[SOLVED]Dynamic texture updated by thread

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.
Post Reply
User avatar
thejesusbr
Gnoblar
Posts: 15
Joined: Thu Feb 03, 2011 8:17 pm
Location: Campinas, SP Brasil

[SOLVED]Dynamic texture updated by thread

Post by thejesusbr » Fri May 27, 2011 1:23 am

Hi, everyone!

I am coding a app using Ogre where I need to update a dynamic texture. The update speed will be ~30fps, but I need that the main loop runs at ~120fps. I have a manager class that take the images from a camera, do some processing with OpenCV and so, update the texture. The texture is created by the core application and ist poiter is passed to the manager class.

If I call for the updateTexture method from de manager class inside the main loop, my fps is bound to the update speed of the manager class. I not want this. I need to run the main loop as fast as I can, but the texture will be updated at ~30fps.

I'm thinking on using a trhead to do de updating of the texture, but I am not so sure about what I have to do, so I'm seeking sugestions. Right now, I'm trying to implement threads using ptypes. I'm working on Linux now, but the code will be migrated to Windows, but keeping the portability.

The code of the updateTexture method:

Code: Select all

void VideoManager::updateTexture()
{
	if (videoCapture->isOpened())
	{
		videoCapture->grab();
		videoCapture->retrieve(frame, CV_CAP_OPENNI_BGR_IMAGE);
		videoCapture->retrieve(depthMap, CV_CAP_OPENNI_DEPTH_MAP);
		videoCapture->retrieve(disparityMap, CV_CAP_OPENNI_DISPARITY_MAP);

	}
	else return;

	// Get the pixel buffer
	mPixelBuffer = texturePtr->getBuffer();
	// Lock the pixel buffer and get a pixel box
	mPixelBuffer->lock(Ogre::HardwareBuffer::HBL_DISCARD); // for best performance use HBL_DISCARD!
	const Ogre::PixelBox& mPixelBox = mPixelBuffer->getCurrentLock();
	pDest = static_cast<Ogre::uchar*> (mPixelBox.data);
	if (displayMode == RGB)
	{
		pSour = static_cast<Ogre::uchar*> (frame.data);
		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1) * 3;
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour++; // B
					*pDest++ = *pSour++; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 6; //2 * canais
				}
				pSour += (mCapWidth + 1) * 3;
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour++; // B
					*pDest++ = *pSour++; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}
	else if (displayMode == DEPTH)
	{
		const float scaleFactor = 0.05f;
		cv::Mat show;
		depthMap.convertTo(show, CV_8UC1, scaleFactor);
		pSour = static_cast<Ogre::uchar*> (show.data);
		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1);
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 2;
				}
				pSour += (mCapWidth + 1);
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}
	else if (displayMode == DISPARITY)
	{
		pSour = static_cast<Ogre::uchar*> (disparityMap.data);

		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1);
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 2;
				}
				pSour += (mCapWidth + 1);
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}


	// Unlock the pixel buffer
	mPixelBuffer->unlock();
}
And the code of mainLoop method:

Code: Select all

void OgreManager::mainLoop()
{
	Ogre::LogManager::getSingletonPtr()->logMessage("*** Starting Main Loop ***");
	frameTicker->reset();
	while (stateManager->getCurrentState() != SHUTDOWN)
	{
		if (mainWindow->isClosed())
		{
			stateManager->requestStateChange(SHUTDOWN);
		}
		//Input Manager;
		inputManager->capture();

		Ogre::WindowEventUtilities::messagePump();

		if (stateManager->getCurrentState() == GUI)
		{
			//Mostre a GUI
			pGUIRenderer->setRenderingEnabled(true);
		}
		else if (stateManager->getCurrentState() == SIMULATION)
		{
			//Roda a simulação
			pGUIRenderer->setRenderingEnabled(false);
                        if (videoOn) videoManager->updateTexture();
		}
		physicsManager->addStepSimulation(33);
		ogreRoot->renderOneFrame();
		frameTicker->reset();
	}
I will made an update to this post as long as I modify the code to use threading. Someone have sugestions?

QUICK UPDATE:
I encapsulated the updateTexture() method around the execute() method inherited by de pt::thread class from ptypes/pasync lib and it give me a segfault on the mPixelBuffer->unlock() call. Why it happened? I need to create the texture in the same trhead, maybe?
Last edited by thejesusbr on Mon May 30, 2011 9:49 pm, edited 1 time in total.
0 x

User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2702
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 2

Re: Dynamic texture updated by thread

Post by syedhs » Fri May 27, 2011 9:00 am

I have mentioned this in other thread but here is my pseudocode:-
setup

Code: Select all

create two identical textures.
set active_texture_index to 0 (first one).
lock the texture buffer
create thread with pointer bit as the parameter.

FrameListener::frameStart (loop)
[code]
has active texture been filled up?
if yes then
     unlock the texture buffer
     use this texture buffer
     active_texture_index = 1 - active_texture_index
     lock the texture buffer, and pass it to the thread the pointer bit.
end
p/s: You have to be quite comfortable with thread programming, ie you should know about how thread communicate with the parent process (in Windows, specifically you have to know Event, WaitForMultipleObject). And never allow the thread to quit unless the parent process quit too. If the job is finished, then the thread should be idle awaiting for next command.
0 x
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me

User avatar
thejesusbr
Gnoblar
Posts: 15
Joined: Thu Feb 03, 2011 8:17 pm
Location: Campinas, SP Brasil

Re: Dynamic texture updated by thread

Post by thejesusbr » Fri May 27, 2011 2:17 pm

Thank you for replying, syedhs. I'll do it. Can you post the link for the thread that you mentioned?
0 x

User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2702
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 2

Re: Dynamic texture updated by thread

Post by syedhs » Fri May 27, 2011 5:35 pm

Actually the post in here is much more detail :wink:
0 x
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me

User avatar
thejesusbr
Gnoblar
Posts: 15
Joined: Thu Feb 03, 2011 8:17 pm
Location: Campinas, SP Brasil

Re: Dynamic texture updated by thread

Post by thejesusbr » Mon May 30, 2011 6:02 pm

Let's see if I get it... I will create two identical textures, one in the main thread and another in the video updating thread. So, I'll render the texture that is in the main thread and when the other texture updates, I lock the main thread texture and copy the data from the other thread. That's it?
0 x

User avatar
thejesusbr
Gnoblar
Posts: 15
Joined: Thu Feb 03, 2011 8:17 pm
Location: Campinas, SP Brasil

Re: [SOLVED]Dynamic texture updated by thread

Post by thejesusbr » Mon May 30, 2011 10:08 pm

I managed to solve this by a different aproach. I've made a simple event mechanism based on trhead triggers and a state variable. When a frame arrives, the newFrame flag is setted to true and the thread goes sleep. In the main loop in, another thread, I check the flag. If it's true, I update the texture. In the update method, I set the flag to false and signal the frame grabber thread to wake up. So, the main thread keeps jumping the texture update until the frame arrives again, which causes the flag to be true. It works as I expect. Below, the final code:

The VideoManager code, running the capture process on a background thread

Code: Select all

//Here, goes the code that will be running on a background thread
void VideoManager::execute()
{
	while (!get_signaled())
	{
		printf("Thread Launched\n");
		printf("Capturing...\n");
		if (videoCapture->isOpened())
		{
			videoCapture->grab();
			videoCapture->retrieve(frame, CV_CAP_OPENNI_BGR_IMAGE);
			videoCapture->retrieve(depthMap, CV_CAP_OPENNI_DEPTH_MAP);
			videoCapture->retrieve(disparityMap, CV_CAP_OPENNI_DISPARITY_MAP);
                       //Notify the arrival of a new frame
			newFrame = true;
                        //Set a semaphore to put the thread to sleep
			copyTrigger.wait();
		}
		else return;
	}
}
//Here is the code that updates the texture.
void VideoManager::updateTexture()
{
	// Get the pixel buffer
	Ogre::HardwarePixelBufferSharedPtr mPixelBuffer;
	mPixelBuffer = texturePtr->getBuffer();
	printf("Taking buffer\n");
	// Lock the pixel buffer and get a pixel box
	mPixelBuffer->lock(Ogre::HardwareBuffer::HBL_DISCARD); // for best performance use HBL_DISCARD!
	printf("Buffer locked\n");
	const Ogre::PixelBox& mPixelBox = mPixelBuffer->getCurrentLock();
	printf("Box taken\n");
	pDest = static_cast<Ogre::uchar*> (mPixelBox.data);
	printf("pDest set\n");
	if (displayMode == RGB)
	{
		pSour = static_cast<Ogre::uchar*> (frame.data);
		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1) * 3;
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour++; // B
					*pDest++ = *pSour++; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 6; //2 * canais
				}
				pSour += (mCapWidth + 1) * 3;
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour++; // B
					*pDest++ = *pSour++; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}
	else if (displayMode == DEPTH)
	{
		const float scaleFactor = 0.05f;
		cv::Mat show;
		depthMap.convertTo(show, CV_8UC1, scaleFactor);
		pSour = static_cast<Ogre::uchar*> (show.data);
		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1);
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 2;
				}
				pSour += (mCapWidth + 1);
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}
	else if (displayMode == DISPARITY)
	{
		pSour = static_cast<Ogre::uchar*> (disparityMap.data);

		if (isMirrored)
		{
			for (size_t j = 0; j < mCapHeight; j++)
			{
				pSour += (mCapWidth - 1);
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
					pSour -= 2;
				}
				pSour += (mCapWidth + 1);
			}
		}
		else
		{
			for (size_t j = 0; j < mCapHeight; j++)
				for (size_t i = 0; i < mCapWidth; i++)
				{
					*pDest++ = *pSour; // B
					*pDest++ = *pSour; // G
					*pDest++ = *pSour++; // R
					*pDest++ = 0; // A
				}
		}
	}
	printf("Buffer copied\n");
	// Unlock the pixel buffer
	mPixelBuffer->unlock();
	printf("Buffer unlocked\n");
       //In the end of the updating process, notify the main thread that the data is up-to-date
	newFrame = false;
       //Signals the semaphore to wake up the frame grabber. So, the threads waits until a
       //new frame arrives from the cam to notify the main thread again
	copyTrigger.post();
}
The main loop running on the main thread:

Code: Select all

void OgreManager::mainLoop()
{
	Ogre::LogManager::getSingletonPtr()->logMessage("*** Starting Main Loop ***");
	frameTicker->reset();
	while (stateManager->getCurrentState() != SHUTDOWN)
	{
		if (mainWindow->isClosed())
		{
			stateManager->requestStateChange(SHUTDOWN);
		}
		//Input Manager;
		inputManager->capture();

		Ogre::WindowEventUtilities::messagePump();

		if (stateManager->getCurrentState() == GUI)
		{
			//Mostre a GUI
			pGUIRenderer->setRenderingEnabled(true);
		}
		else if (stateManager->getCurrentState() == SIMULATION)
		{
			//Roda a simulação
			pGUIRenderer->setRenderingEnabled(false);
			//Each loop, check if a new frame arrives. If true, update the texture
			if (videoOn && videoManager->getNewFrame()) videoManager->updateTexture();
		}
		physicsManager->addStepSimulation(33);
		ogreRoot->renderOneFrame();
		frameTicker->reset();
	}
	Ogre::LogManager::getSingletonPtr()->logMessage("*** Application shuting down... ***");
	ogreRoot->shutdown();
}
The frame rate drops too low, leaving it pratically imperceptible, since the copy is made on a separate thread from the updating. Very good. It not the final solution that I want (updating in background), but have solve the problem, anyway.
Syedhs, thanks for the reply, it helped a lot! Next step, set a independent timer to update the physics... Quite easy!
0 x

Post Reply