Using video background

[tct]

Ogre Version: 2.1
Operating System: Windows, Linux
Render System: OpenGL 3+

I'd be interested to know what the best / recommended way is of getting a video background to work. Basically I'd like to have the viewport completely filled with a background video and then render 3D objects that move around accordingly.

The video background needs to work either with just a basic pre-recorded video (MP4 container) and with the video coming from a webcam.

Any help is highly appreciated.

[dark_sylinc]

Texture (video) streaming is going to be more efficient in 2.2, but anyway to answer:

2.1:
Create 3 textures (with DYNAMIC + DISCARD flags), cycle one each frame, map one per frame, and fill their pixels.

The cycling would be like this (do this only once per frame):

Code: Select all

const PixelBox& box = textureToUse[frameIdx]->getBuffer()->lock();
... fill box ...
textureToUse[frameIdx]->getBuffer()->unlock();
frameIdx = (frameIdx + 1u) % 3u

2.2:
You get a StagingTexture every frame, map it, fill its pixels, and upload the StagingTexture into just one texture.
You can checkout Tutorial_SSAOGameState::createScene01, Terra::createHeightmapTexture on how to use StagingTextures:

Code: Select all

StagingTexture *stagingTexture = textureManager->getStagingTexture( image.getWidth(),
                                                                    image.getHeight(),
                                                                    1u, 1u,
                                                                    image.getPixelFormat() );
stagingTexture->startMapRegion();
TextureBox texBox = stagingTexture->mapRegion( image.getWidth(), image.getHeight(), 1u, 1u,
                                               image.getPixelFormat() );

//for( uint8 mip=0; mip<numMipmaps; ++mip )
texBox.copyFrom( image.getData( 0 ) );
stagingTexture->stopMapRegion();
stagingTexture->upload( texBox, m_targetTexture, 0, 0, 0 );
textureManager->removeStagingTexture( stagingTexture );
stagingTexture = 0;

You can also create 3 staging textures (one for each frame) and keep them around (instead of calling removeStagingTexture) and cycle through them every frame like in the 2.1 case.


Both versions:
Use a pass_quad to render the texture in the compositor with a low level material

Note: because mp4 format and videocams usually decompresses to either yuv422 or yuv444, you probably want to create three R8_UNORM textures (x3 for each frame) one for y, another for u and another for v at different resolutions (Y is at full resolution, U & V are at half resolution when using yuv422). Then in the pixel shader you assemble back yuv422 to RGB, which is much faster than doing the assembling from C++

[paroj]

also see https://github.com/OGRECave/ogre-audiovideo which has sort of v2-1 compatibility