OGRE and Quad Buffer Stereo

[RigoCL]

Hi RigoCL,
I didn't look at your patch, but from the post it looks like you support anaglyph stereo. The first version I'm posting supports quad buffer, so I could try to work with you in order to incorporate anaglyph.

...

How did you implement stereo in your patch? Is it similar? I certainly appreciate any feedback - especially if you have stereo knowledge

...

Actually is not my implementation, I meant I integrated the code in my own Ogre project, so I really don't know how it was implemented. I read almost the entire design document you mentioned, but I can't tell the differences, that's why I was asking.

All I can tell from the code I mentioned is that it supports several modes:

Code: Select all

	/// Anaglyph red/cyan
	SM_ANAGLYPH_RC,
	/// Anaglyph yellow/blue
	SM_ANAGLYPH_YB,
	/// Dual output off-axis mode : suitable for two projectors with polarized filters or head mounted display
	SM_DUALOUTPUT,
	/// Verticaly interlaced mode
	SM_INTERLACED_V,
	/// Horizontaly interlaced mode
	SM_INTERLACED_H,
	/// Interlaced mode with a checkerboard pattern
	SM_INTERLACED_CB,

And it also uses two viewports:

Code: Select all

/*** Stereo : Global variables and defines */
Ogre::StereoManager *gStereoManager = NULL;
Ogre::Viewport *gLeftViewport = NULL;
Ogre::Viewport *gRightViewport = NULL;
Ogre::SceneManager *gSceneMgr;

#define SCREEN_DIST 10.0f
#define EYES_SPACING 0.06f

void createScene(void)
{
	mStereoManager.init(gLeftViewport, gRightViewport, "stereo.cfg");
	mStereoManager.createDebugPlane(mSceneMgr);
	mStereoManager.enableDebugPlane(false);

	...
}

It also uses a stereo.cfg (for storing "eye spacing" and "focal length" values), a Stereoscopy.cg and a Stereoscopy.compositor files.

The code inside the createScene() method was almost all the work I had to do to make this stereo version work.

So far now I've only been able to test Anaglyph mode, though I'm still not satisfied with the results, but I'm assuming I have to find the right values for "eye spacing" and "focal length".

[crefvik]

Hi RigoCL,

This doesn't look like it supports quad buffer stereo and currently that is what I'm supporting. In my document, I give some background to other stereo types such as the composited stereo types you mentioned. However, I haven't implemented these. As mentioned in my document, you could use my implementation with OGRE's compositor to implement composited stereo. This looks like how it was implemented. As mentioned in my last post, the state of the left or right eye is stored in the viewport class itself. This makes it possible to support composited or quad buffer stereo.

In terms of setting the camera settings for both viewports, this is requires the most thought. I've modified the camera's position and the focal length and frustum offset of each camera to get the stereo image. OGRE's camera implementation provides setFocalLength and setFrustumOffset to do this:
Ogre::Camera

Using the default FOV and assuming the scale of the world is 1m, then offsetting the frustums about 3cm from center and setting the focal length about 10m works pretty well, but I've played around with other settings which work well too.

In any case, I can certainly add composited stereo but I don't have support today.

Thanks,
Christian

[acorn]

Hello Christian,

Thank you for writing this quad buffer support in Ogre! I was able to get Rviz (a robot visualization program that uses Ogre) to render in stereo using your branch.

Do you have any comments about the progress of your pull request getting into the main Ogre tree? I realize you may not be the one to ask about that .

Thanks!
-Acorn

P.S. in case you are interested the Rviz code is here:
https://github.com/ros-visualization/rviz/pull/718
I am using this hw/sw
- Asus VG278H monitor (with included NVIDIA 3DVision IR active glasses)
- NVIDIA Quadro K4000
- Ubuntu 12.04 linux

[crefvik]

Hi Acorn,

Glad it works for you! I'm working with the OGRE team to get the feature checked in. I know that Wolfmanfx (Murat Sari) has been working on the pull request and checking it out on a stereo display.

FYI - I'm actually going to try out the robot visualization as soon as I get a chance.

Take care,
Christian

[AllyBarrow]

Hi there, thanks for this patch, it's a real life saver for our current project.

I've pulled the 1.9 version from your branch. Had to add the following to OgreD3D9StereoDriverAMD.cpp

Code: Select all

#ifdef _DEBUG
		viewPort.Width = mRightBuffer->Width;
		viewPort.Height = mRightBuffer->Height;
#else
		D3DSURFACE_DESC *pDesc;
		mRightBuffer->GetDesc(pDesc);
		viewPort.Height = pDesc->Height;
		viewPort.Width = pDesc->Width;
#endif

To get it to compile due to this block in d3d9.h DECLARE_INTERFACE_(IDirect3DSurface9, IDirect3DResource9)

Code: Select all

  
 #ifdef D3D_DEBUG_INFO
    LPCWSTR Name;
    UINT Width;
    UINT Height;
    DWORD Usage;
    D3DFORMAT Format;
    D3DPOOL Pool;
    D3DMULTISAMPLE_TYPE MultiSampleType;
    DWORD MultiSampleQuality;
    DWORD Priority;
    UINT LockCount;
    UINT DCCount;
    LPCWSTR CreationCallStack;
    #endif

Just wondering if that's of any interest to you or indicates I've compiled against a different DX version?

Thanks again for this contribution, looking forward to getting it up and running
Cheers,
Ally

[crefvik]

Hi Ally,

Thanks for the feedback. All the Direct3D work was done using the Microsoft DirectX June 2010 SDK. I believe this also what the OGRE team uses to build the pre-built binaries. I looked back at the source repository and see your issue. The m_rightBuffer is a IDirect3D9Surface object, which does not have a way to retrieve the width or height directly. As you mentioned, you need to get its description. This is strange because I actually did quite a bit of testing on the AMD Radeon - let me check this out.

In any case, thanks for pointing this out. I'll check it out and fix the issues ASAP.

Take care,
Christian

[alexspmiranda]

Hi,
I tried your implamentation, but the output show me this error -> "fatal error LNK1120: 1 unresolved externals" and "LINK : error LNK2001: unresolved external symbol _WinMainCRTStartup". I already look to code line by line, but i don't know why.

I'm new Ogre user. thks.


INFO:

VisualStudio 2008
Ogre 1.8.1
NVidia GeForce 7900 GT/GTO

[crefvik]

Hi alexspmiranda,

I'll send an email to you and then we can see if we can resolve the issue that you're having. Once we've figured out the root cause, we can post a solution back to this forum.

Thanks,
Christian

[alexspmiranda]

Thks Crefvik,
I'm reading right now and I'll answer soon.

[simedj]

Cool project

I read that this implementation essentially boils down to separate left/right view-ports. Does that mean it will work on a regular monitor for testing, i.e. you'd see a split-screen effect like when you watch a 3D TV channel on a non 3D TV?

[crefvik]

Hi simedj,

Thanks! And you got it; it basically boils down to having two or more viewports in the application, which means one camera per eye. And if you wanted to have a PIP window in your application, you can do it with this approach - you just need three viewports. Two viewports for the main scene and one viewport for the monoscopic PIP. Or you could even have a stereo PIP if you wanted - just use two viewports for the PIP. Since there are devices out there that do head tracked stereo, I wanted the app to have full control over the cameras so they may set the view and projection matrices of each eye independently and maybe an offset is not good enough. And as you pointed out, knowing which viewport is for which eye, we can certainly support different stereo formats such as the composited stereo formats I listed at the top of the document. For example, Oculus Rift supports the side-by-side format I listed in my document as do many 3D TVs.

Good question about the split screen. If you used quad buffer stereo on a non-stereo monitor, you would see a flickering image - especially if the refresh rate was 60Hz or less. As mentioned in the document, L/R frames are constantly being sent to the GPU which causes the flicker on a non-stereo monitor since each image is different. The display's EDID (Extended Display Identification Data) actually contains bits that determine whether or not the display is stereo. I've never looked into it, but the GPU could use this information to determine whether it should send both L/R frames or just one frame so you don't see the flicker. If I had to guess though, I believe the flicker comes from the way the OpenGL driver continues to support the latest pixel format regardless of the display type. I noticed that the Direct3D drivers don't seem to have the same behavior, because they give you more control of the stereo state. Regardless, I've put some effort so into the OGRE implementation tries to make sure that both the OpenGL and Direct3D implementation behaves the same.

Take care,
Christian

[RigoCL]

Does this solution work with Ogre Overlay and the GUIs that use it? like CEGUI, SdkTray and MyGUI, they really look 3D?

The stereo solution I have in my project does not display SdkTray in 3D, seems that gui is only rendered to one viewport.

[crefvik]

You would need to modify the SDKTray, CEGUI, etc. to use two viewports to render in stereo. Please keep in mind that if you were using OpenGL or Direct3D - this is exactly what you would need to do. I've just abstracted both OpenGL and Direct3D to a common OGRE interface and allow the application to do whatever it needs to do.

If the UI is really just made of traditional 2D UI widgets like buttons, menus, check boxes, etc., you probably want to keep the UI at zero parallax (ie. keep it in 2D) and you can use a single viewport for that. However, if you are building 3D UI, then you'll need two cameras to generate images for the left and right eyes ... and therefore two viewports. 3D UI can be beneficial for context menus on scene objects or custom widgets that allow you to manipulate an object. I'm sure there are plenty of research papers you can search for that discuss developing 3D user interfaces or stereoscopic 3D user interfaces.

[simedj]

Regarding split-screen/flicker - are you saying both viewports are full-screen i.e. from (0,0) - (1,1) so they overlap? To allow testing on a regular monitor - probably a useful thing(?) - you'd just need to make the viewports not overlap. Ogre itself wouldn't care, whether it would matter for the hardware I couldn't comment but you would be rendering at lower resolution.

Just my $0.02, it seems like being able to select left OR right OR both side-by-side on your regular monitor would be valuable for debugging/testing?

[crefvik]

Yes, the viewports are fullscreen and they overlap 100%. This is the benefit of quad buffer stereo over composited formats such as left-right, top-bottom, or checkerboard. I actually discuss these formats in the document I wrote and posted at the top of this thread. The way quad buffer stereo works is that the GPU alternates between sending a left and right buffer data to the display. This is the reason that quad buffer supports fullscreen resolution.

The way side-by-side works is that you pack two images into a single buffer and the GPU sends that buffer to the display. The benefit is that you don't require a quad buffer GPU, which are generally more expensive. And as you mentioned, you can implement this by using two viewports that don't overlap. You just place one viewport to the top or left of the render target and the other to the bottom or right of the render target. And you're correct that you'll lose half the resolution. This is one reason why Oculus Rift and other displays only work at 720p.

In terms of making it easy to support left-right or top-bottom stereo, that sounds good to me and I strongly recommend we should do this. Since there are stereoscopic displays that only support left-right or top-bottom stereo, we really should support these formats in addition to quad buffer. The way I always thought this would work was to add a "Left-Right" stereo mode and a "Top-Bottom" stereo mode in addition to the "Frame Sequential" mode. Then you could easily change the stereo format.

In terms of debugging, there is some benefit in doing this by testing both camera setups and viewports simultaneously. For example, maybe you are walking around a building and you want to test whether the right image is being clipped by the camera while the left image is not, which you may not want. However, testing stereo comfort in the scene would require a stereo display.

Take care,
Christian

[simedj]

Sounds good. If I get any R&D money or hear anyone interested in using this in my area of work (medical visualisation) this is going to be amazing; it seems with it being added to Ogre we'll get all this practically "for free" in terms of work needed to use it?

[AllyBarrow]

Hi Again,

Have been playing with your QB support now and really impressed, so thanks again.

Quick question - I've been using Ogre's auto create window functionality but I'm now integrating stereo into our main engine so need to create custom windows and can't seem to find the right switch.

If I write to back left/right without doing any specific setup unsurprisingly I get an error.

If I set: WindowParams["Stereo Mode"] = "Frame Sequential" when creating my window the error goes away but it doesn't actually activate the stereo.

Seems like there's a setup step I've missed?

Thanks,
Ally

[crefvik]

Hi Ally,

I think you are using AMD Direct3D quad buffer stereo, but let me know if that's incorrect.

First, I only added support for AMD Direct3D 9 stereo. Its on my to-do list to add support for AMD Direct3D 11 stereo. I was originally working on Direct3D stereo support in OGRE 1.8, and Direct3D 11 support was still being developed so I didn't add stereo support.

Second, AMD stereo only works in fullscreen mode. This is stated in my OGRE stereo document, but you can checkout the AMD Quad Buffer SDK for the details including the supported GPUs. NVIDIA Direct3D stereo supports both windowed and fullscreen modes. In your app, add something to toggle between fullscreen and windowed. You should be able to see the stereo image.

Finally, OGRE stereo is not fully automatic. OGRE enables the driver for you, but you have to setup the viewports by creating two cameras that are slightly offset from each other. If you look at OGRE's Frustum API, you'll see some ways that you can manipulate the focal length and frustum offset. Or you can calculate custom view and projection matrices for each eye by using the set frustum extents API.

In any case, I hope one of those suggestions helps. If you still can't see stereo, please let me know.

Take care,
Christian

[AllyBarrow]

Hi Christian,

Thanks for the rapid reply! Could you check something for me?

I'm actually currently using QB OGL - mainly because I didn't realise you could do windowed stereo in any D3D configuration, nice!

I've stepped through to see what was happening and it seems like the only place Win32GLSupport::mStereoMode gets set seems to be in Win32GLSupport::createWindow but guarded by if(autoCreateWindow)

There seems to be dual, slightly different name value pairs relating to stereo: "Stereo Mode" for 'Ogre Level' and "stereoMode" for Window level. Without "Stereo Mode" getting passed into autoCreateWindow only some of the stereo flags seem to get set.

As a temporary fix I've put:

Code: Select all

else
        {
#if OGRE_NO_QUAD_BUFFER_STEREO == 0
			mStereoMode = SMT_FRAME_SEQUENTIAL;
#endif
            // XXX What is the else?
            return NULL;
        }

At the bottom of Win32GLSupport::createWindow just to get things moving. Any thoughts?

Cheers,
Ally

[crefvik]

Hi Ally,

I've tested the creation of a stereo window by writing code like what is shown below. This should show a configuration dialog which will allow you to choose "Frame Sequential" for the stereo mode. You must be creating the stereo window by setting the root configuration procedurally. Can you post the code or send me an email on how you are configuring root and creating the window? Using your feedback, I'll write a quick sample to test and make sure it works.

Thanks,
Christian

Code: Select all

int main(void)
{
  // Create an instance of OGRE root
  Ogre::Root* root = new Ogre::Root("plugins_d.cfg");

  // Show the configuration dialog, but if it doesn't work close the app
  if (!root->showConfigDialog())
    return -1;

  // Create a render window and verify that stereo is enabled
  Ogre::RenderWindow* window = root->initialise(true, "Stereo Example");
  bool isStereoEnabled = window->isStereoEnabled();

[AllyBarrow]

Hi again,

Yes, we completely hide the Ogre interface in our app and create windows at run time as necessary.

I've modified the main function in the demo code you posted to create a window manually:

Code: Select all

int main(void)
{
  // Create an instance of OGRE root
  Ogre::Root* root = new Ogre::Root("plugins_d.cfg");

   root->showConfigDialog();

	const Ogre::RenderSystemList& renderSystemList = root->getAvailableRenderers();

  // Initialise but don't create a window yet:
  	bool autoCreate = false;
	std::string customCapabilities = ""; 
	root->initialise(autoCreate, std::string(""), customCapabilities);


	//Build up the extra window info:
	Ogre::NameValuePairList params;
	params["monitorIndex"] =  "0";
	params["colourDepth"] = "32";
	params["vsync"] = "false";
	params["depthBuffer"] = "true";	
	params["stereoMode"] = "Frame Sequential";	

	//Now create the window:
	Ogre::RenderWindow* window = root->createRenderWindow(std::string("Stereo Test"), 1024, 768, false, &params);

  bool isStereoEnabled = window->isStereoEnabled();

  // Load the resources for the scene
  loadResources("resources_d.cfg");

  // Create the scene manager and the scene
  Ogre::SceneManager* sceneManager = root->createSceneManager(Ogre::ST_GENERIC);
  Ogre::Entity* sinbadEntity = NULL;
  Ogre::SceneNode* sinbadNode = NULL;
  createScene(sceneManager, sinbadEntity, sinbadNode);

  // Create the cameras and viewports
  Ogre::Real focalLength = 10.0f;
  Ogre::Vector2 frustumOffset(-3.0f, 0.0f);
  Ogre::Viewport* leftViewport = NULL;
  createViewport(window, sceneManager, "LeftCamera", focalLength, frustumOffset, 0, Ogre::CBT_BACK_LEFT, leftViewport);
  frustumOffset.x = 3.0f;
  Ogre::Viewport* rightViewport = NULL;
  createViewport(window, sceneManager, "RightCamera", focalLength, frustumOffset, 1, Ogre::CBT_BACK_RIGHT, rightViewport);

  bool isSinbadRunning = false;
  Ogre::FrameListener* animationFrameListener = new AnimationFrameListener(sinbadEntity, &isSinbadRunning);
  root->addFrameListener(animationFrameListener);

  Ogre::FrameListener* oisFrameListener = new OISFrameListener(window, sinbadNode, &isStereoEnabled, &isSinbadRunning);
  root->addFrameListener(oisFrameListener);

  // Start OGRE rendering
  root->startRendering();

  root->removeFrameListener(animationFrameListener);
  delete animationFrameListener;

  root->removeFrameListener(oisFrameListener);
  delete oisFrameListener;

  // Delete the root object
  delete root;

  return 0;
}

That should hopefully reproduce the same issue.

Thanks again!
Ally

[AllyBarrow]

Hi, one more query,

There might be a conflict between stereo and compositors.

I use a compositor to mirror each camera's output horizontally, if I activate this compositor to a stereo enabled camera under OGL I get an OGL invalid operation warning when Ogre tries to set the draw buffer.

What seems to be happening is that when Ogre creates a compositor it uses the same camera and a different target (RTT). During the render process it sets the camera to BACK to draw to the RTT target then to LEFT/RIGHT to draw to the camera's window target. Which would seem to be right, however, somewhere in the switch OGL is getting confused and throwing up an error.

In DX9 the problem seem more trivial, if I add a compositor and turn on stereo, on exit, inside:

Code: Select all

    //---------------------------------------------------------------------
    void D3D9RenderSystem::destroyRenderTarget(const String& name)
    {       
#if OGRE_NO_QUAD_BUFFER_STEREO == 0
		D3D9StereoDriverBridge::getSingleton().removeRenderWindow(name);
#endif

        detachRenderTargetImpl(name);

        // Do the real removal
        RenderSystem::destroyRenderTarget(name);    
    }

detachRenderTargetImpl(name); throws an exception trying to delete NULL, perhaps removeRenderWindow is deleting a resource detachRenderTargetImpl expects to delete as well?

If you have any thoughts on the OGL problem I'd be really grateful, I'll have to find another way to flip the output if I can't get compositors to work.


Cheers,
Ally

[crefvik]

Hi Ally,

Yes, we completely hide the Ogre interface in our app and create windows at run time as necessary.

I was able to reproduce the issue. The problem was that the OpenGL PFD is actually set in the OpenGL support object, and the stereo setting wasn't being updated by the OpenGL window. If you auto-create the window, then it works fine, but if you don't auto-create the window nothing happens. I'm not clear on the history of why its exactly done this way. In any case, I fixed the issue and submitted a pull-request to the ogre repository.

I haven't looked into the compositor issue you are having, but I'll do that soon.

Take care,
Christian

[AllyBarrow]

Hi Christian,

That's excellent, I'll check that out.

The compositor issue doesn't actually seem to impact the output, just throws up OGL error messages, so have suppressed them and using as is.

I am getting a slightly odd effect in stereo though, a kind of ghosting effect but not actual ghosting (dark in one eye + light in the other resulting in a half way brightness) but something that looks almost like an error calculating shadows in the stencil buffer (have turned off shadows to make sure it isn't though) and the result is a perfect 'negative' of a shape in one eye ending up in the other.

I haven't been able to verify if that's my issue or something in Ogre yet, need to create a simpler example to test with and compare D3D/OGL. Just wondering if you'd noticed anything similar?

Thanks again,
Ally

[AllyBarrow]

Hi, remember me?

Thanks for the update, OGL now opens and runs a manual stero window!

But...

Are you able to open a DX9 stereo window manually? Having a similar problem as in OGL. Fine on auto, no stereo first time opening a manual window and then crashes on susequent attempts. This is with NVIDIA hardware this time.

Any thoughts?

Cheers,
Ally