VR with HlmsUnlit

[Hilarius86]

Ogre Version: 2.2
Operating System: Windows 10
Render System: Direct3D11

Hi, I am in the process of upgrading our software from Ogre 2.1 to 2.2. Most things are working smoothly, thanks for the improvements! Last step is getting VR to work again. In OgreV1 and up until 2.1 I used a self-made solution to update to camera position from VR, render the two cameras and present to OpenVR. I decided to move away from it and adopt the demo implementation to get the benefits of Ogre’s implementation of radial density, hidden area mesh, a working system (with possible future updates) and all this without adopting my own solution to the new way of handling textures.
In my scene I use the HlmsPbs with some custom modifications. This is working ok so far. Now I tried to reactivate the HlmsUnlit that I use to render lines. In the Ogre branch 2.2 I had a problem with HlmsUnlit not being aware of instanced stereo and therefore only rendering to one eye. I was able to adapt the 2.2 source code with the changes from master aka the "instanced stereo unlit" branch. Now my lines are rendering to both eyes - yay. Next problem now is culling. Both eyes use their own culling camera.

I and now trying to analyse the rest of the changes I did not port yet to find what is necessary for the culling to work correctly.

1. Is the current HlmsUnlit implementation capable of using only a single culling camera for VR?
   If it should be capable, I will try to replicate my issues by extending the demo.
2. If it is not, and this is my guess at this moment, I am trying to understand the changes in HlmsPbs with ForwardPlus and instanced stereo and I am hitting a wall. As unlit does not need lighting I am unsure which parts are important to port.
   This is a snippet I found in the ForwardPlus file that indicates, that culling is correctly compensated for PBS.

   Code: Select all

   @property( hlms_instanced_stereo )
   		float2 fwdFragCoord = inPs.cullCamPosXY.xy * (1.0f / inPs.cullCamPosXY.z);
   		fwdFragCoord.xy = fwdFragCoord.xy * 0.5f + 0.5f;

[dark_sylinc]

Hi!

Great that you were able to adapt Instanced Stereo for Unlit in 2.2!
Unlit started supporting Instanced Stereo in 2.3 out of the box.

Is the current HlmsUnlit implementation capable of using only a single culling camera for VR?
If it should be capable, I will try to replicate my issues by extending the demo.

It should be!
Frustum Culling is independent of the Hlms implementation and is in C++.

For the sake of simplicity the InstancedStereo sample performs no tricks and only relies on the left eye's culling for the right eye. Thus, for the InstancedStereo sample, there is a slight chance the right eye does not see things that it should see.

However the Tutorial_OpenVR performs a proper setup for frustum culling. It creates a camera (see OpenVRCompositorListener::mVrCullCamera) called the "culling camera" and use cull_camera property of the Compositor.

The trick is to simply move the culling camera backwards until it encloses both the left and right eye. This causes a few more false positives but avoids having to cull all objects twice.

The exact location of this culling camera depends on FOV settings and eye separation (details that InstancedStereo does not care about and thus are skipped).

This is a snippet I found in the ForwardPlus file that indicates, that culling is correctly compensated for PBS.

Ignore that. It's not related to culling. Forward+ subdivides the screen in a grid, and must create a list of lights per cell.

Since InstancedStereo divides the screen in 2; Forward+ must account for this becase on the pixel shader coordinates go [0; width) but what we want is to go [0; width / 2) twice while looking at different grids (one for each eye).

Cheers.

[Hilarius86]

Thanks for the feedback, well rounded as always!

I am actually already using the Listener from the Tutorial_OpenVR with the sample compositor script for now. So I am using the culling camera and also the cull_camera property. I also debugged to see if the culling camera was found and linked and for my eyes it seemed plausible. Maybe I made a simple mistake and also my other objects are not culled correctly, but for whatever reason, it was not as noticeable. Very big object and centered in the scene...

I guess I will do 2 checks next. Verify the bounds of my line objects and see if the culling of the other objects are fine. Then generate a few line objects into the Tutorial scene and see if I can reproduce the failing culling.
I also had a problem that transparent meshes were not displaying correctly, but those were rendered with pbs and I thought that problem was less severe. And I did not really look for a reason yet.

Are there any changes in between 2.2 (2.2.7) and 2.3 that could result in a culling problem?

Bonus Question: Is there an easy way to get RenderDoc to capture VR?

[dark_sylinc]

Do you have a picture I can see?
Perhaps it's obvious when I see it.

Also to check the obvious: Make sure the MovableObject's local AABB is updated correctly (which you can set via MovableObject::setLocalAabb).

[Hilarius86]

To give an update: I used WireAABB::track to view the bounds and found that the bounds of my lines were broken.
Long story short: When upgrading to 2.0 I skipped a refactor and kept a design problem. I am generating the Items and afterwards altering the meshes and bounds again. I created a little hack with item::_deintitialize and saw it working as expected in 2.0/2.1. Now I am also detaching the item from the node and recreating the Item and the reattaching it to the node and the Wireframe looks right without VR.

With VR you get the following stereo view.
I generated the view with the WireAABB to remove my broken lines from the equation, but the problem persists.

[Hilarius86]

Maybe I got it now.
#define finalDrawId (input.drawId >> 1u)
to make outVs.drawId the correct Id...

Maybe there is some other define that I didn't catch, but it looks like its wrong in master as well.

[dark_sylinc]

Hi!

I'm glad you found the problem!

#define finalDrawId (input.drawId >> 1u)
to make outVs.drawId the correct Id...

Maybe there is some other define that I didn't catch, but it looks like its wrong in master as well.

Nice catch! It seems D3D11 had the bug while GL/Vulkan/Metal did not.

Fixed on 2.3 & master