Page 1 of 1

Cubemap as postprocess in VR

Posted: Thu Oct 10, 2019 10:12 pm
by xrgo
Hello! I want to implement tunneling (https://github.com/sigtrapgames/VrTunnellingPro-Unity) with a grid for VR and my progress so far is this:

I made a post process like the TutorialSky_Postprocess that renders a cubemap (that later will be a grid cage) and just masking it with a vignette.
the thing is that I am using the new instanced stereo rendering so the post process I am applying it in two pass quads with different viewports, like this:

Code: Select all

        //TUNNELING
        {
            Ogre::CompositorTargetDef *targetDef = renderNodeDef->addTargetPass( "renderAuxTunneling" );
            targetDef->setNumPasses( 2 );

            {
                //QUAD
                Ogre::CompositorPassQuadDef *passQuadDef = static_cast<Ogre::CompositorPassQuadDef*>( targetDef->addPass( Ogre::PASS_QUAD ) );
                passQuadDef->setAllLoadActions( Ogre::LoadAction::DontCare );
                passQuadDef->setAllStoreActions( Ogre::StoreAction::Store );
                passQuadDef->mMaterialName = "TunnelingMat";
                passQuadDef->mFrustumCorners = Ogre::CompositorPassQuadDef::CAMERA_DIRECTION;
                passQuadDef->addQuadTextureSource( 0, "renderAuxPost" );
                passQuadDef->mProfilingId = "STEREO_FINAL_PROCESSING_QUAD_PASS";

                passQuadDef->mVpRect[0].mVpLeft = 0.0f;
                passQuadDef->mVpRect[0].mVpTop = 0.0f;
                passQuadDef->mVpRect[0].mVpWidth = 0.5f;
                passQuadDef->mVpRect[0].mVpHeight = 1.0f;
                passQuadDef->mVpRect[0].mVpScissorLeft = 0.0f;
                passQuadDef->mVpRect[0].mVpScissorTop = 0.0f;
                passQuadDef->mVpRect[0].mVpScissorWidth = 0.5f;
                passQuadDef->mVpRect[0].mVpScissorHeight = 1.0f;
            }
            {
                //QUAD
                Ogre::CompositorPassQuadDef *passQuadDef = static_cast<Ogre::CompositorPassQuadDef*>( targetDef->addPass( Ogre::PASS_QUAD ) );
                passQuadDef->setAllLoadActions( Ogre::LoadAction::Load );
                passQuadDef->mMaterialName = "TunnelingMat2";
                passQuadDef->mFrustumCorners = Ogre::CompositorPassQuadDef::CAMERA_DIRECTION;
                passQuadDef->addQuadTextureSource( 0, "renderAuxPost" );
                passQuadDef->mProfilingId = "STEREO_FINAL_PROCESSING_QUAD_PASS";

                passQuadDef->mVpRect[0].mVpLeft = 0.5f;
                passQuadDef->mVpRect[0].mVpTop = 0.0f;
                passQuadDef->mVpRect[0].mVpWidth = 0.5f;
                passQuadDef->mVpRect[0].mVpHeight = 1.0f;
                passQuadDef->mVpRect[0].mVpScissorLeft = 0.5f;
                passQuadDef->mVpRect[0].mVpScissorTop = 0.0f;
                passQuadDef->mVpRect[0].mVpScissorWidth = 0.5f;
                passQuadDef->mVpRect[0].mVpScissorHeight = 1.0f;
            }
        }
TunnelingMat2 just add an offset so it shows the right eye, and I am passing worldViewProj as params with the matrix of each eye

And it works! buuuuut! since each quad is rendering in just half the texture (due to the viewport rect) the cubemap looks stretched, and I guess, that for the same reason... it looks like its at 10cm from my face. I was guessing that some math in the sampling coordinates in the shader would do the trick but I can't find any, I feel there should be a CAMERA_DIRECTION_HALF or something.

is this even possible? as long as the cubemap looks like its at the infinity its good to me

thanks!!

Re: Cubemap as postprocess in VR

Posted: Mon Oct 14, 2019 4:37 pm
by dark_sylinc
Is there any picture?

For an object to look like they're at infinity, both eyes should see exactly the same (i.e. no offset).

Re: Cubemap as postprocess in VR

Posted: Tue Oct 15, 2019 5:34 pm
by xrgo
I have a video =)

see how objects deform (people get fat and skinny) as I rotate the camera
I am guessing that fixing it would make the cubemap look like its at infinity

Re: Cubemap as postprocess in VR

Posted: Tue Oct 15, 2019 6:27 pm
by dark_sylinc
I wonder if that problem is present in non-stereo rendering too. Rolling the camera is not common in regular 3D manipulation.
I'll take a look when I get some time.

Re: Cubemap as postprocess in VR

Posted: Tue Oct 15, 2019 7:08 pm
by xrgo
in stereo the people are noticeable stretched in every case, it doesn't seems to be a problem in non stereo the people look normal... but I tried to remove the viewport rect stuffs and I see one eye filling the whole texture (both eyes) and I thought that would make the aspect ratio to be correct but its the same, I really think the problem is in the normals that are being feeded in to the quad.. I'll try to get more clues =)
thanks!

Re: Cubemap as postprocess in VR

Posted: Tue Oct 15, 2019 7:13 pm
by dark_sylinc
xrgo wrote: Tue Oct 15, 2019 7:08 pm I really thing the problem es in the normals that are being feeded in to the quad.. I'll try to get more clues =)
Yes, the problem is in the normals being fed. But we'll have to look why are they wrong and what transformation needs to be done to stay correct.
Sounds like an Aspect Ratio issue.

Re: Cubemap as postprocess in VR

Posted: Wed Oct 16, 2019 3:51 am
by xrgo
ok I solved (workarounded) by making dummy cameras for the left and right eye, setting them the openvr matrices, etc. (old school style vr camera setup) and use them in QuadPassDef->mCameraName... but! still looks like its too near, or some strange barrel distortion, not sure what it is.

anyways!! I was thinking and I will ditch this approach and instead render my grid cage like a normal object in a second sceneManager, this way I can adjust the size (maybe an infinite box is not the best option to reduce sickness) and I can move around inside the cage using positional tracking

thanks!!