[2.2] Instanced stereo questions. Unlit, quad passes, etc

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
Post Reply
zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

[2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

Hello!

I am trying out the instanced stereo support in 2.2 (GL3Plus RS). For the most part it seems to be working well, but I have found a couple of issues.

1. The Unlit shaders seem to be missing support for instanced stereo. Am I correct in this assumption? I suppose that only some smaller shader modifications are missing, although I am somewhat lost in all the macro expansion stuff going on.

2. Is there some nice way of handling instancing together with quad passes, for example for sky rendering? As an example, can a setup similar to TutorialSky_Postprocess be adapted to handle both instanced and non-instanced rendering?

Thanks

zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

zxz wrote:
Wed Jun 03, 2020 3:13 pm
2. Is there some nice way of handling instancing together with quad passes, for example for sky rendering? As an example, can a setup similar to TutorialSky_Postprocess be adapted to handle both instanced and non-instanced rendering?
Hmm. I suppose that the quad pass must be adjusted to draw two quads with the camera directions of each individual eye's frustum corners. Ogre::Camera doesn't seem to provide corners for both eyes, so some code needs to be added to compute those using the eye projection matrices.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

Hi!

We had issues trying to get instanced stereo to work with "sky as a postprocess" technique (there's the clipping against the middle of the screen, we also had aspect ratio issues which cause dizziness when you tilt your head and due to the head's movement nature this is impossible to prevent), which is why we implemented SceneManager::setSky( true, SceneManager::SkyCubemap, texture ); method.

zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

Hi!

SceneManager::setSky seems to use the same technique for drawing the sky, and the sky is only drawn in the left eye viewport when it is used. From what I can see there is nothing in its implementation that takes stereo rendering into consideration. Perhaps I am misunderstanding you?

I don't understand what the problems you describe are. If each eye were to be rendered individually with each eye's individual camera direction vectors, things should look good. That should be achievable if quad passes are aware of the camera's VrData. That should give the same result as when rendering the whole scene twice (once for each eye), which looks good. Is there something that prevents that from working?

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

zxz wrote:
Fri Jun 05, 2020 9:33 am
SceneManager::setSky seems to use the same technique for drawing the sky, and the sky is only drawn in the left eye viewport when it is used. From what I can see there is nothing in its implementation that takes stereo rendering into consideration.
Uh oh, this must be Unlit's missing support for stereo instancing. Should be easy for me to fix.
zxz wrote:
Fri Jun 05, 2020 9:33 am
I don't understand what the problems you describe are. If each eye were to be rendered individually with each eye's individual camera direction vectors, things should look good. That should be achievable if quad passes are aware of the camera's VrData. That should give the same result as when rendering the whole scene twice (once for each eye), which looks good. Is there something that prevents that from working?
I thought the same too, but it was tricky to get the math right. We could've made a mistake in the formulas though.

zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

dark_sylinc wrote:
Fri Jun 05, 2020 3:32 pm
Uh oh, this must be Unlit's missing support for stereo instancing. Should be easy for me to fix.
The sky implementation uses old materials with custom shaders, not unlit, so I don't think they are related. In any case, I don't think the instancing the same primitive will take care of the eye differences without some additional mechanism making use of the individual projection matrices. I suppose it would work if an actual skycube was used though.

I am also very interested in instanced stereo support for Unlit though, as we cannot begin using instanced stereo until that works. I was looking into implementing it for a bit, and saw that Unlit differs somewhat from Pbs. It has support for identity world transforms, and uses complete worldViewProj matrices where Pbs uses only viewProj, and thus passes somewhat different data to the shader. So it's not quite as trivial as a copy/paste which was my initial hope.

I too think that it would be easy for you to fix :wink:. Now I don't mean to preempt your plans, but it would be very useful to know if that is something that might be fixed in the near future, or if I should go ahead and attempt to hack together some partial solution. These two things (sky and unlit) are what's blocking us from fully functioning VR at the moment. Anyway, it is really nice to (almost) be able to just flip a switch and get stereo rendering without having to setup multiple compositors and managing multiple cameras manually. Just some details missing.

Best regards!

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

I took a look and I forgot Unlit was never ported to shared code (i.e. most code written in .any files). I'm beginning to do that now.

zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

Nice! How is it coming along? I suppose that the instanced stereo support comes along for the ride when the shared code is used?

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

I've pushed the "unification" changes to its own branch (to prevent accidentally breaking everyone until it's more tested). I'm done for now (for now I'm limiting it to vertex shaders, since it's all we need for instanced stereo changes).

I've used up the time I had left to focus on this issue for this weekend , so I will be doing the instanced stereo changes during next weekend.

Without the unification, the changes would've needed to be done 3x times.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

I just pushed Instanced Stereo support for HlmsUnlit to its own branch.

I could not test it thoroughly since I don't have the use cases like you do (also I don't have the VR headset anymore, they were borrowed); but I tried to simulate a basic one and it seemed to be working as intended.

If you find no issues I'll be merging it to master.

zxz
Gremlin
Posts: 160
Joined: Sat Apr 16, 2016 9:25 pm
x 12

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by zxz »

Hello!

I have briefly tested the branch and it seems to work well so far! I have tested using a HMD where it looks good. It will get some more testing as we start testing VR more later.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.2] Instanced stereo questions. Unlit, quad passes, etc

Post by dark_sylinc »

Ahhh nice!

This is a low risk change (last famous words) so I'm going to merge it into master

Post Reply