I actually made some slight progress. I now have a black screen on the rift.
It's not supposed to be black (should be pink with a green statue in front), but it's not crashing, so that's a good thing I guess!
It seems that any time I make a bit of progress, the oculus sdk changes and I get set back (and then give up for a while). With the CV1 on the way (knowing the sdk could change again before 1.0) plus Vive and OSVR having sdks released, it's hard to keep up enthusiasm.
Evolution of the oculus sdk:
- dk1 early versions: simply render in stereo and use a compositor with a shader from the sdk docs. Rendering is done as a standard ogre full screen app, just pick the right monitor (the rift one).
- dk2 versions: things that were open source are now closed and hidden behind a network link, making debugging harder. Direct mode added that uses driver hacks to trick windows into not seeing a monitor. Still rendering the same way for extended mode (using the rift as a monitor). More of a push to "don't do it yourself, let us handle things".
- shader based distortion changed to mesh based distortion. Instead of rendering the post process using a single quad, we now need to use a mesh of around 8000+ tris. That's getting trickier to do (Ogre 2.1 has custom mesh compositors, but it seemed very complex to use).
- directx 9 dropped (where I do most of my ogre work)
- removal of extended mode and client rendering. No more simple rendering to a monitor or user distortion rendering. Now we render stereo to a texture and hand it to the oculus sdk. It does all the distortion rendering.
- latest sdk: we no longer give the oculus sdk the texture. Instead it wants to create the texture and give it to us to render into. It's the reverse of the previous sdk style. But ogre's texture management code is massive. Especially for DX11 and GL3 (which I'm not familiar with, I'm a dx9 coder), I'm not sure how much of ogre's code is needed. The texture manager doesn't support injecting textures from outside sources.
So my current plan is to leave ogre as is and try to trick the oculus sdk into using our textures instead of it's own. I think I'm missing setting a couple of member variables. Currently I'm giving ogre dx11 render textures to the oculus sdk and it's turning on and doing frame submits without errors, it's just showing black instead of the scene.
So no guarantees, but I'm slightly distracted enough to resume some hacky tries.