Videostream to Ogre

[peetcreative]

Hey,
I started a project with Ogre.
The aim is to display live video-stream of a stereo camera in a VR headset, additionally I want to add objects so they overlay the scene from the video-stream.
Sadly I haven't done anything before with ogre. I choose ogre 2.2 because I found the openvr example from dark_sylinc. (https://www.ogre3d.org/tag/openvr)

I implemented drawing the stereo video-stream in the following way, and I wonder if this is the proper way one would do this in ogre:
Every Frame
Load Videoframe from OpenCv Capture or ROS...
Resize in OpenCv::Mat
Write the resized Mat to the "Background"-Texture, which resides on GPU as well as RAM:
I've done it the way described in the doku: (https://ogrecave.github.io/ogre-next/ap ... anges.html)

I have to wait VaoManager->getDynamicBufferMultiplier Frames.
Does anyone has a better or the proper approach streaming content to an ogre scene? You can also have a look on my repo: https://github.com/peetCreative/svr
Loading images is done in SVR.cpp and writing them to texture in OpenVrCompositorListener.cpp
If you find programming or application design fault, please just let me know.

Thanks!

[xrgo]

Hello! and welcome!!
I have made something similar using a camera like this:


So I have some comments of my short experience:
-the fov was very low, even tho I purchased a wide one, don't remember the degrees it supposed to have
-the fps was very low (it was a usb2.0 camera) so you get a lot of delay while moving the head, you could implement a "timewarping" technique but for that to work consistently you'll need to fix the exposure so it wont look good on every scenario
-resolution was very bad (again usb2.0, so no surprise)

I hope I don't sound too discouraging, but it was the camera's fault, Ogre is definitely up to the task. I didn't find any other camera that would work except for this one https://www.stereolabs.com/zed-mini/ but its very expensive, which camera are you using? the better alternative is to use a VR headset that has stereo cameras already like a vive pro.

I have some code I used to test this:

but be aware its very messy (just for testing) and it's definitely not the proper way to do it, but you can get some ideas anyways. I had to run the capture in a separate thread so it wont hold the main rendering thread which needs to be at 90 fps (depending on vr headset used), so for simplicity of testing I just used a mCaptureReady bool flag as mutex. And I had to crop and center manually the texture to fit the real world and look stereo.

I haven't looked at your code, sorry, really busy

Saludos!

[peetcreative]

Thank you for your response. Most interestingly is your frameRenderingQueued function. Hope I can learn sth from it.
I thaught about technical latencies as well and it's one of my first steps meassurig and reducing them to a minimum. It's a student project with a medical stereoendoscope so I hope to have acceptable latencies on this side.
The forum editor dropped this message two times so it got a bit shorter.