Hey,
I started a project with Ogre.
The aim is to display live video-stream of a stereo camera in a VR headset, additionally I want to add objects so they overlay the scene from the video-stream.
Sadly I haven't done anything before with ogre. I choose ogre 2.2 because I found the openvr example from dark_sylinc. (https://www.ogre3d.org/tag/openvr)
I implemented drawing the stereo video-stream in the following way, and I wonder if this is the proper way one would do this in ogre:
Every Frame
Load Videoframe from OpenCv Capture or ROS...
Resize in OpenCv::Mat
Write the resized Mat to the "Background"-Texture, which resides on GPU as well as RAM:
I've done it the way described in the doku: (https://ogrecave.github.io/ogre-next/ap ... anges.html)
I have to wait VaoManager->getDynamicBufferMultiplier Frames.
Does anyone has a better or the proper approach streaming content to an ogre scene? You can also have a look on my repo: https://github.com/peetCreative/svr
Loading images is done in SVR.cpp and writing them to texture in OpenVrCompositorListener.cpp
If you find programming or application design fault, please just let me know.
Thanks!
Videostream to Ogre
-
- OGRE Expert User
- Posts: 1148
- Joined: Sat Jul 06, 2013 10:59 pm
- Location: Chile
- x 168
Re: Videostream to Ogre
Hello! and welcome!!
I have made something similar using a camera like this:
So I have some comments of my short experience:
-the fov was very low, even tho I purchased a wide one, don't remember the degrees it supposed to have
-the fps was very low (it was a usb2.0 camera) so you get a lot of delay while moving the head, you could implement a "timewarping" technique but for that to work consistently you'll need to fix the exposure so it wont look good on every scenario
-resolution was very bad (again usb2.0, so no surprise)
I hope I don't sound too discouraging, but it was the camera's fault, Ogre is definitely up to the task. I didn't find any other camera that would work except for this one https://www.stereolabs.com/zed-mini/ but its very expensive, which camera are you using? the better alternative is to use a VR headset that has stereo cameras already like a vive pro.
I have some code I used to test this:
but be aware its very messy (just for testing) and it's definitely not the proper way to do it, but you can get some ideas anyways. I had to run the capture in a separate thread so it wont hold the main rendering thread which needs to be at 90 fps (depending on vr headset used), so for simplicity of testing I just used a mCaptureReady bool flag as mutex. And I had to crop and center manually the texture to fit the real world and look stereo.
I haven't looked at your code, sorry, really busy
Saludos!
I have made something similar using a camera like this:
So I have some comments of my short experience:
-the fov was very low, even tho I purchased a wide one, don't remember the degrees it supposed to have
-the fps was very low (it was a usb2.0 camera) so you get a lot of delay while moving the head, you could implement a "timewarping" technique but for that to work consistently you'll need to fix the exposure so it wont look good on every scenario
-resolution was very bad (again usb2.0, so no surprise)
I hope I don't sound too discouraging, but it was the camera's fault, Ogre is definitely up to the task. I didn't find any other camera that would work except for this one https://www.stereolabs.com/zed-mini/ but its very expensive, which camera are you using? the better alternative is to use a VR headset that has stereo cameras already like a vive pro.
I have some code I used to test this:
but be aware its very messy (just for testing) and it's definitely not the proper way to do it, but you can get some ideas anyways. I had to run the capture in a separate thread so it wont hold the main rendering thread which needs to be at 90 fps (depending on vr headset used), so for simplicity of testing I just used a mCaptureReady bool flag as mutex. And I had to crop and center manually the texture to fit the real world and look stereo.
I haven't looked at your code, sorry, really busy
Saludos!
-
- Gnoblar
- Posts: 7
- Joined: Tue Feb 11, 2020 10:19 pm
Re: Videostream to Ogre
Thank you for your response. Most interestingly is your frameRenderingQueued function. Hope I can learn sth from it.
I thaught about technical latencies as well and it's one of my first steps meassurig and reducing them to a minimum. It's a student project with a medical stereoendoscope so I hope to have acceptable latencies on this side.
The forum editor dropped this message two times so it got a bit shorter.
I thaught about technical latencies as well and it's one of my first steps meassurig and reducing them to a minimum. It's a student project with a medical stereoendoscope so I hope to have acceptable latencies on this side.
The forum editor dropped this message two times so it got a bit shorter.