DirectShowVMR9 Plugin

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

DirectShowVMR9 Plugin

Post by Pagghiu »

Hi all,
following this thread i've created this Wiki Entry which has all the necessary info to make VMR9 and Ogre cooperate.
Please move this thread if this is not the right zone to post :roll:
I've created a texture plugin that uses VMR9 to load video streams
directly into texture (using available hardware acceleration).

In this simple demo I'm acquiring video from two cameras (USB and Firewire) and streaming an AVI file.
The firewire camera gets deinterlaced in hardware by VMR9 (that is the main reason why I've coded this beast).

Image

This sample only works on the Direct 3D 9 RenderSystem...

Have a nice day :)
Last edited by Pagghiu on Mon Mar 22, 2010 5:32 pm, edited 1 time in total.
User avatar
CaseyB
OGRE Contributor
OGRE Contributor
Posts: 1335
Joined: Sun Nov 20, 2005 2:42 pm
Location: Columbus, Ohio
x 3

Post by CaseyB »

Very Nice!!
Image
Image
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Post by xavier »

For some of us, D3D9 is enough. ;)

BTW, are you locking frame rate to 25 Hz to match the camera output, or does streaming in video drag down performance that much?
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Post by Pagghiu »

no, the code is structured to run video streaming in a separate thread.
I've got 25 fps because streaming from 2 cameras and a divx video is too much for my poor 2 ghz single core laptop :wink:
If i use only one camera for example (the firewire can deliver only 25 fps) I can get higher FPS
Image

Edit:
I'm working on an integration with the ARToolkitPlus Library http://studierstube.icg.tu-graz.ac.at/h ... itplus.php as it was done
here ,here and here.

I Could not get the first example to work...and it uses some strange coordinate system conversion that I've not understood :wink:
The OgreAR library is a wrapping of the first example and is hardcoded for PGR cameras so it's not usable.
The third link refers to Mr. Planet which uses ARTK and not ARTKPlus for tracking, and I need ARTKPlus :)

So I got it to working quite well by myself

Image

I wanted some hints on the best way to integrate this into a plugin for Ogre. To get perfect overlapping between markers and 3D Objects I need to set the View matrix to identity and a custom Proj Matrix.

I my code I do something like this:

Code: Select all

		Entity* cube = mSceneMgr->createEntity("TestCube",SceneManager::PT_CUBE);
		cube->getSubEntity(0)->setUseIdentityView(true);
		SceneNode* cubeNode = mSceneMgr->getRootSceneNode()->createChildSceneNode();
		cube->setMaterialName("arcube");
		SceneNode* subNode		= cubeNode->createChildSceneNode();
		subNode->attachObject(cube);
because subNode->setUseIdentityView(true) does not work.
Is a simple way of setting setUseIdentityView() on all subentities of a node?

I was trying something like this:

Code: Select all

	void setIdentityViewTrueOnAllEntities(Ogre::Node* node)
	{
		SceneNode::ObjectIterator it = node->getAttachedObjectIterator();
		while(it.hasMoreElements())
		{
			MovableObject* obj = it.getNext();
			String type = obj->getMovableType();
			if(type=="Entity")
			{
				Entity* ent = (Entity*) obj;
				int num = ent->getNumSubEntities();
				for(int i = 0; i < num; i++)
				{
					ent->getSubEntity(i)->setUseIdentityView(true);
				}
			}
		}
		SceneNode::ChildNodeIterator nodeIt = node->getChildIterator();
		while(nodeIt.hasMoreElements())
		{
			//recurse
			setIdentityViewTrueOnAllEntities(nodeIt.getNext());
		}
	}
but this does not compile as getAttachedObjectIterator() works only on scenenode.
getChildIterator() returns nodes instead of scene nodes
Is there a simple way of knowing if a node is a scenenode :?:
User avatar
SiENcE
Goblin
Posts: 231
Joined: Thu May 11, 2006 3:07 pm
Location: Berlin

Post by SiENcE »

Nice work.

Why do you have so less fps?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Cool. Just to make the wiki entry more long-lasting and not dependent on your web server, I would create a small archive with just the source (no binaries, and no external deps) and upload it to the wiki instead. The wiki is fine for hosting relatively small files and that way it's self-contained.
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Post by Pagghiu »

Oh yes, I will post the updated sources without the binaries.

Regarding the other question:
In the above shots I had some debug code enabled but there is another reason.

If you also need to capture the frame content for additional software postprocessing, like i did for the AR application, you can't simply insert a samplegrabber in the dshow chain.
I've tried this and reading the samples is very slow (It seems they are allocated in video memory).
The solution is to insert an infinite tee filter and split the video signal before going to VMR9. One stream goes directly to VMR9 and gets accelerated. The other stream goes to the sample grabber and delivers frames to the application that can process them. this stream use additional threads.

So, if you need software capture, on single core machine is probably better using the "classic" approach of grabbing frames and dumping them into a dynamic texture.
If you only need to display the texture instead, VMR9 is faster than dynamic texture.

In an updated version of my code I've added the ability to enable/disable VMR9 on a per material basis.

Code: Select all

		texture_source dsvmr9_video
		{
                    //filename try.avi
                    camera_number 0 //Defines which webcam to use.
                    use_vmr9 true
                    use_yuvmixing true
                    capture false
		}
I can also enable/disable YUV mixing (that enables the hardware acceleration).
'capture' parameter specifies if software capture is needed for this material, to avoid the stream branching I was talking above.

If I disable Capturing I easily go over 100-150 fps using VMR9.



There is also a third problem I've come across.
The ExternalTextureSource::createAdvancedTexture gets called for every parsed material script...so the first version of my code inited all my collection of "avi" materials. Not very optimal :)
I did something along the lines of:

Code: Select all

	bool OgreDSVMR9Controller::frameStarted(const Ogre::FrameEvent &e)
	{
		mtCamTexs::iterator i;
		for(i = mOgreDSVMR9TextureList.begin(); i != mOgreDSVMR9TextureList.end(); ++i)
		{
			(*i)->UpdateTexture();
		}
		return true;
	}

	bool OgreDSVMR9Texture::UpdateTexture()
	{
		if(mMaterial->getLoadingState()==Resource::LOADSTATE_LOADED &&
			!mbAlreadyLoaded)
		{
			//let's load
			if(!mbAlreadyFailed)
			{
				if(!load())
					mbAlreadyFailed = true;
				else
					mbAlreadyLoaded = true;
			}
		}
		if(mOgreDSVMR9)
			return mOgreDSVMR9->Update();
		return false;
	}

Is there a way of getting notified when the associated material gets "loaded"?

thanks
hippynerd
Gnoblar
Posts: 2
Joined: Fri Dec 28, 2007 12:48 pm

Post by hippynerd »

Hi Pagghiu, this is fantastic. I spent all yesterday reading up on VMR9 so that I could implement this myself, and here you have it nicely wrapped up :) Good thing I looked at the Wiki. I haven't had a chance to delve into your code yet, I've only run the included binary, but from what I can see, this is great.
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Post by Pagghiu »

Thanks :)

I just wanted you to know that there are some minor bugs in the posted code...I'll post an updated/cleaned version as soon as I'll free myself from an imminent deadline :roll:
Expect it in a couple of weeks :wink:
3D Coder
hippynerd
Gnoblar
Posts: 2
Joined: Fri Dec 28, 2007 12:48 pm

Post by hippynerd »

Heya Pagghiu,

Hope you made your deadline last year :) Made anymore progress with this, (or otherwise documented any of those bugs?)

cheers
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Post by Pagghiu »

Oh yes I did it :)

I've updated the wiki page linked at the top of this thread, check it out.
3D Coder
totenhose
Gnoblar
Posts: 8
Joined: Tue Nov 06, 2007 5:04 pm

Post by totenhose »

Great work Pagghiu with DirectShowVMR9 plugin.
I have some questions too.

1. Could you compile the plugin for Ogre 1.4.6. I tried but didn't succeed.
2. Is it possible to implement software video demultiplexing - to write different frame contents on different textures?
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Post by Pagghiu »

1. take it at http://pagghiu.italica.info/tmp/ogre/DS ... e1.4.6.zip

2. I have absolutely no idea on how to implement demultiplexing using VMR9...anyway if you want it you'll have to deeply modify sources as they assume a single video stream per graph.

I've recently found some problems regarding videos streaming from file using vmr9 mode...seems like my pattern that waits into Allocator::GetSurface is not well accepted by VMR9. Sometimes it "blocks" the whole graph from delivering video frames.
Everything works ok with live video feed (from camera, etc.).
One day I'll investigate this issue :roll:
3D Coder
User avatar
manowar
Orc
Posts: 419
Joined: Thu Apr 07, 2005 2:11 pm
Location: UK

Post by manowar »

Hello,

I am using that plugin to play a video. It works fine, except that it seems to lock the framerate to the video fps. I am rendering a scene on the top of the video and 25fps is a bit too low. I tried to look at the code but considering that I do not have any experience at all with dshow, I do not understand much. Is it possible to change that ?

Thanks
User avatar
chilligirl
Kobold
Posts: 31
Joined: Sun Apr 13, 2008 10:28 pm

Runtime error problem

Post by chilligirl »

Hey there, I have a problem, I used your awesome code to implement cutscenes for my game, the only problem is that it works fine with any video footage that I downloaded from the web e.g. short videos, and it works fine, but as soon as I insert video that I made and compressed myself, it throws a runtime error. Any help? I compile my code in debugging mode in VS 2005 and then test it with ctrl-f5 to run it without the debugger, because DirectX complains. I compress my RAW AVI with Xvid and I do test it in media player and media player classic. Please help?!
The knack in flying lies in throwing your body at the ground and miss...
neonovo
Gnoblar
Posts: 4
Joined: Mon Mar 22, 2010 11:04 am

Re: DirectShowVMR9 Plugin

Post by neonovo »

the link seems to be dead ... Is there another way to download the plugin ?
User avatar
Pagghiu
Gnoblar
Posts: 15
Joined: Fri Dec 07, 2007 1:15 am
Location: Foggia,Italy

Re: DirectShowVMR9 Plugin

Post by Pagghiu »

I've updated the link on the wiki page :)
3D Coder
neonovo
Gnoblar
Posts: 4
Joined: Mon Mar 22, 2010 11:04 am

Re: DirectShowVMR9 Plugin

Post by neonovo »

thanks a lot :)
neonovo
Gnoblar
Posts: 4
Joined: Mon Mar 22, 2010 11:04 am

Re: DirectShowVMR9 Plugin

Post by neonovo »

Hi,

I am trying to understand how works your plugin. I have done a custom allocator like you:
- I create a normal texture in ogre (RGBA for instant ... but I needs to be yuv to have accelerated dxvA)
- I pass the IDirect3DTexture9 to the custom allocator hacking ogre like this :
IDirect3DTexture9 *texture;
Ogre::D3D9TexturePtr td=(Ogre::D3D9TexturePtr)ogreTexture;
texture=td.getPointer()->getNormTexture();
allocator->setTexture(texture);
- In the PresentImage call : I do this :
if( m_ogreTexture != NULL )
{
IDirect3DSurface9Ptr surface;
FAIL_RET( m_ogreTexture->GetSurfaceLevel( 0 , & surface ) );

// copy the full surface onto the texture's surface
hr=(m_direct3DDevice9->StretchRect(lpPresInfo->lpSurf,NULL,surface,NULL,D3DTEXF_NONE));

but the StretchRect failed ( I have recompiled ogre directx plugin with extraFlags |= D3DCREATE_MULTITHREADED; )

It seems you don't do that ?
Stickymango
Hobgoblin
Posts: 540
Joined: Wed Nov 17, 2004 4:36 pm
Location: Middlesbrough, England

Re: DirectShowVMR9 Plugin

Post by Stickymango »

HI There,

I'm using this plugin to render a video as a background within a 3D scene and it works great, however I'd like to reuse the texture in either another technique or another material but it doesn't seem to work.

I've tried manually cloning the original material and setting the texture name to the name returned by the video texture unit, but it fails to render any video content.

I've also tried to create a second technique and setting the same texture name with the same results.

Obviously it works fine if I duplicate the video texture technique but it also renders the video file twice which is want to avoid, is it possible to re-use the video texture multiple times? is there a function for getting the Ogre::Texture used or do I need to modify the source?

Thanks,
Stickymango
Hobgoblin
Posts: 540
Joined: Wed Nov 17, 2004 4:36 pm
Location: Middlesbrough, England

Re: DirectShowVMR9 Plugin

Post by Stickymango »

Solved, I needed to ensure the OgreDSVMR9Texture was loaded before setting it in another textureUnit, thanks to this plugin I've now got a 1080i live video feed running at 170fps!! :)

Thanks for this excellent plugin!
kiske
Greenskin
Posts: 106
Joined: Wed Jul 09, 2003 1:15 pm
Location: Trentino - ITALY
x 1

Re: DirectShowVMR9 Plugin

Post by kiske »

Hi!

for use with new Ogre version, I've tried to recompile the plugin, but when it links it didn't find dsvld.lib. Could someone please point me out where to find or build it?

many thanks in advance!
<Kiske>

Well can you blame us? For living our dreams...
- Twisted Sister -
kiske
Greenskin
Posts: 106
Joined: Wed Jul 09, 2003 1:15 pm
Location: Trentino - ITALY
x 1

Re: DirectShowVMR9 Plugin

Post by kiske »

For anyone who needs to use this nice plugin with 1.7 here's some indications:

http://www.ogre3d.org/forums/viewtopic.php?f=2&t=61946
<Kiske>

Well can you blame us? For living our dreams...
- Twisted Sister -
mciccarone
Gnoblar
Posts: 4
Joined: Fri Feb 04, 2011 11:43 pm
Location: Orlando, FL

Re: DirectShowVMR9 Plugin

Post by mciccarone »

I know i'm resurrecting a zombie thread here, but I'd like to know if the links have been updated for the VMR9 plugin.

Thanks
Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany
x 110

Re: DirectShowVMR9 Plugin

Post by Transporter »

The source code from OGRE Addon SVN is not working anymore. You have to modify it. Maybe I can fix it. During the last years the SDK stuctures has changed. DirectShow is now part of the platform SDK and the used Qedit.h has been removed completely.