Head-tracking projection...

[_tommo_]

*

In fact i wasn't saying that your method is wrong... now you set the view matrix using setPosition and setOrientation, but you don't take into account depth, so you need to tweak the FOV.

And after this i think that also the projection matrix should be tweaked, but i don't know how.

BTW this can be actually done with ogre:
http://www.ogre3d.org/phpBB2/viewtopic.php?t=37376
I think that we would just need to apply that portal code to the screen, putting the camera in the real-world player position...

[Zoldan]

I just implemented the matrices from the video, which have the same effect. If i move my head away from the screen, it looks like i get closer. additionally the effect when moving around my head isnt "vr like" with them.

so if someone is out there, who knows how to do this, please help, cause i surrender. my code using the matrices is found below.

Code: Select all

		Vector3 cameraPosition = Vector3(-mHeadX, mHeadY, mHeadDist);
		Vector3 cameraTarget = Vector3(-mHeadX, mHeadY, 0);
		Vector3 cameraUpVector = Vector3(0, 1, 0);

		Vector3 zaxis = (cameraPosition - cameraTarget).normalisedCopy();
		Vector3 xaxis = cameraUpVector.crossProduct(zaxis).normalisedCopy();
		Vector3 yaxis = zaxis.crossProduct(xaxis);



		Matrix4 view = Matrix4(	xaxis.x, yaxis.x, zaxis.x, 0,
								xaxis.y, yaxis.y, zaxis.y, 0,
								xaxis.z, yaxis.z, zaxis.z, 0,
								-xaxis.dotProduct(cameraPosition), -yaxis.dotProduct(cameraPosition), -zaxis.dotProduct(cameraPosition), 1);


		mCamera->setCustomViewMatrix(true, view.transpose());




		float nearPlane = .05f;
		float farDist = 100;
		
		float left = nearPlane * (-.5f * mCamera->getAspectRatio() + mHeadX) / mHeadDist;
		float right = nearPlane * (.5f * mCamera->getAspectRatio() + mHeadX) / mHeadDist;
		float bottom = nearPlane * (-.5f - mHeadY) / mHeadDist;
		float top = nearPlane * (.5f - mHeadY) / mHeadDist;
		
		Real inv_w = 1 / (right - left);
		Real inv_h = 1 / (top - bottom);
		Real inv_d = 1 / (nearPlane - farDist);

		Real A = 2 * nearPlane * inv_w;
		Real B = 2 * nearPlane * inv_h;
		Real C = (right + left) * inv_w;
		Real D = (top + bottom) * inv_h;

		Real q, qn;
		q = - (farDist) * inv_d;
		qn = -2 * (farDist * nearPlane) * inv_d;

		Matrix4 proj = Matrix4::ZERO;
		proj[0][0] = A;
		proj[0][2] = C;
		proj[1][1] = B;
		proj[1][2] = D;
		proj[2][2] = q;
		proj[2][3] = qn;
		proj[3][2] = -1;

		mCamera->setCustomProjectionMatrix(true, proj);

[_tommo_]

I just implemented the matrices from the video, which have the same effect. If i move my head away from the screen, it looks like i get closer. additionally the effect when moving around my head isnt "vr like" with them.

Tried -mHeadDist in the position? The z axis should be positive entering the screen and negative in the other direction.
The head is outside the screen, so its world z position should be negative... at least by logic.

For the matrix code i don't know what to say...

[beaugard]

Does this help?

http://www.gamedev.net/community/forums ... _id=467203

[Downsider]

Dude, PLEASE recreate this, and make it goddamn Vista compatable. The "offical" Johnny Lee creation doesn't run on Vista z_Z.

[chrisbaume]

Hey everyone,

I've just signed up as I'm developing a low-cost head tracking device for my final year university project (aka dissertation, thesis). It only uses 4 LDRs, 2 LEDs and a microprocessor, so it should be extremely cheap to make and will take over the world once we find a manufacturer. It's not GPL (especially as it's patented) but I don't see any reason why you won't be able to read my project report and build your own when I'm done.

Anyway, I was looking to make a demo of its applications, hence I'm here to find out more about Ogre. I was quite pleased to find lots of people talking about headtracking after the now famous Johnny Lee recently popularised the idea. Hopefully we'll be able to share some ideas so I can put together a kick-ass demo program!

Chris

[cybereality]

I was quite pleased to find lots of people talking about headtracking after the now famous Johnny Lee recently popularised the idea. Hopefully we'll be able to share some ideas so I can put together a kick-ass demo program!

Sounds cool man. I've been trying to work on a headtracking demo for a while now, I finally have the equipment I need, just need some more time. I'm not using the wiimote, I have another method for the tracking but I don't want to give away too much before I release a demo. After I test my idea I will be more than happy to share.

[Grim]

I have my Projection-Plane and my Eye-Position, can anybody tell me how i get a ProjectionMatrix from these?

[cybereality]

Is there a whitepaper or reference material that details this method of projection?

I've downloaded the WiiVR example, but there are some magic values in there and I'd like to understand the mechanics behind it and not just copy and paste Johnny's code.

[matthieu]

Ok, it took me half a day, but I finaly got the correct formula.

J. Lee is "cheating", as DirectX has "PerspectiveOffCenterLH" which does the trick.


For lateral (X-Y) headtracking, you need to rmultiply the projection matrix with M=
1 0 X X*znear
0 1 Y Y*znear
0 0 1 0
0 0 0 1

(newP = oldP*M)

X Y being the head position, znear being the near clipping pane distance.
This will shear the truncated pyramid matrix, leaving the znear pane (which is your screen) where it is.

I made a small openGL/glut demo that you can find here:
http://www.nottale.net/glenv.cc
Headtracking is simulated by pressing keys.

Hope it helps.

[cybereality]

Ok, it took me half a day, but I finaly got the correct formula.

J. Lee is "cheating", as DirectX has "PerspectiveOffCenterLH" which does the trick.

Hope it helps.

Thanks a lot. That was exactly what I was looking for.

I'd looked through Lee's code, but most of it was to calculate the heads 3d position (which I already have). I got as far as getting to the PerspectiveOffCenterLH, but I wasn't sure how that would work in Ogre. I tried just playing with the projection and view matricies but that was giving wonky results (although some cool ideas for making like a music visualizer).

Much appreciation for the tip.

[Kojack]

I've had the parts for my home made sensor bar on my floor mounted on a breadboard for a week, I just haven't gotten around to soldering and stuff. Tested it though, and the wiimote picks it up ok.

[matthieu]

I have the wiimotes, but the IR leds I ordered did not arrive yet

In the meantime I replaced the wiimote/IR with a webcam, using the face detection code in OpenCV.
It works quite well for X-Y, not that well for distance. However I need to use low camera resolution (160x120) otherwise the algorithm eats too much CPU.

[Y_Less]

Hi

I was wondering how far people got with implementing this. The reason I ask is that I thought this would be a good project for Google's summer of code. As you may have gathered I am contemplating applying to do some work for OGRE and this was my first thought, however it would be very pointless to do if it's already been done.

A few things spring to mind after reading this topic.

1) All current discussion seems to revolve around Windows and DirectX, this is perfectly good start point, and most likely where I would begin, however in order for it to be really useful to OGRE it would need porting over, and this would hopefully serve to increase the project to the levels required for a GSOC project.

2) Another thing I have seen no mention of is collision, this could simply be because no-one has got that far yet, you simply didn't think of it, or that you decided it was outside the realms of scope of a graphics engine, I don't know. Anyway, while the latter item may well be true there should be some form of interface to restrict movement, and to stop updating the display angle beyond a certain point, so although OGRE may not be handling the world interaction, it can still react to, and prevent you peering through, walls and other items.

I appologise for the long initial post, however I was wondering if, assuming no-one else has done it already, this would be something people would like to see in OGRE, and if so if there would be anyone about willing to mentor it as a GSOC project.

Thanks in advance

Alex "Y_Less" Cole

[andytheb]

Hi,

there is Trackhead as a proof of concept:
http://www.burtzlaff.de/trackhead.html

It uses a webcam and two bright leds (not necessarily irleds) for head tracking and Irrlicht as 3D engine. Porting to Ogre should be trivial.
Please mind, that the generation of the projection matrix according to the opengl specs does not work for some strange and unknown reason.

Andy

[mr. iknoweverything]

guys, why always those LEDs on the friggin head? AFAIK eyes reflect IR light very nicely. there you go, you not only get head tracking, eye tracking for free too!1!!!!zomg!1!!

[Sundown]

Did anyone manage to get this working in the end? I've been looking at doing a similar thing in my project, but I don't want to re-invent the wheel so to speak!

[NazguL]

I'm upping this post, because I would really like to know if anyone managed to create a working head tracking with OGRE.
I tried the one present on the wiki but it seems to have some problems...

[iapafoto]

If it could help someone, this code works fine for me, I use it with OpenNi with good results.

• * Head coordinates are relative to the center of the screen (z is positive).
  * Center of the screen is 0,0,0.
  * mScreenX and mScreenY are screen size in the same unit as posHead coordinates what enables to represent virtual objects with true sizes.

Code: Select all

void Application::updateCamera(Vector3 posHead)
{
	mCamera->setPosition(posHead);
	mCamera->setDirection(0,0,-1);

	float near=1;
	mCamera->setNearClipDistance(near); 

	float l = near*(-.5 * mScreenX - posHead.x)/posHead.z; 
	float r = near*( .5 * mScreenX - posHead.x)/posHead.z; 
	float b = near*(-.5 * mScreenY - posHead.y)/posHead.z; 
	float t = near*( .5 * mScreenY - posHead.y)/posHead.z; 

	mCamera->setFrustumExtents(l, r, t, b);
}

Good results also using :

Code: Select all

void Application::updateCamera(Vector3 posHead)
{
	mCamera->setPosition(posHead);
	mCamera->setDirection(0,0,-1);
	mCamera->setNearClipDistance(1); 
	mCamera->setFOVy(2*Radian(atan((mScreenY/2) / fabs(posHead.z))));
	mCamera->setFrustumOffset(Vector2(-posHead.x/posHead.z, -posHead.y/posHead.z));
}

[jacmoe]

1 post and 1 kudos - that's the way !
Thanks for the code, Iapafoto.

[Kojack]

Hmm, forgot about this topic.

Did anyone manage to get this working in the end? I've been looking at doing a similar thing in my project, but I don't want to re-invent the wheel so to speak!

I'm upping this post, because I would really like to know if anyone managed to create a working head tracking with OGRE.

I'm bit late, but better than nothing.
After playing with the wiimote stuff like I mentioned back in 2007 in the beginning of the thread and showing it off in class, some of my students in 2008 decided to make a game with it. Time Crisis style rail shooter using ogre, a projector, wiimote to shoot, another wiimote for head tracking and ir led headset. You physically dodge incoming fire using full body motion.
http://www.youtube.com/watch?v=6fYIZvCYpOQ
(The view in the game is scripted at various points like when moving forward, so sometimes the video doesn't seem to respond to the player's movement).
It worked really well. Kinect would be better (so no head mounted ir lights), but we didn't have one 3 years ago. I'd still want wiimotes for aim though.
I don't have the source code, so can't give anything out.