Head-tracking projection...

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 3
Contact:

Post by _tommo_ »

Zoldan wrote:*
In fact i wasn't saying that your method is wrong... now you set the view matrix using setPosition and setOrientation, but you don't take into account depth, so you need to tweak the FOV.

And after this i think that also the projection matrix should be tweaked, but i don't know how.

BTW this can be actually done with ogre:
http://www.ogre3d.org/phpBB2/viewtopic.php?t=37376
I think that we would just need to apply that portal code to the screen, putting the camera in the real-world player position...
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games

Zoldan
Gnoblar
Posts: 9
Joined: Wed Dec 22, 2004 4:14 am

Post by Zoldan »

I just implemented the matrices from the video, which have the same effect. If i move my head away from the screen, it looks like i get closer. additionally the effect when moving around my head isnt "vr like" with them.

so if someone is out there, who knows how to do this, please help, cause i surrender. my code using the matrices is found below.

Code: Select all

		Vector3 cameraPosition = Vector3(-mHeadX, mHeadY, mHeadDist);
		Vector3 cameraTarget = Vector3(-mHeadX, mHeadY, 0);
		Vector3 cameraUpVector = Vector3(0, 1, 0);

		Vector3 zaxis = (cameraPosition - cameraTarget).normalisedCopy();
		Vector3 xaxis = cameraUpVector.crossProduct(zaxis).normalisedCopy();
		Vector3 yaxis = zaxis.crossProduct(xaxis);



		Matrix4 view = Matrix4(	xaxis.x, yaxis.x, zaxis.x, 0,
								xaxis.y, yaxis.y, zaxis.y, 0,
								xaxis.z, yaxis.z, zaxis.z, 0,
								-xaxis.dotProduct(cameraPosition), -yaxis.dotProduct(cameraPosition), -zaxis.dotProduct(cameraPosition), 1);


		mCamera->setCustomViewMatrix(true, view.transpose());




		float nearPlane = .05f;
		float farDist = 100;
		
		float left = nearPlane * (-.5f * mCamera->getAspectRatio() + mHeadX) / mHeadDist;
		float right = nearPlane * (.5f * mCamera->getAspectRatio() + mHeadX) / mHeadDist;
		float bottom = nearPlane * (-.5f - mHeadY) / mHeadDist;
		float top = nearPlane * (.5f - mHeadY) / mHeadDist;
		
		Real inv_w = 1 / (right - left);
		Real inv_h = 1 / (top - bottom);
		Real inv_d = 1 / (nearPlane - farDist);

		Real A = 2 * nearPlane * inv_w;
		Real B = 2 * nearPlane * inv_h;
		Real C = (right + left) * inv_w;
		Real D = (top + bottom) * inv_h;

		Real q, qn;
		q = - (farDist) * inv_d;
		qn = -2 * (farDist * nearPlane) * inv_d;

		Matrix4 proj = Matrix4::ZERO;
		proj[0][0] = A;
		proj[0][2] = C;
		proj[1][1] = B;
		proj[1][2] = D;
		proj[2][2] = q;
		proj[2][3] = qn;
		proj[3][2] = -1;

		mCamera->setCustomProjectionMatrix(true, proj);

User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 3
Contact:

Post by _tommo_ »

Zoldan wrote:I just implemented the matrices from the video, which have the same effect. If i move my head away from the screen, it looks like i get closer. additionally the effect when moving around my head isnt "vr like" with them.
Tried -mHeadDist in the position? The z axis should be positive entering the screen and negative in the other direction.
The head is outside the screen, so its world z position should be negative... at least by logic.

For the matrix code i don't know what to say... :?
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games

beaugard
OGRE Contributor
OGRE Contributor
Posts: 265
Joined: Sun Mar 25, 2007 1:48 pm
x 2

Post by beaugard »


Downsider
Gnoblar
Posts: 8
Joined: Sat Dec 01, 2007 10:21 pm

Post by Downsider »

Dude, PLEASE recreate this, and make it goddamn Vista compatable. The "offical" Johnny Lee creation doesn't run on Vista z_Z.

chrisbaume
Gnoblar
Posts: 1
Joined: Mon Jan 07, 2008 4:50 pm

Post by chrisbaume »

Hey everyone,

I've just signed up as I'm developing a low-cost head tracking device for my final year university project (aka dissertation, thesis). It only uses 4 LDRs, 2 LEDs and a microprocessor, so it should be extremely cheap to make and will take over the world once we find a manufacturer. It's not GPL (especially as it's patented) but I don't see any reason why you won't be able to read my project report and build your own when I'm done.

Anyway, I was looking to make a demo of its applications, hence I'm here to find out more about Ogre. I was quite pleased to find lots of people talking about headtracking after the now famous Johnny Lee recently popularised the idea. Hopefully we'll be able to share some ideas so I can put together a kick-ass demo program!

Chris

User avatar
cybereality
Hobgoblin
Posts: 563
Joined: Wed Jul 12, 2006 5:40 pm
x 12

Post by cybereality »

chrisbaume wrote:I was quite pleased to find lots of people talking about headtracking after the now famous Johnny Lee recently popularised the idea. Hopefully we'll be able to share some ideas so I can put together a kick-ass demo program!
Sounds cool man. I've been trying to work on a headtracking demo for a while now, I finally have the equipment I need, just need some more time. I'm not using the wiimote, I have another method for the tracking but I don't want to give away too much before I release a demo. After I test my idea I will be more than happy to share.

Grim
Gnoblar
Posts: 11
Joined: Mon Oct 08, 2007 3:24 pm

Post by Grim »

I have my Projection-Plane and my Eye-Position, can anybody tell me how i get a ProjectionMatrix from these?

User avatar
cybereality
Hobgoblin
Posts: 563
Joined: Wed Jul 12, 2006 5:40 pm
x 12

Post by cybereality »

Is there a whitepaper or reference material that details this method of projection?

I've downloaded the WiiVR example, but there are some magic values in there and I'd like to understand the mechanics behind it and not just copy and paste Johnny's code.

matthieu
Gnoblar
Posts: 2
Joined: Sat Jan 19, 2008 7:59 pm

Post by matthieu »

Ok, it took me half a day, but I finaly got the correct formula.

J. Lee is "cheating", as DirectX has "PerspectiveOffCenterLH" which does the trick.


For lateral (X-Y) headtracking, you need to rmultiply the projection matrix with M=
1 0 X X*znear
0 1 Y Y*znear
0 0 1 0
0 0 0 1

(newP = oldP*M)

X Y being the head position, znear being the near clipping pane distance.
This will shear the truncated pyramid matrix, leaving the znear pane (which is your screen) where it is.

I made a small openGL/glut demo that you can find here:
http://www.nottale.net/glenv.cc
Headtracking is simulated by pressing keys.

Hope it helps.

User avatar
cybereality
Hobgoblin
Posts: 563
Joined: Wed Jul 12, 2006 5:40 pm
x 12

Post by cybereality »

matthieu wrote:Ok, it took me half a day, but I finaly got the correct formula.

J. Lee is "cheating", as DirectX has "PerspectiveOffCenterLH" which does the trick.

Hope it helps.
Thanks a lot. That was exactly what I was looking for.

I'd looked through Lee's code, but most of it was to calculate the heads 3d position (which I already have). I got as far as getting to the PerspectiveOffCenterLH, but I wasn't sure how that would work in Ogre. I tried just playing with the projection and view matricies but that was giving wonky results (although some cool ideas for making like a music visualizer).

Much appreciation for the tip.

User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7153
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 519

Post by Kojack »

I've had the parts for my home made sensor bar on my floor mounted on a breadboard for a week, I just haven't gotten around to soldering and stuff. Tested it though, and the wiimote picks it up ok.

matthieu
Gnoblar
Posts: 2
Joined: Sat Jan 19, 2008 7:59 pm

Post by matthieu »

I have the wiimotes, but the IR leds I ordered did not arrive yet :)

In the meantime I replaced the wiimote/IR with a webcam, using the face detection code in OpenCV.
It works quite well for X-Y, not that well for distance. However I need to use low camera resolution (160x120) otherwise the algorithm eats too much CPU.

Y_Less
Gnoblar
Posts: 3
Joined: Tue Mar 18, 2008 2:28 pm

GSOC Project

Post by Y_Less »

Hi

I was wondering how far people got with implementing this. The reason I ask is that I thought this would be a good project for Google's summer of code. As you may have gathered I am contemplating applying to do some work for OGRE and this was my first thought, however it would be very pointless to do if it's already been done.

A few things spring to mind after reading this topic.

1) All current discussion seems to revolve around Windows and DirectX, this is perfectly good start point, and most likely where I would begin, however in order for it to be really useful to OGRE it would need porting over, and this would hopefully serve to increase the project to the levels required for a GSOC project.

2) Another thing I have seen no mention of is collision, this could simply be because no-one has got that far yet, you simply didn't think of it, or that you decided it was outside the realms of scope of a graphics engine, I don't know. Anyway, while the latter item may well be true there should be some form of interface to restrict movement, and to stop updating the display angle beyond a certain point, so although OGRE may not be handling the world interaction, it can still react to, and prevent you peering through, walls and other items.

I appologise for the long initial post, however I was wondering if, assuming no-one else has done it already, this would be something people would like to see in OGRE, and if so if there would be anyone about willing to mentor it as a GSOC project.

Thanks in advance

Alex "Y_Less" Cole

andytheb
Gnoblar
Posts: 1
Joined: Sun Apr 06, 2008 3:11 pm

Post by andytheb »

Hi,

there is Trackhead as a proof of concept:
http://www.burtzlaff.de/trackhead.html

It uses a webcam and two bright leds (not necessarily irleds) for head tracking and Irrlicht as 3D engine. Porting to Ogre should be trivial.
Please mind, that the generation of the projection matrix according to the opengl specs does not work for some strange and unknown reason.

Andy

mr. iknoweverything
Halfling
Posts: 72
Joined: Thu Jul 12, 2007 5:39 pm
Location: berlin

Post by mr. iknoweverything »

guys, why always those LEDs on the friggin head? AFAIK eyes reflect IR light very nicely. there you go, you not only get head tracking, eye tracking for free too!1!!!!zomg!1!!

User avatar
Sundown
Gnoblar
Posts: 24
Joined: Fri Jun 15, 2007 3:32 pm

Post by Sundown »

Did anyone manage to get this working in the end? I've been looking at doing a similar thing in my project, but I don't want to re-invent the wheel so to speak!

NazguL
Kobold
Posts: 33
Joined: Tue Mar 09, 2010 2:17 am

Re: Head-tracking projection...

Post by NazguL »

I'm upping this post, because I would really like to know if anyone managed to create a working head tracking with OGRE.
I tried the one present on the wiki but it seems to have some problems...

iapafoto
Gnoblar
Posts: 1
Joined: Sun Mar 20, 2011 10:39 pm
x 1

Re: Head-tracking projection...

Post by iapafoto »

If it could help someone, this code works fine for me, I use it with OpenNi with good results.
  • * Head coordinates are relative to the center of the screen (z is positive).
    * Center of the screen is 0,0,0.
    * mScreenX and mScreenY are screen size in the same unit as posHead coordinates what enables to represent virtual objects with true sizes.

Code: Select all

void Application::updateCamera(Vector3 posHead)
{
	mCamera->setPosition(posHead);
	mCamera->setDirection(0,0,-1);

	float near=1;
	mCamera->setNearClipDistance(near); 

	float l = near*(-.5 * mScreenX - posHead.x)/posHead.z; 
	float r = near*( .5 * mScreenX - posHead.x)/posHead.z; 
	float b = near*(-.5 * mScreenY - posHead.y)/posHead.z; 
	float t = near*( .5 * mScreenY - posHead.y)/posHead.z; 

	mCamera->setFrustumExtents(l, r, t, b);
}
Good results also using :

Code: Select all

void Application::updateCamera(Vector3 posHead)
{
	mCamera->setPosition(posHead);
	mCamera->setDirection(0,0,-1);
	mCamera->setNearClipDistance(1); 
	mCamera->setFOVy(2*Radian(atan((mScreenY/2) / fabs(posHead.z))));
	mCamera->setFrustumOffset(Vector2(-posHead.x/posHead.z, -posHead.y/posHead.z));
}

User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: Head-tracking projection...

Post by jacmoe »

1 post and 1 kudos - that's the way ! :)
Thanks for the code, Iapafoto.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.

User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7153
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 519

Re: Head-tracking projection...

Post by Kojack »

Hmm, forgot about this topic. :)
Did anyone manage to get this working in the end? I've been looking at doing a similar thing in my project, but I don't want to re-invent the wheel so to speak!
I'm upping this post, because I would really like to know if anyone managed to create a working head tracking with OGRE.
I'm bit late, but better than nothing.
After playing with the wiimote stuff like I mentioned back in 2007 in the beginning of the thread and showing it off in class, some of my students in 2008 decided to make a game with it. Time Crisis style rail shooter using ogre, a projector, wiimote to shoot, another wiimote for head tracking and ir led headset. You physically dodge incoming fire using full body motion.
http://www.youtube.com/watch?v=6fYIZvCYpOQ
(The view in the game is scripted at various points like when moving forward, so sometimes the video doesn't seem to respond to the player's movement).
It worked really well. Kinect would be better (so no head mounted ir lights), but we didn't have one 3 years ago. :) I'd still want wiimotes for aim though.
I don't have the source code, so can't give anything out.

Post Reply