Problems manipulating ogre bones manually (with kinect)

[brunobg]

I'm trying to manipulate bones from models individually, to animate them using kinect. I know there's a lot of posts here about that, but even after reading them all I'm still facing some issues. My code is somewhat based on tuxsinbad, but I'm making it more flexible by not hardcoding bones. I have a map<ogre bone name, kinect joint> that handles the conversion. I'm setting up the bones (all bones) with:

Code: Select all

setupBone(const std::string &name) {
	Ogre::Bone* bone = entity->getSkeleton()->getBone(name);
	Ogre::Quaternion initial = bone->_getDerivedOrientation(); // save orientation
	bone->setManuallyControlled(true);
	bone->setInheritOrientation(false);
	bone->resetToInitialState();
	bone->resetOrientation();
	bone->setOrientation(initial); // restore it
	bone->setInitialState();
}

This keeps the model correct and gets rid of that ugly hardcoded setup in https://github.com/ttair/TuxSinbad/blob ... ntroller.h lines 696-726. Is it correct? It looks correct when rendered, but it may have something to do with the problem below.

Next is to alter the orientations of the bones. Here's how I'm doing it:

Code: Select all

void transformBone(const Ogre::String& modelBoneName, XnSkeletonJoint skelJoint) {
	Ogre::Skeleton* skel = entity->getSkeleton();
	Ogre::Bone* bone = skel->getBone(modelBoneName);
	Ogre::Quaternion qI = bone->getInitialOrientation();
	Ogre::Quaternion newQ = Ogre::Quaternion::IDENTITY;

	XnSkeletonJointOrientation jointOri = kinect->getSkeletonJointOrientation(skelJoint);

	if (jointOri.fConfidence > 0) {
		Ogre::Matrix3 matOri(jointOri.orientation.elements[0], -jointOri.orientation.elements[1], jointOri.orientation.elements[2],
				-jointOri.orientation.elements[3], jointOri.orientation.elements[4], -jointOri.orientation.elements[5],
				jointOri.orientation.elements[6], -jointOri.orientation.elements[7], jointOri.orientation.elements[8]);

		newQ.FromRotationMatrix(matOri);

		bone->resetOrientation(); //in order for the conversion from world to local to work.
		newQ = bone->convertWorldToLocalOrientation(newQ);

		bone->setOrientation(newQ * qI);
	}}
}

And now things get crazy Here's what happens to a human avatar (yeah, that thing on the top right is the head):

The individual parts are being tracked and linked correctly, but why does this odd scaling happens? I'm only changing orientations and the quaternions seem to be normalized. Sinbad.mesh does not have this elastic deformation, but it does not work well either: moving an arm does not move it exactly as it should, for example (but it's not completely crazy). And moving the right arm rotates the torso

Help, please!

[brunobg]

Apparently the weird deformation in the picture is caused by a problem in exporting from Blender that was described here: http://ogre3d.org/forums/viewtopic.php?f=2&t=68584 I tried exporting the Sinbad model myself and got the same problem (but the Sinbad model that comes with Ogre doesn't suffer from it, as I mentioned about). I'm reading about it, but if anybody have extra tips they would be appreciated.

This still does not explain why Sinbad does not pose as it should (anybody knows how the angles are measured in openni and in ogre?) and why the arm rotates the torso of Sinbad. For reference, this is the mapping I'm using:

Code: Select all

		bonesMap["Stomach"] = XN_SKEL_TORSO;
		bonesMap["Waist"] = XN_SKEL_WAIST;
		bonesMap["Root"] = XN_SKEL_WAIST;
		bonesMap["Chest"] = XN_SKEL_TORSO;
		bonesMap["Humerus.L"] = XN_SKEL_LEFT_SHOULDER;
		bonesMap["Humerus.R"] = XN_SKEL_RIGHT_SHOULDER;
		bonesMap["Ulna.L"] = XN_SKEL_LEFT_ELBOW;
		bonesMap["Ulna.R"] = XN_SKEL_RIGHT_ELBOW;
		bonesMap["Thigh.L"] = XN_SKEL_LEFT_HIP;
		bonesMap["Thigh.R"] = XN_SKEL_RIGHT_HIP;
		bonesMap["Calf.L"] = XN_SKEL_LEFT_KNEE;
		bonesMap["Calf.R"] = XN_SKEL_RIGHT_KNEE;

[azer89]

be careful with initial orientation of sinbad mesh. i never use Blender but you can check it using OgreMeshy



and how OpenNI can estimate joint estimation, roughly is similar to this (based on psi pose):
https://www.assembla.com/code/kinect-ms ... lculator.h

what the done is they need to obtain the three orthonormal axes (X, Y, and Z) and create Quaternion to change every bone orientation.

you don't have to worry about this joint orientation estimation, OpenNI done this automatically

[azer89]

oh yeah, i just remember have you checked the skin binding via Blender?
this skin binding means you give weighting how a bone will affect every vertex on skin when you move them.

why the torso is affected perhaps by automatic skin binding the hand joints have some effect for torso (which is should be zero)

[brunobg]

Updates:

Apparently the weird deformation in the picture is caused by a problem in exporting from Blender that was described here: http://ogre3d.org/forums/viewtopic.php?f=2&t=68584 I tried exporting the Sinbad model myself and got the same problem (but the Sinbad model that comes with Ogre doesn't suffer from it, as I mentioned about). I'm reading about it, but if anybody have extra tips they would be appreciated.

I followed those instructions, adding a root bone at the origin that is parent to every other bone. But I still get the same "elastic man" effect. It is not the skin binding: it also happens to the Sinbad mesh, and I checked the weighting. At least on Blender everything is fine, and I'm trying to get OgreMeshy to work. I think the problem is probably in exporting the mesh.

why the torso is affected perhaps by automatic skin binding the hand joints have some effect for torso (which is should be zero)

This is strange, because I'm using the Sinbad model and it works fine with the Ogre-OpenNI example. So it is not a model problem, and I don't know what in the code could be causing this influence...

[brunobg]

Opening in ogremeshy showed the problem: the skeleton is translated in relation to the mesh! This is certainly being caused by the ogre exporter. Any tips of how to solve it?

EDIT: part of the problem seems related to exporting the model, so I moved the question to the blender ogre exporter thread: http://www.ogre3d.org/forums/viewtopic. ... 40#p456640

[brunobg]

I seem to have solved the problems with model exportation, but I'm still facing problems with the animation. It seems I'm almost there. Here's what happens:

Apparently the bones which are not being controlled directly are not moving. I'm setting only these bones to manual mode and to not inherit orientation. Any tips?

[brunobg]

I have ported my changes to TuxSinbad, sample code is at https://github.com/brunobg/TuxSinbad

Still buggy, if anybody has suggestions...