Generic Pose Loader

[brentrossen]

I'm working on a generalized Mesh pose loader, but I'm encountering some difficulties.

Motivation: I'm teaching a college course on interactive animation next semester (hopefully using Ogre), my student's will have little programming knowledge, so I need to be able to black box things like pose loading for them. Also, I think a generalized pose loader could be useful for everyone.

My two options:
1) Load all poses ahead of time for each submesh
2) Load Poses just in time

The problem with option 1:
I need the list of poses to load. It seems that when I try to get the list using

Code: Select all

Mesh::PoseIterator poseIter = mesh->getPoseIterator(); //or getPoseList()

after:

Code: Select all

mesh = MeshManager::getSingleton().load(headFile, ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

but still in the same function, the pose list turns up empty.

The problem with option 2:
If I don't call

Code: Select all

Ogre::VertexAnimationTrack *vt = poseAnim->createVertexTrack(curSubMesh, VAT_POSE);
			vpKeyFrame = vt->createVertexPoseKeyFrame(0);

within the same function as the mesh load I get the 'not vertex animated or has no dedicated submesh' error. I've loaded and used my test mesh manually by specifying which poses to load on each submesh, so I'm fairly certain it's not the .mesh itself.

Here is the code I've been trying for just in time loading:

Code: Select all

/** Goes through all the submeshes and all the poses finding and updating poses with this name 
*If they aren't yet loaded, also loads all poses as the list is traversed*/
void MeshPoseManager::updatePose(std::string poseName, float weight){	
	//get the pose list
	Mesh::PoseIterator poseIter = mesh->getPoseIterator();
	int curPoseIndex = 0;

	//loop through each submesh
	int numSubMeshes = mesh->getNumSubMeshes();
	//skip submesh 0 since it is the shared geometry, and we have no poses on that
	for(int curSubMesh = 1; curSubMesh <= numSubMeshes; curSubMesh++){
		//if we haven't called updatePose before, create a pose reference for each submesh
		if(!posesInitialized){
			Ogre::VertexAnimationTrack *vt = poseAnim->createVertexTrack(curSubMesh, VAT_POSE);
			vpKeyFrame = vt->createVertexPoseKeyFrame(0);
		}
	
		//while the poses apply to the current submesh, check to see if the current pose matches the name, then get the next pose
		while(poseIter.peekNext()->getTarget() == curSubMesh-1){
			//if we haven't called updatePose before, create a pose reference for each pose
			if(!posesInitialized){
				vpKeyFrame->addPoseReference(curPoseIndex, 0.0f);
			}
	
			//get next pose and check if it's the one we want to update
			if(!poseIter.getNext()->getName().compare(poseName)){
				//We found our pose, update it
				Ogre::VertexAnimationTrack *vt = poseAnim->getVertexTrack(curSubMesh);
				Ogre::VertexPoseKeyFrame *keyFrame = vt->getVertexPoseKeyFrame(0);
				keyFrame->updatePoseReference(curPoseIndex, weight);
			}
			curPoseIndex++;
			//go to the next pose
		}

		//go to the next submesh
	}
	
	//enable poses animation state if it is not yet
	if(!posesInitialized){
		manualAnimState = head->getAnimationState("posesAnimState");
		manualAnimState->setTimePosition(0);
		manualAnimState->setEnabled(true);
	}

	posesInitialized = true; //we have initialized our poses
}

My questions:

1) When does the pose list get populated? What is it waiting for? Is the

Code: Select all

mesh = MeshManager::getSingleton().load(headFile, ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

shuffled off to a separate thread? Is that why the pose list isn't filled yet?
2) I would prefer to load poses just in time. Is it possible to load poses after the mesh has finished loading and is attached to its scene node? Or has the hardware buffering been completed and can't be altered?

My guesses are just shots in the dark, if someone could illuminate the situation or give me another avenue for making a generic pose loader I would very much appreciate it.

[sinbad]

1) The pose list is completed as part of mesh loading - see MeshSerializerImpl::readPose which calls Mesh::createPose, which fills the internal mPoseList which is what the iterator is based on. I can only assume that your .mesh file doesn't have any poses defined in it.

2) You can add more poses after a mesh is loaded, yes - again Mesh::createPose is the method.

I guess I'm a little confused as to what you need a generic pose loader for anyway. Do you mean creating new poses, or just creating a track which references exising poses so that you can do something similar to Demo_FacialAnimation?

[brentrossen]

1) I know the mesh has poses in it, so I'll give getting the list immediately after load a try again. Perhaps something else was wrong and I interpreted it as an empty list. Though that won't help for loading just in time.

2) I misused terminology in this question, I didn't mean load poses, the poses should be loaded automatically from the mesh. I meant: Can you always add vertex tracks, keyframes, and pose references? I could create each of the vertex tracks for each submesh right away, then add pose references just in time. Also, does each submesh need a separate animation for it's tracks? Right now I'm trying to put all vertex animation tracks onto the same animation, when I did it manually by pose index I put each vertex track on a separate animation for each submesh.

------------------------------------------------------------------
The reason I want a generic pose loader:

It will first be used in an academic research study on virtual humans in medicine, it may then be used as part of a course next semester.

The Meshes
I have a face that has 6 submeshes and 32 poses (visemes, emotions, races, and genders)
I have a body with 1 submesh and 1 pose (breathing)
Both are skeletally animated

There may be more than one of each of these meshes in the scene at a given time.

When I add a blend shape to the face using Maya, it doesn't necessarily apply to all of the submeshes. So I need to decompress the .mesh to .xml and figure out the pose indexes all over again each time. Since there are so many poses and my facial mesh is 10k faces, the .mesh file is 62mb, when decompressed its around 250mb. It's difficult edit to say the least.

I've written a system that takes audio and makes the facial mesh speak by calling the appropriate viseme pose, and ramping it up and down so it looks like the character speaks naturally. It works correctly once I have all of the poses mapped to the appropriate index.

I don't have a prewritten script, and I have hundreds of audio files that this character can say, so it has to be very general, I can't use canned morph animation. I would much rather be able to call the poses by name than create my own maps from pose index to pose names. Once this is done, I can create any face with the appropriately named visemes and have it say anything.

[sinbad]

2) Ok, that makes more sense. Yes, you can always add more tracks & keyframes etc. And yes, you can add multiple tracks to one animation each referencing a different submesh.

The project sounds really cool

[brentrossen]

Thank you, I hope to post some screenshots soon. We'll be conducting a study within the next two months, so I'm sure we'll have some interesting stuff by then. We hook up our participants with a tracked hat and project the system onto a wall so our characters are lifesize.

In case it's useful for anyone, here are the pertinent parts of the general pose loader solution I came up with:

Required Variables:

Code: Select all

Ogre::Entity* head;
Ogre::MeshPtr mesh;
const char * poseAnimationStateName = "PoseAnimationState";
map<int, Ogre::VertexPoseKeyFrame*> vertexPoseKeyFrameMap;

Loading Code:

Code: Select all

void MeshPoseManager::loadHead(std::string meshFile){
	try{
		// Pre-load the mesh so that we can tweak it with a manual animation
		mesh = MeshManager::getSingleton().load(meshFile, ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

		//we'll read through this list while loading
        PoseList poseList = mesh->getPoseList();

		//used to loop through each submesh
		int numSubMeshes = mesh->getNumSubMeshes();
		int curPoseIndex = 0;
	
		//create the animation on the mesh
		Animation* anim = mesh->createAnimation(poseAnimationStateName, 0);

		//skip submesh 0 since it is the shared geometry, and we have no poses on that
		for(int curSubMesh = 1; curSubMesh <= numSubMeshes; curSubMesh++){		
			//create the VertexTrack on this animation
			Ogre::VertexAnimationTrack *vt = anim->createVertexTrack(curSubMesh, VAT_POSE);
			//create the keyframe we will use to update this vertex track
			//keep all our keyframes in a map for later updating
			vertexPoseKeyFrameMap[curSubMesh] = vt->createVertexPoseKeyFrame(0);
			//add the references to each pose that applies to this subMesh
			while(poseList[curPoseIndex]->getTarget() == curSubMesh-1){
				//create a pose reference for each pose
				vertexPoseKeyFrameMap[curSubMesh]->addPoseReference(curPoseIndex, 0.0f);
				curPoseIndex++;
			}
		}

		//create the head
		head = mSceneMgr->createEntity("Head", meshFile);
		head->getAnimationState(poseAnimationStateName)->setTimePosition(0);
		head->getAnimationState(poseAnimationStateName)->setEnabled(true);

		//put entity in scene
        SceneNode* headNode = mSceneMgr->getRootSceneNode()->createChildSceneNode();
        headNode->attachObject(head);
	}
	catch(runtime_error err){
		std::string error = "ERROR: ";
		error.append(__FILE__);
		error.append(err.what());
		LogManager::getSingleton().logMessage(error);
	}
}

Update Code:

Code: Select all

/** Goes through all the submeshes and all the poses finding and updating poses with this name*/
void MeshPoseManager::updatePose(std::string poseName, float weight){	
	//get the pose list
	PoseList poseList = mesh->getPoseList();
	int curPoseIndex = 0;

	//loop through each submesh
	int numSubMeshes = mesh->getNumSubMeshes();
	//skip submesh 0 since it is the shared geometry, and we have no poses on that
	for(int curSubMesh = 1; curSubMesh <= numSubMeshes; curSubMesh++){
		//while the poses apply to the current submesh, check to see if the current pose matches the name, then get the next pose
		while(poseList[curPoseIndex]->getTarget() == curSubMesh-1){
			//get next pose and check if it's the one we want to update
			if(!poseList[curPoseIndex]->getName().compare(poseName)){
				//We found our pose, update it
				vertexPoseKeyFrameMap[curSubMesh]->updatePoseReference(curPoseIndex, weight);
				// Dirty animation state since we're fudging this manually
				//If we don't notify dirty, Ogre doesn't know to update this state 
				//and nothing happens
				head->getAnimationState(poseAnimationStateName)->getParent()->_notifyDirty();				
			}
			curPoseIndex++;
			//go to the next pose
		}
		//go to the next submesh
	}	
}

As you can see, the mesh I was loading was a head, but the solution should generalize to any mesh with poses on its submeshes. Please post here if you have any troubles with the code above.

[tgraupmann]

Thank you, I hope to post some screenshots soon. We'll be conducting a study within the next two months, so I'm sure we'll have some interesting stuff by then. We hook up our participants with a tracked hat and project the system onto a wall so our characters are lifesize.

How did this project turn out? Do you have any screenshots?

[Yanko]

hey there, maybe i should link my question in this topic too

if anyone has some experience with this piece of code, please take a look at this

http://www.ogre3d.org/phpBB2/viewtopic.php?t=33639

thx!