IK with Ogre

[noche]

Hello,

I did some character inverse knematic with a personal project and since I didn't have a real animation system (loading / playing / blending animations) I'd like to redo this project using Ogre.

My goal is to create a rigged character in Ogre as an artist is able to rig a character in 3dsmax or any other DCC application.

In 3dsmax and in my project I do this by affecting controllers to the skeleton bones :
- IK limb solver (for arms and legs)
- Orientation constraint controller (for hands and feets)
- LookAt controller (head, torso)
- hierarchy
- ...

So I was wondering what architecture I should use to create this in Ogre.
Would you suggest to use the Ogre controllers or something else ?

[noche]

Since I'd like to blend the IK with an existing animation, maybe the best way would be to wrap the IK in an AnimationState.
It would make a procedural animationState that I could blend for free with standard animation.

I'm going to try that.

[xavier]

That's actually an interesting approach -- I hadn't thought of that.

IK was supposed to be part of a GSoC project this summer but it didn't end up working out -- this sounds like an interesting way of doing it though. Keep us posted on progress and issues if you could.

[noche]

I'm a bit new to Ogre, so don't hesitate to tell me if I miss something.
Here is how I started :

I've seen Ogre have 3 main kind of animations :
- NumericAnimationTrack
- NodeAnimationTrack
- VertexAnimationTrack

Those tracks can be created from an Animation instance, itself created from a Skeleton instance. I find a little anoying that the Ogre::Animation must create itself the Ogre::AnimationsTracks because this mean that I cannot add a new kind of animation track. A method Animation::addAnimationTrack(AnimationTrack*) would be nice since anyone could create his own AnimationTrack.

So to do my tests I have to override Skeleton Animation and AnimationTrack :

Code: Select all

//-----------------------------------------------------------------------------
class ProceduralAnimationTrack : public AnimationTrack
{
public:
    ProceduralAnimationTrack(Animation* parent, unsigned short handle, Node* target);

    void        apply(const TimeIndex& timeIndex, Real weight = 1.0, Real scale = 1.0f);
    void        getInterpolatedKeyFrame(const TimeIndex& timeIndex, KeyFrame* kf) const;
    KeyFrame*   createKeyFrameImpl(Real time);
private:
    Node* node;
};

//-----------------------------------------------------------------------------
class ProceduralAnimation : public Ogre::Animation
{
public:
    ProceduralAnimation(const String& name);

    void                        apply(Real timePos, Real weight, Real scale);
    ProceduralAnimationTrack*   createProceduralTrack(unsigned short handle, Node* target);
    bool                        hasProceduralTrack(unsigned short handle) const;
private:
    typedef std::map<unsigned short, ProceduralAnimationTrack*> ProceduralTrackList;
    typedef ConstMapIterator<ProceduralTrackList> ProcTrackIterator;
    ProceduralTrackList mProceduralTrackList;
};

//-----------------------------------------------------------------------------
class ProceduralSkeleton : public Ogre::Skeleton
{
public:
    ProceduralSkeleton(Skeleton* skeleton);
    ProceduralAnimation* createProceduralAnimation(const String& name);
private:
    Skeleton* _skeleton;
};

This is inside this method that all the magic should happend : ProceduralAnimationTrack::getInterpolatedKeyFrame.
A keyframe should be calculated according the IK contraints (an some other kind of contraints too). This way, the procedural animation could be blended with other regular animations. For now, I tried to create a key inside this method with some constant rotation to see if it works :

Code: Select all

void ProceduralAnimationTrack::getInterpolatedKeyFrame(const TimeIndex& timeIndex, KeyFrame* kf) const
{
    TransformKeyFrame* kret = static_cast<TransformKeyFrame*>(kf);
    kret->setRotation(Quaternion(Degree(50), Vector3(1, 0, 0)));
    kret->setTranslate(Vector3::ZERO);
    kret->setScale(Vector3(1, 1, 1));
}

To test it, I tryed to update the animation by doing this :

Code: Select all

Bone* _boneLeftHand;
Skeleton* skeleton;
ProceduralSkeleton* _proceduralSkeleton;
ProceduralAnimation* _proceduralAnimation;
ProceduralAnimationTrack* _trackLeftHand

init : 
_proceduralSkeleton = new ProceduralSkeleton(_skeleton);
_proceduralAnimation = _proceduralSkeleton->createProceduralAnimation("BipedAnimation");
_trackLeftHand= _proceduralAnimation->createProceduralTrack(0, _boneLeftHand);

update:
 _proceduralAnimation->apply(0.0f, 1.0f, 1.0f);

But I dont see any visual changes on my Entity. The bone is not moving. If you have any idea, of how to update the entity ? (I checked I pass in ProceduralAnimationTrack::getInterpolatedKeyFrame and ProceduralAnimationTrack::apply)

The apply method is copy pasted from NodeAnimationTrack::apply :

Code: Select all

void ProceduralAnimationTrack::apply(const TimeIndex& timeIndex, Real weight, Real scale)
{
    TransformKeyFrame kf(0, timeIndex.getTimePos());
    getInterpolatedKeyFrame(timeIndex, &kf);

    // add to existing. Weights are not relative, but treated as absolute multipliers for the animation
    Vector3 translate = kf.getTranslate() * weight * scale;
    node->translate(translate);

    // interpolate between no-rotation and full rotation, to point 'weight', so 0 = no rotate, 1 = full
    Quaternion rotate;
    Animation::RotationInterpolationMode rim =
        mParent->getRotationInterpolationMode();
    if (rim == Animation::RIM_LINEAR)
    {
        rotate = Quaternion::nlerp(weight, Quaternion::IDENTITY, kf.getRotation(), true);
    }
    else //if (rim == Animation::RIM_SPHERICAL)
    {
        rotate = Quaternion::Slerp(weight, Quaternion::IDENTITY, kf.getRotation(), true);
    }
    node->rotate(rotate);

    Vector3 scl = kf.getScale();
    // Not sure how to modify scale for cumulative anims... leave it alone
    //scale = ((Vector3::UNIT_SCALE - kf.getScale()) * weight) + Vector3::UNIT_SCALE;
    if (scale != 1.0f && scl != Vector3::UNIT_SCALE)
    {
        scl = Vector3::UNIT_SCALE + (scl - Vector3::UNIT_SCALE) * scale;
    }
    node->scale(scl);
}

Here is the rest of the cpp code...

[xavier]

Animations are blended using AnimationState -- you get them from the Entity. Bones either are controlled by an animation track, or they aren't -- I don't really know if what you are doing there will work or not, but if you have named animations on an entity that can be found with getAnimationState() then you can blend them together.

[noche]

Animations are blended using AnimationState -- you get them from the Entity. Bones either are controlled by an animation track, or they aren't -- I don't really know if what you are doing there will work or not, but if you have named animations on an entity that can be found with getAnimationState() then you can blend them together.

Do you know if there is a way to add a custom animation type or should I modify ogre source code ?

[noche]

Finaly I modified Ogre source code. I added a procedual animation track that inherit from NodeAnimationTrack :

Code: Select all

    
class _OgreExport ProceduralAnimationTrack : public NodeAnimationTrack
{
public:
	ProceduralAnimationTrack(Animation* parent, unsigned short handle, Node* targetNode, ProceduralAnimationTrackController* controller);

	virtual ~ProceduralAnimationTrack();

	virtual void apply(const TimeIndex& timeIndex, Real weight = 1.0, Real scale = 1.0f);

	virtual void applyToNode(Node* node, const TimeIndex& timeIndex, Real weight = 1.0, Real scale = 1.0f);

	virtual void getInterpolatedKeyFrame(const TimeIndex& timeIndex, KeyFrame* kf) const;

protected:
	KeyFrame* createKeyFrameImpl(Real time);

	ProceduralAnimationTrackController* _controller;
}; 

It has a controller. The controller will be responsible for calculating the keyframe. It must implement this interface :

Code: Select all

    class _OgreExport ProceduralAnimationTrackController 
	{
	public:
		virtual void getInterpolatedKeyFrame(const TimeIndex& timeIndex, KeyFrame* kf) const = 0;
	};

I added the creation of this kind of animation track in the Animation class.
The track is added in the mNodeTrackList, since a ProceduralAnimationTrack is a NodeAnimationTrack :

Code: Select all

    ProceduralAnimationTrack* Animation::createProceduralTrack(unsigned short handle, Node* node, ProceduralAnimationTrackController* controller)
    {
        if (hasNodeTrack(handle))
        {
            OGRE_EXCEPT(Exception::ERR_DUPLICATE_ITEM, 
                "Node track with the specified handle " +
                StringConverter::toString(handle) + " already exists",
                "Animation::createProceduralAnimationTrack");
        }
        ProceduralAnimationTrack* ret = new ProceduralAnimationTrack(this, handle, node, controller);
        mNodeTrackList[handle] = ret;
        ret->setAssociatedNode(node);
        return ret;
    }

Here is a part of the applyToNode code. It's the same as NodeAnimationTrack but it doesn't check if a key exist, and it asks the controller to set the key.

Code: Select all

    void ProceduralAnimationTrack::applyToNode(Node* node, const TimeIndex& timeIndex, Real weight, Real scl)
    {
		// if ( mKeyFrames.empty() ||  !weight || !node)

		if (!weight || !node)
			return;

		TransformKeyFrame kf(0, timeIndex.getTimePos());
		_controller->getInterpolatedKeyFrame(timeIndex, kf);
   ....
    }

Finaly here is a test of a ProceduralAnimationTrackController that rotate and translate a bone :

Code: Select all

class MyController : public ProceduralAnimationTrackController
{
     void getInterpolatedKeyFrame(const TimeIndex& timeIndex, KeyFrame* kf) const
     {
        static float angle = 0;
        angle += 0.1f;
        TransformKeyFrame* kret = static_cast<TransformKeyFrame*>(kf);
        kret->setRotation(Quaternion(Degree(angle), Vector3(1, 0, 0)));
        kret->setTranslate(Vector3(Math::Sin(angle), 0, 0));
        kret->setScale(Vector3(1, 1, 1));
     }
} 

Also the code that create the procedural animation :

Code: Select all

_procAnimation = _skeleton->createAnimation("BipedAnimation", 1.0f);
_leftArmTrack = _procAnimation->createProceduralTrack(leftArm->getHandle(),                                                                 leftArm, &armController); 
_entity->refreshAvailableAnimationState();

AnimationState* state = _entity->getAnimationState("BipedAnimation");
state->setEnabled(true);
state->setWeight(0.5f);

Now its the time to do some IK

[Falagard]

Sweet, can't wait to see what you come up with!

I'm actually thinking of using the animation system to blend between ragdoll physics and animations. For example, let's say a character can be shot where he temporarily falls down using ragdoll, then stands back up using an animation and returns to normal.

It'd also be interesting to see if it's possible to mix them together, so while a character is falling he's waving his arms, etc. though I believe that would require something different (attempt to apply forces to the ragdoll physics based on an animation).

[noche]

I wonder if there is a way to specify in Ogre that an animation state is played in an additive mode with another animation ? It seems weights are only used to interpolate (average) between different animations.

It would be nice to have the choice of the blending type. This way a walking animation could be used for the lower body, and a shooting animation or a procedural animation for the upper body.

[noche]

Sweet, can't wait to see what you come up with!

Here is the result for now, a walking animation and a controller doing only a rotation on the arm :
video

I still wonder why the arm get flipped.

[noche]

The reason why the arm got flipped is because it was using the shortest path to interpolate the rotation between the 2 animations.

_leftArmTrack->setUseShortestRotationPath(false);

this resolve the issue.

[PolyVox]

Cool, looks like an interesting project! Make some screenshots and post in the showcase forum when you're ready - I'm sure you'll get a lot of interest!

[Falagard]

I wonder if there is a way to specify in Ogre that an animation state is played in an additive mode with another animation ? It seems weights are only used to interpolate (average) between different animations.

There's a flag that does this, I believe, when blending animations. Or at least there's a flag that definitely does something

Nice video by the way

[noche]

There's a flag that does this, I believe, when blending animations. Or at least there's a flag that definitely does something

I didn't find the one that does something about additive animations. There is the Animation::setInterpolationMode
that define how it should interpolate (linear or spline) but its still about blending (averaging) N animations.

I may have to add an Additive mode in ogre, otherwise my work will be almost useless.

[xavier]

There is ANIMBLEND_CUMULATIVE and ANIMBLEND_AVERAGE -- it's on the Skeleton class though.

Note the Eihort notes on the difference:

ANIMBLEND_AVERAGE now only rebalances animation weights when the cumulative weight exceeds 1.0f. Therefore the only difference between ANIMBLEND_CUMULATIVE and ANIMBLEND_AVERAGE is now when you have total animation weights above this value, when all weights will be renormalised to sum to 1.0f. This means you can have weightings under 1.0f in average blending mode which is more convenient.

[Falagard]

How about this?

enum SkeletonAnimationBlendMode {
/// Animations are applied by calculating a weighted average of all animations
ANIMBLEND_AVERAGE,
/// Animations are applied by calculating a weighted cumulative total
ANIMBLEND_CUMULATIVE
};

Oops, late by 1 minute.

[noche]

I missed this one Thx a lot guys !

I also wonder what would be the way to get the inital world position and orientation of a bone, at any time. Right now I must save the position and orientation before the skeleton gets animated :

_initalWorldPosition = bone->getWorldPosition();

Is there a way to get it while the animation is played ? I think Bone::getInitialPosition returns local coordinates, but I would need them in world space.

[Falagard]

I missed this one Thx a lot guys !

I also wonder what would be the way to get the inital world position and orientation of a bone, at any time. Right now I must save the position and orientation before the skeleton gets animated :

_initalWorldPosition = bone->getWorldPosition();

Is there a way to get it while the animation is played ? I think Bone::getInitialPosition returns local coordinates, but I would need them in world space.

Convert them to world space then. Take a look at Node's code for converting local to world space. Might have been SceneNode. Anyhow, one of them does it to keep the world position and orientation cached variables up to date.

[noche]

Convert them to world space then.

Yes I know I could convert them to world space, but this sound overkill because this means to runs all the hierarchy from my bone to the root, to multiply all the transformations. And since the result I want is constant, I was hoping there was some cache in ogre that store the initial world transformations of the skeleton.

Thanks for your answer.

[noche]

Here is a little video of my results for now : video

The skeleton is using a cumulative blend mode.

First there is only an IK animation played.
Second there is the IK animation and an animation exported from 3dsmax that has only rotation keyframes on the arm.
Third there is the IK animation and another animation exported from 3dsmax that move the spine.

The result is still not really good foor the 3rd test, I guess I dont use the right world matrices. The nice thing is to be able to play with animation state weights to use more or less the IK animation.

[xavier]

I really like that you've taken this on -- it's a different direction than what I had planned to do but it's far simpler, less intrusive and a more elegant solution.

If you need help analyzing the problem, go ahead and post again some of the code in question.

I hope in the end that you plan to submit a patch to Ogre for this -- it really does solve a whole set of problems that the current Ogre animation system cannot address.

[Falagard]

Awesome!

So, for each bone you have to set up joint constraints - only rotate along a certain axis within certain angles, etc.

Very very nice. Are you planning on sharing the code when you're finished?

I assume this could be used for walking on uneven ground so the feet touch the ground at the proper place, as well as for things like turning the head in a direction of interest, such as something you can pick up, or turning the whole torso for something like.. aiming a gun? Though I assume for it to look right in most cases it'd be a combination of animation blending between two animations, as well as the IK blending.

Anyhow, very impressive!

[KungFooMasta]

Awesome!

[Kencho]

Can't see the video... What codec have you used?

[xavier]

Guessing Fraps -- install the free Fraps and try again?