root motion Topic is solved

[slapin]

Hi, all!
I want to implement root motion for my character. For that I need to extract root bone transform delta at each frame and set that to identity to clear motion. Any ideas about where to look?

[sercero]

I'm not sure my solution is the best, but I did this:

Code: Select all

    // Cambiar los desplazamientos absolutos del root bone por deltas asi puedo llamara a node->translate() directamente.
    // El esqueleto contiene los tracks de las animaciones de los hueso (cada hueso es un track)
    // La clase hueso hereda de Node y por lo tanto las animaciones de los huesos son del tipo nodo
    // Los KeyFrames tambien son KeyFrames especializados en nodos, no la clase KeyFrame base.
    for(int i = 0; i < State::NUM_STATES - 1; i++) {

    // Si no es una animacion estatica ignorar
    if(mStates[i]->mLoop)
        continue;

    //Ogre::Animation::NodeTrackIterator it = baseSkeleton->getAnimation(animacionesEstaticas[i])->getNodeTrackIterator();
    Ogre::Animation::NodeTrackIterator it = baseSkeleton->getAnimation(mStates[i]->mAnimationState->getAnimationName())->getNodeTrackIterator();

    while (it.hasMoreElements()) {
        Ogre::NodeAnimationTrack* track = it.getNext();
        if(track->getAssociatedNode()->getName() == "root") {
            //std::cout << "Track: " << baseSkeleton->getAnimation(animacionesEstaticas[i])->getName() << std::endl;
            Ogre::Vector3 delta = Ogre::Vector3::ZERO;
            for(int j = 0; j < track->getNumKeyFrames(); j++) {
                Ogre::Vector3 trans = track->getNodeKeyFrame(j)->getTranslate();
                //std::cout << "KF (" << j << ") -> " << track->getNodeKeyFrame(j)->getRotation() << std::endl;
                track->getNodeKeyFrame(j)->setTranslate(trans - delta);
                delta = trans;
            }
        }
    }
}

[slapin]

Thanks a lot!

[slapin]

Thanks a lot for the example, I see it creates track full of deltas; but how do you apply this track to the body?
I need to understand this part too...

[slapin]

I implemented this using Ogre master branch

Code: Select all

       for (i = 0; i < NUM_ANIMS; i++) {
            mAnims[i] = mBodyEnt->getAnimationState(animNames[i]);
            mAnims[i]->setLoop(true);
            mAnims[i]->setEnabled(true);
            mAnims[i]->setWeight(0);
            mFadingIn[i] = false;
            mFadingOut[i] = false;
            mSkelAnimations[i] = mSkeleton->getAnimation(animNames[i]);
            for (const auto& it : mSkelAnimations[i]->_getNodeTrackList()) {
                       Ogre::NodeAnimationTrack* track = it.second;
                        Ogre::String trackName = track->getAssociatedNode()->getName();
                        if (trackName == "mixamorig:Hips") {
                                mHipsTracks[i] = track;
                        } else if (trackName == "Root") {
                                mRootTracks[i] = track;
                        }
            }
            Ogre::Vector3 delta = Ogre::Vector3::ZERO;
            Ogre::Vector3 motion = Ogre::Vector3::ZERO;
            for(j = 0; j < mRootTracks[i]->getNumKeyFrames(); j++) {
                        Ogre::Vector3 trans = mRootTracks[i]->getNodeKeyFrame(j)->getTranslate();
                        trans.y = 0.0f;
                        delta = trans - motion;
                        if (delta.z > 0.03f)
                                delta.z = 0.03f;
                        if (delta.z < 0.02f)
                                delta.z = 0.03f;
                        mRootTracks[i]->getNodeKeyFrame(j)->setTranslate(delta);
                        motion = trans;
            }
        }

Now I somehow need to transform "Root" track motion into btRigidBody motion. Should I just reference "Root" node and use its position each frame after animations to move btRigidBody or is there some better way?

[sercero]

Hello, this is the code to move the player:
(please take into account that this is probably not the best way to do this, I'm a lousy coder)
(another thing: I'm using something called "AnimationBlender" you can find that in the forum)

Code: Select all

//------------------------------------------------------------------------------------
// Metodo que actualiza la posicion y orientacion del jugador segun el tipo de camara
// -----------------------------------------------------------------------------------
void Player::updateTransform(btVector3 &ghostPositionGoal, float timeSinceLastFrame)
{
    Ogre::Vector3 src = mPlayerNode->getOrientation().zAxis(); // * Ogre::Vector3::UNIT_Z;
    Ogre::Quaternion quat = Ogre::Quaternion::IDENTITY;

if(mCameraManager->getStyle() == CameraMan3::CS_LOCK) {
    // En el caso de Strafe, si se presiona W+S / W+A / D+S / D+A entonces hay que rotar un poco mas
    Ogre::Radian rotAngle = Ogre::Radian(0);

    if(mInputDirection.x > 0 && mInputDirection.y > 0)
        rotAngle = Ogre::Radian(Ogre::Math::PI / 4);

    if(mInputDirection.x < 0 && mInputDirection.y > 0)
        rotAngle = -1 * Ogre::Radian(Ogre::Math::PI / 4);

    if(mInputDirection.x < 0 && mInputDirection.y > 0)
        rotAngle = -1 * Ogre::Radian(Ogre::Math::PI / 4);

    if(mInputDirection.x < 0 && mInputDirection.y < 0)
        rotAngle = Ogre::Radian(Ogre::Math::PI / 4);

    quat = src.getRotationTo(mCameraManager->getFacing()) * Ogre::Quaternion(rotAngle, Ogre::Vector3::UNIT_Y);

    // Ademas corregir la direccion de la cabeza
    /*
    mAnimationState[mStateCurrent]->setBlendMaskEntry(mNeckBone->getHandle(), 0);
    mNeckBone->setManuallyControlled(true);

    Ogre::Radian delta = quat.getYaw() - mNeckBone->getOrientation().getYaw() + rotAngle;

    // Ambos tienen que ser iguales sino hace zig zag en el ajuste
    Ogre::Radian adjust = Ogre::Radian(1.0 * timeSinceLastFrame / 1000);

    if((Ogre::Math::Abs(delta) > 2 * adjust) && (Ogre::Math::Abs(mNeckBone->getOrientation().getYaw()) < Ogre::Radian(0.4 * Ogre::Math::PI))) {
        mNeckBone->yaw(adjust * Ogre::Math::Sign(delta));
    }
    */

    //mAnimationState[mStateCurrent]->setBlendMaskEntry(mNeckBone->getHandle(), 0);
    //mNeckBone->setManuallyControlled(true);
    //mNeckBone->setOrientation(mCameraManager->getFacing() * Ogre::Quaternion(rotAngle, Ogre::Vector3::UNIT_Y));
    //mNeckBone->yaw();
}

if(mCameraManager->getStyle() == CameraMan3::CS_ORBIT) {
    quat = src.getRotationTo(mTranslateVector);	// Get a quaternion rotation operation
}

Ogre::Quaternion delta = Ogre::Quaternion::nlerp(timeSinceLastFrame / 1000 * TURN_SPEED, mPlayerNode->getOrientation(), quat * mPlayerNode->getOrientation(), true);
mPlayerNode->setOrientation(delta);

/*
std::cout << "quat = " << quat << std::endl;
if(quat.y * quat.y > .9) {
    mStateNext = State::TURNAROUND;
    mStateTimeout.init(mAnimationState[State::TURNAROUND]->getLength() / TIMESCALE);
    mStateTimeout.start();
}
*/

// Traslada el nodo del jugador teniendo en cuenta el blending actual
if(quat.w > 0.9) {
    //mPlayerNode->translate(mTranslateVector * timeSinceLastFrame * TIMESCALE * (1 - mAnimationBlender->getProgress()));
    Ogre::Vector3 nodeTranslation = mTranslateVector * timeSinceLastFrame * TIMESCALE * (1 - mAnimationBlender->getProgress());
    ghostPositionGoal += btVector3(nodeTranslation.x, nodeTranslation.y, nodeTranslation.z);
}
}

[slapin]

Thanks a lot for the code!

Reading the code I don't quite understand how the offset is calculated. I see how the rotation is handled, but not the motion.
Can't get where mTranslateVector goes from and what is ghostPositionGoal.
Will look at AnimationBlender.

[slapin]

After some time struggling I managed to make the whole thing work.
First I modified animation preparation code to this:

Code: Select all

       for (i = 0; i < NUM_ANIMS; i++) {
            mAnims[i] = mBodyEnt->getAnimationState(animNames[i]);
            mAnims[i]->setLoop(true);
            mAnims[i]->setEnabled(true);
            mAnims[i]->setWeight(0);
            mFadingIn[i] = false;
            mFadingOut[i] = false;
            mSkelAnimations[i] = mSkeleton->getAnimation(animNames[i]);
            for (const auto& it : mSkelAnimations[i]->_getNodeTrackList()) {
                        Ogre::NodeAnimationTrack* track = it.second;
                        Ogre::String trackName = track->getAssociatedNode()->getName();
                        if (trackName == "mixamorig:Hips") {
                                mHipsTracks[i] = track;
                        } else if (trackName == "Root") {
                                mRootTracks[i] = track;
                        }
            }
            Ogre::Vector3 delta = Ogre::Vector3::ZERO;
            Ogre::Vector3 motion = Ogre::Vector3::ZERO;
            for(j = 0; j < mRootTracks[i]->getNumKeyFrames(); j++) {
                        Ogre::Vector3 trans = mRootTracks[i]->getNodeKeyFrame(j)->getTranslate();
                        if (j == 0)
                                delta = trans;
                        else
                                delta = trans - motion;
                        mRootTracks[i]->getNodeKeyFrame(j)->setTranslate(delta);
                        motion = trans;
            }
        }

Then the actual motion code I do after animations played for this frame.

Code: Select all

void CharacterController::updateRootMotion(Real delta)
{
        Ogre::Vector3 boneMotion = mRootBone->getPosition();
        /* Kinematic motion */
        Ogre::Quaternion rot = mBodyNode->getOrientation();
//        Ogre::Vector3 gravity(0, -9.8, 0);
        Ogre::Vector3 rotMotion = rot * boneMotion;
//        mBodyNode->setPosition(mBodyNode->getPosition() + rotMotion + gravity);
        mBodyNode->setPosition(mBodyNode->getPosition() + rotMotion);
}

mBodyNode is top SceneNode of character where both physics and entity are.

What is missing is node rotation update from root bone rotation and kinematic collisions (another topic).
After collisions implementation I will be able to add gravity and implement jumping...

[slapin]

Actually looped animations when prepared as in code above are fine too and work very well.

[paroj]

is there something that benefits both of you and we should put directly into Ogre, so everybody can share the same implementation?

[sercero]

@paroj one thing that would be cool is for OGRE to have some simplification in terms of animations is an animation blending system.
I was looking at this: https://www.ogre3d.org/forums/viewtopic.php?t=80704, but never got to implement it.
There is also the famous technofreak animation system which is pretty elaborate.
All of this integrated with a phyisics character controller would go a long way towards simplifying things for people who want to implement a game engine.
Although it would drift away from OGREs purpose which is to be a Graphics engine.

[slapin]

Well, my solution heavily depends on how character model is oriented.
The animation patch-up is universal thing and is much better than playing with manual bone and applying animation tracks to it.
Probably that part can be shared, but actual use of the transform might depend on how the model is made and other factors.
I.e. for Mixamo rig with not split hips the root bone will be hips but you will have to process the transform to filter-out high frequency motions
which will complicate things. I do that using Blender python script for exporting separate Root and Hips bones.
To make this more flexible I guess the animation system could be extended for easier keyframe processing and filtering,
but I have no idea how.
If animation system supported bones postprocessing that could just apply changes after physics and animation processing
so there would be no need for manual bones and track application so one could just apply bone pose after all animations, that would solve both this problem and would allow for activge ragdall and IK easily implemented. Otherwise the thing is doable even today but a bit complicated and requires direct engine guts access.

[slapin]

I think it is at least not in a way of Ogre to be a graphics engine as making animations easier to use adds to benefits of Ogre as whole.

[paroj]

Well, my solution heavily depends on how character model is oriented.

we could start by adding the code that works for you/ the mixamo format. Then the next guy can start there if more is needed.

[slapin]

Looks like my approach to root motion I did per @sercero advice doesn't work too well.
It works only when each frame we have new animation frame. However when we go with unlocked FPS, there is a lot
of repeat frames which leads to lots of additive errors... (3000 FPS!!!11)

So one have to check current animation position for changes or just avoid animation patching and use full transform
calculating previous/current delta. Need to think about it...

[slapin]

It looks like best approach for root motion is via Ogre::AnimationTrack::Listener. I accidentally stumbled upon it and it can update currently playing keyframe regardless of animation blend and all. This eleminates the need to mangle animation tracks and have problems because of animation instability due to FPS differencies (you end up movig more than intended this way, even if your animation FPS match game FPS).

The code is very simple:

Code: Select all

class RootMotionListener : public Ogre::NodeAnimationTrack::Listener {
	Ogre::Vector3 prevTranslation;
	mutable Ogre::Vector3 deltaMotion;
	flecs::entity e;

public:
	RootMotionListener(flecs::entity e)
		: Ogre::NodeAnimationTrack::Listener()
		, e(e)
		, prevTranslation(Ogre::Vector3::ZERO)
		, deltaMotion(Ogre::Vector3::ZERO)
	{
	}
	bool getInterpolatedKeyFrame(const Ogre::AnimationTrack *t,
				     const Ogre::TimeIndex &timeIndex,
				     Ogre::KeyFrame *kf) override
	{
		Ogre::TransformKeyFrame *vkf =
			static_cast<Ogre::TransformKeyFrame *>(kf);
		Ogre::KeyFrame *kf1, *kf2;
		Ogre::TransformKeyFrame *k1, *k2;
		unsigned short firstKeyIndex;
		float tm = t->getKeyFramesAtTime(timeIndex, &kf1, &kf2,
						 &firstKeyIndex);
		k1 = static_cast<Ogre::TransformKeyFrame *>(kf1);
		k2 = static_cast<Ogre::TransformKeyFrame *>(kf2);
		Ogre::Vector3 translation;
		Ogre::Quaternion rotation;
		if (tm == 0.0f) {
			rotation = k1->getRotation();
			translation = k1->getTranslate();
			deltaMotion = translation;
		} else {
			rotation = Ogre::Quaternion::nlerp(
				tm, k1->getRotation(), k2->getRotation(), true);
			translation =
				k1->getTranslate() +
				(k2->getTranslate() - k1->getTranslate()) * tm;
			deltaMotion = translation - prevTranslation;
			if (deltaMotion.squaredLength() >
			    translation.squaredLength())
				deltaMotion = translation;
		}
#if 0
			std::cout << "time: " << tm
				  << " Position: " << deltaMotion;
			std::cout << " Quaternion: " << rotation;
			std::cout << std::endl;
#endif
		vkf->setTranslate(deltaMotion);
		// vkf->setTranslate(translation);
		vkf->setRotation(rotation);
		vkf->setScale(Ogre::Vector3(1, 1, 1));
		prevTranslation = translation;
		// updating end motion via flecs :)
		e.get_mut<CharacterBase>().mBoneMotion = deltaMotion;
		e.get_mut<CharacterBase>().mBonePrevMotion = prevTranslation;
		e.modified<CharacterBase>();
		return true;
	}
};

(needs to be added via

Code: Select all

 mRootTrack->setListener(mListener);

for each root track in each animation) and that motion is used as usual i.e. can divide by delta to get velocity,
add gravity and other forces, rotate by direction, etc. Also rotation delta can be used when needed.

[sercero]

Cool, thanks for sharing

[chilly willy]

Here is a much simpler approach. I only implement translation but it I think it could be extended for orientation.

Preparation:

1. Detach root bone's children from root bone. (I do it per SkeletonInstance but it could probably be done on a Skeleton before instancing it (ie before creating an Entity))

   Code: Select all

   // Diconnect root bone for this skeleton instance.
   rootBone->removeAllChildren();
   

2. Leave root bone animation track alone. (Leave it as absolute keyframes not deltas from previous keyframe.)

3. Measure total movement for the root.

   Code: Select all

   // Measure total translation for root.
   Ogre::TransformKeyFrame * tkfBeg = (Ogre::TransformKeyFrame*)mRootTrack->getKeyFrame(0);
   Ogre::TransformKeyFrame * tkfEnd = (Ogre::TransformKeyFrame*)mRootTrack->getKeyFrame(mRootTrack->getNumKeyFrames()- 1);
   mRootTranslation = tkfEnd->getTranslate() - tkfBeg->getTranslate();
   

Then, every rendering frame:

Code: Select all

// Don't assume AnimationState::addTime()'s internal time-updating (ie modulo) implementation.

float lastTime = mAnimationState->getTimePosition();
mAnimationState->addTime(timeDelta);
float thisTime = mAnimationState->getTimePosition();

float length = mAnimationState->getLength();
bool loop = mAnimationState->getLoop();
int loops = loop ? (int)std::round((lastTime + timeDelta - thisTime) / length) : 0;

// Apply Movement

Ogre::TransformKeyFrame tkf(0, 0);
	
// Get root bone position before update
mRootTrack->getInterpolatedKeyFrame(lastTime, &tkf);
Ogre::Vector3 lastRootPos = tkf.getTranslate();
	
// Get root bone position after update
mRootTrack->getInterpolatedKeyFrame(thisTime, &tkf);
Ogre::Vector3 thisRootPos = tkf.getTranslate();
	
mSceneNode ->translate((thisRootPos - lastRootPos) + (loops * mRootTranslation), Ogre::Node::TS_LOCAL);

Advantages:

• Works for any number of entities with same skeleton
• Don't have to change animation track
• Works for looping and non-looping animations
• Works forward and backward (positive or negative timeDelta)
• Keyframes can be as sparse or as dense as required for animation
• SceneNode moves smoothly every frame, doesn't jump every keyframe
• Bounding box moves with SceneNode, model stays in center of bounding box, no culling artifacts from model walking out of bounding box

[paroj]

it would be cool, if you could contribute a small sample to the SampleBrowser to make this more discoverable and easier to iterate upon

[paroj]

btw this is what chatgpt suggests:
https://chatgpt.com/share/696fef1d-7d28 ... 4ec9b339b7

[slapin]

ChatGPT solution is misleading.

1. You can't set root bone position to anything if it is not set to manual.
2. You simply don't have any gap you can do anything between animations and rendering. All you can do is to run entity's
   animations by calling ent->_updateAnimation() (and do bone magic after that) and I dunno how safe that is. All you do before that is useless as bones will be set by animation tracks. Or not as according to forum some people have to apply tracks manually for manual bones... But for me manual bones are animated by tracks, but allow setting after animation, but by default there is no documented way to do so because there is no hook or listener which runs
   after animations are applied/applying but before rendering. Of course there are workarounds for some case but general hook would make this so much easier it is insane it is not there.

We need better listener which will behave like trackListener but running per entity to avoid lots of problems and have straightforward solution not only for root motion but also for IK and other constraints.

[slapin]

Here is a much simpler approach. I only implement translation but it I think it could be extended for orientation.

Preparation:

1. Detach root bone's children from root bone. (I do it per SkeletonInstance but it could probably be done on a Skeleton before instancing it (ie before creating an Entity))

   Code: Select all

   // Diconnect root bone for this skeleton instance.
   rootBone->removeAllChildren();
   

2. Leave root bone animation track alone. (Leave it as absolute keyframes not deltas from previous keyframe.)

3. Measure total movement for the root.

   Code: Select all

   // Measure total translation for root.
   Ogre::TransformKeyFrame * tkfBeg = (Ogre::TransformKeyFrame*)mRootTrack->getKeyFrame(0);
   Ogre::TransformKeyFrame * tkfEnd = (Ogre::TransformKeyFrame*)mRootTrack->getKeyFrame(mRootTrack->getNumKeyFrames()- 1);
   mRootTranslation = tkfEnd->getTranslate() - tkfBeg->getTranslate();
   

Then, every rendering frame:

Code: Select all

// Don't assume AnimationState::addTime()'s internal time-updating (ie modulo) implementation.

float lastTime = mAnimationState->getTimePosition();
mAnimationState->addTime(timeDelta);
float thisTime = mAnimationState->getTimePosition();

float length = mAnimationState->getLength();
bool loop = mAnimationState->getLoop();
int loops = loop ? (int)std::round((lastTime + timeDelta - thisTime) / length) : 0;

// Apply Movement

Ogre::TransformKeyFrame tkf(0, 0);
	
// Get root bone position before update
mRootTrack->getInterpolatedKeyFrame(lastTime, &tkf);
Ogre::Vector3 lastRootPos = tkf.getTranslate();
	
// Get root bone position after update
mRootTrack->getInterpolatedKeyFrame(thisTime, &tkf);
Ogre::Vector3 thisRootPos = tkf.getTranslate();
	
mSceneNode ->translate((thisRootPos - lastRootPos) + (loops * mRootTranslation), Ogre::Node::TS_LOCAL);

Advantages:

• Works for any number of entities with same skeleton
• Don't have to change animation track
• Works for looping and non-looping animations
• Works forward and backward (positive or negative timeDelta)
• Keyframes can be as sparse or as dense as required for animation
• SceneNode moves smoothly every frame, doesn't jump every keyframe
• Bounding box moves with SceneNode, model stays in center of bounding box, no culling artifacts from model walking out of bounding box

Thatks a lot, implemented this in animation tree environment, works like a charm!