Project Idea - IK support in Ogre's skeletal animation

Post by **xavier** » Fri Mar 16, 2007 11:23 pm

Rationale:

With the increasing (over)use of physical simulation in games and other interactive applications, the line is blurring about which part of the system is the actual "point of contact" for control over an actor's animation.

This has been requested from time to time over the years, and the need for it really wasn't all that great. However, when physics is brought into play, the normal actor update sequence starts to show its weaknesses.

It would simplify the actor update path if the physical environmental effects were more easily updated in an actor's animation; as it is now, with Ogre's FK-only animation, there is really no way to alter a skeletally-animated actor's animations as a result of physical simulation. Adding IK support to Ogre's animation would solve this issue.

This might be a bit much for a summer project -- but I thought I'd toss it out there if anyone were interested. It certainly would make Ogre even that much more attractive for commercial game development.

Comments?

Post by **Praetor** » Fri Mar 16, 2007 11:31 pm

I think one thing that could be done at first is generalizing the way animations work. So that taking manual control is easy. I have no specific ideas, but creating a way for generating multiple "systems" of animation (like FK, as well as several mode of IK, and others) can sit side by side, working together or not, as the application programmer needs. The API should be simple and general allowing mixing and matching animation control systems as well as easy plugging in of custom ones.

Post by **tuan kuranes** » Sat Mar 17, 2007 1:44 pm

Just wanted to point out those ideas are listed in Animation Enhancements.

Initially listed here.

Those proposition are indeed in need of good graphic/physic programmer, but students has to know that many papers and even some available code are on the web.

Alexander · Post by **Alexander** » Sat Mar 17, 2007 8:57 pm

IK would be a great addition.

jarek108 · Post by **jarek108** » Fri Mar 23, 2007 1:41 pm

i`m doing my master in artificial inteligence, working with IK on my robotics project for some time. I`m very interested in doing a project for gSoC, but i was working on a diffrent proposal focusing on computer vision. Since you as a promotor say that you are interested in IK perhaps i could also make second proposal, as it fits my background.

Could you please give me even more information and make more clear what features would aim at specificly in supporting phisics based animation(example of commercial existing system or a reference to scientific article would help me decide if this is something for me or should i rather focus on my 1st proposal), because my exp. with IK is rather about robot control and human motion capture rather that animation support.

Thanks.

Post by **tuan kuranes** » Fri Mar 23, 2007 2:59 pm

You should really read link posted above (there's paper, links, demos with source code inside)

About commercial existing product, as posted in link above, look at cryengine 2.0 and its videos, at endorphin, Cat (used in Ankh), cal3d

jarek108 · Post by **jarek108** » Fri Mar 23, 2007 3:14 pm

thanks tuan kuranes this is higly useful, im right on it but i`d also like to know what xavier had in mind. xavier is it the same stuff you wanted in your project?

Post by **xavier** » Fri Mar 23, 2007 6:46 pm

More or less items 2 and 3 in the Animation Enhancements section, with some of 4 thrown in for good measure (animators love IK, and it does tend to make the exported data sizes less).

CaseyB · Post by **CaseyB** » Fri Mar 23, 2007 8:52 pm

I think this would be a great project! Hopefully a student will like it and be willing to take it on!

jarek108 · Post by **jarek108** » Sat Mar 24, 2007 3:36 pm

ok so i`ve done some reading, the thing is really interesting! i`ve decided that this will be my second proposal. I do not have the big picture complete on this one i need to do more research before i can give a good proposal here. xavier, tuan kuranes could you take a look here
:
http://www.ogre3d.org/phpBB2/viewtopic. ... 683#211683

i`ve just posted my first version of 1 proposal. i`d like to ask you for a comment if you find it also interesting. I think in long term this will more beneficial but in short term IK may seem more atractive. If i had both of them accepted(right..) i would choose the first one also because I already know the subiect, i feel what i can finish during gSoC, and I`ve already spend a lot of effort on it. So please any advice that will give me more chance to be selected are very welcome.

jarek108 · Post by **jarek108** » Sat Mar 24, 2007 10:01 pm

ok so it looks this is my only proposal now (first one was too far from ogre,ehhh);). i`m trying to do some fast research to check what`s possible here. i think doing all the points you mentioned is too much for 3 months at least for me, but i may be wrong.

while following links and articles from tuan kuranes i found this:
http://www.mmandel.com/blog/?page_id=7

i`m too short to judge the quality but it looks like interesting marriage of simulation and animation. i`m inspired and i think i`d like to go deeper here and perhaps build my gSoC proposal around system like this(not sure yet what i want to change or how and how much of that can one do during gSoC, but while i`ll read myself more in the problem ideas will pop out).

ok so assuming for a second that i want to do exaclly that, what do you think guys? is it a good direction? is that more less what you expect?

Alexander · Post by **Alexander** » Sat Mar 24, 2007 10:32 pm

I believe there was an article in one of the game developer gems about "ragdolls with constraints" or something like that. Interesting but not really what I think is proposed about an IK solution, plus that particular link doesn't seem to have anything that resembles natural looking (it may just be the quality of animations used).

jarek108 · Post by **jarek108** » Sat Mar 24, 2007 11:08 pm

as i understand the system we are talking about is trying to combine predefined library of human motions with IK and higher level controllers to enable both nice human like behavior and interaction with physical world.

i did n`t understand 100% but i think this is what xavier was talking about (by the way what do you mean "actor update path" xavier? is it just a sequence of animation steps for the actor or what? i`m not familiar with that). what do you propose alexander? just implementing some IK alhoritm? this is far less than what the guy did. and as i understand will not be easy to combine that with predefined animations/phisical interactions you still need some controllers and you end you doing a system like the one discussed. please tell me what do you mean.

and from what the guy says on his www he was the one to write the article for the gems:).

Post by **xavier** » Sat Mar 24, 2007 11:14 pm

The order of update is important. Often you want to have the authored (keyframed) animation run first, and the results of either IK and/or physics update the character pose after that (usually with a blend between the authored and procedural animations). However, this should be configurable by the user -- I would suggest before and after events from the animation system that the developer can capture and handle as they see fit; possibly also expose the top-level animation update APIs (one for authored, one for procedural). It's a complicated topic -- we're going through designing such a system for our tech at work currently as well, which is sort of what brought this to mind for Ogre for me.

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 12:29 am

as i understand what you proposition it is a high level api for using:
1. predefined animations
2. phisical interactions
3. IK + some "logical" high level controllers for IK
allowing above to take place in user defined order? (sorry if i get it all wrong). is that the picture?

if that`s it, than it seems that the master thesis above is addressing similar goal but with other methods. am i right?

it seems you are close to that problem so i`d like to as for help(you too mr tuan, please any remarks), could i ask you to use your expertise to help me evaluate what in your opinion can be done during 3 months(this seems like a terrible lot of difficult work), what features definetly to include, where could my work end in each point?

I understand i would be to build api allowing controll of type and the order of aplying modyfications to body positions, implement some IK alghoritms and allow users to define more advanced things?

what level of inteligence do you expect me to implement in body controllers? will some toy example do?

how much would i have to do in phisics? can i live phisics entirely to external systems (just call them and allow them to modify position of body in some fazes?)

i hate it that i did n`t find out that it is gong to be my proposal before, i`d do sooo much more reading and would ask less silly questions. anyway, the time is running out for me:), i`d like to write the proposal tomorrow, and i want it to make sense:). thank you for any remarks.

Post by **xavier** » Sun Mar 25, 2007 2:53 am

I would leave physics out of this project entirely, and simply design/change the high-level animation interface (AnimationState, perhaps) such that any procedural methods (including IK) can alter/blend with an authored animation.

Then, the fun part is actually implementing an efficient IK solver that can be used in real-time applications.

This would be an entirely new type of animation, but it would operate on the bones of an existing skeleton (so you don't need to worry about parenting relationships -- they are already there).

And then figuring out how the data would make it from a modeling application into the Ogre resource management system (figuring out the data pipeline, in other words).

Does that help?

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 7:42 am

sure it helps, things are getting clearer.

1. whitch IK solver do you propose?
2. whitch aproach/alghoritm do you sugest for blending procedural and authored animations?

any resources helpful in implementation?

Post by **tuan kuranes** » Sun Mar 25, 2007 8:30 am

1) CCD is a good start, and well documented on the web. here and here

Keep in mind that the goal is to make the blending accessible outside of Ogre by users (so that they can plug their physic engine for instance.)

So maybe IK solver should/could end anyway in a Ogre Plugin, as it will anyway should use same exported "animation blending" interface as any other code outside Ogre will have to use.

The GDC Ode physic ragdoll controller was listed as a simple example of using that exported interface. Other example are users that would like to use Endorphin's "Natural Motion", or implement their own muscle animation technique

2) Read, search on the web to understand the in/out. Here's a good start

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 9:48 am

CCD than. just reading the articles about ccd and blending you pointed.

about the plugin stuff: i thought rather that this would be like accessible in the code level through class interfaces (so people can call it and attach a physics engine of their choice) and some scripting for blending configuration etc. + perhaps some small editor. You say that i should build full external editor?
Is that what you mean by "accessible outside of Ogre by users"? perhaps i should learn something more about how those Ogre plugins work but it looks that this would be 1. a bit tough and taking my focus from central IK stuff 2. would limit real time interaction between user application and my engine, no?

Post by **tuan kuranes** » Sun Mar 25, 2007 10:20 am

I mean only that Ogre core modification should be minimal and aimed at easing blending of "user" (procedural physics anything) animation with skeletal animation, giving an interface that is accessible outside of Ogre core.

Then IK animation control would/should/could be a Ogre plugin (like particleFX) that uses this interface. Then users using IK, or with their own animation controllers, can have the choice of disabling it or replacing it wihtout hacing into Ogre core code. (so maybe have a look at "Ode controller" paper/code in order to have an more general idea of ins and out of animation Controllers other that IK.)

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 12:51 pm

ok i belive it is time to make my proposal, trying to take all what you have been so kind to point me out.

i`m posting it here for now, please move it to separate thread anyone if you feel it should be there.

All the time i ask for comments how to make it better, more realistic to complete and stuff. I`d like to thank you guys for intensive and professional help.

Proposal:
Based on ogre3d forum discussions and project ideas:
http://www.ogre3d.org/phpBB2/viewtopic. ... 812#211812
http://www.ogre3d.org/wiki/index.php?ti ... hancements

Iâ€™d like to build a high level animation control plug-in for ogre. It would enable user to blend together multiple animations in a way similar to proposed here by ogre mentor tuan kuranes:
http://www.ogre3d.org/phpBB2/viewtopic. ... c&start=50
and to apply procedural animation controllers(eg apply IK and physics) blending them all together.

Some features:
-supply users with interfaces allowing them to define on a high (by scripts?) authored animations to be blended and the way thay should be blended(like in example in post above)
- provide abstract interface for animation controllers influencing the actor to build later user defined controllers(IK algorithms etc)
- allow user to easily attach external physics engine that will influence the actor.
-implement of CCD IK algorithm using the abstract interface
-if time allows work monitor mentors/users for any feature that will be expected and possible to implement

Why?
As I have been informed by ogre mentors high level animation controller including IK and smart multiple animation blending is something that ha been requested by users for years. It would allow users to create more attractive natural animations with less effort, allowing to build interesting new interactions with the environment. This would also be easily expanded by future implementation of human(non-human?) like motion controllers to acheave more autonomous, interesting behavior.

Why Me?
I'm a polish student on my way to two master's degrees (computer science and AI), i have previously for 2 years studied mathematics. Since 6 years my passion is AI (at the beginning neural networks later genetic algorithms, now mostly robotics(including some IK) and computer vision), i have finished various courses from that areas and during them I've implemented many smaller projects gaining a lot of practical knowledge about AI methods. I have quite strong understanding of C++ (more less level of understanding you have from deeply knowing Stroustrup`s â€œC++â€

Post by **tuan kuranes** » Sun Mar 25, 2007 1:28 pm

Just to be sure you know: Proposal are to be made on gsoc site.

And will need a planning and list of deliverables. Please try to be as realistic as possible about what you you'll be able to do or not in that limited time, and be clear about inter-dependencies of features to make priorities clearer.

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 2:57 pm

sure, i`m aware of that (that was about gsoc site)

ok i`ve added this to my proposal, please comment on realism:

Schedule and priorities:
As it is complex problem, Iâ€™ll start before gSoC begins, right after I will know if Iâ€™m accepted so I have more than 12 weeks.
I really feel I can implement system mentioned in weeks 1-4 during that time(perhaps making it look and use nice will take another week) also IK algorithm looks well documented and I think it will not be a problem to implement it in a given time. Here all is mostly clear. As for the system that takes outside physics, IK and standard animation i`m on a stage of reading about some interesting solutions and algorithms, but I feel it would be not fair to promise what it will look like and how much time it will take. I can promise hard work and progress according to my mentors remarks.

Bird`s view of the schedule:

Week 1-4
Designing and implementing system supporting standard ogre skeletal animation, similar to state machine to be controlled by a scripts working in a way described in previous post(â€œanimation blending for dummiesâ€

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 3:24 pm

i`m afraid i`m going abroad now (mallorca;)!) and i will have no chance to be near computer for few days so i`m forced to submit this proposal as it is, i hope you will like it. Of course i`m willing to make any changes that my mentors propose before the gSoC starts or later. so my request of remarks about realism stay up to date.

jarek108 · Post by **jarek108** » Sun Mar 25, 2007 3:27 pm

can someone change the topic to:
submited proposal: High level animation control plugin for OGRE (including IK support)

or should I make another post for that, as this was not originally about my proposal? i was once flamed for multiposting so you decide:).

anyway here is my proposal:

Proposal:
Based on ogre3d forum discussions and project ideas:
http://www.ogre3d.org/phpBB2/viewtopic. ... 812#211812
http://www.ogre3d.org/wiki/index.php?ti ... hancements

Iâ€™d like to build a high level animation control plug-in for ogre. It would enable user to blend together multiple animations in a way similar to proposed here by ogre mentor tuan kuranes:
http://www.ogre3d.org/phpBB2/viewtopic. ... c&start=50
and to apply procedural animation controllers(eg apply IK and physics) blending them all together.
Features:
-supply users with interfaces allowing them to define on a high (by scripts?) authored animations to be blended and the way that should be blended(like in example in post above)
- provide abstract interface for animation controllers influencing the actor to build later user defined controllers(IK algorithms etc)
- allow user to easily attach external physics engine that will influence the actor.
-implement of CCD IK algorithm using the abstract interface
-if time allows work monitor mentors/users for any feature that will be expected and possible to implement

Why?
As I have been informed by ogre mentors high level animation controller including IK and smart multiple animation blending is something that ha been requested by users for years. It would allow users to create more attractive natural animations with less effort, allowing to build interesting new interactions with the environment. This would also be easily expanded by future implementation of human(non-human?) like motion controllers to acheave more autonomous, interesting behavior.

Why Me?
I'm a polish student on my way to two master's degrees (computer science and AI), i have previously for 2 years studied mathematics. Since 6 years my passion is AI (at the beginning neural networks later genetic algorithms, now mostly robotics(including some IK) and computer vision), i have finished various courses from that areas and during them I've implemented many smaller projects gaining a lot of practical knowledge about AI methods. I have quite strong understanding of C++ (more less level of understanding you have from deeply knowing Stroustrup`s â€œC++â€