SmartBody - a character animation system for Ogre

[Demarii]

Points 1,2,3 would be terrific and go a long way in aiding artists/programmers of Ogre clients making use of the system. As an artist I think those cover the major bases, but I will ask our projects coders if they can think of anything other points.

[TheSHEEEP]

3) Examples using standard Ogre assets, such as standard Ogre characters (do these exist?)

Well, there would be the ninja, the robot and the ogre
The first two are always with the samples, have been since years. And the sinbad mesh is with the SDK since 1.7

[duststorm]

Is there anything else that someone can think of that I have overlooked?

I believe the Ogre sample is still compiled against Ogre 1.6
Updating it to 1.8 would also be a big plus for the community. That would also be the first thing I would do when I get to playing around with it.

[Demarii]

Yep seeing it working with 1.8 is high on the list for our project as well.

Another suggestion from our lead client coder was that it would be very beneficial if you looked into using a DCVS like Git or Mercurial instead of Subversion as it greatly improves the level of collaboration, fixes, and extensions that the Ogre community and others can offer back to the project in return.

I think having compiled binaries would be nice for artist types like myself who are not handy with compiling stuff, as we could have a set of working tools to work and learn with to prepare our assets so we have something ready for when our projects get the Smartbody system implemented. Not to mention you save yourself the trouble of having to help us through the process. Thanks again for that.... but for most project coders I would imagine they will want to work with the code itself.

[arishapiro]

Is there anything else that someone can think of that I have overlooked?

I believe the Ogre sample is still compiled against Ogre 1.6
Updating it to 1.8 would also be a big plus for the community. That would also be the first thing I would do when I get to playing around with it.

The current Ogre version that is part of the SmartBody distribution is 1.7.4, but I could upgrade to the Ogre 1.8 version.

Ari

[duststorm]

The current Ogre version that is part of the SmartBody distribution is 1.7.4, but I could upgrade to the Ogre 1.8 version.

Oh, my bad.
Transitioning from 1.7 to 1.8 should be very fast. Chances are you don't need to change anything.
The API changes between those versions are really small.

[f00bar]

have played with it a little, here my 2 cents (did not update in a while, maybe documentation is more detailed now):

• documentation: there is some documentation, but how it works underneath is not clear for me (e.g. how to use bml viewer? what is the overhead if I have to retarget all animations (had to do that with my collada files, bvh worked without retarget, but could not apply animations to my models correctly if using bvh skeleton)? Is it possible to do some kind of real-time retargetting if all I need is to rotate some bones?)
• modularity: in one of my projects I already have steering, can smartbody be compiled with just the features I need? e.g. without steering, without reaching, or without face animation?
• complete game character controller example would be awesome: walking, running, strafing, jumping, crouching, foot-alignment to slope, swimming, throwing grenades,.... , all in a single example, or some recommendations how to do that (e.g. is adding foot-alignment to slope with IK a good idea (strafing is more important to me, so I do not want to give up a blend-dimension for that)?)
• open content license: in the project where I already have steering, I also have some animations. I would have to bring those animations into smartbody, because the smartbody animations cannot be used freely. However, atm I only have the basic animations so I would have to create all missing ones first to use smartbody.

[Edit:] My point about documentation is not, that you should invest much time in it. I think smartbody has a nice documentation for an open source project. As more people use it, your forum will answer most questions and it will become clear which points are missing/not clear in the documentation. My point is just that documentation is important, e.g. if you have presentations you use to explain the architecture to students or other design documents, publishing those could help, or collecting good questions from the forums for the FAQ, or giving pointers which of your papers explains what aspect of smartbody.

[arishapiro]

The current Ogre version that is part of the SmartBody distribution is 1.7.4, but I could upgrade to the Ogre 1.8 version.

Oh, my bad.
Transitioning from 1.7 to 1.8 should be very fast. Chances are you don't need to change anything.
The API changes between those versions are really small.

Yes, I updated SmartBody to Ogre 1.81 without any problems.

Ari

[arishapiro]

have played with it a little, here my 2 cents (did not update in a while, maybe documentation is more detailed now):

• documentation: there is some documentation, but how it works underneath is not clear for me (e.g. how to use bml viewer? what is the overhead if I have to retarget all animations (had to do that with my collada files, bvh worked without retarget, but could not apply animations to my models correctly if using bvh skeleton)? Is it possible to do some kind of real-time retargetting if all I need is to rotate some bones?)
• modularity: in one of my projects I already have steering, can smartbody be compiled with just the features I need? e.g. without steering, without reaching, or without face animation?
• complete game character controller example would be awesome: walking, running, strafing, jumping, crouching, foot-alignment to slope, swimming, throwing grenades,.... , all in a single example, or some recommendations how to do that (e.g. is adding foot-alignment to slope with IK a good idea (strafing is more important to me, so I do not want to give up a blend-dimension for that)?)
• open content license: in the project where I already have steering, I also have some animations. I would have to bring those animations into smartbody, because the smartbody animations cannot be used freely. However, atm I only have the basic animations so I would have to create all missing ones first to use smartbody.

[Edit:] My point about documentation is not, that you should invest much time in it. I think smartbody has a nice documentation for an open source project. As more people use it, your forum will answer most questions and it will become clear which points are missing/not clear in the documentation. My point is just that documentation is important, e.g. if you have presentations you use to explain the architecture to students or other design documents, publishing those could help, or collecting good questions from the forums for the FAQ, or giving pointers which of your papers explains what aspect of smartbody.

Yes, and I agree with you. SmartBody currently has about 160 pages of documentation, and another 121 pages of Python API documentation, but I probably need double or triple that amount to explain the features thoroughly.

Regarding the various features, yes, you can disable some features (like steering) and any controller (such as gazing or reaching) that you want.

Regarding the complete character control, that kind of thing is one of the features; I have a group of designed actions called behaviorsets (currently: locomotion, reaching, gesturing, jumping) and you can retarget those behavior sets onto any character and control them. I'll post a video here when I am finished with the Ogre skeleton support that I am currently working on.

[arishapiro]

All,



My team has added support for the Ogre XML format (skeletons, animations, mesh and some textures) to SmartBody.
The following are examples of gazing, locomotion, reaching and jumping using Sinbad the Ogre using the SmartBody controllers and retargeting:

Sinbad Gazing (procedural gaze control)
http://smartbody.ict.usc.edu/HTML/video ... adGaze.mp4

Sinbad Jump (parameterized blend obtained from mocap)
http://smartbody.ict.usc.edu/HTML/video ... adJump.mp4

Sinbad Locomotion (parameterized blend with steering)
http://smartbody.ict.usc.edu/HTML/video ... motion.mp4

Sinbad Reaching (full body IK based on mocap)
http://smartbody.ict.usc.edu/HTML/video ... dReach.mp4

Sinbad Crowd (multiple Sinbads)
http://smartbody.ict.usc.edu/HTML/video ... Crowd2.mp4

Over the next week or so, we will add the commands to get these examples working to the main SmartBody distribution. Right now, Ogre assets in XML form can be parsed and read by SmartBody and rendered in Ogre.

Regards,

Ari Shapiro

[Jabberwocky]

Super cool.

[drwbns]

So awesome, a few glitches here and there but so cool, and a little funny as well I was going to say the steering looks a little odd with his legs so close together.

[nitro3]

It would be great if on your sourceforge page you could provide a compiled dll lib of the current stable version of SmartBody as well. So far I have not had success in trying to compile a working lib and I fear i am not alone. Really looking forward to trying this out. Thanks.

[arishapiro]

It would be great if on your sourceforge page you could provide a compiled dll lib of the current stable version of SmartBody as well. So far I have not had success in trying to compile a working lib and I fear i am not alone. Really looking forward to trying this out. Thanks.

Yes, binaries are one of the next things I will be providing in the near future. Right now, SmartBody supports many platforms: Windows (vs2008 and vs2010), OSx, linux, iOS, Android) and it isn't unusual for a checkin to break the build on one or more platforms. We have an automated build process for a few, but not all platforms.

If your having trouble with the build, would you mind either sending me the build errors directly, or post them to the SmartBody forum so that I can make the fixes to the codebase? At the time of this writing, the current revision (r4018) should build on Windows, linux and OSx.

Ari

[Lee04]

Hi Ari

Great work.
Very impressive.

Yes supporting Ogre formats natively would be nice.

Another thing that would be nice is to see some of the animation graph editors available for Ogre animations work natively with SmartBody directly in these tools view ports...

Tools are always good.

Lee

[arishapiro]

Hi Ari

Great work.
Very impressive.

Yes supporting Ogre formats natively would be nice.

Another thing that would be nice is to see some of the animation graph editors available for Ogre animations work natively with SmartBody directly in these tools view ports...

Tools are always good.

Lee

Can you post a link of an animation graph editor that you are thinking of? SmartBody can use the same animation assets, which can be manipulated and viewed in standard editors. When blending/playing animations, SmartBody them handles them in its own way, so it would be a matter of imitating the editor/GUI to give SmartBody the same high-level editing capability.

Ari

[Deathgunner]

Can you post a link of an animation graph editor that you are thinking of? SmartBody can use the same animation assets, which can be manipulated and viewed in standard editors. When blending/playing animations, SmartBody them handles them in its own way, so it would be a matter of imitating the editor/GUI to give SmartBody the same high-level editing capability.

Ari

I think this is what you are looking for.

[arishapiro]

Can you post a link of an animation graph editor that you are thinking of? SmartBody can use the same animation assets, which can be manipulated and viewed in standard editors. When blending/playing animations, SmartBody them handles them in its own way, so it would be a matter of imitating the editor/GUI to give SmartBody the same high-level editing capability.

Ari

I think this is what you are looking for.

Right, that looks like an animation blending tool for blend trees. Blend trees have been the standard basic animation design paradigm for awhile. Originally, my intention was to do the difficult work of designing a complete, custom blend architecture that covered many aspects of character movement, without requiring the user to build it themselves (including subtleties along the lines of making sure that the eyelids track the vertical position of the eyes and are simultaneously synchronized with the blinks, or ensuring that you can gaze at an object while moving, etc.). It looks like enough people want the blend tree architecture included. The capability is already included in SmartBody - it's mostly a matter of exposing it to the user.

Ari

[Mako_energy]

arishapiro,

SmartBody looks like a great library that had a substantial amount of work put into it and I greatly appreciate you making it available to all of us under such a liberal license.

I do have a few gripes about your library I wanted to mention. First was that I found the folder/file layout of your SDK download to be extremely counter-intuitive and locating the source code of the SmartBody library was a task in itself. For reference it appears to be in /core/smartbody/smartbody-lib/src .

SmartBody has an impressive feature list but seems to have been built with not much more than it's own demo's in mind as an end goal. By that I mean you are pulling in the functionality of many libraries (such as steering, locomotion, and physics) and are integrating them quite tightly with SmartBody. You have clarified earlier in this thread that there are abstractions/interfaces for physics. I haven't had a chance to delve into them very deeply, but they seem to store their own copies of transform data, which if they are truly wrapping objects from other libraries seems like a waste of memory. Maybe the degree of integration is necessary, forgive my ignorance if so, but it makes it sloppier for me to try and integrate it into my engine which has that functionality existing elsewhere. Another obstacle for integration is the lack of documentation. There is little to no source documentation and your manual only really explains things as an overview which doesn't do much to help me use SmartBody outside of the Demos.

I would love to integrate SmartBody into my engine, but these hurdles make that too difficult. As it stands SmartBody is more useful as a reference to use to create my own animation library. Is SmartBody still under active development? Is there a chance these things could be alleviated or even fixed?

[arishapiro]

arishapiro,

SmartBody looks like a great library that had a substantial amount of work put into it and I greatly appreciate you making it available to all of us under such a liberal license.

Thanks.

I do have a few gripes about your library I wanted to mention. First was that I found the folder/file layout of your SDK download to be extremely counter-intuitive and locating the source code of the SmartBody library was a task in itself. For reference it appears to be in /core/smartbody/smartbody-lib/src .

Yes, I agree. I plan on reorganizing it once I finalize the API in order to conform with more standard organizations, such as a bin/, include/, lib/ directories.

Another obstacle for integration is the lack of documentation.
...
There is little to no source documentation and your manual only really explains things as an overview which doesn't do much to help me use SmartBody outside of the Demos.

The documentation is currently consists of a 171-page .pdf, and a couple of hundred pages of Python documentation (which are reflective of the C++ API as they are generated using boost::Python directly from similarly named C++ classes). I haven't documented the source itself, as my expectation would be that people would not want to tinker too much with the internals (or at least, those weren't whom I considered to be the primary users). I agree that there needs to be more documentation. In the meantime, I invite you to post on my forum to ask any questions that might help to clarify its use as I make the platform more robust and complete over time.

You have clarified earlier in this thread that there are abstractions/interfaces for physics. I haven't had a chance to delve into them very deeply, but they seem to store their own copies of transform data, which if they are truly wrapping objects from other libraries seems like a waste of memory.

Yes, my team has constructed the basic interface for using different physics engines, but we haven't made them completely separately yet, nor tested the interface.

I would love to integrate SmartBody into my engine, but these hurdles make that too difficult. As it stands SmartBody is more useful as a reference to use to create my own animation library. Is SmartBody still under active development? Is there a chance these things could be alleviated or even fixed?

First let me say that SmartBody is under active development (you can look at the SourceForge commits and see that there are at least a few checkins every day). I have a team of three people (including myself) as well as an internal support team since we use the platform internally here.

You are welcome to glean ideas from the code - you may not need all of the features that SmartBody offers (i.e., do you need lip synching to voice, facial expressions, eye saccades, and object manipulation, or do you just want your characters to walk from point A to point B). As with any integration effort, there is a balance between taking advantage of the features that you want, versus being overloaded with features that you don't need. Of course, I appreciate your feedback in helping to shape the platform into a more generally-usable system. If you are struggling with certain aspects of it, I"m sure others are as well.

Over the short term, I will be trying to address some of the packaging issues (API, documentation, code organization), while other team members will be adding new functionality (such as online retargeting).

Regards,

Ari Shapiro

[arishapiro]

All,

We've added online retargeting as well as kinect support. The online retargeting allows you to transfer the kinect motion data directly onto arbitrary characters, even ones with varying pre- and post- rotational aspects.

You can see a video of this here:

http://smartbody.ict.usc.edu/HTML/video ... apture.mp4

We are using the Microsoft Kinect drivers and SDK.

Regards,

Ari Shapiro

[Xplodwild]

Hi,

I tried to start using SmartBody - and thanks for bringing such a cool thing to the open-source world, and most specifically us Ogre3D users.

Unfortunately, I ran into quite too much troubles:
- First, I use Visual Studio 2012. The source contains libs for VS 2008 and VS 2010, so I needed to rebuild a few libraries on VS 2012 (Festival, est, ...). There are a few compilation errors due to some C++11 changes (but very minor to fix, just remove the <T1, T2> template type from make_pair calls)
- Then, I use Boost 1.50.0 in my project, and SmartBody uses Boost 1.44. I tried to link against Boost 1.50 instead, upgraded the code to fit Boost changes, but ran into countless link errors from Boost.Python module. I struggled 4 hours with the libs, compiling in static and dynamic, changing the preprocessor NO_LIB define, to no avail: I couldn't get the linker to be happy. And Boost 1.44 doesn't support VS2012 out-of-the-box.
- I tried CMake: It cannot find festival and OpenAL even when forcing the libs paths, CMake returns some errors about install directories, I cannot get ALUT to build... Which raises another question: Why so much recursive dependencies? Why do you need Text-To-Speech and quite a lot of other audio libraries for a 3D animation framework?
- Last thing I tried was cleaning everything back, building required libs on VS2012, and keep on using provided boost 1.44 (with a small hack to make it build on VS2012). I could FINALLY get it to build, BUT no examples works: The Ogre-Viewer just stalls after initializing Ogre::Root, and every single other app crashes because Python cannot bind the socket (error 10048). I couldn't find the port it tries to bind to in the manual, but in any case I'm not running much network services on my machine. I'm not sure if this is due to lib conflict or what, but it simply doesn't work.

The only thing in the end that -should- work, is smartbody-lib. I read that you're encouraging Python use, but I just don't want to deal with python anymore. So it raises one question: Is there any doc or quick example on how to use SmartBody directly in C++, without using the Python bindings?

I understand that you want SB to be as generic as possible to fit with most engines, but it just makes the integration harder (esp. because of all the dependencies).

Thanks

[arishapiro]

Hi,

I'm not surprised that a Visual Studio 2012 build doesn't work properly; each Visual Studio uses its own runtime libraries and needs its own version of the library dependencies. SmartBody currently has support for Visual Studio 2010 and 2008, and when I have a chance, I'll add a 2012 version (maybe stop supporting the 2008 version?) which mainly involves rebuilding all the supporting libraries. I'd encourage you to perhaps downloading Visual Studio 2010 Express and trying again? The CMake build is tested on linux and OSX, but not on Windows and Cygwin. The plan for Windows users was to simply download, build and run without needing to do any extra work. Microsoft's new 2012 version of Visual Studio means more work for my team to support that platform as well. It can be a daunting task; we currently support six different platforms (VS 2010, VS2008, OSX, Linux, Android, iOS).

We have plans to upgrade the BOOST version as well, but there are some small API changes between 1.44 and 1.50 that need to be accommodated.

The entire SmartBody system contains a lot of components that aren't strictly necessary for an animation framework, like the text-to-speech system that you pointed out (also Kinect, Python support, mobile device support, etc). However, part of the goal of the project is to create a complete system that handles all aspects related to the character: walking, talking, touching, gazing, character physics, whatever. If we had more engineering help, we could do a better job of letting the use 'configure' the aspects that they want, and leave out the ones they don't.

You can build SmartBody without Python by adding the variable SB_NO_PYTHON, and use the API directly.

As I've mentioned earlier in the thread, I'd like to find a way to effectively 'package' it for Ogre in particular, perhaps as a plugin with some Ogre-specific configurations. I'm open to ideas.

Regards,

Ari Shapiro

[Xplodwild]

Thanks for the quick reply.

I'd be happy to help you with the Ogre wrapper. A great open source library like this deserves this kind of users contributions. I think it should work in a similar way to physics libraries bindings, hiding most of the internal classes and Interacting directly with ogres classes.

If you could tell me how to get started with the API (initialization, etc), I can dig through the code and start working on a wrapper/binding.

[Mind Calamity]

I've been delaying digging into this for a while now, even though it's been really tempting.

I agree with Xplodwild about that, and will help out as much as I can as soon as I get some more time.