[SOLVED] Easy way to support > 256 bones?

[TheSHEEEP]

Hey,

We are currently re-developing our engine at work, using Ogre, and some of our characters have animations with far over 256 bones.
As the animation system really is too much for work me to dive into right now, I'm asking here:

Does anyone know a way to circumvent that limit?
I saw that almost everything animation-related is stored in (unsigned) shorts, so my guess is that this is a "not possible".

I could try replacing every short with an int, but I fear what would actually be needed for more bones is a rewrite of the animation system.
But I hope that I am wrong
Edit: Yeah, I mistook short for char, sorry for that

Btw. we don't use hardware skinning, and IIRC that is the may reason for that 8 bit limit. If that is not the reason, then what is the reason for it?
For more high-poly characters it is not unusual to have more than 256 bones.

[mkultra333]

unsigned shorts don't limit you to 256, they limit you to 65536. Did you mean unsigned chars?

But an easy way to do this, I think, is just to manually update the mesh verts yourself on the cpu. I think the meshes normally max out at 65536 verts, but you could even have multiple bones per vert that way, so theoretically there'd be no limit to the number of bones you could have in the anim.

Edit: In fact, just realized I sort of do this myself already in my project. I have 512 cubes used as boundaries for small deferred shading lights, and I move each cube around independently by updating the mesh. So it's like having 512 bones, except I only alter position, not orientation. But such a system could be expanded to include orientation, scaling, and more than one "bone" influence per vert. I guess the tricky part is if you want to use the normal Ogre animation system as well, the animation tracks, skeleton bone hierarchy and stuff. I guess you'd either have to break the anim up into different skeletons each with a max of 256 bones and then use the bone info to manually update the mesh verts, or abandon the Ogre system entirely and roll your own.

[TheSHEEEP]

Ah, yes, forget that thing about 256 due to short. Stuff like that happens to my mind on Mondays.
Also on Fridays, btw.

But then I really do wonder.. why the limit to 256, as short is now out of the question?
OGRE_MAX_NUM_BONES is only used two or three times in the code, and in most cases only to cause an OGRE_EXCEPTION (in one case it is used to initialize a string array). But I do not see anything that would break the system if it was, like.. 512 or even more. Guess I'll just try it out as soon as I get back to work.

I won't be able to create my own animation system (time limitations), but it could be possible to split them up, indeed. Especially during the export, that should be possible.
I'll have to think about that, maybe I can come up with something.

[bstone]

Yeah, shorts are not that short

[TheSHEEEP]

Damn, now I'm stuck with that
Well, serves me right.

But please, I still have that bone number problem, so if anyone knows...
a) why it is there in the first place
b) how to circumvent it without creating my own animation system
... that would be great.

I will see if it is a possible solution for us to split up the skeleton, but won't be able to check that before Wednesday, as I'll be at an "event" on Tuesday.

[bstone]

Well, the main limit should be in a hardware skinning implementation since it's usually bound by parameters number available to the vertex programs, but that also means way lower than 256 bones (more like 20-30). But if you're fine with software skinning then just bump the OGRE_MAX_NUM_BONES value up and it should work - the shorts are not that short and the underlying container is std::vector<Bone*>, so you should be safe (although I guess having lots of simultaneous 256+ bone animations might impact the performance too bad).

[TheSHEEEP]

Simply changing OGRE_MAX_NUM_BONES to a higher number worked.
Which makes that magical number even more strange, to be honest, as it seems completely arbitraty and not neccessary at all. Not sure if that is worth a bug report or papercut...

We won't have performance impacts as there will never be more than a handful of animated characters in a scene. Currently, there is exactly one

[bstone]

One sounds good enough

I guess The 256 bones limit could be a remnant from an older Ogre version where it could use a fixed array with unsigned char indices for e.g.

[mkultra333]

Glad it worked. Nice when things turn out to be simple.

Out of curiosity, why do you have one character with over 256 bones?

[bstone]

I have one explanation - he hated the character so much that he actually went and fractured all its bones, one by one

[TheSHEEEP]

Nope, the actual reason is that we really have some very advanced animations for the body, and in addition a lot of bones for speak animations as we are doing lip syncing (basically, our characters move/dance while talking). We are creating videos with that.

Also, in our last engine - or rather current, as our Ogre engine is not yet done, that's what I'm doing - it was easier for us to have one skeleton fit as many characters as possible, including stuff like tails, hats, etc. so there are extra bones that are not actually needed by all the characters. Plus, for the speak animations, we have each bone at least twice, because the old engine simply required that as a silly workaround for a bug/feature.

And as time is of the essence, we cannot re-create all the skeletons/animations for Ogre, to use less bones. I'd love to, as that would also mean a minor performance boost, but that's the simple business reality. You never have enough time. Ogre has to work with the current animations or it won't work at all.
But hey, all seems fine now
At least for our animations.

[bstone]

Yep, that explains why you hated it so much.

[Kojack]

The reasons for 256 bones:
DirectX 9 has a limit of 256 world matrices (bones) within one batch.

The Flexible Vertex Format (used for fixed function) datatype for bone blend indices is D3DDECLTYPE_UBYTE4, which gives 4 bone indices affecting each vertex, with a max of 256 possible bones.

In shader model 2, there isn't enough constant memory to pass in anywhere near enough matrices for 256 bones (the real limit is around 30. To get ogre to render 40 bone models with hardware skinning I had to change the shader to SM3 instead of 2).

[Wolfmanfx]

Also the 4 weights per bone is the hardcoded max inside optimised util

[bstone]

Thanks to Kojack, the mystery is unveiled now! That also confirms that it's better to love your characters and keep their bones safe

[Kojack]

Note: I'm not saying 40 is the limit of SM3. My students had a game with a group of characters running around with 40 or so bones each. It was killing framerate with software skinning, so we switched to hardware and the shaders failed. Changing the profile to SM3 and bumping up the bone arrays let it run. It could have gone higher I guess.
(40 bones was overkill for these models, but it was easier to change the shader than to tell the artists to redo them better)

[PhilipLB]

Maybe it would be good to determine the maximum amount of bones of a mesh dependent on the used shader modell and the render system? If the mesh is above this, print a warning to the log.

[spacegaier]

Maybe it would be good to determine the maximum amount of bones of a mesh dependent on the used shader modell and the render system? If the mesh is above this, print a warning to the log.

+1. Either at runtime and then a log entry, or at least a quick table / page in the wiki for everyone to easily look up in similar situations.

[bstone]

That highly depends on the particular vertex shader that does the animation and as we know vertex shaders tend to do some other stuff as well. Can't see how you could do that in a general way. Maybe with RTSS you could do something like that but I lack the details. Anyway, if the number of bones exceeds the limit - you will see that right away

[dark_sylinc]

Ok a few notes:

* +256 is insane for any model. It doesn't have to do whether it is high or low poly. An rig like that is probably flawed, shame on the rigger. The only way to get that amount of bones and still think it's reasonable, is that it's animating a really big, large and complex model, something like mechanical animation. A Transformers movie comes to mind, and even then, they don't use >256 bones (you can ask on animation studio's Youtube channel, they usually answer these type of questions and are very friendly); but if you batch a whole castle with animations anywhere as a single batch, then it's possible to reach that.

* The average human rig has between 40 & 60 bones, depending on head & hands (fingers) complexity.

* Like Kojak said, the limit happens because bone indices are stored as ubyte4 in the vertex stream, causing a hard limit of 256.

* As for the 4 weights per vertex, again it's because we use one ubyte4 (we would need two ubyte4 to get 4 additional weights) and because the number 4 plays nice with the packed architecture of GPUs & SSE. More than 4 weights per vertex is rare, as it is very hard to manage for the animator, and offers diminishing returns at the expense of lower performance.

* SM 3.0 & 2.0 have the same limit to pass matrices to the vertex shader. The theoretical maximum is 85 (256 constant registers over 3). VS 1.1 has a theoretical maximum of 24 (96 constant registers over 3)
Of course, if you use the theoretical max; you'll loose the ability to pass any other kind of information.

* If you really want to use >256 bones, split the mesh in half (or more) and problem solved. An easy way to do it is to assign different materials in the modeling application before exporting. Don't worry about having twice the render calls, performance will be crippled because of updating the skeleton, caused by the high number of bones; specially since profiling reveals it's a bottleneck. I'm preparing a doc about that.

[bstone]

* If you really want to use >256 bones, split the mesh in half (or more) and problem solved. An easy way to do it is to assign different materials in the modeling application before exporting. Don't worry about having twice the render calls, performance will be crippled because of updating the skeleton, caused by the high number of bones; specially since profiling reveals it's a bottleneck. I'm preparing a doc about that.

Actually, yeah. The vertex shader limitation is per sub-mesh as Ogre only passes the matrices used in the rendered sub-mesh. dark_sylinc is spot on about that.

[Kojack]

SM 3.0 & 2.0 have the same limit to pass matrices to the vertex shader. The theoretical maximum is 85 (256 constant registers over 3). VS 1.1 has a theoretical maximum of 24 (96 constant registers over 3)

Good point. It would have been vs 1.1 that I changed to vs 3. It was years ago.

[TheSHEEEP]

I agree that we would not need > 256, but even if we went and reduced the unneccessary ones, I'd still expect us to be somewhere between 100 and 250.
We have multiple bones for each finger(!!), and you can't reduce that if you want realistic finger animations. In the facial region alone, we have over 40 bones, and yes, you do need them if you want good and flexible facial expressions (which are pretty much requirement for emotional lip syncing). Just have a look at the vast facial editors in games like Skyrim. Try to count the bones, you'll end up with a surprisingly large number.
Also, as I said, currently, the skeletons include bones not needed for many characters (skirts, hats, tails, etc. which in the future will be their own meshes with own animations to attach as props) and some that are needed as workarounds for the current engine, and we do not have the time to re-do hundreds (yes, hundreds) of animations.

Also, our meshes are already split into a variable number of submeshes. This, I guess, is the reason why everything works, now that I have increased that MAX_NUM_BONES.
Maybe the exporter/serializer checked for the limit at the wrong place (not at submesh-level, but at skeleton level for the whole mesh), and therefore caused the exception even if > 256 was not really a problem for the mesh as a whole.
If we encounter any problems with that from now on, I'll sure remember this thread.
Thanks for all the insight, guys!

[FlorianGeorge]

I am using too a model that, when exported from Blender to Ogre, complains about having more than 4 weights for some bones. The workaround was using poses to simulate the more complex bones. The detailed explanations above (thanks!) seem to reaffirm that is was still the easier solution than somehow hacking in support for more than 4 weights.

[dark_sylinc]

We have multiple bones for each finger(!!), and you can't reduce that if you want realistic finger animations. In the facial region alone, we have over 40 bones, and yes, you do need them if you want good and flexible facial expressions (which are pretty much requirement for emotional lip syncing). Just have a look at the vast facial editors in games like Skyrim. Try to count the bones, you'll end up with a surprisingly large number.

Hands usually are 3 bones per finger (thumb is 2), sometimes + 1 more bone per finger, and then 1 or 2 to the hand. Worst case scenario (4 * 4 + 2 * 3 + 2) = 24
A normal human will have around 30 bones for the rest. That makes 30 + 24 = 54 bones.
Plus 40 head bones, 40 + 54 = 104 bones.
That's not even close to 256. Let's make it harder and add the toe's finger bones. 1 per finger is more than enough (using good weighting), that's 10 bones more. It's still 114.
You can double the previous number and still not get to 256.

Note that you may be having actor bones, IK bones, IK Target bones, etc; all of which may not be needed to be exported at all. If those get exported, it would be then reasonable that you're exceeding the 256 limit. Consult the documentation of your exporter plugin to see how to tag some particular bones as "do not export"

For the face, I would say it would much easier to manage with shape keys. But admittedly the 4 blendshapes/morph target per constant limit on Ogre is horrible (when doing it in the vertex shader). It forces a lot of bandwidth if you want to reach the VS 3.0's max of 28 blend shapes at the same time. Again, the document I'm writing for Ogre 2.0 covers this issue too, extensively.

Tip: Note that games like Skyrim often switch between models. When you're customizing they switch to a model with lots of shape keys and bones (and depending on the game, it's also higher poly to give you the illusion that's the quality all the time).
But once you've defined the customization, they'll bake all the results into a new mesh and switch to that model, which now contains much less bones and blendshapes (but enough so there's still facial animation, as well as detailed movement animation).
During cinematics, most games also switch models.