Quake (MD2, MD3 and MD5) to Ogre Mesh Converter (v2.1)

[mkultra333]

Ha! That's excellent. Funny to see Betruger in an Ogre demo.

[AndiNo]

Haha, that really is excellent Betruger could speak, too!^^ Seems as if you already solved all possible problems with MD5 as you already have animations in place. Very good work!

[Devil N]

Yes, all the major problems concerning the MD5 conversion have been solved, so support for the format is pretty much complete. There are a couple of small inconveniences remaining, but essentially the project has now become more of an engineering challenge than a research one. I'm giving the rest of the application a bit of an overhaul as well.

Let me share some of the details of what I've done the last two weeks. Getting the mesh into OGRE was easy enough. The source code that mkultra333 conveniently pointed to already had a function that generates a mesh, given a skeleton and a list of weights. All I had to do was call that function with the bind-pose skeleton and there I would get a nice bind-pose mesh. The only weird thing about MD5 meshes is that they don't have any normals; you have to generate them yourself from the mesh the old-fashioned way. Any irregular shapes in the normals are added through the normal maps.

Converting the skeleton was a bit more challenging. You see, MD5 skeletons have all their bones in object space, whereas OGRE expects bones to be defined in joint-local space. It's nothing a little math can't solve, but I had made it slightly difficult on myself because my quaternion-to-angle-axis function accidentally used a cos() instead of an acos(). Much pointless debugging ensued.

The conversion of the animations was where things got really confusing. Again, MD5 animated bones are all in object space, while in OGRE they are relative to their non-animated selves and relative to their animated parent bones, which in turn are also relative to their non-animated selves. As I write it down here, I can't even remember exactly how it works. Math came to the rescue here too and I thought I had it nailed, but there were a couple of instances where bones would move in the wrong direction, most notably root bones. I spent a few days in utter frustration, until I went over the whole thing again step-by-step, when I finally found out the solution. In short, I was cutting a few too many corners in my previous solution.

After that, I spent waaaay too much time (about three evenings' worth) setting up a bunch of scenes, tweaking lighting and composition, capturing the exact right moments, and finally splicing and editing, all to produce a mere 30 seconds of video. I can be such a fiddly perfectionist bastard sometimes. You have seen the results of that work.

As for Betruger speaking, that is thanks to the wonderful magic of overdubbing.

[AndiNo]

Seems like it was a great amount of work... Anyhow, you did a great job on that!

PS: What model format is next?

[prchakal]

I think this tool have to detect all the animations and do the automatic conversion intestead of fill xml information.

[Devil N]

I think this tool have to detect all the animations and do the automatic conversion intestead of fill xml information.

That would be the best, I agree. The problem is that going from any of the MDx formats to OGRE is not a 1-on-1 conversion. In some cases MDx misses information that OGRE needs, so you have to add that info yourself, and in other cases MDx offers more than OGRE can handle, so you have to choose what you want to keep. If the program were to do everything automatically, then it would have to make certain assumptions, and not all of those assumptions will be what you - the user - will want. That's why customization in the form of config XML files is needed.

Having said that, I did mention I wanted to make the app more batch-processing friendly, and automatic conversion might be part of it. Config XML files will stay, but say that you drag and drop an MDx file onto the executable, it could try to make the sanest assumptions possible and work with that. If you're not happy with the results, you can always create a custom config afterward.

P.S. the reason why I chose XML for configuration is because I had already included the TinyXML library for outputting OGRE meshes. So I figured, why not use it for input as well? It sure as hell is less confusing to use than endless command-line arguments, and it's easier to program too.

I'm still trying to work out how I'm going to implement meshes into my project. The complicating issue is that the level editor is based on DarkRadiant, which works with md5 models. Changing it to use ogre .mesh would be very tricky so it would probably be easier to change my ogre project to load md5 models and convert them to .mesh on the fly. This would also make the project more interesting and usable to the typical Q3A/Doom3/Q4 user.

But there's the extra overhead on level load of doing the mesh conversion. And support for the .mesh format in editors is generally better than for md5. I may opt to make the project support both, depending on how fast the conversion runs.

If I do use the md5 format I think your code will be really handy.

I still wanted to respond to this post.

As you know, the code in this project is written to be used offline, and it outputs OGRE mesh/skeleton XML instead of binary files. It shouldn't be too difficult to adapt the code to work in an online environment; all the conversion steps remain the same. You would just have to change all the output routines from using TinyXML to using the OGRE API instead, and a couple of math routines would have to be rewritten using OGRE classes.

Inside your OGRE project, you would then have to use MeshManager::createManual() with a custom ManualResourceLoader object that can load an MD5 file into a Mesh object. Of course I haven't tested any of this, but I think that's how it would work.

[prchakal]

You can do the automatic detection of animations, so, this is the more ugly part.

With the animation detection, frames and animation name i think it will help much more, the other configurations we customize.

[Rico]

First at all, sorry my bad english. ):

Hi, I am new here as you all can see.
I was just searching for a help/code/howto/howto begin or how it WOULD begin to OGRE3D read/load MD3, MD2, MD5, MDL, all quake models series.
That is the best thing I ever found, its a pretty good work, I cant say good job, only excellent job!
I have almot the same doubt that was asked some replies before in Page 2.
Your code already have any LICENSE?
Because I really want to use it trying to make what I said before, make OGRE3D load this without any problem, or some issues, nothing is perfect
Your code is just what I might need, it has the Ogre Mesh information and MD3, MD2 information, so I could try make it.
Thanks since now for this awesome Converter... I will not say piece of work because I don't really know what expression means... (little work? good work? GREAT WORK )

[Devil N]

First of all, welcome here and thank you for your compliments. I'm flattered.

Yes, I did include an MIT license file with the source for version 1.1. This means you're allowed to do with the code whatever you wish. I'm considering adding an MIT license header to all the source files as well, because it drives the point home better than a single LICENSE file does.

As I mentioned earlier, my code does not actually create OGRE meshes, just XML files that can be converted to OGRE meshes using the OgreXmlConverter tool. You can adapt the code to directly output OGRE meshes, but it requires some extra work. It should be a good place to start though, because it does show you how to solve most of the challenges.

Anyway, before you get busy with my code, you might want to wait for the next release, which shouldn't be too far from now. Aside from adding MD5 support, I'm also giving the rest of the source code a bit of an overhaul, which includes refactoring the MD2 and MD3 conversion classes into a more logical Builder pattern. It should make the code more useful for reuse. I've got a long checklist of other things I'd also like to do, but a lot of those could probably better wait until another release further down the road.

[jacmoe]

If you aren't hosted elsewhere, ask for a place in ogreaddons. I'd really hate to see this disappear.

[Rico]

If you aren't hosted elsewhere, ask for a place in ogreaddons. I'd really hate to see this disappear.

So do I would

Thank you all, ok, I will wait the update (:
Thank you again!

[Sslaxx]

thread necromancy

This looks a pretty interesting tool - any new developments?

[Devil N]

Dang, are you guys telepathic or something? Just last week I picked up work again, after having not touched the project for several months (yeah, shame on me). I was already thinking to myself, "I'll bet you see someone will be posting in that topic right now", and guess what, not a day later someone does.

Basically, the new release has been in a state of near completion since October, but has stagnated on a single feature which I think is important, but that will create a mess of the code no matter how I look at it. That, and the fact I've been working on my master's project and I couldn't really use the distraction, have been the reasons for the delay. Anyway, with my master thesis currently undergoing scrutiny, I thought I'd take another fresh look at the problem. To start, I replaced all the ugly C-style vector and quaternion math with easier to understand object oriented routines, that are also interface-compatible with Ogre's classes. The aforementioned problem with the last feature remains the same, but I think I should just suck it up and implement it regardless. Other than that, there's just a bit of testing to be done and it should be ready for release. Documentation and better examples have also been done for a while, so in other words, the next release shouldn't be too far off now.

[Devil N]

Well, here it is, finally. Check the opening post for details and a link to version 2.0.

Back when I started working on MD5 models in October, I never would have guessed the eventual release would be in February. But the last few months have flown by frighteningly fast for me, with too much to do in too little time. But better late than never, I suppose, and hopefully the tool will still be useful to some. If there are any questions about the tool's functionality or how to use it, feel free to drop a note here.

[jacmoe]

Of course it's useful!
Thanks a lot for taking the time to do this.

[mkultra333]

Cool. When I get up to incorporationg meshes into my Q3A related ogre project, which should be soonish, I'll probably be using this code as my foundation.

[Devil N]

I just read on sinbad's Twitter that he plans to add support for nlerp'ed normals in morph/pose animation. That should fix the lighting problems with animated MD2 and MD3 models after conversion to Ogre mesh format. The Quake-to-Ogre converter should be forward compatible with this new feature (if indeed the Ogre XML Converter is updated too), but I will test it as soon as it is available. To be continued...

[AndiNo]

Sounds interesting. Please keep us up to date.

[mkultra333]

Interpolated normals would be very handy. I'm getting closer to the stage where I may be using this stuff. Thought I'd mention I'm integrating Ogre and Q3A at this very time, you can have a look at the latest results here, http://www.ogre3d.org/forums/viewtopic. ... 82#p386582

BTW, if you have any inclination to join in the fun, let me know, you'd be most welcome.

[Devil N]

I just checked the Ogre commit log, and it looks like sinbad already delivered on his promise back in late June. Support for interpolated normals in morph/pose animated models is now available in the Hg default branch. I checked the modified DTD for mesh XML files, and it looks like I guessed right what it was going to look like. This means that the current version of QuakeToOgre should already be compatible with this new feature. I haven't tested it yet though, I will do that soon-ish if I can find the time.

[jacmoe]

Would you consider putting the code up at Bitbucket, Google Code or similar?

[Devil N]

I might just do that. I've been meaning to do some experimenting with Mercurial, and this seems like a good excuse. The source code currently resides on my personal Subversion server, which technically speaking has always been publicly readable, just not publicly known. I think I'll move the source to Bitbucket before the next release.

Yeah that's right, I said next release. I've tested the whole morph animation normals thing with Ogre 1.8 and it works well as expected, but when sinbad said that it wouldn't be compatible with stencil shadowing, he wasn't fooling around. When stencil shadows are enabled, Ogre basically ignores the included normals and treats them as vertex positions, which royally messes up the mesh. That's why I'm currently preparing a version 2.1, which makes the generation of keyframe normals for MD2 and MD3 models an optional feature, disabled by default.

[jacmoe]

WooHoo!
I'll let you work in peace - but it's a very cool project you've got here!

[Devil N]

Well, here it is then, QuakeToOgre version 2.1.

Not much has changed functionally this time. The most obvious change is the move to Bitbucket, as I already announced. It actually happened the very same evening the above two posts were made. The conversion from Subversion to Mercurial was so straightforward and easy, that it didn't take me long to make the jump. So far I've been really pleased by both Mercurial and Bitbucket.

Other than that, there's the promised addition of an <includenormals> tag, to control the inclusion of normals in morph animation keyframes. I made some small modifications to ensure full DTD compatibility with Ogre's XML spec, and I reorganized some bits and pieces. The binary package now also doesn't include the source code anymore -- you can easily get that from Bitbucket by downloading the corresponding tag.

Besides that, I also did a whole bunch of testing to check for any regression, which you obviously don't see back in the Mercurial changelog. Ironically enough, the only major problem I found is a couple of syntax- and semantic errors in Ogre 1.8's XML mesh DTD file. I'll submit a patch for it soon, as soon as I get access to a scanner so that I can submit the Contributor's Agreement.

[darkbrain]

Hello
i've some problems, converting an md5 mesh, i'm just trying to convert player.md5mesh from Quake4

1. Small issue, alfer i obtained my player.mesh.xml and player.skeleton if i try to execute OgreXMLConverter i obtain the error: Unable to load skeleton player.skeleton for Mesh conversion. Workaround i pass the player.skeleton directly to OgreXMLConverter, and it's works.
2. It's seems not compatible with Ogre1.9, OgreXMLConverter says: Exception caught: The Mesh you have supplied does not have its bounds completely defined. Define them first before exporting.Unregistering ResourceManager for type Skeleton.
3. When i try to load the mesh, i obtain the error: Cannot find serializer implementatino for current version [Serializer_v1.10]

And a queston: How can i specify all textures?

tnx