mdl2mesh - Neverwinter Nights model converter

[Marc]

Did you ever try to use a model that uses dds textures? For some reason, irfanview complains about not being able to read their header and ogre also has problems with it:

Error #: 9
Function: D3D9Texture::_loadNormTex
Description: Unable to load texture from c_av_salamander.dds.

Is nwn-dds some other/special/non dds?


Btw. did you build lua.exe by yourself? It seems as if it is a debug compile because it needs MSVCRTD.DLL .


I guess I also found models that don't work:

c_gynosphinx_02.mdl
c_gynosphinx_03.mdl
c_gynosphinx_04.mdl
c_gynosphinx_05.mdl

from http://nwvault.ign.com/View.php?view=ha ... il&id=4801

only produce this for the .mesh.xml file:

Code: Select all

<mesh>
    <submeshes>

They look correct in biowares model viewer.

[Marc]

I wrote a batch converter that you can download here: http://rochel.s83.deinprovider.de/temp/massconvert.zip (minimal source included, it's not that much.)

It converts a whole directory created by extracting an nwn-erf file. It filters mdls that

- are binary models
- inherit animations
- use textures that are not included in the pack

Afterwards, you have only working Ogre models without missing textures.
Furthermore it creates one universal.material file where all materials of all models are defined in. That way, overlapping material definitions caused by multiple nwn-models using the same texture are solved.

This converter calls mdl2mesh for each model. Therefore, you need to change the path to ogrexmlconverter at the beginning of mdl2mesh.lua. The script is left unchanged except that the ogrexmlconverter gets -t as a parameter for generating tangets. Otherwise stencil shadows don't work (at least not for me)

You have to differentiate between static and animated models. Call

mc {directory of extracted erf-file data}

for models that have animations and skeletons or

mcstatic {directory of extracted erf-file data}

for static models.

One bug/issue: I'm a bit uncertain about a material called BaseWhiteNoLighting. Sometimes, Ogre crashes when trying to load the converted models and that material is defined and sometimes it does crash if it is not defined. Maybe it has to do with static vs. animated models? I don't know. Atm I created such a material at the beginning of the universal.material file. If, after converting a directory, a model doesn't work, just remove that material from the universal.material file.

[bugshake]

all the models on the demo seem to be facing the -y direction...

so why not say (x, y, z) = (x, z, -y) or something? 90 degree rotations are always easy to do without special math functions
BTW great project!

[Borundin]

Did you ever try to use a model that uses dds textures? For some reason, irfanview complains about not being able to read their header and ogre also has problems with it

Have never tried the dds files from NWN. I have however converted them to TGA using DDSTools.exe (available from bioware site). This could have been because OGRE wasnt able to load them. Was last year so dont remember the details.

I guess I also found models that don't work:

c_gynosphinx_02.mdl

I will look into this ASAP. Thanks for bug report!

I wrote a batch converter that you can download here: http://rochel.s83.deinprovider.de/temp/massconvert.zip (minimal source included, it's not that much.)

It converts a whole directory created by extracting an nwn-erf file. It filters mdls that
- are binary models
- inherit animations
- use textures that are not included in the pack

Afterwards, you have only working Ogre models without missing textures.
Furthermore it creates one universal.material file where all materials of all models are defined in. That way, overlapping material definitions caused by multiple nwn-models using the same texture are solved.

Sounds like a great tool and thanks for sharing! Havent had time yet to test it unfortunately. The problem with overlapping material definitions was something that I had already thought of but didnt know quite how to solve. Binary models shouldnt be difficult to handle if you use the mdl2ascii or NWN_compDcomp as a prefilter.

The script is left unchanged except that the ogrexmlconverter gets -t as a parameter for generating tangets. Otherwise stencil shadows don't work (at least not for me)

Hmm, maybe we can put this parameter into main distribution? Only if it doesnt cause any problems of course.

[Borundin]

from http://nwvault.ign.com/View.php?view=ha ... il&id=4801

414Mb .... sigh

[Marc]

I just downloaded one of these mega-hakpaks. I figured it is much easier to use them in comparison to all those tiny paks when using my tool
I guess the models are in some smaller hakpak, too. I also discovered some other models producing the same output. It's about 20 I know of now. They are all in that pak. But I guess, if you can solve for these sphinx models, then the others probably will work either.

Have never tried the dds files from NWN. I have however converted them to TGA using DDSTools.exe (available from bioware site). This could have been because OGRE wasnt able to load them. Was last year so dont remember the details.

Ah cool. I'll try it in perhaps mass convert all .dds to tga in front, too

I will look into this ASAP. Thanks for bug report!

Right, should I post it somewhere else then?

Sounds like a great tool and thanks for sharing! Havent had time yet to test it unfortunately. The problem with overlapping material definitions was something that I had already thought of but didnt know quite how to solve. Binary models shouldnt be difficult to handle if you use the mdl2ascii or NWN_compDcomp as a prefilter.

I tried one a few weaks ago but it complained that I don't have nwn installed and it refused to work without it. But I'll have a look into the other one.

Hmm, maybe we can put this parameter into main distribution? Only if it doesnt cause any problems of course.

I don't know of any problems with that parameter. However, adding tangents to the models makes them larger. So if you don't need them it's probably better to not include them.

[Marc]

I added dd2 -> tga conversion and binary-mdl -> ascii-mdl conversion. I use DDSTools.exe from http://nwvault.ign.com/View.php?view=Ot ... ail&id=343 and nwnmdlcomp.exe from http://nwvault.ign.com/View.php?view=Ot ... ail&id=343. The only thing I'm not sure about is if it's allowed to include those tools in one package for our needs. It's not so easy to contact the authors about that. E.g. ddstools.exe is delivered in a zip file with nothing else in it. The comments on nwvault show that noone seems to know the author. For the nwnmdlcomp, there was a frontend included in the zip whos author wrote his email adress in there. I contacted him if he knows the author... :/

But I've got quite a lot of ogre models on my disk now. Does someone have an Ogre model thumbnail generator at hand ?

[Borundin]

Some new bugs showed up in the latest version unfortunately
Seems like default -nobonerot and -noanimpos parameters got inverted in the code. Im going to fix this in next version (coming soon). Im sorry but this might break your batch app Marc Though Id rather change it now than later.

[Marc]

I don't know exactly what those parameters do anyway. I just figured out that I have to set them for static models and leave them away fo animated - at least in your last version. If you just invert them, I can invert them on my side, too. No big deal. Or do you mean something else? If it's a bug then change it.

[Borundin]

Third version of mdl2mesh.lua have been uploaded to OgreAddons. Will probably take a few hours before it trickles down to anonymous access but if you wish you can also download the latest version from this link

Changes:
- Bugfix: c_gynosphinx_02.mdl can now be converted
- Inverted the meaning of parameter -nobonerot and -noanimipos

Marc: The parameter change means that if you previously used -nobonerot and was happy with the result then you probably will need to remove that parameter now and likewise if you didnt have it before you will probably have to add it with this new version. Same with -noanimpos parameter.

[Marc]

Cool. I plugged it in and modified my script according to your changes. I updated the zip file of the previous location. http://www.mpe-online.org/files/massconvert.zip
Again, set up mdl2mesh.lua as always. Then use mc {dir} or mcstatic {dir} respectively.

Be careful! The script converts all binary mdl files to ascii mdl files and overwrites the binary mdl files in that directory. You should keep a copy of your mdl files in case the binary2ascii converter breaks something. But since you probably download some files from nwvault anyway, this is not too dangerous

[edit]I updated the batch tool. It fixes 2 smaller bugs I encountered related to special characters in names. Now, you can specify the scale at cmdline too like mc[static] {dir} {scale} .[/edit]

[saetrum]

Hi guys...I've got some questions I hope you can help me with:

First off, thank you Borundin for sharing this very useful tool.

I've run into a problem with the converter. See this post about models with a NULL bone. Any ideas? How is everyone getting around this?

2nd - I see the hakpaks that are available, but I am unsure how to use them. I don't own NWN (I know I'm missing out), is there a way to get the models, textures, etc. out of a hakpak without NWN or am I out of luck?

Thanks!

[jacmoe]

am I out of luck?

You are.
But: NWN is probably in the bargain bins now - so grab a cheap copy.

[Borundin]

Well,.. hrmm it seems like there is an error in my script . For the Giant_Spider model the quick fix is to edit Giant_Spider.mdl with notepad for example, then delete the space character after "parent NULL" on line 11. Then save and rerun the converter script and you will be fine.

Code: Select all

node dummy Giant_Spider
  parent NULL <---- delete space chacter here

Note: For this particular model you have to use "Apply bone rotation: No" !!
I will change mdl2mesh.lua so that this will not be a problem in the future.

[Edit:]
About you 2nd question. It depends really. Alot of hakpaks just includes the changed/new textures to save bandwidth. In some cases the entire hakpak is new content that does not use any NWN textures and then you will get everything you need from the hakpak itself.

[Edit2:]
I use the Erfedit tool to extract models and textures from the hakpak file. Dont think you need NWN to run that tool.

[saetrum]

Thank you Borundin! Deleting a space is an easy workaround for now. Is there a way to know if I need to apply bone rotation or not?

I'll search for erfedit and see if it works for me...otherwise, maybe I'll see if I can find NWN cheap.

Thanks again!

[Borundin]

Is there a way to know if I need to apply bone rotation or not?

Unfortunately the only way to find out so far is by testing. If the different bones looks like they are out of place try changing this parameter.
Since the MDL Viewer from Bioware doesnt have this problem there must be a way to find out of course but I havent been able to see the difference yet.

[saetrum]

Unfortunately, it seems that after deleting the space, the animations no longer work. The conversion is sucessful, but when I bring the model up in ogre (or the OGRE mesh viewer) the model doesn't animate. Any ideas? Or should I just wait for an updated version of the script?

edit: For Mass convert to work, do the models have to be in binary format?

[Borundin]

Unfortunately, it seems that after deleting the space, the animations no longer work. The conversion is sucessful, but when I bring the model up in ogre (or the OGRE mesh viewer) the model doesn't animate. Any ideas?

That is strange because before I posted the solution I also made sure that it in fact showed up properly inside ogre. I tested with the cwalk animation and I have a screenshot to prove that it actually works

Did you forget to copy the new mesh and skeleton file to your media directory?

[Marc]

edit: For Mass convert to work, do the models have to be in binary format?

No. It only converts binary to text format automatically, deleting the binary version. The conversion to .mesh applies to all of them then afterwards.

[saetrum]

Unfortunately, it seems that after deleting the space, the animations no longer work. The conversion is sucessful, but when I bring the model up in ogre (or the OGRE mesh viewer) the model doesn't animate. Any ideas?

That is strange because before I posted the solution I also made sure that it in fact showed up properly inside ogre. I tested with the cwalk animation and I have a screenshot to prove that it actually works

Did you forget to copy the new mesh and skeleton file to your media directory?

Really? I can't seem to get it to work. All the animations are there, and I can use them as with other models, but they don't seem to do anything. I'm not using the spider model, and perhaps that is the problem, I'm using the Umberhulk model (also created by the NWN team, available from here http://nwdownloads.bioware.com/neverwin ... erhulk.exe ), but I thought the same solution would work. I tried it in both the mesh viewer and my own program with no luck. I wish it were as simple as me forgetting to copy the file, but unfortunately, there seems to be a bigger problem that I can't figure out.

No. It only converts binary to text format automatically, deleting the binary version. The conversion to .mesh applies to all of them then afterwards.

I keep getting exceptions thrown (Memory access errors) when I try to use it with models that are already in ascii format, I'll have to troubleshoot a little more, and see if I can figure out why.

[Marc]

I keep getting exceptions thrown (Memory access errors) when I try to use it with models that are already in ascii format, I'll have to troubleshoot a little more, and see if I can figure out why.

Well in the current version, you most likely get that when some files are in wrong directories. You have to be in the directory of the extracted massconvert.zip file and execute mc[static] {dir} {scale} where {dir} is the dir of the mdl files you want to convert. That's it. What hakpak/models did you try? Maybe there is a critical bug in it. I have used it successfully on approx. 5 packages though.

[Borundin]

I'm not using the spider model

You didnt say that and the spider model is the one we have been discussing earlier.
Anyway, I will try and have a look at Umberhulk later.

[saetrum]

You didnt say that and the spider model is the one we have been discussing earlier.
Anyway, I will try and have a look at Umberhulk later.

Oh, I realize that I should have told you which model I was using. My fault completely. Don't get me wrong, I really appreciate your help, and I am not expecting you to solve every little thing that goes wrong with my project. Once again, thank you for your help. I appologize I didn't specify which model, this is a problem I have seen with almost every model I've tried to use (that had a NULL root bone), so I assumed it was a common problem regardless of the model.

Well in the current version, you most likely get that when some files are in wrong directories. You have to be in the directory of the extracted massconvert.zip file and execute mc[static] {dir} {scale} where {dir} is the dir of the mdl files you want to convert. That's it. What hakpak/models did you try? Maybe there is a critical bug in it. I have used it successfully on approx. 5 packages though.

Let me explain my directory structure:
massconvert directory is on the same level as the model parent dir
ie. massconvert DLATreant
with in massconvert, I have the .exe's for massconvert
in DLATreant I have the .mdl files, tga files, etc.
I go into the massconvert directory and type mc ../DLATreant

It finds the models and converts one which is binary but after the conversion, I get the exception mentioned previously.

thanks for your help. I realize I could just use each tool individually for such a small hakpak, but I wanted to see how massconvert worked before I attempted a large hakpak.

Thanks!

[Marc]

It finds the models and converts one which is binary but after the conversion, I get the exception mentioned previously.

That's an easy one. The scale parameter isn't optional. Try "massconvert ..\DLATreant 1" (Also note the \ instead of the /. )

I tried that hakpak to be sure. For me it run till the end without an error. But when loading it, some nodes moved wrong. I guess that's again this problem with these parameters you can give to mdl2mesh.

If you need to change these parameters for massconvert, you can replace some 0's by ones in mc{static}.bat. In "@massconvert todo.dat %2 1 0 0 %1" represents "@massconvert todo.dat %2 {skel} {bonerot} {animpos} %1". mc and mcstatic.bat only differ by those bits btw. For the majority of the models I tried the default values worked, for that DLATreant they seam to be wrong.

[saetrum]

That's an easy one. The scale parameter isn't optional

I'm an idiot!
Thanks!


edit: The scale parameter did the trick. I'm now trying to play around with the other parameters to get the animation correct. Still can't get the Umberhulk to work...don't know why.