Lightwave converter

[Kolobrod]

The new version of converter crashes after exporting material script... Previous converter tool works fine, so i need to use MeshUpgrade tool.
--
I use Deep Exploration tool (from http://www.righthemisphere.com/ ) to remove discontinuous UV maps from LWO files.

[dennis]

Yes, it's because of a MeshManager thats needs to be instantiated. It's a couple of posts back. Patch submitted.

http://sourceforge.net/tracker/?group_i ... tid=302997

Let's try to keep the number of tools needed to a minimum, ok?

[_mental_]

The patch has now been applied to the Hastur branch.

[dennis]

Hmmm ... it seems NVidia is quite pleased with their NvTriStrip solution for polygon optimization. It's promoted in almost every presentation about the GFX pipeline I have seen from them. A demo did show a dramatic improvement over (non-indexed!) trianglelists on my GF2MX, but predicting the level of improvement over indexed triangle lists is not really my cup of tea. When polygons are rendered in random order, the vertex cache hit should be very close to 0. That is the current situation.

The general solution for optimizing Ogre meshes would be to incorporate this tristripper (modified for indexed triangle lists?) into the mesh serialization process somehow. That way ALL models will be optimized ... not just the ones coming out of this converter. Microsoft does offer an alternative: ID3DXMesh::OptimizeInplace. It can be integrated into Ogre, but it is crude and DX only. (Ogre mesh -> DX Mesh -> Optimize -> Ogre Mesh)

Some more research got me this info: All results of the operations (transformations, vertex shaders) on vertices are cached in the vertexcache. The vertices are stored in the vertexcache on a FIFO basis. It doesn't matter in what order the vertices are stored in the vertexbuffer, although vertex locality is preferred. But the order of the polygons is very important. Theoretically, the ideal order would be a zig-zag shape of triangles:

This should maximize vertexcache usage (extra re-use of 8 vertices?) with a maximum vertex cache size of 16.

[tuan kuranes]

ogreaddons meshviewer has a tool to convert it ogremesh in tristrip that sould be more cache friendly. ( it seems to uses this code http://www.codercorner.com/Strips.zip from this article : http://www.codercorner.com/Strips.htm )
As you said, You can also modify nvtristrip to support indexed tri list instead of strips.

Don't know if it's already the case, but what is also important would be to make submesh that use the same material to use a shared vertex buffer in order to help the "batch, batch, batch.". It's almost as important as vertex caching.
For example a character with one atlas mapping but several submeshes (for animation) should share the same vertex buffer.

[dennis]

I've just read the article and this stripper :
- is not very vertex cache-friendly
- has some improvement in memory consumption

Creating strips in a straight direction means that only the last two vertices are ever re-used, which is a big improvement over no cache hit at all. Zig-zagging is a far more cache-friendly. So creating an indexed tristrip that zigzags has the best of both. But hey, it seems I invented something so I call it a trizigzag. Expect a siggraph paper soon

[tuan kuranes]

I think i've seen your technique in this paper :
http://www.cs.technion.ac.il/~gotsman/A ... Render.pdf (look the figuire 2 a) snake raster)

Optimization of Mesh Locality for Transparent Vertex Caching is the siggraph paper describing the DX vertex cache optimisation :
http://research.microsoft.com/~hoppe/tvc.pdf

[tuan kuranes]

http://www.cs.technion.ac.il/~gotsman/caching/
Sample based on the paper seems interesting. I like the idea of the vertex
cache simulator. Too bad they uses vrml...

[sinbad]

I still prefer for the tool to do stripification / cache optimisation rather than the Serializer, since the Serializer's job is just to write data out. The tool should have full control over the data and not be overridden by the serializer. Besides, this feature should not be limited to .mesh, custom geometry could use it too.

I've thought about including an optimizer helper class but it just hasn't registered on my TODO list yet, which is always chocka (==packed full, incase that's not obvious). Besides, I wasted several days on a tristrip optimizer a while back only to find I got worse performance. I realise that cache optimisation is more likely to give benefits though; if I get time, I'll look at it but don't hold your breath.

A cache optimisation routine which operates on a VertexData and IndexData instance (not built into Mesh, since it can be used elsewhere) as a patch would be welcome, if anyone has time to tackle it.

[dennis]

I'll make a start with that and implement the simplest algorithm as a testcase. This algorithm would simply pick a polygon and traverse over the edge-connected polygons.

[dennis]

Heh, I was looking for the dependencies .zip and stumbled on the tools download page where I notices the Lightwave converter has been downloaded the most!

http://www.ogre3d.org/modules.php?op=mo ... load&cid=3

Oh, IndexData optimization is almost done. I promise it will be better than having no optimization at all. I'm even throwing in a simple VertexCache profiler just for laughs.

[sinbad]

Heh, I was looking for the dependencies .zip and stumbled on the tools download page where I notices the Lightwave converter has been downloaded the most!

Not wanting to burst your bubble, but it's because the other tools were updated for 0.15 so have had their counts reset. Sorry

If you send me an updated recompiled Lightwave exporter I'll update the download too (resetting your count too though).

[dennis]

Heh, I was looking for the dependencies .zip and stumbled on the tools download page where I notices the Lightwave converter has been downloaded the most!

Not wanting to burst your bubble, but it's because the other tools were updated for 0.15 so have had their counts reset. Sorry

If you send me an updated recompiled Lightwave exporter I'll update the download too (resetting your count too though).

DOH! ... but I guess I'll wait a little while longer

What about some optimizing news ...

Code: Select all

lwo2mesh Toutatis.lwo
Triangulating layer 0, Polygons before: 12796, Polygons after: 12796
MaterialSerializer : writing material default to queue.
MaterialSerializer : done.
MaterialSerializer : writing material(s) to material script : Toutatis.material
MaterialSerializer : done.
Surface: | hits | misses |  hits | misses |
default  | 8913 |  29475 | 22927 |  15461 |
MeshSerializer writing mesh data to Toutatis.mesh...
File header written.
Writing mesh data...
Writing submesh...
Submesh exported.
Exporting bounds information....
Bounds information exported.
Exporting submesh name table...
Submesh name table exported.
Mesh data exported.
MeshSerializer export successful.

With this object we have a winner. However, the results do vary a lot ... and I think I may have to eat my words about not making things worse. But: the algorithm is so incredibly stoopit it can only get better, and it's a start isn't it. I will make the patch tomorrow.

[sinbad]

Don't bother with a patch, I've given you direct commit access to the Lightwave exporter folder.

[P]

Hi all, this is my first post on the forums.

Dennis, i m experiencing several problems while trying to convert my .lwo files to .mesh using your lwo2mesh converter. I m not saying the converter is doing bad work, i maybe forget to do something to make it work. I think you can help me.

I m not able to export meshes with working uv coordinates. I tried with a cube, and many others meshes more complex. The result i m getting into Ogre, is a mesh with texture on it, like i expected it, but with some parts badly uv mapped, badly textured.

The only way i found to fix that, is to use the 'unweld' vertex command inside Lightwave. Here i m getting good mapping.

But if i m unwelding the vertex inside Lightwave, i can only get hard edges on my mesh, and of course i m doubling the number of vertex.

What i want to do is finding the way i must follow, to model my mesh inside Lightwave, and beeing able to get a smooth effect or hard edges if necessary, with correct texture mapping on it. It seems that the UV coordinates are modified during the conversion process, and never reach the 1.0 value.

Time for questions now :

I guess Ogre is able to render mesh which have shared vertex, right ?
Inside lightwave we adjust the smoothing, with setting up a maximum angle of smoothing between 2 polys. Are you able to convert that thing ? I guess the only way is to convert this to smoothing groups, using several submeshes maybe ?

I m really hoping to be able to get the same result as the one i m getting inside the Lighwave viewport. I know that Lightwave is maybe not saving datas a way that make it easy to read from Ogre.

Can you help me ?

(sorry for my bad english)

Cya

P.

[P]

I spoke with Sinbad an hour ago, and most of my questions have find answer.

I ll ask again later if i need help

[P]

I use Deep Exploration tool (from http://www.righthemisphere.com/ ) to remove discontinuous UV maps from LWO files.

How do u do to remove discontinuous uv with this tool ? the only result i m able to get is to lost my uv

isn t there a command inside lightwave to convert sicontinuous uv to basic uv ? i have tried exporting to lightwave 5 .lwo but this way i m loosing uv too

thx

(sorry if these post aren t at the right place, maybe better to hae them inside the help section)

[dennis]

Nono, this is the right place for any Lightwave Converter issues. (If Sinbad ever creates a converter/export specific forum, I would ask him to move this topic there.) I have looked into the discontinuous vertex map problem only to conclude a rewrite of a big piece of code is needed.

Up until now, the number of vertices in the .mesh was the same number as in the .lwo file. This will change because all vertices with more than 1 UV coord in the same map have to be duplicated. (Oh, think about the fun stuff of disecting multiple discontinuous vertex maps while keeping the number of vertices in check! )

Unwelding is a quick hack, but leads to problems with smoothing. (A small workaround could be to map seams on relatively flat surfaces.) I WILL solve this particular problem, but please bare with me and have some patience ... (or ... try it yourself if you need it now.)

[P]

Thx you for your answer


I think the way to go is :

going throught polys, looking at vertex position
{
if vertex position not exist in the vertex list1
copy vertex position into the vertex list1, position data
}

going throught polys, looking at vertex position and uv while looking at the vertex list1
{
copying vertex position into the vertex list2, position data
if uv not exist in the vertex list2, uv data
copy uv values into the vertex list2, uv data
else
split a new vertex into the list2 with the same position value
copy the uv value into the list2 new vertex, uv data
}

I m not sure i m clear

Btw there is many others things to consider, like copying the normals and it s becoming a bit complex for me. Anyway I ll try to fix it myself but i m not sure i ll be able to do it

thx again

[P]

And there is another thing to do, to split a new vertex when we want a hard edge.

I don t know how the smooth information is coded inside Lightwave, but i have downloaded the SDK, begin to read docs about it, and with your code and a lot of time i hope to be able to do a plugin for Lightwave to export a .mesh

SDK is helping a lot with picking information we need

I would like to say that in case i will not be able to fix, i can wait

cya

[dennis]

I don t know how the smooth information is coded inside Lightwave

The surface has an angle parameter for when to smooth a vertex. Please look at lwLayer::calculateVertexNormals for details. Basically when the angle between polygons is below the surface smoothing parameter and smoothing is enabled, the vertexnormal is recalculated (altered average of the connected polygons)

[P]

Hi

I have started to code a Lightwave plugin (.dll which can be loaded by Lightwave) which export all objects into a scene into .xml data for the meshes with their .material scripts.

It s pretty easy, first because Lightwave SDK is easy to use, and because it comes with a great example to read datas from objects and store them into a DB.


What the plugin do for now :

It loads all objects which are into a scene (which have been sent to the layout editor), look at the maxSmoothingAngle between polygons surface parameter, and split a new vertex if the angle between 2 polys is greater than the one defined into the surface parameter.

Then, it writes a .xml which can been converter with OgreXmlConverter.exe, and a .material (only color parameters for now)

The plugin exports vertex positions, normals and uvmaps.


The problem !! :

Lightwave is storing a uvcoord by vertex, not per vertex and per poly, and here comes the problems. This means that if the vertex is on the uv seam, it will have only one uv coord (one u and one v of course), so when i split a new vertex, i m not able to give him a different uv coordinate like i have to do in order it to works.

The thing is, Lightwave is only using shared vertrices, and is able to draw the uv map correctly, even if there is a uv seam.

I saw that it s possible inside Lightwave to assign uv per poly (polys uv) but it means that the user will have to prepare his mesh before the export (which is bad) or find a way to not do any uv seam (like applying 2 textures with 2 uvmaps for a sphere). And i have not be able to assign per poly uv inside it, when i tried it myself.

So i thought there were a way to bypass this problem. First unweld all vertices inside Lightwave, then call the plugin and modify the plugin in order to weld vertices which has to been preivously unweld. But even if vertex are unwelded, i ll find the same uv value on the both vertices evne if they are on a seam...

Why Lightwave don t need to assign differents uv on a seam ? i guess it s because it is able to play with "discontinuous textures".

Any idea ? Something i m understading bad ?

Thx you for your help in advance

P.

[dennis]

This is the exact same challenge I'm facing: dealing with discontinuous vertex maps. (VMAD) Unwelding everything won't solve anything and will make life harder as well. Unwelding only the vertices on the seam is a quick and dirty work around. The seam then does need to be in a position where unwelding isn't that noticeable.

[P]

The seam then does need to be in a position where unwelding isn't that noticeable.

You mean because unwelding result in a hard edge ?
I guess, but i haven t tried it yet, that to get a smooth shading between 2 polys which are not sharing vertices, you have to recalculate vertex normals depending on the 2 polys normals.

Do u know how i can inside Lightwave, convert a vmap into a VMAD ?

I really see no other solution at all, and i begin to think that the only way is to avoid uv seams inside lightwave

[P]

hum something just being a light into my mind :

When i was using your converter, i was able to get correct uv map (with a seam) if i was unwelding uv vertices inside Lightwave. Is it because Lightwave convert vmap into a VMAD (coded inside the .lwo i found info on vmad) if vertex are unweld ?

I don t know how to read vmad with the sdk. But i found you can read them from .lwo

P.