Ogre Forums

Are there any news regarding XMLConverter? Ogre 1.10 nor 2.0 can't be released with this issue not getting fixed.

I'm hoping he's still around. He hasn't logged on in 4 months...

I'm hoping he's still around. He hasn't logged on in 4 months...

Hehe...I'm pretty inactive currently and limited in time, but I will get a mail if somebody replies to this thread.
However sometimes it doesn't work, because after a message, I need to log in and check the thread again to get a new email for newer posts. Maybe some forum admins could change this behavior?

Are there any news regarding XMLConverter? Ogre 1.10 nor 2.0 can't be released with this issue not getting fixed.

I'm still not sure whether putting mesh==>mesh operations (which has nothing to do with XML) into XMLConverter is a good idea. At least it should have a proper name like OgreMeshTool as scrawl suggested. If the user wants to have a binary with name XMLConverter, we can copy the file (and symlink on non-M$ operating systems). However I'm not officially an Ogre3D developer so my opinion is unimportant to this...so who likes this idea to have an allinone XMLConverter should do that part...


Some other good news is that there are new features to be expected, but bad news is that currently they are on hold for months and I dont have time currently to finish them.

These features are working somewhat (no threading and some flag combinations are not working too):
1. firstLodIsOriginalMesh: Allows to replace original index buffer (not revertable!)
2. trimOriginalVertexBuffer: Removes unused vertices from vertex buffer including vertices, which were in malformed triangles or never used or reduced or duplicated. (optimizes vertex buffer)
3. recalculateNormals: If you have used firstLodIsOriginalMesh, it will recalculate the normals for the reduced mesh intelligently, taking into account hard/soft edges from original mesh. (This caused the biggest graphical issues in the output)
4. better corner case handling: separated submesh borders, texture ripping, hard edges handling for better quality.
5. if vertex buffer has less than 65k vertices and it has used 32bit indices, then always convert them into 16bit indices. (Makes index buffer smalled. Maybe I will add an option for this)
6. move components from generateLodLevels() parameter to LodConfig for cleaner interface. (At least this should go into Ogre 1.10, because it changes the interface. Other features dont.)
7. merge equal vertices: For example OgreMax has exported my collada models in a way which was suboptimal. It was never reusing the same vertex even if the position, normal and uv coords was equal ( in other words: index count = vertex count). The problem is that my algorithm is marking "special vertices" (like texture seams hard edges in self shadowing etc), by marking vertices which were twice in the vertex buffer, but in OgreMax exported meshes, it was marking every vertex as special. Actually, on a 200MB mesh with millions of vertices, this doubled the FPS without any other changes, because this way you have less vertices and more optimized mesh after optimizing the vertex buffer with this option.

planned features:
8. when using workqueue, then process 1% of the calculation on main thread to prevent memory overhead. Instead of copying the vertex and index buffers, build up the LodData datastructure on main thread, it would be easier to build the LodData in the main thread and pass that to the worker thread.
9. Create an optimize(mesh) function, which will not generate any Lod levels, but optimize the vertex and index buffers for fastest rendering (just enabling some of the above features and generating new index and vertex buffers).
10. currently, the new features doesn't support threading or compression. (which is a huge part of the work)
11. generate Lod levels as separated meshes, which allows to recalculate normals for each Lod Level for best quality then load them as manual Lod levels.
12. There are some bugs in MeshLodTests, which needs fixes. Also we need tests for the new features too.
13. Resolution based autoconfiguredLod: In PixelCountLodStrategy with a 4K display, the first Lod level is too far away. This would optimize reduction for given resolution. Of course you would need to generate the LOD levels on the fly, because every user would have a different LOD reduction amount based on resolution used.

Disadvantages of the new features:
-50% bigger memory footprint: We now store for every vertex position: the vertexID list, including submesh, normal and UV coords. So we also need to touch the UV buffer and generate lot of extra data.
-20% slower processing time in general (with new features turned off or on. The new features doesnt need much calculations, but to generate the new members/information required in LodData).

Next weekend I will try to go through the changes and push interface breaking changes (shouldn't be so much), but I think most features can only go to Ogre 1.11, because I dont have time to finish it currently. I may make an Ogre 1.11 experimental branch with the incomplete features.

sajty wrote:I'm still not sure whether putting mesh==>mesh operations (which has nothing to do with XML) into XMLConverter is a good idea. At least it should have a proper name like OgreMeshTool as scrawl suggested. If the user wants to have a binary with name XMLConverter, we can copy the file (and symlink on non-M$ operating systems). However I'm not officially an Ogre3D developer so my opinion is unimportant to this...so who likes this idea to have an allinone XMLConverter should do that part...

XMLConverter is the key tool for importing assets from 3rd party modelling software. It is THE most important tool for content creation.
As such, it must support all important features (edge list generation, tangent generation, vertex duplication removal, etc).
OgreMax is a 3rd party sw that doesn't rely on this tool, but others do. And XMLConverter is the base reference for checking the conversion works.

7. merge equal vertices: For example OgreMax has exported my collada models in a way which was suboptimal. It was never reusing the same vertex even if the position, normal and uv coords was equal ( in other words: index count = vertex count).

Nice to have, but FYI the OgreMax plugin has an option for this in the settings. Ask the developers why it's off by default

9. Create an optimize(mesh) function, which will not generate any Lod levels, but optimize the vertex and index buffers for fastest rendering (just enabling some of the above features and generating new index and vertex buffers).

Take in mind most important optimizations are:

• Cache optimization (relevant literature: here, here, here, some code). We already have a cache optimizer and an AMD Tootle integration, but I don't know their current state or quality (IndexData::optimiseVertexCacheTriList, VertexCacheProfiler::profile). May be worth checking out.
• Compression. Store position as 16-bit half. Normal tangents and bitangents as QTangents. UVs as 16-bit half. This is already done in Ogre 2.0 though (we use a different mesh file format)

11. generate Lod levels as separated meshes, which allows to recalculate normals for each Lod Level for best quality then load them as manual Lod levels.

I would place much effort on manual Lod levels. They're a PITA to support and thus getting abandoned in 2.0.
You could still separate the meshes, use the recalculation functionality, and then merge back. But manual lod levels are most likely going to disappear.

Disadvantages of the new features:
-50% bigger memory footprint: We now store for every vertex position: the vertexID list, including submesh, normal and UV coords. So we also need to touch the UV buffer and generate lot of extra data.

At runtime in general or just the tool?

Nice to have, but FYI the OgreMax plugin has an option for this in the settings. Ask the developers why it's off by default

Hehe, but you cant trust the mesh input to be correctly generated. A wrongly exported/generated mesh should also have a nice LOD and in the LOD generation process it comes handy if the mesh also gets optimized.

Cache optimization (relevant literature: here, here, here, some code). We already have a cache optimizer and an AMD Tootle integration, but I don't know their current state or quality (IndexData::optimiseVertexCacheTriList, VertexCacheProfiler::profile). May be worth checking out.

Yeah, I know these and I have lot of great ideas in my head to improve the generator, but currently I dont have enough time to work on it.

Compression. Store position as 16-bit half. Normal tangents and bitangents as QTangents. UVs as 16-bit half. This is already done in Ogre 2.0 though (we use a different mesh file format)

Actually, I have read your Ogre 2.0 pdf after implementing these vertex buffer related features, so porting to 2.0 will also be harder, because now the algorithm is directly processing the position, normals and UV. I dont have checked ogre2.0 until now, but I hope the implementation will be transparent between different number representation (However that may cause overhead).

At runtime in general or just the tool?

Just the MeshLodGenerator, when generating the LOD levels. So its not that big deal, because normally you can preprocess your meshes and you will not get any slowdown ingame...if you generate them on the fly, then you should use a background thread anyways, so hopefully that 20% slowdown when generating will not get noticed by the user.