Lightwave converter

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

There's a couple of ways. Firstly, you could set the first texture layer's blend mode to 'replace' using:

Code: Select all

mat->getTextureLayer(0)->setColourOperation(LBO_REPLACE);
This changes the default blending from modulation with the vertex colours to total replacement by the texture. However this will turn off any influence of the vertex colours, including dynamic lighting so this may not be for you.

The second way is just to set the base material properties to Colour::White. When I wrote a 3DS converter (before the exporter) I had an option to export material colours but I usually left this off, since colours are generally used to tag different objects / groups rather than actually being a genuine rendering colour.

NB I'd suggest that you use .material scripts rather than trying to export material settings from the modeller; modellers typically have very different material settings from realtime renderers (lots of procedural stuff) so what I normally do is put placeholder materials in the modeller to line up textures, then define a Material with the same name in a .material script using all of OGRE's features, this replaces the one in the model at runtime. Most of the time I don't even bother putting material definitions in my .mesh files (it's an optional component of the file).
In Lightwave the color attribute of a surface can contain both an rgb and a texture. When a texture image can't be loaded, the rgb shows and the rgb is part of the calculations for procedural textures. In fact, the color attribute is the only attribute that can contain an RGB.

Should I multiply the rgb values with the diffuse value (0..1) and the specularity value (0..1) to get the parameters for setDiffuse and SetSpecular?
Hmm, yes I guess this is your only option. It's a bit odd for a renderer not to support different base diffuse and specular colours; that stops you from colouring the highlights of the model slightly differently. Unless LW does it a different way that is.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

lwo2mesh.exe v0.82 (Win32)

OgreMain.dll, devil.dll and stlport_vc645.dll are included in the package.

This version should convert geometry, UVmaps and materials correctly.

Test it and please post you feedback here or send me an e-mail.
ChronosWS
Gnoblar
Posts: 1
Joined: Sun Jan 26, 2003 12:58 am
Location: Redmond, WA

Post by ChronosWS »

Would be nice if the source is platform agnostic, if you can. If you don't know how to make it so, or don't have access to the documentation for Linux, I'd be happy to help out.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

ChronosWS wrote:Would be nice if the source is platform agnostic, if you can. If you don't know how to make it so, or don't have access to the documentation for Linux, I'd be happy to help out.
The source is as platform independent as can be. Even the byteswapping for different-endian platforms is in there. But of all the platforms Lightwave used to run on, the only one left that counts is Win32. (>95%?)

(From another topic:)
patlecat wrote:The same counts for the Lightwave exporter!! We need animation :!:
Hey Pat, well the best I can do for you is export a skeleton and that's what I'm looking into right now. With Lightwave, animation of objects is done in the scene, not in the objects themselves.
User avatar
patlecat
Gnoblar
Posts: 8
Joined: Fri Dec 27, 2002 1:00 pm
Location: Switzerland

Post by patlecat »

Marvelous Dennis :D

Would it be abusing your kindness if you could also look for a scene exporter?


Greetz, pat le blissed :twisted:
"What the heck do I need to test my code for? It compiled man!" (an anonymous purist)
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

I have sent the latest version v0.87 to Sinbad. It includes the source. Exporting skeletons is NOT implemented, although I made a small start with it. I probably won't continue developing it, but I hope someone else does. Have fun with it!

regards,

Dennis
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

When the VBO branch was merged into the HEAD branch somebody forgot to send a message to a contributor who wrote a conversion utility to convert Lightwave objects into Ogre meshes ...

Looking at the Milkshape exporter things on the mesh-building front have become reaheally complicated! A lot of variables that used to be in GeometryData simply don't exist anymore. Anybody on the Ogre-team up for making the changes quickly ? (lwo2mesh.cpp and lwo2mesh.h) Please let me know because if I'm going to do it, it's going to take a lot of time.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Yeah, a few of the exporters which we didn't write will need updating, I was hoping people would pick up on them from the new snapshot. I'm trying to resolve a D3D7 problem and write a tutorial on the VBO changes (which should hopefully help with your confusion).

Once you understand the VBO structures I'm sure you won't find it hard to update the exporter. In many ways they're easier to use than the old GeometryData because they're more explicit about vertex formats (wheras these were implied previously by pointers and stride values). If you look at the existing code you'll find the same pattern everywhere. To help get you started before I finish the tute, here's a potted sequence:

:arrow: Create a VertexData structure either on the Mesh or on the SubMesh depending on whether the SubMesh uses shared geometry (this option has the same effect as before)
:arrow: Update the vertex declaration on VertexData to reflect the elements of the vertex. This is done by calling 'addElement' once per vertex element (e.g. positions, normals etc).
:arrow: Each element has 5 important attributes.
1. The 'source' is the buffer the data comes from (as an index, not a pointer, because you can re-bind vertex buffers without changing the element)
2. The 'offset' is the byte offset from the start of a vertex at which this element starts. This is only > 0 if a single buffer contains multiple elements.
3. The 'type' is the data format. For example positions are normally VET_FLOAT3. This is used for sizing and to allow different dimensions of data to be used than just x,y,z. It is separate from the semantic (see below) because this is important for future shader support - some shaders might accept 4D vertex positions, for example.
4. The 'semantic' is the meaning of the data, ie is it a position, a normal, a texture coordinate etc. The use of enumerated semantics rather than predefined named pointers allows for significantly more flexibility now and for the future (cards are already starting to accept new semantics like patch data)
5. The 'index' is optional and applies if you're supplying more than one element of the same semantic, e.g. multiple texture coords or indeed multiple positions (for vertex shader interpolation).
:arrow: Once you've defined the vertex declaration, you need to create the vertex buffers, you do this using HardwareBufferManager. Notice that you only need to supply the number of vertices and the size, not the vertex format; this is because the buffer is independent of the format, again for flexibility since you can use the same buffer to supply data to multiple declarations if you so wished. Buffers are reference counted so they delete themselves when all users have finished with them. The docs explain the 'usage' parameter, but for an exporter you don't need to worry about it because you'll be using the base buffer manager (DefaultHardwareBufferManager - you should create this in your exporter) which creates buffers in software with no restrictions.
:arrow: You then need to 'bind' the buffer(s) to the indexes referred to in the 'source' values of the vertex element using the setBinding method on VertexData's vertexBufferBinding member.
:arrow: Now set the vertexStart and vertexCount members of VertexData which describes the subset of the buffers you're planning to use. In a simple case this is just 0 and the vertex count of your model.
:arrow: Now set up the IndexData on the SubMesh (no need to create, all SubMeshes create their own). This is simpler, just an indexStart and indexCount, and an indexBuffer which you should create through HardwareBufferManager again. The only real choice here is whether you need 32-bit indexes or not, best to use 16-bit if you can since they're more efficient.
:arrow: Populate your buffers by either using their writeData methods to copy data from system memory buffers, or write data into them directly by using their 'lock' methods. Check the way it's done elsewhere, and note useful utility functions like VertexElement::baseVertexPointerToElement which take a pointer to the start of a vertex and give you back a pointer to the start of that element. There are in fact lots of useful utility methods on VertexDeclaration and VertexElement to help you, in many ways this is much easier than it was before because you don't have to just organise the memory yourself.

Well, that ended up sounding long and complex, but having done a lot of these I can tell you it's actually very easy once you adjust your thinking! :) HTH
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

That got me going in the right direction. Some more questions:
- What's the best way to determine if a VertexData has any normals?
- How do I reference to a hardwarebuffer from a VertexData structure ?
(the buffers are created in a setupVertexData method (see: setupGeometry) and data is copied to them in a copyDataToVertexData method (see: copyDataToGeometry))
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

dennis wrote: - What's the best way to determine if a VertexData has any normals?
Like this:

Code: Select all

VertexElement* elem = vertexData->vertexDeclaration->findElementBySemantic(VES_NORMAL);
If it returns non-null then you know it has normals, and plus you have the element which defines the normals.
- How do I reference to a hardwarebuffer from a VertexData structure ?
(the buffers are created in a setupVertexData method (see: setupGeometry) and data is copied to them in a copyDataToVertexData method (see: copyDataToGeometry))
Each element has a 'source' which is an index value from 0-n where n is the number of bound vertex buffers. You supply the source when you create the element. You bind your vertex buffers using vertexData->vertexBufferBinding->setBinding. Here's a complete example taken from the Milkshape exporter which takes the simplistic approach of using a separate buffer per element:

Code: Select all

        // Set up mesh geometry
        ogreSubMesh->vertexData = new Ogre::VertexData();
        ogreSubMesh->vertexData->vertexCount = msMesh_GetVertexCount (pMesh);
        ogreSubMesh->vertexData->vertexStart = 0;
        Ogre::VertexBufferBinding* bind = ogreSubMesh->vertexData->vertexBufferBinding;
        Ogre::VertexDeclaration* decl = ogreSubMesh->vertexData->vertexDeclaration;
        // Always 1 texture layer, 2D coords
        #define POSITION_BINDING 0
        #define NORMAL_BINDING 1
        #define TEXCOORD_BINDING 2
        decl->addElement(POSITION_BINDING, 0, Ogre::VET_FLOAT3, Ogre::VES_POSITION);
        decl->addElement(NORMAL_BINDING, 0, Ogre::VET_FLOAT3, Ogre::VES_NORMAL);
        decl->addElement(TEXCOORD_BINDING, 0, Ogre::VET_FLOAT2, Ogre::VES_TEXTURE_COORDINATES);
        // Create buffers
        Ogre::HardwareVertexBufferSharedPtr pbuf = Ogre::HardwareBufferManager::getSingleton().
            createVertexBuffer(decl->getVertexSize(POSITION_BINDING), ogreSubMesh->vertexData->vertexCount, 
                Ogre::HardwareBuffer::HBU_DYNAMIC, false);
        Ogre::HardwareVertexBufferSharedPtr nbuf = Ogre::HardwareBufferManager::getSingleton().
            createVertexBuffer(decl->getVertexSize(NORMAL_BINDING), ogreSubMesh->vertexData->vertexCount, 
                Ogre::HardwareBuffer::HBU_DYNAMIC, false);
        Ogre::HardwareVertexBufferSharedPtr tbuf = Ogre::HardwareBufferManager::getSingleton().
            createVertexBuffer(decl->getVertexSize(TEXCOORD_BINDING), ogreSubMesh->vertexData->vertexCount, 
                Ogre::HardwareBuffer::HBU_DYNAMIC, false);
        bind->setBinding(POSITION_BINDING, pbuf);
        bind->setBinding(NORMAL_BINDING, nbuf);
        bind->setBinding(TEXCOORD_BINDING, tbuf);
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Yes I know this example, I've been using it as reference. In this example the buffer is created and then filled in one go. If I want to keep the structure I've created intact this is not an option. What if I want to change something in a buffer. How do you get to the vertex data once the buffer reference is lost ?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

The buffer reference is never lost - if it was the buffer would be deleted because they're reference counted. You can get a reference by calling vertexData->vertexBufferBinding->getBuffer(source);
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Here's a new one:

Code: Select all

c:\cvs\ogre\ogrenew\tools\lightwaveconverter\include\lwenvelope.h(49) : error C2065: 'm_alloc_free' : undeclared identifier
c:\cvs\ogre\ogrenew\tools\lightwaveconverter\src\main.cpp(493) : error C2872: 'm_alloc_free' : ambiguous symbol
c:\cvs\ogre\ogrenew\tools\lightwaveconverter\src\main.cpp(609) : error C2872: 'm_alloc_free' : ambiguous symbol
This only happens when compiling the debug configuration. Weird, Ogre/STL problem ?

UPDATE: shifting some includes solved this one .... but still weird.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Probably something to do with the Ogre debug memory manager - some includes don't like it (like ODE) and you have to surround them with #include "OgreNoMemoryMacros.h" and #include "OgreMemoryMacros.h".
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Almost there ...

Code: Select all

C:\CVS\Ogre\ogrenew\Tools\LightwaveConverter\bin\Debug>lwo2mesh -g -d p 0.5 4 1000.0 2000.0 4000.0 8000.0 -m -s yellowsub.lwo
Triangulating layer 0, Polygons before: 9634, Polygons after: 9634
Generating 4 lower LODs for mesh yellowsub.mesh.
MeshSerializer writing mesh data to yellowsub.mesh...
File header written.
Writing mesh data...
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Exporting LOD information....
Could not export to file: yellowsub.mesh
:? Weird, Sinbad could you check if the LOD's are there or not?
Last edited by dennis on Wed Oct 20, 2004 8:26 am, edited 1 time in total.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

I'd say not, since using the xml converter on it leads to:
An exception has been thrown!

-----------------------------------
Details:
-----------------------------------
Error #: 7
Function: MeshSerializerImpl::readMeshLodInfo
Description: Missing M_MESH_LOD_USAGE chunk in conversion.
File: e:\projects\ogrecvs\ogrenew\ogremain\src\ogremeshserializerimpl.cpp
Line: 1166
Stack unwinding: <<beginning of stack>>
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Code: Select all

C:\CVS\Ogre\ogrenew\Tools\LightwaveConverter\bin\Debug>lwo2mesh -g -d p 0.1 4 1000.0 2000.0 4000.0 8000.0 -m -s yellowsub.lwo
Triangulating layer 0, Polygons before: 9634, Polygons after: 9634
Generating 4 lower LODs for mesh yellowsub.mesh.
MeshSerializer writing mesh data to yellowsub.mesh...
File header written.
Writing mesh data...
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Writing submesh...
Submesh exported.
Exporting LOD information....
LOD information exported.
Exporting bounds information....
Bounds information exported.
Mesh data exported.
MeshSerializer export successful.
It appears that the LOD generating/exporting isn't robust enough. The generator could only generate 3 more levels but the exporter tries to export 4. It's in Mantis. Oh, and a small feature request: reduction/distance and/or poly/distance pairs would be great.

I'm going to continue with the sharedVertexData problem ...
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Ok, the LOD generator needs to report back when it can't reduce as much as has been asked. As for the feature request, yeah I'm trying to balance simplicity of use and tweakability - I may have to overload some members for people who want more advanced control over it. Not something I'll do right now though, we're just in the final stages of stabilising everything for the release.

Do you have an ETA on the converter? I'd like to include the updated version in the release if possible.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Actually, having looked at the code again, I don't understand how you've created the situation where the exporter tries to export more levels than it has. The exporter simply exports the number of LODs the mesh reports that it has, and if you use Mesh::generateLodLevels this gets set correctly. So how did you get it into the state you describe?

BTW, please put more detail on your mantis bugs in future - a one line bug report really isn't very useful.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

sinbad wrote:Ok, the LOD generator needs to report back when it can't reduce as much as has been asked. As for the feature request, yeah I'm trying to balance simplicity of use and tweakability - I may have to overload some members for people who want more advanced control over it. Not something I'll do right now though, we're just in the final stages of stabilising everything for the release.

Do you have an ETA on the converter? I'd like to include the updated version in the release if possible.
Actually this could be done VERY quickly by replacing the single reduction value with a vector of values. The old thing wouldn't have to be broken if you just copy the single reduction value into this vector for each LOD-level.

The ETA should be this evening.
sinbad wrote:Actually, having looked at the code again, I don't understand how you've created the situation where the exporter tries to export more levels than it has. The exporter simply exports the number of LODs the mesh reports that it has, and if you use Mesh::generateLodLevels this gets set correctly. So how did you get it into the state you describe?

BTW, please put more detail on your mantis bugs in future - a one line bug report really isn't very useful.

Code: Select all

		if (flags[GenerateLOD])
		{
			ProgressiveMesh::VertexReductionQuota quota;

			if (flags[UseFixedMethod])
				quota = ProgressiveMesh::VRQ_CONSTANT;
			else
				quota = ProgressiveMesh::VRQ_PROPORTIONAL;
					
			ogreMesh->generateLodLevels(distanceList, quota, reduction);
		}
		
		try
		{
			meshserializer.exportMesh(ogreMesh, fname);
		}
		catch (...)
		{
			cout << "Could not export to file: " << fname << endl;
		}
Apparently it does not get set correctly. I will try to pinpoint the exact place of error when I finish the sharedVertexData problem. If you wish, I can send you what I have NOW for examination. Including the yellow submarine. If I could be any more descriptive about this bug, I would be. If the lower LOD's could not be generated, they will be ignored completely, right? No halting reporting or breaking I hope.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Well I solved copying VertexData data by using the code from VertexData::Clone(). So I have one tiny little problem left that is probably out of my hands anyway. Converting the Yellow Sub with shared geometry AND LODs leads to an exception when loading it. :?

Code: Select all

...
17:57:37: Mesh: Loading yellowsub.mesh .
17:57:37: An exception has been thrown!

-----------------------------------
Details:
-----------------------------------
Error #: -2005530516
Function: D3D9HardwareIndexBuffer::D3D9HardwareIndexBuffer
Description: Cannot create D3D9 Index buffer: Invalid call. 
File: C:\CVS\Ogre\ogrenew\RenderSystems\Direct3D9\src\OgreD3D9HardwareIndexBuffer.cpp
Line: 51
Stack unwinding: D3D9RenderSystem::startRendering(..) <- <<beginning of stack>>
17:57:40: Render Target 'OGRE Render Window' Average FPS: 20.520912 Best FPS: 45.680241 Worst FPS: 0.224215
17:57:40: D3D9 : Shutting down cleanly.
17:57:40: *-*-* OGRE Shutdown
...
Note: Using only Shared Geometry or only LOD's or neither does work.

Sinbad, I will send you the files. If you are going to add the reduction/distance and/or poly/distance pairs before the release, I will update the converter to use them accordingly. (If the release is not before tuesday that is ...)
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

I'm busy finishing up other things for the release so I'm not going to get time to make that change you're asking for yet, we'll save it for the next one. Besides, I'd like to make the interface as helpful as possible, 2 vectors going into the method (which must be the same size) seems a little hacky to me, I'll do it properly when I have the time.

One thing your code sample omitted was how the distanceList was built up - I'll have to investigate that aspect of it asfter you send me the code.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

Ok. what I actually meant was an overloaded generateLodLevels.
User avatar
dennis
Gremlin
Posts: 157
Joined: Mon Nov 11, 2002 4:21 pm
x 3

Post by dennis »

I've been doing a lot of stuff the last couple of months and well ... I'm going to support the Lightwave Converter again, at least on a "keep it running with the current Ogre version" basis. Provided Steve keeps me informed of the changes in Ogre that probably need fixing in lwo2mesh of course.

The current changes in Ogre that were interface-breaking are:

Code: Select all

c:\cvs\ogre\ogrenew\tools\lightwaveconverter\src\lwo2mesh.cpp(34) : error C2039: 'createDeferred' : is not a member of 'MaterialManager'
        c:\cvs\ogre\ogrenew\ogremain\include\ogrematerialmanager.h(54) : see declaration of 'MaterialManager'
changed to:

Code: Select all

create();
and:

Code: Select all

c:\cvs\ogre\ogrenew\tools\lightwaveconverter\src\lwo2mesh.cpp(77) : error C2039: 'addTextureLayer' : is not a member of 'Material'
        c:\cvs\ogre\ogrenew\ogremain\include\ogrematerial.h(76) : see declaration of 'Material'
changed to:

Code: Select all

ogreMat->getTechnique(0)->getPass(0)->createTextureUnitState(texname);
And added code to set the bounding sphere and box. (Should I make a patch that adds recalcBoundingBox() and recalcBoundingSphere() to OgreMesh ?)

Catch the patch.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Originally I omitted a calculateBounds method because I wanted to make it clear that this is something that should be set by the exporter, not calculated by Ogre (becuse to calculate it requires potentially expensive locking / reading of the VBOs). However provided it's documented that it's an expensive operation and really it should be done offline then that's ok I guess; exporters might also want to consider calculating their own bounds anyway, especially for animated meshes.

As for the updates required, the simplest way to find that out is to look at the change history of the Milkshape exporter; these changes were made around 4 months ago now.