OgreMeshTool, LODs and v2 format

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
Hrenli
Halfling
Posts: 73
Joined: Tue Jun 14, 2016 12:26 pm
x 19

OgreMeshTool, LODs and v2 format

Post by Hrenli »

Hi,

I am trying to get autogenerated LODs directly to v2 .mesh file using OgreMeshTool. Everything seems to work fine if I use v1 format. If I try to generate LODs directly into v2 format I am getting this error:
LOD Generation only works on v1 meshes at the moment.
Export it as -v1, run the command again, and re-export it to -v2
If I follow the error's suggestion by running:
OgreMeshTool.exe -autogen file.mesh.xml
OgreMeshTool.exe -v2 file.mesh
LODs seems to get lost from the file during second step no matter what... Is it me doing something stupid or it's not possible to get v2 .mesh file with LODs inside at the moment?
User avatar
SolarPortal
OGRE Contributor
OGRE Contributor
Posts: 203
Joined: Sat Jul 16, 2011 8:29 pm
Location: UK
x 51
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by SolarPortal »

Hi, i know this isnt for the OgreMeshTool.exe, but i had troubles when doing it in c++ and generating a manual v2 mesh that needed lods.
The problem stemmed from not being able to set the lod values to the v2 mesh / submesh and save to disk; so i made these changes:

* Add an additional 3 commands to OgreMesh2.cpp at around line 400: ( I only used the last 2 functions )

Code: Select all

    
...   
        const LodValueArray* _getLodValueArray(void) const                      { return &mLodValues; }
        void clearLodValues() { mLodValues.clear(); }
        void setLodValue(Ogre::Real index) { mLodValues.push_back(index); }
...
* Compile and then run in your app with this sort of function:

Code: Select all

void setupLodValues(int numLods, Ogre::MeshPtr mesh)
{
	Ogre::Real distances[5] = { 0, 20, 40, 50, 60 };
	mesh->clearLodValues();
	for (int i = 0; i < numLods; i++){
		mesh->setLodValue(distances[i]);
	}
}
As long as you add the lod vao to the submesh with something similar to this:

Code: Select all

	
	...	
	Ogre::VertexArrayObject* lodvao = vaoManager->createVertexArrayObject(vertexBuffers, indexBuffer, Ogre::OT_TRIANGLE_LIST);
	subMesh->mVao[Ogre::VpNormal].push_back( lodvao );
	subMesh->mVao[Ogre::VpShadow].push_back( lodvao );
	...
then you can add your own lod meshes and it will save to the v2 mesh format and can be read back from the file.

Hope this helps :)
Lead developer of the Skyline Game Engine: https://aurasoft-skyline.co.uk
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

I should probably add this to my TODO list. I didn't know it was broken.

Thanks for posting the workaround :)
User avatar
SolarPortal
OGRE Contributor
OGRE Contributor
Posts: 203
Joined: Sat Jul 16, 2011 8:29 pm
Location: UK
x 51
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by SolarPortal »

no problem :)
Lead developer of the Skyline Game Engine: https://aurasoft-skyline.co.uk
Hrenli
Halfling
Posts: 73
Joined: Tue Jun 14, 2016 12:26 pm
x 19

Re: OgreMeshTool, LODs and v2 format

Post by Hrenli »

Thanks SolarPortal, I played with loading of different files a bit more and you are right. It seems that mLodValues is the only thing which is really missing. mNumLods is always 1 nowdays for v2 meshes but it seems not to be used at all. And mVao-s for submeshes seem to be loading correctly already with LODs.
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

What's the status of this?
I mean, is it possible to generate LODs for a v1 mesh (using ogremeshtool or ogremeshlod library), import it in a v2 mesh and then have LODs working in the created item?

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

Honestly I don't remember (probably not fixed)
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: OgreMeshTool, LODs and v2 format

Post by Zonder »

dark_sylinc wrote: Fri May 22, 2020 12:38 am Honestly I don't remember (probably not fixed)
Not quite clear on this. Was it broken and you put a fix in for it?
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

I don't remember taking a look at any of the LOD bugs mentioned in this post, hence it's likely that it's not fixed.
But I can't be sure.
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

The good news: the LOD system works even for V2 meshes!. The bad news: the strategies based on pixels count (in my opinion, the most useful ones) are fucked as hell (and you will also experience a crash trying them).

Let's start from the good news.

Code: Select all

LodStrategyManager::getSingleton().setDefaultStrategy(DistanceLodSphereStrategy::getSingletonPtr());

v1::MeshPtr mesh_v1 = v1::MeshManager::getSingleton().load("Sinbad.mesh", ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

LodConfig config(mesh_v1, LodStrategyManager::getSingleton().getDefaultStrategy());
config.createGeneratedLodLevel(10.0f, 0.25f, LodLevel::VRM_PROPORTIONAL);
config.createGeneratedLodLevel(20.0f, 0.50f, LodLevel::VRM_PROPORTIONAL);
config.createGeneratedLodLevel(30.0f, 0.75f, LodLevel::VRM_PROPORTIONAL);

MeshLodGenerator generator;
generator.generateLodLevels(config);

MeshPtr mesh_v2 = MeshManager::getSingleton().createManual("Sinbad_lod.mesh", ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
mesh_v2->importV1(mesh_v1.get(), false, false, true);

Item* item = m_sceneManager->createItem(mesh_v2, SCENE_STATIC);

SceneNode* node = m_sceneManager->getRootSceneNode()->createChildSceneNode(SCENE_STATIC);
node->attachObject(item);
The code here above generates three LOD levels for the v1 Sinbad.mesh:
  • at 10.0 units distance, reduce vertices count by 25%
  • at 20.0 units distance, reduce vertices count by 50%
  • at 30.0 units distance, reduce vertices count by 75%
The v1 mesh is then imported into a v2 mesh and an Item is created from it. As you can see from the images here below, OGRE correctly picks a different LOD level based on the distance of the camera from the mesh.

Image Image Image Image

Let's move on to the broken things.

First thing first, inside the implementations of the PixelCountLodStrategyBase::lodUpdateImpl, a nullptr is returned when you take the Viewport object from the camera. This happens because the viewport is binded to the camera when CompositorPass::setRenderPassDescToCurrent is invoked, but this happens AFTER the LODs evaluation step (call to sceneManager->updateAllLods in CompositorPassScene::execute). Matias, could you please fix it? It's quite impossible to create a pixels count based strategy without viewport infos.

Secondarily, pixels count based strategies cannot work as the LOD picking code works right now. In LodStrategy::lodSet which LOD to use is chosen by a std::lower_bound algorithm which, to work as intended, needs an ascending ordered vector. This is true for the distance based strategies, while it's not for the pixels count based ones. Indeed, in distance based strategies the distance increases while the camera steps away from the object, while the contrary happens in pixels count based strategies: the pixels count increases while the camera approaches the object. I guess that this discrepancy between the two family of strategies was taken in account in OGRE 1.x (see here and here), but the code to handle it has been lost/not ported in OGRE 2.x (MeshLodUsageList is commented out in v2 Mesh class).
I see two viable solutions. The first is to subtract pixels count from the total of pixels (e.g. instead of "when objects cover x pixels, reduce vertices count by y%" do "when objects doesn't cover total - x pixels, reduce vertices count by y%"). The second is to negate the pixels count value. In particular, I believe the latter is the solution that OGRE pursues, but it doesn't work as it is right now. This because MovableObject::mLodMesh always contains a 0.0 value as first element (the LOD value of the original mesh). Probably, sort MovableObject::mLodMesh here and here, when the LOD has been generated for a pixels count based strategy, would solve the issue (but I don't know how much is safe to do this).

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

Nice detective work!

It's clear that "artist-friendly" values should be exposed to the user, but internally we should convert those values back and forth to whatever is more efficient for us.
First thing first, inside the implementations of the PixelCountLodStrategyBase::lodUpdateImpl, a nullptr is returned when you take the Viewport object from the camera. This happens because the viewport is binded to the camera when CompositorPass::setRenderPassDescToCurrent is invoked, but this happens AFTER the LODs evaluation step (call to sceneManager->updateAllLods in CompositorPassScene::execute). Matias, could you please fix it? It's quite impossible to create a pixels count based strategy without viewport infos.
Order-of-execution bugs strike again!
We'll have to do some workaround, because we can't call setRenderPassDescToCurrent() before doing executing the shadow node (well we can... but it'd cost performance), and we need the LODs calculated before running the shadow node. Chicken and egg problem.
Fortunately all the LOD calculations care about is viewport sizes, and that is something we can provide.

I still advocate we use negative values for LOD sorting. It's just that as you pointed out, instead of using 0 for LOD0, we use a large negative value. We need to ensure a 0 doesn't slip through to LOD0 when using Pixel count strategies.
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

dark_sylinc wrote: Thu May 28, 2020 5:46 pm Order-of-execution bugs strike again!
We'll have to do some workaround, because we can't call setRenderPassDescToCurrent() before doing executing the shadow node (well we can... but it'd cost performance), and we need the LODs calculated before running the shadow node. Chicken and egg problem.
Fortunately all the LOD calculations care about is viewport sizes, and that is something we can provide.
Is it something you'll be able to fix soon? I'd like to move the "LOD" post-it in the "Done" column :D
dark_sylinc wrote: Thu May 28, 2020 5:46 pm I still advocate we use negative values for LOD sorting. It's just that as you pointed out, instead of using 0 for LOD0, we use a large negative value. We need to ensure a 0 doesn't slip through to LOD0 when using Pixel count strategies.
My fault this time. OGRE already set the correct value for LOD0 based on the default strategy in use (see here). It's enough to set the default strategy before any mesh is being created and choose LOD "distances" accordingly to the strategy behaviour in the LodConfig object (positive values for distance based strategies, negative values for pixel counts based strategies).
dark_sylinc wrote: Thu May 28, 2020 5:46 pm It's clear that "artist-friendly" values should be exposed to the user, but internally we should convert those values back and forth to whatever is more efficient for us.
You're right, I could provide to artists a front-end that shows LOD distances in a human-readable form, but the problem is I cannot understand the math which is behind the pixels count based strategies and so I'm unable to do the conversion back and forth. E.g. the ScreenRatioPixelCountLodStrategy should returns value in the following range [-k, 0.0) (where k is greater-equal to 1 when the projected bounding sphere covers all the screen), but it's not true.
So, I decided to redo the math from scratch (see here; n.b. I choose to didn't take in account what they call "sphere horizon" and so I replaced sqrt(d^2 - r^2) with d).

Radius of the bounding sphere in screen space is:
Image

This leads us to the area of the bounding sphere which in screen space is:
Image

Resolving the exponent and taking some shortcuts (cot(fovy / 2) is equal to 1 / tan(fovy / 2) which is equal to projMatrix[1][1]) leads us to:
Image

Divide all this stuff per the total pixels count (viewport_width * viewport_height) gives us the screen coverage ratio:
Image

Which is quite different from the formula used by ScreenRatioPixelCountLodStrategy:
Image

I know that LodStrategy::lodUpdateImpl is a sensitive place (the fact that SIMD operations are in play warns me about it!), but my formula consists just in two multiplications and a division more. I don't know how much these additional operations can affect performances, but the outputs are much more consistent respect to the ones returned by ScreenRatioPixelCountLodStrategy.


EDIT: viewport_height / viewport_width can be replaced by 1 / aspectRatio which is an already computed value in Camera object. Doing so, one of the two additional multiplactions vanish.

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

Your new formula is going to stay. I just tested it. It's SUPER accurate. WAY more accurate than the current one.

I took ShadowMapDebugging sample, removed all cubes but one. Left only one cube and the plane.

I get close, and align the camera so the cube is roughly 1/4th of the screen. The value of lodValues[1]? Around -0.25

I get really far. The value of lodValues[1] is -0.005341
I take a picture, load it in GIMP. I cut a tight rectangle around the cube and copy paste it as its own layer. GIMP says the layer is 80x79 in resolution. The window resolution is 1600x900. What's (80*79) / (1600*900) ? 0.004388889

Then I go really far away, looking from above, the plane is roughly fitting the screen (it can't be a perfect screen because my aspect ratio is 16:9 and the plane is squared). What's lodValues[0]? -0.978748083114624

Damn! It's a very good approximation. This change is getting implemented.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by dark_sylinc »

Your formulas were pushed.

Let me know if you spot an error.

I'm thinking ScreenRatioPixelCountLodStrategy should become the default, now that it's more accurate and it is far more intuitive to work with. Thoughts?
paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by paroj »

TaaTT4 wrote: Fri May 29, 2020 2:18 pm Divide all this stuff per the total pixels count (viewport_width * viewport_height) gives us the screen coverage ratio:
Image

Which is quite different from the formula used by ScreenRatioPixelCountLodStrategy:
Image

EDIT: viewport_height / viewport_width can be replaced by 1 / aspectRatio which is an already computed value in Camera object. Doing so, one of the two additional multiplactions vanish.

Code: Select all

proj[0][0] = proj[1][1]/aspect
which reduces the difference between the two formulas to the factor 1/4
paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by paroj »

TaaTT4 wrote: Fri May 29, 2020 2:18 pm Radius of the bounding sphere in screen space is:
Image
the factor 1/2 is superficial. This resolves the 1/4 in the result.
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

paroj wrote: Sat May 30, 2020 1:39 am

Code: Select all

proj[0][0] = proj[1][1]/aspect
which reduces the difference between the two formulas to the factor 1/4
Well spotted! I hadn't considered this further optimization.
paroj wrote: Sat May 30, 2020 12:36 pm
TaaTT4 wrote: Fri May 29, 2020 2:18 pm Radius of the bounding sphere in screen space is:
Image
the factor 1/2 is superficial. This resolves the 1/4 in the result.
Why are you saying so? If you take out the denominator from the rightmost fraction (viewport_height / 2), don't you get to have the original formula used in ScreenRatioPixelCountLodStrategy at the end (which is clearly not so precise and unconsistent)?

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

paroj
OGRE Team Member
OGRE Team Member
Posts: 1994
Joined: Sun Mar 30, 2014 2:51 pm
x 1074
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by paroj »

TaaTT4 wrote: Sat May 30, 2020 1:36 pm
paroj wrote: Sat May 30, 2020 12:36 pm
TaaTT4 wrote: Fri May 29, 2020 2:18 pm Radius of the bounding sphere in screen space is:
Image
the factor 1/2 is superficial. This resolves the 1/4 in the result.
Why are you saying so? If you take out the denominator from the rightmost fraction (viewport_height / 2), don't you get to have the original formula used in ScreenRatioPixelCountLodStrategy at the end (which is clearly not so precise and unconsistent)?
it doesnt currently make sense to me why it should be viewport_height / 2 instead of viewport_height. I am currently writing a test to find out which is actually correct.

edit: the 1/4 is correct because the NDC height is 2
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

paroj wrote: Sat May 30, 2020 2:09 pm it doesnt currently make sense to me why it should be viewport_height / 2 instead of viewport_height.
I believe it's because the formula takes in account just the vertical semi-frustum (fovy / 2). You probably get rid of the viewport_height division by 2, if you use the entire fovy.

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: OgreMeshTool, LODs and v2 format

Post by TaaTT4 »

dark_sylinc wrote: Fri May 29, 2020 9:55 pm Let me know if you spot an error.
Had a look at the math and it seems all correct to me. I also compared the values returned by ScreenRatioPixelCountLodStrategy with the ones returned by the strategy I made to evaluate my formula and they are the same. Just FYI, I didn't checked the AbsolutePixelCountLodStrategy nor the cases when the camera is of PT_ORTHOGRAPHIC type.
dark_sylinc wrote: Fri May 29, 2020 9:55 pm I'm thinking ScreenRatioPixelCountLodStrategy should become the default, now that it's more accurate and it is far more intuitive to work with. Thoughts?
I'm totally agree with this. I consider ScreenRatioPixelCountLodStrategy the most intuitive and flexible LOD strategy. Also commercial engines (see here and here) follow the same approach.
But to be consistent with this choice there's a last thing to fix in MeshLodGenerator::getAutoconfig. First of all, the default strategy must be changed even here. Secondarily, MeshLodGenerator::getAutoconfig automatically finds plausible values for LOD distance and reduction values. While the reduction value is computed in relation to mesh bounding sphere radius, the distance value is computed starting from a magic number that represents some strange pixels count value (see here). Looking at this (especially the "Automatic mesh Lod management system" paragraph), if I've correctly understood the idea behind the algorithm, that magic number must be replaced with -1.0 (which is the ideal LOD0 value for which the mesh covers all the screen).

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: OgreMeshTool, LODs and v2 format

Post by rujialiu »

TaaTT4 wrote: Wed Jun 03, 2020 12:12 pm I'm totally agree with this.
I Agree!
Post Reply