Upgrading from 1.11.2 to 1.12.13 Topic is solved

Problems building or running the engine, queries about how to use features etc.
rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

HBL_DISCARD can be used here https://github.com/OGRECave/ogre/blob/v ... .cpp#L2007

For the objects being rendered with software vertex animation, it always becomes HBL_DISCARD in that function when I debug it.

There is no GPU>CPU transfer here (shadow buffer should be used): https://github.com/OGRECave/ogre/blob/1 ... .cpp#L1977

I don't know exactly what that means, but when I debug the srcPosLock variable, it shows: "mShadowBuffer = empty" and "mUsage = 5".

on exit from the function, you should end up here: https://github.com/OGRECave/ogre/blob/1 ... r.cpp#L223 and the whole buffer should be discarded

It calls D3D11HardwareBuffer::_updateFromShadow 3 times when the softwareVertexBlend ends, and all calls contain a non-empty mShadowBuffer.
The first call it comes into this if-case here:

Code: Select all

if(mShadowBuffer && mShadowUpdated && !mSuppressHardwareUpdate)
{
	bool discardWholeBuffer = mLockStart == 0 && mLockSize == mSizeInBytes;
	copyDataImpl(*mShadowBuffer, mLockStart, mLockStart, mLockSize, discardWholeBuffer);
	mShadowUpdated = false;
}

The second call it does not come into a single if-case, so nothing happens in the function.
The third call it comes into this if-case here:

Code: Select all

if (mUseTempStagingBuffer)
{
	// delete
	mShadowBuffer.reset();
	mUseTempStagingBuffer = false;
}

I also tried to replace this whole function (Mesh::softwareVertexBlend) with the one from 1.11.2, but it had no effect on the FPS at all.

paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Upgrading from 1.11.2 to 1.12.13

Post by paroj »

I don't know exactly what that means, but when I debug the srcPosLock variable, it shows: "mShadowBuffer = empty" and "mUsage = 5".

this is likely the problem. We are reading from an unshadowed GPU_ONLY buffer here.

try commenting out

Code: Select all

mShadowBuffer.reset();

that you mentioned above and see whether it improves performance

rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

This is the new code (only commented out that one line):

Code: Select all

void D3D11HardwareBuffer::_updateFromShadow(void)
{
	if(mShadowBuffer && mShadowUpdated && !mSuppressHardwareUpdate)
	{
		bool discardWholeBuffer = mLockStart == 0 && mLockSize == mSizeInBytes;
		copyDataImpl(*mShadowBuffer, mLockStart, mLockStart, mLockSize, discardWholeBuffer);
		mShadowUpdated = false;
	}

if (mUseTempStagingBuffer)
{
	// delete
	//mShadowBuffer.reset();
	mUseTempStagingBuffer = false;
}
}

It seems to be working pretty good now! :D

However, for InstancingHW, it became worse performance again on Direct3D11.
So I tried changing back the mShadowBuffer and it became normal performance again, but then the software vertex animation performance issue comes back of course, so it has to be fixed some other way.
So I instead tried with HBU_DYNAMIC_WRITE_ONLY in InstanceBatchHW like we were talking about before, and it went up to the expected FPS (which was poison for D3D11 before, so that is pretty strange, but I am happy).
Direct3D9 was not affected by this change from what I could see in the FPS, so this change works.

These are my results for many different tests with exactly the same settings in options (pretty high settings):

Normal game scene 1.11.2 -> 1.12.13:
D3D11 with GPU animation ON: 235 -> 241
D3D11 with GPU animation OFF: 195 -> 205
D3D9 with GPU animation ON: 247 -> 247
D3D9 with GPU animation OFF: 222 -> 202

Instancing stress test 1.11.2 -> 1.12.13:
D3D11: 72 -> 75
D3D9: 99 -> 97

Grass stress test 1.11.2 -> 1.12.13:
D3D11: 55 -> 56
D3D9: 60 -> 63

Particle stress test 1.11.2 -> 1.12.13:
D3D11: 133 -> 133
D3D9: 125 -> 138

So the results show quite clearly it is almost exactly the same regarding all tests compared to my older 1.11.2 version.
Now I have no more things I need to upgrade from the old version from what I can see, I only need to test the new stuff in this version (more specifically, multithreaded loading of HLSL shaders), which might create a new thread in the future.

So I will mark this whole post as solved now.
Thank you for all your help! :mrgreen:

Code: Select all

while(true)
	ShowMessage("Thank you!")
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Upgrading from 1.11.2 to 1.12.13

Post by paroj »

this is interesting as for me (NVidia) commenting this line makes no difference.

Note that commenting this line adds a shadow buffer to all HBU_GPU_ONLY buffers. You might want to rather add

Code: Select all

if(options == HardwareBuffer::HBL_READ_ONLY)

here https://github.com/OGRECave/ogre/blob/3 ... r.cpp#L147

so only buffers that are read-back get that.

edit: even better - use https://ogrecave.github.io/ogre/api/lat ... d17ef60daa to only add shadow buffers to skeletal animated meshes.

rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

this is interesting as for me (NVidia) commenting this line makes no difference.

I am also using Nvidia. This line made a huge difference (but only if we are talking about software vertex animation of course).

Note that commenting this line adds a shadow buffer to all HBU_GPU_ONLY buffers. You might want to rather add

if(options == HardwareBuffer::HBL_READ_ONLY)
here https://github.com/OGRECave/ogre/blob/3 ... r.cpp#L147

so only buffers that are read-back get that.

edit: even better - use https://ogrecave.github.io/ogre/api/lat ... d17ef60daa to only add shadow buffers to skeletal animated meshes.

It seems to work pretty fine for sure.
I use setVertexBufferPolicy on animated/skeletal meshes before they are loaded and I am now again using "mShadowBuffer.reset();".

I have not attempted all tests, but it seems to work as well as before (possibly even better now with instancing since I can remove the HBU_STATIC_WRITE_ONLY -> HBU_DYNAMIC_WRITE_ONLY in OgreInstanceBatchHW.cpp).

However, where do I actually put the setVertexBufferPolicy code correctly?
Doing it in user code is extremely ugly since at the time of a mesh not being loaded, it is impossible to know if it is animated or not (at resourceLoadStarted for example).
That means I need to keep a list of names of meshes that has animations or skeletons, and then use that code every time I load a mesh through a resource manager, "dynamic" resource manager (no "load" called) or manual loading, which is a big pain to have to do.

Do you know a location I can place it so it does this automatically for all animated/skeletal meshes?

rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

MeshSerializerImpl::readSubMesh (which in turns calls MeshSerializerImpl::readGeometryVertexBuffer) loads the vertex buffer with mVertexBufferUsage/mVertexBufferShadowBuffer and seems to be used before MeshSerializerImpl::readSkeletonLink or MeshSerializerImpl::readAnimations.

That means it is impossible for a mesh that has not yet been loaded to know if it is animated or has a skeleton, and as soon as it is loaded it is impossible to change it to use the shadow buffer. It is a catch-22.

Does that mean that it makes this an impossible task, and that I have to keep a list of mesh names instead?
This makes it impossible to handle dynamically loaded meshes from other locations...

rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

I did not understand your other solution:

Note that commenting this line adds a shadow buffer to all HBU_GPU_ONLY buffers. You might want to rather add

CODE: SELECT ALL

if(options == HardwareBuffer::HBL_READ_ONLY)
here https://github.com/OGRECave/ogre/blob/3 ... r.cpp#L147

You mean the code should look like this?:

Code: Select all

if(options == HardwareBuffer::HBL_READ_ONLY)
     mUseTempStagingBuffer = true;

Instead of this?:

Code: Select all

mUseTempStagingBuffer = true;
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Upgrading from 1.11.2 to 1.12.13

Post by paroj »

Does that mean that it makes this an impossible task, and that I have to keep a list of mesh names instead?

yes. I thought that you might have more context in your game besides the name. But if not, that is a list of mesh names then.
If we could determine that before loading the buffers, we could solve this within Ogre. Maybe we could update the mesh writer to store any skeleton links in front of the buffers without breaking the current file format.

You mean the code should look like this?:

Code: Select all

if(options == HardwareBuffer::HBL_READ_ONLY)
     mUseTempStagingBuffer = true;

Instead of this?:

Code: Select all

mUseTempStagingBuffer = true;

yes, but I messed up the contion. It shoud rather be

Code: Select all

if(options != HardwareBuffer::HBL_READ_ONLY)
     mUseTempStagingBuffer = true;
rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

yes. I thought that you might have more context in your game besides the name. But if not, that is a list of mesh names then.

If each mesh has a .skeleton file with the same name, it is rather easy of course.
But mesh names can be different from skeleton files, and that makes it impossible.

Some meshes also have vertex animation without a skeleton. I am pretty sure those are affected by this performance issue as well, so they are also impossible to fix for without a list of mesh names.

Also, user-imported meshes are very hard to fix it for (as people can import stuff in the map editor with little difficulty).

But with the below code you showed me, these "issues" are completely irrelevant anyway.

if(options != HardwareBuffer::HBL_READ_ONLY)
mUseTempStagingBuffer = true;

This works very well and the performance is as good as before from what I can see.
I also had to alter my custom manual object class to use a shadow buffer (on createIndexBuffer and createVertexBuffer) to be able to get optimal performance with grass in Direct3D11. That might have to also be done in Ogre::ManualObject, but as I don't use it I won't do that change in my source.

Though I don't really know what the shadow buffer actually does, it increases performance. From the comments in the source it seems it is a buffer in the CPU that is updating towards the GPU when it feels like it, which I don't really know what it means. Maybe updates are slower because of that, but I don't see it affecting my scene in any way with the grass.

So thank you again! The performance issues are gone. :D

paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Upgrading from 1.11.2 to 1.12.13

Post by paroj »

rpgplayerrobin wrote: Sun May 15, 2022 2:40 pm

Though I don't really know what the shadow buffer actually does, it increases performance.

see https://ogrecave.github.io/ogre/api/lat ... ow-Buffers

rpgplayerrobin
Gnoll
Posts: 617
Joined: Wed Mar 18, 2009 3:03 am
x 353

Re: Upgrading from 1.11.2 to 1.12.13

Post by rpgplayerrobin »

So it means that a shadow buffer is only good for when you need to read stuff in the CPU in user code?
Strange in that case, since I never read data from my manual object class (only write with HBU_GPU_ONLY), so the performance should then be worse with a shadow buffer, but instead it is much faster.
Shouldn't it be the same with animated meshes? Because I never read their vertex buffers or anything like that in the CPU.

paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Upgrading from 1.11.2 to 1.12.13

Post by paroj »

no shadow buffers also ensure that buffer updates are write combined. I updated the docs for this.

Post Reply