[SOLVED] [2.0] Upgrading from 1.9

[rpgplayerrobin]

Hello!

I am in the final process of upgrading my game (Spellheart) from Ogre 1.9 to Ogre 2.0.
Upgrading it to Ogre 2.1 would be too much work, and would take months to do (since all my specific custom shaders has to be upgraded, and fixed function materials etc).
So I figured that I could at least upgrade to Ogre 2.0 to get some performance boost.

I will post all my notes on how to upgrade from Ogre 1.9 to Ogre 2.0 when these last two questions are answered (I want to be able to write how I solved these questions as well).

As seen on the "What version to choose?" page (http://www.ogre3d.org/about/what-version-to-choose) upgrading from Ogre 1.9 to Ogre 2.0 seems to go two "stable" versions towards the future.
There it also says that Ogre 2.0 should have a better performance than Ogre 1.9 ("Fast" vs "Ok").

To my surprise, the performance does not seem to be better for my game at all, and I am wondering if I am doing something wrong.

Here are some FPS statistics with my game (not really scientific, but at least something):
(I had to use an image, tabs and whitespaces did not work correctly with normal text)



Currently the FPS is almost exactly the same in 1.9 and 2.0 for my application as you can see above.

Question #1:
Is there anything I should think of that would increase my FPS more in Ogre 2.0?
Altering the parameters to the "createSceneManager" does not seem to give me any FPS differences at all.
Also, if I upgrade all my shaders/materials to Direct3D 11 (since Ogre 2.0 supports that?), will the FPS be increased?


Question #2:
Memory seems to be acting a bit wierd in Ogre 2.0.
When just looking on a scene in the sample browser, the memory goes up by around 4 kb per second.
When looking away from the scene, the memory goes up by more than 8 kb per second.

But after a while, the memory just stops going up.
This strange memory seem to happen only when using "m_Root->renderOneFrame".

If you edit the shadows sample to never cast shadows, and to only use the floor and remove all lights and the penguin, this memory increase stops happening.
As soon as you add the first light (the directional light for example), this memory increase happens again (it should not be because of shadows, as they are disabled).

This also happens in my main application, and after waiting for 38 minutes in a very simple scene, 21 mb had been added and then the memory stopped going up.
When the memory stops going up, there is no FPS increase in case you are wondering.

Same increase in memory happens in Process Explorer (separate application, so it does not just happen in the Task Manager).
I tried this in Ogre 1.9 and it does not happen there.

I have no idea what this strange memory behaviour is, but I am glad that it finally stops after going up for such a long time (38 minutes in my application).

Do anyone have any idea what this memory increase is, and if I can somehow stop it from happening?

[Zonder]

The improvements were only CPU side (I think :S) you might be GPU bound so the new version can't help you. Maybe check your GPU status.

[paroj]

As seen on the "What version to choose?" page (http://www.ogre3d.org/about/what-version-to-choose) upgrading from Ogre 1.9 to Ogre 2.0 seems to go two "stable" versions towards the future.
There it also says that Ogre 2.0 should have a better performance than Ogre 1.9 ("Fast" vs "Ok").

yeah, the hyper fast/ mega fast/ ultra fast part is a little confusing. updated that to be more specific.

[dark_sylinc]

Hi, I can be specific to clear doubts:

There are four main reasons a 1.x Ogre app would be running "slow":

1. Frustum culling, scene graph traversal (aka many scene nodes / many Entity / many InstancedEntity). Ogre was very slow at this; and this is the area that got a major performance upgrade (between 3x-4x). Many applications were bottlenecked by this, thus it's common to see performance improvements by just moving to Ogre 2.0.

2. Postprocessing in Ogre 1.x was very wasteful because there was some RTT management behind your back. While migrating to Ogre 2.0 won't automatically make it faster; you can manage it yourself which means you can avoid any possible inefficiency. See the Postprocess sample in Ogre 2.0

3. RenderSystem API overhead. Ogre 1.x was particularly bad at this, and so is 2.0. This problem got fixed in Ogre 2.1. Nonetheless, you can solve this problem by using InstanceManager (see "New Instancing" sample from Ogre 1.x & 2.0) which requires adapting your code to use instancing (both C++ & shaders). The InstanceManager became obsolete in 2.1 though.

4. Your application is GPU bottlenecked. You're submitting too many vertices, your custom shaders are very fat or unoptimized, you're submitting very few vertices per draw, you have lots of triangles that are are very tiny (occupy one pixel or so), your textures are huge, you lack mipmaps, you're drawing back to front, you're using alpha testing on everything. Lots of potential reasons. This won't be fixed by Ogre, your scene needs to be analized and optimize what's bottlenecking you (use meshes with less vertices, optimize shaders, use more aggressive LODs, reduce the resolution of the texture, don't use features you don't use like turning on alpha testing on everything even if you don't need it)

First you need to identify your bottleneck is. Considering you said "would be too much work, and would take months to do (since all my specific custom shaders has to be upgraded, and fixed function materials etc)" I'm suspecting you don't use instancing and you do use custom shaders, which would mean your problem is either problem # 3. or 4. But profiling will tell.
Your memory usage sounds off though. Ogre 2.0 got MUCH better at memory efficiency per Entity/InstancedEntity and per Node (should consume 20-40% of the original consumption); so your comparison of 644mb vs 665mb doesn't sound right at all; unless most of that is your mem. consumption and not Ogre's.

Also, if I upgrade all my shaders/materials to Direct3D 11 (since Ogre 2.0 supports that?), will the FPS be increased?

No. The best RenderSystem in 1.x and 2.0 era is Direct3D 9. In fact Direct3D11 support is quite bad and only recommended to those who require it (e.g. because they're targeting Windows RT, Windows Phone, UWP, you want texture array support, etc).
It was in Ogre 2.1 that D3D11 became really good and stable (and also D3D9 was removed), alongside GL3+.

[dark_sylinc]

Do anyone have any idea what this memory increase is, and if I can somehow stop it from happening?

Looking at Process Explorer/taskmgr's memory consumption is a rough estimate, but not an indicative of exact memory consumption.

Memory increasing by a few bytes or kb and then stopping (or even getting suddely lower then slowly back to rising again) is indicative of the OS doing internal work when allocating/deallocating small chunks of memory and fragmenting/defragmenting blocks of memory. As long as it doesn't rise forever (which would indicate a leak) this is harmless; and the OS will defragment and reclaim memory shall another process need it.
Though I do admit ideally Ogre shouldn't do any per-frame allocation, it's hard to keep up with a 16-year-old engine. I do kill off any per-frame allocation whenever I detect one; and I've fixed quite a few during these years of Ogre 2.1 development, but I'm not keeping track.

Also please mind that this does not only come to Ogre alone, but also could come from any your code or third party libraries.

[rpgplayerrobin]

Thank you for your answers!

The improvements were only CPU side (I think :S) you might be GPU bound so the new version can't help you. Maybe check your GPU status.

This is very likely the issue here, that I am too much GPU bound that the CPU improvements does not seem to matter at all.

2. Postprocessing in Ogre 1.x was very wasteful because there was some RTT management behind your back. While migrating to Ogre 2.0 won't automatically make it faster; you can manage it yourself which means you can avoid any possible inefficiency. See the Postprocess sample in Ogre 2.0

I actually almost copy pasted the compositors from that sample, so I am only using those two textures for an arbitrary amount of post processing effects.
Even with that, they are slower than the 1.9 version (by 2 FPS only though).

4. Your application is GPU bottlenecked. You're submitting too many vertices, your custom shaders are very fat or unoptimized, you're submitting very few vertices per draw, you have lots of triangles that are are very tiny (occupy one pixel or so), your textures are huge, you lack mipmaps, you're drawing back to front, you're using alpha testing on everything. Lots of potential reasons. This won't be fixed by Ogre, your scene needs to be analized and optimize what's bottlenecking you (use meshes with less vertices, optimize shaders, use more aggressive LODs, reduce the resolution of the texture, don't use features you don't use like turning on alpha testing on everything even if you don't need it)

This must be the issue here. Otherwise I would notice the CPU boost from 2.0.
I am using alpha testing on some specific materials, but walls and such never use anything like that.
Most of my vegitation are using alpha rejection though (is that the same as alpha testing?), but they do not use any kind of "scene_blend x" in the materials.

Your memory usage sounds off though. Ogre 2.0 got MUCH better at memory efficiency per Entity/InstancedEntity and per Node (should consume 20-40% of the original consumption); so your comparison of 644mb vs 665mb doesn't sound right at all; unless most of that is your mem. consumption and not Ogre's.

My memory other than Ogres memory looks exactly the same when I upgraded, it might have something to do with the compositor system though, since the post processing compositor (copied from the sample) creates two textures for other post processing effects. So I am not very worried about this part.

No. The best RenderSystem in 1.x and 2.0 era is Direct3D 9. In fact Direct3D11 support is quite bad and only recommended to those who require it (e.g. because they're targeting Windows RT, Windows Phone, UWP, you want texture array support, etc).
It was in Ogre 2.1 that D3D11 became really good and stable (and also D3D9 was removed), alongside GL3+.

Good to know! Then I will not even try to do that hehe.

Also please mind that this does not only come to Ogre alone, but also could come from any your code or third party libraries.

I also tried this memory increase in the sample browser, and the same thing happened there:

If you edit the shadows sample to never cast shadows, and to only use the floor and remove all lights and the penguin, this memory increase stops happening.
As soon as you add the first light (the directional light for example), this memory increase happens again (it should not be because of shadows, as they are disabled).

[Zonder]

Do not use process explorer for exact measure of memory allocation., Windows doesn't remove the memory attached to the process because it might need it and it's slow to do. It removes it when another app requests.

[rpgplayerrobin]

Do not use process explorer for exact measure of memory allocation., Windows doesn't remove the memory attached to the process because it might need it and it's slow to do. It removes it when another app requests.

Good to know.

As my questions have been answered I now feel that I am ready to release my notes on how to upgrade to Ogre 2.0 from Ogre 1.9.
Should I do that in a new post or should I add it here (its a huge amount of text on how to fix things)?

[rpgplayerrobin]

As promised I have now made a new post about how I successfully upgraded to Ogre 2.0.
I hope that might help other people.

http://www.ogre3d.org/forums/viewtopic.php?f=25&t=92538