MaterialManager Memory Hogging (Pass Graveyard)

[FlorianGeorge]

The following minimum repro sample appears to cause Ogre to reserve more memory than it should:

Code: Select all

    Ogre::Root* root = new Ogre::Root();

    Ogre::D3D9Plugin* d3d9plugin = new Ogre::D3D9Plugin();
  	root->installPlugin(d3d9plugin);

    Ogre::RenderSystem* rsys = root->getRenderSystemByName("Direct3D9 Rendering Subsystem");
    root->setRenderSystem(rsys);

    root->initialise(false);

    Ogre::NameValuePairList windowOptions;
    windowOptions["show"] = "false";

    root->createRenderWindow(
      "TestWindow",
      640,
      480,
      false, 
      &windowOptions);

    std::cout << "TextureTest" << std::endl;

    for (int i = 0; i < 100000; i++)
    {
      Ogre::TexturePtr tex = Ogre::TextureManager::getSingleton().create("testText", "General");
      Ogre::TextureManager::getSingleton().remove("testText");
    }

    std::cout << "MaterialTest" << std::endl;

    for (int i = 0; i < 100000; i++)
    {
      Ogre::MaterialPtr mat = Ogre::MaterialManager::getSingleton().create("testMat", "General");
      Ogre::MaterialManager::getSingleton().remove("testMat");
    }

    std::cout << "End of Test" << std::endl;

    delete root;

On my system (Windows 7 SP1 x64), up until createRenderWindow, the application uses ~5MB Memory, after createRenderWindow is uses ~20MB.
During "TextureTest", the Memory Footprint doesn't change significantly.
During "MaterialTest", the Memory Footprint increases up to ~75MB, and would continue to do so indefinitely if the loop wouldn't stop after 100k iterations. This is only released after deleting Ogre::Root.

I reproduced this behavior under Ogre 1.8.1, as well as the current up-to-date revision of Ogre 1.9 in the repository:

Changeset: 4790 (a01cca8e4f96) Bugfix commit 9fb67da (https://bitbucket.org/sinbad/ogre/commi ... daf1a24047
Branch: v1-9
User: Transporter
Date: 2013-05-29 00:47:05 +0200 (16 hours)
Parent: 4788 (fba4e499e38e) [GLES2] Add a runtime extension check to prevent calls to null GL function point
Tags: tip

Any reason why this is happening, a bug or wrong usage of Ogre on my side?

[bstone]

Try replacing

Code: Select all

Ogre::MaterialManager::getSingleton().remove("testMat");

with

Code: Select all

Ogre::MaterialManager::getSingleton().remove( mat );

[FlorianGeorge]

Try replacing

Code: Select all

Ogre::MaterialManager::getSingleton().remove("testMat");

with

Code: Select all

Ogre::MaterialManager::getSingleton().remove( mat );

A MaterialPtr will not be implicitly converted to a ResourcePtr, at least using Visual Studio 2008 SP1. But you can convert explicitly.

All three versions

Code: Select all

Ogre::MaterialManager::getSingleton().remove("testMat");

Code: Select all

Ogre::MaterialManager::getSingleton().remove(mat->getHandle());

Code: Select all

Ogre::ResourcePtr res = mat;
Ogre::MaterialManager::getSingleton().remove(res);

have the same behavior in my TestCase.

[bstone]

Good to know. At least that excludes the problem with finding the resource by name.

[FlorianGeorge]

It appears that the Pass Graveyard is the culprit for the memory growth behavior.

When a new Material is created, in my code with

Code: Select all

      Ogre::MaterialPtr mat = Ogre::MaterialManager::getSingleton().create("testMat", "General");

the constructor of Ogre::Material will call Ogre::Material::applyDefaults, which will create a default Material with one Technique and one Pass.

When this Material is removed, in my code e.g. with

Code: Select all

    Ogre::MaterialManager::getSingleton().remove("testMat");

the destructor of Ogre::Material will call Ogre::Material::removeAllTechniques, which will lead to Ogre::Technique::removeAllPasses, which will lead to Ogre::Pass::queueForDeletion.

There, the ever growing Graveyard is created:

OgrePass.h:

Code: Select all

        /// The place where passes go to die
        static PassSet msPassGraveyard;

OgrePass.cpp:

Code: Select all

		{
			OGRE_LOCK_MUTEX(msPassGraveyardMutex)
			msPassGraveyard.insert(this);
		}

After 100k Materials being created and removed, msPassGraveyard will contain 100k dead Passes, all of them happily sitting on the memory they are still allocating. This Graveyard will only be cleaned up by Ogre::Pass::processPendingPassUpdates. I found three places in the Ogre code where Ogre::Pass::processPendingPassUpdates is called, Ogre::Root::~Root, Ogre::RenderQueue::clear and Ogre::RenderQueue::~RenderQueue. Calling Ogre::Root::renderOneFrame will not call it, and I haven't seen it being called (in my small example) after running Ogre::Root::startRendering neither. This means in my code, and possibly other developer's code as well, the Graveyard is never cleared while running, only at shutdown.

So currently, a simple solution for me is to call Ogre::Pass::processPendingPassUpdates manually whenever I feel like the Graveyard should be cleaned up. Would this be the "officially" suggested behavior? Or should there be a mechanism that will call it also when e.g. Ogre::Root::renderOneFrame is called, if the Graveyard size exceeds a certain adjustable limit (or always if the limit is set to zero)?

[bstone]

That's really interesting. Thanks for your investigation. I doubt that's the official way to clean up memory consumed by materials.

[Kojack]

I found three places in the Ogre code where Ogre::Pass::processPendingPassUpdates is called, Ogre::Root::~Root, Ogre::RenderQueue::clear and Ogre::RenderQueue::~RenderQueue. Calling Ogre::Root::renderOneFrame will not call it, and I haven't seen it being called (in my small example) after running Ogre::Root::startRendering neither. This means in my code, and possibly other developer's code as well, the Graveyard is never cleared while running, only at shutdown.

I just stuck a breakpoint in it, Pass::processPendingPassUpdates is called every frame via Ogre::RenderQueue::clear. The call chain:
Root::renderOneFrame
Root::_updateAllRenderTargets
RenderSystem::_updateAllRenderTargets
RenderTarget::update
RenderTarget::updateImpl
RenderTarget::_updateAutoUpdatedViewports
RenderTarget::update
Viewport::update
Camera::_renderScene
SceneManager::_renderScene
SceneManager::prepareRenderQueue
RenderQueue::clear
Pass::processPendingPassUpdates

(This is in a version of ogre 1.9 from a few months ago)