ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle Topic is solved

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Lax
Orc Shaman
Posts: 701
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 73

ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by Lax »

Hi dark_sylinc,

I’m running into a deterministic crash in ParticleFX2 that appears to be an engine-side bookkeeping bug, not script misuse. After extensive isolation and testing, this can be reproduced with valid and relatively simple particle definitions.

Observed crash

The crash consistently happens in:

Code: Select all

Ogre::ParticleSystemDef::deallocParticle()

Code: Select all

Call stack (debug):

FastArray<unsigned __int64>::operator[]()
bitset64::findLastBitSetPlusOne()
ParticleSystemDef::deallocParticle()
ParticleSystemManager2::updateSerialPos()
ParticleSystemManager2::update()

At the time of crash:

Code: Select all

bitset64 internalArraySize = 3
mValues.size() = 2

This indicates a capacity mismatch between bitset64 and its internal FastArray, leading to an out-of-bounds access.

Key observations:

  • Happens after some runtime, not immediately
  • Happens in debug and release
  • Reproduced with:
    • single emitter
    • multiple emitters
    • ColourInterpolator (no ColourFader)
    • generous quota (180+)
  • Still occurs even with constant emission rate and TTL
  • Script complexity does not matter once runtime is long enough

This rules out script misuse, quota exhaustion, or affector misuse.

Root cause (analysis)

The issue appears to be triggered by non-FIFO particle deallocation, which can happen naturally when:

  • multiple particles expire in the same frame
  • frame delta fluctuates slightly
  • TTL-based death occurs in batches

In ParticleSystemDef::deallocParticle(), this code path is reached:

Code: Select all

mLastParticleIdx =
    static_cast<uint32>(
        mActiveParticles.findLastBitSetPlusOne( mLastParticleIdx )
    );

However, at this point:

  • mLastParticleIdx can be greater than the bitset capacity
  • findLastBitSetPlusOne(startFrom) then assumes more storage blocks than actually allocated
  • This results in bitset64 believing it has more blocks than mValues actually contains

Hence:

Code: Select all

internalArraySize > mValues.size()

-> out-of-bounds read -> crash

This is pure engine-side state drift, not something a particle script can prevent.

Minimal safe fix (suggested)

Clamping mLastParticleIdx to the bitset capacity before calling findLastBitSetPlusOne() fully resolves the crash in my tests.

Example fix:

Code: Select all

if( mActiveParticles.empty() )
{
    mFirstParticleIdx = 0;
    mLastParticleIdx  = 0;
    return;
}

const uint32 cap = static_cast<uint32>( mActiveParticles.capacity() );

if( mLastParticleIdx > cap )
    mLastParticleIdx = cap;

const uint32 safeStart = std::min( mLastParticleIdx, cap );

mLastParticleIdx =
    static_cast<uint32>(
        mActiveParticles.findLastBitSetPlusOne( safeStart )
    );

This:

  • preserves behavior
  • prevents invalid memory access
  • has no measurable performance impact
  • aligns with how wraparound should be handled safely

Conclusion

ParticleFX2 currently assumes FIFO-ish deallocation, but its own code explicitly supports non-FIFO paths — those paths are where the bookkeeping breaks.

This makes long-running particle systems inherently unstable without engine fixes.

Thanks for your time, and for Ogre-Next in general — this is a great system, just one sharp edge that’s easy to miss.

Best regards
Lax

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5551
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1402

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by dark_sylinc »

  1. Does this happen in Single Threaded? Or is Multi Threading required?
  2. What were the values of mLastParticleIdx, mLastParticleIdx and quota at the time of crash?
  3. Can you provide an MRP? (Minimum Reproduction Program)

Looking at the code involved, I'm almost certain the fix is:

Code: Select all

mLastParticleIdx =
	static_cast<uint32>( mActiveParticles.findLastBitSetPlusOne( lastParticleIdxWrapped );
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5551
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1402

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by dark_sylinc »

Upon further examination I am certain that was the problem.

Fixed. Thanks for the report!

Lax
Orc Shaman
Posts: 701
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 73

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by Lax »

Hi,

ok now the crash is gone.
I have another request. For ParticleEmitter is protected:

Code: Select all

class _OgreExport EmitterDefData : protected ParticleEmitter

That is really bad. I use particle system template files. I clone a particle and want to control more via c++ code. For example i want to fadeout an particle if its at its end. But thats not possible, because of the protected ParticleEmitter restriction, so I'm not able to change Emitter data.

Could this be made public?

I show an example, why i need some api:

Code: Select all

if (false == particleManager->hasParticleSystemDef(this->finalParticleDefName))
{
				Ogre::ParticleSystemDef* clonedParticleSystemDef = baseDef->clone(this->finalParticleDefName, particleManager);

			if (nullptr == clonedParticleSystemDef)
			{
				Ogre::LogManager::getSingletonPtr()->logMessage(Ogre::LML_CRITICAL, "[ParticleFxComponent] Clone failed for: " + this->finalParticleDefName);
				success = false;
				return;
			}

			const Ogre::Vector2 scale = this->particleScale->getVector2();
			const Ogre::Real uniformScale = (scale.x + scale.y) * 0.5f;

			for (Ogre::EmitterDefData* emitter : clonedParticleSystemDef->getEmitters())
			{
				// Scale particle dimensions
				Ogre::Vector2 dim = emitter->getInitialDimensions();
				dim.x *= scale.x;
				dim.y *= scale.y;
				emitter->setInitialDimensions(dim);

				// Scale emitter position to fix offset when scaling down
				auto emitterPos = emitter->getPosition();
				emitterPos.x *= uniformScale;
				emitterPos.y *= uniformScale;
				emitterPos.z *= uniformScale;
				emitter->setPosition(emitterPos);

				// Scale velocity range
				Ogre::Real minVel = emitter->getMinParticleVelocity();
				Ogre::Real maxVel = emitter->getMaxParticleVelocity();
				emitter->setParticleVelocity(minVel * uniformScale, maxVel * uniformScale);
			}

			finalParticleSystemDef = clonedParticleSystemDef;
}

I have for example also scaling issues. So many particles are to huge. node->setScale(...), does not affect particles. I tested it. So i clone a particle for my manipulation and also change setParticleVelocity etc.

Best Regards
Lax

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5551
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1402

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by dark_sylinc »

Use asParticleEmitter().

Note that as the documentation says, some functionality may not behave as expected or documented, or may not be available.

Code: Select all

        /// ParticleEmitter is a protected base class of EmitterDefData.
        ///
        /// This is because the new system was designed to be backwards compatible
        /// as much as possible with old the system, however it may not map 1:1.
        /// Thus in order to access the old interface, one must cast explicitly.
        ParticleEmitter *asParticleEmitter() { return this; }
Lax
Orc Shaman
Posts: 701
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 73

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by Lax »

Hi dark_sylinc,

I now fully implemented ParticleFX features. I struggle with delete Ogre::Root. I get always a crash. I analyzed it with the help of several LLM's. I first thought, the bug is on my side. But then we analyzed Ogre code and it came out, that something is wrong on Ogre site. I post the summary:

Summary:
Crash occurs in ParticleSystemManager::_destroyRenderer() during Root::~Root() when using cloned ParticleSystemDef objects. The renderer factory map iterator returns corrupted/uninitialized memory (0xCCCCCCCCCCCCCCCC pattern in Visual Studio debug builds).

Ogre-Next Version:
Latest master branch (as of January 2025)

Platform:
Windows 10/11, Visual Studio 2022, Debug build

Steps to Reproduce:

  1. Create a ParticleSystemDef from a template

  2. Clone it for dynamic particle creation:

    Code: Select all

    Ogre::ParticleSystemDef* baseDef = particleManager->getParticleSystemDef("MyTemplate");
    Ogre::ParticleSystemDef* clone = baseDef->clone("MyClone_" + uniqueId, particleManager);
  3. Create ParticleSystem2 instances from the clone:

    Code: Select all

    Ogre::ParticleSystem2* ps = sceneManager->createParticleSystem2("MyClone_" + uniqueId);
  4. Destroy the ParticleSystem2 instances properly:

    Code: Select all

    const Ogre::ParticleSystemDef* def = ps->getParticleSystemDef();
    Ogre::ParticleSystemDef* mutableDef = const_cast<Ogre::ParticleSystemDef*>(def);
    mutableDef->_destroyParticleSystem(ps);
  5. Destroy SceneManager before Root

  6. Delete Root

Expected Result:
Clean shutdown with no crashes.

Actual Result:
Crash in ParticleSystemManager::_destroyRenderer() with call stack:

Code: Select all

OgreMain_d.dll!Ogre::ParticleSystemManager::_destroyRenderer(renderer=0x...)
OgreMain_d.dll!Ogre::ParticleSystem::~ParticleSystem()
OgreMain_d.dll!Ogre::ParticleSystemDef::~ParticleSystemDef()
OgreMain_d.dll!Ogre::ParticleSystemManager2::~ParticleSystemManager2()
OgreMain_d.dll!Ogre::SceneManager::~SceneManager()
OgreMain_d.dll!Ogre::Root::~Root()

Debug Analysis:

In Visual Studio debugger at crash point:

ParticleSystemManager state:

Code: Select all

mRendererFactories: { size=1 }
["billboard"]: 0x0000023a39bb4b80 {valid BillboardParticleRendererFactory pointer}
mutex: locked

Crash location variables:

Code: Select all

void ParticleSystemManager::_destroyRenderer(ParticleSystemRenderer* renderer)
{
    ParticleSystemRendererFactoryMap::iterator pFact = 
        mRendererFactories.find(renderer->getType());
    
    // pFact->_Ptr = 0xCCCCCCCCCCCCCCCC  ← UNINITIALIZED/CORRUPTED!
    
    if (pFact == mRendererFactories.end())  // This throws/crashes
}

The iterator returned by find() has garbage memory (0xCCCC pattern), even though the map itself contains valid data.

Root Cause Analysis:

The issue appears to be a destruction order problem in Root::~Root():

Code: Select all

Root::~Root()
{
    shutdown();  // ← Destroys SceneManagers (including ParticleSystemManager2)
                 //   ParticleSystemDef destructors run HERE
                 //   They try to destroy v1 ParticleSystem objects
                 //   which need to destroy renderers
    
    // ... many other cleanups ...
    
    delete mParticleSystemManager;  // ← v1 ParticleSystemManager destroyed
    
    // ... more cleanups ...
    
    unloadPlugins();  // ← Plugin_ParticleFX2 DLL unloaded
}

When ParticleSystemDef::~ParticleSystemDef() runs (during shutdown()), it contains v1 ParticleSystem objects that have mRenderer pointers. These destructors call ParticleSystemManager::_destroyRenderer(), but at this point:

  1. The ParticleSystemManager's mutex is in a locked/inconsistent state
  2. The map's internal structure is corrupted (iterator returns 0xCCCC)
  3. The plugin might be in the process of being unloaded

Why the Ogre samples don't crash:

The official ParticleFX2 sample doesn't use cloning:

Code: Select all

// Sample just creates systems directly:
Ogre::ParticleSystem2* ps = sceneManager->createParticleSystem2("Examples/PurpleFountain");

// And relies on automatic cleanup
void destroySystems(...)
{
    delete graphicsSystem;  // Everything cleaned up internally
}

They don't manually destroy ParticleSystemDef clones or create complex destruction scenarios.

Proposed Fix:

Option 1: In ParticleSystem::~ParticleSystem(), add safety check:

Code: Select all

ParticleSystem::~ParticleSystem()
{
    // ... existing cleanup ...
    
    if (mRenderer)
    {
        // Check if ParticleSystemManager singleton still exists
        if (ParticleSystemManager::getSingletonPtr())
        {
            try
            {
                ParticleSystemManager::getSingleton()._destroyRenderer(mRenderer);
            }
            catch (...)
            {
                // Manager is shutting down, just null the pointer
                // OS will clean up renderer memory on process exit
            }
        }
        mRenderer = 0;
    }
}

Option 2: Change Root destruction order to destroy SceneManagers LAST:

Code: Select all

Root::~Root()
{
    // ... all other cleanup ...
    
    delete mParticleSystemManager;  // v1 manager first
    unloadPlugins();                // then plugins
    
    shutdown();  // SceneManagers LAST (this destroys ParticleSystemManager2)
}

Option 3: Have ParticleSystemDef destructor NOT destroy v1 ParticleSystems if ParticleSystemManager is shutting down.

Workaround for Users:

Currently, the only reliable workaround is to skip Root destruction entirely:

Code: Select all

Core::~Core()
{
    // Clean up MyGUI, etc.
    // ...
    
    // Skip Root deletion to avoid crash
    // OGRE_DELETE this->root;
    this->root = nullptr;
    
    // OS will clean up all memory on process exit
}

This is acceptable for game applications where clean shutdown isn't critical, but not ideal for tools/editors that need proper cleanup.

Additional Notes:

  • This only affects applications that dynamically create and destroy cloned ParticleSystemDef objects
  • The crash is reproducible 100% of the time when using clones
  • Static particle systems (created once at startup) don't trigger the issue
  • The mutex being locked during destruction suggests potential thread safety issues as well

Question for dark_sylinc:

Is the mixing of v1 (ParticleSystem/ParticleSystemManager) and v2 (ParticleSystemDef/ParticleSystemManager2) particle systems intentional? It seems like ParticleSystemDef internally creates v1 ParticleSystem objects which hold renderer pointers, creating a complex destruction dependency that's difficult to manage during shutdown.

Would it be better to either:

  1. Keep v1 and v2 particle systems completely separate
  2. Or ensure ParticleSystemDef doesn't use v1 ParticleSystem internally
  3. Or add proper shutdown ordering to handle this intertwined destruction

Best Regards
Lax

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5551
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1402

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by dark_sylinc »

Is the mixing of v1 (ParticleSystem/ParticleSystemManager) and v2 (ParticleSystemDef/ParticleSystemManager2) particle systems intentional?

No, it's not. At least not here. ParticleSystemRenderer should NOT be getting used for v2 particles. A quick glance at the code looks like ParticleSystem::setRenderer got accidentally called. Find out how/why and post its callstack when it happens.

The most likely culprit is ParticleSystem::operator=. In ParticleSystem() constructor, bCreateRenderer must be false for v2 particles.

Lax
Orc Shaman
Posts: 701
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 73

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by Lax »

Ah thanks for the clue and yes i think its called by accident. Look at this stack trace:

Code: Select all

 	OgreMain_d.dll!Ogre::ParticleSystem::setRenderer(const std::string & rendererName={...}) Zeile 1069	C++
 	OgreMain_d.dll!Ogre::ParticleSystem::CmdRenderer::doSet(void * target=0x000001ea2546e6b0, const std::string & val={...}) Zeile 1563	C++
 	OgreMain_d.dll!Ogre::StringInterface::setParameter(const std::string & name={...}, const std::string & value={...}) Zeile 85	C++
 	OgreMain_d.dll!Ogre::ParticleSystemTranslator2::translate(Ogre::ScriptCompiler * compiler=0x000001ea62b70c10, const Ogre::SharedPtr<Ogre::AbstractNode> & node={...}) Zeile 6420	C++
 	OgreMain_d.dll!Ogre::ScriptCompiler::compile(const Ogre::SharedPtr<std::list<Ogre::SharedPtr<Ogre::ConcreteNode>,std::allocator<Ogre::SharedPtr<Ogre::ConcreteNode>>>> & nodes={...}, const std::string & group={...}) Zeile 353	C++
 	OgreMain_d.dll!Ogre::ScriptCompiler::compile(const std::string & str={...}, const std::string & source={...}, const std::string & group={...}) Zeile 259	C++
 	OgreMain_d.dll!Ogre::ScriptCompilerManager::parseScript(Ogre::SharedPtr<Ogre::DataStream> & stream={...}, const std::string & groupName={...}) Zeile 1811	C++
 	OgreMain_d.dll!Ogre::ResourceGroupManager::parseResourceGroupScripts(Ogre::ResourceGroupManager::ResourceGroup * grp=0x000001ea7a31b9b0) Zeile 1134	C++
 	OgreMain_d.dll!Ogre::ResourceGroupManager::initialiseAllResourceGroups(bool changeLocaleTemporarily=true) Zeile 177	C++

The grp="ParticleFX2"
rendererName="billboard"

It comes via the scriptCompiler...

cfg:

Code: Select all

[ParticleFX2]
FileSystem=../../media/ParticleFX2

Here of course i have some sample particles, like in the ogre samples.

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5551
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1402

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by dark_sylinc »

Thanks!

Pushed a fix. Now it will issue a warning instead of causing a crash.

Lax
Orc Shaman
Posts: 701
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 73

Re: ParticleFX2: deterministic crash in ParticleSystemDef::deallocParticle

Post by Lax »

Youre welcome,

and maybe you could add here this little fix:

in order that Ocean does not crash at the beginning, if there is no terra. In OgreHlmsPbs:

Code: Select all

uint32 HlmsPbs::fillBuffersFor( const HlmsCache *cache, const QueuedRenderable &queuedRenderable,
                                bool casterPass, uint32 lastCacheHash, CommandBuffer *commandBuffer,
                                bool isV1 )
{
    assert( dynamic_cast<const HlmsPbsDatablock *>( queuedRenderable.renderable->getDatablock() ) );
    const HlmsPbsDatablock *datablock =
        static_cast<const HlmsPbsDatablock *>( queuedRenderable.renderable->getDatablock() );

if( OGRE_EXTRACT_HLMS_TYPE_FROM_CACHE_HASH( lastCacheHash ) != mType )
{
    if( mCurrentPassBuffer == 0 )
        return 0;

...
}

This is necessary:

if( mCurrentPassBuffer == 0 )
return 0;

Else i get early crashes using Ocean, because at specific point of time mCurrentPassBuffer is 0 and then is populated. So that guard will add savity.

Best Regards
Lax

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62