[3.0.0] Incorrect datablocks being applied to objects Topic is solved

Problems building or running the engine, queries about how to use features etc.
User avatar
haloman30
Kobold
Posts: 27
Joined: Mon Aug 29, 2022 2:53 pm
x 2

[3.0.0] Incorrect datablocks being applied to objects

Post by haloman30 »

Ogre Version: 3.0.0
Operating System: Windows 7 x64
Render System: Direct3D11

Hello!

Apologies for posting another help topic so soon after another one - but I'm running into another weird issue with datablocks.

I'm unsure if this is some kind of limitation on the amount of datablocks that can exist or what, but the issue I'm running into is that, when generating a large amount of unlit datablocks at runtime, it appears that objects end up using the wrong ones.

I've done some basic testing to ensure that datablocks aren't actually just being reused, and they seem not to be. The amount of datablocks in question is a bit shy of 1,000. I do intend to make some optimizations to drastically reduce this number, but looking back I think I've run into this once or twice before with much fewer datablocks. As such, if this is some kind of hard limit of Ogre, or if there's something I'm just doing wrong - I'd like to know either way. Having searched around, it seems as though Ogre should be able to support several times as many datablocks as I'm using currently - which is what makes me thing that there's something I'm doing wrong somehow.

I've attached a couple screenshots - one showing what things generally should look like, and another showing what happens if this large amount of datablocks are created.

Some other details that I can think of which could be relevant:

  • The datablocks in question are currently generated per-primitive.
  • Each primitive (both 2D and 3D) are their own ManualObjects, with their own datablocks to allow for per-object color/transparency (and yes I know that this is pretty inefficient - I've already got a few ways I can think of to slim this down heavily)
  • The ID string for each primitive includes a UUID string. I also tried using a random uint64 converted to a string instead, in case that the UUIDs were faulty and duplicating, but this appears to not be the case.
  • I've tried keeping track of all datablocks generated in this way and checking to ensure that the newly created datablock addresses do not match any of the ones created previously - and it appears that all datablocks are in fact unique
  • Through checking values at runtime, I've confirmed for certain that in fact multiple objects are using the same datablock. Even creating a quick and dirty ImGUI window to edit the colors of the datablocks shows it reflecting on both the intended and the unintended objects simultaneously (this can be seen in the second screenshot)

Intended Look:
Image

Bugged Look:
Image

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5433
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1341

Re: [3.0.0] Incorrect datablocks being applied to objects

Post by dark_sylinc »

Hi!

Are you running in Debug mode? Because by default we use 32-bit hashes for datablock names, which can cause collisions (they usually appear at around 10.000 datablocks but it can happen sooner).

Debug mode will assert if collisions appear, but Release mode will just ignore them and cause bugs like these.

If collisions are indeed the issue, you can build OgreNext with CMake option OGRE_IDSTRING_USE_128 turned on to quickly fix it. This will make OgreNext to use 128-bit hashes instead of 32.

If this is not the problem, then I'll ask you if you can create a small repro by modifying one of the samples and upload the diff here so I can take a look. Thanks!

User avatar
haloman30
Kobold
Posts: 27
Joined: Mon Aug 29, 2022 2:53 pm
x 2

Re: [3.0.0] Incorrect datablocks being applied to objects

Post by haloman30 »

dark_sylinc wrote: Tue Jun 25, 2024 3:21 pm

Hi!

Are you running in Debug mode? Because by default we use 32-bit hashes for datablock names, which can cause collisions (they usually appear at around 10.000 datablocks but it can happen sooner).

Debug mode will assert if collisions appear, but Release mode will just ignore them and cause bugs like these.

If collisions are indeed the issue, you can build OgreNext with CMake option OGRE_IDSTRING_USE_128 turned on to quickly fix it. This will make OgreNext to use 128-bit hashes instead of 32.

If this is not the problem, then I'll ask you if you can create a small repro by modifying one of the samples and upload the diff here so I can take a look. Thanks!

Hey!

Yes, I was running in debug mode - and I had already preemptively enabled the 128-bit ID string hashes, as it just made sense to me to have it (and by extension my own engine) able to handle more materials and such. I never ran into any assertions, either.

In either case, I was able to modify the decals sample (as that's the one I happened to be tinkering with to begin with before), and simply generating 900 datablocks results in the same issue happening. The only file I changed was DecalsGameState.cpp, however I did also need to add the Unlit HLMS includes to the Visual Studio project settings.

At the top of the file, I added these includes and 3 plain functions:

Code: Select all


#include "OgreHlmsUnlitDatablock.h"
#include "OgreManualObject2.h"

#include <random>

std::string random_int_string()
{
    std::random_device random_device;
    std::mt19937_64 generator( random_device() );

std::uniform_int_distribution<uint64_t> distribution( std::numeric_limits<std::uint64_t>::min(),
                                                      std::numeric_limits<std::uint64_t>::max() );

return std::to_string( distribution( generator ) );
}

Ogre::HlmsUnlitDatablock *create_test_unlit_datablock( Demo::GraphicsSystem *mGraphicsSystem )
{
    Ogre::HlmsManager *hlmsManager = mGraphicsSystem->getRoot()->getHlmsManager();
    Ogre::Hlms *hlmsUnlit = hlmsManager->getHlms( Ogre::HLMS_UNLIT );

Ogre::HlmsMacroblock macroblock = Ogre::HlmsMacroblock();
Ogre::HlmsBlendblock blendblock = Ogre::HlmsBlendblock();
blendblock.setBlendType( Ogre::SceneBlendType::SBT_TRANSPARENT_ALPHA );
Ogre::HlmsParamVec paramvec = Ogre::HlmsParamVec();

std::string datablock_id = random_int_string();

Ogre::HlmsUnlitDatablock *new_datablock = (Ogre::HlmsUnlitDatablock *)hlmsUnlit->createDatablock(
    datablock_id, datablock_id, macroblock, blendblock, paramvec, true );

new_datablock->setUseColour( true );
new_datablock->setAlphaTest( Ogre::CompareFunction::CMPF_EQUAL );

return new_datablock;
}

void unlit_testing(Demo::GraphicsSystem* mGraphicsSystem)
{
    for (int i = 0; i < 900; i++)
    {
        create_test_unlit_datablock( mGraphicsSystem );
    }

Ogre::SceneManager *sceneManager = mGraphicsSystem->getSceneManager();
Ogre::ManualObject *manual_object = sceneManager->createManualObject();

Ogre::HlmsUnlitDatablock *datablock = create_test_unlit_datablock( mGraphicsSystem );
datablock->setColour(Ogre::ColourValue(1.0, 0.0, 0.0, 1));


{
    manual_object->begin( *datablock->getNameStr(), Ogre::OperationType::OT_TRIANGLE_LIST );

    manual_object->position( -1.0f, 1.0f, 1.0f );
    manual_object->position( 1.0f, 1.0f, 1.0f );
    manual_object->position( -1.0f, 1.0f, -1.0f );
    manual_object->position( 1.0f, 1.0f, -1.0f );
    manual_object->position( -1.0f, -1.0f, 1.0f );
    manual_object->position( 1.0f, -1.0f, 1.0f );
    manual_object->position( -1.0f, -1.0f, -1.0f );
    manual_object->position( 1.0f, -1.0f, -1.0f );

    manual_object->triangle( 0, 1, 2 );
    manual_object->triangle( 1, 3, 2 );
    manual_object->triangle( 4, 6, 5 );
    manual_object->triangle( 5, 6, 7 );
    manual_object->triangle( 0, 2, 4 );
    manual_object->triangle( 4, 2, 6 );
    manual_object->triangle( 1, 5, 3 );
    manual_object->triangle( 5, 7, 3 );
    manual_object->triangle( 0, 4, 1 );
    manual_object->triangle( 4, 5, 1 );
    manual_object->triangle( 2, 3, 6 );
    manual_object->triangle( 6, 3, 7 );

    manual_object->end();
}

Ogre::SceneNode *sceneNode = sceneManager->createSceneNode(Ogre::SCENE_DYNAMIC );
sceneNode->attachObject( manual_object );
sceneNode->setPosition( 0, 20, 0 );
}

The only other change made to the file was calling unlit_testing near the end of DecalsGameState::createScene01, right before the call to TutorialGameState::createScene01();:

Code: Select all

void DecalsGameState::createScene01()
{
    // ...

Ogre::uint32 visibilityMask = mGraphicsSystem->getSceneManager()->getVisibilityMask();
visibilityMask &= ~0x00000002u;
visibilityMask &= ~0x00000001u;
mGraphicsSystem->getSceneManager()->setVisibilityMask( visibilityMask );

unlit_testing( mGraphicsSystem );

TutorialGameState::createScene01();
}

Later

There will be a new unlit cube above the main area (a bit high up), generated using a ManualObject in the unlit_testing function.

By default, the cube will appear white - when it should instead be red. Commenting out the for loop at the start of unlit_testing will skip generating the 900 datablocks, and the cube will be red. Haven't directly checked memory or anything like that, but seems to be pretty clearly the same issue.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5433
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1341

Re: [3.0.0] Incorrect datablocks being applied to objects

Post by dark_sylinc »

HOLY!

Such an easy repro. Thanks for the repro. I will look into it!!!

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5433
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1341

Re: [3.0.0] Incorrect datablocks being applied to objects

Post by dark_sylinc »

Phew!

This bug was laughably easy to fix, thankfully!

It was introduced in v2.3.

Fixed.

Thank you for the report!

Update: Added a test to prevent regressing on this.

User avatar
haloman30
Kobold
Posts: 27
Joined: Mon Aug 29, 2022 2:53 pm
x 2

Re: [3.0.0] Incorrect datablocks being applied to objects

Post by haloman30 »

dark_sylinc wrote: Wed Jun 26, 2024 3:40 pm

Phew!

This bug was laughably easy to fix, thankfully!

It was introduced in v2.3.

Fixed.

Thank you for the report!

Update: Added a test to prevent regressing on this.

Wow nice, literally just one line to fix it - glad to have been accidentally helpful!