Ogre-Next preload some textures

[Lax]

Hi @dark_sylinc,

I would like to pre-load some necessary textures, at engine startup. E.g. for ParticleUniverse, because first time particle is played, it takes some time, the texture is shown and its sprite.

How is that possible?

Would be something like this possible:

Code: Select all

void Core::preLoadTextures(void)
{
	std::vector<Ogre::String> filters = { "png", "jpg", "bmp", "tga", "gif", "tif", "dds" };

std::set<Ogre::String> textureNames;

Ogre::TextureGpuManager* textureManager = Ogre::Root::getSingleton().getRenderSystem()->getTextureGpuManager();

for (auto& resourceGroupName : this->resourceGroupNames)
{
	if ("Vehicles" != resourceGroupName)
	{
		continue;
	}

	// Ogre::StringVector extensions = Ogre::Codec::getExtensions();
	// for (Ogre::StringVector::iterator itExt = extensions.begin(); itExt != extensions.end(); ++itExt)
	for (auto& filter : filters)
	{
		Ogre::StringVectorPtr names = Ogre::ResourceGroupManager::getSingletonPtr()->findResourceNames(resourceGroupName, "*." + filter/**itExt*/);
		for (Ogre::StringVector::iterator itName = names->begin(); itName != names->end(); ++itName)
		{
			Ogre::String textureName = *itName;
			// Idea was: If textures e.g. from vehicles are pre-loaded in simulation chaning a texturename at runtime is faster, but that was not the case :(
			Ogre::Image2 image;
			image.load(textureName, resourceGroupName);

			Ogre::uint32 textureFlags = Ogre::TextureFlags::AutomaticBatching;

				textureFlags &= ~Ogre::TextureFlags::AutomaticBatching;
				textureFlags |= Ogre::TextureFlags::RenderToTexture;
				textureFlags |= Ogre::TextureFlags::AllowAutomipmaps;

			Ogre::uint32 filters = Ogre::TextureFilter::FilterTypes::TypeGenerateDefaultMipmaps;

			// Really important: createOrRetrieveTexture when its created, its width/height is 0 etc. so the texture is just prepared
			// it will be filled with correct data when setDataBlock is called
			Ogre::TextureGpu* texture = textureManager->createOrRetrieveTexture(textureName,
				Ogre::GpuPageOutStrategy::SaveToSystemRam, textureFlags, Ogre::TextureTypes::Type2D,
				resourceGroupName, filters, 0u);

			texture->setResolution(image.getWidth(), image.getHeight());
			texture->setNumMipmaps(PixelFormatGpuUtils::getMaxMipmapCount(image.getWidth(), image.getHeight()));

			// Check if its a valid texture
			if (nullptr != texture)
			{
				// Really important: GpuResidency must be set to 'OnSystemRam', else cloning a datablock does not work!
				if (texture->getResidencyStatus() != Ogre::GpuResidency::OnSystemRam)
				{
					try
					{
						texture->scheduleTransitionTo(Ogre::GpuResidency::Resident);

						bool canUseSynchronousUpload = texture->getNextResidencyStatus() == Ogre::GpuResidency::Resident && texture->isDataReady();

						if (false == canUseSynchronousUpload)
						{
							texture->waitForData();
						}
					}
					catch (const Ogre::Exception& exception)
					{

					}
				}
			}
		}
	}
}
}

I'm testing it with the resource group "Vehicles", because I want to change a diffiuse texture name for a datablock at runtime. E.g. setting a different car color texture.

But now, If I use the preLoadTextures() and change a texture. It will not work anymore. The texture does look weird.

Best Regards
Lax

[dark_sylinc]

You're loading the textures as RenderToTexture and without AutomaticBatching.

That sounds strange. If you really wanted that, when removing AutomaticBatching make the type Texture2DArray explicitly, otherwise it will work in Unlit but it won't in Pbs.

Also you have HlmsDatablock::preload if you want to do it per material.

I don't know what you're trying to do here:

Code: Select all

bool canUseSynchronousUpload = texture->getNextResidencyStatus() == Ogre::GpuResidency::Resident && texture->isDataReady();

if (false == canUseSynchronousUpload)
{
	texture->waitForData();
}

Since the value should return always false. It's better to issue all the load requests and then wait for all of them with waitForStreamingCompletion.

Code: Select all

// Really important: GpuResidency must be set to 'OnSystemRam', else cloning a datablock does not work!

This shouldn't be right. Is there a bug in OgreNext I'm missing?

[Lax]

Hi,

thanks for you useful advices.
By accident I worked with:

Code: Select all

Ogre::uint32 textureFlags = Ogre::TextureFlags::AutomaticBatching;

			textureFlags &= ~Ogre::TextureFlags::AutomaticBatching;
			textureFlags |= Ogre::TextureFlags::RenderToTexture;
			textureFlags |= Ogre::TextureFlags::AllowAutomipmaps;

But they were just for the reflection mode:

Code: Select all

if (Ogre::PBSM_REFLECTION == pbsTextureType)
{
				textureFlags |= ~Ogre::TextureFlags::AutomaticBatching;
				textureFlags |= Ogre::TextureFlags::RenderToTexture;
				textureFlags |= Ogre::TextureFlags::AllowAutomipmaps;
				internalTextureType = Ogre::TextureTypes::TypeCube;
}

So I added the wrong peace of code

Now I load like this for a specific folder like "Vehicles" folder:

Code: Select all

void Core::preLoadTextures(void)
{
	std::vector<Ogre::String> filters = { "png", "jpg", "bmp", "tga", "gif", "tif", "dds" };

std::set<Ogre::String> textureNames;

Ogre::TextureGpuManager* textureManager = Ogre::Root::getSingleton().getRenderSystem()->getTextureGpuManager();

for (auto& resourceGroupName : this->resourceGroupNames)
{
	if ("Vehicles" != resourceGroupName)
	{
		continue;
	}

	// Ogre::StringVector extensions = Ogre::Codec::getExtensions();
	// for (Ogre::StringVector::iterator itExt = extensions.begin(); itExt != extensions.end(); ++itExt)
	for (auto& filter : filters)
	{
		Ogre::StringVectorPtr names = Ogre::ResourceGroupManager::getSingletonPtr()->findResourceNames(resourceGroupName, "*." + filter/**itExt*/);
		for (Ogre::StringVector::iterator itName = names->begin(); itName != names->end(); ++itName)
		{
			Ogre::String textureName = *itName;

			Ogre::uint32 textureFlags = Ogre::TextureFlags::AutomaticBatching;
			Ogre::uint32 filters = Ogre::TextureFilter::FilterTypes::TypeGenerateDefaultMipmaps;

			// Really important: createOrRetrieveTexture when its created, its width/height is 0 etc. so the texture is just prepared
			// it will be filled with correct data when setDataBlock is called
			Ogre::TextureGpu* texture = textureManager->createOrRetrieveTexture(textureName,
				Ogre::GpuPageOutStrategy::SaveToSystemRam, textureFlags, Ogre::TextureTypes::Type2D,
				resourceGroupName, filters, 0u);

			// Check if its a valid texture
			if (nullptr != texture)
			{
				try
				{
					texture->scheduleTransitionTo(Ogre::GpuResidency::Resident);
						
				}
				catch (const Ogre::Exception& exception)
				{

				}
			}
		}
	}
}

So what is interesting for me: Using textureManager->waitForStreamingCompletion(); at the end. Can I call that twice?

I want to load a complete dot scene with all models etc. Could I call that after the scene load:

textureManager->waitForStreamingCompletion()

At the end of the scene loading and also call it after preload some specific textures?

Is it wise e.g. to let also the designer for a level deside, which textures from which resource group should be preloaded?

Or is this no more necessary, if textureManager->waitForStreamingCompletion() is called after the scene is loaded but before the first renderOneFrame?

// Really important: GpuResidency must be set to 'OnSystemRam', else cloning a datablock does not work!
This shouldn't be right. Is there a bug in OgreNext I'm missing?

I tested it, maybe there was in the past a bug. Now cloning a datablock does work

Best Regards
Lax

[dark_sylinc]

So what is interesting for me: Using textureManager->waitForStreamingCompletion(); at the end. Can I call that twice?

You can call it multiple times without worry. It's not free, but it's not something you have to avoid at all costs.

I want to load a complete dot scene with all models etc. Could I call that after the scene load:

textureManager->waitForStreamingCompletion()

At the end of the scene loading and also call it after preload some specific textures?

Sounds fine.

Is it wise e.g. to let also the designer for a level deside, which textures from which resource group should be preloaded?

Yes. In Alliance Air War we have a list of textures that we preload very early (i.e. mostly UI textures and the one from the main scene in the main menu) to minimize loading times.

Since the game supports multiple "eras" (World War 2, Iron Curtain, Modern Era), the moment you tap on the era in the main menu we unload the textures of the current era and start preloading the textures of major planes from that new era you just chose.

By the time the user ends up navigating the UI to get to the hangar, the textures are likely already loaded.

Thus we basically "anticipate" what the user will do. This makes the game feel a lot more responsive.

I recommend you automate this, and only have the level decide intervene "don't load this texture" or "load this texture the automated system missed", because otherwise it becomes a very time consuming and error-prone process.
e.g. In the case of Alliance Airwar, we already know which textures belong to each era. But in the case of UI textures, we just spawn a dummy UI and put it behind a black screen that says "Loading...". This saves us the effort of checking what textures are used by the UI.

Or is this no more necessary, if textureManager->waitForStreamingCompletion() is called after the scene is loaded but before the first renderOneFrame?

If you always call waitForStreamingCompletion() before renderOneFrame, there's no need to call waitForStreamingCompletion() anywhere else (unless you start loading textures inside listeners inside renderOneFrame, which is possible but I don't recommend).

Calling waitForStreamingCompletion() before renderOneFrame ensures you never see a "white/dummy" texture because the real one is still loading. However the ultimate goal is if you can avoid calling waitForStreamingCompletion as much as possible and the user in practice (almost) never sees the texture "pop" because you correctly guessed what to preload.

[Lax]

Thanks for all the useful details!
I will try a similiar approach, but since a game engine with level editor. I try a most generic approach.
I already tag each texture, which is used, during scene load.

If someone creates an own game, he can use a functionality, which creates a folder and loads all his necessary meshes, materials and textures in this folder. So this folder indeed could be preLoaded if the game is started.

By the way: Nice game you produced!

Best Regards
Lax