texture initialization from memory

[bishopnator]

I need to initialize Ogre::Texture from memory. So I load file content into memory (from my internal media library - own format). Problem is with loading DDS file, because implementation of Ogre::Texture for both implemented render systems (DirectX and OpenGL) contain special code for loading dds files (test whether texture is in dds format is determined by its name and then ie. for DX render system DX method is called to load texture). I can modified plugins - texture will contain methods ie. loadFromMem() and input parameters will be format of memory and memory itself.

Anybody was solving this problem? I think this improvements will be very useful for anyone, who wants implement own texture loader (ie. texture can be stored in executable file and loaded to memory during runtime - if dds file will be stored in this way, it will NOT be loadable).

[tuan kuranes]

Isn't that already possible if you use Ogre::Image::loadFromMemory and then load texture from that Image ?

[bishopnator]

This is possible for all textures except textures in DDS format. It is not possible to initialize texture from DDS file, which is in memory. Maybe you load Ogre::Image and initialize texture in some way, but point is in using compressed format DXTn (DXT1 - DXT5). If I have texture in DDS file which is in non-compressed format (so all except DXTn), I am not able to load texture. This is done for DirectX render system by calling method D3DXCreateTextureFromFileInMemoryEx, but I haven't access to this function. ie. method void D3D9Texture::_loadNormTex() contains 2 parts:
1st for DDS texture (determined by extension)
2nd for rest of formats - this part can be copied from this plugin and used anywhere in application (it is independent from render system)

So this is the reason why I wrote, that method loadFromMem will be useful (this render-system depended code will be in this method)

I have problem only with DDS textures which are in compressed format DXTn. Other formats can be loaded using Ogre::Image::loadFromMemory and Ogre::Texture::_loadImages.

[bishopnator]

I see, that I made some mistakes in my last replay :-\
So correct is:
I am NOT able to load textures from DDS format if they are in compressed format (DXTn). If they are in non-compressed format, loading is possible via
Ogre::Image::loadFromMemory and Ogre::Texture::_loadImages.

[sinbad]

Yes, a known limitation right now with DXT compressed data, it's down to how the data is streamed through the 2 different rendersystems (DirectX sucks DDS straight in, GL needs to separate the metadata from the main data). The Image class has already broken the metadata apart so it in theory needs to be reconstituted for feeding into DirectX, but that in turn needs access to the DDS codec; it's a bit messy.

You're generally better to load these via a DataStream, which can be streamed from memory. That way the DDS itself is still in one piece rather than having been partly decoded by Image.

[bishopnator]

So what do u plan to do with this? It's clear, that used solution for loading textures from dds files is used due to saving memory (textures from these files should be compressed in graphic card). So I see, that using decompression files and converting to other format (ie. A8R8G8B8) is inefficient.

I looked into implementation of Ogre::Texture for DX and GL plugins and in both implementations there is code for loading file content into memory and this memory is later processed to load texture data into graphic card. What I need is to have method which accept DataStreamPtr and flag, which indicates format of data (dds, bmp, tga, etc.) Change is only in split existing loading methods into 2 parts:
1) load file contents from media folder (default loading)
2) process loaded data

I think that this approach also helps to write manual loaders for resources, because user only needs to rewrite 1st step (load file data into memory) and 2nd step should be same in all cases (maybe some special cases like resources generation will be different)

[dudiz]

I also encountered the same problem while trying to load DDS data from memory.

I did a little hack:

First Stage: write a DDSFileInMemory archive that holds your MemoryDataStream pointer:

Code: Select all

#ifndef _CFileInMemoryArchive_H__
#define _CFileInMemoryArchive_H__

#include "stdafx.h"
#include "OgreArchiveFactory.h"

class CFileInMemoryArchive : public Ogre::Archive   
{
public:
	CFileInMemoryArchive(const Ogre::String& name, const Ogre::String& archType ) : Ogre::Archive(name, archType) 
	{
		Ogre::StringVector* stringVector = new Ogre::StringVector();
		stringVector->push_back(name);
		m_StringVectorPtr.bind(stringVector);
		size_t pos = mName.find_last_of("#");
		Ogre::String baseName = mName.substr(0, pos);
		Ogre::MemoryDataStream* dataStream = (Ogre::MemoryDataStream*)atoi(baseName.c_str());
		m_DataStreamPtr.bind(dataStream);
		Ogre::FileInfo fileInfo;
		fileInfo.basename = name;
		fileInfo.archive = this;
		fileInfo.filename = name;
		fileInfo.path = "";
		fileInfo.compressedSize = dataStream->size();
		fileInfo.uncompressedSize = dataStream->size();
		Ogre::FileInfoList* fileInfoList = new Ogre::FileInfoList();
		fileInfoList->push_back(fileInfo);
		m_FileInfoListPtr.bind(fileInfoList);

	};
	~CFileInMemoryArchive()
	{
	};

	/// @copydoc Archive::isCaseSensitive
	bool isCaseSensitive(void) const {return true;};

	/// @copydoc Archive::load
	void load(){};
	/// @copydoc Archive::unload

	void unload()
	{
	};

	/// @copydoc Archive::open
	Ogre::DataStreamPtr open(const Ogre::String& filename) const
	{
		return m_DataStreamPtr;
	}

	/// @copydoc Archive::list
	Ogre::StringVectorPtr list(bool recursive = true, bool dirs = false) {return m_StringVectorPtr;};

	/// @copydoc Archive::listFileInfo
	Ogre::FileInfoListPtr listFileInfo(bool recursive = true, bool dirs = false) {return m_FileInfoListPtr;};

	/// @copydoc Archive::find
	Ogre::StringVectorPtr find(const Ogre::String& pattern, bool recursive = true,
		bool dirs = false) 
	{
		return m_StringVectorPtr;
	}

	/// @copydoc Archive::findFileInfo
	Ogre::FileInfoListPtr findFileInfo(const Ogre::String& pattern, bool recursive = true,
		bool dirs = false) {return m_FileInfoListPtr;};

	/// @copydoc Archive::exists
	bool exists(const Ogre::String& filename) {return filename == mName;}

	Ogre::StringVectorPtr m_StringVectorPtr;
	Ogre::DataStreamPtr m_DataStreamPtr;
	Ogre::FileInfoListPtr m_FileInfoListPtr;
};

/** Specialisation of ArchiveFactory for FileSystem files. */
class CFileInMemoryArchiveFactory : public Ogre::ArchiveFactory
{
public:
	virtual ~CFileInMemoryArchiveFactory() {}
	/// @copydoc FactoryObj::getType
	const Ogre::String& getType(void) const
	{
		static Ogre::String name = "DDSFileInMemory";
		return name;
	}
	/// @copydoc FactoryObj::createInstance
	Ogre::Archive *createInstance(const Ogre::String& name) 
	{
		return new CFileInMemoryArchive(name, "DDSCoreFileInMemory");
	}
	/// @copydoc FactoryObj::destroyInstance
	void destroyInstance(Ogre::Archive* arch) {delete arch;}
};



#endif

Second Stage:
Create Ogre::MemoryDataStream which contains the DDS memory buffer.

Code: Select all

Ogre::StringUtil::StrStreamType mmfileName;
const BYTE* buffer = tile->GetBuffer(); // get DDS memory buffer.
size_t size = tile->GetBufferSize(); // get DDS memory buffer size.

Ogre::MemoryDataStream* memDataStream = new MemoryDataStream((void*)buffer, size);


// The file name is concatanating the address of the pointer to the buffer and the point of the class
// in order to supply the FileInMemory archive access to the buffer
// and to avoid duplication of the name of the texture between different tiles
// note that the memDataStream pointer is deleted when the mmfileName is unloaded

mmfileName << ((unsigned int)memDataStream) << "#" << ((unsigned int)this) << ".dds";

ResourceGroupManager::getSingleton().addResourceLocation(
mmfileName.str(), "DDSFileInMemory", ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

mTex = TextureManager::getSingleton().load(mmfileName.str().c_str(),ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

ResourceGroupManager::getSingleton().removeResourceLocation(
				mmfileName.str(), ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);

ArchiveManager::getSingleton().unload(mmfileName.str());

The "trick" is that OGRE treats our file as a DDS, but it reads the buffer from the MemoryDataStream.

If anyone has a better solution please let me know...

[bishopnator]

thanks for reply, your solution looks nice .. I consider modification of Ogre -> implementing abstract method Ogre::Texture::loadFromMem(). I looked into the Ogre sources and every method from Ogre::Texture (impl. for DX and GL) loads file into memory and than initialize texture (I suppose that also other methods for initializing resources work similar). So I think that splitting these methods into 2 parts (load file to memory and than initialize textury from memory) will be better solution. Disadvantage of this it is in ogre modification - if new version of Ogre come, I must rewrite all my modifications to new Ogre :-\ I have problems with posting patches (I dont now why, but always connection to server failed, so even if this modification was useful for others, I will not be able to post it :-\ )

[Game_Ender]

If you really can't use the SF patch track you can always zip up you post and post a link here in the development forum.