DDS/DXT textures and main memory consumption

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
User avatar
Injector
Gremlin
Posts: 174
Joined: Wed Jan 21, 2004 2:42 pm
Location: Frankfurt, Germany

DDS/DXT textures and main memory consumption

Post by Injector »

Yesterday I thought I'd give dds textures a shot and batch converted all our tga files to dds files (using dxt1 for a start).

While the total file size decreased tremendously, I was disappointed when I checked the application's memory usage: There was no significant change while displaying our biggest scene (which is quite texture heavy).
So, my question is: When loading compressed dds textures, are they decompressed by Ogre at any time and held in main memory uncompressed?
btw: I am using a GeForce6800 GT so dxt textures are supported (the Ogre log seconds that).

Thanks for your input.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

Devil does uncompress them. (at least unpatched version for sure)
there will be a huge difference in Video Card Memory.

Don't use dxt1 as it is often badly supported on old card.
( cf http://udn.epicgames.com/Two/TextureSpecifications )

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

Ogre does not hold the textures in main memory, in either compressed or uncompressed form, after loading. If it still takes the same amount of memory it's due to your GL/D3D driver holding it in case the device is lost.

User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19265
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

tuan kuranes wrote:Devil does uncompress them. (at least unpatched version for sure)
It shouldn't if you tell it to keep DDS compressed, which we do when we use it to load DDS in GL. All DevIL does is strip the metadata out into readable attributes - we then load the compressed data up to GL as-is. Also, DevIL only handles DDS under GL, in D3D9 DDS loading bypasses DevIL entirely and it gets directly loaded in compressed form by D3D, since DDS is a binary dump of the D3D surface(s) anyway.

Both options therefore pass the compressed data through to the driver without decompressing it.

I wouldn't expect your app memory to change with DXT anyway. We don't hold onto texture data for any texture in OGRE, it all gets held by the driver, therefore no matter what the size it's not going to be making an impact on your app memory usage (except, as wumpus says, if the driver keeps a copy which is out of our control). DXT saves GPU memory, not app memory.

User avatar
Injector
Gremlin
Posts: 174
Joined: Wed Jan 21, 2004 2:42 pm
Location: Frankfurt, Germany

Post by Injector »

I see. Thanks for your answers.
Don't use dxt1 as it is often badly supported on old card.
It was only a test anyway. I chose dxt1 as it produced the smallest files, that's all.

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

tuan kuranes wrote: Don't use dxt1 as it is often badly supported on old card.
( cf http://udn.epicgames.com/Two/TextureSpecifications )
It's well supported, just not if you use transparency. If you just use it to store RGB images it's the best (smallest) choice as it doesn't waste any space on alpha data.

User avatar
Injector
Gremlin
Posts: 174
Joined: Wed Jan 21, 2004 2:42 pm
Location: Frankfurt, Germany

Post by Injector »

I guess he was referring to this (from the link):
DXT1 is a four-bit compressed color format that allows for opaque, and one-bit alpha textures; A hardware bug in all nVidia chipsets, including the NV20 (GeForce3), potentially makes DXT1 textures gross and ugly. Test your DXT1 textures on nVidia hardware before committing to their use. All other DXTC formats on nVidia hardware are okay, as textures are decompressed in 32-bit color internally.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

yes, now you should really use dxt5 fro transparent images.

About dds not atomatically decompressing, it seems rather strange to me, as in il_dds.c:line 446 , it seems there's no way to prevent devil to call "Decompress()" lying there ? Perhaps there's a way to get dds header that doesn't use "iLoadDdsInternal" that I did miss ?

Along other problems with dds opengl, That's why I made a dds gl patch a long time ago (dds cubemap, mipmaps and volume texture problems)

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

tuan kuranes wrote:it seems there's no way to prevent devil to call "Decompress()" lying there ? Perhaps there's a way to get dds header that doesn't use "iLoadDdsInternal" that I did miss ?
From OgreILImageCodec:

Code: Select all

                    // Compare DXT size returned by DevIL with our idea of the compressed size
                    if(imageSize == ilGetDXTCData(NULL, 0, dxtFormat))
                    {
                        // Retrieve data from DevIL
                        ilGetDXTCData((unsigned char*)output->getPtr()+offset, imageSize, dxtFormat);
                    } else
                    {
                        LogManager::getSingleton().logMessage(
                            "Warning: compressed image "+input->getName()+" size mismatch, devilsize="+StringConverter::toString(ilGetDXTCData(NULL, 0, dxtFormat))+" oursize="+
                            StringConverter::toString(imageSize));
                    }
Along other problems with dds opengl, That's why I made a dds gl patch a long time ago (dds cubemap, mipmaps and volume texture problems)
I've tested all of those a while back and they worked fine, given your version of DevIL is up to date.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

Problem is, before that call, Ogre does call "ilLoadL()", that make DevIL call "iLoadDdsInternal", that call "Decompress" in all cases.

So it seems it still decompress in all cases.

At time of the patch, there was a problem with compressed VTC texture, in devil (still in actual cvs) they say that VTC compressed need a hack..
search for "// Microsoft bug, they're not following their own documentation." in il_dds.c. But I couldn't open any VTC compressed texture using their "hack" with VTC texture coming from DxTex utility.

Patch was also adressing a cubemap problem that was making cubemap not the same in dx and gl.

Didn't make any test since, I just forgot this patch, while I was waiting a response from devil maintainers about how to adress those problem.

(and I no more uses 3d texture or cubemaps DDS for horizon mapping... so I did not follow accuratly the thing... kind of my fault...)

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

tuan kuranes wrote:Problem is, before that call, Ogre does call "ilLoadL()", that make DevIL call "iLoadDdsInternal", that call "Decompress" in all cases.
But as the IL image object is deleted afterwards, so decompressed image is removed from memory, I don't see why that matters?
At time of the patch, there was a problem with compressed VTC texture, in devil (still in actual cvs) they say that VTC compressed need a hack..
Ahh, I never tried VTC for 3d textures yet, that might be a problem yes. (as it's a NVidia only thing I could never try)

But cubemaps, even compressed ones, work in both GL and D3D.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

Right, that's not very important, just annoying when loading levels, which comes incredibly long those days, doom3, and hl2 loading just recalls of those c64 days, loading games from band ;)

VTC bug was in the decompression thing, so it's in the case of no compressed VTC support. But that's not widely used, we had only one forum call about that in one year.

Cubemap bug was harder to find... it was just 1 face of the whole cubemap that was inverted between dx and gl. But I guess it had surely been corrected while changing the opengl texture load mechanism.

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

tuan kuranes wrote:Right, that's not very important, just annoying when loading levels, which comes incredibly long those days, doom3, and hl2 loading just recalls of those c64 days, loading games from band ;)
True :) Some games still take ages to load a level.

Then again, DevIL might not have the best design, but it works, and it seems noone feels like refactoring it.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

that's the why of the rise of http://freeimage.sourceforge.net/
(dds, hdr, etc...)
Too bad they didn't choose lgpl.

Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Post by Vectrex »

but you can use their own licence which is fine for closed source commercial stuff (isn't it?)

User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19265
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Last time I checked Freeimage still lagged behind DevIL unfortunately. We may well switch later if it's proven to be stable and has all the features we need.

User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

It seems to have the same overall features as DevIL (but no DDS writing at all :( )

Unless FI provides very clear advantages I don't think someone of the Ogre team will spend time on this in the near future.

Of course, contributions are welcome. Writing an Ogre image codec that uses FreeImage is probably quite simple, it'd work like OgreILImageCodec.

Post Reply