File level fallback for textures

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.
Kai-Peter
Greenskin
Posts: 133
Joined: Tue Oct 15, 2002 10:14 am
Location: Helsinki, Finland
x 1

File level fallback for textures

Post by Kai-Peter »

Here is the problem:
- Artists prefer to use formats like TGA for editing textures. Easy to edit and has nice alpha channel support. The files are usually named "*.tga"
- Releases (and users on slow connections) prefer small optimized formats. Let these files be named "*.dds"
- Material scripts (and a lot of other places) need a single file name.

So how to get that single filename to load tga for artists and something else for end users? With the same name?

Here is my solution:

- All image references that need this functionality should end with "shi" (for ShortHike Image).
- The artist creates a file "path/to/image.tga" and enters it into a material script as "path/to/image.shi".
- I run a custom Archive implementation. It encapsulates a standard archive (a FileSystem in my case) that is created at the same time when my custom Archive is created.
- The custom archive passes most calls through to the FileSystemArchive. Except when someone looks for a "foo.shi" file. In exists() it looks if "foo.dds" or "foo.tga" exists and if it can find either one returns true.
- When "path/to/image.shi" is opened it returns an empty named MemoryDataStream("path/to/image.shi", (void*)NULL, 0).
- My custom ImageCodec is registered to handle "shi" files. When it is asked to decode an image it looks first for the ".tga" file and if found asks the TGA ImageCoded to decode it. If again it's a ".dds" file asks the DDS image codec to decode it.

The actual formats are naturally arbitrary. You could even have the codec check for three or four file formats if you wanted to. Or splice together images from two different files (like jpg for RGB and zip for A). Also, I haven't implemented a custom MemoryDataStream that would encode the resourcegroup, as currently that is hardcoded for me. But that would be a few lines of code anyway.

Suggestions and feedback? Is there an easier way to do this? (It took just a few hours to write so you really need to try..) :D
Kai Backman, programmer (Blog)
ShortHike - Space Station Game
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

You could run a release processor, which parses scripts, checks for better formats, and alters the script if they're available? Or perhaps even generates the .dds files for you. I don't think this needs to be a runtime complication.
Kai-Peter
Greenskin
Posts: 133
Joined: Tue Oct 15, 2002 10:14 am
Location: Helsinki, Finland
x 1

Post by Kai-Peter »

Good point from our Channel Islands participant. Preprocessing is a great way to save overhead. And yes, this is part of my CacheManager for creating memorymapped raw cache files with preprocessed DXT files. Full agreement on the best format issue.

But is preprocessing more convenient? This takes a few lines of code for each .shi file but makes it fully transparent to the user. Any image will get the same benefits. And all future image references as well. A script parser would need to be updated for every new type of script (materials, icons, UI widgets etc.). Also having a distinct suffix makes these files stand out and you can still use real tga files if you wish. :D
Kai Backman, programmer (Blog)
ShortHike - Space Station Game
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66

Post by sinbad »

Fair enough - although I'm thinking this could even be done with awk ;) I don't think you'd need separate parsers - just looking for <something>.tga would be reliable across all scripts. You're right that doing it automatically is simpler, but since you'd have to convert the image files anyway somehow, I'm thinking you have another processing step 'somewhere' anyway to plug this into.
Last edited by sinbad on Sat May 14, 2005 11:07 pm, edited 1 time in total.
User avatar
:wumpus:
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3067
Joined: Tue Feb 10, 2004 12:53 pm
Location: The Netherlands
x 1

Post by :wumpus: »

I'm in favor of preprocessing as well in this case, especially if you use compressed dds. The artist can see how the final result looks, which is the entire reason for looking at his texture in Ogre anyway, I suppose?

Apart from that, seems you jumped through quite a few hoops to get it to work this way, I don't think you should change it again :)
Kai-Peter
Greenskin
Posts: 133
Joined: Tue Oct 15, 2002 10:14 am
Location: Helsinki, Finland
x 1

Post by Kai-Peter »

Yep, I'm planning to do the compression in the build system (suing Jam). But that is a file name matching problem, so it isn't as bad as the AWKing through all the scripts. Oh, and the above was actually just a few hours of work. The Ogre:: interfaces are good. ;)

I actually got another idea from James of Reflexive who told that their system let's the artist see the compressed image at runtime. Ie. the artists plugs in a TGA file and then uses the level editor to look at the image after the compression. They can then tweak a quality variable in the editor to see what's the quality they need and this setting is then saved for the installer build step. I tought that was a pretty interesting idea for direct manipulation.. I might do that myself .. :D

So here's the plan now, thanks for the input:
1. TGA->Compressed at build time
2. Compressed->Uncompressed raw file at first startup
3. Blazing fast loads by memorymapping file (and using yet another codec that just returns the memory location).

Tweaking, tweaking ..
Kai Backman, programmer (Blog)
ShortHike - Space Station Game