Make the Resource System more modular.

[Mako_energy]

Recently I started to research the possibility of taking over I/O in my engine entirely, including the I/O for Ogre's resources. The short version is that it doesn't seem possible. Here is the thread where I shared my logic for that. If I am wrong about that, please share.

Somehow, I'd like to make that possible. The simplest solution would probably be to extend the ScriptCompilerEvent class so that it includes all the information needed to manually load the texture, as well as add overrides to the MeshSerializer class that accept a void* for a buffer in memory. Both of these seem like bandaid solutions however. Plus there are likely more issues going that route that I haven't found just yet that would lead me back here.

Implementing interface classes was another idea I've seen elsewhere on these forums a while ago, but I realize that is a tall order to ask. Are there any existing idea's on the part of the Ogre team for changing the resource system any time soon?

[masterfalcon]

This may not be a bad idea to propose for discussion as a GSoC project.

[nikki]

Maybe generalizing the DataStream idea so that you can provide your own streams that may correspond to network resources, custom archive types, data from memory, or such? That sounds like an interesting idea indeed. How far do you think the current setup of Ogre scales in this context? It may be a matter of simply finding the minimal interface Ogre requires from a file system, and allowing the user to provide his own implementation for this (by default use the normal file system).

[Mako_energy]

It may be a matter of simply finding the minimal interface Ogre requires from a file system, and allowing the user to provide his own implementation for this (by default use the normal file system).

I actually like this idea quite a bit. It allows me to accomplish all that I want to accomplish plus there is an example of something that does that already in Ogre. I think this still requires some level of interface classes, but as I understand it the LogManager in Ogre is currently setup to initialize with all defaults if Root is created before it is. But in the Root constructor it checks to see if one already exists and does nothing if there is. That way you have to make the LogManager yourself before Root is created to setup anything custom. It may be feasible to do something like that with a minimal set of interface classes which I could implement to access my system and then drop in before I make root. This would also keep the system around for those that use it. An idea anyway.

[stealth977]

Well, actually its pretty simple to implement all that you need. We did it in Ogitor. All you need is to derive a class from Ogre::Archive and implement the functions just like what you need.

For example in Ogitor we use it to make OGRE use our own filesystem. The DataStreams OGRE use are very simple to derive from and supply whatever functionality you need. Only problem there may be is async datastreams like creating a datastream that fetches its data async from a network path, since OGRE requires your DataStreams to reach the underlying data instantly (or block till data is available)

[Mako_energy]

Looking at the class again, I'm not sure why I missed using it like that. Someone mentioned deriving my own Archive class and I more or less misinterpreted it. Although I'm not entirely sure that would suit my needs completely, although on the surface it does appear to cover the basics. My use case is a bit unusual, and probably a bit extreme.

First is that I have my own API, and I hide/wrap the API's of other libraries we use, including Ogre. So I'd not only have to write the implementation classes, but also write the wrapper classes. Not the end of the world, but inconvenient. The best alternative is something that is possible, but has a caveat. Populating a memory buffer and then passing it into the appropriate manager to be constructed into the proper resource after it's done loading. I can do this using the MemoryDataStream class, however the caveat is when resources load other resources. When I last looked around the source, there aren't enough overrides to intercept when this happens. Such as when a mesh loads skeleton/material, or a material loads a texture, etc. There are places where you can add logic when this happens, but after that logic runs Ogre still tries to open it's own data stream and load the resource itself. If I am wrong about this, please let me know. Even if I go with the memory buffer route, that is a lot of interface and implementation classes that bloat up the binary needlessly.

The other complication is with our multi-threading strategy which we haven't delved into implementing just yet, but tons of planning. We want to implement a dependency graph, which I'm sure has it's own list of issues of compatibility with the way Ogre does things. That is a bit of a larger beast and I think possibly outside the scope of this thread, unless you have any insights to share there as well.