Make the Resource System more modular.

What it says on the tin: a place to discuss proposed new features.
Post Reply
User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Make the Resource System more modular.

Post by Mako_energy » Tue Mar 20, 2012 1:24 am

Recently I started to research the possibility of taking over I/O in my engine entirely, including the I/O for Ogre's resources. The short version is that it doesn't seem possible. Here is the thread where I shared my logic for that. If I am wrong about that, please share.

Somehow, I'd like to make that possible. The simplest solution would probably be to extend the ScriptCompilerEvent class so that it includes all the information needed to manually load the texture, as well as add overrides to the MeshSerializer class that accept a void* for a buffer in memory. Both of these seem like bandaid solutions however. Plus there are likely more issues going that route that I haven't found just yet that would lead me back here.

Implementing interface classes was another idea I've seen elsewhere on these forums a while ago, but I realize that is a tall order to ask. Are there any existing idea's on the part of the Ogre team for changing the resource system any time soon?
0 x

User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
Contact:

Re: Make the Resource System more modular.

Post by masterfalcon » Tue Mar 20, 2012 2:29 am

This may not be a bad idea to propose for discussion as a GSoC project.
0 x

User avatar
nikki
Old One
Posts: 2730
Joined: Sat Sep 17, 2005 10:08 am
Location: Princeton, New Jersey, USA
Contact:

Re: Make the Resource System more modular.

Post by nikki » Sat Mar 24, 2012 4:53 am

Maybe generalizing the DataStream idea so that you can provide your own streams that may correspond to network resources, custom archive types, data from memory, or such? That sounds like an interesting idea indeed. How far do you think the current setup of Ogre scales in this context? It may be a matter of simply finding the minimal interface Ogre requires from a file system, and allowing the user to provide his own implementation for this (by default use the normal file system).
0 x

User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Re: Make the Resource System more modular.

Post by Mako_energy » Sun Mar 25, 2012 5:44 am

nikki wrote:It may be a matter of simply finding the minimal interface Ogre requires from a file system, and allowing the user to provide his own implementation for this (by default use the normal file system).
I actually like this idea quite a bit. It allows me to accomplish all that I want to accomplish plus there is an example of something that does that already in Ogre. I think this still requires some level of interface classes, but as I understand it the LogManager in Ogre is currently setup to initialize with all defaults if Root is created before it is. But in the Root constructor it checks to see if one already exists and does nothing if there is. That way you have to make the LogManager yourself before Root is created to setup anything custom. It may be feasible to do something like that with a minimal set of interface classes which I could implement to access my system and then drop in before I make root. This would also keep the system around for those that use it. An idea anyway.
0 x

User avatar
stealth977
Gnoll
Posts: 638
Joined: Mon Dec 15, 2008 6:14 pm
Location: Istanbul, Turkey

Re: Make the Resource System more modular.

Post by stealth977 » Tue Apr 24, 2012 8:11 pm

Well, actually its pretty simple to implement all that you need. We did it in Ogitor. All you need is to derive a class from Ogre::Archive and implement the functions just like what you need.

For example in Ogitor we use it to make OGRE use our own filesystem. The DataStreams OGRE use are very simple to derive from and supply whatever functionality you need. Only problem there may be is async datastreams like creating a datastream that fetches its data async from a network path, since OGRE requires your DataStreams to reach the underlying data instantly (or block till data is available)
0 x
Ismail TARIM
Ogitor - Ogre Scene Editor
WWW:http://www.ogitor.org
Repository: https://bitbucket.org/ogitor

User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Re: Make the Resource System more modular.

Post by Mako_energy » Sat Jun 02, 2012 1:48 am

Looking at the class again, I'm not sure why I missed using it like that. Someone mentioned deriving my own Archive class and I more or less misinterpreted it. Although I'm not entirely sure that would suit my needs completely, although on the surface it does appear to cover the basics. My use case is a bit unusual, and probably a bit extreme.

First is that I have my own API, and I hide/wrap the API's of other libraries we use, including Ogre. So I'd not only have to write the implementation classes, but also write the wrapper classes. Not the end of the world, but inconvenient. The best alternative is something that is possible, but has a caveat. Populating a memory buffer and then passing it into the appropriate manager to be constructed into the proper resource after it's done loading. I can do this using the MemoryDataStream class, however the caveat is when resources load other resources. When I last looked around the source, there aren't enough overrides to intercept when this happens. Such as when a mesh loads skeleton/material, or a material loads a texture, etc. There are places where you can add logic when this happens, but after that logic runs Ogre still tries to open it's own data stream and load the resource itself. If I am wrong about this, please let me know. Even if I go with the memory buffer route, that is a lot of interface and implementation classes that bloat up the binary needlessly.

The other complication is with our multi-threading strategy which we haven't delved into implementing just yet, but tons of planning. We want to implement a dependency graph, which I'm sure has it's own list of issues of compatibility with the way Ogre does things. That is a bit of a larger beast and I think possibly outside the scope of this thread, unless you have any insights to share there as well.
0 x

Post Reply