Owen wrote:For every resource, Ogre needs a way to link it to the resource system that loaded it.
Picture an application which uses its' own resource system and CEGUI. CEGUI should not, for the purposes of interacting with Ogre, need to know anything about the application's resource system. Instead, the idea is that it can create the Resources directly and provide it's own ResourceLoader for them (like, if memory serves, it provides a ManualResourceLoader now).
So, a resource needs a link to its' loader, and that loader likely needs some way to identify the resource (being as, as I've mentioned, resource names/IDs are on my chopping block)
We could put a void*/Orge::Any in the Resource, and let the loader use that... but to me it feels simpler and more elegant for the resource system to just store that information in the ResourceLoader instance
Think of it as every resource being loaded by a ManualResourceLoader. Thats the paradigm I want to head to.
I don't see how the description I made of the system would be incompatible of what you're describing.
There would be a ResourceLoader provided by CEGUI, just it's not per Resource instance, it's global. The ResourceSystem still needs to know how to fill the resource, and that's why a custom ResourceLoader is needed.
As for storing the needed data as void*/Ogre::Any; I don't get it. If CEGUI needs custom resources, then deriving from the base Resource is the right solution:
CEGUITextureLoader : public Ogre::ResourceLoader ---> Handles filling of Resources.
CEGUITexture : public Ogre::Texture ---> CEGUI's texture, data handled & filled by CEGUITextureLoader
Owen wrote:
Agreed. As mentioned, I'm making Resource completely thread unsafe except that
- The reference count will be maintained atomically
- The main thread guarantees to update the resource state atomically (i.e. it is legal to read it from other threads)
For the first one, some day Ogre will have a defined way of ditching resources and reference count will hopefully go away (or stay in a way more like Havok does). But not today, so, acceptable.
Could you elaborate on the latter (what do you mean to be legal to read the state from other threads, and why the main thread would be the one updating the resource state)?
Owen wrote:This approach is unworkable - it would require to traverse all Entities and similar whenever the render system drops the device on top of us, in order to erase their resource references (and then instigate reloading of them)
1. Render Systems don't drop the device often.
2. Even if it did, the system would still handle it gracefully. On device lost the resource system stalls everything else until everything is reloaded. This approach is serialized, but it's pointless to talk about "background loading" when there's nothing in the foreground to do.
Owen wrote:Ogre is a retained mode graphics API. Resources should be retained also.
When I'm talking about releasing the Resource pointer, I'm not saying that it will become a dangling ptr. The same pointer will be delivered when the resource is given back.
The reason I'm saying we should encourage to release the Resource is because it's multithreaded (aka. avoid the user from screwing up, he's not expected to know much about MT safety or how the ResourceSys works); you can't use a resource that is not yet loaded or in an inconsistent state. You either wait (stall), or try again next time. If you choose the latter, it's safer if we take away the dangerous toy.
This is made for the sake of multithreading, scalability, and safety. If you wish for all sakes to retain the pointer, use blocking loads & reloads for all your Resources. With enough anticipation, blocking loads will never stall; though I admit for reloads it would always stall unless you give away the Resource for a while and do something else until is reloaded. Nevertheless any other multithreading aproach that avoids this problem means more frequent locks to check the state.
My approach needs to lock once per frame. Blocking loads may or may not need to lock once per call (they definitely don't if the resource was already loaded; if it wasn't it... depends, keep reading for an explanation)
Owen wrote:
Additionally, I don't like the idea of keeping blocking resource loads (For example, the application's resource system might rely upon the main thread servicing an event loop of some kind in order to keep the resource system functional)
Well, I'm strongly inclined against anything that could lead to such kind of inter-dependency because it ends up in listeners pre- and post- anything you'll perform (I'm speaking generally, not just Resources) which leads to:
a. if( listener ) listener->callVirtualFunction() -> This is bad if done too often.
b. Break assumptions on the state of what you're working with, that can lead to preventing optimizations on how to treat the data you're processing -> The worst side effect. Node::processQueuedUpdates is a prime example of this misfortune.
And 99% times, there's a better way of doing it (sometimes, this way already existed but was ignored).
However, I do recognize there may be times where this is needed, may be because of time constraints, legal (i.e. not all source code is available), etc. which may render this inter-dependency necessary. So, shall that be the case, my answer is:
Don't take my words so literally. Making getLoadedResource 'blocking' doesn't necessarily mean the thread gets suspended. Sure that's probably what's often gonna happen. But what it really means is that we guarantee that getLoadedResource won't return until the pointer is loaded, it doesn't matter how.
In your particular example, one solution is that the thread gets suspended and waits for the Background to thread to wake him up to service the event loop you mentioned and then goes to sleep again. After the resource is loaded, the main thread wakes up again for a second time, this time to return from getLoadedResource.
That's one solution, another attempt is to flag all those cases and load the resource inside getLoadedResource (that is, no multithreading at all). I can think of other similar single threaded solutions.
Another multithreaded solution is that a task that will service the event loop is spawned before the main thread goes to sleep.
Owen wrote:One of the resultant benefits of my proposed excision of the resource system is that innovation
I'm looking forward to that.
Cheers