[GSoC 2013 - accepted] Resource System Redesign

[Owen]

Exams are over for another 4 months! Time to tackle this thing

[Owen]

Sample browser now builds. It crashes immediately upon starting to load the overlays, which is exactly in line with expectations. Compositors are probably completely busted, but we can come back to that, or maybe completely ignore it because it looks completely different on the other side of the merge anyway.

Next step is to fix the overlays to work.

[Klaim]

Sorry for the delay, here is my review of the document:

1. I think that a sequence diagram of a typical multi-threaded loading of resource would help clarify how the resource loading implementation communicate with Ogre (even if it's almost clear in the current document).

2. "The general rule is that modifications to a resource may only be made upon the main thread (that is, the one driving the Ogre rendering process), with one exception: the loading state."

This sentence might be ambiguous. Do you mean: "The general rule is that once ready, modifications to a resource may only be made by the thread driving Ogre rendering process (which should always be the same), with one exception: the loading state, which is the communication device between the Ogre thread and the resource loader."

Or something like that? I am not totally sure I understood correctly the initial sentence.

3. "Once startedLoadingResource function has been called, " -> +"by the loading thread"?

4. "resource; up to that point, the Resource object is confined such that the only operations which may be performed on it from other threads are queries as to its’ state and reference count modifications."

What are "other threads"? Ogre threads? That part is a bit confusing (to me at least), certainly because of the lack of clarification of the role of the different actors in the whole system.

5. A list of the different classes of the new resource system, with a short description, might help too.

6. "If you’re implementing your own resource system, you’ll need to consult the documentation to determine in what phase(s) to make what calls."

Which documentation exactly? The API documentation of the specific resource types?

7. The explainations of how to setup your own resource system should be gathered in a separate dedicated section.

8. "For these cases, the loaders that the default resource system uses are exposed as callbacks. [NB. At this time this isn’t implemented. The existing codecs are largely being stripped out and will be reappearing in the next phase]"

Is this note still true today?

9. "If you’re not interested in writing a new resource system, then the changes for you are much simpler."

Here by "the changes", you mean "upgrading your Ogre user code to this new system". Am I correct?

10. Titles to the different sections would help getting to the part interesting to the different users, like the last part which is what you have to know for migrating to the new system if I'm correct.


I hope to try the new resource system as soon as it's stabilized enough that you, Owen, think users can begin to experiment with it.

[Owen]

Sorry for the delay, here is my review of the document:

1. I think that a sequence diagram of a typical multi-threaded loading of resource would help clarify how the resource loading implementation communicate with Ogre (even if it's almost clear in the current document).

2. "The general rule is that modifications to a resource may only be made upon the main thread (that is, the one driving the Ogre rendering process), with one exception: the loading state."

This sentence might be ambiguous. Do you mean: "The general rule is that once ready, modifications to a resource may only be made by the thread driving Ogre rendering process (which should always be the same), with one exception: the loading state, which is the communication device between the Ogre thread and the resource loader."

Best clarification would be "with the exception that the resource loader may modify it from another thread while it is in the LOADING state"

3. "Once startedLoadingResource function has been called, " -> +"by the loading thread"?

By the resource system on the main thread

startedLoadingResource is the function you call to tell Ogre that you're about to start doing background stuff on another thread.
You call aboutToLoadResource to notify it that you're done with that and going to finalize up your work on the main thread
You finish everything off by calling loadedResource.

All of these are state transitions, they must execute on the main thread.

4. "resource; up to that point, the Resource object is confined such that the only operations which may be performed on it from other threads are queries as to its’ state and reference count modifications."

What are "other threads"? Ogre threads? That part is a bit confusing (to me at least), certainly because of the lack of clarification of the role of the different actors in the whole system.

The ONLY things which are safe to do from a background thread are manipulate the reference count (through IntrusivePtr<T>) and read the state of the object (the Resource::LoadState enum), except for if you are the resource loader and are doing a background load from another thread

5. A list of the different classes of the new resource system, with a short description, might help too.

Resource and ResourceLoader. Add in ResourceCodec and ScriptLoader if you want to hook into the "standard" loaders.

6. "If you’re implementing your own resource system, you’ll need to consult the documentation to determine in what phase(s) to make what calls."

Which documentation exactly? The API documentation of the specific resource types?

The Doxygen comments for Resource and ResourceLoader

7. The explainations of how to setup your own resource system should be gathered in a separate dedicated section.

Agreed

8. "For these cases, the loaders that the default resource system uses are exposed as callbacks. [NB. At this time this isn’t implemented. The existing codecs are largely being stripped out and will be reappearing in the next phase]"

No

9. "If you’re not interested in writing a new resource system, then the changes for you are much simpler."

Here by "the changes", you mean "upgrading your Ogre user code to this new system". Am I correct?

Yes

10. Titles to the different sections would help getting to the part interesting to the different users, like the last part which is what you have to know for migrating to the new system if I'm correct.

Indeed. It was an early draft

I hope to try the new resource system as soon as it's stabilized enough that you, Owen, think users can begin to experiment with it.

Experimenting it is definitely possible now, but expect bugs and little areas to not be working. Compositors are definitely busted, for example, as are overlays.

[nickG]

is possible create resource system as in UE?(loading only target package+dependence)

[Owen]

is possible create resource system as in UE?(loading only target package+dependence)

You can do whatever you want. The new default resource system supports something similar.

[Mako_energy]

Owen,

What is the current timeline for this project looking like? Will you be able to complete what remains of the work prior to the next GSoC? Could completing this project be another GSoC(if there is even enough work left for it to be one)?

[volca]

Hey,

I am really looking forward to see this in vanilla ogre - are there any plans to get this into v2-0 or is it not a viable time frame to fill?