[GSoC 2013 - accepted] Resource System Redesign

Threads related to Google Summer of Code
Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Owen » Thu Feb 06, 2014 1:47 pm

Exams are over for another 4 months! Time to tackle this thing :-)
0 x

Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Owen » Thu Feb 06, 2014 3:34 pm

Sample browser now builds. It crashes immediately upon starting to load the overlays, which is exactly in line with expectations. Compositors are probably completely busted, but we can come back to that, or maybe completely ignore it because it looks completely different on the other side of the merge anyway.

Next step is to fix the overlays to work.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Klaim » Thu Feb 06, 2014 6:52 pm

Sorry for the delay, here is my review of the document:

1. I think that a sequence diagram of a typical multi-threaded loading of resource would help clarify how the resource loading implementation communicate with Ogre (even if it's almost clear in the current document).

2. "The general rule is that modifications to a resource may only be made upon the main thread (that is, the one driving the Ogre rendering process), with one exception: the loading state."

This sentence might be ambiguous. Do you mean: "The general rule is that once ready, modifications to a resource may only be made by the thread driving Ogre rendering process (which should always be the same), with one exception: the loading state, which is the communication device between the Ogre thread and the resource loader."

Or something like that? I am not totally sure I understood correctly the initial sentence.

3. "Once startedLoadingResource function has been called, " -> +"by the loading thread"?

4. "resource; up to that point, the Resource object is confined such that the only operations which may be performed on it from other threads are queries as to its’ state and reference count modifications."

What are "other threads"? Ogre threads? That part is a bit confusing (to me at least), certainly because of the lack of clarification of the role of the different actors in the whole system.

5. A list of the different classes of the new resource system, with a short description, might help too.

6. "If you’re implementing your own resource system, you’ll need to consult the documentation to determine in what phase(s) to make what calls."

Which documentation exactly? The API documentation of the specific resource types?

7. The explainations of how to setup your own resource system should be gathered in a separate dedicated section.

8. "For these cases, the loaders that the default resource system uses are exposed as callbacks. [NB. At this time this isn’t implemented. The existing codecs are largely being stripped out and will be reappearing in the next phase]"

Is this note still true today?

9. "If you’re not interested in writing a new resource system, then the changes for you are much simpler."

Here by "the changes", you mean "upgrading your Ogre user code to this new system". Am I correct?

10. Titles to the different sections would help getting to the part interesting to the different users, like the last part which is what you have to know for migrating to the new system if I'm correct.


I hope to try the new resource system as soon as it's stabilized enough that you, Owen, think users can begin to experiment with it.
0 x

Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Owen » Thu Feb 06, 2014 7:41 pm

Klaim wrote:Sorry for the delay, here is my review of the document:

1. I think that a sequence diagram of a typical multi-threaded loading of resource would help clarify how the resource loading implementation communicate with Ogre (even if it's almost clear in the current document).

2. "The general rule is that modifications to a resource may only be made upon the main thread (that is, the one driving the Ogre rendering process), with one exception: the loading state."

This sentence might be ambiguous. Do you mean: "The general rule is that once ready, modifications to a resource may only be made by the thread driving Ogre rendering process (which should always be the same), with one exception: the loading state, which is the communication device between the Ogre thread and the resource loader."
Best clarification would be "with the exception that the resource loader may modify it from another thread while it is in the LOADING state"
3. "Once startedLoadingResource function has been called, " -> +"by the loading thread"?
By the resource system on the main thread

startedLoadingResource is the function you call to tell Ogre that you're about to start doing background stuff on another thread.
You call aboutToLoadResource to notify it that you're done with that and going to finalize up your work on the main thread
You finish everything off by calling loadedResource.

All of these are state transitions, they must execute on the main thread.
4. "resource; up to that point, the Resource object is confined such that the only operations which may be performed on it from other threads are queries as to its’ state and reference count modifications."

What are "other threads"? Ogre threads? That part is a bit confusing (to me at least), certainly because of the lack of clarification of the role of the different actors in the whole system.
The ONLY things which are safe to do from a background thread are manipulate the reference count (through IntrusivePtr<T>) and read the state of the object (the Resource::LoadState enum), except for if you are the resource loader and are doing a background load from another thread
5. A list of the different classes of the new resource system, with a short description, might help too.
Resource and ResourceLoader. Add in ResourceCodec and ScriptLoader if you want to hook into the "standard" loaders.
6. "If you’re implementing your own resource system, you’ll need to consult the documentation to determine in what phase(s) to make what calls."

Which documentation exactly? The API documentation of the specific resource types?
The Doxygen comments for Resource and ResourceLoader
7. The explainations of how to setup your own resource system should be gathered in a separate dedicated section.
Agreed
8. "For these cases, the loaders that the default resource system uses are exposed as callbacks. [NB. At this time this isn’t implemented. The existing codecs are largely being stripped out and will be reappearing in the next phase]"
No
9. "If you’re not interested in writing a new resource system, then the changes for you are much simpler."

Here by "the changes", you mean "upgrading your Ogre user code to this new system". Am I correct?
Yes
10. Titles to the different sections would help getting to the part interesting to the different users, like the last part which is what you have to know for migrating to the new system if I'm correct.
Indeed. It was an early draft

I hope to try the new resource system as soon as it's stabilized enough that you, Owen, think users can begin to experiment with it.
Experimenting it is definitely possible now, but expect bugs and little areas to not be working. Compositors are definitely busted, for example, as are overlays.
0 x

nickG
Greenskin
Posts: 122
Joined: Fri Jan 20, 2012 6:44 pm
Location: Russia,Moscow

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by nickG » Wed Feb 12, 2014 7:35 pm

is possible create resource system as in UE?(loading only target package+dependence)
0 x

Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Owen » Fri Feb 14, 2014 3:15 am

nickG wrote:is possible create resource system as in UE?(loading only target package+dependence)
You can do whatever you want. The new default resource system supports something similar.
0 x

User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by Mako_energy » Tue Feb 18, 2014 11:48 am

Owen,

What is the current timeline for this project looking like? Will you be able to complete what remains of the work prior to the next GSoC? Could completing this project be another GSoC(if there is even enough work left for it to be one)?
0 x

User avatar
volca
Gnome
Posts: 393
Joined: Thu Dec 08, 2005 9:57 pm
Contact:

Re: [GSoC 2013 - accepted] Resource System Redesign

Post by volca » Wed Apr 30, 2014 8:11 pm

Hey,

I am really looking forward to see this in vanilla ogre - are there any plans to get this into v2-0 or is it not a viable time frame to fill?
0 x
Image

Post Reply