[SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Threads related to Google Summer of Code
Locked
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

tuan kuranes wrote:
compositor textures will have to exist once per object
That has to be specified somehow by the user, as some pass resulting may be just intermediate, and can be reused in other "single effect material". In the skin example, the first diffuse pass RTT is to be reused, where the second pass RTT (blurred diffuse) has to be kept until material applied.
Isn't there something like that in current Compositor ? Intermediate RTT reuse ?
There is - thats the 'shared' possibility that sinbad talked about. I was talking about the 'result' RTT of these passes. If we have 10 objects that use this material, that means that 10 copies of the result RTT will have to coexist. Not necessarily a bad thing, as the objects that will use this will probably be the very-high quality objects, and not too many of them will coexist in the first place.
It's probably a better idea to have this conversation at the end of the SoC if the project gets selected.
I still think we have to make sure that gsoc design will be at least compliant, if not designed to ease that future work.
Thinking all along the wrok how/where shadow framework could be replaced/shared seems a good thing to me.

Specifically, the "shadow material" current "linking" inside material could be shared/replace/enhanced by some way to fit both compositors and shadow framework ?
Hm, yea good point. That's part of the reason i did decide to "think out loud" in this topic. The way I see the design I'm going towards right now will make these ideas much easier. Its important to remember that we can't remove features, as Ogre does have users that use its features. The migration path is probably something like declaring the 'old' version (shadow_texture and such) deprecated for a stable version and internally implementing it using the new system, and then after a while (unlike MS who do it every tuesday with DX) - maybe a complete stable cycle (Ogre 2.0?) remove it.

The conversation is healthy for this SoC project and for Ogre, and I think its causing me to believe more in the steps that I'm describing. I think my design in its original form actually fits with these ideas very well. I will have to make sure that it stays the case when I get down to the finer details of the implementation, but I do vision it all connecting together quite nicely when the post-SoC work will be done.
User avatar
iloseall
Gremlin
Posts: 156
Joined: Sun Sep 14, 2003 3:54 am
Location: Beijing China
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by iloseall »

In FX, we can setting light as camera,so we can control the shadow or other light space effect.
Is it good idea that add camera or frustum or light control to compositor?
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

That will be possible, but probably not directly in scripts.
There are (IMO) too many possible options for configuring these custom render sequences (for example, the camera will be where the light is, but what will be its orientation and clip distances? What will be its location if its a directional light? Etc). So, the option will be to auto-attach a compositor listener, using the script-triggered-binding that I will add. This will make it more explicit, and when someone will read the script they will see that there is some code that comes along with it.
I don't think it will be possible to solve the problem without code, and we won't be able to solve the flexibility issue in scripts only without using a proper scripting language, and I'm not about to add a scripting language as a dependency to ogre...
User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany
x 1

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by xadhoom »

Hi Noman!

By reading your proposal I´m surprised that you don´t mention the possibilities which arise for "deferred shading".
About the Ogre example it is often said that its not done the right way and as far as I know chaining compositors is exactly the way it should be done. Maybe it would be very attractive for the community (at least for me) if your work would lead to a new "Deferred Shading Example" for Ogre which clearly shows the functionality the new compositor framework provides when you have finished the work.

xad
User avatar
stealth977
Gnoll
Posts: 638
Joined: Mon Dec 15, 2008 6:14 pm
Location: Istanbul, Turkey
x 42

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by stealth977 »

xadhoom wrote:Hi Noman!

By reading your proposal I´m surprised that you don´t mention the possibilities which arise for "deferred shading".
About the Ogre example it is often said that its not done the right way and as far as I know chaining compositors is exactly the way it should be done. Maybe it would be very attractive for the community (at least for me) if your work would lead to a new "Deferred Shading Example" for Ogre which clearly shows the functionality the new compositor framework provides when you have finished the work.

xad
BTW, the current state of major Game Titles show that deferred shading is the way to go (they at least use a hybrid method). Also the GPU manufacturers are adding features to ease the system.

One of my main goals is to provide deferred shading in Ogitor too, so i am looking forward to changes in the compositor framework...

ismail,
Ismail TARIM
Ogitor - Ogre Scene Editor
WWW:http://www.ogitor.org
Repository: https://bitbucket.org/ogitor
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

Erm...
Deferred shading is THE leading usecase for this project. While I'm not adding 'one click' deferred shading to Ogre, many of the changes are geared toward making deferred rendering much easier to setup and control, and giving more modular options to the user. Some of the conversations in this thread are strictly about deferred shading.
The final currently planned stage is a showcase example, a refactoring of the deferred shading demo to work with the new architecture. It will contain shadows and post-processing options (that will be toggle-able) that use the Gbuffer as their input (like SSAO).
User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany
x 1

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by xadhoom »

Ah I missed your first post :roll:

I´m eager to see how this will evolve :P

xad
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

Thanks!

Some design consideration updates.
I've been thinking about how I'm going to handle the compositor/code binding. A while back I wrote about the Compositor Logic Framework, which worked well but was a bit limited. It was implicit (didn't show up in the compositor script, so you had no way of raising an error if the expected plugin does not exist) and only worked from DLL's, so you had to have an exact-named plugins in the directory for it to work.

Instead of implementing this in a plugin-like way, I'm thinking for a factory pattern.
The CompositorManager will have a 'registerCompositorListener' method that will receive the name of the listener and a pointer to a CompositorListenerFactory (new API, pretty obvious in functionality).
The compositors (in both scripts and code) will be able to specify a named listener. When an instance of the compositor is created, a listener will be created from the registered factory and attached to the compositor... And thats it!

The advantages of this method are :
  • Simple. No dynamic loading of DLLs via naming conventions, just plain and simple code.
  • Explicit. When a compositor requires code, the compositor 'announces' it. When taking a compositor from a sample, you will know that there is some code attached.
  • Error checking. If the code that complements a compositor is missing, instead of not functioning correctly, the compositor will fail to load.
  • Flexible. The named listeners can be shared between different compositors, and they can be registered in code or plugins (including multiple listeners in a single plugin, etc).
The disadvantage (only one I can think of right now), is that contrary to the CompositorLogic Framework, the plugins will not be loaded on demand when the compositor is used for the first time. I don't think its a big drawback at all.

The resulting scripts would look something like :

Code: Select all

compositor MyCompositor
{
    ... (texture definitions)
    ... (render passes)

    named_listener MyCompositorListener
}
There is also a possibility of adding param passing to the listener (using similar curly bracket syntax or such), but I'm not sure if its necessary in the first stage of the new interface.

Thoughts?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Assaf Raman »

I like the idea of declaring, but I think that "named_listener" should be "Compositor_Logic".
The reason I don't want the "declare listener" idea is that it is inconsistent to our scripts naming convention.
The logic should be a resource, smiler to a shader.
In the future I want an option to have different types of logic resources - meaning cross platform logic in LUA or other scripting languages and such.
Watch out for my OGRE related tweets here.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Praetor »

I don't see that as a drawback at all. Basically, compositor logic classes would act as normal Ogre plugins. Just like you request a render system, and its DLL is not auto-loaded, I would not expect auto-loading of compositor logic either. Doing it the way you are proposing makes the logic code fit into the current Ogre conventions very well.

I also agree with Assaf about the script name. Small detail, to be sure, though.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

OK...
So besides names, the question is should the management of CompositorLogics be as a simple registration function like I wrote, or another type of Resource? Having them as a resource means a new resource manager etc (Introduction of CompositorLogicManager etc), but it's still pretty much the same.

I started looking deeper into the compositor code to start designing the cross-compositor referencing, and thinking about how to implement this.
First question I'm asking (myself, mostly) is will we support defining textures as BOTH public (available to other compositors in the chain) and shared? It is theoretically possible, but makes the job of keeping the shared texture pool a tougher one. I will have to make sure that a texture doesnt get reused after its been filled but before one of the referencing compositors uses it. Possible task, but requires quite a bit of code...
However, it is important to be able to share textures to an extent. I want to reuse textures from multiple viewports - so if two same-sized viewports are using deferred shading. They'll use the same set of MRTs for the GBuffer. Seeing as the GBuffer is 16MB in the common cases, its pretty important to reuse it. Perhaps a slightly different scope of texture sharing should be introduced for this case?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by sinbad »

Yes, I think you need to allow a compositor texture to be both public AND shared separately, and perhaps simultaneously - the 2 concepts are orthogonal, and there are very real reasons why you'd want a surface to be both in some cases.

I think a close examination of the purposes, and maknig sure they are clearly defined is key here.

'shared' (as currently defined)
As things stand, 'shared' means 'share the space, but not the content'. A surface can be re-used within the compositor (provided there isn't a dependency issue), across active compositors on the same and different viewports, and when switching from one compositor technique to another or otherwise turning on / off compositors. The key facility 'shared' brings is performance and memory efficiency. Content of the surface is considered discardable unless dependency is explicitly indicated within one compositor, which is why it has to be turned on explicitly.

'public' (as proposed)
This means 'share the content'. That means that the scope of that content being important is now much less easy to define. However, you still might want to share the surfaces for efficiency purposes - for example if you only used the 'public' surface for other compositors on the same viewport, then these surfaces should be available for other compositors on other viewports to re-use, if the sizes and other features match. Therefore, a combination of both 'public' and 'shared' is feasible.

I actually wonder if 'public' is actually misleading as a term. There seem to be actually 2 states - 'public to all active compositors on the same viewport (locally public)' - probably the most common set-up, and 'public to any compositor on any viewport (globally public)' - less common I would guess. When you combine 'shared' with each of these, the limitations become clearer I think: for simplicity 'locally public' surfaces would never be sharable within the viewport, but could be sharable with other viewports, and also if a compositor technique was switched on the same viewport, the surfaces could be re-used. 'globally public' surfaces would probably never be sharable.

This also defines the scope of the validity of the content in those surfaces too. 'locally public' surfaces will always keep their content valid within the sequence for that viewport, but the contents cannot be guaranteed after that - this is what makes them sharable. The content in 'globally public' surfaces would always have to be valid, making them basically just like a regular unique render texture.
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

Yea... Thats what I thought about. I started thinking about identifying situations in which locally public textures could be reused within the same chain, but a lot of code will be involved, and it might not be worth the risk. I'm thinking about using this scheme exactly. By the way - Is there any reason for chain-scoped textures to NOT be shared amongst different viewports? What was motive behind adding the 'shared' keyword for textures and not just sharing them whenever it doesn't affect the final result?

As for scopes, I originally also thought mainly about 'chain-scope' textures and not 'global-scope' textures, but then I saw this thread in the feature requests forums which essentially, is asking for the 'global-scope' functionality.

Naming is indeed a slight problem with this project, choosing the correct keywords in scripts etc, but that can easily be changed as long as its done before its merged into trunk and people actually start using it.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by sinbad »

Noman wrote:Is there any reason for chain-scoped textures to NOT be shared amongst different viewports? What was motive behind adding the 'shared' keyword for textures and not just sharing them whenever it doesn't affect the final result?
I'm not sure what you mean - 'shared' textures are shared via the CompositorManager which means they are already shared across different viewports. The only problem comes when they're shared in the same viewport in fact, since it's at this stage that content dependencies are potentially a problem.

[edit]Oh I see what you mean. The reason I didn't default 'shared' on is that some people already use the contents created in a compositor elsewhere - e.g. they might set up a compositor, then pull out one of the intermediate stages and wire it up to something else, like a manual render or something else. I know, I've worked on projects that do this :) Sometimes it's essentially a manual version of what you're proposing in this project, but sometimes it's for something more specific. Basically, I didn't want to break these peoples projects by making the content 'discardable' as soon as *we* thought it was - in practice we don't have perfect knowledge of the lifecycle people depend on for compositor texture content so I played it safe and made it something they had to turn on. Debugging cases where the content suddenly started getting overwritten at a different stage to what it normally did would be an absolute nightmare for users.
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

Ok, fair enough.

So, compositor textures will have two different parameter types - one will be the sharing flag (this will be the same flag that is already present, but the implementation might change depending on the other param) and the other will be the scope flag, which will be either compositor-scope (this is the current mode of work, and will be the default scope), chain-scope (the common use case for the techniques i'm targeting) and global-scope (one texture for everyone, as requested in the forums).

Perhaps the sharing flag needs to be renamed with these changes, since 'sharing' becomes a slightly ambiguous term (the scoping is also sort of sharing the texture with other compositors). I would suggest something like 'pooled' to differ from the inter-compositor communication and still be self-explanatory, but again, the actual names used in scripts aren't pressing issues etc.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by sinbad »

Agreed, with the addition of this feature 'shared' might be ambiguous. 'pooled' is probably better since it suggests that it's the memory that's being shared, not the content.
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Submitted] Improve Ogre's Compositor Framework

Post by Noman »

Project accepted!

http://socghop.appspot.com/org/home/goo ... jstreeting

This makes it even more important for me to get feedback from the community about this project. If you are a heavy user of compositors (or avoided using it because of shortcomings), please let me know what you think and what features you would like to see!
User avatar
omniter
OGRE Contributor
OGRE Contributor
Posts: 424
Joined: Thu Mar 19, 2009 8:08 am
Location: Canada
x 44

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by omniter »

Congrats, Noman! Can't wait to see how this turns out. :D
User avatar
subquantum
Goblin
Posts: 270
Joined: Tue Oct 02, 2007 10:23 pm
Location: Rochester, NY
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by subquantum »

Indeed, congratulations! I'll definitely be following the progress on this project, as I've been playing with compositors and working towards a sort-of hybrid deferred system for a while now, and coming up with the same frustrations and workarounds the others in this thread have been discussing. I'd be glad to help out with suggestions or testing, although I'm far from being an experienced, wizardly Ogre hacker (yet) :)
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by Assaf Raman »

Congrats.
Now comes the hard part.
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by sinbad »

Congrats, here's where the work starts though ;)

Here's your wiki page for the year: http://www.ogre3d.org/wiki/index.php/SoC2009_Compositor
And your development branch is here: https://svn.ogre3d.org/svnroot/ogre/bra ... compositor

You obviously know the drill from last year - please update the wiki page with your consolidated notes & plans so it's the 'current reference' on your thinking based on both the application and the discussions since, and feel free to start using the SVN branch whenever you like (your login is still valid).
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by Noman »

Cool.

I'll probably get to putting all the info on the wiki page properly in the weekend. I'll also use it as an opportunity to merge all the insights from the conversations in this thread to the actual plan, and work from there. I want to get the inter-compositor communication design down to a pretty exact plan before the coding time starts (yea, I can theoretically start before that, but its probably a good idea to design first..)
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by Noman »

Wiki page is now updated with all the latest good stuff.

I also added a Target Design section in the wiki page, which consists of the pseudo-code and pseudo-scripts that are planned to be possible at the end of the project. They illustrate the design decisions and explain how these features will be used. Its still in progress, but already has some information there. Since this project is mainly about exposing new options for better pipeline control, I think that this is a very important design aspect (if not THE most important design aspect) of the project.

Feel free to have a look...
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by sinbad »

Nice. Re the proposed script format, the only detail comment I have is that although the '/' character is a very natural delimiter between the compositor name and the local texture name, it's used a lot already in the names of compositors, and (AFAIK) they're not prohibited characters in the local texture names either. So I'm not sure you could easily differentiate between what is just a name with a slash in it, and what is an external reference.

In fact, to make the whole syntax easier, I'd suggest introducing a new "local" texture definition line, which pulls in a texture from another compositor and then makes it available such that it 'looks' like a local texture. This makes writing scripts which use that same texture multiple times much easier, while also resolving the ambiguity on the naming. E.g.

Code: Select all

shared_texture <src_compositor> <src_name> <local_name>

Code: Select all

shared_texture DeferredPass output_tex deferredMRT
Then you can use 'deferredMRT' as if it were a local texture (including the MRT surface number).
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: [SoC 2009 - Accepted] Improve Ogre's Compositor Framework

Post by sinbad »

Actually 'texture_ref' would probably be a term that's more consistent with our other scripts rather than shared_texture.
Locked