[GSoC 2011 - Accepted] Modern Illumination Techniques
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
[GSoC 2011 - Accepted] Modern Illumination Techniques
Personal Details
Name: Andrei Radu
Email: andrei.radu@siggraph.org
OGRE Forum username: andrei_radu
Project Proposal
My proposal focuses on implementing several modern illumination techniques, that are either related to, or benefit from deferred shading. Most of these have been used in several shipped games, while a
couple are still considered 'bleeding edge'. My list of features is prioritized, and I tried to make sure that each milestone delivers an usable and thoroughly documented piece of code.
Light Pre-pass
This technique is an alternative to deferred shading, promising lower memory bandwidth requirements and a somewhat higher material system flexibility, at the expense of adding another geometry pass(which is relatively cheap). It was first introduced by Wolfgang Engel via his blog[1], and after developed trough several articles in the ShaderX and Gpu Pro series.
The rendering process starts by rendering normals and depth information into a slim G-Buffer(the normals can be stored in polar coordinates, which leaves 16 bits for depth; this should be enough
precision, and requires a single RGBA8 render target; we use a 16 bit floating point render target for clarity). Next, for each light in the scene, N*L*light_color and RV^n(for the phong model) are rendered into a light buffer(also a single RGBA8 target, or a RGBA16F to support HDR). A second geometry pass is performed(while keeping the Z-Buffer from the first pass, thus there
is no over-draw), and the lighting equation is reconstructed, using the material info.
This technique has been used in several shipped games, and a rather impressive demo, involving a large number of light sources, can be found at[2]. A larger performance gain is achieved when some post-processing effects like motion blurred or precomputed ambient occlusion are performed(and thus would require a larger G-Buffer in a classical deferred shading approach).
In my implementation, the alpha channel of the light buffer contains the specular term raised at a light-specific value, and multiplied by NL and attenuation. The specular term(NH) is recalculated at merging by division with the luminance of the diffuse channel.
Inferred Lighting[3]
This builds on the previous technique and tries to use smaller resolution buffers. The main issue that arises is that rope-like artifacts appear on object edges(due to sampling issues). The proposed solution is using another buffer that stores both linear depth and a semi-unique ID of that surface(two adjacent surfaces shouldn't have the same ID). This is used in the final material pass to bias the bilinear filtering that would occur due to sampling a smaller light buffer.
The main purpose for this is reducing the cost of illumination, since lighting is performed at a lower resolution.I used a light buffer with 0.75 the height and 0.75 the width of the viewport, so a ~44% reduction in lighting calculations. Artifacts are present along curve surfaces, and lighting detail on pixel sized objects or highlights is lost(temporal aliasing occurs due to this as well).
Another important issue is that performance can be easily traded for visual accuracy(we simply need to use smaller or larger light/G buffer). To the extent of my knowledge, this has not yet been used in a shipped game.
Reflective Shadow Maps
Reflective shadow maps are used, in this implementation, as a form of instant radiosity. The purpose is to provide one bounce global illumination for diffuse surfaces. The technique starts with the idea that every point that would give one bounce GI is captured by the light's shadowmap. So we store extra information in the shadowmap(world-space position, normal and flux - basically the ammount of energy that reaches that point), and generate Virtual Point Lights(VPLs) based on that. We then use these pointlights to light the scene. Since several hundred point lights are needed to get good results, using a deferred technique is mandatory.
Light Propagation Volumes[4]
This is a global illumination approximation technique, that can be used for low frequency lighting situations. The first step is partitioning the scene in a grid. Normally, the grid would have varying cell size, but for my implementation I will only implement a fixed size grid. The scene is then rendered into a set of Reflective Shadow Maps Each VPL's position is then determined, and converted to a spherical harmonics representation[5]. The VPL's in each cell are then 'accumulated'(we perform a summation in SH space). Low frequency direct illumination can be approximated as well by creating VPL's from environment maps or area lights.
Next the depth and surface normal at each point is used to reconstruct a coarse approximation of the scene's geometry(surface information stored in the RSMs is also used; additional information can be
obtained by using depth peeling). This geometry information is used per cell to obtain occlusion data for each incoming direction. This is also converted into SH space and accumulated.
Implementation Details: The whole technique can be implemented as a single compositor. The grids will consist of flattened 3D textures(2D textures that contain several slices) and a custom rendering pass will be used to inject the RSMs into it(sample the RSM and render the point lights directly in SH coordinates). Light propagation is computed in a fixed number of passes, using ping-pong buffers(in each pass each cell looks at it's six neighbors).
Two open-source stand-alone implementations exist at [6]and [7].
Demos
The demo extends the deferred shading demo with a drop-down that selects the lighting technique used. The options will be: forward rendering, deferred shading, deferred lighting, inferred lighting, spherical harmonics lighting and light propagation volumes. Additionally, when selecting deferred shading, deferred lighting or inferred lighting, there will be an option to activate Reflective Shadow Maps. Debugging outputs are also extended to allow the viewing of the light buffer, for deferred lighting and inferred lighting, and object IDs for inferred lighting.
Schedule
The only other commitment I have this summer are my third year exams, and I'll finish those around the 10th of June. That gives me roughly 10 weeks of actual work.
• Preparation period – Find out what the community expects from this project. Get familiar with the inner workings of the compositor framework.
• [done] Weeks 1-[s]2[/s] 4(June 12th)– Implement and optimize Light Pre-Pass and Inferred lighting
•[s]Week 3(June 26th)Implement Depth Peeling(useful for both LPV and SSDO)[/s]
•[s]Week 4(July 3rd)Implement SSDO[/s]
•Week 5(July 10th)Implement the rendering of 'fat' shadow maps
• Week 6(July 17th)Implement the RSM sampling and generating of VPLs - VPLs will work as a special kind of deferred lights(they provide light over an hemisphere); after each shadow casting light is processed and it's shadow map generated, vpl's for that light are generated and rendered. The number of VPLs will depend on scene settings and the light's intensity.
•Week 7(July 24th) Tweak VPL generation to obtain good visuals & solve issues - I expect issues due to the relative small number of VPLs that will be generated for real time performance to be achievable. Each VPL's contribution needs to be clamped to avoid unnatural highlights. Implement Spherical Harmonic Lighting (this is a step of implementing LPV) - will generate a set of coeficients for the cathedral scene and use them for (directional only) lighting.
•Week 8 - 9(July 31st) Build a spacial grid containing the scene and render the geometry into it - this involves rendering each shadow map into a 'flat' 3D texture
•Week 10(August 14th)''Implement propagation & occlusion between grid cells, and generate the SH coefficients for each of them. Use these coefficients to light the scene.
Some further development ideas:
• The specular term in deferred lighting can be computed in other ways with various tradeoffs(an extra render target can be used, or the diffuse term can be compressed).
• LPV can be optimized by allowing cascading grids(recursively subdivided; think of a octree)
• The SH representation from LPV can be used as a basis for further development in SSDO(instead of accessing a cube-map for each direction, we could use the per-cell representation)
Why I'm The Person For This Project
I am a computer science student, in my third year, and the courses I followed span from object oriented programming to software engineering, to computer graphics and computer architectures. I consider
myself to be well prepared in all of these fields, especially since I have taken a special interest in them.
I'm also quite a talkative and sociable person, and enjoy working with others. I am currently involved in a large scale virtual reality project that uses an Ogre based client. You can find more about my work from my blog[9] or my projects page[10].
Why OGRE?
I have a great interest in computer graphics and engine design, and hope to work in the field in the near future and has helped me improve in both fields. I also learned a lot from Ogre, both in coding style and in terms of engine design(I often find myself skimming trough open source engines' doxygen documentation to see how different modules work together).
Anything Else
My proposal might look a little ambitious, or unfeasible, but I believe that I have the needed skills and motivation to implement it. Plus, I don't have anything planned until early September
[1]http://diaryofagraphicsprogrammer.blogs ... derer.html
[2]http://www.confettispecialfx.com/river- ... ts#more-92
[3]http://graphics.cs.uiuc.edu/~kircher/in ... _paper.pdf
[4]http://www6.incrysis.com/Light_Propagation_Volumes.pdf
[5]Reflective Shadow Maps
[6]http://www.ppsloan.org/publications/StupidSH36.pdf
[7]http://lee.fov120.com/lpv.zip
[8]http://blog.blackhc.net/wp-content/uplo ... totype.zip
[9]http://andreiradu.blogspot.com
[10]http://graphics.cs.pub.ro/~andreiradu
Andrei
Name: Andrei Radu
Email: andrei.radu@siggraph.org
OGRE Forum username: andrei_radu
Project Proposal
My proposal focuses on implementing several modern illumination techniques, that are either related to, or benefit from deferred shading. Most of these have been used in several shipped games, while a
couple are still considered 'bleeding edge'. My list of features is prioritized, and I tried to make sure that each milestone delivers an usable and thoroughly documented piece of code.
Light Pre-pass
This technique is an alternative to deferred shading, promising lower memory bandwidth requirements and a somewhat higher material system flexibility, at the expense of adding another geometry pass(which is relatively cheap). It was first introduced by Wolfgang Engel via his blog[1], and after developed trough several articles in the ShaderX and Gpu Pro series.
The rendering process starts by rendering normals and depth information into a slim G-Buffer(the normals can be stored in polar coordinates, which leaves 16 bits for depth; this should be enough
precision, and requires a single RGBA8 render target; we use a 16 bit floating point render target for clarity). Next, for each light in the scene, N*L*light_color and RV^n(for the phong model) are rendered into a light buffer(also a single RGBA8 target, or a RGBA16F to support HDR). A second geometry pass is performed(while keeping the Z-Buffer from the first pass, thus there
is no over-draw), and the lighting equation is reconstructed, using the material info.
This technique has been used in several shipped games, and a rather impressive demo, involving a large number of light sources, can be found at[2]. A larger performance gain is achieved when some post-processing effects like motion blurred or precomputed ambient occlusion are performed(and thus would require a larger G-Buffer in a classical deferred shading approach).
In my implementation, the alpha channel of the light buffer contains the specular term raised at a light-specific value, and multiplied by NL and attenuation. The specular term(NH) is recalculated at merging by division with the luminance of the diffuse channel.
Inferred Lighting[3]
This builds on the previous technique and tries to use smaller resolution buffers. The main issue that arises is that rope-like artifacts appear on object edges(due to sampling issues). The proposed solution is using another buffer that stores both linear depth and a semi-unique ID of that surface(two adjacent surfaces shouldn't have the same ID). This is used in the final material pass to bias the bilinear filtering that would occur due to sampling a smaller light buffer.
The main purpose for this is reducing the cost of illumination, since lighting is performed at a lower resolution.I used a light buffer with 0.75 the height and 0.75 the width of the viewport, so a ~44% reduction in lighting calculations. Artifacts are present along curve surfaces, and lighting detail on pixel sized objects or highlights is lost(temporal aliasing occurs due to this as well).
Another important issue is that performance can be easily traded for visual accuracy(we simply need to use smaller or larger light/G buffer). To the extent of my knowledge, this has not yet been used in a shipped game.
Reflective Shadow Maps
Reflective shadow maps are used, in this implementation, as a form of instant radiosity. The purpose is to provide one bounce global illumination for diffuse surfaces. The technique starts with the idea that every point that would give one bounce GI is captured by the light's shadowmap. So we store extra information in the shadowmap(world-space position, normal and flux - basically the ammount of energy that reaches that point), and generate Virtual Point Lights(VPLs) based on that. We then use these pointlights to light the scene. Since several hundred point lights are needed to get good results, using a deferred technique is mandatory.
Light Propagation Volumes[4]
This is a global illumination approximation technique, that can be used for low frequency lighting situations. The first step is partitioning the scene in a grid. Normally, the grid would have varying cell size, but for my implementation I will only implement a fixed size grid. The scene is then rendered into a set of Reflective Shadow Maps Each VPL's position is then determined, and converted to a spherical harmonics representation[5]. The VPL's in each cell are then 'accumulated'(we perform a summation in SH space). Low frequency direct illumination can be approximated as well by creating VPL's from environment maps or area lights.
Next the depth and surface normal at each point is used to reconstruct a coarse approximation of the scene's geometry(surface information stored in the RSMs is also used; additional information can be
obtained by using depth peeling). This geometry information is used per cell to obtain occlusion data for each incoming direction. This is also converted into SH space and accumulated.
Implementation Details: The whole technique can be implemented as a single compositor. The grids will consist of flattened 3D textures(2D textures that contain several slices) and a custom rendering pass will be used to inject the RSMs into it(sample the RSM and render the point lights directly in SH coordinates). Light propagation is computed in a fixed number of passes, using ping-pong buffers(in each pass each cell looks at it's six neighbors).
Two open-source stand-alone implementations exist at [6]and [7].
Demos
The demo extends the deferred shading demo with a drop-down that selects the lighting technique used. The options will be: forward rendering, deferred shading, deferred lighting, inferred lighting, spherical harmonics lighting and light propagation volumes. Additionally, when selecting deferred shading, deferred lighting or inferred lighting, there will be an option to activate Reflective Shadow Maps. Debugging outputs are also extended to allow the viewing of the light buffer, for deferred lighting and inferred lighting, and object IDs for inferred lighting.
Schedule
The only other commitment I have this summer are my third year exams, and I'll finish those around the 10th of June. That gives me roughly 10 weeks of actual work.
• Preparation period – Find out what the community expects from this project. Get familiar with the inner workings of the compositor framework.
• [done] Weeks 1-[s]2[/s] 4(June 12th)– Implement and optimize Light Pre-Pass and Inferred lighting
•[s]Week 3(June 26th)Implement Depth Peeling(useful for both LPV and SSDO)[/s]
•[s]Week 4(July 3rd)Implement SSDO[/s]
•Week 5(July 10th)Implement the rendering of 'fat' shadow maps
• Week 6(July 17th)Implement the RSM sampling and generating of VPLs - VPLs will work as a special kind of deferred lights(they provide light over an hemisphere); after each shadow casting light is processed and it's shadow map generated, vpl's for that light are generated and rendered. The number of VPLs will depend on scene settings and the light's intensity.
•Week 7(July 24th) Tweak VPL generation to obtain good visuals & solve issues - I expect issues due to the relative small number of VPLs that will be generated for real time performance to be achievable. Each VPL's contribution needs to be clamped to avoid unnatural highlights. Implement Spherical Harmonic Lighting (this is a step of implementing LPV) - will generate a set of coeficients for the cathedral scene and use them for (directional only) lighting.
•Week 8 - 9(July 31st) Build a spacial grid containing the scene and render the geometry into it - this involves rendering each shadow map into a 'flat' 3D texture
•Week 10(August 14th)''Implement propagation & occlusion between grid cells, and generate the SH coefficients for each of them. Use these coefficients to light the scene.
Some further development ideas:
• The specular term in deferred lighting can be computed in other ways with various tradeoffs(an extra render target can be used, or the diffuse term can be compressed).
• LPV can be optimized by allowing cascading grids(recursively subdivided; think of a octree)
• The SH representation from LPV can be used as a basis for further development in SSDO(instead of accessing a cube-map for each direction, we could use the per-cell representation)
Why I'm The Person For This Project
I am a computer science student, in my third year, and the courses I followed span from object oriented programming to software engineering, to computer graphics and computer architectures. I consider
myself to be well prepared in all of these fields, especially since I have taken a special interest in them.
I'm also quite a talkative and sociable person, and enjoy working with others. I am currently involved in a large scale virtual reality project that uses an Ogre based client. You can find more about my work from my blog[9] or my projects page[10].
Why OGRE?
I have a great interest in computer graphics and engine design, and hope to work in the field in the near future and has helped me improve in both fields. I also learned a lot from Ogre, both in coding style and in terms of engine design(I often find myself skimming trough open source engines' doxygen documentation to see how different modules work together).
Anything Else
My proposal might look a little ambitious, or unfeasible, but I believe that I have the needed skills and motivation to implement it. Plus, I don't have anything planned until early September
[1]http://diaryofagraphicsprogrammer.blogs ... derer.html
[2]http://www.confettispecialfx.com/river- ... ts#more-92
[3]http://graphics.cs.uiuc.edu/~kircher/in ... _paper.pdf
[4]http://www6.incrysis.com/Light_Propagation_Volumes.pdf
[5]Reflective Shadow Maps
[6]http://www.ppsloan.org/publications/StupidSH36.pdf
[7]http://lee.fov120.com/lpv.zip
[8]http://blog.blackhc.net/wp-content/uplo ... totype.zip
[9]http://andreiradu.blogspot.com
[10]http://graphics.cs.pub.ro/~andreiradu
Andrei
Last edited by andrei_radu on Wed Jul 13, 2011 11:59 am, edited 10 times in total.
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- Kobold
- Posts: 36
- Joined: Thu Sep 25, 2008 10:34 am
- Location: Russia. Nakhodka.
- x 4
Re: [GSoC 2011] Inferred Lighting
Post more details. It very interesting.
-
- OGRE Retired Moderator
- Posts: 2653
- Joined: Wed Sep 24, 2003 8:07 am
- Location: Haute Garonne, France
- x 4
Re: [GSoC 2011] Inferred Lighting
maybe a bit too small scope ?
Lights techniques that could be demoed in a sample app: pre pass/inferred, deferred, tiled (dice papers), LPV (crysis papers), etc...
Lights techniques that could be demoed in a sample app: pre pass/inferred, deferred, tiled (dice papers), LPV (crysis papers), etc...
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011] Inferred Lighting
I still need to figure out a couple of things before I decide what my demo will do. For example, I would like to see if I can add MSAA support(there's a neat article on doing that in shaderX^7, and it only requires some directX10.1 support). As I said in the first post, I'd like to see if I can use two different lighting models in the same scene(I was thinking of using a multi-pass approach, and leveraging the stencil buffer, as opposed to using dynamic branching).
I think the Light Propagation Volumes technique is a bit too complex to be squeezed in with a bunch of other stuff.
I'm not familiar with the dice papers, unless they're the ones that talk about splitting the screen in tiles, and computing which light affects which tile.
As for the light pre-pass, sure, the main difference between it and inferred lighting is the DSF.
I think the Light Propagation Volumes technique is a bit too complex to be squeezed in with a bunch of other stuff.
I'm not familiar with the dice papers, unless they're the ones that talk about splitting the screen in tiles, and computing which light affects which tile.
As for the light pre-pass, sure, the main difference between it and inferred lighting is the DSF.
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Moderator
- Posts: 2653
- Joined: Wed Sep 24, 2003 8:07 am
- Location: Haute Garonne, France
- x 4
Re: [GSoC 2011] Inferred Lighting
A thing to do is to check current dx10/dx11 ogre rendersystem plugin to check what's in or not, either to limit choices or to point in your application what you''ll have to add.
LPV has several opensource implementations on the net tht can ease implementation a lot.
Dice papers about splitting the screen in tiles, and computing which light affects which tiles are the ones.
LPV has several opensource implementations on the net tht can ease implementation a lot.
Dice papers about splitting the screen in tiles, and computing which light affects which tiles are the ones.
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011] Inferred Lighting
Ok, how does this sound like a list of deliverables?
Two global illumination approximation techniques: Screen space directional occlusion(link) and LPV(link
Two deferred rendering techniques: light pre-pass(link and inferred lighting(link)
One optimization: Screen space classification(link) - this is basically a generalization of tiling
Depending on the status of the directX11 render, I might attempt a MSAA implementation.
Any thoughts?
Two global illumination approximation techniques: Screen space directional occlusion(link) and LPV(link
Two deferred rendering techniques: light pre-pass(link and inferred lighting(link)
One optimization: Screen space classification(link) - this is basically a generalization of tiling
Depending on the status of the directX11 render, I might attempt a MSAA implementation.
Any thoughts?
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Moderator
- Posts: 2653
- Joined: Wed Sep 24, 2003 8:07 am
- Location: Haute Garonne, France
- x 4
Re: [GSoC 2011] Inferred Lighting
Good list.
Now you would have to go into details, for each deliverables, what path you'll follow to implement it in Ogre and what you'll have to fix/change/enhance in the compositor/shader/rendersystem in order to get it working ?
(ie: cuda/directcompute/opencl plugin, scene manager "per tile" culling and light/renderable list, etc...)
Now you would have to go into details, for each deliverables, what path you'll follow to implement it in Ogre and what you'll have to fix/change/enhance in the compositor/shader/rendersystem in order to get it working ?
(ie: cuda/directcompute/opencl plugin, scene manager "per tile" culling and light/renderable list, etc...)
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011] Inferred Lighting
My laptop is currently down(ironically the GPU crashed), so I've been stuck installing all my stuff on a friend's laptop, so I didn't get to do a lot of work these couple of days. I have, however, edited the first post, detailing my proposal a bit, and giving some personal info.
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Team Member
- Posts: 714
- Joined: Mon Jan 31, 2005 7:21 pm
- Location: Israel
- x 2
Re: [GSoC 2011] Inferred Lighting
I think that a main point that you need to think about (and shape your application accordingly) is this :
How does this SoC project benefit the OGRE project and its users as a whole, rather than just being a cool demo?
I think this question should be an important one that affects decisions you make even at an early point in the project, and should definitely be addressed directly in the application.
How does this SoC project benefit the OGRE project and its users as a whole, rather than just being a cool demo?
I think this question should be an important one that affects decisions you make even at an early point in the project, and should definitely be addressed directly in the application.
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011] Inferred Lighting
Thanks for the pointer, updated the first post.
I have a question regarding the current deferred shading implementation. Why isn't it implemented as a separate renderer plugin(as in having directX, ogl, directX-deferred,etc)? What would be the downside of such an approach(besides added complexity in implementation)?
I have a question regarding the current deferred shading implementation. Why isn't it implemented as a separate renderer plugin(as in having directX, ogl, directX-deferred,etc)? What would be the downside of such an approach(besides added complexity in implementation)?
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- Goblin
- Posts: 287
- Joined: Mon Dec 08, 2008 4:49 pm
- x 10
Re: [GSoC 2011] Inferred Lighting
Deferred Shading is a method, where DirectX and OpenGL are RenderSystems. Basically, the rendersystem's task is to convert Ogre's data in such a way that the 3D drivers understand it. What you want to do is one level higher. I think you can generalize it in these layers:
Having deferred shading as a plugin is a good Idea, but I don't think it belongs on the level of render systems.
Btw, I already made a start with creating a separate plugin, see this post
- Game / Scene management : creating, moving, destroying meshes, lights, etc
- rendering management : compositors, deferred shading, selecting material scheme
- Render System: OpenGL, DirectX, GLES
Having deferred shading as a plugin is a good Idea, but I don't think it belongs on the level of render systems.
Btw, I already made a start with creating a separate plugin, see this post
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011] Inferred Lighting
Sent my proposal trough google, and updated the first post. Removed tiling due to time constraints and to the fact that it's an optimization mainly aimed at consoles. Also changed the topic name to better reflect the content of the proposal.
Any thoughts?
Any thoughts?
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Moderator
- Posts: 2653
- Joined: Wed Sep 24, 2003 8:07 am
- Location: Haute Garonne, France
- x 4
Re: [GSoC 2011]Modern Illumination Techniques
very nice proposal, well written.
Only missing parts are more details on necessary Ogre modifications (compositor/scene management/etc.) so that you can get a more detailed planning.
Ogre compositor and scene management definitely some improvement/optimisation on the deferred shading side. Compositor are not very suited for it, and many resort to their own ogre "compositr/deferred" code instead
=> http://www.ogre3d.org/forums/viewtopic. ... 232#p41981
=> http://www.ogre3d.org/tikiwiki/Deferred ... get_passes_
=> http://liquidrockgames.blogspot.com/201 ... roach.html <= blog post serie on deferred in Ogre from mkultra333
About "deferred" render system plugin, I think there is something really interesting there.
I think it's more about adding a level up of abstraction, with a rendering_scheme plugin on top of existing render system plugins, which would end in having a "forward" rendering scheme plugin, and a deferred rendering scheme plugin (and for a future "tiled" plugin). It would help decoupling scene/render code that are actually pretty mixed are hard to code/maintain/debug, specially when using compositor/shadow/deferred.
That would definitely help solve the "render_scene" compositor pass reuse, material_scheme at the target scope and not at the pass scope, and many other compositor, material and scene management that are right on the way of deferred shading. (thing is, plugin path might be ill suited because of the possibility of having both deferred and forward at the same time, but different and decoupled code path still seems necessary)
( thinking, pointing about work to do, and proposing would definitely address the " How does this SoC project benefit the OGRE project and its users as a whole, rather than just being a cool demo? ")
Only missing parts are more details on necessary Ogre modifications (compositor/scene management/etc.) so that you can get a more detailed planning.
Ogre compositor and scene management definitely some improvement/optimisation on the deferred shading side. Compositor are not very suited for it, and many resort to their own ogre "compositr/deferred" code instead
=> http://www.ogre3d.org/forums/viewtopic. ... 232#p41981
=> http://www.ogre3d.org/tikiwiki/Deferred ... get_passes_
=> http://liquidrockgames.blogspot.com/201 ... roach.html <= blog post serie on deferred in Ogre from mkultra333
About "deferred" render system plugin, I think there is something really interesting there.
I think it's more about adding a level up of abstraction, with a rendering_scheme plugin on top of existing render system plugins, which would end in having a "forward" rendering scheme plugin, and a deferred rendering scheme plugin (and for a future "tiled" plugin). It would help decoupling scene/render code that are actually pretty mixed are hard to code/maintain/debug, specially when using compositor/shadow/deferred.
That would definitely help solve the "render_scene" compositor pass reuse, material_scheme at the target scope and not at the pass scope, and many other compositor, material and scene management that are right on the way of deferred shading. (thing is, plugin path might be ill suited because of the possibility of having both deferred and forward at the same time, but different and decoupled code path still seems necessary)
( thinking, pointing about work to do, and proposing would definitely address the " How does this SoC project benefit the OGRE project and its users as a whole, rather than just being a cool demo? ")
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011]Modern Illumination Techniques
First off thanks for the feedback.
I plan to implement deferred lighting slightly differently than mkultra333 presented in his post(in that I only plan to use one render target for the G-buffer since there are a couple of ways of approximating the specular power).
Do you mean something like scene manager calls this rendering_scheme plug-in that in turn calls the render system? If so, I'm on board with that. Where would the compositor fit in? And by the way, where does the scene traversal take place? I've been looking trough the source code but wasn't able to find it.
Also changed the render formats in my proposal to 16f; totally forgot about HDR.
I plan to implement deferred lighting slightly differently than mkultra333 presented in his post(in that I only plan to use one render target for the G-buffer since there are a couple of ways of approximating the specular power).
Could you please elaborate on this?I think it's more about adding a level up of abstraction, with a rendering_scheme plugin on top of existing render system plugins, which would end in having a "forward" rendering scheme plugin, and a deferred rendering scheme plugin (and for a future "tiled" plugin). It would help decoupling scene/render code that are actually pretty mixed are hard to code/maintain/debug, specially when using compositor/shadow/deferred.
Do you mean something like scene manager calls this rendering_scheme plug-in that in turn calls the render system? If so, I'm on board with that. Where would the compositor fit in? And by the way, where does the scene traversal take place? I've been looking trough the source code but wasn't able to find it.
Also changed the render formats in my proposal to 16f; totally forgot about HDR.
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Moderator
- Posts: 2653
- Joined: Wed Sep 24, 2003 8:07 am
- Location: Haute Garonne, France
- x 4
Re: [GSoC 2011]Modern Illumination Techniques
Could you please elaborate on this?
Do you mean something like scene manager calls this rendering_scheme plug-in that in turn calls the render system? If so, I'm on board with that. Where would the compositor fit in? And by the way, where does the scene traversal take place? I've been looking trough the source code but wasn't able to find it.
Scene traversal is done in scenemanager, traversal fills render queue, queue are sorted (render order, batching, user needs) and only then "render" pass takes place.
It's the render pass part that needs some love in regards with new techniques.
Best check with perfhud + debugger with a step by step the scene manager render on a scene with shadows + compositor so that you'll see the current state
(re-entrant/recursive calls and excessive branching in that code that makes it hard to evolve debug, difficulty to make "hash" of a render pass so that we can cache it et reuse, etc... some keywords: SceneManager::IlluminationRenderStage, SceneManager::RenderContext, RenderQueueInvocation::RENDER_QUEUE_INVOCATION_SHADOWS).
Note that it's the most complex part of Ogre, but if targeting deferred, you'll have to understand the how and why before diving in.
A good idea is to check Ogre v2.0 Tindalos notes for some future ideas&directions of Ogre : http://www.ogre3d.org/tikiwiki/TindalosNotes
-
- Gnoblar
- Posts: 15
- Joined: Fri Jul 23, 2010 6:34 pm
Re: [GSoC 2011]Modern Illumination Techniques
Hi Andrei
I like your idea for your project for GSoC.
An Ogre implementation of LPV would be really interesting. I would really like to see that.
With the SSDO stuff I am not quite sure. It is a good technique which produces an awesome effect. But it combines the sampling of the occlusion with the calculation of the lighting, by calculating the influence of the lighting for each un-occluded direction. You do this either by sampling an env. map or by calculating it directly for given light sources.
What is really cool about SSDO is the fact that you can use the sampling to calculate first bounce GI. That is what most people implement when they include SSDO in their pipeline.
The idea of using SH instead of env. maps is quite good.
Actually I am just finishing my masters thesis, which implemented the idea of the directional occlusion from SSDO into SH-occlusion sampling. So you have The SH function of the local directional occlusion for each pixel calculated through a screen space sampling method.
This SH function is then used by a SH lighting algorithm to calculate the lighting of each pixel.
This is maybe the same idea you had. I will post some screen shoots an some demo code in the next days when I am done writing my thesis. If you want I could help you out with some code.
Cheers
Sebastian
I like your idea for your project for GSoC.
An Ogre implementation of LPV would be really interesting. I would really like to see that.
With the SSDO stuff I am not quite sure. It is a good technique which produces an awesome effect. But it combines the sampling of the occlusion with the calculation of the lighting, by calculating the influence of the lighting for each un-occluded direction. You do this either by sampling an env. map or by calculating it directly for given light sources.
What is really cool about SSDO is the fact that you can use the sampling to calculate first bounce GI. That is what most people implement when they include SSDO in their pipeline.
The idea of using SH instead of env. maps is quite good.
Actually I am just finishing my masters thesis, which implemented the idea of the directional occlusion from SSDO into SH-occlusion sampling. So you have The SH function of the local directional occlusion for each pixel calculated through a screen space sampling method.
This SH function is then used by a SH lighting algorithm to calculate the lighting of each pixel.
This is maybe the same idea you had. I will post some screen shoots an some demo code in the next days when I am done writing my thesis. If you want I could help you out with some code.
Cheers
Sebastian
-
- OGRE Retired Team Member
- Posts: 714
- Joined: Mon Jan 31, 2005 7:21 pm
- Location: Israel
- x 2
Re: [GSoC 2011]Modern Illumination Techniques
The application is good.
My biggest concern is definitely the timeline you set up. It feels a bit like you took all the tasks you wanted to do and divided them almost evenly across the time of the project. Do you really feel you can finish light pre-pass and inferred lighting and optimize them in two weeks? And bring them to a state usable in other projects?
I'd feel safer if there were a bit less techniques in the project and more time devoted to each one. I don't want you to address this by removing a technique from the SoC plan, but by thinking a bit deeper into the plan and asking yourself if you think you can pull everything off. If your answer is yes, then by all means go for it. However, make sure to plan in a way that you leave the Ogre community with something if you don't finish 100% of your plans.
Also, you did a good job of describing the techniques you plan to implement from a theoretical point of view, but not from a project/code point of view. How do you plan to implement them? Compositors? Manual code? A new interface?
Also, how do you plan to test/showcase your work? A demo for each technique? One unified demo? Are you going to compare it to the existing forward/deferred pipelines? This is very critical for the project.
You're in a very good direction, keep up the good work!
My biggest concern is definitely the timeline you set up. It feels a bit like you took all the tasks you wanted to do and divided them almost evenly across the time of the project. Do you really feel you can finish light pre-pass and inferred lighting and optimize them in two weeks? And bring them to a state usable in other projects?
I'd feel safer if there were a bit less techniques in the project and more time devoted to each one. I don't want you to address this by removing a technique from the SoC plan, but by thinking a bit deeper into the plan and asking yourself if you think you can pull everything off. If your answer is yes, then by all means go for it. However, make sure to plan in a way that you leave the Ogre community with something if you don't finish 100% of your plans.
Also, you did a good job of describing the techniques you plan to implement from a theoretical point of view, but not from a project/code point of view. How do you plan to implement them? Compositors? Manual code? A new interface?
Also, how do you plan to test/showcase your work? A demo for each technique? One unified demo? Are you going to compare it to the existing forward/deferred pipelines? This is very critical for the project.
You're in a very good direction, keep up the good work!
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011]Modern Illumination Techniques
@tuan kuranes:
Thanks a lot for the info I'll try to work on that in the next couple of days.
@racoon:
My point is that I already have the SH representation for each grid cell, and using that in SSDO is almost free, and should give better results than sampling an env. map. Sure, one bounce GI can be added on top of that with relative ease, but I tried focusing my efforts on LPV, which is a more involved technique.
I'd really love to read your thesis, and I'd more than welcome any sort of code sample.
@Noman:
Well, my process was a bit backwards than what you described I actually tried to fragment the main tasks into smaller ones that could be done in a similar amount of time. I work well with one week or two week milestones, since that's generally the time I have for a school assignment.
I actually _don't_ think I can implement light pre-pass and inferred in two weeks, if I start from scratch. But Ogre already has a deferred shading demo, and I've gotten quite familiar with it. I also started seeing how light pre-pass can be implemented(and inferred is just a somewhat slight modification away).
Also, although I stated that I can start coding at the beginning of June, I actually expect to get a lot of things done by then(I generally code a lot between exams to keep me in shape ), so that's why the first stage got such a short time-frame. So yes, I'm sure I can pull this off
I tried to make my schedule so that each step is something valuable in itself. Depth peeling, RSMs and LPV without glossy or diffuse lighting are still extremely useful(RSMs are a valid GI technique, and depth peeling can help a lot with transparency).
I also tried describing a coarse implementation plan, but the fact is that I'm still delving into the depths of ogre's compositor framework, so that part of my proposal is vague on purpose. I understand how things _could_ be done, but I don't yet know what the best way of doing them is.
About the demo, that's really my bad; my initial post contained some info on that, but forgot to add the info to my proposal. Will edit it in as soon as possible.
Thanks a lot for the info I'll try to work on that in the next couple of days.
@racoon:
My point is that I already have the SH representation for each grid cell, and using that in SSDO is almost free, and should give better results than sampling an env. map. Sure, one bounce GI can be added on top of that with relative ease, but I tried focusing my efforts on LPV, which is a more involved technique.
I'd really love to read your thesis, and I'd more than welcome any sort of code sample.
@Noman:
Well, my process was a bit backwards than what you described I actually tried to fragment the main tasks into smaller ones that could be done in a similar amount of time. I work well with one week or two week milestones, since that's generally the time I have for a school assignment.
I actually _don't_ think I can implement light pre-pass and inferred in two weeks, if I start from scratch. But Ogre already has a deferred shading demo, and I've gotten quite familiar with it. I also started seeing how light pre-pass can be implemented(and inferred is just a somewhat slight modification away).
Also, although I stated that I can start coding at the beginning of June, I actually expect to get a lot of things done by then(I generally code a lot between exams to keep me in shape ), so that's why the first stage got such a short time-frame. So yes, I'm sure I can pull this off
I tried to make my schedule so that each step is something valuable in itself. Depth peeling, RSMs and LPV without glossy or diffuse lighting are still extremely useful(RSMs are a valid GI technique, and depth peeling can help a lot with transparency).
I also tried describing a coarse implementation plan, but the fact is that I'm still delving into the depths of ogre's compositor framework, so that part of my proposal is vague on purpose. I understand how things _could_ be done, but I don't yet know what the best way of doing them is.
About the demo, that's really my bad; my initial post contained some info on that, but forgot to add the info to my proposal. Will edit it in as soon as possible.
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- OGRE Retired Team Member
- Posts: 714
- Joined: Mon Jan 31, 2005 7:21 pm
- Location: Israel
- x 2
Re: [GSoC 2011]Modern Illumination Techniques
Just went over the edits. It is a very ambitious project, but you seem to mean it
I have no further comments about your application, so you are welcome to upload it (make sure that the one on google-melange is synced with your changes in this thread).
Good luck!
I have no further comments about your application, so you are welcome to upload it (make sure that the one on google-melange is synced with your changes in this thread).
Good luck!
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011]Modern Illumination Techniques
Thanks for the confidence
I still need to address the issues raised by tuan kuranes but I have a really tough week at school. Is it ok if I present my solution in a comment to the initial proposal after the deadline?
I still need to address the issues raised by tuan kuranes but I have a really tough week at school. Is it ok if I present my solution in a comment to the initial proposal after the deadline?
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- Gnoblar
- Posts: 15
- Joined: Fri Jul 23, 2010 6:34 pm
Re: [GSoC 2011]Modern Illumination Techniques
Hi andrei
you may wanna look at this paper if you haven't read it already: http://www.cs.columbia.edu/~cs4162/slid ... ghting.pdf
Another thing I that I don't think that the SSDO is so free as you might think. SSDO also samples the screen space like SSAO. I am not sure if you ever tried to implement an SSAO algorithm, which gives you good results at an acceptable speed? That is not so easy. you have to think about sampling patterns like interleaved sampling and filters. If you use SSDO and want to to it right you may need to call the sampling twice. 1: one to calculate the illumination from unoccluded direction. 2: the indirect illumination. Because you don't have the color values for the occluders in the first pass you have to rerun the SSDO sampling. One thing you could do is to calculate the diffuse lighting with a simple lighting method while generating your G-Buffer , so you can run everything in one pass. But then the indirect light send by an occluder may not be right (when calculating it the occlusion was not taken not account).
Don't get me wrong I really like your ideas and think it would be great but your timeline is really packed. If you believe you can do it, than it great. but you could maybe consider to concentrate on the LPV and a solid implementation of SH functionality in Ogre3D because that would be useful for a lot of people I think. If you rally want to implement more then on method have you ever thought about "irradiance Volumes"? They also use SH for calculating diffuse indirect illumination. The ground idea is quite similar to LPV.
If you would consider implementing this instead of SSDO you would have the dam basis for both method, which is SH.
again I really like you proposal and I am happy to follow your project to see what comes out of it. I only wanted to share some of my thoughts about it because my masters-thesis is about a similar topic and I know how complicated some of the stuff you mentioned can be, because I've just been there .
Cheers
Sebastian
The idea of using the SH from LPV is good. Actually that's the next thing I want to try after my thesis, because I am calculating the directional occlusion in SH combining it with the SH from the LPVs is the logical next step. But I believe on problem is and I am sure you will also face it if you combine the SSDO sampling with SH from the LPVs is, that the SH from the LPVs only uses 2 bands. That means you have really low frequency incoming light representation. The resolution of the representation is not high enough for the effects you can see in the SSDO paper. It will work but it may look more like AO then like directional occlusion. To get a higher resolution you need more bands in the LPVs than that leads to other problems like rotating the SH of more then 2 bands in real time etc. SH can be a pain.andrei_radu wrote: @racoon:
My point is that I already have the SH representation for each grid cell, and using that in SSDO is almost free, and should give better results than sampling an env. map. Sure, one bounce GI can be added on top of that with relative ease, but I tried focusing my efforts on LPV, which is a more involved technique.
I'd really love to read your thesis, and I'd more than welcome any sort of code sample.
you may wanna look at this paper if you haven't read it already: http://www.cs.columbia.edu/~cs4162/slid ... ghting.pdf
Another thing I that I don't think that the SSDO is so free as you might think. SSDO also samples the screen space like SSAO. I am not sure if you ever tried to implement an SSAO algorithm, which gives you good results at an acceptable speed? That is not so easy. you have to think about sampling patterns like interleaved sampling and filters. If you use SSDO and want to to it right you may need to call the sampling twice. 1: one to calculate the illumination from unoccluded direction. 2: the indirect illumination. Because you don't have the color values for the occluders in the first pass you have to rerun the SSDO sampling. One thing you could do is to calculate the diffuse lighting with a simple lighting method while generating your G-Buffer , so you can run everything in one pass. But then the indirect light send by an occluder may not be right (when calculating it the occlusion was not taken not account).
Don't get me wrong I really like your ideas and think it would be great but your timeline is really packed. If you believe you can do it, than it great. but you could maybe consider to concentrate on the LPV and a solid implementation of SH functionality in Ogre3D because that would be useful for a lot of people I think. If you rally want to implement more then on method have you ever thought about "irradiance Volumes"? They also use SH for calculating diffuse indirect illumination. The ground idea is quite similar to LPV.
If you would consider implementing this instead of SSDO you would have the dam basis for both method, which is SH.
again I really like you proposal and I am happy to follow your project to see what comes out of it. I only wanted to share some of my thoughts about it because my masters-thesis is about a similar topic and I know how complicated some of the stuff you mentioned can be, because I've just been there .
Cheers
Sebastian
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011]Modern Illumination Techniques
Hi Sebastian, thanks a lot for the wealth of information, and for the interest. Haven't really thought about the low precision of the SH representation. This would probably depend on the grid resolution(increasing resolution might be a better option than increasing SH bands).
I meant SSDO is almost free in comparison to SSAO(which I is, indeed, quite expensive, especially when used in forward renderers), and I chose SSDO over other GI techniques because I can build on the current SSAO implementation.
Andrei
I meant SSDO is almost free in comparison to SSAO(which I is, indeed, quite expensive, especially when used in forward renderers), and I chose SSDO over other GI techniques because I can build on the current SSAO implementation.
Andrei
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page
-
- Gnoblar
- Posts: 15
- Joined: Sat Jul 18, 2009 9:37 pm
Re: [GSoC 2011]Modern Illumination Techniques
yeah would be awesome.About "deferred" render system plugin, I think there is something really interesting there.
-
- OGRE Retired Team Member
- Posts: 714
- Joined: Mon Jan 31, 2005 7:21 pm
- Location: Israel
- x 2
Re: [GSoC 2011]Modern Illumination Techniques
Congratulations! Your project has been accepted to Google Summer of Code 2011, and I will be your mentor.
We'll be setting up a working environment in the coming weeks. In the meantime, could you please update the wiki page of your project with the latest version of your proposal?
http://www.ogre3d.org/tikiwiki/SummerOf ... evelopment
We'll be setting up a working environment in the coming weeks. In the meantime, could you please update the wiki page of your project with the latest version of your proposal?
http://www.ogre3d.org/tikiwiki/SummerOf ... evelopment
-
- Google Summer of Code Student
- Posts: 55
- Joined: Fri Mar 18, 2011 8:37 pm
- x 8
Re: [GSoC 2011 - Accepted]Modern Illumination Techniques
Thanks for the confidence, I'll be sure to update the wiki page and set up to use the latest mercurial version. In the meanwhile, is there some other documentation on compositors than the one on the wiki and the ogre book?
My Google summer of code 2011 topic: Modern Illumination Techniques
My Google summer of code thread
My Google summer of code wiki page
My Google summer of code thread
My Google summer of code wiki page