[SoC 2009 - Accepted] Improve Ogre's Compositor Framework

[aguru]

Looks like all is good on linux now
Thanks for the great work!

[lukeneorm]

hi,
first of all THANK YOU Noman for your great work.. I was looking for a sollution to improve performance to support multiple lights in my project, and I think this is the right way.
So.. I'm trying to use deferred shading technique in my project. In the Deferred Shading wiki, under "integration steps", is described how to integrate the deferred shading:
1. Register GBufferSchemeHandler and DeferredLightCompositionPass with their respective managers.
2. Add the GBuffer and ShowLit compositors to the viewports you want deferred shaded.

So I include the cpp, h and shaders files in my project, and I followed steps 1 and 2 for integration, adding this lines in my code:

Code: Select all

	CompositorManager &compMan = CompositorManager::getSingleton();
	MaterialManager::getSingleton().addListener(new GBufferSchemeHandler, "GBuffer");
	MaterialManager::getSingleton().addListener(new NullSchemeHandler, "NoGBuffer");
	compMan.registerCustomCompositionPass("DeferredLight", new DeferredLightCompositionPass);
	CompositorInstance* DSInstance;
	DSInstance = compMan.addCompositor(myVp, "DeferredShading/GBuffer");
	DSInstance->setEnabled(true);
	DSInstance = compMan.addCompositor(myVp, "DeferredShading/ShowLit");
	DSInstance->setEnabled(true);

but.. the result is the scene as if there are no lights. The left figure show the normal scene (without deferred shading), while the right one show the scene after the attemp to integrate deferred shading:



As you can see, in the right figure there are no lights. I'm using a spotlight, but the result is the same also with point or directional lights. The original material I use for my scene is the nullsquare's Normal AO Specular Mapping Shader [http://www.ogre3d.org/wiki/index.php/No ... ing_Shader].

If I enable DeferredShading/ShowDepthSpecular, DeferredShading/ShowNormals or DeferredShading/ShowColour I obtain this:



As you can see, the ShowNormals compositor doesn't display the correct result: the boxes and the floor shouldn't be flat (they have a normal map applied).

..am I missing some steps to fully integrate this technique with my project?

[Noman]

Good to see people trying this out now that this is in trunk!
First of all, just to make sure - the deferred shading demo works correctly on your system (to rule out hardware issues etc) ?

Now, lets split your problems into two sections - writing the G-Buffer and rendering the lights from the G-Buffer.

Writing to the g-buffer is doing its 'basic' job fine (you can see that the debug visualizations make sense) but the advanced techniques (normal mapping, for example) aren't doing anything. See the next section in the article (Adapting the framework) on other places you might need to modify to get things working. A good place to see if things are working properly is the result of the GBufferSchemeHandler::inspectPass call. Does it find the normal map in passes that have a normal map?
Also note, that deferred shading completely bypasses the normal shading pipeline, so nullsquared's shaders do not get used. If you want to add features like specular mapping, you would have to do something similar to what I did with normal mapping - add a specular map convention and add code to the g-buffer material generator to generate the shader that uses specular maps.

Reading from the Gbuffer is where the main lighting problem is happening. It seems like only the ambient light gets rendered. This section of the rendering is DeferredLightCP.cpp. Try debugging DeferredLightRenderOperation::execute and check that it gets called and that it enters the loop for every light in the scene. Disable shadow casting for now if you have it enabled, will make things simpler and less error prone. If things haven't cleared up by now, try debugging into DLight::updateFromParent / updateFromCamera to make sure that the geometry that gets built makes sense.

Tell me what happened and we'll continue from there...

[Wolfmanfx]

The demo works correct on mac the problem that sinbad has with the demo is that he starts the demo not through xcode (xcode sets the working directory) so there is a problem with the resources(i assume)

[Vectrex]

Trunk works fine here.

Windows 7 : rev 7100
NVidia 8800 GT 512meg
Intel core 2 8400, 3ghz per core
2 gig ram

1920 x 1080 = DX 130 fps, OGL 115 fps
with SSAO = DX 55 fps, OGL 50 fps (HARSH! )

Is it me or is the triangle count broken?

[lukeneorm]

Hi Noman, thank you for your reply.

First of all, just to make sure - the deferred shading demo works correctly on your system (to rule out hardware issues etc) ?

It works very well. I rise the spotlights number up to 20, and I obtain 40/35 FPS (with/without SSAO) on my laptop (2.53GHz Intel Core 2 Duo, nVidia GeForce 9600M GT 512MB) using D3D. Here it is a couple of screenshots.



The problem with the spotlight was in the attenuation parameters. Now everything works fine! I use a DeferredDemo.material template for now, my priority is to manage more than a couple of lights with shadows and ssao and obtain a decent FPS. Later I'll try to add other shader effects.
Now I have a problem with deferred SSAO compositor. In the original SSAO compositor (not deferred) I follow nullsquare suggestion to adapt SSAO with the size of the scene and I tweak linear depth by multiplicating it by a COEF value:

Code: Select all

	//[geom_ps shader in diffuse.cg, original ssao code by nullsquared]:
	//float4 OUT.c contains this values: (linear depth value, normal.x, normal.y, normal.z)
	
	//instead of this..
    //OUT.c = float4(length(vp.xyz) / far, normalize(vn.xyz).xyz);
	//..I use this
	float COEF = 0.1;
	OUT.c = float4(length(vp.xyz) * COEF / far, normalize(vn.xyz).xyz);

Now, my problem is that if I use deferred SSAO in my scene leaving all 'as it is', fixed spotlights and their shadows works well, but I obtain bad SSAO artifacts (see the circle on the walls) because SSAO input depth value isn't tweak:



If I change GBufferMaterialGeneratorImpl::generateFragmentShader code to tweak depth writing:

Code: Select all

//change this..
#ifdef WRITE_LINEAR_DEPTH
    ss << "	oColor1.a = length(iViewPos) / cFarDistance;" << std::endl;
//..into this
#ifdef WRITE_LINEAR_DEPTH
    ss << "	oColor1.a = length(iViewPos) * 0.1 / cFarDistance;" << std::endl;

I obtain the correct SSAO result, but spotlights and shadows stop to work in the right way.



I try to leave generateFragmentShader code unchanged and change only the deferred ssao ssao_ps shading code, so as to affect only SSAO input depth value:

Code: Select all

	//[ssao_ps shader in ssao.cg, deferred ssao code]
	//I try to change this..
    float depth = geom.w * COEF;
	//..into this
	#define COEF 0.1
    float depth = geom.w * COEF;

But the result isn't the same as when I changed generateFragmentShader code. Shouldn't it be the same? I mean: for deferred SSAO depth interpretation, if generateFragmentShader code writes this depth value..
oColor1.a = length(iViewPos) * 0.1 / cFarDistance;
..isn't the same as if generateFragmentShader writes this depth value:
oColor1.a = length(iViewPos) / cFarDistance;
and then ssao_ps multiplies that value by 0.1?

Where am I wrong? :\

[Noman]

@Vectrex :
SSAO is a very pixel-expensive technique. There is not much you can do about it... Thats the reason that at high-resolutions you will get a performance hit. The upside, is that this performance hit will be the same regardless of scene complexity, which is nice.

@lukenorm
Good to hear that you're understanding the workings of the framework and making progress! The screenshot from the modified demo is actually quite impressive! We would need ~20 shadow textures at the same time to get the same result without deferred shading. Cool stuff!
Modifying generateFragmentShader had side effects because the (not automatically generated) post-processing shaders did not get modified to compensate for this difference. However, for accuracy purposes I don't think its a good idea to make this change.
All in all, the SSAO works well when the scale system is 1 unit = 1 meter. I don't know why they are not having the same effect on your SSAO results... You can debug the SSAO compositor by commenting out the "input 0 scene" line in the SSAO compositor in the target_output pass, and then enabling SSAO will give you the SSAO buffer. Can you post screenshots of it and maybe that will help us debug the problem?

[lukeneorm]

Good to hear that you're understanding the workings of the framework and making progress! The screenshot from the modified demo is actually quite impressive! We would need ~20 shadow textures at the same time to get the same result without deferred shading. Cool stuff!

Yes, your work is excellent: I'm currently using up to 25 spotlight with shadows in my project, and I still obtain 20 FPS..

All in all, the SSAO works well when the scale system is 1 unit = 1 meter. I don't know why they are not having the same effect on your SSAO results... You can debug the SSAO compositor by commenting out the "input 0 scene" line in the SSAO compositor in the target_output pass, and then enabling SSAO will give you the SSAO buffer. Can you post screenshots of it and maybe that will help us debug the problem?

I'm using 1 unit = 1 meter scale in my scene, and SSAO works indeed quite well. The problem is that since my scene use one only texture, I need to rise up SSAO strenght output to better highlight scene geometry. But if I do that, circle artifacts become visible. With the trick I explained in the previous post for depth value writing, I could decrease that kind of artifacts. But if it is not possible.. then I'll use SSAO in his original state. Here are a couple of screenshots (left: normal SSAO, right: SSAO effect with streght rised up to 3 times):


Now another interesting issue: what if I'd like to use different shadow technique? I see that checkShadow function in LightMaterial_ps.cg perform a simple texture shadow check between shadowDistance and distanceFromLight.
The first question is: checkShadow returns a void value.. how can this function affect total_light_contrib? I mean: usually I write a shadow function that returns a shadow amount value, and then multiplicate that value by the final color, so as to attenuate the color output if that pixel is in shadow. But in this case I can't see where you use checkShadow internal computation.. am I missing something?
The second question is: to use PCF shadow technique, would it be sufficient to replace checkShadow function with something like this?

Code: Select all

float checkShadow_PCF(
	float2 offset //added: it is inverse_texture_size.xy
	sampler2D shadowMap,
	float3 viewPos,
	float4x4 invView,
	float4x4 shadowViewProj,
	float shadowFarClip,
#if LIGHT_TYPE == LIGHT_DIRECTIONAL
	float3 shadowCamPos
#else
	float distanceFromLight
#endif
	)
{
	float3 worldPos = mul(invView, float4(viewPos, 1)).xyz;
#if LIGHT_TYPE == LIGHT_DIRECTIONAL
	float distanceFromLight = length(shadowCamPos-worldPos);
#endif
	float4 shadowProjPos = mul(shadowViewProj, float4(worldPos,1));
	shadowProjPos /= shadowProjPos.w;
	float2 shadowSampleTexCoord = shadowProjPos.xy;

    float2 uv = shadowSampleTexCoord.xy;
    float3 o = float3(offset, -offset.x) * 0.3f;

    // Note: We using 2x2 PCF. Good enough and is alot faster.
    float c =   (distanceFromLight <= tex2D(shadowMap, uv.xy - o.xy).r) ? 1 : 0; // top left
    c +=        (distanceFromLight <= tex2D(shadowMap, uv.xy + o.xy).r) ? 1 : 0; // bottom right
    c +=        (distanceFromLight <= tex2D(shadowMap, uv.xy + o.zy).r) ? 1 : 0; // bottom left
    c +=        (distanceFromLight <= tex2D(shadowMap, uv.xy - o.zy).r) ? 1 : 0; // top right
    return c / 4;
}

This is because if it would be possible to improve shadow quality.. 25 spotlight + PCF shadows would be a VERY nice effect..
Thank you!

[Noman]

I am not a big SSAO expert. I pretty much took it as is from nullsquared's demo, so maybe that thread is a better place for this type of questions. I do know that SSAO is supposed to be a soft technique - not supposed to affect the scene too much, so maybe that is not the way to highlight scene geometry properly...

As for the shadow technique - of course its possible to implement a different one. I chose the simplest one out there that just does depth comparison between a naive linear depth map and drops the pixel if shadowed (the clip(...) line in the checkShadow function is the line that allows shadows to work without returning a value).
The shadow casting code is completely integrated with ogre's shadowing framework so you can do whatever you like there to generate the shadow maps, you just need to modify the deferred light shaders to use the technique as well (you are looking at the right shader for that task).
I don't think its a good idea to integrate this type of work with the demo, because its sophisticated enough as it is - I don't want to confuse the user even more with advanced shadowing techinques... However, the framework was definitely built with the ability to use other shadow techniques in mind. Good luck!

[lukeneorm]

Hi,
I recently send a PM to Noaman to fully understand how he can pass only visible lit pixels to the lit shader. Since Noaman answer was very helpful to me (thank you!:)), he suggest to write the issue in this thread to help other people understand whats going on.
So I write my question and Noaman answer altogether, trying to describe how the visible lit pixels are selected and passed to the lit shader.

1* DeferredLightRenderOperation::execute is a render_custom target pass called every frame registered with registerCustomCompositionPass(). It asks for what lights are affecting the frustum with _getLightsAffectingFrustum().
2* For each light affecting frustom
2.1* Find and mark visible lit pixels
- DLight::rebuildGeometry builds the convex light geometry (for example a sphere for a point light, a cone for a spot light). Let's for example examine a spotlight geometry: a cone.
- If camera is outside the current light convex geometry, the cone-geometry has:
* a CULL_CLOCKWISE culling mode, so only triangles whose vertices are passed/indexed in anticlockwise order are rendered: in other words, all the cone-mesh triangles outside-oriented are rendered, because the camera is looking at the cone-mesh from the outside;
* a CMPF_LESS_EQUAL depth function, so the pixel is written if it is closer (or at the same distance) than the current contents.
- Otherwise, If camera is inside the current light convex geometry, the cone has:
* a CULL_ANTICLOCKWISE culling mode, so only triangles whose vertices are passed/indexed in clockwise order are rendered: in other words, all the cone-mesh triangles inside-oriented are rendered, because the camera is looking at the cone-mesh from the inside;
* a CMPF_GREATER_EQUAL depth function, so the pixel is written if it is further (or at the same distance) than the current contents.
The question is: how can we isolate visible lit pixels using only culling mode + depth function? For example, if we put a spotlight in this way (on the left the image taken using nVidia perfHud, on the right i manually erased cone-edges to better figure out how the cone intersects the wall):

the camera is outside the cone, so only the outside-triangle are taken into account from CULL_CLOCKWISE. But CMPF_LESS_EQUAL tells to pass to the LightMaterial_ps shader pixels on cone surface that are closer (or at the same distance) than the current contents. Now, the pixels in the A cone-surface-area are closer than the pixels in the background (B): why they are (correctly) not lit?
The answer is:

the pixels marked by A do get passed to the pixel shader, but the calculations there lead to having no effect. This code is at the end of the function :

Code: Select all

		
    #if LIGHT_TYPE == LIGHT_SPOT
       float spotlightAngle = saturate(dot(lightDir.xyz, -objToLightDir));
       float spotFalloff = saturate((spotlightAngle - spotParams.x) / (spotParams.y - spotParams.x));
       total_light_contrib *= (1-spotFalloff);
    #endif

In the case of a pixel being outside the spotlight's range, spotFalloff will be 1, causing total_light_contrib to be zeroed out. Since we are doing additive lighting, this means that the colour will not change.

Now we can continue our step-by-step road to the pixel shader

- DLight::updateFromCamera sets LightMaterial_ps.cg farCorner, shadowCamPos, shadowFarClip, and LightMaterial_vs farCorner parameters.
2.2* Render shadow map
- DeferredLightRenderOperation::execute calls sm->prepareShadowTextures(cam, mViewport, &ll); and init ShadowMap shader param.
2.3* Shade lit pixels
- DeferredLightRenderOperation::execute calls sm->_injectRenderWithPass(pass, rend, false, false, lightList); with a lightList with one only light: the current light. It shades lit pixels and add them to framebuffer.

Hope that Noaman answer helps someone to better understand the deferred workflow!

[Assaf Raman]

Michael Grosberg just donated this model for your demo.
If you want to use it - give him credit in a text file next to the model.
The model is inspired by this Escher painting:

Attached is a 3dmax model, mesh file and all textures.

screenshot.jpg (353.61 KiB) Viewed 4877 times

screenshot-viewer.jpg (401.91 KiB) Viewed 4877 times

[xadhoom]

When the camera in the deferred demo is near the athena (guessing it depends if the camera is inside the front light´s radius). The lighting starts flickering on the athena as if the lights are put on/off very chaotic. Maybe it belongs to the mesh drawings/calculations in the light compositor?

This is better visible if you increase the number of lights around the athena...

xad

[Noman]

Assaf and Michael - thanks!
It looks like a great model! I would use it in the demo, but it would require changing the scene creation portion of it. I might get to it later, but I don't see it as something urgent.

xadhoom - hmm, I think it has to do with the calculation of are we inside/outside camera. Perhaps the calculation needs to be tweaked a bit (for example, to take clip distances and geometry/pure math differences into consideration) to get a more accurate result... Maybe even a small bias can do the trick instead of more advanced calculations...

[Noman]

Heads up - discussion regarding some of the features proposed in this thread but not implemented in the SoC project is now happening in this thread.

[sinbad]

I've locked this topic now since everything in it is now core - if you want to discuss it now, please do so in the other normal forums!