Dynamic Ambient Occlusion

[blackgun]

Dynamic AO, looks like:

http://www.ogre3d.org/phpBB2/viewtopic.php?t=34166

[jjp]

This is nothing you can have as a feature in a general purpose engine. There are many different approaches, all with advantages and disadvantages. No general solution exists that would work for most scenarios.

If you want something like this: lots of articles on the topic and implementation shouldn't be too hard with Ogre. But there is no way to have a general ambient occlusion feature.

[jacmoe]

I would love to have a demo or two featuring some possible ways of doing ambient occlusion with Ogre.

[Praetor]

Don't tempt me jacmoe!

[jjp]

We(=Cabalistic and me) will definitely use some kind of dynamic AO within our next project (and make a spin-off demo..). But that won't start earlier than December. Probably more like early next year...

[ILikeCake]

How does one go about doing ambient occlusion on the fly?

As far as I understand it, ambient occlusion is normally done by integrating a visibility function over a sphere or hemisphere. You can use Monte Carlo methods, but I would think that would be prohibitively expensive in real-time to get any kind of reasonable accuracy.

So what's the trick?

[voxel]

How does one go about doing ambient occlusion on the fly?

As far as I understand it, ambient occlusion is normally done by integrating a visibility function over a sphere or hemisphere. You can use Monte Carlo methods, but I would think that would be prohibitively expensive in real-time to get any kind of reasonable accuracy.

So what's the trick?

AO is very low-order approximation. I've seen HW AO (the Nvidia method?) that works like cube mapping (rendering view from different faces).

The other is to use 3d basis functions - ala pre-computed radiance transfer with spherical harmonics + wavelets to compute the "visibility function" but that only works for rigid + local shadowing.

[cloud]

The gems 2 way is to create a hiararchy of disc shaped surface elements that you can traverse or go down based on the solid angle which is a sort of percentage of intersection of 2 discs hemispheres, you do each vertex and sum up the contributions gives an amount of occulsion. I'm still trying to do it, the hard bit is updating the control maps, position, normal, area.

[ILikeCake]

I was sort of having trouble figuring out how you would store the disc position, normal and area and use them to update the vertex colour for your mesh within Ogre. The algorithm is pretty simple, but the specific API implementation was a bit beyond my fairly basic Ogre knowledge.

I'd be interested to see your code if you got this working as described in the paper!

[voxel]

I saw something in Spherical Harmonic Exponentiation that seemed pretty neat

http://research.microsoft.com/research/ ... pubid=1617

http://download.microsoft.com/download/ ... aphics.zip
(see Advanced Lighting PPT)

Basically they are computing SH co-efficients per subarea by NOT rendering ta cube map (then project that onto SH basis functions) from the subarea, but projecting spheres/shapes (semi-manually created) onto the SH basis functions...

[sinbad]

Does anyone except me find all these techniques too low frequency and too reliant on a static scene to be truly that useful in a generic sense?

AO is great, because it's just one extra lighting cue that when combined with more discrete dynamic lighting gives it extra depth. Dynamic AO techniques seem to fall between 2 stools to me - they're not flexible enough to do truly dynamic lighting (because they can only handle low frequency distant lights) and can't cope with a changing scene without excessive precomputation. Layering AO with regular dynamic light / shadow complements quite well so you get some depth and dynamism, doing it all with AO just seems, well 'fuzzy' and restrictive to me.

Nice as a tech demo but I don't find them that useful in practical cases. Maybe it's just me.

[Praetor]

It as certainly given good results in practice. I believe Valve uses is for everything since HL2. But yeah, I'm usually pretty pessimistic about things which are not truly dynamic. What it usually means is you MUST make some sort of tool through which everything is fed. Not always a horrible thing, but it just seems like every new tool you shove in a pipeline makes that pipeline a little less flexible.

However, indirect lighting is getting to be a must. Because, I'm starting to notice and snicker at direct-lighting only games. Take Gears of War. Great fun with coop. But some parts are so dark, for apparently no reason at all. We look up at the sky, and it is fairly light, one wall of an alleyway is lit a little, and the other is absolutely pitch black. It was jarring, and a little amusing. Lighting is so very tricky. To me, the more effects added, the more I notice how it doesn't look real. Another example is Bioshock. It is a work of art, really, and yet there are just so many lighting effects it seems to only highlight that the lighting is still just a little "wrong."

[Falagard]

God I hate lighting. It's so tricky to get things to look good, let alone look perfect. There are so many factors.

It depends entirely on the type of game you're going for.

If your game is a level based game that takes place at a specific time of day, your best bet is to use a technique like Half Life 2's precalculated global illumination with directional lightmaps (radiosity normal mapping) for your static geometry and ambient cubes + texture shadows and dynamic lighting for your dynamic geometry.

If your game has any sort of diurnal (day/night) cycle like Oblivion or an MMO, your best bet is to use precalculated ambient occlusion and texture shadows for outdoor areas. Instead of ambient cubes you could use hemispheric lighting. Indoor areas that don't receive light from outside could use techniques similar to HL2 techniques above though. It gets trickier if you want to be able to see outside through windows and have light stream through them inside.

Anyhow, it's all tricky stuff

Then you factor in HDR and cry.

[voxel]

However, indirect lighting is getting to be a must.

This is key. Local illumination / direct lighting looks so... 90s.

AO is being completely abused. Just slapping it on with direct lighting makes it look muddy (a known thing in film/TV). It's the greyness of the shadows that changes the tone of the visuals. And using lambert/phong/blinn + AO just compounds the weird mix.

However, spherical Harmonics / hemispherical lighting will remain for a while - it's great to getaway from using directionals all the time.

[sinbad]

It as certainly given good results in practice. I believe Valve uses is for everything since HL2.

Nah, that's their radiosity lightmaps and local cubic environment maps you're seeing there, it's not PRT as far as I know. Sure they encode the radiosity as SH vectors but it's an entirely different calculation because it doesn't let the light be dynamic.

That's the thing, I think precalculated but directional radiosity looks a lot better for static scenes, and PRT and other dynamic approaches only copes with 'distant, low frequency lights' which generally means the sun / skylight which doesn't change fast enough to make a sod of difference in most games. Hence why I'm failing to see the benefits of having a system which can't deal with a dynamic scene (which is more likely to be needed) but can deal with dynamic sun (which is less likely). If you're going to say the scene has to be static, at least with radiosity lightmaps you get better shadow definition at the expense of it not being dynamic, yes, but since PRT can't deal with anything but distant lights it's dynamic elements are not that useful in practice IMO. The specular created from the local cubemaps gives a nice dynamicish feel even though it's not.

[Praetor]

I'm getting all these complicated GI techniques jumbled. But agreement on the dynamic part of it all.

[voxel]

Hence why I'm failing to see the benefits of having a system which can't deal with a dynamic scene (which is more likely to be needed) but can deal with dynamic sun (which is less likely).

You can always project lights onto the SH basis function to "add" these lights to the SH vectors.

Also, you can use different sets of SH vectors for different areas - i.e under a bridge, inside a room, etc..

If a light turns on in a room - you can project this light to create new SH vectors.

But everything is all low-frequency of course.

If you're going to say the scene has to be static, at least with radiosity lightmaps you get better shadow definition at the expense of it not being dynamic, yes, but since PRT can't deal with anything but distant lights it's dynamic elements are not that useful in practice IMO. The specular created from the local cubemaps gives a nice dynamicish feel even though it's not.

The problem with lightmaps is texture memory and constant state switching. Works great for games that move slowly (FPS) but not anything fast - action/racing.

PRT is impractical due to the crazy polygon requirements... It's a nice idea without a future.

High-frequency lighting has the same issues. Rarely does anybody generate a dynamic cube map on the fly (all 6 faces - yes, I've done it for a console game). It's a massive hit on the GPU. Everybody uses pre-generated cubemaps.... why not use pre-generated SH vectors for the low frequency lightint?

[Praetor]

You can always project lights onto the SH basis function to "add" these lights to the SH vectors.

I suspect that this is an expensive operation. So, wouldn't be hard to marry dynamic with realtime with this?

[voxel]

You can always project lights onto the SH basis function to "add" these lights to the SH vectors.

I suspect that this is an expensive operation. So, wouldn't be hard to marry dynamic with realtime with this?

I don't recall it is.... (almost 4 years ago I dug into the math). You project the light onto 9 basis functions x 3 (once per R,G,B). The more basis functions you have the better the representation... Projecting lights onto SH basis functions is like taking a sawtooth/stepped curve and pumping into FFT with sines and cosines.

The lights are conveniently represented in a nice functional/procedural format. The math is a little weird - especially rotating the SH co-efficients (barf) which you try avoiding.

Rendering the lights into cube / sphere map and then using that to generate SH vectors would be expensive and slow (how most people generate SH vectors). That is the only way to project lights that don't have a nice procedural representation... like a HDRI textured domes.

[Kojack]

I was planning on experimenting with combining SH with ogre, but a hard drive crash put that on hold.

Anyway, have a read of http://home.comcast.net/~tom_forsyth/bl ... otes%5D%5D
for a cool description of how Tom got a Gamecube to handle 50+ lights in a room using SH (when a Gamecube has no shaders and only 8 opengl lights).

[voxel]

I was planning on experimenting with combining SH with ogre, but a hard drive crash put that on hold..

Using the SH co-efficients is easy - it's just 3 (R,G,B) matrix multiplies and dot products in a vertex shader to generate the colour.

I'd start with the SH vectors from the Grace Cathedral found here:

http://graphics.stanford.edu/papers/envmap/prefilter.c

Code: Select all

% prefilter -swap grace_probe.float 1000

         Lighting Coefficients

(l,m)       RED        GREEN     BLUE
L_{0,0}    0.078908  0.043710  0.054161
L_{1,-1}   0.039499  0.034989  0.060488
L_{1,0}   -0.033974 -0.018236 -0.026940
L_{1,1}   -0.029213 -0.005562  0.000944
L_{2,-2}  -0.011141 -0.005090 -0.012231
L_{2,-1}  -0.026240 -0.022401 -0.047479
L_{2,0}   -0.015570 -0.009471 -0.014733
L_{2,1}    0.056014  0.021444  0.013915
L_{2,2}    0.021205 -0.005432 -0.030374

MATRIX M: RED
 0.009098 -0.004780  0.024033 -0.014947 
-0.004780 -0.009098 -0.011258  0.020210 
 0.024033 -0.011258 -0.011570 -0.017383 
-0.014947  0.020210 -0.017383  0.073787 

MATRIX M: GREEN
-0.002331 -0.002184  0.009201 -0.002846 
-0.002184  0.002331 -0.009611  0.017903 
 0.009201 -0.009611 -0.007038 -0.009331 
-0.002846  0.017903 -0.009331  0.041083 

MATRIX M: BLUE
-0.013032 -0.005248  0.005970  0.000483 
-0.005248  0.013032 -0.020370  0.030949 
 0.005970 -0.020370 -0.010948 -0.013784 
 0.000483  0.030949 -0.013784  0.051648 

Your scenes should look this:

http://graphics.stanford.edu/papers/env ... filter.jpg

[voxel]

I decided to play with a SH co-efficient using shader. My OGRE/cg skills kinda suck so feel free to improve:

Code: Select all

void sphericalharmonics_vp(float4 position : POSITION, 
			float3 normal   : NORMAL,
		  out float4 oPosition : POSITION,
		  out float4 oColor    : COLOR,
		  uniform float4x4 worldViewProj,
		  uniform float4x4 redMatrix,
		  uniform float4x4 greenMatrix,
		  uniform float4x4 blueMatrix)
{
	oPosition = mul(worldViewProj, position);
	float4 N = float4(normal[0], normal[1], normal[2], 1.0);
	float4 temp = mul(redMatrix, N);
	float red = dot(N, temp);
	float4 temp2 = mul(greenMatrix, N);
	float green = dot(N, temp2);
	float4 temp3 = mul(blueMatrix, N);
	float blue = dot(N, temp3);
	oColor = float4(red, green, blue, 1.0);
} 

// Hard-coded for testing - also all values are scaled by 10

Code: Select all

vertex_program SphericalHarmonics cg
{
	source BLAHBLAH.cg
	default_params
	{
		param_named_auto worldViewProj worldviewproj_matrix
   		param_named redMatrix matrix4x4 0.09098 -0.04780  0.24033 -0.14947 -0.04780 -0.09098 -0.11258  0.20210  0.24033 -0.11258 -0.11570 -0.17383 -0.14947  0.20210 -0.17383  0.73787 
   		param_named greenMatrix matrix4x4 -0.02331 -0.02184  0.09201 -0.02846 -0.02184  0.02331 -0.09611  0.17903  0.09201 -0.09611 -0.07038 -0.09331 -0.02846  0.17903 -0.09331  0.41083  
   		param_named blueMatrix matrix4x4 -0.13032 -0.05248  0.05970  0.00483 -0.05248  0.13032 -0.20370  0.30949  0.05970 -0.20370 -0.10948 -0.13784 0.00483  0.30949 -0.13784  0.51648 
	}

	entry_point sphericalharmonics_vp
	profiles vs_1_1 arbvp1
}

Code: Select all

material my_sh_coeff_shader
{
	receive_shadows off
	technique
	{

// This is totally useless I think. The second part should produce the right colours given the correct co-effs.
		pass
		{
			ambient 1.0 1.0 1.0 1
			diffuse 0.0 0.0 0.0 1
			specular 0 0 0 1
			emissive 0 0 0
		}
		pass
		{
                        // This is problem wrong
			scene_blend modulate
			vertex_program_ref SphericalHarmonics
			{ 
			} 
		}
	}
}

[anomalous_underdog]

the nvidia method i think is this

http://http.download.nvidia.com/develop ... ic_amb_occ

pretty impressive. although maybe its nvidia hardware specific. i don't know

[stimuli]

For the future, when GPUs can actually handle this sort of computation on a meaningful scale, having this ability in Ogre would be great. Every instance of real-time AO I've seen has been limited to one simple character in an empty scene, which suggests that day has not yet arrived.

Verts that haven't moved much/aren't within the AO threshold don't need re-calculation. Or is all AO calculated per object, per frame?

Obviously, for environments, one would want to bake out AO passes in Maya/similar and then composite those results with the standard DO pass.

[Vectrex]

For the future, when GPUs can actually handle this sort of computation on a meaningful scale, having this ability in Ogre would be great. Every instance of real-time AO I've seen has been limited to one simple character in an empty scene, which suggests that day has not yet arrived.

Verts that haven't moved much/aren't within the AO threshold don't need re-calculation. Or is all AO calculated per object, per frame?

Obviously, for environments, one would want to bake out AO passes in Maya/similar and then composite those results with the standard DO pass.

I recommend checking out Screen Space AO (SSAO) which various people have had success with. Most notably Crytek. Google Crytek SSAO, it works really well