Soft Shadows (w/o EA) Demo (99% done, released)

[nullsquared]

shadowTextureCasterPreViewProj() is defined a bit lower down in main.cpp, use that. As for no shadows ... Well, what kind of shaders are you using? This isn't really a plug-n-play type of thing, since you need to integrate the actual shadow code into your own lighting shaders. 00diffuse.material and diffuse.cg are what you can base your shadow code off, most of it is encapsulated in a single shadow() function. Or, if you want, you can always just use that shader in your own project and modify it to your needs if you don't feel like writing your own.

[TheSHEEEP]

shadowTextureCasterPreViewProj() is defined a bit lower down in main.cpp, use that.

Thanks, using that now.

As for no shadows ... Well, what kind of shaders are you using? This isn't really a plug-n-play type of thing, since you need to integrate the actual shadow code into your own lighting shaders. 00diffuse.material and diffuse.cg are what you can base your shadow code off, most of it is encapsulated in a single shadow() function. Or, if you want, you can always just use that shader in your own project and modify it to your needs if you don't feel like writing your own.

Aha! Well I was thinking it was (more or less) plug-n-play (mainly because I think that is how shadows should work ).
So my next (and hopefully last) question:

How do I integrate the actual shadow code into my own lighting shaders.

I tried to figure it out by looking into your demo, but I can't even see where you did that. I don't see any reference to either the diffuse material or the cg.

Sorry for sounding completely noobish, but I am when it comes to shaders

[nullsquared]

Aha! Well I was thinking it was (more or less) plug-n-play (mainly because I think that is how shadows should work ).

Shadows are completely scene dependent - different models/configurations/scenes/sizes/formations/environments/etc. call for different shadowing. This VSM shadowing is extremely basic, just the tip of the iceberg meant to be used as a "base", more or less.

So my next (and hopefully last) question:

How do I integrate the actual shadow code into my own lighting shaders.

If you have no shader experience what-so-ever, then you probably aren't using an lighting shaders to begin with. In which case, if you want to use these shadows, you'd be best off with playing with the provide 00diffuse.material file.

I tried to figure it out by looking into your demo, but I can't even see where you did that. I don't see any reference to either the diffuse material or the cg.

Well, the 00diffuse.material file specifies a material "template" (Eihort ), from which other materials (look at the other .material files) "inherit" (Eihort ). Then they simply choose their own textures that the diffuse shader will use. See how those materials work, and try to accommodate your own materials to do the same thing.

Also - my SSAO demo has the same but cleaned-up shadowing.. and the diffuse shader has some added specular lighting. You might want to take those in place of the old ones for this shadow demo.

[TheSHEEEP]

Ah, okay, so I have to include diffuse_template in all materials that I want to receive shadows, right?

Code: Select all

material karte : diffuse_template
{
    set_texture_alias ambient_tex Mat_Farbe_neu9.tif
    set_texture_alias diffuse_tex Mat_Farbe_neu9.tif
}

Tried it with some random jpg, too, with the same result: white screen with a floating Lemming.
I am using both the *.materials and *.cg from the SSOA demo (which rocks, btw). Now the mesh using that material is white.
From the Ogre.log:

19:15:48: Mesh: Loading LevelEntity.mesh.
19:15:48: WARNING: material karte has no supportable Techniques and will be blank. Explanation:

Irony: The "explanation" is blank, as well

Btw... I'm beginning to think that my unexperienced questions would make up for an excellent FAQ.

[nullsquared]

That's odd. Which materials, exactly, did you copy over? Technically, you only need 00diffuse.material and diffuse.cg for the actual shadow "receiving" and lighting (as well as the shadow caster materials). Can I see the whole log? It seems the diffuse material choked on something.

[TheSHEEEP]

I copied

00diffuse.material
diffuse.cg
shadow_caster.material
shadow_caster.cg


I just looked over the whole log again and found some other error lines before the one I posted:

08:33:47: Error at line 242 of Level_01.material: parent material: diffuse_template not found for new material:karte
08:33:47: Parsing script 00diffuse.material
08:33:47: OGRE EXCEPTION(2:InvalidParametersException): Parameter called invSMSize does not exist. in GpuProgramParameters::_findNamedConstantDefinition at ..\src\OgreGpuProgram.cpp (line 883)
08:33:47: Error at line 33 of 00diffuse.material: Invalid param_named_auto attribute - Parameter called invSMSize does not exist.
08:33:47: Error in material diffuse_template at line 89 of 00diffuse.material: Invalid vertex_program_ref entry - vertex program ambient_vs has not been defined.
08:33:47: Error in material diffuse_template at line 93 of 00diffuse.material: Invalid fragment_program_ref entry - fragment program ambient_ps has not been defined.

Some of those seemed to require the ambient files as well (00ambient.material and ambient.cg).
So I copied them, too, and now the log shows this (as well as the one with the blank "explanation"):

08:36:32: Error at line 242 of Level_01.material: parent material: diffuse_template not found for new material:karte
08:36:32: Parsing script 00ambient.material
08:36:32: Parsing script 00diffuse.material
08:36:32: OGRE EXCEPTION(2:InvalidParametersException): Parameter called invSMSize does not exist. in GpuProgramParameters::_findNamedConstantDefinition at ..\src\OgreGpuProgram.cpp (line 883)
08:36:32: Error at line 33 of 00diffuse.material: Invalid param_named_auto attribute - Parameter called invSMSize does not exist.

edit: I placed the material "karte" in a new file. Didn't help, as well.

[nullsquared]

Uh. This is a tough one. You need to get the '00' materials to be parsed _before_ your other materials (this is on Eihort, Shoggoth has a better way of doing this).

[TheSHEEEP]

Ahhh, it finally works.

Well, almost.

The files from the SSAO demo make the Entity not receiving shadows and reflect light like it was made of metal

So I just used the files from the normal demo.

Basically it works, but it is very, very pixelated (meaning the shadows consist of very few squares) and the shadows are a bit too transparent (may be the same problem, though).
Also, the running around Lemmings don't cast shadows on themselves, although it is enabled in the Scene Manager.
Do they have to have diffuse_template in their materials as well?
If yes, what if they don't use textures?

This is because we don't use textures in most of our models since we want to achieve comic look, which works fine. Also, no texturing to do for our artists. Hooray

Anyway, thanks for your help!
Our project will have to be finished on July 30th. After that, I will some time to write a simple Tutorial for this, once it works nice for me. Remind me of that, I will probably forget it!

[nullsquared]

Well, at least it sounds like we're getting somewhere

1) what scale are you using for your models and world? As in, 1 unit = 1 meter, or 1 unit = 1 centimeter, or 1 unit = 1 foot, what?

2) can I get a screen shot of what you mean by the blocky/transparent shadows?

3) anything that you want to receive a shadow (including self-shadowing) will need to make use of the shadow map. For you, this would mean using diffuse_template for all of your materials that should be shadowed. This is why I meant for the diffuse_template to be a simple "base" for your own lighting shaders so that you can integrate the same shadowing into your own pipeline

[xadhoom]

The shadow quality depends on the distance between the light point A
(occluder) and point B (shadow receiver). If you use your light e.g. as sun
at the sky this may lead to very bad shadows. Think about shadow mapping
where you look through the light and see all these very small (e.g.) lemmings
from above. If they are only one pixel in your map then you will get very pixelated
shadows if you look through your camera on the ground.
The other thing mentioned above is a distance inaccuracy which can occur
with such a large distance (sun - lemming - ground). The lemming and the
ground maybe have only a small offset between them in comparison to the
whole way up to the sunlight.

Your shadow should be completely black in its center if you don´t add several
passes (e.g. a shadow independent ambient pass). In the shader there is one last
calculation like this: resultingColourWithoutShadows * shadows
This leads to Colour = (0,0,0) if the shadow is (0,0,0).

Cheers!

[xadhoom]

@Null: Btw did you already look at the Nvidia PCSS implementation? The pictures look great as always...

[TheSHEEEP]

Well, at least it sounds like we're getting somewhere

Hell, yeah

1) what scale are you using for your models and world? As in, 1 unit = 1 meter, or 1 unit = 1 centimeter, or 1 unit = 1 foot, what?

We usually use 1 unit = 1 inch.
At least this is the scale of the units with the models I am able to check right now.
What is the impact of that?

2) can I get a screen shot of what you mean by the blocky/transparent shadows?

sure:

3) anything that you want to receive a shadow (including self-shadowing) will need to make use of the shadow map. For you, this would mean using diffuse_template for all of your materials that should be shadowed. This is why I meant for the diffuse_template to be a simple "base" for your own lighting shaders so that you can integrate the same shadowing into your own pipeline

Okay, I guess that one is clear now


@ xadhoom:

I get the idea behind the distance thing, but is there any easy way to change that behaviour?

[nullsquared]

Hm. What format shadow texture are you using, and what size? What hardware do you have? Can you verify that in 00diffuse.material the shadow texture_unit block has bilinear filtering? How far away is the light from your shadow casters and shadow receivers?

@ xadhoom: Yeah, I already implemented my own version of PCSS using randomly rotated PCF. In my opinion, they're too slow and too flaky for real use:
- blocker search can be very slow if you have a large search region, or just a large shadow map
- as the penumbra gets bigger, you'll also need a bigger blocker search (leading to the first problem) since otherwise you get hard edges at the penumbra edges (back to square 1)
Though now that I think about it, it might be a good idea to implement PCSS with VSM.

[TheSHEEEP]

Hm. What format shadow texture are you using, and what size? What hardware do you have?

Are you talking about this?

Code: Select all

mgr->setShadowTextureCount( 1 );
		mgr->setShadowTextureSize( 512 );
		mgr->setShadowTexturePixelFormat(PF_FLOAT16_RGB);

I changed the TextureSize to 2048.
Shadows now look less pixelated, much better basically. Nice
Still, if there are ways to improve it...

Can you verify that in 00diffuse.material the shadow texture_unit block has bilinear filtering?

I can verify that it is anispotric. I changed it to bilinear, with the same result as in the screenshot.

How far away is the light from your shadow casters and shadow receivers?

The light is at:
x="-816.37866" y="1870.793" z="-503.94312" (from the osm file)
The levelEntity (receiver) is at 0, 0, 0.

How can I get/set the position of the shadow caster?
I thought it was basically the light.

Also, how can I use the SSAO demo without using that shiny reflection? Not that it doesn't look good, it just wouldn't work with our game.

[nullsquared]

I believe the distance might have something to do with the pixelation. My units are at 1 unit = 1 meter; there's probably a precision error creeping. Does changing to FLOAT32 instead of FLOAT16 change anything? As for the shadow map size, definitely shouldn't need to be all the way at 2048, 1024 should be fine with a proper configuration.

@ shiny stuff: It's just some basic shader code for the most part. You'd need to remove the "specularContribution" part from the shader (and possibly anything that is now left unused).

[TheSHEEEP]

I believe the distance might have something to do with the pixelation. My units are at 1 unit = 1 meter; there's probably a precision error creeping. Does changing to FLOAT32 instead of FLOAT16 change anything?

Yup. The shadows look the same, but the framerate drops significantly
What is better for the quality of the shadows, far away or close light?

As for the shadow map size, definitely shouldn't need to be all the way at 2048, 1024 should be fine with a proper configuration.

I tried with 1024 but the shadows really look pixelated with that. I'd try tweaking other values, but I don't really know which.

@ shiny stuff: It's just some basic shader code for the most part. You'd need to remove the "specularContribution" part from the shader (and possibly anything that is now left unused).

Okay, I'll try that, but I already got errors from the Ogre.log as well, similar to the ones I had before. Meaning: No shadows, but "shiny stuff". In what order do the SSOA files have to be parsed?

[nullsquared]

Switching to the SSAO version of the shaders will probably cause more trouble than good, pretty much all I did was tweak a few small things here and there. Though, there is one change that I want you to test, so it'd be nice if you can post the log here. Just to clarify:
- ambient shaders should come from shadows demo
- shadow caster and diffuse shaders should come from SSAO demo

What type of light are you using? Spotlights give the best results since they can most closely be related to a camera frustum. Did you properly set your light's range? Light::setAttenuation(range, ...) Closer lights are better for the shadows.

[TheSHEEEP]

I'm at home right now, but I will test those things tomorrow when I'm at school (it's a student project).

But I can answer some light questions:

I am using a pointlight (Omni light in 3ds max) and did not do anything with the range. That's something I will also do once I can
Unfortunately I won't be able to switch to spotlights ( the light should "simulate" a strong moonlight). Well, only for the lanterns, perhaps. That depends on the framerate.

[nullsquared]

Well, the problem here is that shadow maps only cover a certain frustum; but point lights have 6 frustums (up/down, left/right, back/front). Using point lights is a big no no with standard shadow maps. What you can do is simply make your light a spot light, point it in the general direction that you want shadow casting to happen, and then shade it as a point light. This is as easy as simply removing the spotlight calculations from the shader. Now you make both the shadows happy and the lighting happy

Sorry about making this such a tough process to integrate, I was mainly targeting people who had some shader experience and just wanted a base example to work with .

[TheSHEEEP]

What you can do is simply make your light a spot light, point it in the general direction that you want shadow casting to happen, and then shade it as a point light. This is as easy as simply removing the spotlight calculations from the shader.

Errrhhhh.... Getting a spotlight in there shouldn't be the problem. But shading it as another light type? Well, I'll see if I can do that tomorrow. But I already doubt it.

Sorry about making this such a tough process to integrate, I was mainly targeting people who had some shader experience and just wanted a base example to work with .

No problem. It still seems alot easier than other shaders I tried. And on top of that, not one person in my team has ever worked with classic light maps, so that is no alternative.
Also, I have to start getting into shaders somewhere, eh?
Wish I had more time just playing around with them. But the project has to be finished, and I want f*cking non-lagging shadows in there

[nullsquared]

What you can do is simply make your light a spot light, point it in the general direction that you want shadow casting to happen, and then shade it as a point light. This is as easy as simply removing the spotlight calculations from the shader.

Errrhhhh.... Getting a spotlight in there shouldn't be the problem. But shading it as another light type? Well, I'll see if I can do that tomorrow. But I already doubt it.

It's as simple as setting the light type and setting the direction to where you want shadows (as if a spot light was casting the shadows). Then you just open up diffuse.cg and:

Code: Select all

// find
// uniform float4 spotlightParams,
// and just delete it

// find
// calculate the spotlight effect
// float spot = dot(ld0, normalize(-IN.sdir)); // angle between spotlight dir and actual dir
// spot = saturate((spot - spotlightParams.y) / (spotlightParams.x - spotlightParams.y));
// and delete it

// find
// (LdotN0 * lightDif0 * difTex.xyz) * la * spot
// and remove that find multiply by spot so it becomes
// (LdotN0 * lightDif0 * difTex.xyz) * la

Now you have a spotlight that's shaded as a point light.

Sorry about making this such a tough process to integrate, I was mainly targeting people who had some shader experience and just wanted a base example to work with .

No problem. It still seems alot easier than other shaders I tried. And on top of that, not one person in my team has ever worked with classic light maps, so that is no alternative.

Personally, I'm an all-dynamic type of guy. I have a thing against light maps since you can't just take a ceiling light, cut it's string off, and use it as a plate to beat someone's head in

Also, I have to start getting into shaders somewhere, eh?
Wish I had more time just playing around with them. But the project has to be finished, and I want f*cking non-lagging shadows in there

Right, well, good luck!

[TheSHEEEP]

Okay, first thing I did: Replacing the Omni light with a spotlight.

Switching to the SSAO version of the shaders will probably cause more trouble than good, pretty much all I did was tweak a few small things here and there. Though, there is one change that I want you to test, so it'd be nice if you can post the log here. Just to clarify:
- ambient shaders should come from shadows demo
- shadow caster and diffuse shaders should come from SSAO demo

When doing that, I couldn't find the lines to remove in diffuse.cg you quoted above (think those were from the shadow demo, right?).
Anyway, the results were: No shiny stuff, but also no shadows.

Now I replaced everything with the files from the shadow demo and deleted/replaced the lines like you explained.

This is a screen of how it looks like now, without doing anything with attenuation:



As you can see, they work, but miss any diffusion o.O
The pixelation gets worse when I decrease the texture size to 1024, so it will stay at 2048 until we found a way to make it look non-pixelated with a smaller size

Next thing I'll try is playing with attenuation settings. No time for that right now

[SwenSin]

Hmmm I wonder what is going on with the real-time graphic dictionary. I've been in the 3d biz (realtime, non-realtime) for 18 years now and "soft-shadows" always reffered to shadows that got continuosly soft (blurred) with distance since the light had been given a virtual volume/area (soft-shadows were always equal to area-shadows) (just like http://www.youtube.com/watch?v=xyPcA9QbC0k&NR=1&fmt=18 ). For what you are trying to acheive here guys is name "blurry shadows" a much much better name since your shadows are not dealing with the any virtual emission surface.

[SwenSin]

Hmmm I wonder what is going on with the real-time graphic dictionary. I've been in the 3d biz (realtime, non-realtime) for 18 years now and "soft-shadows" always reffered to shadows that got continuosly soft (blurred) with distance since the light had been given a virtual volume/area (soft-shadows were always equal to area-shadows) (just like http://www.youtube.com/watch?v=xyPcA9QbC0k&NR=1&fmt=18 ). For what you are trying to acheive here guys is name "blurry shadows" a much much better name since your shadows are not dealing with the any virtual emission surface.

[nullsquared]

They're soft shadows. Whether they're correct soft shadows or not is a different story. I hate aliased shadows, and this is what most soft shadows out there aim to fix - at reasonable speeds. Once you get correct soft shadows running at 100+ FPS for 4+ lights, tell us. And that stencil-based demo you linked to is completely scene-specfic, stencil based (misses all of the benefits of shadow mapping), oh, and slow