Shadow improvements

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Shadow improvements

Post by dark_sylinc »

I'll be working the next few weeks on implementing shadow map changes. These will probably break some compositor scripts and C++ (all which should be easy to port to the new interface), so I'm giving you a heads up.

The current sytem is ok, but it has 4 main problems:
  • No support real for UV atlases or texture arrays: We need 1 shadow map per light (and in the case of PSSM, one shadow map per split!). This wastes valuable texture slots, unnecessarily eats valuable GPU registers, and prevents certain optimizations like issuing a single clear to the whole depth texture atlas (instead of issuing N clears)
  • No support for point lights.
  • Using PSSM without a directional light in scene causes a crash.
  • No support for lazily updated lights. Some people can get away with many shadow maps because they are rarely updated. We currently don't support that because lights get sorted per frame based on distance to camera (so if Light A was assigned to shadow map 3 in the previous frame, it may be assigned to shadow map 4 in this frame). The user should be able to "freeze" and "reserve" a specific shadow map to a specific light; and this shadow map should not update unless specifically requested. Of course, if you share an UV atlas with other dynamic lights and your compositor script clears the whole atlas, that would be your fault (you should best keep an UV atlas for dynamic lights, and an UV atlas for static lights; or one shadow map per static light).
The compositor syntax will end up like this (subject to last minute changes; btw the UV offsets right now are overlapping, just ignore that):

Code: Select all

compositor_node_shadow ShadowMapDebuggingShadowNode
{
	technique pssm

	texture dynamicTex0 8192 8192 PF_D32_FLOAT
	texture staticTex0 8192 8192 PF_D32_FLOAT
	texture tmpCubemap 1024 1024 PF_D32_FLOAT  cubemap

	num_splits		2
	pssm_lambda		0.99
	shadow_map 0 dynamicTex0 light 0 split 0
	shadow_map 1 dynamicTex0 uv 0.0 0.0 0.5 0.5 light 0 split 1
	shadow_map 2 dynamicTex0 uv 0.0 0.5 0.5 0.5 light 0 split 2

	technique focused
	shadow_map 3 dynamicTex0 uv 0.0 0.0 0.5 0.5 light 1 spot point
	shadow_map 4 dynamicTex0 uv 0.0 0.5 0.5 0.5 light 2 spot point
	shadow_map 5 staticTex0 uv 0.0 0.5 0.5 0.5 light 3 spot point
	shadow_map 6 staticTex0 uv 0.0 0.5 0.5 0.5 light 4 spot point
	shadow_map 7 staticTex0 uv 0.0 0.5 0.5 0.5 light 5 spot point

	initial
	{
		target dynamicTex0
		{
			//Will clear everything in one swoop
			pass clear
			{
				colour_value 1 1 1 1
			}
		}
	}

	shadow_map_target_type directional spot
	{
		shadow_map 0 1 2 3 4
		{
			pass render_scene
			{
			}
		}

		shadow_map 5 6 7
		{
			//Will clear the particular region affected that could be dirty.
			pass render_quad
			{
				material Shadows/ClearRegion
			}
			pass render_scene
			{
			}
		}
	}

	shadow_map_target_type point
	{
		target tmpCubemap +X : cubemap_target {}
		target tmpCubemap -X : cubemap_target {}
		target tmpCubemap +Y : cubemap_target {}
		target tmpCubemap -Y : cubemap_target {}
		target tmpCubemap +Z : cubemap_target {}
		target tmpCubemap -Z : cubemap_target {}

		shadow_map 3 4
		{
			pass render_quad
			{
				material dpsm/converter
				input 0 tmpCubemap
			}
		}

		shadow_map 5 6 7
		{
			//Will clear the particular region affected that could be dirty.
			pass render_quad
			{
				material Shadows/ClearRegion
			}
			pass render_quad
			{
				material dpsm/converter
				input 0 tmpCubemap
			}
		}
	}
}
  • dynamicTex0 is the dynamic atlas
  • staticTex0 is the static atlas
  • tmpCubemap is a temporary RTT used for point lights
  • What's in "initial" gets executed in all paths at the beginning (regardless of whether the light is point or spot)
  • Shadow maps [0; 2] are used for PSSM for directional lights only.
  • Shadow maps [3; 4] are dynamic and support point and spot lights.
  • Shadow maps [5; 7] are static, support point and spot lights, and will require additional C++ code to handle their updates.
With this change, the textures are separated from shadow map definitions, as multiple shadow maps could share the same texture.

The overall syntax is a little uglier and perhaps not so straightforward compared to the current system; however this new one is much more powerful; and shadow nodes is something you deal once (usually copy paste a reference) and then forget about it.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

Woooo!!!

sounds great... while you are at it... may I ask for a feature? one I've always wanted :$
Stable pssm shadows http://www.garagegames.com/community/blogs/view/21284 http://www.ogre3d.org/forums/viewtopic.php?f=11&t=71142

thanks!
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

That's beyond the scope I'm afraid.

However implementing it shouldn't be that hard, as it involves adding a different camera setup (or modfiying existing PSSM). This is more about how textures are handled.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

dark_sylinc wrote:That's beyond the scope I'm afraid.
:(
dark_sylinc wrote:This is more about how textures are handled.
yes I did noticed.... but I guessed that since:
dark_sylinc wrote: implementing it shouldn't be that hard
... but I tried once and fail lol :p
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Shadow improvements

Post by al2950 »

xrgo wrote:
dark_sylinc wrote: implementing it shouldn't be that hard
... but I tried once and fail lol :p
Lol I would also quite like this, and have had a look at it. But the last time I looked at it I got scared off by the shadow camera code in Ogre!
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: Shadow improvements

Post by rujialiu »

Wow, that's fantastic! Just to confirm, does that mean "static lights" can do everything that "dynamic lights" do, only more powerful?

I'm thinking about simply marking every shadow-cast light as static light, and implement generic custom behavior in C++ that handles lazy update, assigning shadow maps to lights, does it sound good? Do I have to use dynamic light for some reason? Or is "dynamic light" only here to make it easier to use dynamic shadows (because it works out-of-box and doesn't need C++ code).
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

rujialiu wrote:I'm thinking about simply marking every shadow-cast light as static light, and implement generic custom behavior in C++ that handles lazy update, assigning shadow maps to lights, does it sound good? Do I have to use dynamic light for some reason? Or is "dynamic light" only here to make it easier to use dynamic shadows (because it works out-of-box and doesn't need C++ code).
Yes, and "dynamic light" is there because that's most people want or need and "it just works".
Also for directional lights, statically updated lights don't make sense.

Static lighting will require some maintenance (the obvious: if you the scene moves you're responsible for telling the light/shadow to update itself otherwise it will be out of date; you link N shadow maps to N lights... so if you change to a different different environment where there are still N lights but they're all different, you''ll have to unlink the previous lights and link to the new ones).
The advantage is that if the lights rarely require being updated, it could be quite fast to support many lights with shadow maps (though watch out for memory consumption, specially on iOS where memory is limited and iOS may suddenly terminate your app if you consume what they think is "too much").
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Update:

Whoa! A lot was done during this weekend. Progress was much faster than I hoped.
D3D11 & Metal support is broken (the shaders just haven't been ported; I need to first settle down the whole system in one API). Other than that:
  • Feature parity with the previous system (i.e. nothing the old system could do cannot be done with the new one; except for D3D11 & Metal of course)
  • UV Atlas support. UV atlas is working. On Mesa using the atlas gives a 1.7% performance boost, so I guess it's something. (though we didn't do this just for performance, but also to free up texture slots so they can be used for... you know... user textures)
  • Shadow maps supporting specific light types is working. This is important so we don't assign a point light to a shadow map that can only do directional lights (for example). This fixed a very old bug where Ogre 2.1 would crash if you enabled PSSM shadows and didn't have a directional light. This was extremely satisfying to fix.
  • Implemented executing a pass that can be used to render the contents of a shadow map, without being the shadow map itself (i.e. temporary auxiliary textures, like a shared cubemap for point lights)
  • Centralized all or most of shadow mapping shader code into one template file. Hlms template logic and macros can deal with the small syntax differences; while sharing most of the math. This is easier to maintain.
This basically paved the way for point light shadow mapping.

Now what remains:
  • Conditionally execute shadow map passes based on the light type assigned to this shadow map.
  • Write Pixel Shader that converts cubemap to DPSM (Dual Paraboloid Shadow Map).
  • Write PBS code that can handle DPSM shadow maps.
  • Add support for statically updated lights.
  • Port to D3D11 & Metal.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

dark_sylinc wrote:using the atlas gives a 1.7% performance boost
yay! <3 I love performance! you are the best!
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Small update:

Conditionally executing passes based on light type is done. Now I'm halfway there writing the DPSM converter. The converter seems to be done but it seems the actual problem is in rendering to the cubemap, something is wrong. Haven't been able to check why yet.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

thanks for the update =D
dark_sylinc wrote: Now I'm halfway there writing the DPSM converter
I was (offtopicly) wondering... can this be used for reflections? can we just render a dualparaboloid with two cameras? wold that bring any benefit?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Please note we will be rendering to cubemaps, then converting to DPSM. We won't be rendering directly to DPSM as testing shows it deforms too much when tessellation is low. We could support it, but it's not a priority.
Same goes for reflections.

The reason to use scene -> Cubemap -> DPSM is so that we keep a reasonable memory footprint and be atlas friendly. If we use cubemaps directly and want to support 8 point lights at 1024x1024, then we would have to do 1024x1024x6x8 = 192MB. However with DPSM it would be 8 DPSM and 1 cubemap: 1024x1024x4x8 + 1024x1024x4x6 = 56MB.

We do plan doing the same for PCC (Parallax Corrected Cubemaps) for iOS; since PCC are often meant to be static (you could update them in real time, but for many probes it's not very fast) and we would have the memory saving benefit (iOS memory is very limited), plus we would be able to select from arrays for enhanced auto selection (planned after the texture refactor) as iOS and DX10 level hardware do not have cubemap arrays.

Though please note a 1024x1024 DPSM has less than 1MP of effective resolution (the corners are wasted) while a 1024x1024 cubemap has 6MP of effective resolution. For shadow mapping this is not a big deal, but for storing colour data with precise information, 1 MP may not be enough.
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

thank you for the explanation!
User avatar
GlowingPotato
Goblin
Posts: 211
Joined: Wed May 08, 2013 2:58 pm
x 10

Re: Shadow improvements

Post by GlowingPotato »

Hi! A question from someone that doesn't do coding, me.

Does this new shadowing improvements have any performance or quality gain?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Hi!
From a non-coder's perspective, these improvements mean:
  • No significant performance changes. While it could be noticed performance gains in some cases (e.g. I measured 1.7% performance boost using Mesa with Radeon HD 7770 on Linux), likewise you could notice performance degradations (e.g. it's not impossible a particular GPU/driver combo ends up being 1% slower. Anyway small numbers)
  • No significant quality changes.
  • If you use the atlas, more texture slots become available. This has been a particular issue for those users with lots of textures in their materials (to be specific: many textures in the same material) which have been running into Ogre exceptions (Forward3D/Clustered eats 2 texture slots, shadows were eating several slots, etc). This should be even less of a problem after the texture refactor, but until that arrives, it's nice to have more room.
  • Point light shadow maps support. That's the biggie. You could consider that a feature or put it in "quality". Previously you could only run spot or directional lights with shadows.
  • Ogre no longer crashes if you use PSSM shadows and no directional lights are present in the scene (stability improvement)
  • Static shadow maps mean you can get a significant performance improvement if you take advantage of it (and you gain artist control over the scene's perceived quality).
The last one, as a non-coder explanation: Ogre right now recalculates the shadow maps every single frame (even if nothing has changed). However if YOU know that whatever a light is illuminating is not changing at all (or barely changing), with static shadow maps you are the one who tells Ogre when to update it (e.g. you may only need to update it three times during the whole level); hence the framerate goes up.
Perceived quality may also go up because by default Ogre applies shadow mapping on the closest lights; so shadows flip on and off as you move the camera (because lights that had no shadows get closer while lights that were using shadows get farther away). While often this is desirable, there are cases where the artist may want a particular light to always have shadows (regardless of distance); with static shadow maps you can force that; hence the perceived quality may go up (but that's up to the talent of the artist and the scene in particular).
User avatar
GlowingPotato
Goblin
Posts: 211
Joined: Wed May 08, 2013 2:58 pm
x 10

Re: Shadow improvements

Post by GlowingPotato »

Thanks a lot!

Very helpful information!
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Shadow improvements

Post by xrgo »

dark_sylinc wrote:Static shadow maps mean you can get a significant performance improvement if you take advantage of it
I am definitely going to use that... I would like to use it this way:
one big static shadow covering the whole scene... maybe 8k? maybe resolution its not that important anymore since it will be static, so it will not flick.
and one pssm shadow for dynamic objects.
the thing is... how can achieve a blend/merge for this 2 shadowmaps? is it possible to generate 2 shadowmaps with different techniques form 1 unique light (the sun!) and then both maps should be multiplied right? its done automatically?

thank youuuuuuuuuu
User avatar
GlowingPotato
Goblin
Posts: 211
Joined: Wed May 08, 2013 2:58 pm
x 10

Re: Shadow improvements

Post by GlowingPotato »

Could you guys give me an advice?

In sunset rangers we have a dynamic day/night system, but the light direction is not updated every frame, instead, we update every half a second (or close to this), would I benefit from using the static shadow map, and updating manually when we update the light direction ? Or the latency in that particular frame would be too big and stuttering would be to noticeable?
Or... I completely get the idea of the static shadow map wrong ?

Thanks.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Hi, for directional maps you can't stop updating the shadow maps because even though the light doesn't change; what is being lit (i.e. all the objects in the scene) are still moving.

Static lights are best when both the light (spot and point) and what's around it doesn't change, and only update when one of the these change. In some cases, if there's only a few dynamic objects, you can use visibility masks / render queues to have the dynamic objects not cast shadows and get away with it.
More advanced users could do "compositing" by having the static shadow have the static objects cached; and then render on top the few dynamic objects (this should be theoretically possible, I think, once I'm done).

But directional lights cover a very far range, often with many dynamic objects, which makes this unfeasible. To make it worse, directional lights (unlike spot and point) also depend on camera placement.
Although what xrgo said is not far fetched. I didn't think of it and the changes I'm doing are not meant for that use in particular (sorry xrgo), though I think what he asks shouldn't be too hard to implement.
What he is saying is that our of the 3 or 4 PSSM splits, the last one contains a shadow of the whole scene, instead of what should be covered by the last split (this makes it lower quality) and is updated rarely (if ever) with no dynamic objects in it. Because the last split puts shadows only on what's far away, the artifacts aren't as noticeable (but it is lower quality, and if your scene is very big, the quality change can be very noticeable... which you can alleviate by using huge resolutions) and the performance goes up because you save one pass from not updating the last PSSM split. A few AAA games actually do this.
User avatar
Crystal Hammer
Gnome
Posts: 317
Joined: Sat Jun 23, 2007 5:16 pm
x 77
Contact:

Re: Shadow improvements

Post by Crystal Hammer »

That's great news. I'm gonna have to build Ogre 2.1 again to check out the new stuff.
But, can you tell how will this compare to the CSM demo? Topic here http://www.ogre3d.org/forums/viewtopic.php?f=11&t=71142
I remember Cascaded Shadow Mapping being the best, at least for my purposes.
Does current PSSM still:
- freak out, go nuts, and turn to crap, when camera angle is aligned with light direction? (looses precision very badly after 1st shadowmap)
- do the texels of shadow flicker (on borders), with every camera move (even smallest)?
- and because of that even smoothing that shadow flickers
Those were the biggest elephants in the room for me, with old PSSM. And CSM fixed those.
Is there a chance of (by the way) integrating CSM camera setup into Ogre 2.1 too?
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Shadow improvements

Post by al2950 »

Good news! PSSM is way better, in fact all shadow mapping is. There is certainly no popping or 'turning to crap' issues, quality is kept when the camera is inline with the light. The only issue that remains is the 'swimming' around the borders of shadows when the camera translates/rotates, which is caused by the shadow camera not fixing its orientation, this was in fact mentioned earlier in this post.

The shadow improvements fixes is in fact one of the main reasons I decided to take the leap into 2.1, other than performance. If you have issues with shadows in your current system, I would highly recommend you checkout ogre 2.1 and build with OGRE_BUILD_SAMPLES2 cmake flag and have a look for yourself!
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

Well Good News everyone!

All the changes I proposed doing are done. Added StaticShadowMaps demo to show how to use static shadow maps (on spot and point lights).

All that's left is:
  • Try to see what to do with Metal in iOS. It works fine, but clearing the whole atlas in one go is inefficient in that platform (which is the opposite to desktop). That's fixable if you use a different shadow node definition for iOS, but it would be nice if we could share the scripts and let Ogre sort that out.
  • Test a bit more before merging to 2.1 and rolling these changes to everyone else. If someone wants to try and tell us your results that will be appreciated.
  • I see point light shadow maps are a bit buggy. I need to research on that. Fixed.
  • Update the docs
  • Fix Terra sample (it's broken now)
Cheers
Matias
farrer
Halfling
Posts: 64
Joined: Mon Sep 12, 2011 7:35 pm
x 13

Re: Shadow improvements

Post by farrer »

Just to point that I've tested here on an old Nvidia 630M notebook card, Linux (with proprietary drivers), and the average framerate of the Sample_ShadowMapDebugging goes up from the ~270 to ~380 FPS! And, what seems strange (at least for me), is that the Sample_StaticShadowMaps is running at a worst ~320 FPS. But both are better than the previous one, great work (although performance improvements weren't you main objective here)!

Edit: Ignore the framerate comparasion between the old and the new one: just remembered that my 2.1 branch was compiled with debug support :oops:

Will check latter, when I'll have some more time, the point light shadows on my project.

Thanks!
Last edited by farrer on Mon Mar 20, 2017 2:36 pm, edited 1 time in total.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadow improvements

Post by dark_sylinc »

farrer wrote:Just to point that I've tested here on an old Nvidia 630M notebook card, Linux (with proprietary drivers), and the average framerate of the Sample_ShadowMapDebugging goes up from the ~270 to ~380 FPS!
Great! That's awesome. That's a lot more than 1% and I wasn't expecting that.
farrer wrote:Just to point that I've tested here on an old Nvidia 630M notebook card, Linux (with proprietary drivers), and the average framerate of the Sample_ShadowMapDebugging goes up from the ~270 to ~380 FPS! And, what seems strange (at least for me), is that the Sample_StaticShadowMaps is running at a worst ~320 FPS. But both are better than the previous one, great work (although performance improvements weren't you main objective here)!
That's because the static shadow maps are using, on purpose, 4096x4096 shadow maps instead of 2048x2048 for the spotlights to get better quality. If you press F2 to animate the objects you will see the framerate goes down, and if you press F3 to stop updating the shadow maps even if the objects are animating, the framerate goes back to 320 fps.
Press F1 for the explanation text.
farrer
Halfling
Posts: 64
Joined: Mon Sep 12, 2011 7:35 pm
x 13

Re: Shadow improvements

Post by farrer »

dark_sylinc wrote:
farrer wrote:Just to point that I've tested here on an old Nvidia 630M notebook card, Linux (with proprietary drivers), and the average framerate of the Sample_ShadowMapDebugging goes up from the ~270 to ~380 FPS!
Great! That's awesome. That's a lot more than 1% and I wasn't expecting that.
As I edited my message at the same time you answered: I forgot that my 2.1 branch was compiled with debug mode. Recompiled it now and the correct values are from ~330 (old) to ~380 (new), which still is more than the 1% (but not so huge as my wrong comparasion). Great work, anyway!
dark_sylinc wrote: That's because the static shadow maps are using, on purpose, 4096x4096 shadow maps instead of 2048x2048 for the spotlights to get better quality. If you press F2 to animate the objects you will see the framerate goes down, and if you press F3 to stop updating the shadow maps even if the objects are animating, the framerate goes back to 320 fps.
Press F1 for the explanation text.
That's make sense, I've forgot about the shadow maps size.
Post Reply