Improving Shadows on Ogre 2.1

[dark_sylinc]

HW bilinear PCF is what you're missing:

PCF.png

We sample in those locations e.g. in location 1 and the HW will perform 2x2 PCF on samples 00 01 10 and 11. Sampling location 2 will perform 2x2 PCF on samples 10 20 11 and 21
Thus for 3x3 PCF we need to perform 4 samples.

[xrgo]

ohhh I see, so if I implement a 16 samples that should be called 5x5, and 25 samp 6x6? is that ok?

also: the offset[0] in 4x4 is (0,0)... shouldn't be that (-1,-1)? so its centered? in fact by doing that I notice that when having peter panning (or acne) I have the same amount on both sides of an object, when its (0,0) I have more peter panning (or acne) in one side, so I think it need to be centered

thank you!

[dark_sylinc]

ohhh I see, so if I implement a 16 samples that should be called 5x5, and 25 samp 6x6? is that ok?

Sounds correct

also: the offset[0] in 4x4 is (0,0)... shouldn't be that (-1,-1)? so its centered? in fact by doing that I notice that when having peter panning (or acne) I have the same amount on both sides of an object, when its (0,0) I have more peter panning (or acne) in one side, so I think it need to be centered

I have no idea. I wonder if you're right and it could improve quality.

[xrgo]

thanks!
I've made a PR:
https://github.com/OGRECave/ogre-next/pull/85

Can I also ask where in your todo list is the stable pssm xD? viewtopic.php?p=514707#p514707
to be honest the only reason I need pcf 6x6 is to avoid a little bit the flickering due to unstable pssm (which in VR is super duper annoying), implementing pcf 6x6 is in my skillset, but making the shadows stable nope (I tried)

Thanks!!!

[dark_sylinc]

I thought I had replied about this but since it was only in my head:

That technique (stable PSSM) has a few strong drawbacks which is why I stopped caring about it (at least that particular implementation):

1. The way the current PSSM works is by dividing the visible area in chunks:

PSSM.png

The pictured scenario is the ideal one (light perpendicular to lighting direction), but basically each split contains the most tight shadow map to get the most quality out of it (without sacrificing correctness within that split).

However when you rotate the camera or change the lighting direction, the size and shape of these splits change (the green, yellow and red rectangles), and that's what causes the instability.

Moving the camera doesn't change the shape of the splits, thus it is stable to position-only changes.

2. This alternate way of calculating PSSM splits was much more basic and simple:

PSSM02.png

It just grows from the center of the camera. The green split wastes space by including geometry in the shadow map which is behind the camera. But the benefit of doing that is that rotating the camera is now very stable as the split is exactly the same irrespective of camera rotations.

The problem starts... as we pay attention to the further away splits. There's a lot of wasted space (the yellow split even includes objects that were rendered for the green split; the red one includes the yellow and green ones) which will affect both performance and quality.
Like 70% of the shadow map space for the red split is wasted. Far away splits already have poor quality, this makes it a lot worse.

Perhaps a fusion of both methods (using a concentric square/sphere centered around the camera for the 1st and maybe the 2nd split; while using optimal split for the 3rd split) could be a better balance or compromise between both methods:

PSSM03.png

Notice that the green and yellow split are just concentric squares/spheres while the red one was calculated to tightly fit the frustum.

[dark_sylinc]

UPDATE: Writing things down helps a lot.
I tried a 20 minute fix and I got the hybrid working. As expected the quality is lower (more shadows look a lot more staircase for the same resolution) but are stable to camera rotations.

I will push tomorrow to its own branch for evaluation.


UPDATE 2: The only thing so far I see broken is PSSM split smooth fading between the splits that are using different methods, which is not a big surprise.

[xrgo]

Fantastic!!

Moving the camera doesn't change the shape of the splits, thus it is stable to position-only changes.

you mean this for the current implementation?, if that so, I can confirm it's not stable to position-only changes (just checked with the sample by just pressing WASD, no mouse move) for that to be stable I think the shadow map should move/snap in a world-size texel size

Perhaps a fusion of both methods

wouldn't that cause the last split to not be stable? I usually set stuff so that the last split looks very low res so the problem wold be noticeable, and in some cases I use only two splits.

you should have a look at this CSM implementation
viewtopic.php?t=71142
I remember it to be perfect when I use it back in my 1.9 days, has split smooth fading, 100% stable, even the poisson filtering was stable (it's in world space) (but pcf might be better). There's a download link at the bottom of the first post.
Maybe is easier to integrate it as an option?

edit: ohhh I see here in the still alive pictures viewtopic.php?p=467151#p467151 that this works as you mentioned before (concentric splits)

[dark_sylinc]

you mean this for the current implementation?, if that so, I can confirm it's not stable to position-only changes (just checked with the sample by just pressing WASD, no mouse move) for that to be stable I think the shadow map should move/snap in a world-size texel size

That is correct.

But the level of stability is on a whole different level when compared to camera rotation.

wouldn't that cause the last split to not be stable?

Correct, the last splits wouldn't be stable

I usually set stuff so that the last split looks very low res so the problem wold be noticeable, and in some cases I use only two splits.

You're vastly underestimating how bad it looks on the last splits. It looks really, really bad. As in "wtf is that?? it doesn't even resemble a shadow" bad.

edit: ohhh I see here in the still alive pictures viewtopic.php?p=467151#p467151 that this works as you mentioned before (concentric splits)

Yes, and as described in that post, that CSM implementation had issues based on the light direction.

[dark_sylinc]

I've added the v2-2-stable-pssm branch.
Check it out.

Use:

Code: Select all

num_stable_splits 1

In compositor node scripts to have the 1st split use the stable variant.
Use num_stable_splits >= num_splits to have all splits use the stable variant (num_stable_splits > num_splits is the same as num_stable_splits = num_splits).

In code it's ShadowTextureDefinition::numStableSplits

There are artifacts (for some reason, yet to be researched) when the pixels switch from stable to the unstable variant. If it weren't for that, I would merge it with ifd branch
Edit: And just like that... the artifacts are gone. I cannot reproduce them anymore. Maybe I fixed something while preparing the code for pushing to git?

Edit 2: Could you test how it works for you? If all is well, I'll merge to the main v2-2-irradiance-field branch

[xrgo]

mmmm, clean build, just added planarreflections and buildtests on cmake

[dark_sylinc]

I just checked and it doesn't seem like I forgot to commit anything.

ConcentricShadowCamera is defined in OgreMain/include/OgreShadowCameraSetupConcentric.h

But I just noticed that this was missing, which is important for MSVC:

Code: Select all

#include "OgreStableHeaders.h"

Could you pull & try again?

[xrgo]

Thanks! yes that was it
its working, but as you mentioned only for rotation changes, not for position changes, so I am 100% certain that is not going to improve anything in VR since its imposible to not translate your head as you look around, even when you think you're standing still camera position values change, so it will flicker

its missing this important part (from the csm code):

Code: Select all

		// Round local x/y position based on a world-space texel; this helps to reduce
		// jittering caused by the projection moving with the camera
		Real worldTexelSize = (texCam->getOrthoWindowWidth()) / texCam->getViewport()->getActualWidth();

		//convert world space camera position into light space
		Vector3 lightSpacePos = q.Inverse() * pos;
		
		//snap to nearest texel
		lightSpacePos.x -= fmod(lightSpacePos.x, worldTexelSize);
		lightSpacePos.y -= fmod(lightSpacePos.y, worldTexelSize);

		//convert back to world space
		pos = q * lightSpacePos;

But the level of stability is on a whole different level when compared to camera rotation.

I am going to try now anyways

It looks really, really bad. As in "wtf is that?? it doesn't even resemble a shadow" bad.

yeah I noticed xD maybe I can play with resolution, but thanks for giving an easy way to change numStableSplits

Yes, and as described in that post, that CSM implementation had issues based on the light direction.

I used it a lot back in the days and never noticed any issue, might be a very rare case

[xrgo]

OK, I tried in VR and if I "stand still" it flicks like 80% less than before, but if I move around it flicks like 10% less than before. knowing that the later case is practically 100% of the time in VR is not a great gain, considering that it also looks more pixelated, the only way is to make it stable for position changes.

I tried to integrate the code in csm

Code: Select all

Vector3 dir;
    Quaternion q;

// Calculate texCam direction, which same as directional light direction
    dir = -light->getDerivedDirection();  // backwards since point down -z
    dir.normalise();

    Vector3 up = Vector3::UNIT_Y;
    // Check it's not coincident with dir
    if( Math::Abs( up.dotProduct( dir ) ) >= 1.0f )
    {
        // Use camera up
        up = Vector3::UNIT_Z;
    }
    // cross twice to rederive, only direction is unaltered
    Vector3 left = dir.crossProduct( up );
    left.normalise();
    up = dir.crossProduct( left );
    up.normalise();
    // Derive quaternion from axes
    q.FromAxes( left, up, dir );


// Round local x/y position based on a world-space texel; this helps to reduce
    // jittering caused by the projection moving with the camera
    Real worldTexelSize =
        ( texCam->getOrthoWindowWidth() ) / 2048;//hardcoded value to test

    // convert world space camera position into light space
    Vector3 lightSpacePos = q.Inverse() * shadowCameraPos;

    // snap to nearest texel
    lightSpacePos.x -= fmod( lightSpacePos.x, worldTexelSize );
    lightSpacePos.y -= fmod( lightSpacePos.y, worldTexelSize );
    lightSpacePos.z -= fmod( lightSpacePos.z, worldTexelSize );
    // convert back to world space
    shadowCameraPos = q * lightSpacePos;

and it flicks waaaaaay less frecuent, but the intensity of the flick is the same (not sure if that makes sense), and the fact that happens less periodically makes it more distracting xD, so it have to be perfect :/

Thanks!

[xrgo]

ok, apparently this works (at least in the sample):

Code: Select all

        Quaternion q = light->getParentSceneNode()->_getDerivedOrientationUpdated();

		// Round local x/y position based on a world-space texel; this helps to reduce
        // jittering caused by the projection moving with the camera
        Real worldTexelSize =
            ( texCam->getOrthoWindowWidth() ) / 2048; //the first split is 2048 so it works perfectly int that split

        // convert world space camera position into light space
        Vector3 lightSpacePos = q.Inverse() * shadowCameraPos;

        // snap to nearest texel
        lightSpacePos.x -= fmod( lightSpacePos.x, worldTexelSize );
        lightSpacePos.y -= fmod( lightSpacePos.y, worldTexelSize );
        // convert back to world space
        shadowCameraPos = q * lightSpacePos;

I just need to know how to get the split resolution for that division

Edit: doh... viewportRealSize!! seems to be working really perfectly on the sample, going to check in vr now

Edit2: OMG it looks really really really goooood, finally no more flickery mess, I am going to test other projects

Edit3: I tested a couple of other projects and is really good, I only get a tiny tiny flick sometimes but its just one pixel in the border of the shadow, its very very minimal (pretty sure is because the object casting that shadow is a rigid body with constraints that seems to be still but its actually vibrating a bit) so I'll call it perfect!

should I make a PR? I also had to add this:

Code: Select all

        static void createShadowNodeWithSettings( CompositorManager2 *compositorManager,
                                                  const RenderSystemCapabilities *capabilities,
                                                  const String &shadowNodeName,
                                                  const ShadowNodeHelper::
                                                  ShadowParamVec &shadowParams,
                                                  bool useEsm,
                                                  uint32 pointLightCubemapResolution=1024u,
                                                  Real pssmLambda=0.95f, Real splitPadding=1.0f,
                                                  Real splitBlend = 0.125f, Real splitFade = 0.313f,
                                                  uint32 numStableSplits = 0,       <<<<<<<<<<<<<<<<<<<<<<<THIS (and the cpp impl obviously)
                                                  uint32 visibilityMask = VisibilityFlags::RESERVED_VISIBILITY_FLAGS );

I am going to sleep now, very very happy, thanks for everything, I love you <3

[TaaTT4]

should I make a PR?

Yes, please! I've put VR on hold atm, but I'm interested in this anyway since I have the same problem when the car is stopped with the engine turned on (the camera has micro-shakes due to the engine vibration).


O.T.: I guess you have a similar problem with alpha masked fences. How do you alleviate the issue in that case?

[TaaTT4]

UPDATE 2: The only thing so far I see broken is PSSM split smooth fading between the splits that are using different methods, which is not a big surprise.

Yes, because when I made that PR I didn't take in account that splits could also being concentric (don't have the crystall ball ). Can't have a look at it atm, but I guess that the PSSM splits blend system can be easily adapted to work with your changes.

[dark_sylinc]

UPDATE 2: The only thing so far I see broken is PSSM split smooth fading between the splits that are using different methods, which is not a big surprise.

Yes, because when I made that PR I didn't take in account that splits could also being concentric (don't have the crystall ball ). Can't have a look at it atm, but I guess that the PSSM splits blend system can be easily adapted to work with your changes.

Actually the artifact is gone; and your code should work without modification since the input your code relies on isn't affected it (distance from view point to slice).
Anyway the artifact doesn't seem to be anywhere now so I'm guessing it was a bug from one of the hardcoded values while I was prototyping.

should I make a PR? I also had to add this:

Code: Select all

        static void createShadowNodeWithSettings( CompositorManager2 *compositorManager,
                                                  const RenderSystemCapabilities *capabilities,
                                                  const String &shadowNodeName,
                                                  const ShadowNodeHelper::
                                                  ShadowParamVec &shadowParams,
                                                  bool useEsm,
                                                  uint32 pointLightCubemapResolution=1024u,
                                                  Real pssmLambda=0.95f, Real splitPadding=1.0f,
                                                  Real splitBlend = 0.125f, Real splitFade = 0.313f,
                                                  uint32 numStableSplits = 0,       <<<<<<<<<<<<<<<<<<<<<<<THIS (and the cpp impl obviously)
                                                  uint32 visibilityMask = VisibilityFlags::RESERVED_VISIBILITY_FLAGS );

Nice!!!
Yes please, submit a PR!

[xrgo]

PR done: https://github.com/OGRECave/ogre-next/pull/86

yes, I can confirm that fade between splits works fine

I guess you have a similar problem with alpha masked fences. How do you alleviate the issue in that case?

yes! I think that's one of the last flickering sources left, he problem is workarounded by using alpha blend instead of alpha clip, but that's expensive, I have no real solution for now, haven't looked much though

thank you for everything

[xrgo]

thanks for the merge and the corrections/improvements!
I'll report in a few days if I see something strange