Unexpected Results with Shadows and Transparent Materials

Problems building or running the engine, queries about how to use features etc.
dcavallini
Gnoblar
Posts: 20
Joined: Fri Jul 16, 2021 10:04 am
x 2

Unexpected Results with Shadows and Transparent Materials

Post by dcavallini »

Version 14.0.0 (Tsathoggua):
Mac OS e Windows 11:
GL 3+,Direct3D11:

I enabled the shadow technique SHADOWTYPE_STENCIL_ADDITIVE and set the scene blending in the material to SBT_TRANSPARENT_ALPHA. When I try to render a textured mesh, I get an incorrect result, as shown in the attached image, when there is an object positioned behind it. What could be the issue?
Image

rpgplayerrobin
Orc Shaman
Posts: 710
Joined: Wed Mar 18, 2009 3:03 am
x 391

Re: Unexpected Results with Shadows and Transparent Materials

Post by rpgplayerrobin »

This is a transparency issue. Try also with depth write being off in the material.
Make sure that the floor is not using any transparency at all.

Also, usually, trees leaves are not using transparency, instead they are using alpha rejection, which completely solves the problem you are having as well, since that does not require scene_blend alpha_blend or depth_write off in the material.

And by the way, the shadow will always be square like that since stencil has no idea of how the texture looks, so stencil is an outdated shadow technique.

dcavallini
Gnoblar
Posts: 20
Joined: Fri Jul 16, 2021 10:04 am
x 2

Re: Unexpected Results with Shadows and Transparent Materials

Post by dcavallini »

If I enable alpha rejection small improvement:
Image
If I turn off shadows good result:
Image

User avatar
Crystal Hammer
Orc
Posts: 402
Joined: Sat Jun 23, 2007 5:16 pm
x 112

Re: Unexpected Results with Shadows and Transparent Materials

Post by Crystal Hammer »

Why are you using SHADOWTYPE_STENCIL? Aren't stencil shadows a thing of the past, like from 20 years ago?
https://ogrecave.github.io/ogre/api/lat ... adows.html
Any reasons not to use texture shadowmaps?
SHADOWTYPE_TEXTURE_MODULATIVE or SHADOWTYPE_TEXTURE_ADDITIVE
Been a while since I've seen stencil shadows, but since those are just triangles, I think the 1st image is the most that you can get for leaves, shadow will be from triangles here. I belive you can only get nice leaves, alpha tested shadows with texture shadowmap only.

dcavallini
Gnoblar
Posts: 20
Joined: Fri Jul 16, 2021 10:04 am
x 2

Re: Unexpected Results with Shadows and Transparent Materials

Post by dcavallini »

I know using the texture based shadows technique is better in many ways. But the problem described here:
https://forums.ogre3d.org/viewtopic.ph ... 18#p553018
I couldn't solve.

rpgplayerrobin
Orc Shaman
Posts: 710
Joined: Wed Mar 18, 2009 3:03 am
x 391

Re: Unexpected Results with Shadows and Transparent Materials

Post by rpgplayerrobin »

The problem with that post is that you are using simple texture shadows, which makes it so that either an object is a shadow caster OR it is a shadow receiver.

But RTSS (or your own shaders) can instead use depth texture shadows, which solves this problem.

Depth shadows are needed for objects to know when the shadow starts, because as you may have seen with normal texture shadows, the shadow is actually shadowing things behind AND in front of the object, which is bad. So depth texture shadows solves all problems.

However, as I have no idea how to actually use RTSS (I instead wrote my own shaders for this), I cannot help you further. You would have to research a bit on how RTSS works and how to enable depth shadows for it.

You could start from here:
https://ogrecave.github.io/ogre/api/lat ... ow_mapping