[2.1] Screen Space Decals bug Topic is solved

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

[2.1] Screen Space Decals bug

Post by rujialiu »

Hi!

Out of curiosity, I've tried the decals branch. It's really nice overall, but I found two bugs:

1. If I only use emissive decals, not diffuse/normal decals, I would get the following shader compilation error:
...\HlmsDebug\537035522PixelShader_ps.hlsl(800,42-70): error X4000: variable 'decalMask' used without having been completely initialized

It looks trivial to fix, though.

In Forward3D_pieces_ps.any, after this:

Code: Select all

        @property( hlms_decals_diffuse )
...
        @end
I added some codes:

Code: Select all

        @property( !hlms_decals_diffuse )
            decalMask = isOutsideDecal ? 0.0f : 1.0f;
        @end
That makes sure decalMask is always initialized.

Then it works most of the time. However, sometimes part of the decals just "disappear". Looks like a bug in Forward Cluster intersection code? Anyway, I've simplified my use-case and modified the sample. When you hit F4, some parts of the spheres are not covered by the decal. Here is the diff:

Code: Select all

 Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp b/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp
index ff4e878..47ab591 100644
--- a/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp
+++ b/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp
@@ -64,7 +64,7 @@ namespace Demo
             Ogre::SceneNode *sceneNode = sceneManager->createSceneNode();
             sceneNode->attachObject( decal );
             sceneNode->setPosition( Ogre::Vector3( 0, 0.4, 0 ) );
-            sceneNode->setOrientation( Ogre::Quaternion( Ogre::Degree( 45.0f ), Ogre::Vector3::UNIT_Y ) );
+            sceneNode->setOrientation( Ogre::Quaternion( Ogre::Degree( 45.0f ), Ogre::Vector3::UNIT_X ) );
             sceneNode->setScale( Ogre::Vector3( 10.0f ) );
             wireAabb->track( decal );
 
@@ -102,8 +102,8 @@ namespace Demo
 
         Ogre::v1::MeshPtr planeMeshV1 = Ogre::v1::MeshManager::getSingleton().createPlane( "Plane v1",
                                             Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME,
-                                            Ogre::Plane( Ogre::Vector3::UNIT_Y, 1.0f ), 50.0f, 50.0f,
-                                            1, 1, true, 1, 4.0f, 4.0f, Ogre::Vector3::UNIT_Z,
+                                            Ogre::Plane( Ogre::Vector3::UNIT_Z, 1.0f ), 50.0f, 50.0f,
+                                            1, 1, true, 1, 4.0f, 4.0f, Ogre::Vector3::UNIT_X,
                                             Ogre::v1::HardwareBuffer::HBU_STATIC,
                                             Ogre::v1::HardwareBuffer::HBU_STATIC );
 
@@ -221,8 +221,8 @@ namespace Demo
                     Ogre::SceneNode *sceneNode = sceneManager->getRootSceneNode( Ogre::SCENE_DYNAMIC )->
                             createChildSceneNode( Ogre::SCENE_DYNAMIC );
                     sceneNode->setPosition( Ogre::Vector3( armsLength * x - startX,
-                                                           1.0f,
-                                                           armsLength * z - startZ ) );
+                                                           armsLength * z - startZ, 
+                                                           4.0f));
                     sceneNode->attachObject( item );
                 }
             }
It's basically rotate the decal (don't know whether it's crucial, though), the "background plane" and the carefully place the spheres.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

Thanks for the bugfix! You told me in private but I completely forgot! Pushed.

As for the bug you mention... there is no bug! At least, not in the scene you provided.
I will soon be pushing a debug utility to help visualize the internal representation.

This is the same scene you posted, with the debug visualizer:

The plane in the middle is the actual decal plane that is being projected, the cube is the "area of influence" of the decal.



As you can see the decal is working as intended, it's just that either you misunderstood how it works, or you thought the orientation you thought you set is different from the actual orientation.

Cheers
Matias
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: [2.1] Screen Space Decals bug

Post by rujialiu »

dark_sylinc wrote: Fri Sep 21, 2018 4:26 pm As you can see the decal is working as intended, it's just that either you misunderstood how it works, or you thought the orientation you thought you set is different from the actual orientation.
Thanks! It looks like I misunderstood how it works, and I still don't know... Could you briefly explain it? Also, in my real use case, the disappearing part is view-dependent. The area of influence is a long and narrow vertical cube, which intersects a big vertical plane, and I want to project the decal onto the plane. When I go approaching the vertical plane, sometimes a part of the decal disappeared, but when I move the camera closer/farther, the SAME part appear again. Is that possible?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

Pushed debug visualization.
rujialiu wrote: Fri Sep 21, 2018 5:05 pm Thanks! It looks like I misunderstood how it works, and I still don't know... Could you briefly explain it?
Decals are 2D in nature, like a sticker. Their natural representation is being a plane in the XZ space, which can be rotated in XYZ space.

However a 2D plane has infinitely small thickness. And we also need a way to tell the rendering engine where the sticker is pasted onto.
That's where the 3rd component enters into action:

This is the decal, acting like a sticker on the floor:


When we activate the debug visualization, we see the sticker was placed slightly above the ground (this is not relevant) and it has depth:


Everything that is inside that cube will be affected by the "sticker" and now's when the sticker analogy begins to fall part, because even if it's from behind the spheres get affected:


With debug visualizer off:


If we rotate the sticker, we'll see it gets "plastered" into the objects that are within the cube, in the direction of the arrows:


With debug visualizer off:


The sticker analogy is not entirely correct: With a sticker in real life, if you paste that sticker into a tube, the sticker will bend and envelop the tube.
With decals, the texture gets projected into the tube, giving you a distorted image. See slide 60 "Side Stretching" of the Warhammer's Screen Space Decals presentation.
rujialiu wrote: Fri Sep 21, 2018 5:05 pm Also, in my real use case, the disappearing part is view-dependent. The area of influence is a long and narrow vertical cube, which intersects a big vertical plane, and I want to project the decal onto the plane.
Mmm... while making the photos above for your example, I noticed this bug. Requires further investigation.

The following scene reveals the bug:

Code: Select all

diff -r c0f7516b49c6 Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp
--- a/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp	Fri Sep 21 14:18:55 2018 -0300
+++ b/Samples/2.0/ApiUsage/Decals/DecalsGameState.cpp	Fri Sep 21 14:48:57 2018 -0300
@@ -130,9 +130,9 @@
             Ogre::Decal *decal = sceneManager->createDecal();
             Ogre::SceneNode *sceneNode = sceneManager->createSceneNode();
             sceneNode->attachObject( decal );
-            sceneNode->setPosition( Ogre::Vector3( 0, 0.4, 0 ) );
-            sceneNode->setOrientation( Ogre::Quaternion( Ogre::Degree( 45.0f ), Ogre::Vector3::UNIT_Y ) );
-            sceneNode->setScale( Ogre::Vector3( 10.0f ) );
+            sceneNode->setPosition( Ogre::Vector3( 0, 2.1, 0 ) );
+            sceneNode->setOrientation( Ogre::Quaternion( Ogre::Degree( 45.0f ), Ogre::Vector3::UNIT_X ) );
+            sceneNode->setScale( Ogre::Vector3( 5.0f, 1.5f, 5.0f ) );
             wireAabb->track( decal );
 
             Ogre::HlmsTextureManager::TextureLocation texLocation;
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

Pushed fix for the culling bug.

Let me know if that fixes your problems.

Cheers
Matias
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: [2.1] Screen Space Decals bug

Post by rujialiu »

dark_sylinc wrote: Fri Sep 21, 2018 6:42 pm Decals are 2D in nature, like a sticker. Their natural representation is being a plane in the XZ space, which can be rotated in XYZ space.
So when in its natural representation, it's only projecting from top of the cube to the bottom of the cube?
Everything that is inside that cube will be affected by the "sticker" and now's when the sticker analogy begins to fall part, because even if it's from behind the spheres get affected:
This is where I'm confused. By setting the postion/orientation/scale of the decal's parent scene node, are we manupulating the "area of influence cube"? If so, how is the "sticker" determined? Just the bottom side of the cube (before rotating) with some depth?
If we rotate the sticker, we'll see it gets "plastered" into the objects that are within the cube, in the direction of the arrows:
In the picture, it looks like only the sticker is rotated, but the cube is not. How is it possible???
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

rujialiu wrote: Sat Sep 22, 2018 2:53 am
dark_sylinc wrote: Fri Sep 21, 2018 6:42 pm Decals are 2D in nature, like a sticker. Their natural representation is being a plane in the XZ space, which can be rotated in XYZ space.
So when in its natural representation, it's only projecting from top of the cube to the bottom of the cube?
We could say it that way. But please note going from bottom to top is the same top to bottom.
rujialiu wrote: Sat Sep 22, 2018 2:53 am
Everything that is inside that cube will be affected by the "sticker" and now's when the sticker analogy begins to fall part, because even if it's from behind the spheres get affected:
This is where I'm confused. By setting the postion/orientation/scale of the decal's parent scene node, are we manupulating the "area of influence cube"? If so, how is the "sticker" determined? Just the bottom side of the cube (before rotating) with some depth?
Perhaps a different way of looking at it is: Forget about stickers, and think about a rectangular flashlight with a texture on it. Everything hit by the flashlight will have the texture on it.... even if it's facing away from the light (i.e. normals are ignored)

To get a sense of how the projection works, I made these two videos:



In the first video, the projection deforms as the objects stop being aligned with the decal.
In the second video, I am using a very depth decal so that it affects many objects. Note that going from the front or from behind the texture is the same.
rujialiu wrote: Sat Sep 22, 2018 2:53 am
If we rotate the sticker, we'll see it gets "plastered" into the objects that are within the cube, in the direction of the arrows:
In the picture, it looks like only the sticker is rotated, but the cube is not. How is it possible???
It IS rotated. But because it is projected instead of being plastered, it may not look like it.

I made a few videos to help you understand it. But if you're having trouble, I'd suggest you try to make your decals interactive, that can be moved with the cursor. It should help you understand it.

For this video, I set the following to the decal's node:

Code: Select all

sceneNode->setPosition( Ogre::Vector3( 5, 0.4, 0 ) );
sceneNode->setOrientation( Ogre::Quaternion( Ogre::Degree( 90.0f ), Ogre::Vector3::UNIT_Y ) *
                           Ogre::Quaternion( Ogre::Degree( 90.0f ), Ogre::Vector3::UNIT_X ) );
sceneNode->setScale( Ogre::Vector3( 10, 2.0f, 2 ) );
And then I rotated it around. This is the result:


Note that we can customize the shader code further so that the object's normal is taken into account and prevent the decal from showing if it's receiving it "from behind". That would be a 5 minute tweak to the shader code.
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: [2.1] Screen Space Decals bug

Post by rujialiu »

dark_sylinc wrote: Sat Sep 22, 2018 4:41 am Note that we can customize the shader code further so that the object's normal is taken into account and prevent the decal from showing if it's receiving it "from behind". That would be a 5 minute tweak to the shader code.
That would be helpful (maybe a switch?). Usually I don't want other 4 sides except top/bottom of the "area of influece" cube to receive decals. These are not receiving "from behind", but perpendicular.

I'll spend some time to understand your explanations & videos, try your fix and analyze my use case. Thanks again!
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: [2.1] Screen Space Decals bug

Post by rujialiu »

dark_sylinc wrote: Fri Sep 21, 2018 8:44 pm Pushed fix for the culling bug.
Let me know if that fixes your problems.
This fixed my (partially disappearing decals) problems. Thanks!!!
However, the latest commit still didn't solve the compile error problem.

I still have to add the following

Code: Select all

        @property( !hlms_decals_diffuse )
            decalMask = isOutsideDecal ? 0.0f : 1.0f;
        @end
before your new code:

Code: Select all

        @property( hlms_decals_normals || hlms_decals_emissive )
            @property( hlms_decals_diffuse )
                decalMask = ignoreAlphaDiffuse ? 1.0f : decalMask;
            @end
            decalMask = isOutsideDecal ? 0.0f : decalMask;
        @end
Otherwise the generated code is still wrong:

Code: Select all

float decalMask;
decalMask = isOutsideDecal ? 0.0f : decalMask;
Lax
Hobgoblin
Posts: 583
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 50

Re: [2.1] Screen Space Decals bug

Post by Lax »

Hi,

I'm having issues getting decals to work. I orientated on the example. I get the following directX errors:

Code: Select all

Cannot compile D3D11 high-level shader 537199626PixelShader_ps Errors:
(1029,5-10): error X3018: invalid subscript 'xyz'
 in D3D11HLSLProgram::compileMicrocode at D:\Ogre\GameEngineDevelopment\external\Ogre2.1SDK\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 549)
I compared the 2.0 resources with my resources and found no differences.

Has anybody an idea, what is going wrong?

Thanks in advance.

Regards
Lax

http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

Could you upload the generated 537199626PixelShader_ps.hlsl?
Thanks
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: [2.1] Screen Space Decals bug

Post by rujialiu »

Now we successfully integrated decals to our project to do some custom/advanced lighting. After one minor fix to the shader, everything is working now.

In Forward3D_piece_ps.any:

Code: Select all

float2 decalUvDdx;
decalUvDdx.x = dot( invWorldView0.xyz, posDdx.xyz );
decalUvDdx.y = dot( invWorldView2.xyz, posDdx.xyz );
float2 decalUvDdy;
decalUvDdy.x = dot( invWorldView0.xyz, posDdy.xyz );
decalUvDdy.y = dot( invWorldView2.xyz, posDdy.xyz );
It turns out that sometimes dot is negative, causing artifacts around the edges of the decals. I added four clamp(dot(...), 0.0f, 1.0f) around each of the four dot calls and it solves the problem. I've only tested DX11, though.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Screen Space Decals bug

Post by dark_sylinc »

Does the problem go away if you use abs() instead of clamping?
Can I see a pic of the problem?

Thanks
Post Reply