[2.1] Hlms Normal Map question

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

Yes!!! Now is packing everything!!
Thank you very much

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

I think I finally solved it!!!!! (the original post problem), after more than a year living with this issue xD

apparently this is a common problem:
https://forums.unrealengine.com/showthr ... ues-in-4-4
http://polycount.com/discussion/128766/ ... ements/p43
http://forum.unity3d.com/threads/calcul ... 781/page-4
http://forum.unity3d.com/threads/unity- ... rd.366291/

and they mention Mikktspace, that it seems to be an standard for tangent space normals calculation, that was the key, so here:
http://eat3d.com/forum/questions-and-fe ... on-scaling
and here:
https://wiki.blender.org/index.php/Dev: ... ormal_Maps
are the formulas, I just used them in the shader (instead of the TBN) and it works!! now my smooth cube with only 8 vertices looks like the flat cube with 24 vertices... obviously, its not perfect, you can see the pixels of the normal map depending on the resolution (like the images in this topic
http://forum.unity3d.com/threads/unity- ... rd.366291/)

relevant code

Code: Select all

        //in the pixel shader
	vec3 vBinormal	= normalize( cross( inPs.normal, inPs.tangent )@insertpiece( tbnApplyReflection ) );
        //nNormal = normalize( TBN * nNormal );
	nNormal = normalize(nNormal.x * inPs.tangent + nNormal.y * vBinormal + nNormal.z * inPs.normal);
I do have to test with some more meshes, just tested with a cube and its working :P
Last edited by xrgo on Fri Mar 18, 2016 11:14 pm, edited 1 time in total.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.1] Hlms Normal Map question

Post by dark_sylinc »

Thanks for all the links. Very interesting. I'll read them later.

For the record, the only change you've made was that pixel shader snippet you posted?

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

dark_sylinc wrote:For the record, the only change you've made was that pixel shader snippet you posted?
yes! I think so, I have edited many many stuffs in the shaders, but I think that should be enough.
I edited the code in the post to fit the default pbs shader

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

I just tested with a more complex mesh it works great, one problem is that you can notice when the mip changes, and when too far it starts looking non flat again, because its using a very high mip. Its not really a problem, its barely noticeable, and one can always sample the texture with textureLod(,,0) and its gone :P.
Another problem for me specifically (not sure if this also happens with exporters), since I read the the data from the blender file and I construct the mesh in code, quads sometimes gets triangulated in the different direction, and the normals look bad, the solution is simply to triangulate the mesh before baking so I can guarantee that triangles are exactly the same.

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

And I am having another problem! that I had even before this mikktspace fix...

when I scale the object normals go wrong, in other word it looks good only at scale(1,1,1).
But! I solved for any uniform scale!, int the vertex shader, normalize normal and tangent:

Code: Select all

outVs.normal = normalize( mat3(@insertpiece( worldViewMat )) * @insertpiece(local_normal) );
outVs.tangent = normalize( mat3(@insertpiece( worldViewMat )) * @insertpiece(local_tangent) );
it has to be normalized in the vertex shader!! if done in the pixel shader, mikktspace fix wont work and normals would look wrong :P

but don't fix for non-uniform scale =(, the problem seems to come from the "worldViewMat" but I have no idea how to solve it :P

EDIT: here they talk about nonuniform scale and normal maps http://stackoverflow.com/questions/1382 ... ng-in-glsl but its still chinesse to me :lol:

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.1] Hlms Normal Map question

Post by dark_sylinc »

xrgo wrote:EDIT: here they talk about nonuniform scale and normal maps http://stackoverflow.com/questions/1382 ... ng-in-glsl but its still chinesse to me :lol:
Translating the chinese: The solution would be incredibly expensive.
We exploit a property that when all three axis are 90° apart from each other, like in the picture:
Image
This is called being orthogonal. The inverse of a matrix is now equal to the transpose. Calculating the transpose is cheap and easy (just swap matrix[j] for matrix[j]). Calculating the inverse is expensive. You can take a look at Matrix3::Inverse from our C++ codebase if you're interested. But I guess you'll prefer living with the artifact.

The TBN matrix translates from world space to tangent space. Thus we need actually use the inverse of the TBN matrix to translate from tangent space to world space. Since we ensure the TBN matrix is orthogonal, we just use the transpose.

For the record, this is how non-orthogonal axes look like:
Image
Note one of the axis is not 90° from the others.

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

dark_sylinc wrote:The solution would be incredibly expensive.
:(
expensive for the cpu?, that would be done in this line?

Code: Select all

    const Ogre::Matrix4 &worldMat = queuedRenderable.movableObject->_getParentNodeFullTransform();
can we detect when the scale is uniform and use the transpose and when not use the inverse? so it wont be expensive in every case... can be done in the shader?

I am having trouble with this, not only with this smooth-flat cube cases... I have a desert like terrain and I want to use some assets with different scales, like they do in this video (ue4) https://youtu.be/MlgvfEicdwU?t=184 but normal's looks really bad... :(

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.1] Hlms Normal Map question

Post by dark_sylinc »

It's expensive because you would have to calculate the inverse in the pixel shader.
I'll have to take a look to see all the resources you've posted and experiment to see what can be done to alleviate the artifacts you talk about; but I don't plan on doing that very soon. Your links definitely prove we have room for improvement though.

Taking a quick glance at the videos it appears the best solution short-term solution would be to bake the scale into the mesh and recompute the normals (i.e. use the artifact version during editing, once you're done editing start baking the scale of those meshes if it's non uniform and recalculate tangents).

PS. Very cool video.

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

dark_sylinc wrote:I'll have to take a look to see all the resources you've posted and experiment to see what can be done to alleviate the artifacts you talk about; but I don't plan on doing that very soon. Your links definitely prove we have room for improvement though.
Thank you!!
dark_sylinc wrote:the best solution short-term solution would be to bake the scale into the mesh and recompute the normals (i.e. use the artifact version during editing, once you're done editing start baking the scale of those meshes if it's non uniform and recalculate tangents).
yes, that's what I have in mind

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

I am going to leave this here for later

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4511
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 943
Contact:

Re: [2.1] Hlms Normal Map question

Post by dark_sylinc »

xrgo wrote:
Mon Feb 03, 2020 2:13 pm
I am going to leave this here for later
New ticket, PRs & help is welcomed :P

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

Thanks for the tips! I am going to try this weekend :D

User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 241
Joined: Wed Apr 23, 2014 3:49 pm
x 45

Re: [2.1] Hlms Normal Map question

Post by TaaTT4 »

Hey @xrgo,

Any progress on this topic?
Senior game programmer at Vae Victis
Working on Racecraft

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Hlms Normal Map question

Post by xrgo »

TaaTT4 wrote:
Thu Mar 12, 2020 2:54 pm
Hey @xrgo,

Any progress on this topic?
Nope, sorry =(, really hard to find the time

User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 241
Joined: Wed Apr 23, 2014 3:49 pm
x 45

Re: [2.1] Hlms Normal Map question

Post by TaaTT4 »

I'm resuming this old post because I'd like to tackle the issue described here (see also here). To be honest, I've already replaced TangentSpaceCalc with MikkTSpace in a local branch of my OGRE fork, but this causes a lot of visual artifacts. I believe there's something broken behind or I'm missing some detail, so let's take a step back.

Following what @xrgo said in his first message, I've created the following Blender scene (hoping to not have made mistakes: it's the first time ever that I use Blender).
Image
Smooth, Smooth_norm and Smooth_norm_tan are cubes with "shade smooth", while Flat is a cube with "shade flat". UV unwrap has been made using the "Smart UV Project" tool. In addition, since I've read that could affect the normal map baking process, I triangulate faces of Smooth_norm and Flat cubes.

Then, I've followed this article to bake the normal map of Flat cube over Smooth_norm. I've baked both versions: DirectX (with -Y) and OpenGL (with +Y). I've then binded the baked normal map (OpenGL version) to the materials of Smooth_norm and Smooth_norm_tan cubes and they became flat.
Image
Note the the hard edges, in yellow, as well as some visual artifacts (reflection maybe?!), in red.

I've then proceeded with the export of the cubes. The Smooth, Smooth_norm and Flat cubes has been exported without letting Blender to calculate their tangents (to being able to generate them in OGRE then). The contrary instead has been done for Smooth_norm_tan cube. QTangents generation has been disabled for all cubes.
Image Image

Then, I've converted the baked normal maps from PNG to DDS (BC5_SNORM format) and I've generated their mipmaps. I've also created the following material:

Code: Select all

hlms Smooth_norm_dx pbs
{
	normal_map		Smooth_norm_dx.dds
	reflection_map	SaintPetersBasilica.dds
}

hlms Smooth_norm_gl pbs
{
	normal_map		Smooth_norm_gl.dds
	reflection_map	SaintPetersBasilica.dds
}
I've then put all the assets in OGRE and I've modified the Sample_PbsMaterials with the following patch:

Code: Select all

From 9451aab3bb4db3d704263b6fd351fcef535143b7 Mon Sep 17 00:00:00 2001
From: TaaTT4 <raffaele.bratta@gmail.com>
Date: Thu, 11 Jun 2020 18:38:40 +0200
Subject: [PATCH] FOO

---
 .../PbsMaterials/PbsMaterialsGameState.cpp    | 78 +++++++++++++++++++
 1 file changed, 78 insertions(+)

diff --git a/Samples/2.0/Showcase/PbsMaterials/PbsMaterialsGameState.cpp b/Samples/2.0/Showcase/PbsMaterials/PbsMaterialsGameState.cpp
index 10e21d53e..8263aed4c 100644
--- a/Samples/2.0/Showcase/PbsMaterials/PbsMaterialsGameState.cpp
+++ b/Samples/2.0/Showcase/PbsMaterials/PbsMaterialsGameState.cpp
@@ -220,6 +220,84 @@ namespace Demo
 
         mCameraController = new CameraController( mGraphicsSystem, false );
 
+        Ogre::uint32 visibilityMask = mGraphicsSystem->getSceneManager()->getVisibilityMask();
+        visibilityMask &= ~0x00000003;
+        mGraphicsSystem->getSceneManager()->setVisibilityMask( visibilityMask );
+
+        sceneManager->destroyLight(
+            static_cast<Ogre::Light *>( mLightNodes[1]->getAttachedObject( 0 ) ) );
+
+        sceneManager->destroyLight(
+            static_cast<Ogre::Light *>( mLightNodes[2]->getAttachedObject( 0 ) ) );
+
+        // ----------------
+        // Smooth
+        Ogre::v1::MeshPtr smooth_v1 = Ogre::v1::MeshManager::getSingleton().load(
+            "Cube.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        Ogre::MeshPtr smooth_v2 = Ogre::MeshManager::getSingleton().create(
+            "Smooth_v2.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        smooth_v2->importV1( smooth_v1.get(), false, false, false );
+
+        Ogre::Item *smooth = sceneManager->createItem( smooth_v2, Ogre::SCENE_STATIC );
+
+        static_cast<Ogre::HlmsPbsDatablock *>( smooth->getSubItem( 0 )->getDatablock() )
+            ->setTexture( Ogre::PBSM_REFLECTION, "SaintPetersBasilica.dds" );
+        // ----------------
+
+        // ----------------
+        // Smooth_norm
+        Ogre::v1::MeshPtr smooth_norm_v1 = Ogre::v1::MeshManager::getSingleton().load(
+            "Cube.001.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        smooth_norm_v1->buildTangentVectors( Ogre::VES_TANGENT, 0, 0, false, false, true );
+
+        Ogre::MeshPtr smooth_norm_v2 = Ogre::MeshManager::getSingleton().create(
+            "Smooth_norm_v2.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        smooth_norm_v2->importV1( smooth_norm_v1.get(), false, false, false );
+
+        Ogre::Item *smooth_norm = sceneManager->createItem( smooth_norm_v2, Ogre::SCENE_STATIC );
+        smooth_norm->setDatablock( "Smooth_norm_dx" );
+        // ----------------
+
+        // ----------------
+        // Smooth_norm_tan
+        Ogre::v1::MeshPtr smooth_norm_tan_v1 = Ogre::v1::MeshManager::getSingleton().load(
+            "Cube.002.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        Ogre::MeshPtr smooth_norm_tan_v2 = Ogre::MeshManager::getSingleton().create(
+            "Smooth_norm_tan_v2.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        smooth_norm_tan_v2->importV1( smooth_norm_tan_v1.get(), false, false, false );
+
+        Ogre::Item *smooth_norm_tan = sceneManager->createItem( smooth_norm_tan_v2, Ogre::SCENE_STATIC );
+        smooth_norm_tan->setDatablock( "Smooth_norm_dx" );
+        // ----------------
+
+        // ----------------
+        // Flat
+        Ogre::v1::MeshPtr flat_v1 = Ogre::v1::MeshManager::getSingleton().load(
+            "Cube.003.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        Ogre::MeshPtr flat_v2 = Ogre::MeshManager::getSingleton().create(
+            "Flat_v2.mesh", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME );
+
+        flat_v2->importV1( flat_v1.get(), false, false, false );
+
+        Ogre::Item *flat = sceneManager->createItem( flat_v2, Ogre::SCENE_STATIC );
+
+        static_cast<Ogre::HlmsPbsDatablock *>( flat->getSubItem( 0 )->getDatablock() )
+            ->setTexture( Ogre::PBSM_REFLECTION, "SaintPetersBasilica.dds" );
+        // ----------------
+
+        Ogre::SceneNode *node = rootNode->createChildSceneNode( Ogre::SCENE_STATIC );
+        node->attachObject( smooth );
+        node->attachObject( smooth_norm );
+        node->attachObject( smooth_norm_tan );
+        node->attachObject( flat );
+
         TutorialGameState::createScene01();
     }
     //-----------------------------------------------------------------------------------
-- 
2.24.0.windows.2
These are the results:
Image Image
The hard edges are visible, but visual artifacts are stronger than in Blender (plus there's some shadow acne).

But I was expecting this. Indeed, based on what is said here, Blender calculates tangents using MikkTSpace algorithm and thus the pixel shader has to be aware of this. So, I've changed it following MikkTSpace indications (which is the same thing that @xrgo did some time ago and which apparently solved his issues):

Code: Select all

From 54b00f536a2490e11cce84ca73273727b5a0a654 Mon Sep 17 00:00:00 2001
From: TaaTT4 <raffaele.bratta@gmail.com>
Date: Thu, 11 Jun 2020 18:59:00 +0200
Subject: [PATCH] FOE

---
 Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any b/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
index c1d767554..783998c25 100644
--- a/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
+++ b/Samples/Media/Hlms/Pbs/Any/Main/800.PixelShader_piece_ps.any
@@ -632,7 +632,7 @@
 			@insertpiece( forwardPlusApplyDecalsNormal )
 
 			@property( normal_map )
-				pixelData.normal = normalize( mul( TBN, pixelData.normal ) );
+				pixelData.normal = normalize( pixelData.normal.x * vTangent + pixelData.normal.y * vBinormal + pixelData.normal.z * pixelData.geomNormal );
 			@end
 
 			@insertpiece( DoDirectionalShadowMaps )
-- 
2.24.0.windows.2
This lead us to the same result as before. No appreciable differences, while I was expecting to see Smooth_norm_tan cube (the one with the tangents calculated by Blender in MikkTSpace) very similar to how I see it in Blender.
Image

Even if I can submit a draft PR of MikkTSpace tangents calculation, it's pointless at this time since OGRE is unable to handle them properly yet. Could something have been broke moving from OGRE 2.1 to OGRE 2.2 (I'm just thinking loudly since @xrgo said that the pixel shader modifcation solved his issues and, from what I've understood, he let Blender to calculate the tangents). In an article I've already linked before, the author talks about per-vertex bitangent vs per-pixel bitangent: could be this difference that is messing things up?

My skills end here, I'm officially lost. Here you can find the blend project and the assets I used.
Senior game programmer at Vae Victis
Working on Racecraft

Post Reply