Page 1 of 1

glTF2/PBR with Displacement map, possible?

Posted: Tue May 06, 2025 9:42 pm
by Arthurfernandes

Ogre Version: 14.3.4
Operating System: Windows 10
Render System: DirectX 11

Hey everyone,

I'm currently exploring Ogre a bit deeper, especially around materials. I found the PBR example in the SampleBrowser and started applying it to all my entities. For testing purposes, I'm using every map available in the material, and so far everything is working great.

However, the last thing I'm trying to implement is a Displacement Map, and that's where things are not going so well. I only understand the basics of shaders, so I've been using ChatGPT to help me through the process.

Here are the changes it suggested:

Changes to .material file:

  • At the beginning:

Code: Select all

set $Displacement white.png
set $DisplacementScale 0.05
  • In the vertex_program_ref glTF2/PBR_vs block:

Code: Select all

param_named      u_DisplacementScale float $DisplacementScale
param_named      u_DisplacementSampler int 8
preprocessor_defines HAS_NORMALS;HAS_UV;HAS_TANGENTS

Changes to pbr-vert.glsl:

  • In OGRE_UNIFORMS:

Code: Select all

uniform sampler2D u_DisplacementSampler;
uniform float u_DisplacementScale;
  • In the vertex shader's main function (MAIN_DECLARATION):

Code: Select all

MAIN_DECLARATION
{
  vec4 pos = mul(u_ModelMatrix, position);
  v_Position = vec3(pos.xyz) / pos.w;

  #ifdef HAS_NORMALS
  #ifdef HAS_TANGENTS
  vec3 normalW = normalize(mul(u_ModelMatrix, vec4(normal.xyz, 0.0)).xyz);
  vec3 tangentW = normalize(mul(u_ModelMatrix, vec4(tangent.xyz, 0.0)).xyz);
  vec3 bitangentW = cross(normalW, tangentW) * tangent.w;
  v_TBN = mtxFromCols(tangentW, bitangentW, normalW);
  #else // HAS_TANGENTS != 1
  v_Normal = normalize(vec3(mul(u_ModelMatrix, vec4(normal.xyz, 0.0))));
  v_Normal = normalW;
  #endif
  #endif

  #ifdef HAS_UV
  v_UV = uv0;
  #else
  v_UV = vec2(0.,0.);
  #endif
  
vec4 worldPos = mul(u_ModelMatrix, position); #ifdef HAS_DISPLACEMENTMAP float displacement = texture(u_DisplacementSampler, v_UV).r; worldPos.xyz += normalW * (displacement * u_DisplacementScale); #endif gl_Position = mul(u_MVPMatrix, worldPos); }

After all these changes, when I try to run the application I get this error:

Code: Select all

Pass 0: vertex program glTF2/PBR_vs cannot be used - compile error

Does anyone know what might be causing this and how to fix it?

Notes:
I verified that HAS_DISPLACEMENTMAP is defined in the .program file.

The texture unit "Displacement" is added to the material.

Shader code seems to compile fine in other contexts.

Thanks in advance for any help


Re: glTF2/PBR with Displacement map, possible?

Posted: Wed May 07, 2025 5:11 pm
by rpgplayerrobin

Where that error appears in the code (or in Ogre.log), there must be more information about why it does not compile.

As I do not use RTSS I am not sure exactly what is wrong here, but the only thing I can think of is that you are using normalW in the shader even though it is not certain that the shader has a normalW variable.
Try this instead:

Code: Select all

  #ifdef HAS_NORMALS
  #ifdef HAS_TANGENTS
  #ifdef HAS_DISPLACEMENTMAP
  float displacement = texture(u_DisplacementSampler, v_UV).r;
  worldPos.xyz += normalW * (displacement * u_DisplacementScale);
  #endif
  #endif
  #endif

Re: glTF2/PBR with Displacement map, possible?

Posted: Sun May 11, 2025 10:59 pm
by paroj

the RTSS supports all variants of that, including POM since 14.0. Take a look:
https://www.ogre3d.org/2023/07/08/ogre- ... on-mapping


Re: glTF2/PBR with Displacement map, possible?

Posted: Mon May 12, 2025 5:43 pm
by Arthurfernandes
paroj wrote: Sun May 11, 2025 10:59 pm

the RTSS supports all variants of that, including POM since 14.0. Take a look:
https://www.ogre3d.org/2023/07/08/ogre- ... on-mapping

Is there any example showing how to use it?


Re: glTF2/PBR with Displacement map, possible?

Posted: Tue May 13, 2025 2:37 pm
by paroj

the screenshots show the bumpmapping sample ;)


Re: glTF2/PBR with Displacement map, possible?

Posted: Tue May 13, 2025 7:42 pm
by rpgplayerrobin

It seems it is integrated into RTSS, here is an example material for it:

Code: Select all

// Steep parallax occlusion mapping
material RTSS/ParallaxOcclusionMapping
{
   technique
   {
       pass
       {
           specular 1 1 1 32

       // Base diffuse texture map
       texture_unit
       {
          texture Bricks076C_diffspec.dds
       }

       texture_unit
       {
           texture Bricks076C_normheight.dds
           rtshader_system
           {
              normal_map parallax_occlusion
           }
       }
   }
   }
}

But your scenario has nothing to do with this of course, since parallax mapping is done in the fragment shader, while yours try to adjust the height of a vertex in the vertex shader (essentially from a heightmap texture).


Re: glTF2/PBR with Displacement map, possible?

Posted: Tue May 13, 2025 11:37 pm
by paroj

POM takes the heightmap into account in the fragment shader. As the geometry is not altered (as this would require a high tesselation) it is technically not the same, but looks very similar.


Re: glTF2/PBR with Displacement map, possible?

Posted: Thu May 15, 2025 4:11 pm
by Arthurfernandes
paroj wrote: Tue May 13, 2025 11:37 pm

POM takes the heightmap into account in the fragment shader. As the geometry is not altered (as this would require a high tesselation) it is technically not the same, but looks very similar.

I saw this, but is there any way to perform real displacement in Ogre — as in actually altering geometry based on a displacement map?


Re: glTF2/PBR with Displacement map, possible?

Posted: Thu May 15, 2025 6:38 pm
by rpgplayerrobin

I would not rely on Ogre being able to do everything, your first post in this thread was logical and I would continue with that if I were you.

You must check why the compile error happens, since that error message that you wrote does not help much.
Every time a compile error happens for me I get the full detail in the ogre log and/or as an error message that pops up when it fails to compile a shader.
If that does not happen for you for some reason, you must go into the source file and put a breakpoint at where the compilation fails (simply search for the error message shown).

Also, make sure to not use a shader cache while you are doing your tests.
I delete my shader cache file each time I change a shader in any way, otherwise it can sometimes create bugs.

Did you try changing the shader to what I wrote in my first reply and see if the compile error still happens?


Re: glTF2/PBR with Displacement map, possible?

Posted: Mon May 19, 2025 4:28 pm
by Arthurfernandes

I did, but the error still occurs. Maybe this is happening because I'm using Ogre with C# bindings, but I can't find any details about the error anywhere — all I got was the error I sent earlier.


Re: glTF2/PBR with Displacement map, possible?

Posted: Mon May 19, 2025 8:14 pm
by rpgplayerrobin

Then you must go into the Ogre source that you have compiled and put a breakpoint at where the compilation fails (simply search for the error message you get).
It will show exactly why the compilation fails in the code.
But if that does not work since you use C# bindings, I would really consider not using C#, since being able to debug the source code is extremely important.

By the way, I have never understood why Mogre and/or C# is used for Ogre.
To me it seems like it is like building a bridge over straight land, essentially just slowing down traffic without any upside.
I understand that Unity uses C#, but that converts it to C++ with Mono, which is in my opinion also just an extra step an may create unoptimized code since you don't really have full control of the stack compared to the heap.
Also, if I am not mistaken, you then also have to install .NET on the target machine and not just the C++ redist.

I just see it as an extra step and there seems to be more bugs related to using the C# version of Ogre than just using pure Ogre with C++.
It may as you say be the culprit to your issue.


Re: glTF2/PBR with Displacement map, possible?

Posted: Tue May 20, 2025 1:36 pm
by paroj

I guess the reason is the same why one would integrate Lua or python into your game.
Its much faster to write application logic in a high level language with proper reflections, where you easily do metaprogramming.
The performance hit is negligible here as that code is not on a hot path.

C# is a nice middleground between being structured and convenient here.

Anyway.. With Python you can attach the C++ debugger to your Python process and step through the C++ code. There are some GDB extensions that extend the stack trace to python code so you get the full picture. Might be worth investigating this for C#.

Also wherever you get "vertex program glTF2/PBR_vs cannot be used" should be also the real compile error if you scroll up.