Depth buffer precision and reverse-z

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Depth buffer precision and reverse-z

Post by zxz »

Hello!

We have been having great issues with z-fighting for quite a long time. Even when pushing the near-clip distance out as far as possible. A common technique these days for solving this is reversing the depth range as mentioned here:
https://developer.nvidia.com/content/de ... visualized

I decided to try it out in Ogre, with excellent results. With reversed-z we have pretty much no z-fighting issues at all anymore.

This page: https://nlguillemot.wordpress.com/2016/ ... in-opengl/ details the necessary changes.

Since I use the GL3Plus rendering system, I had to bump gl3w to a version containing glClipControl in order to set the clip range to 0-1. Beside that, the following changes were needed in order to test reversed-z in Ogre on the GL3Plus render system. The changes should be similar for D3D, aside from not needing to change the default clip parameters.

Code: Select all

diff --git a/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp b/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
index 85addbd..1a3ed58 100644
--- a/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
+++ b/Components/Hlms/Pbs/src/OgreHlmsPbs.cpp
@@ -297,9 +297,9 @@ namespace Ogre
 
             samplerblock.mMinFilter     = FO_LINEAR;
             samplerblock.mMagFilter     = FO_LINEAR;
             samplerblock.mMipFilter     = FO_NONE;
-            samplerblock.mCompareFunction   = CMPF_LESS_EQUAL;
+            samplerblock.mCompareFunction   = CMPF_GREATER;
 
             if( !mShadowmapCmpSamplerblock )
                 mShadowmapCmpSamplerblock = mHlmsManager->getSamplerblock( samplerblock );
 
diff --git a/OgreMain/src/OgreHlmsDatablock.cpp b/OgreMain/src/OgreHlmsDatablock.cpp
index 42cf77a..337f6a7 100644
--- a/OgreMain/src/OgreHlmsDatablock.cpp
+++ b/OgreMain/src/OgreHlmsDatablock.cpp
@@ -54,9 +54,9 @@ namespace Ogre
         BasicBlock( BLOCK_MACRO ),
         mScissorTestEnabled( false ),
         mDepthCheck( true ),
         mDepthWrite( true ),
-        mDepthFunc( CMPF_LESS_EQUAL ),
+        mDepthFunc( CMPF_GREATER ),
         mDepthBiasConstant( 0 ),
         mDepthBiasSlopeScale( 0 ),
         mCullMode( CULL_CLOCKWISE ),
         mPolygonMode( PM_SOLID )
diff --git a/OgreMain/src/OgreRectangle2D.cpp b/OgreMain/src/OgreRectangle2D.cpp
index 4100292..4abf1c4 100644
--- a/OgreMain/src/OgreRectangle2D.cpp
+++ b/OgreMain/src/OgreRectangle2D.cpp
@@ -124,25 +124,25 @@ namespace v1
         {
             //1st Top-left
             *pVerts++ = -1.0f;
             *pVerts++ =  1.0f;
-            *pVerts++ = -1.0f;
+            *pVerts++ = 0.0f;
 
             *pVerts++ =  0.0f;
             *pVerts++ =  0.0f;
 
             //2nd Bottom-left
             *pVerts++ = -1.0f;
             *pVerts++ = -3.0f; //3 = lerp( -1, 1, 2 );
-            *pVerts++ = -1.0f;
+            *pVerts++ = 0.0f;
 
             *pVerts++ =  0.0f;
             *pVerts++ =  2.0f;
 
             //3rd Top-right
             *pVerts++ =  3.0f;
             *pVerts++ =  1.0f;
-            *pVerts++ = -1.0f;
+            *pVerts++ = 0.0f;
 
             *pVerts++ =  2.0f;
             *pVerts++ =  0.0f;
         }
diff --git a/RenderSystems/GL3Plus/src/OgreGL3PlusDepthBuffer.cpp b/RenderSystems/GL3Plus/src/OgreGL3PlusDepthBuffer.cpp
index 1ad715d..b08d6c6 100644
--- a/RenderSystems/GL3Plus/src/OgreGL3PlusDepthBuffer.cpp
+++ b/RenderSystems/GL3Plus/src/OgreGL3PlusDepthBuffer.cpp
@@ -111,9 +111,9 @@ namespace Ogre
                 OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE) );
                 OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST) );
                 OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST) );
                 OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_MODE, GL_NONE) );
-                OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_LEQUAL) );
+                OCGE( glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_COMPARE_FUNC, GL_GREATER) );
                 
                 if( hasGL42 || support->checkExtension("GL_ARB_texture_storage") )
                 {
                     OCGE( glTexStorage2D( GL_TEXTURE_2D, GLint(1), depthFormat,
diff --git a/RenderSystems/GL3Plus/src/OgreGL3PlusRenderSystem.cpp b/RenderSystems/GL3Plus/src/OgreGL3PlusRenderSystem.cpp
index 9db3234..1c26100 100644
--- a/RenderSystems/GL3Plus/src/OgreGL3PlusRenderSystem.cpp
+++ b/RenderSystems/GL3Plus/src/OgreGL3PlusRenderSystem.cpp
@@ -635,9 +635,9 @@ namespace Ogre {
                                                                                &depthFormat,
                                                                                &stencilFormat );
             DepthBuffer::DefaultDepthBufferFormat = PF_D32_FLOAT_X24_S8_UINT;
         }
-
+        glClipControl(GL_LOWER_LEFT, GL_ZERO_TO_ONE);
         mGLInitialised = true;
     }
 
     void GL3PlusRenderSystem::reinitialise(void)
@@ -2229,10 +2229,14 @@ namespace Ogre {
     void GL3PlusRenderSystem::_convertProjectionMatrix(const Matrix4& matrix,
                                                        Matrix4& dest,
                                                        bool forGpuProgram)
     {
+      Matrix4 reverse(1,0,0,0,
+                      0,1,0,0,
+                      0,0,-1,1,
+                      0,0,0,1);
         // no any conversion request for OpenGL
-        dest = matrix;
+        dest = reverse * matrix;
     }
 
     void GL3PlusRenderSystem::_makeProjectionMatrix(const Radian& fovy, Real aspect,
                                                     Real nearPlane, Real farPlane,
The clear passes also have to be adjusted to clear to 0 instead of 1.

What remains to be done is to fix shadows, since the changes above also affect the shadowing passes, while the shaders are not adjusted for it.

I haven't done that yet. I am not sure how it will play with the pseudo-linear depth stuff Ogre uses, or if reversed-z is good or bad when it comes to shadows. In any case, either the shaders need adjustments, or Ogre has to be set up to use different depth comparisons when drawing the shadows.

I think this would be a very useful feature in mainline Ogre, when all issues have been worked out.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Depth buffer precision and reverse-z

Post by dark_sylinc »

Reverse-Z is something I've been wanting for a while.

The technique itself is easy. Change the projection matrix, flip the culling modes, adjust the clear pass depth, change the vertex to for postprocessing rectangles, and voilá, magic!
For best results it's best if handled inside the Frustum class (to avoid losing precision), rather than via matrix multiplication.

However the details is where it gets dirty, which is why I've been procrastinating this task:
  • Where floating point depth buffers is not supported, Reverse Z hurts more than it helps. Thus supporting older HW means we need to be able to switch at will. There also can be cases where just Reverse Z cannot be used or wouldn't make sense (it's not just old HW support)
  • This means we need to support switching on demand based on RTT that is bound. The problem comes from macroblocks needing to flip their culling mode. PSOs support this though, via Hlms::applyStrongMacroblockRules: if there is a setting that needs overriding the macroblock, a new macroblock is created which is what ends up actually being used, and that PSO will keep a strong reference to this new macroblock, instead of keeping a weak reference to the original one.
  • Ideally user shaders (low level materials) should have an easy way to tell toggle the functionality. For example ReconstructPosFromDepth sample needs different math depending on whether the depth buffer used as a texture was written with reverse Z. Same happens to postprocessing effects' vertex shader setting the Z component correctly
  • Clear passes are also affected
  • Directional shadow mapping is the least affected by Rev. Z. Reverse Z here either does nothing, barely improves, or barely degrades. This is because it uses an orthographic projection, so the distribution of depth is linear (postVertex.w is always 1)
  • Spot lights are the best benefit, because they can use use Rev. Z directly, instead of that pseudolinear Z stuff
  • Point lights is a big question mark, because we need to manually store distance from pixel to light, rather than depth. However this is stored to a colour buffer. Depth is still used normally. Therefore point lights may still benefit, or not.
So basically the problem boils down to ensuring Ogre handles all circumstances & edge cases. I have the modified code changes required for OgreFrustum.cpp if you need them, though that's secondary (until the other issues are solved, the matrix multiplication in _convertProjectionMatrix will do)

There is something odd about your changes though. It seems you're making our GL renderer go from [-1; 1] to [1; -1] and then make GL discard what's in the range [-1; 0] via the glClipControl call. That should be rendering broken tris; unless I missed something.
The projection matrix should be getting transformed so it goes from [-1; 1] to [1; 0] to be in sync with the glClipControl call.
This is very easy to debug via C++ by having a couple of Vector4 point to several extremes of the frustum and multiply it against the final viewProj matrix.
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: Depth buffer precision and reverse-z

Post by zxz »

dark_sylinc wrote: Sun Jun 10, 2018 7:49 pm Reverse-Z is something I've been wanting for a while.
Nice. It seems extremely useful and it would be awesome to have much less z-fighting issues.
  • Where floating point depth buffers is not supported, Reverse Z hurts more than it helps. Thus supporting older HW means we need to be able to switch at will. There also can be cases where just Reverse Z cannot be used or wouldn't make sense (it's not just old HW support)
In the NVIDIA article I linked, they conclude that "Reversed-Z with an integer depth buffer is as good as any of the other integer options.". They have examples with a 24-bit integer buffer.
There is something odd about your changes though. It seems you're making our GL renderer go from [-1; 1] to [1; -1] and then make GL discard what's in the range [-1; 0] via the glClipControl call. That should be rendering broken tris; unless I missed something.
The projection matrix should be getting transformed so it goes from [-1; 1] to [1; 0] to be in sync with the glClipControl call.
This is very easy to debug via C++ by having a couple of Vector4 point to several extremes of the frustum and multiply it against the final viewProj matrix.
I'm not surprised if there are mistakes. I just threw it together quickly as a proof of concept. I suppose the rescaling is missing to compress the projection range. I couldn't see any visual artefacts though, but I might have been just lucky.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Depth buffer precision and reverse-z

Post by al2950 »

+1000000 :D
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: Depth buffer precision and reverse-z

Post by xrgo »

+1000001 :D
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: Depth buffer precision and reverse-z

Post by zxz »

Just make sure the counter doesn't overflow :mrgreen:
User avatar
TaaTT4
OGRE Contributor
OGRE Contributor
Posts: 267
Joined: Wed Apr 23, 2014 3:49 pm
Location: Bologna, Italy
x 75
Contact:

Re: Depth buffer precision and reverse-z

Post by TaaTT4 »

All in: pow(1000001, 2);
I guess it's an unsigned long long, we have sapce ;)

Senior programmer at 505 Games; former senior engine programmer at Sandbox Games
Worked on: Racecraft EsportRacecraft Coin-Op, Victory: The Age of Racing

al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Depth buffer precision and reverse-z

Post by al2950 »

In all seriousness, is this something that might be done? The Ogre 2.1 roadmap seems to have gone a bit cold!
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Depth buffer precision and reverse-z

Post by dark_sylinc »

That's because I'm focusing on something else that has been in demand for a while now.

I didn't want to talk about it because it was very WIP (plus had no documentation, it still hasn't got build instructions), but it's beginning to mature.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: Depth buffer precision and reverse-z

Post by al2950 »

I had noticed that before, looks interesting.

Your comment about supporting old hardware.... surely that is what Ogre 1.x is for. 2.x is looking more to the future!? Also given zxz point below, I was wondering if its something we should look at more seriously.
zxz wrote: Sun Jun 10, 2018 8:51 pm In the NVIDIA article I linked, they conclude that "Reversed-Z with an integer depth buffer is as good as any of the other integer options.". They have examples with a 24-bit integer buffer.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Depth buffer precision and reverse-z

Post by dark_sylinc »

al2950 wrote: Wed Jul 04, 2018 3:11 pm Your comment about supporting old hardware.... surely that is what Ogre 1.x is for. 2.x is looking more to the future!? Also given zxz point below, I was wondering if its something we should look at more seriously.
zxz wrote: Sun Jun 10, 2018 8:51 pm In the NVIDIA article I linked, they conclude that "Reversed-Z with an integer depth buffer is as good as any of the other integer options.". They have examples with a 24-bit integer buffer.
I've been thinking about that.
Back when 2.1 started the goal was to support both D3D10 and D3D11 hardware, which was still relevant at the time. 3 years later, D3D11 hw gained a lot of ground over D3D10.
A Core 2 Quad with a NV GTX 280 or a Radeon HD 4850 is still a "powerful" computer that can have lots of benefits from using 2.1 over 1.x.
Nonetheless I don't want to actively remove D3D10 support. There is no reason to not support them, but I guess in 2018+ they can be considered 2nd tier (i.e. IIRC most or all d3d10 hardware supports FP32 depth buffers, just at reduced performance), so if reverse Z negatively affects them (i.e. either by reducing their perf from using FP32, or reducing quality by using INT24) then so be it.

Reviewing my thoughts about the subject, it seems my concerns about Reverse Z over D3D10 hw were exaggerated: these GPUs will still work with reverse Z (whether slower or lower quality, but still works). And the market share has decreased a lot.

D3D9 HW is different, because (aside from no longer being relevant) supporting it requires an unreasonable effort for us. So if for some reason that's your target, then use Ogre 1.x
rujialiu
Goblin
Posts: 296
Joined: Mon May 09, 2016 8:21 am
x 35

Re: Depth buffer precision and reverse-z

Post by rujialiu »

dark_sylinc wrote: Thu Jul 05, 2018 5:19 pm A Core 2 Quad with a NV GTX 280 or a Radeon HD 4850 is still a "powerful" computer that can have lots of benefits from using 2.1 over 1.x.
Nonetheless I don't want to actively remove D3D10 support. There is no reason to not support them, but I guess in 2018+ they can be considered 2nd tier (i.e. IIRC most or all d3d10 hardware supports FP32 depth buffers, just at reduced performance), so if reverse Z negatively affects them (i.e. either by reducing their perf from using FP32, or reducing quality by using INT24) then so be it.
What about dropping D3D10 hw support in Ogre 2.2? :-D
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: Depth buffer precision and reverse-z

Post by zxz »

I haven't been following the development of Ogre recently, since we have been stuck on 2.1 for a good while. However, I recently started some preliminary porting to 2.2, and was pleasantly surprised to find that the feature of this thread has been available for quite some time now.

Thanks!
Post Reply