render depth buffer using compositor

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
andreahmed
Kobold
Posts: 38
Joined: Wed Dec 31, 2014 2:26 am
x 2

render depth buffer using compositor

Post by andreahmed »

Hi All,
I'm trying to render the depth buffer using a RTT then pass it to compositor then render a full screen quad.

but I don't get any depth texture at all

here is my code

Code: Select all

 // Create the depth render texture
        Ogre::TexturePtr depthTexture = Ogre::TextureManager::getSingleton().createManual(
                                                                              "DepthMap", Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME,
                                                                              Ogre::TEX_TYPE_2D, mOgreCamera->getViewport()->getActualWidth(), mOgreCamera->getViewport()->getActualHeight(),
                                                                              0, Ogre::PF_FLOAT16_R, Ogre::TU_RENDERTARGET);
        Ogre::MaterialPtr material = Ogre::MaterialManager::getSingleton().getByName("Ogre/Compositor/SSAO");
        material->getTechnique(0)->getPass(0)->getTextureUnitState(0)->setTextureName("DepthMap");
    //    material->getTechnique(0)->getPass(0)->setVertexProgram( "SSAO_vs" );
      //  material->getTechnique(0)->getPass(0)->setFragmentProgram( "SSAO_fs" );
        //material->load();
        Ogre::RenderTexture* renderTexture = depthTexture->getBuffer()->getRenderTarget();

        renderTexture->addViewport(mOgreCamera);
        renderTexture->getViewport(0)->setClearEveryFrame(true);
        renderTexture->getViewport(0)->setBackgroundColour(Ogre::ColourValue( 0.8f, 0.8f, 0.8f ));
        renderTexture->getViewport(0)->setOverlaysEnabled(false);



and that's the compositor

Code: Select all

compositor DepthMap
{
    technique
    {
        // Temporary textures
        texture rt0 target_width target_height PF_A8R8G8B8

        target rt0
        {
            // Render output from previous compositor (or original scene)
            input previous
        }

        target_output
        {
            // Start with clear output
            input none

            // Draw a fullscreen quad with the black and white image
            pass render_quad
            {
                // Renders a fullscreen quad with a material
                material Ogre/Compositor/SSAO
                input 0 rt0
            }
        }
    }
}
andreahmed
Kobold
Posts: 38
Joined: Wed Dec 31, 2014 2:26 am
x 2

Re: render depth buffer using compositor

Post by andreahmed »

please help me :(
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: render depth buffer using compositor

Post by dark_sylinc »

Ogre 1.x doesn't support reading from the depth buffer.

However you can use PF_FLOAT16_R / PF_FLOAT32_R to render depth to the colour buffer.
If you do that, you need to write your own pixel shader that outputs depth to the colour buffer. Maybe there's a sample already doing it you can look at, but I don't know.
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: render depth buffer using compositor

Post by paroj »

for reference: starting with 1.11, you can use PF_DEPTH (aka. PF_DEPTH16) just as any other PixelFormat to read from the depth buffer.
loath
Platinum Sponsor
Platinum Sponsor
Posts: 290
Joined: Tue Jan 17, 2012 5:18 am
x 67

Re: render depth buffer using compositor

Post by loath »

paroj wrote: Thu Aug 09, 2018 1:21 pm for reference: starting with 1.11, you can use PF_DEPTH (aka. PF_DEPTH16) just as any other PixelFormat to read from the depth buffer.
whoah - just found this post. does this mean with directx 11 we can read from the depth buffer in a fragment shader?

thanks!
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: render depth buffer using compositor

Post by paroj »

loath wrote: Thu Oct 31, 2019 2:01 am
paroj wrote: Thu Aug 09, 2018 1:21 pm for reference: starting with 1.11, you can use PF_DEPTH (aka. PF_DEPTH16) just as any other PixelFormat to read from the depth buffer.
whoah - just found this post. does this mean with directx 11 we can read from the depth buffer in a fragment shader?

thanks!
yes, see: https://www.ogre3d.org/2018/10/24/ogre- ... erm-report
McGroto
Gnoblar
Posts: 2
Joined: Fri Oct 30, 2020 7:19 am

Re: render depth buffer using compositor

Post by McGroto »

I have tried using Ogre's example from your link, but I end up just getting a black image where my scene usually is. For clarity, I am trying to use exactly this:

Code: Select all

material ShowDepth {
    technique {
        pass {
            lighting off
            texture_unit {
                filtering none
            }
        }
    }
}

compositor Texcoord
{
    technique
    {
        texture depthTex target_width target_height PF_DEPTH16
        target depthTex
        {
            pass clear {}
            pass render_scene {}
        }
        target_output
        {
            pass render_quad
            {
                material ShowDepth
                input 0 depth
            }
        }
    }
}
I have also tried changing "input 0 depth" to "input 0 depthTex" with no luck. Any ideas?
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: render depth buffer using compositor

Post by paroj »

RenderSystem/ Ogre version? Also try disabling FSAA
McGroto
Gnoblar
Posts: 2
Joined: Fri Oct 30, 2020 7:19 am

Re: render depth buffer using compositor

Post by McGroto »

According to my x86 folder it looks like OGRE 1.9.0 and RenderSystem GLES2. I'm not sure how to disable FSAA. I am using this in Gazebo, for reference. I am pretty new to this so I may be missing something else somewhere, but I have gotten some other compositors (e.g. lens flare and increased exposure) to work using the following inside of my C++ scripts:

Code: Select all

void MyCompositor::setupCompositor(sdf::ElementPtr sdf, CameraPtr camera)
{
  // Create the listener
  if (!m_compositor_listener)
  {
    m_compositor_listener.reset(new MyCompositorListener(sdf));
  }

  CompositorPtr compositor = Ogre::CompositorManager::getSingleton().getByName("DoF");
  if (compositor.get() != nullptr)
  {
    Ogre::CompositorManager::getSingleton().removeCompositor(
      camera->OgreViewport(), "DoF");
  }

  // Create the compositor and attach the listener to it
  CompositorInstance* ci = Ogre::CompositorManager::getSingleton().addCompositor(
    camera->OgreViewport(), "DoF");
  if (ci)
  {
    ci->setEnabled(true);
    ci->addListener(m_compositor_listener.get());
  }
  ROS_INFO("Setting up DoF comp 1");
}
Where I changed the compositor name from Texcoord to DoF. And within the listener file:

Code: Select all

void MyCompositorListener::notifyMaterialRender(unsigned int pass_id,
                                                    Ogre::MaterialPtr& mat)
{
  GZ_ASSERT(!mat.isNull(), "Null Ogre3D material");
  Ogre::Technique* technique = mat->getTechnique(0);
  GZ_ASSERT(technique != nullptr, "Null Ogre3D technique");
  Ogre::Pass* pass = technique->getPass(pass_id);
  GZ_ASSERT(pass != nullptr, "Null Ogre3D pass");
  Ogre::GpuProgramParametersSharedPtr params = pass->getFragmentProgramParameters();
  GZ_ASSERT(!params.isNull(), "Null Ogre3D material GPU parameters");
  }
That is about all that I am doing in terms of compositor setup. Is there something major I am missing here? My end goal is to create a depth of field compositor but I thought that being able to display depth would be a good first step.
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: render depth buffer using compositor

Post by paroj »

McGroto wrote: Fri Oct 30, 2020 7:58 pm According to my x86 folder it looks like OGRE 1.9.0 and RenderSystem GLES2.
as stated in the link, you need at least Ogre 1.11.3 for this. Also you need GLES3 for depth texture support. (the GLES2 rendersystem will use that if available)
swebb2066
Gnoblar
Posts: 1
Joined: Tue Feb 16, 2021 12:32 am

Re: render depth buffer using compositor

Post by swebb2066 »

Hi, I am receiving the following exception when attempting to render using the above ShowDepth material to a TEX_TYPE_2D texture using RTShaderSystem.

Code: Select all

class Ogre::RenderingAPIException: Ogre::RenderingAPIException::RenderingAPIException: No VertexElement for semantic TEXCOORD0 in shader 3e09fc7ab6c75ef5e45fd0a016abe6d4_VS found in Ogre::D3D11VertexDeclaration::getD3DVertexDeclaration at OgreD3D11VertexDeclaration.cpp (line 101)
The generated shader (3e09fc7ab6c75ef5e45fd0a016abe6d4_VS.hlsl) is

Code: Select all

//-----------------------------------------------------------------------------
// Program Type: Vertex shader
// Language: hlsl
// Created by Ogre RT Shader Generator. All rights reserved.
//-----------------------------------------------------------------------------

//-----------------------------------------------------------------------------
//                         PROGRAM DEPENDENCIES
//-----------------------------------------------------------------------------
#include <OgreUnifiedShader.h>
#include "FFPLib_Transform.cg"
#include "FFPLib_Common.glsl"
#include "FFPLib_Texturing.cg"

//-----------------------------------------------------------------------------
//                         GLOBAL PARAMETERS
//-----------------------------------------------------------------------------

float4x4	worldviewproj_matrix;

//-----------------------------------------------------------------------------
// Function Name: main
// Function Desc: 
//-----------------------------------------------------------------------------
void main
	(
	 in float4	iPos_0 : POSITION, 
	 in float2	iTexcoord_0 : TEXCOORD0, 
	 out float4	oPos_0 : POSITION, 
	 out float2	oTexcoord_0 : TEXCOORD0
	)
{
	float4	lColor_0;
	float4	lColor_1;

	FFP_Transform(worldviewproj_matrix, iPos_0, oPos_0);

	lColor_0	=	vec4(1.00000,1.00000,1.00000,1.00000);

	lColor_1	=	vec4(0.000000,0.000000,0.000000,0.000000);

	oTexcoord_0	=	iTexcoord_0;
}


I am using Ogre 1.12.11.

The code to create the texture is

Code: Select all

    Ogre::MaterialPtr mat = Ogre::MaterialManager::getSingleton().getByName("ShowDepth");
    if (!mat)
    {
        LOG4CXX_WARN(m_log, "Material 'ShowDepth' not found");
        return false;
    }

    m_texPtr = Ogre::TextureManager::getSingleton().createManual
        ( "DepthMap"
        , Ogre::ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME
        , Ogre::TEX_TYPE_2D
        , m_camera_cols
        , m_camera_rows
        , 1 // depth
        , 0 // num_mips
        , Ogre::PixelFormat::PF_DEPTH16
        , Ogre::TU_RENDERTARGET
        );
    mat->getTechnique(0)->getPass(0)->getTextureUnitState(0)->setTexture(m_texPtr);
    auto buffer = m_texPtr->getBuffer();
    LOG4CXX_DEBUG(m_log, "setupDepthTexture:"
        << " hasShadowBuffer? " << buffer->hasShadowBuffer()
        << " bufferIsSystemMemory? " << buffer->isSystemMemory()
        );
    m_depth_map = buffer->getRenderTarget();
    Ogre::Viewport* viewPort = m_depth_map->addViewport(m_camera);
    viewPort->setClearEveryFrame(true);
    viewPort->setBackgroundColour(Ogre::ColourValue::White);
    viewPort->setOverlaysEnabled(false);
    Ogre::CompositorInstance* ci = Ogre::CompositorManager::getSingleton().addCompositor(viewPort, "RenderDepth");
    ci->addListener(m_listener.get());
    Ogre::CompositorManager::getSingleton().setCompositorEnabled(viewPort, "RenderDepth", true);
When using the GL3Plus render system there is no exception but I only get zeros when I fetch the buffer using

Code: Select all

    std::vector<RobotVision::DepthImage::PixelType> tempData(m_camera_rows * m_camera_cols, 0);
    Ogre::PixelBox resultMem(m_camera_cols, m_camera_rows, 1, Ogre::PixelFormat::PF_DEPTH16, &tempData[0]);
    auto buffer = m_texPtr->getBuffer();
    buffer->blitToMemory(resultMem);
I would appreciate any suggetion on how I can help diagnose this issue.

Thanks
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: render depth buffer using compositor

Post by paroj »

swebb2066 wrote: Tue Feb 16, 2021 12:51 am Hi, I am receiving the following exception when attempting to render using the above ShowDepth material to a TEX_TYPE_2D texture using RTShaderSystem.

Code: Select all

class Ogre::RenderingAPIException: Ogre::RenderingAPIException::RenderingAPIException: No VertexElement for semantic TEXCOORD0 in shader 3e09fc7ab6c75ef5e45fd0a016abe6d4_VS found in Ogre::D3D11VertexDeclaration::getD3DVertexDeclaration at OgreD3D11VertexDeclaration.cpp (line 101)
The exception says that the object you are rendering has no texture coordinates. You obviously need those to display the depth*texture*.

Also the snippet above will not work with FSAA - rather use the snipped in the blog post which was updated for that:
https://www.ogre3d.org/2018/10/24/ogre- ... erm-report

Finally, you could draw some inspiration from the OVIS OpenCV implementation:
https://github.com/opencv/opencv_contri ... s.cpp#L840
Post Reply