rendering to a depth texture in Vulkan Topic is solved

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
Post Reply
jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

rendering to a depth texture in Vulkan

Post by jwwalker »

Is there any trick to rendering to a depth texture (depth only, no color) using Vulkan? I'm using code that works fine with Metal on MacOS and Direct3D on Windows 32, but it gives me nothing but black (zero) in Vulkan on Windows 64. I'm creating the texture like this

Code: Select all

txMgr->createTexture( txName,
    Ogre::GpuPageOutStrategy::SaveToSystemRam,
    Ogre::TextureFlags::RenderToTexture,
    Ogre::TextureTypes::Type2D,
    Ogre::BLANKSTRING,
    0,
    Ogre::DepthBuffer::POOL_NON_SHAREABLE );

and using this compositor node:

Code: Select all

compositor_node JWDepthOnlyRender
{
	in 0 rt0
	target rt0
	{
		pass render_scene
		{
			load { depth clear }
			store { depth store }
			rq_first    0
			rq_last		max
		}
	}
	out 0 rt0
}
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

The Ogre::DepthBuffer::POOL_NON_SHAREABLE parameter will do nothing. I think I explained this in another post; this flag is a remnant from the old 2.1 system and should no longer be used by users directly (in this case it's harmless because it does nothing, the poolId for TextureGpuManager::createTexture only matters for AutomaticBatching textures).

None of what you posted indicates whether it's a depth buffer. What matters is that you call:

Code: Select all

tex = txMgr->createTexture( txName,
    Ogre::GpuPageOutStrategy:: ...,
    Ogre::TextureFlags::RenderToTexture,
    Ogre::TextureTypes::Type2D );
tex->setPixelFormat( PFG_D32_FLOAT_S8X24_UINT );

Where the pixel format is a depth format (like PFG_D16_UNORM, PFG_D32_FLOAT, PFG_D32_FLOAT_S8X24_UINT, etc)

Cheers

Update: Pushed two commits, marking POOL_NON_SHAREABLE as deprecated.

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

Fair enough, more of my setup code:

Code: Select all

_offscreenRenderTarget.reset( txMgr->createTexture( txName,
    Ogre::GpuPageOutStrategy::SaveToSystemRam,
    Ogre::TextureFlags::RenderToTexture,
    Ogre::TextureTypes::Type2D ) );
_offscreenRenderTarget->setPixelFormat( Ogre::PFG_D32_FLOAT );
_offscreenRenderTarget->setResolution( viewWidth, viewHeight );
_offscreenRenderTarget->setNumMipmaps( 1u );
_offscreenRenderTarget->scheduleTransitionTo( Ogre::GpuResidency::Resident );
jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

OK, it turns out that if I use PFG_D32_FLOAT_S8X24_UINT instead of PFG_D32_FLOAT, it works on Vulkan. Is there a way that's better than trial and error to figure out which pixel format to use when?

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

So it works with PFG_D32_FLOAT but not with PFG_D32_FLOAT_S8X24_UINT?

This may be a bug unless there's some Vulkan oddity I'm missing. I don't think I ever tried to read from a depth texture created as PFG_D32_FLOAT_S8X24_UINT; so it's not unreasonable to think that path is completely untested.

Vulkan Validation layers may have more info (enable them with vkconfig on Windows)

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

dark_sylinc wrote: Sun Jul 31, 2022 7:43 pm

So it works with PFG_D32_FLOAT but not with PFG_D32_FLOAT_S8X24_UINT?

No, the other way around: it works with PFG_D32_FLOAT_S8X24_UINT but not with PFG_D32_FLOAT.

dark_sylinc wrote: Sun Jul 31, 2022 7:43 pm

This may be a bug unless there's some Vulkan oddity I'm missing. I don't think I ever tried to read from a depth texture created as PFG_D32_FLOAT_S8X24_UINT; so it's not unreasonable to think that path is completely untested.

Vulkan Validation layers may have more info (enable them with vkconfig on Windows)

OK, I'll look into vkconfig.

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

I turned on the Vulkan validation layer and ran my program. In the bad case, using PFG_D32_FLOAT, I get two error reports that may be relevant. When VulkanQueue::prepareForDownload calls vkCmdPipelineBarrier, I get this:

Code: Select all

VUID-vkCmdPipelineBarrier-srcAccessMask-02815(ERROR / SPEC): msgNum: 618171435 - Validation Error: [ VUID-vkCmdPipelineBarrier-srcAccessMask-02815 ] Object 0: handle = 0x27a0adc0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x24d88c2b | vkCmdPipelineBarrier(): .pImageMemoryBarriers[0].srcAccessMask bit VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT is not supported by stage mask (VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT). The Vulkan spec states: The srcAccessMask member of each element of pMemoryBarriers must only include access flags that are supported by one or more of the pipeline stages in srcStageMask, as specified in the table of supported access types (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdPipelineBarrier-srcAccessMask-02815)
    Objects: 1
       [0]  0x27a0adc0, type: 6, name: NULL

and when VulkanAsyncTextureTicket::downloadFromGpu calls vkCmdCopyImageToBuffer, I get this:

Code: Select all

Ogre: ERROR: [Validation] Code 0 : Validation Error: [ VUID-vkCmdPipelineBarrier-srcAccessMask-02815 ] Object 0: handle = 0x27a0adc0, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x24d88c2b | vkCmdPipelineBarrier(): .pImageMemoryBarriers[0].srcAccessMask bit VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT is not supported by stage mask (VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT). The Vulkan spec states: The srcAccessMask member of each element of pMemoryBarriers must only include access flags that are supported by one or more of the pipeline stages in srcStageMask, as specified in the table of supported access types (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdPipelineBarrier-srcAccessMask-02815)
VUID-vkCmdCopyImageToBuffer-aspectMask-00211(ERROR / SPEC): msgNum: 376861408 - Validation Error: [ VUID-vkCmdCopyImageToBuffer-aspectMask-00211 ] Object 0: handle = 0x4f42790000000ebf, name = OSTX2, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0x167672e0 | vkCmdCopyImageToBuffer: pRegion[0] subresource aspectMask 0x1 specifies aspects that are not present in image format 0x7e. The Vulkan spec states: For each element of pRegions, imageSubresource.aspectMask must specify aspects present in pname:srcImage (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdCopyImageToBuffer-aspectMask-00211)
    Objects: 1
       [0]  0x4f42790000000ebf, type: 10, name: OSTX2

In the better case, using PFG_D32_FLOAT_S8X24_UINT, vkCmdCopyImageToBuffer also produces an error but with different text:

Code: Select all

Ogre: ERROR: [Validation] Code 0 : Validation Error: [ VUID-vkCmdPipelineBarrier-srcAccessMask-02815 ] Object 0: handle = 0x28032070, type = VK_OBJECT_TYPE_COMMAND_BUFFER; | MessageID = 0x24d88c2b | vkCmdPipelineBarrier(): .pImageMemoryBarriers[0].srcAccessMask bit VK_ACCESS_COLOR_ATTACHMENT_WRITE_BIT is not supported by stage mask (VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT|VK_PIPELINE_STAGE_LATE_FRAGMENT_TESTS_BIT). The Vulkan spec states: The srcAccessMask member of each element of pMemoryBarriers must only include access flags that are supported by one or more of the pipeline stages in srcStageMask, as specified in the table of supported access types (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-vkCmdPipelineBarrier-srcAccessMask-02815)
VUID-VkBufferImageCopy-aspectMask-00212(ERROR / SPEC): msgNum: -1123803970 - Validation Error: [ VUID-VkBufferImageCopy-aspectMask-00212 ] Object 0: handle = 0x1fd5740000000e65, name = OSTX2, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0xbd041cbe | vkCmdCopyImageToBuffer: aspectMasks for imageSubresource in pRegion[0] must have only a single bit set. The Vulkan spec states: The aspectMask member of imageSubresource must only have a single bit set (https://vulkan.lunarg.com/doc/view/1.3.216.0/windows/1.3-extensions/vkspec.html#VUID-VkBufferImageCopy-aspectMask-00212)
    Objects: 1
       [0]  0x1fd5740000000e65, type: 10, name: OSTX2
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

OK regarding the validation bugs, it seems they caught an obvious bug.

But I really doubt this was the cause of your problem (although try it. If it's fixed then problem solved :lol: ).

I thought you were reading the depth buffer from a shader and displaying it on screen, coming out as black. I now caught on that you're downloading it to CPU and it comes as a pointer full of 0s.

There's a much greater chance there's a subtle synchronization bug in OgreNext causing the transfer to initiate before rendering finishes (or causing render to start without all the necessary data).

Are you able to make a small repro using Ogre samples? (e.g. modify any of the samples). You may find the Readback test (Samples/2.0/Tests/Readback) useful for this job (enable OGRE_BUILD_TESTS in CMake)

I'll also need info on your environment (OS + GPU) once you are able to make the repro.

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

The fix for the "obvious bug" made the vkCmdPipelineBarrier validation error go away, but the vkCmdCopyImageToBuffer validation error still happens.

I'll see what I can do about a reproducible sample.

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

Further examination of the remaining validation error and the code that produced it gives me some understanding of what is going wrong, though I am not quite sure of the right fix:

TextureGpuManager::createAsyncTextureTicket passes the depth texture pixel format PFG_D32_FLOAT to PixelFormatGpuUtils::getFamily which chooses a pixel format family of PFG_R32_UINT. Later, when VulkanMappings::getImageAspect passes that family to PixelFormatGpuUtils::getFlags, the only flag that is returned is PFF_INTEGER, not PFF_DEPTH. As a result, VulkanMappings::getImageAspect returns VK_IMAGE_ASPECT_COLOR_BIT instead of VK_IMAGE_ASPECT_DEPTH_BIT.

There is no documentation for PixelFormatGpuUtils::getFamily, so I am not sure what that function is supposed to accomplish. However, if I just comment out the case for PFG_D32_FLOAT, so that this pixel format becomes its own "family", then the validation error goes away and my code works as desired.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

Oh interesting. Then this should be a repeatable bug easy to repro.

Thanks for the insight! I just need to remember to check this out!

How to fix this may be easy... or not :P (cries just thinking about it)

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

I may not be able to concoct a reproducible case. Whenever I try to run one of the demos under the Visual Studio debugger, my PC gets into this weird state where it does not respond to any mouse clicks, but does respond to the keyboard, and I must kill Visual Studio to get back into control. It happens as soon as the demo gets past SDL_CreateWindow. Maybe I could build a non-SDL reproducing case, but I'm not sure if it's worth my time when I have ways of working around the problem.

I see from the commit history that it was you who introduced PixelFormatGpuUtils::getFamily back in 2017. What is it supposed to do?

Other than changing PixelFormatGpuUtils::getFamily, another possible fix would be to change VulkanAsyncTextureTicket::downloadFromGpu so that it passes textureSrc->getPixelFormat() instead of mPixelFormatFamily to VulkanMappings::getImageAspect.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

What is it supposed to do?

It's a consequence of a terrible decision by Microsoft to make staging textures a thing; instead of just relying on staging buffers (like every other API: OpenGL, Vulkan, Metal, and... D3D12).

D3D11 will only download into a "staging texture", which needs to be in the same family format.

GL/Vulkan/Metal/D3D12 doesn't care about it. It's just bytes. As long as there is enough bytes, it's ok.

PixelFormatGpuUtils::getFamily also makes sense when reinterpretation is desired (e.g. RGBA8_UNORM vs RGBA8_UNORM_SRGB).

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

jwwalker wrote: Wed Aug 03, 2022 10:24 pm

I may not be able to concoct a reproducible case. Whenever I try to run one of the demos under the Visual Studio debugger, my PC gets into this weird state where it does not respond to any mouse clicks, but does respond to the keyboard, and I must kill Visual Studio to get back into control. It happens as soon as the demo gets past SDL_CreateWindow. Maybe I could build a non-SDL reproducing case, but I'm not sure if it's worth my time when I have ways of working around the problem.

Disable mouse grabbing (SdlInputHandler::setGrabMousePointer or SdlInputHandler::setMouseVisible or both; see MainEntryPoints::mainAppSingleThreaded where it is set for most samples).

I can't believe Windows 10 broke this. They keep messing with Input system that worked fine for decades and keeps breaking.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4971
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1164
Contact:

Re: rendering to a depth texture in Vulkan

Post by dark_sylinc »

jwwalker wrote: Wed Aug 03, 2022 10:24 pm

I may not be able to concoct a reproducible case. Whenever I try to run one of the demos under the Visual Studio debugger, my PC gets into this weird state where it does not respond to any mouse clicks, but does respond to the keyboard, and I must kill Visual Studio to get back into control. It happens as soon as the demo gets past SDL_CreateWindow. Maybe I could build a non-SDL reproducing case, but I'm not sure if it's worth my time when I have ways of working around the problem.

Disable mouse grabbing (SdlInputHandler::setGrabMousePointer or SdlInputHandler::setMouseVisible or both; see MainEntryPoints::mainAppSingleThreaded where it is set for most samples).

I can't believe Windows 10 broke this. They keep messing with Input system that worked fine for decades and keeps breaking.

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

dark_sylinc wrote: Thu Aug 04, 2022 1:03 am

It's a consequence of a terrible decision by Microsoft to make staging textures a thing; instead of just relying on staging buffers (like every other API: OpenGL, Vulkan, Metal, and... D3D12).

D3D11 will only download into a "staging texture", which needs to be in the same family format.

GL/Vulkan/Metal/D3D12 doesn't care about it. It's just bytes. As long as there is enough bytes, it's ok.

PixelFormatGpuUtils::getFamily also makes sense when reinterpretation is desired (e.g. RGBA8_UNORM vs RGBA8_UNORM_SRGB).

So, if pixel format families are only required in the D3D11 case, why is a pixel format family passed to every subclass of AsyncTextureTicket? Why not just let the constructor of D3D11AsyncTextureTicket do the call to PixelFormatGpuUtils::getFamily?

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

dark_sylinc wrote: Thu Aug 04, 2022 1:06 am

Disable mouse grabbing (SdlInputHandler::setGrabMousePointer or SdlInputHandler::setMouseVisible or both; see MainEntryPoints::mainAppSingleThreaded where it is set for most samples).

Thanks! I'm now doing

Code: Select all

inputHandler->setGrabMousePointer( false );
inputHandler->setMouseVisible( true );
inputHandler->setMouseRelative( false );

in my personal fork of OgreNext. That makes it less annoying on macOS too. (If I leave setMouseRelative( true ) in there, then the mouse cursor is still hidden in spite of the setMouseVisible( true ).)

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

jwwalker
Gremlin
Posts: 151
Joined: Thu Aug 12, 2021 10:06 pm
x 12

Re: rendering to a depth texture in Vulkan

Post by jwwalker »

The bug is now fixed and closed, so I'll mark this question as answered.

Post Reply