RootLayout with UAV and descriptor sets

[bishopnator]

In this link https://ogrecave.github.io/ogre-next/ap ... youts.html there is following example:

But in Ogre::RootLayout::validate there is a check that non-baked descriptor sets cannot contain UAVs and the baked sets cannot contain DescBindingTypes::ParamBuffer and DescBindingTypes::ConstBuffer so when I try to setup the RootLayout as described in the help, I am getting the exceptions.

Are the examples there outdated? If I split the UAVs description to e.g. mDescBindingRanges[1] and leave all other types in mDescBindingRanges[0] and also additionally set mBaked[1] to true, does it have any impact on HLSL (D3D11 RS) and GLSL (GL3Plus, but not Vulkan) shaders? I think I don't have to change anything in my shaders, right? I didn't study Vulkan nor D3D12 yet, so I am not very familiar with the reasons why the sets were introduced in the first place. I am just trying to avoid exceptions in my code due to "invalid" initialization (enclosed in quotes as I don't think the initialization is invalid - only Ogre-next is not very happy with with )

[dark_sylinc]

Are the examples there outdated?

No, the example is just invalid. I'll rewrite it a little bit.

I was so preocuppied with making sure the example explains how RootLayouts work compared to D3D11 that I didn't realize the example was invalid.

The reason UAVs are in only in baked descriptors is because we need to do a lot of analysis that is better done baked.
And constant params cannot be in baked descriptors due to a couple mistakes early on (originally OgreNext 2.2 started being worked on while I got a few things wrong about Vulkan). Ideally, it should be possible.

If I split the UAVs description to e.g. mDescBindingRanges[1] and leave all other types in mDescBindingRanges[0] and also additionally set mBaked[1] to true, does it have any impact on HLSL (D3D11 RS) and GLSL (GL3Plus, but not Vulkan) shaders?

It should not have any impact.

I think I don't have to change anything in my shaders, right?

You don't, for D3D11 and OpenGL.

But for Vulkan we are using GLSL with a few macros defined by the RootLayout, and that would basically mean that either you write it by hand, or patch up the output from SPIRV-Cross to get what OgreNext wants.

so I am not very familiar with the reasons why the sets were introduced in the first place

The manual section you linked to explains the reasons. Basically Vulkan/D3D12 went away with the table model; and went for baking everything into a baked descriptor. i.e. from a low level perspective instead of D3D11 binding all 128 texture slots(\*):

(\*) This code is just to make a point. In reality drivers use heavy optimizations to send as little data as possible (i.e. they don't loop over all 128 slots every time a texture changes).

Code: Select all

for( int i = 0; i < D3D11_COMMONSHADER_CONSTANT_BUFFER_HW_SLOT_COUNT; ++i );
     set_bind_to_gpu( i, const_buffers[i] );
for( int i = 0; i < D3D11_COMMONSHADER_INPUT_RESOURCE_REGISTER_COUNT; ++i )
     set_bind_to_gpu( i, texture[i] );

Vulkan instead just sets a single pointer:

Code: Select all

set_bind_to_gpu( set_idx, baked_descriptor_set_ptr );

Our abstraction got this wrong, because our baked Descriptors aren't fully baked (we do as much early work as possible; but ultimately the Vulkan descriptor is uploaded every time it's bound; which is not the ideal practice).

Vulkan's idea is that if you're likely going to have a lot of resources that can be bound once, resources that rarely change (e.g. materials), and resources that need to change often (e.g. per-instance world matrices); so we are guaranteed at least 4 sets:

Code: Select all

set_bind_to_gpu( 0, descriptor_of_stuff_that_never_changes );

for each instance
    if( instance->material_pool != last_pool )
          set_bind_to_gpu( 1, instance->descriptors_with_materials );
    set_bind_to_gpu( 2, instance->descriptors_with_instance_data );

If you still have questions, this article describes Vulkan's descriptor sets.

TL;DR: GPUs internally have up to 4 different ways in which their resources are bound; which means "which path is the optimal" is different depending on the HW model.

Root Layouts is not the only solution that can solve this; it is one of possibly many. However I'm quite happy with how Root Layouts ended up because it let us emulate the D3D11/GL model, the C++ code is in control (but optionally the shader can take control), it gives us a lot of predictability on CPU and memory consumption, and it allows many shaders to share the same descriptor signature.

[dark_sylinc]

BTW regarding HLSL, GLSL and Spirv-Cross:

OgreNext currently does not support it, but the path of least resistance for you should be to add HLSL/DXC support to our Vulkan RenderSystem.

DXC can compile HLSL into SPIRV. You'd need a few adjustments:

• Clone OgreVulkanProgram.cpp and modify it so that it supports DXC/HLSL instead of glslang/GLSL.

• HLSL uses [[vk::binding(X[, Y])]] to indicate the set and binding slot. You'd have to write [[vk::binding( ogre_t1 )]] and have the RootLayouts generate the macro.

• Vulkan needs to be initialized with a few specific extensions required to support the SPIRV generated by DXC. You'd have to update the code so that OgreNext asks for those extensions.

It should actually be pretty straightforward (knocks on wood) to add DXC/HLSL support.

[bishopnator]

Thanks for explanation. I am still confused about the term 'baked' and its meaning regarding the RootLayout. If the flag is set to false, there is pretty a lot of flexibility about the slots (only uav buffers and uav textures cannot be set). If it is set to true, there are more checks which are not hard to understand from the implementation of validate() member method, but they are harder to understand from the user perspective. It is probably due to some additional checks done by ogre later in different render systems.

For now I am not too limited as I set all bindings to RootLayout's descriptor set 0 (non-baked) and all uavs to set 1 (baked) and it seems that I am not getting any exceptions.

[dark_sylinc]

Oh! What you said refreshed my memory!

OK so here's the thing:

• In retrospective, it probably would've been better to force all RenderSystems to use RootLayouts (that would've made users porting forwards hearder though). RootLayouts could've easily support tex_buffers : [0; 7] and textures : [0; 7] but because in OpenGL and D3D11 those slots are shared, that was forbidden (in an attempt to keep user code working on all APIs). You must set them as tex_buffers : [0; 7] and textures : [7; 14] because they can't overlap. validate() checks that. In Vulkan we could allow them to overlap, but such code would not run on OpenGL or D3D11!

  • If D3D11 and GL had been forced to use RootLayouts too, that would've allowed us overlapping because we can remap the slots in D3D11/GL so they don't overlap. RootLayouts' slots are "fake" or "virtual".

• Long term plan is to move towards DescriptorSets for most stuff. Since our D3D11 & OpenGL backends were barely using UAVs, I took the chance to only allow UAVs via baked descriptor sets. This was already happening on D3D11 and OpenGL: you may notice there is no _setUavCS( slot, TextureGpu* ) function. There's only _setUavCS( uint32 slotStart, const DescriptorSetUav *set ). Only baked sets are accepted. As such, RootLayouts with UAVs on non-baked sets do not make sense. Even if RootLayout's validation would let you, there's no way from C++ to bind such UAVs.

For now I am not too limited as I set all bindings to RootLayout's descriptor set 0 (non-baked) and all uavs to set 1 (baked) and it seems that I am not getting any exceptions.

Yes. That is the correct way.

But please that for regular textures set via RenderSystem::_setTextures (i.e. via DescriptorSetTexture or DescriptorSetTexture2), you would have to put them in set 1 (baked) or another set 2 (also baked).

Textures slots in the non-baked set must be set via RenderSystem::_setTexture (i.e. individually).

UAVs and textures can live in the same baked set; but you must use DescriptorSetTexture2 for binding, since DescriptorSetTexture will unbind the UAVs (I can't remember why it does that. I just noticed from VulkanRenderSystem::_setTextures implementation).

Edit: TL;DR to answer your question: "baked" means it's bound via DescriptorSet* family of functions.
Edit 2: Vulkan rendersystem wiping out UAV bindings when setting a new DescriptorSetTexture is starting to look like a bug.

[bishopnator]

How the RootLayout is influenced if there are multiple CompositorPassUav preceding CompositorPassScene? It seems like following:

• each CompositorPassUav completely overwrite the DescriptorSetUav in RenderSystem, which means that using multiple CompositorPassUav doesn't make sense - there should be only a single CompositorPassUav which sets all required UAVs, right?
• the HLMS's RootLayout is not aware of any active CompositorPassUav - it just tells Ogre, what the shader expects / uses. If HLMS binds some UAVs by its own and want to keep those bound by CompositorPassUav, it must somehow update current DescriptorSetUav queued in RenderSystem (just trying to figure out some flexible approach about allowing usage of CompositorPassUav with some direct possibility of setting UAVs in HLMS).

It seems that setupRootLayout is called for every compiled shader, but the implementation actually returns always the same layout as the maximum bound slots used by any shader stage in any combination of active properties. Is it correct? Isn't it then better to call it just once for whole HLMS?

As the render targets and UAV slots in d3d11 are shared, I have to write something like this in my pixel shader:

Code: Select all

struct ps_out
{
	float4 color0 : SV_Target0;
};
RWStructuredBuffer<float> my_uav_buffer : register(u1);

Does the RootLayout contain the UAV slots binding [0, 1] or [0, 2] or [1, 2]? My question is more about the bound color buffers - are those ignored by RootLayout? You wrote that the slots in RootLayout are virtual and later mapped properly so I suppose I should specify just [0, 1].

[bishopnator]

I am still struggling with RootLayout(s) between the d3d11 and gl3plus RenderSystems. Can you point me out where the macros ogre_t0, ogre_t1, etc. (and similar macros for other types) are created and injected? I like the idea from the reading about them from comments in the code, but I didn't find the cpp file where they are generated. I have some difficulties with declaring RootLayouts for HLSL and GLSL when I use read-only buffers and uav buffers. In HLSL the uav buffers goes after the color attachments (so when I output 2 colors, my first uav can be bound to u2 (as u0 and u1 are taken by color outputs) and read-only buffers are bound to t registers (t0, t1, etc.). However in GLSL both are SSBOs and hence I need to offset bindings of uavs differently - some discussion I tried to start in viewtopic.php?t=97499

In the ideal case, the RootLayout could be 100% independent of RenderSystem - all buffers could be treated separately which means that I could declare e.g. that I use uav textures at slots 0, 1, 2 and uav buffers also at slots 0, 1, 2 - ogre will treat those slots as different types. The translation layer in all RenderSystems would remap those virtual slots to actual slots supported by the given RenderSystem and generates macros for the shader compilation - here I would even expect more different macros and also closer matches to the ogre's types - instead of ogre_t0 rather ogre_read_only_buffer_0. So in the above example with "shared" slots between uav textures and uav buffers, ogre could provide to the shader compilation macros like ogre_uav_texture_0, ogre_uav_buffer_0 --> according to how RenderSystem handles bindings of the uav texture and buffers, it will generate correct mapping (like #define ogre_uav_texture_0 u1 and #define ogre_uav_buffer_0 u2 in the case of HLSL).

In the case of CbShaderBuffer command, the binding slot should be also virtual or there should be a helper function which translates virtual slot to actual RenderSystem's slot based on the RootLayout.

I am really confused about those rules in the RootLayout as some buffer types share the slots and cannot overlap, but in gl3plus RS there are some internal implementation details e.g. how the read-only buffers are actually created which influences how they are bound in the shaders and hence which slots are (or can be) used.

If ogre shouldn't (or cannot) be modified, I am thinking about implementing some kind of VirtualRootLayout which will work as described above and from which it will be possible to create RootLayout as Ogre expects now. But before doing anything, I would like to collect some more information because I already misunderstood some parts of ogre before and also now I have still feeling, that I am missing something important.

note: I already tried dxc for converting my HLSL shaders to spirv and then using spirv-cross to convert it to glsl, but it doesn't work as smooth as shaderc library. Reagrdless of what library is used to generate spirv from HLSL, there is a problem that I need to remap uav bindings exactly due to above described problems because spirv extracts the bindings from the used registers (e.g. u1 means the SSBO in GLSL will be bound to slot 1, which is incorrect, if it is already used by read-only buffer). Using such "virtual" RootLayout can solve it.

[dark_sylinc]

Can you point me out where the macros ogre_t0, ogre_t1, etc. (and similar macros for other types) are created and injected?

The macros are created in OgreVulkanRootLayout.cpp VulkanRootLayout::generateRootLayoutMacros and they get injected in OgreVulkanProgram.cpp VulkanProgram::getPreamble.

In the ideal case, the RootLayout could be 100% independent of RenderSystem - all buffers could be treated separately which means that I could declare e.g. that I use uav textures at slots 0, 1, 2 and uav buffers also at slots 0, 1, 2 - ogre will treat those slots as different types.

Yes. RootLayout being implemented for all RenderSystems would make things easier. Though we'd have to think on how to keep backwards compatibility with existing D3D11 & GL shaders.

And also I'm not sure if OpenGL will be worth maintaining in the future.

Metal would definitely be benefit, as we've hit some limitations with the current version, that would be gone with RootLayouts.