[2.1+] RenderDoc HLSL debugging with "Fast Shader Build Hack"

[rujialiu]

Dear all,

Today I found that RenderDoc 1.1 added HLSL level debugging: https://github.com/baldurk/renderdoc/releases/v1.1

So I gave it a try. However, when debugging the fragment shader, I noticed that the debugger gave very different result with or without "Fast Shader Build Hack". Actually with the hack, most lighting codes are skipped due to lightMask is mostly zero.

With the hack, worldMaterialIdx is defined as:

Code: Select all

uint4 worldMaterialIdx[2];

and the light mask is retried this way:

Code: Select all

@piece( DeclareObjLightMask )uint objLightMask = worldMaterialIdx[inPs.drawId].z;@end

However, all objLightMask starting from drawId=2, are all zeros. It looks like RenderDoc's debugging is based on its own bytecode intepreter, which detects out-of-bound errros and gives 0 when isPs.drawId >= 2?

Anyway, this is one more reason why Fast Shader Build Hack may be disabled

[dark_sylinc]

Yeah, the main reason "Fast Shader Build Hack" is there is for better shader debugging. The second reason is "just in case" for some reason in the future a vendor decides to break the hack (btw it wouldn't break just us: it would also break lots of major titles).

I've opened a ticket at RenderDoc to request the debugger matches what the GPU is doing.

[rujialiu]

Yeah, the main reason "Fast Shader Build Hack" is there is for better shader debugging. The second reason is "just in case" for some reason in the future a vendor decides to break the hack (btw it wouldn't break just us: it would also break lots of major titles).

I've opened a ticket at RenderDoc to request the debugger matches what the GPU is doing.

baldurk already closed this issue 4 minutes ago because he thought it's a "undefined-behaviour-for-undefined-behaviour compatibility"... anyway, we can add this info in Ogre 2.1 FAQ and source code.