[2.3.0] Issues with Shadow Nodes Topic is solved

[haloman30]

Ogre Version: 2.3.0
Operating System: Windows 7
Render System: Direct3D 11

Hello!

I'm having some trouble figuring out how to get shadow compositor nodes to work properly. With some minor tweaks, I got the basic "focused" shadow node example working from the 2.3.0 docs - had some issues with it not recognizing the pixel format.

It seems to pretty much work as expected - except I end up with a crash anytime any object in the scene uses an unlit material instead of PBS, with the D3D11 vertex/pixel shaders just failing to compile according to the logs.

I'm not entirely sure where to go from here - so I'm reaching out to y'all again in the hopes of another clue of where to look, I'm sure I've just overlooked something somewhere.

For reference, I've included the shadow node in the compositor, the error messages mentioned, and the vertex/pixel shaders that were created (since some other somewhat related topics had people asking for the resulting shaders).

Any help or ideas on things to try are greatly appreciated!

Shadow Node in Compositor:

Code: Select all

compositor_node_shadow myShadowNode
{
    technique focused
    texture focusedTex 2048 2048 PFG_D32_FLOAT keep_content
    shadow_map 0 focusedTex light 0

shadow_map_target_type directional spot
{
    shadow_map 0
    {
        pass clear { colour_value 1 1 1 1 }
        pass render_scene
        {
            rq_first 0
            rq_last 252
        }
    }
}
}

Error Messages:

Code: Select all

06.16.24 19:11:10 ### ERROR OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000009VertexShader_vs Errors: 300000009VertexShader_vs.hlsl(220,5-13): error X3018: invalid subscript 'uv0'  in D3D11HLSLProgram::compileMicrocode at C:\Projects\Ogre\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 539)
06.16.24 19:11:10 High-level program 300000009VertexShader_vs encountered an error during loading and is thus not supported. OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000009VertexShader_vs Errors: 300000009VertexShader_vs.hlsl(220,5-13): error X3018: invalid subscript 'uv0'  in D3D11HLSLProgram::compileMicrocode at C:\Projects\Ogre\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 539)
06.16.24 19:11:10 ### ERROR OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000009PixelShader_ps Errors: 300000009PixelShader_ps.hlsl(221,41-51): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at C:\Projects\Ogre\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 539)
06.16.24 19:11:10 High-level program 300000009PixelShader_ps encountered an error during loading and is thus not supported. OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000009PixelShader_ps Errors: 300000009PixelShader_ps.hlsl(221,41-51): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at C:\Projects\Ogre\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 539)

Pixel Shader:

Code: Select all

#if 0
	***	hlms_uv_count0	2
	***	hlms_pose	0
	***	hlms_uv_count	1
	***	diffuse_map0_idx	0
	***	hlms_disable_stage	0
	***	texcoord	0
	***	fast_shader_build_hack	1
	***	glsl	635204550
	***	uv_diffuse0	0
	***	out_uv_count	1
	***	hlms_high_quality	0
	***	hlms_shadowcaster	1
	***	samplerStateStart	2
	***	alpha_test	1
	***	glsles	1070293233
	***	out_uv0_out_uv	0
	***	diffuse_map0	1
	***	out_uv0_texture_matrix	0
	***	hlms_alpha_to_coverage	0
	***	hlms_shadowcaster_directional	1
	***	hlms_pose_normals	0
	***	hlms_pose_half	0
	***	hlms_alphablend	0
	***	num_textures	1
	***	hlslvk	1841745752
	***	hlms_shadow_uses_depth_texture	1
	***	hlms_tangent	0
	***	hlms_bones_per_vertex	0
	***	out_uv0_tex_unit	0
	***	alpha_test_shadow_caster_only	0
	***	materials_per_buffer	2
	***	hlms_skeleton	0
	***	out_uv0_source_uv	0
	***	partial_pso_clip_distances	0
	***	syntax	-334286542
	***	hlms_tangent4	0
	***	metal	-1698855755
	***	diffuse_map	1
	***	hlms_render_depth_only	1
	***	full_pso_clip_distances	0
	***	textureMaps0	2
	***	hlms_qtangent	0
	***	out_uv_half_count0	2
	***	diffuse	0
	***	out_uv_half_count	1
	***	diffuse_map0_sampler	0
	***	hlms_normal	0
	***	num_samplers	1
	***	glslvk	-338983575
	***	hlsl	-334286542
	DONE DUMPING PROPERTIES
	***	alpha_test_cmp_func	==
	DONE DUMPING PIECES
#endif

#define ushort uint
#define ushort3 uint3
#define ushort4 uint4
#define ogre_float4x3 float4x3

//Short used for read operations. It's an int in GLSL & HLSL. An ushort in Metal
#define rshort int
#define rshort2 int2
#define rint int
//Short used for write operations. It's an int in GLSL. An ushort in HLSL & Metal
#define wshort2 uint2
#define wshort3 uint3

#define toFloat3x3( x ) ((float3x3)(x))
#define buildFloat3x3( row0, row1, row2 ) transpose( float3x3( row0, row1, row2 ) )

#define min3( a, b, c ) min( a, min( b, c ) )
#define max3( a, b, c ) max( a, max( b, c ) )

#define INLINE
#define NO_INTERPOLATION_PREFIX nointerpolation
#define NO_INTERPOLATION_SUFFIX

#define PARAMS_ARG_DECL
#define PARAMS_ARG

#define floatBitsToUint(x) asuint(x)
#define uintBitsToFloat(x) asfloat(x)
#define floatBitsToInt(x) asint(x)
#define fract frac
#define lessThan( a, b ) (a < b)

#define inVs_vertexId input.vertexId
#define inVs_vertex input.vertex
#define inVs_normal input.normal
#define inVs_tangent input.tangent
#define inVs_binormal input.binormal
#define inVs_blendWeights input.blendWeights
#define inVs_blendIndices input.blendIndices
#define inVs_qtangent input.qtangent
#define inVs_colour input.colour

#define inVs_drawId input.drawId

#define finalDrawId input.drawId


#define inVs_uv0 input.uv0
#define outVs_Position outVs.gl_Position
#define outVs_viewportIndex outVs.gl_ViewportIndex
#define outVs_clipDistance0 outVs.gl_ClipDistance0.x

#define gl_SampleMaskIn0 gl_SampleMask
#define interpolateAtSample( interp, subsample ) EvaluateAttributeAtSample( interp, subsample )
#define findLSB firstbitlow
#define findMSB firstbithigh
#define mod( a, b ) (a - b * floor(a / b))

#define outPs_colour0 outPs.colour0
#define OGRE_Sample( tex, sampler, uv ) tex.Sample( sampler, uv )
#define OGRE_SampleLevel( tex, sampler, uv, lod ) tex.SampleLevel( sampler, uv, lod )
#define OGRE_SampleArray2D( tex, sampler, uv, arrayIdx ) tex.Sample( sampler, float3( uv, arrayIdx ) )
#define OGRE_SampleArray2DLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float3( uv, arrayIdx ), lod )
#define OGRE_SampleArrayCubeLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float4( uv, arrayIdx ), lod )
#define OGRE_SampleGrad( tex, sampler, uv, ddx, ddy ) tex.SampleGrad( sampler, uv, ddx, ddy )
#define OGRE_SampleArray2DGrad( tex, sampler, uv, arrayIdx, ddx, ddy ) tex.SampleGrad( sampler, float3( uv, arrayIdx ), ddx, ddy )
#define OGRE_ddx( val ) ddx( val )
#define OGRE_ddy( val ) ddy( val )
#define OGRE_Load2D( tex, iuv, lod ) tex.Load( int3( iuv, lod ) )
#define OGRE_LoadArray2D( tex, iuv, arrayIdx, lod ) tex.Load( int4( iuv, arrayIdx, lod ) )
#define OGRE_Load2DMS( tex, iuv, subsample ) tex.Load( iuv, subsample )

#define OGRE_Load3D( tex, iuv, lod ) tex.Load( int4( iuv, lod ) )

#define bufferFetch( buffer, idx ) buffer.Load( idx )
#define bufferFetch1( buffer, idx ) buffer.Load( idx ).x

#define structuredBufferFetch( buffer, idx ) buffer[idx]

#define ReadOnlyBuffer( slot, varType, varName ) StructuredBuffer<varType> varName : register(t##slot)
#define readOnlyFetch( bufferVar, idx ) bufferVar[idx]
#define readOnlyFetch1( bufferVar, idx ) bufferVar[idx].x

#define OGRE_Texture3D_float4 Texture3D

#define OGRE_ArrayTex( declType, varName, arrayCount ) declType varName[arrayCount]

#define OGRE_SAMPLER_ARG_DECL( samplerName ) , SamplerState samplerName
#define OGRE_SAMPLER_ARG( samplerName ) , samplerName

#define CONST_BUFFER( bufferName, bindingPoint ) cbuffer bufferName : register(b##bindingPoint)
#define CONST_BUFFER_STRUCT_BEGIN( structName, bindingPoint ) cbuffer structName : register(b##bindingPoint) { struct _##structName
#define CONST_BUFFER_STRUCT_END( variableName ) variableName; }

#define FLAT_INTERPOLANT( decl, bindingPoint ) nointerpolation decl : TEXCOORD##bindingPoint
#define INTERPOLANT( decl, bindingPoint ) decl : TEXCOORD##bindingPoint

#define OGRE_OUT_REF( declType, variableName ) out declType variableName
#define OGRE_INOUT_REF( declType, variableName ) inout declType variableName

#define OGRE_ARRAY_START( type ) {
#define OGRE_ARRAY_END }


// START UNIFORM DECLARATION

// END UNIFORM DECLARATION

struct PS_INPUT
{
	
						

};



	
		Texture2D textureMaps0 : register(t2);
	




SamplerState samplerState0 : register(s2);


struct PS_OUTPUT
{
	
		
			
			
				float colour0 : SV_Depth;
			
		
	

	
};




	#define DiffuseSampler0 samplerState0
				#define SampleDiffuse0( tex, sampler, uv ) OGRE_Sample( tex, sampler, uv )
		#define DiffuseTexture0 textureMaps0
	
	
		#define DiffuseUV0 inPs.uv0.
		
// START UNIFORM DECLARATION
	
	
struct Material
{
	float4 alpha_test_threshold;
	float4 diffuse;

	
		uint4 indices0_3;
		uint4 indices4_7;
	
	
};


	CONST_BUFFER( MaterialBuf, 1 )
	{
		Material materialArray[2];
	};

	
	//Uniforms that change per Item/Entity
	CONST_BUFFER( InstanceBuffer, 2 )
	{
		//.x =
		//Contains the material's start index.
		//
		//.y =
		//shadowConstantBias. Send the bias directly to avoid an
		//unnecessary indirection during the shadow mapping pass.
		//Must be loaded with uintBitsToFloat
		//
		//.z =
		//Contains 0 or 1 to index into passBuf.viewProj[]. Only used
		//if hlms_identity_viewproj_dynamic is set.
		
			uint4 worldMaterialIdx[2];
				};

	
// END UNIFORM DECLARATION






	#define outDepth outPs.colour0



void main
(
	PS_INPUT inPs
	
)
{
	PS_OUTPUT outPs;
	

	
float4 diffuseCol = float4( 1.0f, 1.0f, 1.0f, 1.0f );


	

	
		ushort materialId	= worldMaterialIdx[inPs.drawId].x & 0x1FFu;
		#define material materialArray[materialId]
		



// Decode diffuse indices (for array textures)

	ushort diffuseIdx0 = material.indices0_3.x & 0x0000FFFFu;															

	// Load base image
	diffuseCol = SampleDiffuse0( DiffuseTexture0, DiffuseSampler0,
								 DiffuseUV0  ).;

// Load each additional layer and blend it

// Apply material colour
			



	if( material.alpha_test_threshold.x == diffuseCol.w )
		discard;




	
	




}

Vertex Shader:

Code: Select all

#if 0
	***	hlms_uv_count0	2
	***	hlms_pose	0
	***	hlms_uv_count	1
	***	diffuse_map0_idx	0
	***	fast_shader_build_hack	1
	***	glsl	635204550
	***	uv_diffuse0	0
	***	out_uv_count	1
	***	hlms_high_quality	0
	***	hlms_shadowcaster	1
	***	samplerStateStart	2
	***	alpha_test	1
	***	glsles	1070293233
	***	out_uv0_out_uv	0
	***	diffuse_map0	1
	***	out_uv0_texture_matrix	0
	***	hlms_alpha_to_coverage	0
	***	hlms_shadowcaster_directional	1
	***	hlms_pose_normals	0
	***	hlms_pose_half	0
	***	hlms_alphablend	0
	***	num_textures	1
	***	hlslvk	1841745752
	***	hlms_shadow_uses_depth_texture	1
	***	hlms_tangent	0
	***	hlms_bones_per_vertex	0
	***	out_uv0_tex_unit	0
	***	alpha_test_shadow_caster_only	0
	***	materials_per_buffer	2
	***	hlms_skeleton	0
	***	out_uv0_source_uv	0
	***	syntax	-334286542
	***	hlms_tangent4	0
	***	metal	-1698855755
	***	diffuse_map	1
	***	hlms_render_depth_only	1
	***	textureMaps0	2
	***	hlms_qtangent	0
	***	out_uv_half_count0	2
	***	diffuse	0
	***	out_uv_half_count	1
	***	diffuse_map0_sampler	0
	***	hlms_normal	0
	***	num_samplers	1
	***	glslvk	-338983575
	***	hlsl	-334286542
	DONE DUMPING PROPERTIES
	DONE DUMPING PIECES
#endif

//#include "SyntaxHighlightingMisc.h"


#define ushort uint
#define ushort3 uint3
#define ushort4 uint4
#define ogre_float4x3 float4x3

//Short used for read operations. It's an int in GLSL & HLSL. An ushort in Metal
#define rshort int
#define rshort2 int2
#define rint int
//Short used for write operations. It's an int in GLSL. An ushort in HLSL & Metal
#define wshort2 uint2
#define wshort3 uint3

#define toFloat3x3( x ) ((float3x3)(x))
#define buildFloat3x3( row0, row1, row2 ) transpose( float3x3( row0, row1, row2 ) )

#define min3( a, b, c ) min( a, min( b, c ) )
#define max3( a, b, c ) max( a, max( b, c ) )

#define INLINE
#define NO_INTERPOLATION_PREFIX nointerpolation
#define NO_INTERPOLATION_SUFFIX

#define PARAMS_ARG_DECL
#define PARAMS_ARG

#define floatBitsToUint(x) asuint(x)
#define uintBitsToFloat(x) asfloat(x)
#define floatBitsToInt(x) asint(x)
#define fract frac
#define lessThan( a, b ) (a < b)

#define inVs_vertexId input.vertexId
#define inVs_vertex input.vertex
#define inVs_normal input.normal
#define inVs_tangent input.tangent
#define inVs_binormal input.binormal
#define inVs_blendWeights input.blendWeights
#define inVs_blendIndices input.blendIndices
#define inVs_qtangent input.qtangent
#define inVs_colour input.colour

#define inVs_drawId input.drawId

#define finalDrawId input.drawId


#define inVs_uv0 input.uv0
#define outVs_Position outVs.gl_Position
#define outVs_viewportIndex outVs.gl_ViewportIndex
#define outVs_clipDistance0 outVs.gl_ClipDistance0.x

#define gl_SampleMaskIn0 gl_SampleMask
#define interpolateAtSample( interp, subsample ) EvaluateAttributeAtSample( interp, subsample )
#define findLSB firstbitlow
#define findMSB firstbithigh
#define mod( a, b ) (a - b * floor(a / b))

#define outPs_colour0 outPs.colour0
#define OGRE_Sample( tex, sampler, uv ) tex.Sample( sampler, uv )
#define OGRE_SampleLevel( tex, sampler, uv, lod ) tex.SampleLevel( sampler, uv, lod )
#define OGRE_SampleArray2D( tex, sampler, uv, arrayIdx ) tex.Sample( sampler, float3( uv, arrayIdx ) )
#define OGRE_SampleArray2DLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float3( uv, arrayIdx ), lod )
#define OGRE_SampleArrayCubeLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float4( uv, arrayIdx ), lod )
#define OGRE_SampleGrad( tex, sampler, uv, ddx, ddy ) tex.SampleGrad( sampler, uv, ddx, ddy )
#define OGRE_SampleArray2DGrad( tex, sampler, uv, arrayIdx, ddx, ddy ) tex.SampleGrad( sampler, float3( uv, arrayIdx ), ddx, ddy )
#define OGRE_ddx( val ) ddx( val )
#define OGRE_ddy( val ) ddy( val )
#define OGRE_Load2D( tex, iuv, lod ) tex.Load( int3( iuv, lod ) )
#define OGRE_LoadArray2D( tex, iuv, arrayIdx, lod ) tex.Load( int4( iuv, arrayIdx, lod ) )
#define OGRE_Load2DMS( tex, iuv, subsample ) tex.Load( iuv, subsample )

#define OGRE_Load3D( tex, iuv, lod ) tex.Load( int4( iuv, lod ) )

#define bufferFetch( buffer, idx ) buffer.Load( idx )
#define bufferFetch1( buffer, idx ) buffer.Load( idx ).x

#define structuredBufferFetch( buffer, idx ) buffer[idx]

#define ReadOnlyBuffer( slot, varType, varName ) StructuredBuffer<varType> varName : register(t##slot)
#define readOnlyFetch( bufferVar, idx ) bufferVar[idx]
#define readOnlyFetch1( bufferVar, idx ) bufferVar[idx].x

#define OGRE_Texture3D_float4 Texture3D

#define OGRE_ArrayTex( declType, varName, arrayCount ) declType varName[arrayCount]

#define OGRE_SAMPLER_ARG_DECL( samplerName ) , SamplerState samplerName
#define OGRE_SAMPLER_ARG( samplerName ) , samplerName

#define CONST_BUFFER( bufferName, bindingPoint ) cbuffer bufferName : register(b##bindingPoint)
#define CONST_BUFFER_STRUCT_BEGIN( structName, bindingPoint ) cbuffer structName : register(b##bindingPoint) { struct _##structName
#define CONST_BUFFER_STRUCT_END( variableName ) variableName; }

#define FLAT_INTERPOLANT( decl, bindingPoint ) nointerpolation decl : TEXCOORD##bindingPoint
#define INTERPOLANT( decl, bindingPoint ) decl : TEXCOORD##bindingPoint

#define OGRE_OUT_REF( declType, variableName ) out declType variableName
#define OGRE_INOUT_REF( declType, variableName ) inout declType variableName

#define OGRE_ARRAY_START( type ) {
#define OGRE_ARRAY_END }




float4x4 UNPACK_MAT4( StructuredBuffer<float4> matrixBuf, uint pixelIdx )
{
	float4 row1 = matrixBuf[int(pixelIdx << 2u)];
	float4 row2 = matrixBuf[int((pixelIdx << 2u) + 1u)];
	float4 row3 = matrixBuf[int((pixelIdx << 2u) + 2u)];
	float4 row4 = matrixBuf[int((pixelIdx << 2u) + 3u)];

return transpose( float4x4( row1, row2, row3, row4 ) );
}


// START UNIFORM DECLARATION

//Uniforms that change per pass
CONST_BUFFER_STRUCT_BEGIN( PassBuffer, 0 )
{
	//Vertex shader
	
		float4x4 viewProj[2];
							
					float4 depthRange;
						//Pixel Shader
	float4 invWindowSize;
	
}
CONST_BUFFER_STRUCT_END( passBuf );



	//Uniforms that change per Item/Entity
	CONST_BUFFER( InstanceBuffer, 2 )
	{
		//.x =
		//Contains the material's start index.
		//
		//.y =
		//shadowConstantBias. Send the bias directly to avoid an
		//unnecessary indirection during the shadow mapping pass.
		//Must be loaded with uintBitsToFloat
		//
		//.z =
		//Contains 0 or 1 to index into passBuf.viewProj[]. Only used
		//if hlms_identity_viewproj_dynamic is set.
		
			uint4 worldMaterialIdx[2];
				};


// END UNIFORM DECLARATION


	
		// Most common path
		#define masterMatrix worldViewProj
	




// START UNIFORM D3D DECLARATION
ReadOnlyBuffer( 0, float4, worldMatBuf );

// END UNIFORM D3D DECLARATION

struct VS_INPUT
{
	float4 vertex : POSITION;


float2 uv0 : TEXCOORD0;
uint drawId : DRAWID;

};

struct PS_INPUT
{

						

float4 gl_Position : SV_Position;





};

PS_INPUT main( VS_INPUT input )
{
	PS_INPUT outVs;
	


	
		float4x4 worldViewProj = UNPACK_MAT4( worldMatBuf, finalDrawId );
	

	
		outVs_Position = mul( inVs_vertex, masterMatrix );
	



	

	

	
		
			outVs.uv0. =
					inVs_uv0.xy;
		
	

	
		outVs.drawId = finalDrawId;
	




	
						float shadowConstantBias = -uintBitsToFloat( worldMaterialIdx[inVs_drawId].y ) * passBuf.depthRange.y;
				
	
	
	
		//We can't make the depth buffer linear without Z out in the fragment shader;
		//however we can use a cheap approximation ("pseudo linear depth")
		//see http://www.yosoygames.com.ar/wp/2014/01/linear-depth-buffer-my-ass/
		
			outVs_Position.z = outVs_Position.z + shadowConstantBias;
				
		







return outVs;
}

[dark_sylinc]

Hi!

I'll need to check if this is not our bug. Because it sounds like it could be an OgreNext bug.

Meanwhile make sure you're setting the HlmsUnlit properly (i.e. via HlmsUnlit::getDefaultPaths and all), and make sure that you do not put both PBS and Unlit data files (i.e. the data files under Samples/Media/Hlms) in the same folder.

Btw:

Code: Select all

texture focusedTex 2048 2048 PFG_D32_FLOAT keep_content

keep_content should be used when you need to preserve the contents between frames. Since shadow maps (at least by default) are regenerated from scratch every frame, you should not ask for keep_content.

[dark_sylinc]

As I suspected this was an OgreNext bug!

Fixed! Thanks for the report!

However there is an issue: The fix was pushed to 3.0 branch and not the 2.3 one (which is the one you're using), because it would affect ABI stability.

You have two options:

1. Upgrade to 3.0. It is very stable. The only reason it hasn't been officially released yet is because I need to review this PR and I haven't got the time.
2. Stay with 2.3; and use PBS' emissive as a weak replacement for Unlit.

Cheers
Matias

[haloman30]

Btw:

Code: Select all

texture focusedTex 2048 2048 PFG_D32_FLOAT keep_content

keep_content should be used when you need to preserve the contents between frames. Since shadow maps (at least by default) are regenerated from scratch every frame, you should not ask for keep_content.

Funny you mention that - I was experiencing a crash (or more accurately an ogre exception) without keep_content set.

Code: Select all

Transitioning texture focusedTex[Value 0x00000004] from Undefined to a read-only layout. Perhaps you didn't want to set TextureFlags::DiscardableContent / aka keep_content in compositor?

As I suspected this was an OgreNext bug!

Fixed! Thanks for the report!

Awesome! Reassuring at least to know that I wasn't totally out of my mind. I'll likely look into upgrading to 3.0 (assuming it still runs on Windows 7, as I'm targeting my engine to run on 7), or worst case scenario I'll try the PBR emissive method.

[haloman30]

As I suspected this was an OgreNext bug!

Fixed! Thanks for the report!

However there is an issue: The fix was pushed to 3.0 branch and not the 2.3 one (which is the one you're using), because it would affect ABI stability.

You have two options:

1. Upgrade to 3.0. It is very stable. The only reason it hasn't been officially released yet is because I need to review this PR and I haven't got the time.
2. Stay with 2.3; and use PBS' emissive as a weak replacement for Unlit.

Cheers
Matias

Hey - so, I've upgraded to 3.0.0, and while it does appear to work generally, the issue in question still persists. As soon as I add in an object using an unlit datablock, instant crash - and it seems to be exactly the same error as before. For reference, I'm once again including the error messages and the compiled shaders in question.

Error Log:

Code: Select all

06.19.24 17:14:55 ### ERROR OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000002VertexShader_vs Errors: 300000002VertexShader_vs.hlsl(219,4-15): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at K:\Development\ogre-3.0.0\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
06.19.24 17:14:55 High-level program 300000002VertexShader_vs encountered an error during loading and is thus not supported. OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000002VertexShader_vs Errors: 300000002VertexShader_vs.hlsl(219,4-15): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at K:\Development\ogre-3.0.0\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
06.19.24 17:14:55 ### ERROR OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000002PixelShader_ps Errors: 300000002PixelShader_ps.hlsl(208,41-51): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at K:\Development\ogre-3.0.0\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
06.19.24 17:14:55 High-level program 300000002PixelShader_ps encountered an error during loading and is thus not supported. OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 300000002PixelShader_ps Errors: 300000002PixelShader_ps.hlsl(208,41-51): error X3018: invalid subscript 'drawId'  in D3D11HLSLProgram::compileMicrocode at K:\Development\ogre-3.0.0\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)

Vertex Shader:

Code: Select all

#if 0
	***	hlslvk	58199705
	***	hlms_tangent	0
	***	glslvk	-1410682516
	***	diffuse	1
	***	samplerStateStart	2
	***	out_uv_half_count	0
	***	full32	-1395729169
	***	materials_per_buffer	2
	***	hlms_alphablend	1
	***	fast_shader_build_hack	1
	***	alpha_test_shadow_caster_only	0
	***	hlms_render_depth_only	1
	***	hlms_normal	0
	***	hlms_pose_half	0
	***	hlms_tangent4	0
	***	hlms_qtangent	0
	***	syntax	-1201987786
	***	alpha_test	1
	***	glsles	1831466908
	***	hlms_alpha_to_coverage	0
	***	hlms_skeleton	0
	***	hlms_bones_per_vertex	0
	***	hlms_pose_normals	0
	***	precision_mode	-1395729169
	***	hlms_shadowcaster	1
	***	hlms_shadow_uses_depth_texture	1
	***	out_uv_count	0
	***	metal	-1561198854
	***	hlms_pose	0
	***	hlms_uv_count	0
	***	hlsl	-1201987786
	***	glsl	1437276862
	***	relaxed	1464334097
	***	hlms_shadowcaster_directional	1
	***	midf16	405557013
	DONE DUMPING PROPERTIES
	DONE DUMPING PIECES
#endif

//#include "SyntaxHighlightingMisc.h"


#define ushort uint
#define ushort3 uint3
#define ushort4 uint4
#define ogre_float4x3 float4x3

//Short used for read operations. It's an int in GLSL & HLSL. An ushort in Metal
#define rshort int
#define rshort2 int2
#define rint int
//Short used for write operations. It's an int in GLSL. An ushort in HLSL & Metal
#define wshort2 uint2
#define wshort3 uint3

#define toFloat3x3( x ) ((float3x3)(x))
#define buildFloat3x3( row0, row1, row2 ) transpose( float3x3( row0, row1, row2 ) )

#define min3( a, b, c ) min( a, min( b, c ) )
#define max3( a, b, c ) max( a, max( b, c ) )

#define INLINE
#define NO_INTERPOLATION_PREFIX nointerpolation
#define NO_INTERPOLATION_SUFFIX

#define PARAMS_ARG_DECL
#define PARAMS_ARG

#define floatBitsToUint(x) asuint(x)
#define uintBitsToFloat(x) asfloat(x)
#define floatBitsToInt(x) asint(x)
#define fract frac
#define lessThan( a, b ) (a < b)

#define inVs_vertexId input.vertexId
#define inVs_vertex input.vertex
#define inVs_normal input.normal
#define inVs_tangent input.tangent
#define inVs_binormal input.binormal
#define inVs_blendWeights input.blendWeights
#define inVs_blendIndices input.blendIndices
#define inVs_qtangent input.qtangent
#define inVs_colour input.colour

#define inVs_drawId input.drawId

#define finalDrawId input.drawId


#define outVs_Position outVs.gl_Position
#define outVs_viewportIndex outVs.gl_ViewportIndex
#define outVs_clipDistance0 outVs.gl_ClipDistance0.x

#define gl_SampleMaskIn0 gl_SampleMask
#define interpolateAtSample( interp, subsample ) EvaluateAttributeAtSample( interp, subsample )
#define findLSB firstbitlow
#define findMSB firstbithigh
#define mod( a, b ) (a - b * floor(a / b))

#define outPs_colour0 outPs.colour0
#define OGRE_Sample( tex, sampler, uv ) tex.Sample( sampler, uv )
#define OGRE_SampleLevel( tex, sampler, uv, lod ) tex.SampleLevel( sampler, uv, lod )
#define OGRE_SampleArray2D( tex, sampler, uv, arrayIdx ) tex.Sample( sampler, float3( uv, arrayIdx ) )
#define OGRE_SampleArray2DLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float3( uv, arrayIdx ), lod )
#define OGRE_SampleArrayCubeLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float4( uv, arrayIdx ), lod )
#define OGRE_SampleGrad( tex, sampler, uv, ddx, ddy ) tex.SampleGrad( sampler, uv, ddx, ddy )
#define OGRE_SampleArray2DGrad( tex, sampler, uv, arrayIdx, ddx, ddy ) tex.SampleGrad( sampler, float3( uv, arrayIdx ), ddx, ddy )
#define OGRE_ddx( val ) ddx( val )
#define OGRE_ddy( val ) ddy( val )
#define OGRE_Load2D( tex, iuv, lod ) tex.Load( int3( iuv, lod ) )
#define OGRE_LoadArray2D( tex, iuv, arrayIdx, lod ) tex.Load( int4( iuv, arrayIdx, lod ) )
#define OGRE_Load2DMS( tex, iuv, subsample ) tex.Load( iuv, subsample )

#define OGRE_Load3D( tex, iuv, lod ) tex.Load( int4( iuv, lod ) )

#define bufferFetch( buffer, idx ) buffer.Load( idx )
#define bufferFetch1( buffer, idx ) buffer.Load( idx ).x

#define structuredBufferFetch( buffer, idx ) buffer[idx]

#define ReadOnlyBuffer( slot, varType, varName ) StructuredBuffer<varType> varName : register(t##slot)
#define readOnlyFetch( bufferVar, idx ) bufferVar[idx]
#define readOnlyFetch1( bufferVar, idx ) bufferVar[idx].x

#define OGRE_Texture3D_float4 Texture3D

#define OGRE_ArrayTex( declType, varName, arrayCount ) declType varName[arrayCount]

#define OGRE_SAMPLER_ARG_DECL( samplerName ) , SamplerState samplerName
#define OGRE_SAMPLER_ARG( samplerName ) , samplerName

#define CONST_BUFFER( bufferName, bindingPoint ) cbuffer bufferName : register(b##bindingPoint)
#define CONST_BUFFER_STRUCT_BEGIN( structName, bindingPoint ) cbuffer structName : register(b##bindingPoint) { struct _##structName
#define CONST_BUFFER_STRUCT_END( variableName ) variableName; }

#define FLAT_INTERPOLANT( decl, bindingPoint ) nointerpolation decl : TEXCOORD##bindingPoint
#define INTERPOLANT( decl, bindingPoint ) decl : TEXCOORD##bindingPoint

#define OGRE_OUT_REF( declType, variableName ) out declType variableName
#define OGRE_INOUT_REF( declType, variableName ) inout declType variableName

#define OGRE_ARRAY_START( type ) {
#define OGRE_ARRAY_END }




float4x4 UNPACK_MAT4( StructuredBuffer<float4> matrixBuf, uint pixelIdx )
{
	float4 row1 = matrixBuf[int(pixelIdx << 2u)];
	float4 row2 = matrixBuf[int((pixelIdx << 2u) + 1u)];
	float4 row3 = matrixBuf[int((pixelIdx << 2u) + 2u)];
	float4 row4 = matrixBuf[int((pixelIdx << 2u) + 3u)];

return transpose( float4x4( row1, row2, row3, row4 ) );
}


// START UNIFORM DECLARATION

//Uniforms that change per pass
CONST_BUFFER_STRUCT_BEGIN( PassBuffer, 0 )
{
	//Vertex shader
	
		float4x4 viewProj[2];
							
					float4 depthRange;
						//Pixel Shader
	float4 invWindowSize;
	
}
CONST_BUFFER_STRUCT_END( passBuf );



	//Uniforms that change per Item/Entity
	CONST_BUFFER( InstanceBuffer, 2 )
	{
		//.x =
		//Contains the material's start index.
		//
		//.y =
		//shadowConstantBias. Send the bias directly to avoid an
		//unnecessary indirection during the shadow mapping pass.
		//Must be loaded with uintBitsToFloat
		//
		//.z =
		//Contains 0 or 1 to index into passBuf.viewProj[]. Only used
		//if hlms_identity_viewproj_dynamic is set.
		
			uint4 worldMaterialIdx[2];
				};


// END UNIFORM DECLARATION


	
		// Most common path
		#define masterMatrix worldViewProj
	




// START UNIFORM D3D DECLARATION
ReadOnlyBuffer( 0, float4, worldMatBuf );

// END UNIFORM D3D DECLARATION

struct VS_INPUT
{
	float4 vertex : POSITION;


uint drawId : DRAWID;

};

struct PS_INPUT
{

						

float4 gl_Position : SV_Position;





};

PS_INPUT main( VS_INPUT input )
{
	PS_INPUT outVs;
	


	
		float4x4 worldViewProj = UNPACK_MAT4( worldMatBuf, finalDrawId );
	

	
		outVs_Position = mul( inVs_vertex, masterMatrix );
	



	

	

	

	
		outVs.drawId = finalDrawId;
	




	
						float shadowConstantBias = -uintBitsToFloat( worldMaterialIdx[inVs_drawId].y ) * passBuf.depthRange.y;
				
	
	
	
		//We can't make the depth buffer linear without Z out in the fragment shader;
		//however we can use a cheap approximation ("pseudo linear depth")
		//see http://www.yosoygames.com.ar/wp/2014/01/linear-depth-buffer-my-ass/
		
			outVs_Position.z = outVs_Position.z + shadowConstantBias;
				
		







return outVs;
}

Pixel Shader:

Code: Select all

#if 0
	***	hlslvk	58199705
	***	hlms_tangent	0
	***	glslvk	-1410682516
	***	diffuse	1
	***	samplerStateStart	2
	***	out_uv_half_count	0
	***	full32	-1395729169
	***	materials_per_buffer	2
	***	hlms_alphablend	1
	***	fast_shader_build_hack	1
	***	alpha_test_shadow_caster_only	0
	***	hlms_render_depth_only	1
	***	hlms_normal	0
	***	hlms_pose_half	0
	***	partial_pso_clip_distances	0
	***	hlms_tangent4	0
	***	hlms_qtangent	0
	***	syntax	-1201987786
	***	hlms_disable_stage	0
	***	alpha_test	1
	***	glsles	1831466908
	***	hlms_alpha_to_coverage	0
	***	hlms_skeleton	0
	***	hlms_bones_per_vertex	0
	***	hlms_pose_normals	0
	***	texcoord	0
	***	precision_mode	-1395729169
	***	hlms_shadowcaster	1
	***	hlms_shadow_uses_depth_texture	1
	***	out_uv_count	0
	***	full_pso_clip_distances	0
	***	metal	-1561198854
	***	hlms_pose	0
	***	hlms_uv_count	0
	***	hlsl	-1201987786
	***	glsl	1437276862
	***	relaxed	1464334097
	***	hlms_shadowcaster_directional	1
	***	midf16	405557013
	DONE DUMPING PROPERTIES
	***	blend_mode_idx6	@insertpiece( NormalNonPremul)
	***	blend_mode_idx15	@insertpiece( NormalNonPremul)
	***	blend_mode_idx0	@insertpiece( NormalNonPremul)
	***	blend_mode_idx12	@insertpiece( NormalNonPremul)
	***	blend_mode_idx1	@insertpiece( NormalNonPremul)
	***	blend_mode_idx13	@insertpiece( NormalNonPremul)
	***	blend_mode_idx9	@insertpiece( NormalNonPremul)
	***	alpha_test_cmp_func	==
	***	blend_mode_idx10	@insertpiece( NormalNonPremul)
	***	blend_mode_idx14	@insertpiece( NormalNonPremul)
	***	blend_mode_idx2	@insertpiece( NormalNonPremul)
	***	blend_mode_idx8	@insertpiece( NormalNonPremul)
	***	blend_mode_idx3	@insertpiece( NormalNonPremul)
	***	blend_mode_idx5	@insertpiece( NormalNonPremul)
	***	blend_mode_idx4	@insertpiece( NormalNonPremul)
	***	blend_mode_idx11	@insertpiece( NormalNonPremul)
	***	blend_mode_idx7	@insertpiece( NormalNonPremul)
	DONE DUMPING PIECES
#endif

#define ushort uint
#define ushort3 uint3
#define ushort4 uint4
#define ogre_float4x3 float4x3

//Short used for read operations. It's an int in GLSL & HLSL. An ushort in Metal
#define rshort int
#define rshort2 int2
#define rint int
//Short used for write operations. It's an int in GLSL. An ushort in HLSL & Metal
#define wshort2 uint2
#define wshort3 uint3

#define toFloat3x3( x ) ((float3x3)(x))
#define buildFloat3x3( row0, row1, row2 ) transpose( float3x3( row0, row1, row2 ) )

#define min3( a, b, c ) min( a, min( b, c ) )
#define max3( a, b, c ) max( a, max( b, c ) )

#define INLINE
#define NO_INTERPOLATION_PREFIX nointerpolation
#define NO_INTERPOLATION_SUFFIX

#define PARAMS_ARG_DECL
#define PARAMS_ARG

#define floatBitsToUint(x) asuint(x)
#define uintBitsToFloat(x) asfloat(x)
#define floatBitsToInt(x) asint(x)
#define fract frac
#define lessThan( a, b ) (a < b)

#define inVs_vertexId input.vertexId
#define inVs_vertex input.vertex
#define inVs_normal input.normal
#define inVs_tangent input.tangent
#define inVs_binormal input.binormal
#define inVs_blendWeights input.blendWeights
#define inVs_blendIndices input.blendIndices
#define inVs_qtangent input.qtangent
#define inVs_colour input.colour

#define inVs_drawId input.drawId

#define finalDrawId input.drawId


#define outVs_Position outVs.gl_Position
#define outVs_viewportIndex outVs.gl_ViewportIndex
#define outVs_clipDistance0 outVs.gl_ClipDistance0.x

#define gl_SampleMaskIn0 gl_SampleMask
#define interpolateAtSample( interp, subsample ) EvaluateAttributeAtSample( interp, subsample )
#define findLSB firstbitlow
#define findMSB firstbithigh
#define mod( a, b ) (a - b * floor(a / b))

#define outPs_colour0 outPs.colour0
#define OGRE_Sample( tex, sampler, uv ) tex.Sample( sampler, uv )
#define OGRE_SampleLevel( tex, sampler, uv, lod ) tex.SampleLevel( sampler, uv, lod )
#define OGRE_SampleArray2D( tex, sampler, uv, arrayIdx ) tex.Sample( sampler, float3( uv, arrayIdx ) )
#define OGRE_SampleArray2DLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float3( uv, arrayIdx ), lod )
#define OGRE_SampleArrayCubeLevel( tex, sampler, uv, arrayIdx, lod ) tex.SampleLevel( sampler, float4( uv, arrayIdx ), lod )
#define OGRE_SampleGrad( tex, sampler, uv, ddx, ddy ) tex.SampleGrad( sampler, uv, ddx, ddy )
#define OGRE_SampleArray2DGrad( tex, sampler, uv, arrayIdx, ddx, ddy ) tex.SampleGrad( sampler, float3( uv, arrayIdx ), ddx, ddy )
#define OGRE_ddx( val ) ddx( val )
#define OGRE_ddy( val ) ddy( val )
#define OGRE_Load2D( tex, iuv, lod ) tex.Load( int3( iuv, lod ) )
#define OGRE_LoadArray2D( tex, iuv, arrayIdx, lod ) tex.Load( int4( iuv, arrayIdx, lod ) )
#define OGRE_Load2DMS( tex, iuv, subsample ) tex.Load( iuv, subsample )

#define OGRE_Load3D( tex, iuv, lod ) tex.Load( int4( iuv, lod ) )

#define bufferFetch( buffer, idx ) buffer.Load( idx )
#define bufferFetch1( buffer, idx ) buffer.Load( idx ).x

#define structuredBufferFetch( buffer, idx ) buffer[idx]

#define ReadOnlyBuffer( slot, varType, varName ) StructuredBuffer<varType> varName : register(t##slot)
#define readOnlyFetch( bufferVar, idx ) bufferVar[idx]
#define readOnlyFetch1( bufferVar, idx ) bufferVar[idx].x

#define OGRE_Texture3D_float4 Texture3D

#define OGRE_ArrayTex( declType, varName, arrayCount ) declType varName[arrayCount]

#define OGRE_SAMPLER_ARG_DECL( samplerName ) , SamplerState samplerName
#define OGRE_SAMPLER_ARG( samplerName ) , samplerName

#define CONST_BUFFER( bufferName, bindingPoint ) cbuffer bufferName : register(b##bindingPoint)
#define CONST_BUFFER_STRUCT_BEGIN( structName, bindingPoint ) cbuffer structName : register(b##bindingPoint) { struct _##structName
#define CONST_BUFFER_STRUCT_END( variableName ) variableName; }

#define FLAT_INTERPOLANT( decl, bindingPoint ) nointerpolation decl : TEXCOORD##bindingPoint
#define INTERPOLANT( decl, bindingPoint ) decl : TEXCOORD##bindingPoint

#define OGRE_OUT_REF( declType, variableName ) out declType variableName
#define OGRE_INOUT_REF( declType, variableName ) inout declType variableName

#define OGRE_ARRAY_START( type ) {
#define OGRE_ARRAY_END }


// START UNIFORM DECLARATION

// END UNIFORM DECLARATION

struct PS_INPUT
{
	
						

};








struct PS_OUTPUT
{
	
		
			
			
				float colour0 : SV_Depth;
			
		
	

	
};




// START UNIFORM DECLARATION
	
	
struct Material
{
	float4 alpha_test_threshold;
	float4 diffuse;

	
		uint4 indices0_3;
		uint4 indices4_7;
	
	
};


	CONST_BUFFER( MaterialBuf, 1 )
	{
		Material materialArray[2];
	};

	
	//Uniforms that change per Item/Entity
	CONST_BUFFER( InstanceBuffer, 2 )
	{
		//.x =
		//Contains the material's start index.
		//
		//.y =
		//shadowConstantBias. Send the bias directly to avoid an
		//unnecessary indirection during the shadow mapping pass.
		//Must be loaded with uintBitsToFloat
		//
		//.z =
		//Contains 0 or 1 to index into passBuf.viewProj[]. Only used
		//if hlms_identity_viewproj_dynamic is set.
		
			uint4 worldMaterialIdx[2];
				};

	
// END UNIFORM DECLARATION






	#define outDepth outPs.colour0



void main
(
	PS_INPUT inPs
	
)
{
	PS_OUTPUT outPs;
	

	
float4 diffuseCol = float4( 1.0f, 1.0f, 1.0f, 1.0f );


	

	
		ushort materialId	= worldMaterialIdx[inPs.drawId].x & 0x1FFu;
		#define material materialArray[materialId]
		



// Decode diffuse indices (for array textures)

																

// Load each additional layer and blend it

// Apply material colour
		
		diffuseCol *= material.diffuse;
		



	if( material.alpha_test_threshold.x == diffuseCol.w )
		discard;




	
	




}

[dark_sylinc]

Hi!

It seems I forgot to merge 3.0 -> master; thanks for that.

But please beware master is 4.0; double check you've checked out the v3-0 branch.

[haloman30]

Hi!

It seems I forgot to merge 3.0 -> master; thanks for that.

But please beware master is 4.0; double check you've checked out the v3-0 branch.

Yeah - I cloned and locally compiled the v3-0 branch, as I prefer to use it statically (trying to get all the 3rd party libraries I'm using to run that way, so there aren't hundreds of DLL files). Even double-checked that the commit was correct and manually checked a couple of the modified files in that commit, and the new changes you made were in - so it seems (at least to me) like the issue lies elsewhere.

I'll also note that in my case, the datablocks are generated entirely through code, as my engine uses its own content system that everything goes through - unsure if that's helpful or not.

[haloman30]

Some other information I've managed to seemingly work out:

• The issue does NOT happen when I use the same code to create these unlit datablocks within one of the ogre samples
• The issue seems to not originate with my compositor configuration - having my engine use a different compositor (one of the sample ones to be precise) results in the same exception being thrown
• The issue happens within both OpenGL and Direct3D 11 render systems (wasn't able to test Vulkan as I'm still having some issues getting it working, likely due to my own stuff I'm adding it, such as imgui)

I've tried comparing the HLSL output, but knowing very little about shader code in general, I wasn't able to really learn much. From what I can tell, it should be working - as I don't see anything obvious that would indicate drawId being missing, but again, this is beyond my current knowledge, so it's entirely possible I'm missing something obvious.

If anyone has any ideas of things to double-check, or any hints at where the problem could be originating from, I'd greatly appreciate it. I'd prefer to have unlit materials be available, both for my own use and for those using the game engine, so I'd really like to get these working properly if possible. Even if it's not a complete suggestion, even some vague pointer would at least be a start and something I can pursue.

[dark_sylinc]

Hi!

Make sure your copy of the Hlms datafolder was updated. It sounds like it wasn't. That would explain everything.

[haloman30]

Hi!

Make sure your copy of the Hlms datafolder was updated. It sounds like it wasn't. That would explain everything.

Yep, that did in fact fix it! Appreciate it!

I had assumed that most of the stuff in media was just like default assets for the samples and stuff, and not anything more involved like that - but yeah, bringing the 3.0.0 media folder over fixed that, as well as another minor visual issue I was running into before as well.

Thanks for the help!