[2.3] Decals on the Terra system

[Ridder Geel]

When attempting to use decals in a scene with terra objects it crashes due to the shaders/materials probably not supporting the decals.
Whats the right way to fix it? I found the "hlms_enable_decals" property, but i'm not entirely sure what the right way to enable it is. (I'd rather ask for the right way rather than to hack through a bunch of hlms properties)

Code: Select all

12:07:59: OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000004PixelShader_ps Errors:
E:\Game Development\Ogre\ogre-next\build\bin\Debug\700000004PixelShader_ps.hlsl(1432,25): error X3004: undeclared identifier 'decalsDiffuseTex'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
12:07:59: High-level program 700000004PixelShader_ps encountered an error during loading and is thus not supported.
OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000004PixelShader_ps Errors:
E:\Game Development\Ogre\ogre-next\build\bin\Debug\700000004PixelShader_ps.hlsl(1432,25): error X3004: undeclared identifier 'decalsDiffuseTex'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
12:07:59: OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000005PixelShader_ps Errors:
E:\Game Development\Ogre\ogre-next\build\bin\Debug\700000005PixelShader_ps.hlsl(1432,25): error X3004: undeclared identifier 'decalsDiffuseTex'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
12:07:59: High-level program 700000005PixelShader_ps encountered an error during loading and is thus not supported.
OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000005PixelShader_ps Errors:
E:\Game Development\Ogre\ogre-next\build\bin\Debug\700000005PixelShader_ps.hlsl(1432,25): error X3004: undeclared identifier 'decalsDiffuseTex'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)

Thanks in advance!

[dark_sylinc]

Hi!

Are you enabling Forward+? i.e.

Code: Select all

sceneManager->setForwardClustered( true, 16, 8, 24, 96, 4, 0, 2, 50 );

Also for debugging it would help a lot if you post the generated shader that is failing, with debug properties. You can do this via:

Code: Select all

hlmsTerra->setDebugOutputPath( true, true, "C:\\PathToDumpShaders" );

And upload the VS & PS that are failing.

[Ridder Geel]

Yep (I merged both tutorials with the parts that seemed required to keep the code here much simpler):

Code: Select all

   void Tutorial_TerrainGameState::createScene01()
   {
       Ogre::Root *root = mGraphicsSystem->getRoot();
       Ogre::SceneManager *sceneManager = mGraphicsSystem->getSceneManager();

   sceneManager->setForwardClustered( true, 16, 8, 24, 96, 4, 0, 2, 50 );

   // Render terrain after most objects, to improve performance by taking advantage of early Z

Here are the shader dumps:

ShaderDump.zip

[dark_sylinc]

OK I see the problem.

Does it go away if you replace HlmsTerra::getDefaultPaths with...?:

Code: Select all

    void HlmsTerra::getDefaultPaths( String &outDataFolderPath, StringVector &outLibraryFoldersPaths )
    {
        // We need to know what RenderSystem is currently in use, as the
        // name of the compatible shading language is part of the path
        Ogre::RenderSystem *renderSystem = Ogre::Root::getSingleton().getRenderSystem();
        Ogre::String shaderSyntax = "GLSL";
        if( renderSystem->getName() == "Direct3D11 Rendering Subsystem" )
            shaderSyntax = "HLSL";
        else if( renderSystem->getName() == "Metal Rendering Subsystem" )
            shaderSyntax = "Metal";

        // Fill the library folder paths with the relevant folders
        outLibraryFoldersPaths.clear();
        outLibraryFoldersPaths.push_back( "Hlms/Common/" + shaderSyntax );
        outLibraryFoldersPaths.push_back( "Hlms/Common/Any" );
        outLibraryFoldersPaths.push_back( "Hlms/Pbs/Any" );
#ifdef OGRE_BUILD_COMPONENT_ATMOSPHERE
        outLibraryFoldersPaths.push_back( "Hlms/Pbs/Any/Atmosphere" );
#endif
        outLibraryFoldersPaths.push_back( "Hlms/Pbs/Any/Main" );
        outLibraryFoldersPaths.push_back( "Hlms/Pbs/" + shaderSyntax );
        outLibraryFoldersPaths.push_back( "Hlms/Terra/Any" );

        // Fill the data folder path
        outDataFolderPath = "Hlms/Terra/" + shaderSyntax;
    }

Basically I only added a new line:

Code: Select all

outLibraryFoldersPaths.push_back( "Hlms/Pbs/" + shaderSyntax );

[Ridder Geel]

Yes, or well it changes to this error:

Code: Select all

16:54:21: OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000004PixelShader_ps Errors:
E:\Game Development\Ogre\ShaderDump\700000004PixelShader_ps.hlsl(1465,51-61): error X3018: invalid subscript 'normal'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
16:54:21: High-level program 700000004PixelShader_ps encountered an error during loading and is thus not supported.
OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000004PixelShader_ps Errors:
E:\Game Development\Ogre\ShaderDump\700000004PixelShader_ps.hlsl(1465,51-61): error X3018: invalid subscript 'normal'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
16:54:21: OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000005PixelShader_ps Errors:
E:\Game Development\Ogre\ShaderDump\700000005PixelShader_ps.hlsl(1465,51-61): error X3018: invalid subscript 'normal'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)
16:54:21: High-level program 700000005PixelShader_ps encountered an error during loading and is thus not supported.
OGRE EXCEPTION(-2147467259:RenderingAPIException): Cannot compile D3D11 high-level shader 700000005PixelShader_ps Errors:
E:\Game Development\Ogre\ShaderDump\700000005PixelShader_ps.hlsl(1465,51-61): error X3018: invalid subscript 'normal'
 in D3D11HLSLProgram::compileMicrocode at E:\Game Development\Ogre\ogre-next\RenderSystems\Direct3D11\src\OgreD3D11HLSLProgram.cpp (line 554)

ShaderDump.zip

[dark_sylinc]

Ok this is a bug.

Working on it.

[Ridder Geel]

Awesome, if you need any help with it or with testing just let me know!

I was also wondering if there is a way to make the decal only affect certain objects instead of all objects?

[dark_sylinc]

Fixed.

Let me know if you still have issues.

[dark_sylinc]

I was also wondering if there is a way to make the decal only affect certain objects instead of all objects?

Mmm, looking at the code it appears that Decals honors visibility masks, because SceneManager::_collectForwardPlusObjects uses the standard frustum culling code; which in SceneManager::cullFrustum reads from:

Code: Select all

const uint32 visibilityMask =
    request.cullingLights
        ? ( camera->getLastViewport()->getLightVisibilityMask() & mLightMask )
        : ( ( camera->getLastViewport()->getVisibilityMask() & this->getVisibilityMask() ) |
            ( camera->getLastViewport()->getVisibilityMask() &
              ~VisibilityFlags::RESERVED_VISIBILITY_FLAGS ) );

However since we don't look at this mask for caching; it may be impossible without slight modifications to OgreNext.

Options

Option A

I was thinking that the following should be possible:

1. Divide into two render_scene passes.
2. The 1st render_scene will set CompositorPassSceneDef::mVisibilityMask to 1.
3. You set the Decals' mask to 1 and also the objects affected by the mask to 1.
4. For every other object that should be unaffected, you set it to 2 (i.e. 1u << 1u).
5. The 2nd render_scene will set CompositorPassSceneDef::mVisibilityMask to 1.

The source code of OgreNext needs to be modified. The two overloads of ForwardPlusBase::getCachedGridFor would have to account CompositorPassSceneDef::mVisibilityMask; or much better yet create a mDecalsMask property.

This might be expensive though, specially if you have many Forward+ lights.

Option B

Similar to Option A, but create CompositorPassSceneDef::mEnableDecals boolean so that the 2nd pass sets this setting to false. Therefore the 2nd pass can reuse the same cached data from the 1st pass; it's just that HlmsPbs must generate a shader that ignores the decals data; and this is evaluated in HlmsPbs::preparePassHash.

Option C

Modify shader's source code so that the masks are resolved inside the shaders. It's the most flexible but moves the cost to the GPU. The previous methods have a cost on the CPU.

[Ridder Geel]

It works like a charm!

Thanks for the info on the 'filtering' of decal things.
I'm looking into using decals to have the selection for units in an RTS engine/game that I'm making.
So generally most of the decals I will use will be the size of a single unit or perhaps two, I'm not sure which approach would be best in that case.
The decals will probably all be used only on the terrain, so perhaps I could filter it out that way (only have the decals appear on the Terra shader, not the normal PBS one), however if you are looking into adjusting it for Ogre in general, that obviously won't be a solution.

[Ridder Geel]

Do the detail maps of the terra system support normal maps in combination with the decals?
Because things start to look really strange when I add normal maps and add a decal (generally speaking a lot of blackness, and a lot of the normal map seems to disappear)
(So there is a difference in appearance with and without a decal added, even on parts not covered by it)
(Test material code)

Code: Select all

{
    "Terra" :
    {
			"TerraExampleMaterial": {
				"diffuse": {
					"value": [ 1, 1, 1 ],
					"texture": "grass_1024.jpg"
				},

			"detail_weight": {
				"texture": "DetailMap.png",
				"sampler": "unique_name",
				"normalize": true,
				"autogenerate": [
					{
						"slope_min": 0,
						"slope_max": 0,
						"elevation_min": 0,
						"elevation_max": 0
					},
					{
						"slope_min": 0,
						"slope_max": 0,
						"elevation_min": 0,
						"elevation_max": 0
					},
					{
						"slope_min": 0,
						"slope_max": 0,
						"elevation_min": 0,
						"elevation_max": 0
					},
					{
						"slope_min": 0,
						"slope_max": 0,
						"elevation_min": 0,
						"elevation_max": 0
					}
				]
			},

			"detail0": {
				"offset": [ 0, 0 ],
				"scale": [ 128, 128 ],
				"roughness": 1,
				"metalness": 0,
				"diffuse_map": "floor_diffuse.png",
				"normal_map": "floor_bump.PNG"
			},

			"detail1": {
				"offset": [ 0, 0 ],
				"scale": [ 128, 128 ],
				"roughness": 1,
				"metalness": 1,
				"diffuse_map": "floor_diffuse.png",
				"normal_map": "floor_bump.PNG"
			},

			"detail2": {
				"offset": [ 0, 0 ],
				"scale": [ 128, 128 ],
				"roughness": 1,
				"metalness": 1,
				"diffuse_map": "floor_diffuse.png",
				"normal_map": "floor_bump.PNG"
			},

			"detail3": {
				"offset": [ 0, 0 ],
				"scale": [ 128, 128 ],
				"roughness": 1,
				"metalness": 1,
				"diffuse_map": "floor_diffuse.png",
				"normal_map": "floor_bump.PNG"
			}
		}
}
}

On a slightly less related note:
When I attempted to add a variable to the terrain cell's uploadToGpu, I keep breaking the terrain. Which parts do I need to edit to correctly add a "float4 time;" to the cell data. It does not seem as simple as adding it in the 500.Structs_piece_vs_piece_ps.any, adding it in the uploadToGpu, and adding 20 instead of 16 to the currentMappedConstBuffer.. This variable changes every "frame update", so maybe its not the right place to do it, but it seemed like one of the most logical places to do it.

Code: Select all

terrainCell->uploadToGpu( currentMappedConstBuffer );
currentMappedConstBuffer += 16u;

[dark_sylinc]

Hi!

Decals' Normals in Terra's are hard because of how detail normals are calculated by Terra. We also get a catch-22 problem because Terra calculates detail normals in tangent space then converts it to the final normal. And we need that normal to calculate the decals's normals that want to add their own detail normal in tangent space.

It's not impossible to solve but really convoluted; and another problem is that there is no one way to blend in the detail maps. The main reason Terra is provided as a sample and not a component is that nobody agrees in how detail maps should be mixed together; thus everyone wants to customize that part.

If you want to look into it further I strongly recommend you use RenderDoc's source-level shader debugging features for Vulkan and D3D11.

When I attempted to add a variable to the terrain cell's uploadToGpu, I keep breaking the terrain. Which parts do I need to edit to correctly add a "float4 time;" to the cell data.

The terrain is made up of many terrain cells. If you upload the data from TerrainCell::uploadToGpu then you're uploading per cell data. Which doesn't sound like what you want:

Code: Select all

struct CellData
{
	//.x = numVertsPerLine
	//.y = lodLevel
	//.z = vao->getPrimitiveCount() / m_verticesPerLine - 2u
	//.w = skirtY (float)
	uint4 numVertsPerLine;
	int4  xzTexPosBounds;		//XZXZ
	float4 pos;		//.w contains 1.0 / xzTexPosBounds.z
	float4 scale;	//.w contains 1.0 / xzTexPosBounds.w
};
	@property( syntax != metal )
		//Uniforms that change per Item/Entity
		CONST_BUFFER( InstanceBuffer, 2 )
		{
			CellData cellDataArray[256];
		};
	@end

It offers no customization, and if you want to customize it yourself to add more data, you have to lower cellDataArray[256] because otherwise it won't fit into the 64kb limit.

I assume the "time" parameter you want to add is a global time for all terrains; in which case you can use regular PBS custom_passBuffer piece to insert extra data.

If this "time" parameter is per Terrain, then you can add it to the material datablock via custom_materialBuffer (or just use HlmsPbsDatablock::setUserValue!!! you have up to 12 floats worth of arbitrary data reserved for users) and make sure each terrain uses a different material (you can clone them if you have to).

You can see Tutorial_Hlms01_Customization and Tutorial_Hlms02_CustomizationPerObj which are currently living in the ParticleFX2 branch (hasn't been merged into master yet) which show how to do it.

[Ridder Geel]

Ah in this case I will simply look to disable the decal normals, its not like I really need them anyways, but I just thought I'd ask.
Thanks!
As for the time variable, indeed having one time variable for all terrains is more than good enough.
The explanation helps me understand why I was encountering the strange behavior I saw, so thanks again!
I shall look into those examples.