[2.3] HLMS / Cleanest approach to add outline shading for Meshes with Skeletons? Topic is solved

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
Slamy
Gnoblar
Posts: 22
Joined: Sat Mar 27, 2021 10:49 pm
Location: Bochum, Germany
x 10

[2.3] HLMS / Cleanest approach to add outline shading for Meshes with Skeletons?

Post by Slamy »

Hello you all.

This topic might be asked around here previously. But my problem is a little bit different... at least I believe.
I would like to add an outline to objects. I'm aware that this can be performed using a normal offset on vertex positions.

Code: Select all

#version 330 core
#extension GL_ARB_shader_viewport_layer_array : require

vulkan_layout( OGRE_POSITION )	in vec3 position;
vulkan_layout( OGRE_NORMAL )	in vec3 normal;

in int gl_InstanceID;

vulkan( layout( ogre_P0 ) uniform Params { )
	uniform mat4 worldViewProj;
	uniform mat4 worldView;
	uniform mat4 projection;
vulkan( }; )

out vec4 vertexColor; // specify a color output to the fragment shader

void main()
{
	gl_Position = worldViewProj * vec4(position + normal*0.1, 1.0);	
	vertexColor = vec4(0.9, 0.0, 0.0, 1.0);
	gl_ViewportIndex = gl_InstanceID;
}
This piece of code works with static meshes and I'm ok with the results.
But as soon as I would like to use this approach for animated meshes I get into trouble as these are animated using the Vertex Shader of Pbs.

I thought.... well then I just add this small piece of position altering code to a custom piece™ of HLMS. There is even this neatly custom_vs_preTransform right for that purpose. Or is it? I've read through https://ogrecave.github.io/ogre-next/api/2.3/hlms.html and while I do understand the theory behind this, I have problems actually doing something with this information in practice. There is a certain lack of sample code on howto I can add a custom piece for a single material. I've read through the Terrain Pbs code. But It's like spanish.

How do I integrate this outline shader code the cleanest?
Is this custom_vs_preTransform piece in every Pbs based shader? Or can I just put it in one material?
Where can I place a custom piece for a single project. Ressources2.cfg seems to not be used so I can't place it in my own materials folder.

Kind regards,
Slamy
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.3] HLMS / Cleanest approach to add outline shading for Meshes with Skeletons?

Post by dark_sylinc »

Slamy wrote: Thu May 06, 2021 1:51 pm I thought.... well then I just add this small piece of position altering code to a custom piece™ of HLMS. There is even this neatly custom_vs_preTransform right for that purpose.
I can't say it for certain, but looks like it.

As for what you need to do, if it's basic enough with an extra piece file it may be enough.

What you need is to add a folder to Pbs' library containing your customizations.

That's what Terra does to add terrain shadows to all objects (although Terra also uses a C++ listener because it needs to bind a few textures).

What is really important is this bit:

Code: Select all

libraryPbs.push_back( Ogre::ArchiveManager::getSingletonPtr()->load(
                                      rootHlmsFolder + "Hlms/Terra/" + shaderSyntax + "/PbsTerraShadows",
                                      getMediaReadArchiveType(), true ) );
Which points to our customizations.

You can use hlmsPbs->reloadFrom, or better yet include that folder while initializing HlmsPbs for the first time:

Code: Select all

// Create & Register HlmsPbs
// Do the same for HlmsPbs:
Ogre::HlmsPbs::getDefaultPaths( mainFolderPath, libraryFoldersPaths );
Ogre::Archive *archivePbs =
	archiveManager.load( rootHlmsFolder + mainFolderPath, "FileSystem", true );

// Get the library archive(s)
Ogre::ArchiveVec archivePbsLibraryFolders;
libraryFolderPathIt = libraryFoldersPaths.begin();
libraryFolderPathEn = libraryFoldersPaths.end();
while( libraryFolderPathIt != libraryFolderPathEn )
{
	Ogre::Archive *archiveLibrary =
		archiveManager.load( rootHlmsFolder + *libraryFolderPathIt, "FileSystem", true );
	archivePbsLibraryFolders.push_back( archiveLibrary );
	++libraryFolderPathIt;
}

// !!!CUSTOM PATHS HERE!!!
archivePbsLibraryFolders.push_back( archiveManager.load( "custom paths here!!!", "FileSystem", true ) );
// !!!CUSTOM PATHS HERE!!!

// Create and register
hlmsPbs = OGRE_NEW Ogre::HlmsPbs( archivePbs, &archivePbsLibraryFolders );
Ogre::Root::getSingleton().getHlmsManager()->registerHlms( hlmsPbs );
HlmsPbs will enumerate all files in that folder i.e. PbsTerraShadows_piece_vs_piece_ps.glsl, and parse it.

That's where your pieces should be defined.

The Ogre 2.1 FAQ contains a few resources on Hlms customization.

Cheers
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.3] HLMS / Cleanest approach to add outline shading for Meshes with Skeletons?

Post by dark_sylinc »

Btw, an entirely different but better approach to outlines (but intermediate-to-advanced) is to use Ben Golus' approach of using Signed Distance Fields and Jump Flood Algorithm (but it's not fast if you want every single object to have its own outline)
Slamy
Gnoblar
Posts: 22
Joined: Sat Mar 27, 2021 10:49 pm
Location: Bochum, Germany
x 10

Re: [2.3] HLMS / Cleanest approach to add outline shading for Meshes with Skeletons?

Post by Slamy »

dark_sylinc wrote: Fri May 07, 2021 5:34 pm Btw, an entirely different but better approach to outlines (but intermediate-to-advanced) is to use Ben Golus' approach of using Signed Distance Fields and Jump Flood Algorithm (but it's not fast if you want every single object to have its own outline)
I very much appreciate that you provide me some content to read to further improve my skills. But let's take some smaller steps to be more safe. :lol:

I got it working and your post was helpful.
My first working draft is this.

900.OutlineShader_piece_all.any (I chose that name so I need only one file)

Code: Select all

@property( hlms_normal_thicken )
	@undefpiece( DefaultBodyPS )
	@piece( DefaultBodyPS )
		outColour = vec4(0.0, 0.5, 0.0, 1.0);
	@end
@end

@property( hlms_normal_thicken )
	@piece( custom_vs_preTransform )
		@property( hlms_normal || hlms_qtangent )
			worldPos.xyz = worldPos.xyz + worldNorm * 0.01;
		@end
	@end
@end
I've deactivated the default implementation of the pixel shader and altered the vertex shader.

Of course this is nothing for an actual practical use as the color and the outline thickness is hard coded.
My current file looks like this: (For some reason the forum doesn't select the right language syntax highlighting here.)

Code: Select all


@property( hlms_normal_thicken )
	// Pixel Shader Modify. Remove all lighting calculation. Shall only use emissive
	@undefpiece( DoDirectionalLights )
	@undefpiece( DoSpotLights )
	@undefpiece( DoDirectionalShadowMaps )

	// Vertex Shader Modify
	// Diffuse Red Channel defines the width of the outline
	@piece( custom_vs_preTransform )
		@property( hlms_normal || hlms_qtangent )
			ushort materialId	= worldMaterialIdx[inVs_drawId].x & 0x1FFu;
			#define material materialArray[materialId]
			worldPos.xyz = worldPos.xyz + worldNorm * material.kD.x;
		@end
	@end
	
	// Force that the Vertex shader also gets the Material data which is not standard for Pbs
	@piece( custom_vs_uniformStructDeclaration)
		@insertpiece( MaterialStructDecl )
		@property( !(hlms_skeleton || hlms_shadowcaster || hlms_pose) )@insertpiece( InstanceStructDecl )@end
	@end

@end
I've removed most of the pixel shader stuff and use only emissive colors as output.
The vertex shader uses the red channel of the diffuse color to define a factor applied to the normal.

While this solution might not be perfect as it depends on the direction of the normals and therefore the quality of the mesh, it currently is ok for me.

Image

Some example code to use this:

Code: Select all

		Ogre::Hlms* hlmsPbsOutline = hlmsManager->getHlms(Ogre::HLMS_USER0);
		Ogre::HlmsMacroblock macroblock;
		Ogre::HlmsBlendblock blendblock;
		macroblock.mCullMode = Ogre::CULL_ANTICLOCKWISE;
		Ogre::HlmsParamVec param;
		redOutlineDataBlock = static_cast<Ogre::HlmsPbsDatablock*>(
			hlmsPbsOutline->createDatablock("RedOutline", "RedOutline", macroblock, blendblock, param));
		redOutlineDataBlock->setEmissive(Ogre::Vector3(0.0, 1.0, 0));
		redOutlineDataBlock->setDiffuse(Ogre::Vector3(0.01, 0, 0));
		redOutlineDataBlock->setSpecular(Ogre::Vector3(0, 0, 0));

For initialization I just duplicated some lines in GraphicsSystem.cpp:

Code: Select all

{
		// Create & Register HlmsPbsOutline
		HlmsPbsOutline::getDefaultPaths(mainFolderPath, libraryFoldersPaths);
		std::cout << "mainFolderPath " << mainFolderPath << std::endl;

		Ogre::Archive* archivePbs = archiveManager.load(rootHlmsFolder + mainFolderPath, archiveType, true);

		// Get the library archive(s)
		Ogre::ArchiveVec archivePbsLibraryFolders;
		libraryFolderPathIt = libraryFoldersPaths.begin();
		libraryFolderPathEn = libraryFoldersPaths.end();
		while (libraryFolderPathIt != libraryFolderPathEn)
		{
			Ogre::Archive* archiveLibrary =
				archiveManager.load(rootHlmsFolder + *libraryFolderPathIt, archiveType, true);
			std::cout << "archiveLibrary " << archiveLibrary << std::endl;
			std::cout << "libraryFolderPathIt " << *libraryFolderPathIt << std::endl;
			archivePbsLibraryFolders.push_back(archiveLibrary);
			++libraryFolderPathIt;
		}

		{
			Ogre::Archive* archiveLibrary = archiveManager.load("HlmsPbsOutline", archiveType, true);
			archivePbsLibraryFolders.push_back(archiveLibrary);
		}

		// Create and register
		hlmsPbsOutline = OGRE_NEW HlmsPbsOutline(archivePbs, &archivePbsLibraryFolders);
		Ogre::Root::getSingleton().getHlmsManager()->registerHlms(hlmsPbsOutline);
	}

And then this custom class

Code: Select all

/*
 * HlmsPbsOutline.h
 *
 *  Created on: 08.05.2021
 *      Author: andre
 */

#ifndef HLMSPBSOUTLINE_H_
#define HLMSPBSOUTLINE_H_

#include "OgreHlmsPbs.h"

class HlmsPbsOutline : public Ogre::HlmsPbs
{
public:
	HlmsPbsOutline(Ogre::Archive* dataFolder, Ogre::ArchiveVec* libraryFolders) : Ogre::HlmsPbs(dataFolder, libraryFolders)
	{
        mType = Ogre::HLMS_USER0;
        mTypeName = "PbsOutline";
        mTypeNameStr = "PbsOutline";

	}
protected:
	void calculateHashForPreCreate(Ogre::Renderable* renderable, Ogre::PiecesMap* inOutPieces) override
	{
		setProperty("hlms_normal_thicken", 1);

		Ogre::HlmsPbs::calculateHashForPreCreate(renderable, inOutPieces);
	}
};

#endif /* HLMSPBSOUTLINE_H_ */
That's all folks!
Post Reply