[3.0.0] Custom Shaders Topic is solved

[haloman30]

Ogre Version: 3.0.0

Hello - not sure if this is just something I've missed in my searching, but I'm wondering how (or if it's possible with reasonable performance) to implement custom shaders in Ogre-Next.

Through my searching, I've run across a couple approaches - using low-level materials (which supposedly can have negative performance implications for multiple of these), or creating a custom HLMS. As such, I'm pretty sure the HLMS route is where I need to go - however, all the posts and topics I've found seem to not quite be in reference to the kind of system I'm looking for - which is, to allow totally any number of arbitrary pixel/vertex/geometry shaders to be authored by users.

My project is a game engine, with a focus on 3D but also ideally usable for 2D as well, and so users being able to write their own shader code for certain objects to me seems to be a pretty essential feature. It's entirely possible I've overlooked something obvious, but from what I can tell, it seems as though I would need to create an entirely separate HLMS for each user-created shader.

So - is this correct? Or am I mistaken? If not, is there any significant overhead or limits on the number of HLMSes? Or is there some other solution that I've just missed?

As usual, I appreciate any and all info and guidance!

[dark_sylinc]

Hi!

We have an entry in the 2.1+ FAQ regarding custom Hlms implementations.

In the master (4.0) branch we've added tutorials on writing custom Hlms implementations. See Tutorial_Hlms01 through Tutorial_Hlms05.

These tutorials apply to OgreNext 3.0 too. Maybe a virtual function override changed signature parameter, but it should be easy enough to adapt.

Now, onto your question:

however, all the posts and topics I've found seem to not quite be in reference to the kind of system I'm looking for - which is, to allow totally any number of arbitrary pixel/vertex/geometry shaders to be authored by users.

The Hlms is very flexible; so you can define custom pieces per object (i.e. apply custom shader code to a particular object), per material, or per pass.

You can apply different pieces to different objects; effectively creating an arbitrary number of shaders by users (though you'd probably have to write something on top to manage your users' shader code -> map into what goes into which object). See the tutorial.

In the master/4.0 branch we've also added a new per-material setting HlmsDatablock::setCustomPieceFile (with material JSON support keyword "custom_piece_file_vs" and co.) which allows registering a custom piece file per material without C++ additional code.

When customizing existing Hlms implementations you can do things like:

Code: Select all

@undefpiece( DeclareBRDF )
@piece( DeclareBRDF )
 // your lighting code here
@end

To force the PBS component of Hlms to not generate its lighting code and use whatever customization you want.

It seems that you want a shader to start from scratch (rather than customize an existing one). That is also possible.

This is often not recommended because Unlit & Pbs deal with a lot of boilerplate (skeletal animation, shadow casting, etc) so customizing these instead of starting from scratch is usually the easiest path. Specially since you can override this code to do whatever else.

For example Terra from the Terrain Sample completely overrides the vertex & pixel shaders of HlmsPbs while still reusing a lot of individual pieces of code from HlmsPbs.

So - is this correct? Or am I mistaken? If not, is there any significant overhead or limits on the number of HLMSes? Or is there some other solution that I've just missed?

If you mean HLMS types, we support 8 (with LowLevel, Pbs and Unlit taking away 3, so you're left with 5). However for what you want to implement, you'd want to either customize HlmsPbs/HlmsUnlit, or create an extra Hlms type (assuming that you want users to code from scratch) that manages all user's shader code.

This is what HlmsLowLevel essentially does, it looks at what was defined in the older Material system from v1 Ogre and creates the shaders from it. So if the user creates a thousand different shaders using the v1 material system, the HlmsLowLevel will create a thousand shaders.

A single Hlms type can create a virtually unlimited number of shaders.

is there any significant overhead or limits on the number of HLMSes?

Originally the v1 Material system was very slow and unsupported in regular rendering because it was treated as an exception; but those issues were fixed and nowadays the main reason to discourage users from using the v1 Material system for thousands of objects is that its flexibility and ease of use (user-friendly) design goals clash with high performance.

High performance originates from code that can perform assumptions, attempts to treat lots of objects in the same way, and imposes specific limitations on what can be done and what cannot. I'm not talking about OgreNext here, but computers in general.
But "cost" is a relative term here. Applying v1 materials on 100 Items is probably going to be fine on a modern CPU.

But if you apply it on many thousands of them, you won't have the same performance as you'd have with HlmsPbs or Unlit because they have limitations on their interfaces to achieve greater efficiency.

If what you want are shaders from scratch, a custom Hlms type may be what you're looking for; but you may just end up reinventing the v1 Material System (with similar performance pitfalls).

Cheers

[haloman30]

If what you want are shaders from scratch, a custom Hlms type may be what you're looking for; but you may just end up reinventing the v1 Material System (with similar performance pitfalls).

Hey!

Yes, from-scratch shaders is indeed what I'm primarily looking into. The idea of extending PBS and Unlit sounds interesting too, however I do feel it's important regardless to include the ability for new shaders entirely from scratch to be created.

I did some looking into using low-level materials, however I'm getting some issues trying to manually create them (as I'm wanting to avoid relying on a bunch of material scripts since all engine content is in it's own format). Couldn't find much useful on the forum or elsewhere - and tried creating them both using HLMSLowLevel and the v1 MaterialManager.

Creating a HLMSLowLevelDatablock seemingly doesn't populate the proxy material, and creating it via MaterialManager results in a "not supported" error message and no valid techniques being available:

Code: Select all

WARNING: material shaders/solid.shader has no supportable Techniques and will be blank. Explanation:  Pass 0: Vertex program shaders/solid.shader_vtx cannot be used - not supported.

I did find a post suggesting that the issue might be due to the syntax code not being set properly, and even checking supported values (ps_5_0 and vs_5_0 in my case, also tried just 'hlsl' as it also showed up in the list), no luck. I also noticed the following message when creating the material:

Code: Select all

Material shaders/solid.shader was requested with isManual=true, but this is not applicable for materials; the flag has been reset to false

Which has me wondering - is manual creation of low level materials actually possible? Is there some step I'm missing for the program/material creation, or is this a case where I'd need to still create a custom HLMS to achieve this? For reference, the current approach I'm using for creating materials is as follows:

Code: Select all

// tag_path is a std::string with the shader file path
// BLAM_OGRE_RESOURCE_GROUP is a preprocessor macro storing the resource group used for custom datablocks, meshes, etc

Ogre::MaterialManager* material_manager = Ogre::MaterialManager::getSingletonPtr();
material = material_manager->create(tag_path, BLAM_OGRE_RESOURCE_GROUP, true);

Ogre::HighLevelGpuProgramManager* manager = Ogre::HighLevelGpuProgramManager::getSingletonPtr();

if (vertex_program_code.length() > 0)
{
	Ogre::HighLevelGpuProgramPtr program = manager->createProgram(tag_path + "_vtx", BLAM_OGRE_RESOURCE_GROUP,
		"hlsl", Ogre::GpuProgramType::GPT_VERTEX_PROGRAM);

program->setSource(vertex_program_code);
program->setSyntaxCode("hlsl");
program->createParameters();
program->load();
material->getTechnique(0)->getPass(0)->setVertexProgram(program->getName());
}

material->load();

[haloman30]

Hello!

Been a bit as I've been busy with some other stuff - but coming back to it later with fresh eyes, I'm pretty sure I've almost got this working, but running into one last roadblock.

I've gone ahead and gone down the path of using HLMS customizations - currently, using the 4.0 branch and the per-datablock piece files (using setCustomPieceFile and providing code directly within that function rather than using external files).

I'm using Tutorial_Hlms05_CustomizationPerObjData as a reference (and for now just copying its HlmsPbs implementation as-is and using its piece files via the datablocks as mentioned before), and I've confirmed that the per-renderable parameters are getting through, but it seems that when the shaders are created, the use_arbitrary_colour property isn't getting set. Comparing the call stacks in my project vs the tutorial, it looks like calculateHashForPreCreate in my project is only called during item creation, whereas in the tutorial, this is also called again during render as well.

Since during creation, the parameter isn't set yet, the use_arbitrary_colour property never gets set, and so the custom shader code never gets included. Removing the check for that parameter gets it a bit further, and the parameter even works as expected - but then placing a light causes a crash due to the constant buffer ID not getting set.

Is this to do with me setting the custom piece files in the datablock, rather than just having them be part of the HLMS itself? If so, is there by chance some additional step I need to do?

As usual, appreciate any info or pointers!

[dark_sylinc]

Hi!

Ideally you need to call setCustomParameter( kColourId, ... ); before setting the material.

But if that has already happened you can do:

Code: Select all

HlmsDatablock *datablock = renderable->getDatablock();
renderable->_setNullDatablock();
datablock->setDatablock( datablock );

Sometimes it works alright regardless because if the datablock has textures and OgreNext isn't done loading the texture yet, it needs to postpone calculateHashForPreCreate() for later, and by that time setCustomParameter() has already been called. This issue has been raised in another thread and it is indeed a flaw as it is not obvious and could be made to be more robust.

[haloman30]

Hello!

Was going to stick with the former route of HLMS customizations - but due to troubles with error handling, among other things, I went back and dug further into using low-level materials, since that in hindsight seemed a bit more direct.

I have gotten it almost working exactly as I need - the trouble I'm running into is how to properly pass uniforms to them.

I've scoured the forums and elsewhere to try and narrow things down, even looking a bit into Ogre 1 and seeing if there's something I'm missing, and as far as I can tell - and I'm coming up empty. The shader itself compiles and runs without issue - it's just uniforms that aren't working as desired.

Currently, I'm using D3D11, and my vertex/pixel shaders are as follows (trying to pass the uniform into the pixel shader):

Vertex shader (hlsl, vs_4_0):

Code: Select all

float4x4 worldViewProj;

struct VS_INPUT {
    float4 position : POSITION;
    float3 uv : TEXCOORD0;
};

struct VS_OUTPUT {
    float4 position : SV_POSITION;
    float3 uv : TEXCOORD0;
};

VS_OUTPUT main(VS_INPUT input) {
    VS_OUTPUT output;
    output.position = mul(worldViewProj, input.position);
    output.uv = input.uv;
    return output;
}

Pixel shader (hlsl, ps_4_0):

Code: Select all

struct VS_OUTPUT {
    float4 position : SV_POSITION;
    float3 uv : TEXCOORD0;
};

float4 main(VS_OUTPUT input, uniform float4 color_uniform) : SV_Target {
    return color_uniform;
}

I am setting the parameter using setNamedConstant() immediately after creation and compilation of the shader - is there some other point where I should be calling this? Or perhaps a different method I've managed to miss?

One other thing to note, I am creating these materials entirely programatically - not using any material scripts for these - not sure if there's a step I'm maybe missing there for updating parameters besides setNamedConstant?

Any pointers or help is greatly appreciated!

[rpgplayerrobin]

I am pretty certain that you need to create the variable for the shader material as well.

Look at a minimal shader like this one:

Test_VS.hlsl:

Code: Select all

float4x4 modelViewProj;

void main_vs( float4 position : POSITION,
			  out float4 oPosition : POSITION )
{
	oPosition = mul(modelViewProj, position);
}

Test_PS.hlsl:

Code: Select all

float4 specific_color; // This is much easier to handle for many variables than your code does in the main function instead

float4 main_ps() : COLOR0
{
	return specific_color;
}

And their shader materials:

Code: Select all

vertex_program Test_VS hlsl
{
	source Test_VS.hlsl
	entry_point main_vs
	target vs_5_0 // Also note 5 instead of 4, for D3D11 instead of D3D10

default_params
{
	param_named_auto modelViewProj worldviewproj_matrix
}
}

fragment_program Test_PS hlsl
{
	source Test_PS.hlsl
	entry_point main_ps
	target ps_5_0 // Also note 5 instead of 4, for D3D11 instead of D3D10

default_params
{
	param_named specific_color float4 1.0 1.0 1.0 1.0
}
}

After this, you can easily set its value with this code:

Code: Select all

// Check if the pass has a shader
if (pass->hasFragmentProgram())
{
	// Check first if the named constant exists on the pass
	GpuProgramParametersSharedPtr tmpGPUParam = pass->getFragmentProgramParameters();
	CString tmpConstant = "specific_color";
	if (tmpGPUParam->_findNamedConstantDefinition(tmpConstant, false))
		// Set the colour of the pass
		tmpGPUParam->setNamedConstant(tmpConstant, Vector4(colour.r, colour.g, colour.b, colour.a));
}

[haloman30]

And their shader materials:

Code: Select all

vertex_program Test_VS hlsl
{
	source Test_VS.hlsl
	entry_point main_vs
	target vs_5_0 // Also note 5 instead of 4, for D3D11 instead of D3D10

default_params
{
	param_named_auto modelViewProj worldviewproj_matrix
}
}

fragment_program Test_PS hlsl
{
	source Test_PS.hlsl
	entry_point main_ps
	target ps_5_0 // Also note 5 instead of 4, for D3D11 instead of D3D10

default_params
{
	param_named specific_color float4 1.0 1.0 1.0 1.0
}
}

Well, thing is - I'm doing this all programatically. There is no material script, I'm creating and loading the programs directly in code. My engine has a proprietary format for all content, and my goal is to have everything use this system eventually. I did some poking around to see if there was some obvious way to add default parameters, trying to find perhaps how the actual material scripts themselves get loaded and replicate that manually, but haven't yet been able to find anything obvious.

EDIT

I actually checked when debugging - it does seem that the program's constant definitions and default parameters are getting populated - curiously, my parameter appears twice, once as color_uniform and then again as color_uniform[0]. I did try and see if setting color_uniform[0] would get a result, but no such luck still.

EDIT 2

Found the issue - turns out, it was actually working all along - there was some old testing code elsewhere that I entirely forgot about from a while back, that was overriding it after I had set it the first time.

Thanks for the help either way, though!