Struggling with HLMS

[Frankincense]

Hi all,

I recently started porting from 1.10 to 2.1 as I required better control over the default scene compositor and I use a lot of instancing so hopefully can benefit from the performance improvements too. I am now at the point where I need to convert all of my materials/shaders to HLMS. I have been able to convert v1 entities to Items and shown a basic (incorrectly lit) scene using the basic unlit HLMS.

The main shader I used previously was a 'jumbo shader' with many defines to control what was performed, with a matching C++ file to generate the materials and set the defines. The shader has instancing, custom lighting implementation, custom texturing and a bunch of other features I need to port to the new system - the shading is stylised and not 'photo-real' at all.

My initial thinking was to define my own HLMS, update my existing shaders to work with it and go from there. Unfortunately I have had many issues trying this due to not having many examples to work with, little documentation on creating a new HLMS and the Unlit/PBS examples being extremely complex for someone new to HLMS. I have also found that most forum links for help with HLMS are a number of years old and require at least some tweaks to get working. Reading through the links on viewtopic.php?p=524412#p524412 was somewhat helpful but didn't answer some of the more basic questions I have had (mostly about the purpose of each HLMS class & function!)

I used the basic example from here: viewtopic.php?p=519340#p519340 to just get something on-screen, which initially worked, but then trying to add my own textures I found that either the UVs were missing, or the textures weren't loaded correctly - not really sure. Using the Unlit as reference didn't really help as its not really clear why each function/code block is doing what it is, so I would just be guessing as to what needs to change.

I then thought that maybe I should just customise the PBS setup and remove any 'photo-real' lighting or other sections that I don't need. I got a basic 'material' with texturing working by passing the "diffuse_map" parameter to the datablock but the scene lighting is completely wrong, default fresnel and roughness (I understand PBS works best with HDR which I am not interested in). I have tried the "sRGB Gamma Conversion" setting which caused everything to be washed-out, I then looked at some of the samples and found:

setPowerScale ( Ogre::Math::PI ) ; // Since we don't do HDR, counter the PBS' division by PI

But that doesn't make the scene bright enough or make the lighting look any more correct. My scene uses a single 'sun' directional light, all of the Items in the scene have a diffuse texture, default compositor and default camera.


-I there something I am missing with using PBS (with a non-HDR scene) to get lighting that looks anything close to what I had under 1.10?
-Is using PBS as a starting-point for non-photo-real shading a false start and should I instead start with Unlit?
-Are there any examples of a non-photo-real HLMS setup that I can use a reference (preferably with a diffuse texture, basic lighting, etc.)?
-Is the performance benefit of HLMS really worth me learning its complexity, or should I just stick to the old style materials if I am porting from 1.10? (I require heavy use of instancing and I am not currently sure if Item's instancing would work with with the old-style materials)
-Can you change the shading type for PBS?
-Can I remove the unused PBS shader template sections without having to also modify the C++ classes?
-Do most people use the default shading and shadow mapping? Or implement their own?


This post is obviously quite long and sounds very negative, but 1.X materials made sense after a day of working with them, HLMS seems to be a bit out-of-reach for now - I appreciate that most of what HLMS is doing was probably being abstracted in 1.10 by the 'material/program/technique' system and now it's being exposed at a lower level, but for someone just trying to get a game ported to 2.10 its been very difficult so far! I also have a custom shadow-map implementation that I am quite scared of trying to port!


Any help would be much appreciated,
Frank

[Frankincense]

Maybe what is needed is a guide for adding a diffuse texture (and UVs) to the example here viewtopic.php?p=519340#p519340 and hopefully that will help understanding what the Unlit/PBS classes are doing and how to properly add new functionality to a HLMS. That way we can start to build a step-by-step for how HLMS's are put together?

[Frankincense]

For anyone looking at this, I have been looking to get PBS to look similar to my previous setup before modifying it and I was able to use this to setup the legacy lighting:

Code: Select all

      Ogre::HlmsBlendblock blendblock ;
      Ogre::HlmsMacroblock macroblock ;
      Ogre::HlmsParamVec params ;

      params.push_back ( std::make_pair ( "legacy_math_brdf", "1" ) ) ;
      params.push_back ( std::make_pair ( "diffuse_map", texture_name ) ) ;

      auto *datablock = static_cast <Ogre::HlmsPbsDatablock*> ( hlms->createDatablock ( material_name, material_name, macroblock, blendblock, params ) ) ;
      datablock->setBrdf( Ogre::PbsBrdf::BlinnPhongFullLegacy ) ;
      datablock->setSpecular( Ogre::Vector3::ZERO ) ;

With the above, the RGB gamma conversion can be disabled:

Code: Select all

Root->getRenderSystem ()->setConfigOption ( "sRGB Gamma Conversion", "No" ) ;

And the lighting can be a default power level:

Code: Select all

TheSun->setPowerScale ( 1.0f ) ;

I will now look to copy the PBS HLMS classes and remove all of the parts I am not interested in, then I should be able to add my own lighting implementation to replace the PBS Brdf


Edit: After making a full copy of the PBS classes and media files I then found that there was another small line that I had to modify to get the lighting looking much closer to what I previously had with my custom shaders.
In this block in the PBS PixelShader_ps.glsl:

Code: Select all

	@property( !hw_gamma_write )
		//Linear to Gamma space
		outColour.xyz	= sqrt( finalColour );
	@end @property( hw_gamma_write )
		outColour.xyz	= finalColour;
	@end

It was still converting linear to gamma space colours, so in the HlmsPBS I set 'HwGammeWrite' to OFF (0) on this line:

Code: Select all

setProperty( PbsProperty::HwGammaWrite, 0 );

[Frankincense]

Now I have the default PBS looking close to what I had previously the next step is to copy all of the PBS classes (OgreHlmsPbs*.*), rename it to my custom shading and also copy the media in '/Samples/Media/Hlms/Pbs' to a new folder with the same name.

In the copied vertex and fragment shaders I then commented out all of the sections I am not interested in such as SSR (screen-space reflections), Dual Paraboloid Mapping, Tangents, normal maps, all of the non-diffuse texture sampling, parallax cubemaps, envprobes, etc. I found that a few of the sections (e.g. @foreach) didn't like being commented out and had to be completely removed to get it to work.
After commenting out each section I just re-ran to ensure that the shader still worked.

After getting a working cut-down shader I then did the same on the C++ side, commenting out all of the copied OgreHlmsPbs code that I didn't need from above.


There are still a lot of the copied Pbs class stuff that I don't understand and unfortunately a number of the functions don't have any description comments. But hopefully I can add/remove/copy everything I need to fully customise it for my own shading setup.

[dark_sylinc]

There are still a lot of the copied Pbs class stuff that I don't understand and unfortunately a number of the functions don't have any description comments. But hopefully I can add/remove/copy everything I need to fully customise it for my own shading setup.

If you post them here perhaps I can clarify some of them and benefit everyone
Don't be afraid to ask.

[Frankincense]

If you post them here perhaps I can clarify some of them and benefit everyone
Don't be afraid to ask.

Certainly! Hopefully these questions make sense

General:

1. Current HLMS implementations use a large buffer of data with an index to perform the automatic instancing, should a const buffer always be used? If not, when should it not be used?
2. What are the shader buffer commands (CbShaderBuffer) and do they relate to anything in the old material system?
3. I assume 'setProperty' would be equivalent to setting define flags when building a shader?
4. How are textures 'baked' and what purpose does this serve?

Datablocks/hashing:

1. Does each datablock generate a hash so there is only one copy if all of the data is the same?
2. How do the 'calculateHash' functions actually calculate the hash? It seems that they mostly just set properties in the shaders?
3. What should be inside 'calculateHashForPreCreate' vs 'calculateHashForPreCaster'?
4. What does 'PreCreate' and 'PreCaster' mean in this context?
5. Are these functions called only once per object being rendered, or once per unique datablock on creation?
6. Why are some properties set in both 'calculateHashForPreCreate' and 'calculateHashForPreCaster'? How should I decided which one to put my properties in?
7. In 'HlmsPbs::calculateHasForPreCaster' a number of properties are removed from mSetProperties, how are these chosen and what's the effect?
8. What is done in 'createShaderCacheEntry' and what should go in it vs the 'calculateHashX' functions?
9. When is 'preparePassHash' called? I assume its setting all of the data to be passed to the shader each time a shader pass is rendered (e.g. main pass + shadow caster)?
10. Are all of the 'hash' functions called before rendering and set 'static' data, and 'preparePassHash' is called every frame to pass the dynamic data?
11. Does this affectively replace the old 'automatic' material properties to pass this data?
12. In the Datablock 'setter' functions, when should I call 'scheduleConstBufferUpdate' vs 'flushRenderables'?
13. Are the macro/sampler blocks part of the hash generated?

That's plenty questions for now

[dark_sylinc]

Current HLMS implementations use a large buffer of data with an index to perform the automatic instancing, should a const buffer always be used? If not, when should it not be used?

This varies per GPU, but the general rule is that const buffers are better if the index (e.g. the value of 'i' in data[ i]) is the same for a lot (best case: all) of the vertices or pixels while the other buffers are better if the index is highly dynamic (e.g. 'i' can be different for every vertex/pixel)

Also if you need far more than 64kb, then const buffers are not an option.
You can read more from my post in my personal website in UBO vs TBO (UBO = ConstBufferPacked, TBO = TexBufferPacked)

[*]What are the shader buffer commands (CbShaderBuffer) and do they relate to anything in the old material system?

They just bind from C++ the buffer to a specific slot so that it can be available for shaders using that slot.
The type of buffer the shader expects must match what C++ sets (e.g. if shader expects a texture buffer at slot 3, then C++ must set a TexBufferPacked at slot 3)

No, there is no equivalent in the old material system

[*]I assume 'setProperty' would be equivalent to setting define flags when building a shader?

Yes.

[*]How are textures 'baked' and what purpose does this serve?

We try to group together as many textures as possible. We use texture2DArrays for that purpose because they're widely supported in HW. The main restriction is that textures that can be grouped together must have the same format and resolution.

The reason for grouping is performance. Rather than doing:

Code: Select all

for each object
{
   setTextures( textures[i], num_textures[i] );
   draw( object[i].vertexCount );
}

We perform:

Code: Select all

for each group
{
   setTextures( group[i].texturesArray );
   for each object in group
       draw( group[i].object[j].vertexCount );
}

This way we call setTextures() far less frequently (the call instruction isn't that expensive; but there is a lot of work that needs to be done on CPU and GPU sides when swapping too frequently).

[*]Does each datablock generate a hash so there is only one copy if all of the data is the same?

Yes-ish... and no.

This is explained in detail the manual:

There are two components that needs to be evaluated that may affect the shader itself and would need to be recompiled.

1. The Datablock/Material. Does it have Normal maps? Then include code to sample the normal map and affect the lighting calculations. Does it have a diffuse map? If not, avoid sampling the diffuse map and multiplying it against the diffuse colour, etc.
2. The Mesh. Is it skeletally animated? Then include skeletal animation code. How many blend weights? Modify the skeletal animation code appropiately. It doesn't have tangents? Then skip the normal map defined in the material. And so on.

When calling Renderable::setDatablock(), what happens is that Hlms::calculateHashFor will get called and this function evaluates both the mesh and datablock compatibility. If they're incompatible (i.e. the Datablock or the Hlms implementation requires the mesh to have certain feature. e.g. the Datablock needs 2 UV sets bu the mesh only has one set of UVs) it throws.

If they're compatible, all the variables (aka properties) and pieces are generated and cached in a structure (mRenderableCache) with a hash key to this cache entry. If a different pair of datablock-mesh ends up having the same properties and pieces, they will get the same hash (and share the same shader).

In short if two datablocks are identical AND the mesh properties are identical (e.g. vertex format, animation), then the hash will be the same.

This also means that the same datablock applied to two different meshes may have the same hash, or different hashes.

[*]How do the 'calculateHash' functions actually calculate the hash? It seems that they mostly just set properties in the shaders?

The same section of the manual explains it, but it basically analyzes data required (e.g. does it use normal maps?) analyzes the mesh structure (does it use float3 for position? or does it use half4? does it have normals?) and sets the properties.
The hash is calculated on properties set.

[*]What should be inside 'calculateHashForPreCreate' vs 'calculateHashForPreCaster'?

Whatever the derived implementation needs to set. For example HlmsPbs checks if the material needs normal mapping there and if so, sets the properties; then the shader uses this property to add shader code to deal with normal mapping.
HlmsPbs also checks one by one which textures are needed.

The HlmsUnlit implementation doesn't have normal mapping so it does not have such code.

[*]What does 'PreCreate' and 'PreCaster' mean in this context?

Each mesh-material combo needs mostly (at least) two shaders: the main one used for rendering; and the one used during shadow casting.

The shadow casting shader tends to be extremely simple, but it may need to still account for skeletal animation and if alpha tested shadows are required (e.g. foliage shadows) then the caster needs to account some textures to read the alpha.

[*]Why are some properties set in both 'calculateHashForPreCreate' and 'calculateHashForPreCaster'? How should I decided which one to put my properties in?

This was answered by the previous question: former is what's needed for main rendering, latter is only what's necessary for shadow casting.

[*]Are these functions called only once per object being rendered, or once per unique datablock on creation?

When the material is assigned to the Object. Changing a material to the object casues this function to be called again (which may end up reusing an existing hash in the cache, or creating a new one).

The idea is to move most work to creation time (although there is a final step that inevitably happens at runtime every frame, per object: Hlms::getMaterial gets called and if the two hashes haven't been merged into the final hash, then a new shader is generated; else a shader in a cache is provided)

[*]In 'HlmsPbs::calculateHasForPreCaster' a number of properties are removed from mSetProperties, how are these chosen and what's the effect?

Because the caster version is usually a heavily watered-down version; it's easier to remove unnecessary properties rather than evaluate them again.

Technically we could leave all properties set and add an additional that says "is_caster = true" (which we do, btw) so that the template follows a different code path.

However this prevents shader reuse. Two material-object combos may end up with exactly the same shader; however if their properties set are different (even if they're ignored) they'll get different hashes, and if they get different hashes, they'll be treated as different shaders (which hurts performance). That's why we prefer stripping down properties as much as possible

[*]What is done in 'createShaderCacheEntry' and what should go in it vs the 'calculateHashX' functions?

calculateHashX gets called while the shader hasn't even been generated. And properties set this way will be part of the hash.

The function createShaderCacheEntry creates the actual shader; and setting properties here won't affect the hash; which is usually a bad thing (because two different shaders will be seen as the same shader).

[*]When is 'preparePassHash' called? I assume its setting all of the data to be passed to the shader each time a shader pass is rendered (e.g. main pass + shadow caster)?

Also explained in the manual, preparePassHash gets called per pass to set properties that are global to all mesh and materials for that pass.

This means there are 3 hashes:

1. Material-mesh pair combo
2. Pass
3. The final hash, which combines the former two

[*]Does this affectively replace the old 'automatic' material properties to pass this data?

I don't know what are the "old 'automatic' material properties"

[*]In the Datablock 'setter' functions, when should I call 'scheduleConstBufferUpdate' vs 'flushRenderables'?

flushRenderables: Whenever the shader must change. For example the material uses diffuse textures, and now you remove them. This requires the shader to change (a property needs to change its value, or be set/unset). Hence flushRenderables will force all objects using this material to calculate their hashes again (it's basically like unsetting the datablock and setting it again for every object to force a rebuild).

scheduleConstBufferUpdate: when material data living in GPU memory needs to change; without requiring the shader to be recompiled. For example if you set the diffuse colour from blue (0, 0, 1) to red (1, 0, 0); the GPU memory needs to be altered with this change.
It's essentially memcpy( gpu_memory, cpu_memory ); with the new parameters.

[*]Are the macro/sampler blocks part of the hash generated?

Macro & Blendblocks: Yes. The reason is more technical, has to do with how PSOs in modern APIs (Vulkan, Metal & D3D12) work (see we use mLifetimeId).
Sometimes the reason is less technical. For example Hlms checks the blendblock->isAutoTransparent() to see if the shader should be aware that alpha blending is required.

Samplerblocks: Yes, but only the amount of sampleblocks matters (because it results in a different shader)

[Frankincense]

Awesome thanks, that has cleared up a bunch of things - hopefully it will be useful for others too.


Note, if anyone is adding more data to the Datablock and added a new 'setProperty' but for some reason it correctly enabling the '@property( NewProperty )' in your shader for every object - have a look at your Datablock::cloneImpl, you may have missed adding it to the list of data to copy when cloning.

[Frankincense]

I am trying to add a new shader parameter that is the same for all objects, but only used by some of them (an example would be Time). I initially thought to add it to the PassBuffer and then set the data in 'preparePassHash' - but I don't think this will work as its not enabled for every object in the pass (enabled via setProperty for each datablock).

Where should data (like Time) that is the same value for every object, but only conditionally enabled, go in the HLMS?

[dark_sylinc]

The variable should be added to the pass buffer, populated in preparePassHash; and those materials that don't use it will ignore this value (the passbuffer structure defined in the shader should always contain this value, but they would just not use it).

And in calculateHashForPreCreate you set a property to indicate this item-material combo should use (or ignore) that value.

[Frankincense]

The variable should be added to the pass buffer, populated in preparePassHash; and those materials that don't use it will ignore this value (the passbuffer structure defined in the shader should always contain this value, but they would just not use it).

And in calculateHashForPreCreate you set a property to indicate this item-material combo should use (or ignore) that value.

Got it, that's how I was maybe thinking it should be done. I assume that the performance impact of adding a few extra bytes to the PassBuffer is pretty negligible?


Are the datablock material properties part of the hash at all? They don't look to be used in 'calculateHash'.
Wouldn't that mean that changing the diffuse colour for one datablock, for example, would change it for every renderable using that datablock, rather than creating a new datablock?
I would have assumed that the material properties would be part of what makes that datablock unique.

[dark_sylinc]

I assume that the performance impact of adding a few extra bytes to the PassBuffer is pretty negligible?

Indeed.

Are the datablock material properties part of the hash at all? They don't look to be used in 'calculateHash'.
Wouldn't that mean that changing the diffuse colour for one datablock, for example, would change it for every renderable using that datablock, rather than creating a new datablock?
I would have assumed that the material properties would be part of what makes that datablock unique.

Depends:

• Does the material use diffuse colour? -> likely to be turned into a property (unless it's rare to not use a diffuse colour)
• What is the colour is the diffuse? -> not turned into parameter (i.e. that's the difference between scheduleConstBufferUpdate & flushRenderables). You can turn it into a property you want, but that is ill-advised (every single different value will be turned into a new shader using the diffuse colour as a hardcoded value)

Btw are you aware of RenderDoc? If not, download it now. It's a must have for debugging shaders; and will let you understand how our shaders work much faster.

[Frankincense]

Depends:

• Does the material use diffuse colour? -> likely to be turned into a property (unless it's rare to not use a diffuse colour)
• What is the colour is the diffuse? -> not turned into parameter (i.e. that's the difference between scheduleConstBufferUpdate & flushRenderables). You can turn it into a property you want, but that is ill-advised (every single different value will be turned into a new shader using the diffuse colour as a hardcoded value)

So if I am reading that correctly, for the PBS implementation it has been assumed that every object with the same datablock is assumed to also have all the same material properties?
That would mean that setting different material properties (keeping everything else like diffuse/normal maps, etc, the same) would not cause a new datablock to be created, and therefore all of the objects using that datablock would use the material properties that were set last, is that correct?

Btw are you aware of RenderDoc? If not, download it now. It's a must have for debugging shaders; and will let you understand how our shaders work much faster.

I haven't seen that before no, and I have needed a good application for debugging shaders for a while - thanks!