Tutorial on how to pass additional textures to HLMS PBS

[aymar]

Recently I needed to pass additional textures to the default PBS implementation, and I ran into a few barriers until I finally sorted everything out.
So, I decided to write a simple step by step tutorial on how to pass additional textures to the default PBS implementation (probably this is also useful when passing your custom textures to your custom HLMS also).

So, for the sake of the tutorial, let's say you want to pass an additional diffuse map to the PBS implementation, for instance, to render a few decals over your mesh using the second uv channel.

So, what are we going to do in this tutorial is:

• Add a new diffuse map called "decal_map".
• Configure it the same way a diffuse map is configured.
• Set it to use the second uv map.
• Slithgly change the HLMS implementation to actually render this decal map over the diffuse one.

The first thing we have to do is the C++ setup.

First in the OgreHlmsPbsPrerequisites.h you'll find this enum, let's add here the entry for our decal map, right after the diffuse one, it is important to remember the position where you entered your custom map.

Code: Select all

enum PbsTextureTypes
    {
        PBSM_DIFFUSE,
        [b]PBSM_DECAL,[/b]
        PBSM_NORMAL,
        PBSM_SPECULAR,
        PBSM_METALLIC = PBSM_SPECULAR,
        PBSM_ROUGHNESS,
        PBSM_DETAIL_WEIGHT,
        PBSM_DETAIL0,
        PBSM_DETAIL1,
        PBSM_DETAIL2,
        PBSM_DETAIL3,
        PBSM_DETAIL0_NM,
        PBSM_DETAIL1_NM,
        PBSM_DETAIL2_NM,
        PBSM_DETAIL3_NM,
        PBSM_REFLECTION,
        NUM_PBSM_SOURCES = PBSM_REFLECTION,
        NUM_PBSM_TEXTURE_TYPES
    };

That's done!

Now to the OgreHlmsPbs.h, we have to add the IdString used to locate the map in the script and to set it to the shaders. Just add a IdString DecalMap and a IdString UvDecal in the attributes of the struct PbsProperty, I won't paste the code here because it's really pretty straightforward and it doesn't matter where you put it as long as you put it IN the struct PbsProperty.

Now the implementation (OgreHlmsPbs.cpp). Here you define how the hlms script will find the decal map and send it to the shader.

Step 1: Define the actual strings for DecalMap and UvDecal previously declared.

Code: Select all

const IdString PbsProperty::DecalMap = IdString( "decal_map" );
const IdString PbsProperty::UvDecal = IdString( "uv_decal" );

If you note the change we did in the OgreHlmsPbsPrerequisites, you'll notice that we changed the value of NUM_PBSM_SOURCES and NUM_PBSM_TEXTURE_TYPES when we added our entry there, and both changes will affect the whole implementation and need to be accounted for, the next step fixes one point that needs fixing after this change.

Step 2: Setup the uv source for the decal map, you need to put it in the correct location (right after the diffuse map):

Code: Select all

const IdString *PbsProperty::UvSourcePtrs[NUM_PBSM_SOURCES] =
    {
        &PbsProperty::UvDiffuse,
        [b]&PbsProperty::UvDecal,[/b]
        &PbsProperty::UvNormal,
        &PbsProperty::UvSpecular,
        &PbsProperty::UvRoughness,
        &PbsProperty::UvDetailWeight,
        &PbsProperty::UvDetail0,
        &PbsProperty::UvDetail1,
        &PbsProperty::UvDetail2,
        &PbsProperty::UvDetail3,
        &PbsProperty::UvDetailNm0,
        &PbsProperty::UvDetailNm1,
        &PbsProperty::UvDetailNm2,
        &PbsProperty::UvDetailNm3
    };

Step 3: Customize the hash function (calculateHashForPreCreate):

Code: Select all

        setTextureProperty( PbsProperty::DiffuseMap,    datablock,  PBSM_DIFFUSE );
        [b]setTextureProperty( PbsProperty::DecalMap, datablock, PBSM_DECAL );[/b]
        setTextureProperty( PbsProperty::NormalMapTex,  datablock,  PBSM_NORMAL );
        setTextureProperty( PbsProperty::SpecularMap,   datablock,  PBSM_SPECULAR );
        setTextureProperty( PbsProperty::RoughnessMap,  datablock,  PBSM_ROUGHNESS );
        setTextureProperty( PbsProperty::EnvProbeMap,   datablock,  PBSM_REFLECTION );
        setTextureProperty( PbsProperty::DetailWeightMap,datablock, PBSM_DETAIL_WEIGHT );

Step 4: Customize the hash function for the caster pass (Optional).

This finishes the HlmsPbs changes.

Now for the datablock. This bit is a little more complicated, because we'll work simultaneously with 3 blocks of code, and they absolutely NEED to be in sync. The 3 portions are:
C++ Material : In OgreHlmsPbsDatablock.h

Code: Select all

        float   mkDr, mkDg, mkDb;                   //kD
        float   _padding0;
        float   mkSr, mkSg, mkSb;                   //kS
        float   mRoughness;
        float   mFresnelR, mFresnelG, mFresnelB;    //F0
        float   mTransparencyValue;
        float   mDetailNormalWeight[4];
        float   mDetailWeight[4];
        Vector4 mDetailsOffsetScale[8];
        uint16  mTexIndices[NUM_PBSM_TEXTURE_TYPES];
        float   mNormalMapWeight;

GPU Material : In Structs_piece_vs_piece_ps.glsl (and of course, hlsl too, but for the sake of simplicity, let's work on the GLSL for now):

Code: Select all

struct Material
{
	vec4 kD; //kD.w is alpha_test_threshold
	vec4 kS; //kS.w is roughness
	//Fresnel coefficient, may be per colour component (vec3) or scalar (float)
	//F0.w is transparency
	vec4 F0;
	vec4 normalWeights;
	vec4 cDetailWeights;
	vec4 detailOffsetScaleD[4];
	vec4 detailOffsetScaleN[4];

	uvec4 indices0_3;
	//uintBitsToFloat( indices4_7.w ) contains mNormalMapWeight.
	uvec4 indices4_7;
};

And Material Size : In OgreHlmsPbsDatablock.cpp

Code: Select all

const size_t HlmsPbsDatablock::MaterialSizeInGpu = 52 * 4 + NUM_PBSM_TEXTURE_TYPES * 2 + 4;

Please, take a moment to notice that in the default implementation all 3 are in perfect sync. Because in the default implementation NUM_PBSM_TEXTURE_TYPES is 14, meaning it'll pass 14 indices to the GPU. Now please, notice that the GPU Material is a GPU representation of the C++ Material. And the MaterialSizeInGpu used in this line appropriately tells what should be sent to the GPU:

Code: Select all

memcpy( dstPtr, &mkDr, MaterialSizeInGpu );

So if you count it right, from mkDr, counting MaterialSizeInGpu bytes, it is sending exactly that whole C++ Material to the GPU Material. Am I clear at this point? Please, let me know...

So in one side we have the C++ Material, in the other side the GPU Material, and the glue between them in MaterialSizeInGpu.

What will be the result then? Let's explore what each GPU variable will be storing after the C++ send it's variables over:
kD.xyzw <- mkDr, mkDg, mkDb, _padding0;
kS.xyzw <- mkSr, mkSg, mkSd, mRoughness;
F0.xyzw <- mFresnelR, mFresnelG, mFresnelB, mTransparencyValue;
normalWeights.xyzw <- mDetailNormalWeight[0, 1, 2, 3];
cDetailWeights.xyzw <- mDetailWeight[0, 1, 2, 3];
detailOffsetScaleD[0, 1, 2, 3].xyzw and detailOffsetScaleN[0, 1, 2, 3].xyzw <- mDetailsOffsetScale[0, 1, 2, 3, 4, 5, 6, 7].xyzw;
indices0_3.x <- mTexIndices[0] concat mTexIndices[1];
indices0_3.y <- mTexIndices[2] concat mTexIndices[3];
indices0_3.z <- mTexIndices[4] concat mTexIndices[5];
indices0_3.w <- mTexIndices[6] concat mTexIndices[7];
indices4_7.x <- mTexIndices[8] concat mTexIndices[9];
indices4_7.y <- mTexIndices[10] concat mTexIndices[11];
indices4_7.z <- mTexIndices[12] concat mTexIndices[13];
indices4_7.w <- mNormalMapWeight;

They are perfectly synced. But now we went and added one more texture, meaning one more index to mTexIndices. So if we do nothing here, we'll end up with the following to the indices4_7:
indices4_7.x <- mTexIndices[8] concat mTexIndices[9];
indices4_7.y <- mTexIndices[10] concat mTexIndices[11];
indices4_7.z <- mTexIndices[12] concat mTexIndices[13];
indices4_7.w <- mTexIndices[14] concat (first 16 bits of mNormalMapWeight);

Note that mNormalMapWeight is now broken. There's actually a few different ways to fix this, one would be to move mNormalMapWeight declaration to before mTexIndices and grab it in the GPU with a single float., another one would be to declare an extra float in the end of the GPU material, which would receive the remaining 16 bits from mNormalMapWeight, than we would reconstruct it from indices4_7.w and this float. But let's to a third way here, let's just push the mNormalMapWeight 16 bits to the front so it ends up directly where it has to be.

To do this we'll do the following changes:

Code: Select all

uint16  mTexIndices[NUM_PBSM_TEXTURE_TYPES [b]+ 1[/b]];

Code: Select all

const size_t HlmsPbsDatablock::MaterialSizeInGpu = 52 * 4 + (NUM_PBSM_TEXTURE_TYPES [b] + 1[/b])* 2 + 4;

Code: Select all

uvec4 indices0_3;
uvec4 indices4_7;
float normalMapWeight;

Now they are back in sync.

(Continues...)

[aymar]

So, after syncing the C++ memory and the GPU memory, let's continue to customize the datablock implementation.

Now we're gonna add the code to detect a decal_map from the script and correctly set it in the material.
We need to add something like this to the Datablock constructor (OgreHlmsPbsDatablock.cpp)

Code: Select all

        if( Hlms::findParamInVec( params, "decal_map", paramVal ) )
        {
            textures[PBSM_DECAL].texture = setTexture( paramVal, PBSM_DECAL );
            mSamplerblocks[PBSM_DECAL] = hlmsManager->getSamplerblock( HlmsSamplerblock() );
        }

And also this:

Code: Select all

        if( Hlms::findParamInVec( params, "uv_decal_map", paramVal ) )
        {
            setTextureUvSource( PBSM_DECAL, StringConverter::parseUnsignedInt( paramVal ) );
        }

We also need to customize the HlmsPbsDatablock::setTexture method, like this:

Code: Select all

        const HlmsTextureManager::TextureMapType texMapTypes[NUM_PBSM_TEXTURE_TYPES] =
        {
            HlmsTextureManager::TEXTURE_TYPE_DIFFUSE,
            [b]HlmsTextureManager::TEXTURE_TYPE_DIFFUSE,[/b]
            HlmsTextureManager::TEXTURE_TYPE_NORMALS,
            HlmsTextureManager::TEXTURE_TYPE_DIFFUSE,
            HlmsTextureManager::TEXTURE_TYPE_MONOCHROME,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL_NORMAL_MAP,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL_NORMAL_MAP,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL_NORMAL_MAP,
            HlmsTextureManager::TEXTURE_TYPE_DETAIL_NORMAL_MAP,
            HlmsTextureManager::TEXTURE_TYPE_ENV_MAP
        };

Remember to add in the correct position (where you first added the decal entry in the prerequisites).

And finally the HlmsPbsDatablock::suggestMapTypeBasedOnTextureType method:

Code: Select all

 switch( type )
        {
        default:
        case PBSM_DIFFUSE:
        [b]case PBSM_DECAL:[/b]
        case PBSM_SPECULAR:
        case PBSM_DETAIL_WEIGHT:
        case PBSM_DETAIL0:
        case PBSM_DETAIL1:
        case PBSM_DETAIL2:
        case PBSM_DETAIL3:
            retVal = HlmsTextureManager::TEXTURE_TYPE_DIFFUSE;
            break;

Now the C++ work is done, moving on to the shaders.

If you recall we already glued the C++ material to the shader material, now we need to extract the information sent correctly. This code you'll find in the PixelShader_ps.glsl:

Code: Select all

@property( diffuse_map )	diffuseIdx			= material.indices0_3.x & 0x0000FFFFu;@end
[b]@property( decal_map )	decalIdx	    		= material.indices0_3.x >> 16u;@end[/b]
@property( normal_map_tex )	normalIdx			= [b]material.indices0_3.y & 0x0000FFFFu;@end[/b]
@property( specular_map )	specularIdx			= [b]material.indices0_3.y >> 16u;@end[/b]
@property( roughness_map )	roughnessIdx		= [b]material.indices0_3.z & 0x0000FFFFu;@end[/b]
@property( detail_weight_map )	weightMapIdx	= [b]material.indices0_3.z >> 16u;@end[/b]
@property( detail_map0 )	detailMapIdx0		= [b]material.indices0_3.w & 0x0000FFFFu;@end[/b]
@property( detail_map1 )	detailMapIdx1		= [b]material.indices0_3.w >> 16u;@end[/b]
@property( detail_map2 )	detailMapIdx2		= [b]material.indices4_7.x & 0x0000FFFFu;@end[/b]
@property( detail_map3 )	detailMapIdx3		= [b]material.indices4_7.x >> 16u;@end[/b]
@property( detail_map_nm0 )	detailNormMapIdx0	= [b]material.indices4_7.y & 0x0000FFFFu;@end[/b]
@property( detail_map_nm1 )	detailNormMapIdx1	= [b]material.indices4_7.y >> 16u;@end[/b]
@property( detail_map_nm2 )	detailNormMapIdx2	= [b]material.indices4_7.z & 0x0000FFFFu;@end[/b]
@property( detail_map_nm3 )	detailNormMapIdx3	= [b]material.indices4_7.z >> 16u;@end[/b]
@property( envprobe_map )	envMapIdx			= [b]material.indices4_7.w & 0x0000FFFFu;@end[/b]

And also, we need to subtrack 1 from the index (Ogre adds 1 to the indices to avoid 0 indices because HLMS parameters set to 0 equals parameters not set. So in order to actually get the expected value we need to subtract 1 from the index, this is done in the Textures_piece_ps.glsl:

Code: Select all

@sub( diffuse_map_idx, diffuse_map, 1 )
[b]@sub( decal_map_idx, decal_map, 1 )[/b]
@sub( normal_map_tex_idx, normal_map_tex, 1 )

Note that I not only added the decalIdx, I also changed the value of all others Idx after decal, because it ends up changing their values. And if you did everything correctly so far, when you pass a decal map in your script like this:

Code: Select all

hlms FooMaterial pbs{
    diffuse_map foo.dds
    decal_map bar.dds
}

You'll be able to use @property( decal_map ) in the HLMS shader implementation.

Now we also need to correct the normal map weight (since we messed with that a bit).

Just change this line:

Code: Select all

@property( normal_weight_tex )#define normalMapWeight uintBitsToFloat( material.indices4_7.w )@end

To this:

Code: Select all

@property( normal_weight_tex )#define normalMapWeight material.normalMapWeight@end

Now, just to finish it all up, let's use our decal map in the shader. Find the file Textures_piece_ps.glsl and add these changes:

Code: Select all

@property( diffuse_map )
	@property( !hlms_shadowcaster )
		@piece( SampleDiffuseMap )	
            diffuseCol = texture( textureMaps[@value( diffuse_map_idx )], vec3( inPs.uv@value(uv_diffuse).xy, diffuseIdx ) );
            
            [b]@property( decal_map )
                vec4 decal = texture( textureMaps[@value( decal_map_idx )], vec3( inPs.uv@value(uv_decal).xy, decalIdx ) );
                diffuseCol.rgb = mix(diffuseCol.rgb, decal.rgb, decal.w);
            @end[/b]
            
            @property( !hw_gamma_read )	diffuseCol = diffuseCol * diffuseCol;@end 
        @end
	@end @property( hlms_shadowcaster )
		@piece( SampleDiffuseMap )	
            diffuseCol = texture( textureMaps[@value( diffuse_map_idx )], vec3( inPs.uv@value(uv_diffuse).xy, diffuseIdx ) ).w;
        @end
	@end
@end

Important things to point out:
The correct setup of the IdString "decal_map" and the actual texture makes available to the HLMS two things: @property( decal_map ) and @value( decal_map_idx ). The _idx one represents the index in the texture map your texture actually is, and the decal_map_idx is automatically done by Ogre, if you recall we never set anything with this name, even though it'll be available.

Obviously I may have understood some things wrong or missed a few points, if you have any questions or corrections, please point them out.

And of course, if you understand this tutorial you should be able to send over to the GPU custom material data easily, because this needs fewer setup than what we did here.

Cheers!

P.S.: As pointed out by dark_sylinc, the smartest way to add decals to your scene would be to use the detail maps already present in the default implementation, the example used in the tutorial (decal_map) was used just to illustrate the need to have additional textures.

[xrgo]

Thank you very much for this, I already knew this since I made my own pbr implementation (http://www.ogre3d.org/forums/viewtopic.php?f=25&t=84156) but with this tutorial many more can dare to do the same or expand the current one =) thank you so much for taking the time

one thing I believe is missing is something like this:

Code: Select all

@sub( decal_map_idx, decal_map, 1 )

int the Textures_piece_ps.glsl

This will compensate the "+1" in OgreHlmsPbs.cpp

Code: Select all

    void HlmsPbs::setTextureProperty( IdString propertyName, HlmsPbsDatablock *datablock,
                                      PbsTextureTypes texType )
    {
        uint8 idx = datablock->getBakedTextureIdx( texType );
        if( idx != NUM_PBSM_TEXTURE_TYPES )
        {
            //In the template the we subtract the "+1" for the index.
            //We need to increment it now otherwise @property( diffuse_map )
            //can translate to @property( 0 ) which is not what we want.
            setProperty( propertyName, idx + 1 );
        }
    }

[aymar]

Oh yes, I forgot the @sub line, I'll edit the tutorial and include it, many thanks.

[Zonder]

nice work!

[hyyou]

aymar, thank for the great dedication for explaining HLMS PBS in detail.

However, after I followed some of your steps, I began to feel a bit uneasy because most of the steps look like ... a hack.

Are Ogre's users really allowed (or supposed) to change Ogre3D's internal files e.g. OgreHlmsPbs.cpp?

I love your instruction , but hacking is on the border of my comfort zone.
I believe custom modification to any library's source code should generally be avoided, except it is designed to be modified.

(Sorry to revive an old thread, but this seems to be the only one HLMS PBS advanced tutorial.)

[dark_sylinc]

I should point out that PBS already supported Decals, it's via PBSM_DETAIL0 through PBSM_DETAIL3 & PBSM_DETAIL0_NM through PBSM_DETAIL3_NM.
Regardless, this guide is very useful in case you need to add more functionality we don't provide out of the box.

However, after I followed some of your steps, I began to feel a bit uneasy because most of the steps look like ... a hack.

Are Ogre's users really allowed (or supposed) to change Ogre3D's internal files e.g. OgreHlmsPbs.cpp?

I love your instruction , but hacking is on the border of my comfort zone.
I believe custom modification to any library's source code should generally be avoided, except it is designed to be modified.

Modifying PBS is fine by some, uncomfortable by others.
If you're a bit uncomfortable, you may end up achieving the same results by subclassing HlmsPbs, overriding the virtual functions (beware not all of our functions are virtual for performance reasons) and adding code to the Hlms template files by defining the "custom_*" pieces (look in the manual for the pieces you can define).

If the modifications aren't too big, you may prefer to use the HlmsListener instead of subclassing.

In order of flexibility:

• Using an HlmsListener + defining custom_ pieces: You can't achieve custom decals with this, but you can add features that affects all objects like for example add atmospheric effects & fog, extra lighting equations, more shadows, etc.
• Subclassing HlmsPbs + defining custom_ pieces
• Modifying HlmsPbs directly

Usually option 1. is enough for most needs. But it depends on what you want to do. The way of operation is the same regardless of what method you choose (define properties to communicated with the shader template, add shader code, etc)