[2.1] Easier way to communicate with hlms shader?

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
Post Reply
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

[2.1] Easier way to communicate with hlms shader?

Post by xrgo »

Hello, in my previous topic ( http://www.ogre3d.org/forums/viewtopic.php?f=25&t=82878 ) I learned some ways to communicate with the pbs shader from code:
1. modifying the OgreHlmsPbs implementation ( http://www.ogre3d.org/forums/viewtopic. ... 78#p515450 )
2. adding new parameters in HlmsPbsDatablock ( http://www.ogre3d.org/forums/viewtopic. ... 78#p516178 )

And they work flawlessly =)... thing is I don't like the idea of modify Ogre source, because sometimes it gets complicated to merge when there are changes on the repo.

Is there any other (not so invasive) way to communicate with hlms shaders?
like in the old way:
glsl: uniform float myParameter;
c++: myMaterial->getTechnique(0)->getPass(0)->getFragmentProgramParameters()->setNamedConstant("myParameter", value);

Thanks in advance!!

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4501
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 936
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc »

Not yet.

There was/is a plan to allow for customizations interfaces. While they might not end up offering the full flexibility you need (depends on the features to be added...) as too much flexibility will hurt performance or make things harder, it could make the merging easier (or non existant if you're lucky).

The basic idea was that was that you would provide a template where custom pieces would be collected to be inserted in the shaders (so that you can customize the templates without actually modifying the original ones); and provide some sort of listener to setup per-pass data or setup properties at shader creation time (don't expect a listener to be called per object though, that ain't gonna happen!).
This isn't too hard, but might need to be handled with a bit of care.

Don't know when this feature is going to arrive though. There's still some important threading left to do to the Hlms (fillBuffersFor) which shouldn't affect such system, but if I'm wrong then that would make things more complicated if the customization system is already there, or may be not.

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo »

Thank you for your answer, hope to see that feature soon =) thanks

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4501
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 936
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc »

Hi xrgo. I just pushed this feature to the repo: https://bitbucket.org/sinbad/ogre/commi ... 6a3b3d8f26.

The manual explains a bit on how to do it. I haven't written a sample yet, showing how to do it. Do you think you can figure it out?

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo »

Thank you so much!!!!!!!
dark_sylinc wrote:Do you think you can figure it out?
yes I think I can :)

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo »

I just tried it and it worked like a charm =)
Thank you so much for this

aymar
Greenskin
Posts: 145
Joined: Fri Jun 12, 2015 6:53 pm
Location: Florianopolis, Brazil
x 17

Re: [2.1] Easier way to communicate with hlms shader?

Post by aymar »

Hey xrgo,

Can you post some info on how to use the HlmsListener to customize the PBS?

Thanks.

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo »

Here is how I made some simple linear fog:

You have to inherit some class from HlmsListener

Code: Select all

class SomeClass : public Ogre::HlmsListener
and implement these methods:

Code: Select all

    //Comunicate stuffs to the PBS shader
    Ogre::uint32 getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager );

    float* preparePassBuffer( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager,
                                      float *passBufferPtr );
for instance for my Fog I implement them like this:

Code: Select all

Ogre::uint32 SomeClass::getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager ){
        return 32; //( vec4 fogParams + vec4 fogColor )*4
    }

float* SomeClass::preparePassBuffer( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                  bool dualParaboloid, Ogre::SceneManager *sceneManager,
                                  float *passBufferPtr ){

    //Linear Fog parameters
    *passBufferPtr++ = sceneManager->getFogStart();
    *passBufferPtr++ = sceneManager->getFogEnd();
    *passBufferPtr++ = 0.0;
    *passBufferPtr++ = 0.0f;

    *passBufferPtr++ = sceneManager->getFogColour().r;
    *passBufferPtr++ = sceneManager->getFogColour().g;
    *passBufferPtr++ = sceneManager->getFogColour().b;
    *passBufferPtr++ = 1.0f;
    return passBufferPtr;
}
Then you have to tell the hlmsPbs that SomeClass is the listener

Code: Select all

hlmsPbs->setListener(mySomeClass);
Then in the shaders:
If you notice in the file "Structs_piece_vs_piece_ps.glsl" in the "PassBuffer" there is a "@insertpiece( custom_passBuffer )", this means that you can make your own custom passBuffer extension that the hlms precompiler will automatically put where that "@insertpiece( custom_passBuffer )" is.
So you can make a file named: "CustomPassBuffer_piece_vs_piece_ps.glsl" with this inside:

Code: Select all

@piece( custom_passBuffer )

//Fog
	vec4 fogParams;
	vec4 fogColor;
	
@end
now you have in your "fogParams" and "fogColor" that you can use in the pbs shader!
And in the same way if you notice in the file "PixelShader_ps.glsl" there is a "@insertpiece( custom_ps_posExecution )", so you can make a file named "CustomPost_piece_ps.glsl" with this inside:

Code: Select all

@piece( custom_ps_posExecution )
@property( !hlms_shadowcaster )
//Fog
	float fogDistance = length( gl_FragCoord.z / gl_FragCoord.w );   // or better "length( inPs.pos )" for a more accurate distance
	float fogFactor = 1.0-clamp((pass.fogParams.y - fogDistance) / (pass.fogParams.y - pass.fogParams.x),0.0,1.0);
	vec3  fogColor  = vec3(pass.fogColor.xyz);
	outColour.xyz   = mix(finalColour, fogColor, fogFactor );
	
@end
@end
and that way you are actually modifying the shader without modifying the original files =)
Last edited by xrgo on Sun Apr 17, 2016 5:54 pm, edited 1 time in total.

aymar
Greenskin
Posts: 145
Joined: Fri Jun 12, 2015 6:53 pm
Location: Florianopolis, Brazil
x 17

Re: [2.1] Easier way to communicate with hlms shader?

Post by aymar »

That helps a lot!

Do you know if low level materials still support auto params?

Thanks!

xrgo
OGRE Expert User
OGRE Expert User
Posts: 1147
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 166

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo »

yes! low level mats works as always, just no Fixed function pipeline and you need to put the entity on a V1_LEGACY renderqueue group

Lax
Orc
Posts: 476
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 30

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax »

Hi,

I have a question, will this fog code snippets mentioned above be implemented as part of Ogre2.x?
Because if you update the ogre repository and have your own Fog implementation, this could become a mess quickly...

Regards
Lax
Image
http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62
Image

al2950
OGRE Expert User
OGRE Expert User
Posts: 1221
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 154

Re: [2.1] Easier way to communicate with hlms shader?

Post by al2950 »

Lax wrote:
Wed Mar 28, 2018 12:00 pm
Because if you update the ogre repository and have your own Fog implementation, this could become a mess quickly...
I dont see this is as the case, in fact it is not the case! The fog implementation uses the standard interfaces for customizing a HLMS implementation. So you should face no difficulties in updating Ogre using your custom fog functions.

Lax
Orc
Posts: 476
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 30

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax »

Hi al2950,

ok thanks, I will try it out.

Regards
Lax
Image
http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62
Image

Lax
Orc
Posts: 476
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 30

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax »

Hi,

I'm looking at the fog implementation and the following came in my mind:

Code: Select all

hlmsPbs->setListener(mySomeClass);
Sets an active listener for hlms pbs, but if terra is used, there is already a listener "Ogre::HlmsPbsTerraShadows" set, so one listener would overwrite the other one.

I'm wondering, why several listeners cannot be registered like:

Code: Select all

hlmsPbs->addListener(mySomeClass);
Regards
Lax
Image
http://www.lukas-kalinowski.com/Homepage/?page_id=1631
Please support Second Earth Technic Base built of Lego bricks for Lego ideas: https://ideas.lego.com/projects/81b9bd1 ... b97b79be62
Image

hedphelym
Gremlin
Posts: 173
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 16
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym »

Hlsl version:

CustomPassBufferPiece_vs_piece_ps.hlsl

Code: Select all

@piece( custom_passBuffer )
	float4 fogParams;
	float4 fogColor;
@end
CustomPost_piece_ps.hlsl

Code: Select all

@piece( custom_ps_posExecution )
@property( !hlms_shadowcaster )
	float fogDistance =  length( inPs.pos );
	float fogFactor = clamp((passBuf.fogParams.y - fogDistance) / (passBuf.fogParams.y - passBuf.fogParams.x),0.0,1.0);
	outPs.colour0.rgb  = lerp(passBuf.fogColor.rgb,finalColour, fogFactor );
@end
@end

hedphelym
Gremlin
Posts: 173
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 16
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym »

When I run the fog with directx in debug mode it keeps hitting this assert in 'OgreHlmsPbs.cpp' :

Code: Select all

assert( (size_t)(passBufferPtr - startupPtr) * 4u == mapSize );
Has anyone of you tested this? Or know why that happens?
I have used the code exactly as mentioned in the first post, with the HLSL code I posted right above this post.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4501
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 936
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc »

Check that both getPassBufferSize and preparePassBuffer from your listener are getting called.

The listener's getPassBufferSize is supposed to tell Ogre that the buffer must be 32 bytes bigger. And preparePassBuffer should return a pointer that is advanced by 32 bytes (hence the post's code performs *passBufferPtr++ eight times).

The assert is there to check that everything that had to be sent to GPU got written, and that we didn't go past the bounds.
If the assert is failing, either getPassBufferSize/preparePassBuffer are not getting called, they are in disagreement, or something very weird is going on.

Watchout for alignToNextMultiple calls. I can't remember if they affect listener code, but it has bitten a few people a few times. Most variables have to be aligned to 16 bytes and this is painful to enforce when we allow arbitrary code to be inserted.

hedphelym
Gremlin
Posts: 173
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 16
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym »

The solution was very simple when I found it.
The getPassBufferSize function need to be const for it to be called.

Code: Select all

Ogre::uint32 PbsFog::getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager ) const
                                      {
                                      	.....
                                      }
                                      
What happened was that It was not defined as const, thereby not being called, and ogre got '0' instead of '32'.

PBY
Halfling
Posts: 47
Joined: Tue Jul 16, 2013 12:57 pm
x 1

Re: [2.1] Easier way to communicate with hlms shader?

Post by PBY »

As this seems to be the only place where this simple way of adding small shader programs, and the documentation is useless, here is one point missing above:

Not only should your custom file end with the correct extension (like .hlsl), and declare the wanted @piece name, but the filename is also important.
OgreHlms will enumerate all files from the Hlms archive, but will only take file ending (before file extension) with one the following strings:
  • piece_vs
  • piece_ps
  • piece_gs
  • piece_hs
  • piece_ds
It will use the following list for finding the corresponding files:

Code: Select all

PieceFilePatterns[] = { "piece_vs", "piece_ps", "piece_gs", "piece_hs", "piece_ds" };
You should also use the correct ending, as it will take only one of the strings for each type, i.e. for creating the vertex shader, it will only search for piece_vs.

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 4501
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 936
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc »

A few details to what you just said:
  1. You can concatenate these suffixes. MyFile_piece_vs_piece_ps.glsl means it will be parsed both during vertex and pixel shader stage.
  2. Use piece_all means it will be parsed in all shader stages (a shortcut to avoid concatenating all piece_*s suffixes)
  3. You can use the .any extension for files that should be parsed by any RenderSystem, ie. it gets parsed regardless of whether glsl/hlsl/metal is the currently active shader languague. You can also use @property( syntax == glsl )@end or @property( syntax != hlsl )@else @end to handle the minor differences

Post Reply