[2.1] Easier way to communicate with hlms shader?

Design / architecture / roadmap discussions related to future of Ogre3D (version 2.0 and above)
Post Reply
xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

[2.1] Easier way to communicate with hlms shader?

Post by xrgo » Sun Apr 05, 2015 5:02 am

Hello, in my previous topic ( http://www.ogre3d.org/forums/viewtopic.php?f=25&t=82878 ) I learned some ways to communicate with the pbs shader from code:
1. modifying the OgreHlmsPbs implementation ( http://www.ogre3d.org/forums/viewtopic. ... 78#p515450 )
2. adding new parameters in HlmsPbsDatablock ( http://www.ogre3d.org/forums/viewtopic. ... 78#p516178 )

And they work flawlessly =)... thing is I don't like the idea of modify Ogre source, because sometimes it gets complicated to merge when there are changes on the repo.

Is there any other (not so invasive) way to communicate with hlms shaders?
like in the old way:
glsl: uniform float myParameter;
c++: myMaterial->getTechnique(0)->getPass(0)->getFragmentProgramParameters()->setNamedConstant("myParameter", value);

Thanks in advance!!
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc » Sun Apr 05, 2015 7:27 am

Not yet.

There was/is a plan to allow for customizations interfaces. While they might not end up offering the full flexibility you need (depends on the features to be added...) as too much flexibility will hurt performance or make things harder, it could make the merging easier (or non existant if you're lucky).

The basic idea was that was that you would provide a template where custom pieces would be collected to be inserted in the shaders (so that you can customize the templates without actually modifying the original ones); and provide some sort of listener to setup per-pass data or setup properties at shader creation time (don't expect a listener to be called per object though, that ain't gonna happen!).
This isn't too hard, but might need to be handled with a bit of care.

Don't know when this feature is going to arrive though. There's still some important threading left to do to the Hlms (fillBuffersFor) which shouldn't affect such system, but if I'm wrong then that would make things more complicated if the customization system is already there, or may be not.
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo » Sun Apr 05, 2015 6:14 pm

Thank you for your answer, hope to see that feature soon =) thanks
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc » Sat Apr 25, 2015 8:06 pm

Hi xrgo. I just pushed this feature to the repo: https://bitbucket.org/sinbad/ogre/commi ... 6a3b3d8f26.

The manual explains a bit on how to do it. I haven't written a sample yet, showing how to do it. Do you think you can figure it out?
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo » Sat Apr 25, 2015 9:25 pm

Thank you so much!!!!!!!
dark_sylinc wrote:Do you think you can figure it out?
yes I think I can :)
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo » Mon May 18, 2015 8:28 pm

I just tried it and it worked like a charm =)
Thank you so much for this
0 x

aymar
Greenskin
Posts: 145
Joined: Fri Jun 12, 2015 6:53 pm
Location: Florianopolis, Brazil

Re: [2.1] Easier way to communicate with hlms shader?

Post by aymar » Tue Jul 14, 2015 2:28 pm

Hey xrgo,

Can you post some info on how to use the HlmsListener to customize the PBS?

Thanks.
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo » Tue Jul 14, 2015 9:26 pm

Here is how I made some simple linear fog:

You have to inherit some class from HlmsListener

Code: Select all

class SomeClass : public Ogre::HlmsListener
and implement these methods:

Code: Select all

    //Comunicate stuffs to the PBS shader
    Ogre::uint32 getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager );

    float* preparePassBuffer( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager,
                                      float *passBufferPtr );
for instance for my Fog I implement them like this:

Code: Select all

Ogre::uint32 SomeClass::getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager ){
        return 32; //( vec4 fogParams + vec4 fogColor )*4
    }

float* SomeClass::preparePassBuffer( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                  bool dualParaboloid, Ogre::SceneManager *sceneManager,
                                  float *passBufferPtr ){

    //Linear Fog parameters
    *passBufferPtr++ = sceneManager->getFogStart();
    *passBufferPtr++ = sceneManager->getFogEnd();
    *passBufferPtr++ = 0.0;
    *passBufferPtr++ = 0.0f;

    *passBufferPtr++ = sceneManager->getFogColour().r;
    *passBufferPtr++ = sceneManager->getFogColour().g;
    *passBufferPtr++ = sceneManager->getFogColour().b;
    *passBufferPtr++ = 1.0f;
    return passBufferPtr;
}
Then you have to tell the hlmsPbs that SomeClass is the listener

Code: Select all

hlmsPbs->setListener(mySomeClass);
Then in the shaders:
If you notice in the file "Structs_piece_vs_piece_ps.glsl" in the "PassBuffer" there is a "@insertpiece( custom_passBuffer )", this means that you can make your own custom passBuffer extension that the hlms precompiler will automatically put where that "@insertpiece( custom_passBuffer )" is.
So you can make a file named: "CustomPassBuffer_piece_vs_piece_ps.glsl" with this inside:

Code: Select all

@piece( custom_passBuffer )

//Fog
	vec4 fogParams;
	vec4 fogColor;
	
@end
now you have in your "fogParams" and "fogColor" that you can use in the pbs shader!
And in the same way if you notice in the file "PixelShader_ps.glsl" there is a "@insertpiece( custom_ps_posExecution )", so you can make a file named "CustomPost_piece_ps.glsl" with this inside:

Code: Select all

@piece( custom_ps_posExecution )
@property( !hlms_shadowcaster )
//Fog
	float fogDistance = length( gl_FragCoord.z / gl_FragCoord.w );   // or better "length( inPs.pos )" for a more accurate distance
	float fogFactor = 1.0-clamp((pass.fogParams.y - fogDistance) / (pass.fogParams.y - pass.fogParams.x),0.0,1.0);
	vec3  fogColor  = vec3(pass.fogColor.xyz);
	outColour.xyz   = mix(finalColour, fogColor, fogFactor );
	
@end
@end
and that way you are actually modifying the shader without modifying the original files =)
Last edited by xrgo on Sun Apr 17, 2016 5:54 pm, edited 1 time in total.
1 x

aymar
Greenskin
Posts: 145
Joined: Fri Jun 12, 2015 6:53 pm
Location: Florianopolis, Brazil

Re: [2.1] Easier way to communicate with hlms shader?

Post by aymar » Tue Jul 14, 2015 10:54 pm

That helps a lot!

Do you know if low level materials still support auto params?

Thanks!
0 x

xrgo
OGRE Expert User
OGRE Expert User
Posts: 909
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 50

Re: [2.1] Easier way to communicate with hlms shader?

Post by xrgo » Tue Jul 14, 2015 11:12 pm

yes! low level mats works as always, just no Fixed function pipeline and you need to put the entity on a V1_LEGACY renderqueue group
0 x

Lax
Gnome
Posts: 311
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 4

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax » Wed Mar 28, 2018 12:00 pm

Hi,

I have a question, will this fog code snippets mentioned above be implemented as part of Ogre2.x?
Because if you update the ogre repository and have your own Fog implementation, this could become a mess quickly...

Regards
Lax
0 x

al2950
OGRE Expert User
OGRE Expert User
Posts: 1131
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 51

Re: [2.1] Easier way to communicate with hlms shader?

Post by al2950 » Wed Mar 28, 2018 1:43 pm

Lax wrote:
Wed Mar 28, 2018 12:00 pm
Because if you update the ogre repository and have your own Fog implementation, this could become a mess quickly...
I dont see this is as the case, in fact it is not the case! The fog implementation uses the standard interfaces for customizing a HLMS implementation. So you should face no difficulties in updating Ogre using your custom fog functions.
0 x

Lax
Gnome
Posts: 311
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 4

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax » Thu Mar 29, 2018 3:11 pm

Hi al2950,

ok thanks, I will try it out.

Regards
Lax
0 x

Lax
Gnome
Posts: 311
Joined: Mon Aug 06, 2007 12:53 pm
Location: Saarland, Germany
x 4

Re: [2.1] Easier way to communicate with hlms shader?

Post by Lax » Tue May 01, 2018 9:36 am

Hi,

I'm looking at the fog implementation and the following came in my mind:

Code: Select all

hlmsPbs->setListener(mySomeClass);
Sets an active listener for hlms pbs, but if terra is used, there is already a listener "Ogre::HlmsPbsTerraShadows" set, so one listener would overwrite the other one.

I'm wondering, why several listeners cannot be registered like:

Code: Select all

hlmsPbs->addListener(mySomeClass);
Regards
Lax
0 x

hedphelym
Gremlin
Posts: 154
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 7
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym » Fri May 18, 2018 10:20 am

Hlsl version:

CustomPassBufferPiece_vs_piece_ps.hlsl

Code: Select all

@piece( custom_passBuffer )
	float4 fogParams;
	float4 fogColor;
@end
CustomPost_piece_ps.hlsl

Code: Select all

@piece( custom_ps_posExecution )
@property( !hlms_shadowcaster )
	float fogDistance =  length( inPs.pos );
	float fogFactor = clamp((passBuf.fogParams.y - fogDistance) / (passBuf.fogParams.y - passBuf.fogParams.x),0.0,1.0);
	outPs.colour0.rgb  = lerp(passBuf.fogColor.rgb,finalColour, fogFactor );
@end
@end
0 x

hedphelym
Gremlin
Posts: 154
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 7
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym » Thu Aug 23, 2018 1:12 pm

When I run the fog with directx in debug mode it keeps hitting this assert in 'OgreHlmsPbs.cpp' :

Code: Select all

assert( (size_t)(passBufferPtr - startupPtr) * 4u == mapSize );
Has anyone of you tested this? Or know why that happens?
I have used the code exactly as mentioned in the first post, with the HLSL code I posted right above this post.
0 x

User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 3770
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 131
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by dark_sylinc » Fri Aug 24, 2018 3:14 am

Check that both getPassBufferSize and preparePassBuffer from your listener are getting called.

The listener's getPassBufferSize is supposed to tell Ogre that the buffer must be 32 bytes bigger. And preparePassBuffer should return a pointer that is advanced by 32 bytes (hence the post's code performs *passBufferPtr++ eight times).

The assert is there to check that everything that had to be sent to GPU got written, and that we didn't go past the bounds.
If the assert is failing, either getPassBufferSize/preparePassBuffer are not getting called, they are in disagreement, or something very weird is going on.

Watchout for alignToNextMultiple calls. I can't remember if they affect listener code, but it has bitten a few people a few times. Most variables have to be aligned to 16 bytes and this is painful to enforce when we allow arbitrary code to be inserted.
0 x

hedphelym
Gremlin
Posts: 154
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 7
Contact:

Re: [2.1] Easier way to communicate with hlms shader?

Post by hedphelym » Fri Aug 24, 2018 7:30 am

The solution was very simple when I found it.
The getPassBufferSize function need to be const for it to be called.

Code: Select all

Ogre::uint32 PbsFog::getPassBufferSize( const Ogre::CompositorShadowNode *shadowNode, bool casterPass,
                                      bool dualParaboloid, Ogre::SceneManager *sceneManager ) const
                                      {
                                      	.....
                                      }
                                      
What happened was that It was not defined as const, thereby not being called, and ogre got '0' instead of '32'.
0 x

Post Reply