[2.1][SOLVED] Per Object Clipping using Shader Topic is solved

Problems building or running the engine, queries about how to use features etc.
Post Reply
gabbsson
Halfling
Posts: 65
Joined: Wed Aug 08, 2018 9:03 am
x 13

[2.1][SOLVED] Per Object Clipping using Shader

Post by gabbsson »

Ogre Version: 2.1
Operating System: Linux
Render System: GL3Plus

I have managed to implement clipping by adding the custom shader files for the Pbs shader:

CustomPassBuffer_piece_vs_piece_ps.glsl
CustomPost_piece_vs.glsl

I create vec4 planes in the custompassbuffer file:

Code: Select all

vec4 clipPlane0;
...
Which are "filled" using a customized HlmsListener:

Code: Select all

float*OgreClippingHlmsListener::preparePassBuffer(const Ogre::CompositorShadowNode* shadowNode, bool casterPass, bool dualParaboloid, Ogre::SceneManager* sceneManager, float* passBufferPtr)
{
    *passBufferPtr++ = clippingPlaneOne_.x;
    *passBufferPtr++ = clippingPlaneOne_.y;
    *passBufferPtr++ = clippingPlaneOne_.z;
    *passBufferPtr++ = clippingPlaneOne_.w;
    ...
 }
In the CustomPostpiece I simply set each clipdistance:

Code: Select all

gl_ClipDistance[0] = dot(float4(worldPos.xyz, 1.0), passBuf.clipPlane0.xyzw)
...
 
Currently I can't find the thread but I found an example somewhere how to do this on the forum.
Edit: Found it: viewtopic.php?f=25&t=83081
As far as I can tell this part works, I can successfully set my planes in code and it clips.
I have only done this for the Pbs shader, but I would assume the same can be done for Unlit.

The above solution will clip everything which uses the shader, which I want to change to clip only specific objects.
Ideally I want to turn on/off specific planes for specific objects.
My first thought was simply enabling and disabling (glEnable(CLIP_DISTANCE0f for example) the clipping distances in different renderqueues.
While this isn't strictly "per object", it would at least be a compromise. Unfortunately this did not work at all.

For example doing something like this, and putting objects in renderqueue 4, does not work:

Code: Select all

void OgreClippingRenderqueueListener::renderQueueStarted(Ogre::RenderQueue* rq, Ogre::uint8 queueGroupId, const Ogre::String& invocation, bool& skipThisInvocation)
{
	if (queueGroupId == 4)
	{
		glEnable(GL_CLIP_DISTANCE0);
	}
	else
	{
		glDisable(GL_CLIP_DISTANCE0);
	}
}
But if groupId is the last group (255) instead it clips everything, leading me to believe the each renderqueueStarted is called before rendering anything.
Perhaps this is all obvious and I simply misunderstood the documentation for renderqueuelisteners, or modifying GL stuff here is a bad idéa.

I have considered one other option, embedding the plane data into each vertex somehow, but that sounds cumbersome/terrible if I want to update the plane (having to iterate through each vertex and update). I have not even bothered to try this, even though it should technically be "per object".

Either way I am looking for any help I can get in tackling this problem. I hope I have provided enough info.

tl;dr:
The question can be made general: Is it possible to pass information on a per "object/mesh" level to the vertex shader?
If yes, I'd love an example!
Last edited by gabbsson on Tue Oct 16, 2018 2:39 pm, edited 2 times in total.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [2.1] Per Object Clipping using Shader

Post by al2950 »

Short answer is no, without creating your own version or extending a current HLMS implementation.

There are 3 different levels of 'customisations'
1) At the pass level. Ogre supports the ability to customize any HLMS shader at the pass level, which is what you have currently implemented.
2) At the material level. This currently requires you to effectively create your own HLMS
3) At the object level. This currently requires you to effectively create your own HLMS

The reason there is no easy way to manipulate 2 & 3 is because, especially 3, it is at a very performance sensitive area, so having for example a virtual call here would be open to abuse, and potential performance hits.

Having sad this, there have been some changes in the last year, that I have not looked , but may make it easier to add customizations.
gabbsson
Halfling
Posts: 65
Joined: Wed Aug 08, 2018 9:03 am
x 13

Re: [2.1] Per Object Clipping using Shader

Post by gabbsson »

Alright, I had a feeling that was the case.

The application I am writing won't be populated with a large amount of objects, so it might be ok even if it is not preferable do to changes at object level.
I'll start reading up on how to create my own Hlms implementation. Thank you for the quick answer!

I suppose I can leave this as unsolved in case someone has any other input, as you mentioned perhaps something has changed in the last year.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [2.1] Per Object Clipping using Shader

Post by al2950 »

I think if I was in your position, I would take the unlit HLMS and then either edit it directly or inherit from it and customize the functions you need.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Per Object Clipping using Shader

Post by dark_sylinc »

Hi!

To what has already been said:

1. You may also want to checkout this post about overloading default Hlms implementations to customize a specific behavior, whether per object or per material.

2. Indeed calling glEnable(CLIP_DISTANCE0) directly is a bad idea.
Ogre will read the property HlmsBaseProp::PsoClipDistances, set HlmsPso::clipDistances accordingly, and the GL3+ RenderSystem will call glEnable(CLIP_DISTANCEn).
Additionally, I'd suggest not to rely on glEnable(CLIP_DISTANCEn), because it's not a portable behavior: GL can use this function to disable clip distances. Even if the shaders contains clipping code and this setting is disabled, that shader code will be ignored.
The rest of the APIs (D3D11, Metal, Vulkan, D3D12) do not follow this behavior: If the shader contains clipping code, it's as if there were an implicit glEnable(CLIP_DISTANCEn); and if there is no clipping code in the shader, it's as if there's an implicit glDisable(CLIP_DISTANCEn);

In other words, if you want portable behavior, objects in which you do not want to apply clipping should not contain clipping code in the shaders (i.e. HlmsBaseProp::PsoClipDistances = 0) instead of generating a single shader that applies clipping to everything while relying on glEnable/glDisable(CLIP_DISTANCEn) to control which objects get clipped and which don't.
gabbsson
Halfling
Posts: 65
Joined: Wed Aug 08, 2018 9:03 am
x 13

Re: [2.1] Per Object Clipping using Shader

Post by gabbsson »

Hey!

Thanks for the reply, I've read through the post and think I've understood.
Based on your reply in the post I believe I want to inherit and overload HlmsUnlit::fillBuffersFor(..).
Unfortunately I seem to be lacking some understanding to pull that off.

Firstly I'd like to clarify my ultimate goal: let each renderable (what I called object) have its own list of clipping planes.
In my original post I was trying to compromise by doing it per renderqueue, but I'd much prefer to do it per object from the start.

Unfortunately I'm the the precarious situation of being rather new to C++ (even more so Ogre and shaders).
I'm not very confident when using pointers such as *passBufferPtr. From the post that showed how to do it I understood how to use *passBufferPtr and just "accepted" that it managed to place the data in the correct place in the shader. I don't quite see the connection between the buffer and where it will end up in the shader in this new case. (To be fair I don't really understand how any information gets passed to uniforms).

I've given it a shot, but quickly realized I'm not sure what I'm doing. Consider the following something more akin to pseudocode.
Sadly this is as far as I got, let me know if I'm on the right track:
Edit: Updated the code after reading the source/documentation for Renderable::getCustomParameter

Code: Select all


// All clipping planes enabled using HlmsPso::clipDistances
// Turn planes off in individual renderables by setting plane(s) to (0, 0, 0, 0) and back on by setting to non zero

uint32 HlmsClippingUnlit::fillBuffersFor( const HlmsCache *cache, const QueuedRenderable &queuedRenderable,
                                      bool casterPass, uint32 lastCacheHash,
                                      CommandBuffer *commandBuffer, bool isV1 )
{
  	uint32 length = HlmsUnlit::fillBuffersFor(...);

  	Renderable* renderable = queuedRenderable.renderable;
  	
  	size_t planeCount  = 7;
  	
  	for (size_t planeIndex = 0; planeIndex < planeCount; planeIndex++)
  	{
  		Vector4 plane = renderable.getCustomParameter(planeIndex);
  		
  		// Set buffer to the plane values, similar to the passBufferPtr
  		// Post says to use currentMappedConstBuffer
  		
  	}
  	// Increase length by the correct amount
		
  	return length;
}
I've tried to read through the HlmsUnlit::fillBuffersFor() source code but most of it goes straight over my head. I'll experiment and hopefully I figure it out but some more help would go a long way.

Mainly I'm confused about:
  • Do I want to add my planes into the currentMappedConstBuffer?
    If so what is the correct way to do that?
If I were just making a shader I would guess I'd need to create a uniform for each plane in the shader. Then for each renderable update those uniforms. Is the above analogous to that?

About the second point: I'll definitely try to update to the method you recommend, however I don't expect my program to run on anything other than Opengl platforms. For now portability is not an issue.
gabbsson
Halfling
Posts: 65
Joined: Wed Aug 08, 2018 9:03 am
x 13

Re: [2.1] Per Object Clipping using Shader

Post by gabbsson »

I've managed to find quite a few posts about making custom hlms/sending data to shader and so on. Either I just don't understand or I can't see the connection of their code to mine. For example most are basically what I did in my original post (per pass).

I have made progress however. The biggest changes I have made from my last post is to scrap the idea of inheriting fillBuffersFor and simply make my own which I call from inherited fillBuffersForv1 (and v2). I took the inspiraton from ColibriGUI, referenced in one of the posts. I copied the fillBuffersFor from HlmsUnlit and have since tried to understand how to add my own data to either currentMappedConstBuffer or currentMappedTexBuffer.

In the shader piece: Structs_piece_vs_piece_ps.glsl I found the Instance declaration which seems to be what I want ("Uniforms that change per item/entiy").

Code: Select all

...
// Code was not copied, spelling errors etc are just in this post not in the file.
@piece( InstanceDecl )
//Uniforms that change per Item/Entity
layout_constbuffer(binding = 2) uniform InstanceBuffer
{	
	//.x =
	//Contains the material's start index.
	//.y = 
	//shadowCOnstatnBias. Send the bias directly to avoid an unnecessary indirection
	//during the shadow mapping pass.
	//Must be loaded with uintBitsToFloat
	//
	//.z =
	//Contains 0 or 1 to index into passBuf.viewProj[] Only used if
	//hlms_identity_viewproj_dynamic is set.
	uvec4 worldMaterialIdx[4096];
	vec4 clipPlanes[4096];
} instance;
...
Inside is the worldMaterialIdx which I also see mentioned in the fillBuffersFor method. I tried to simply add my own array: vec4 clipPlanes[4096]. To get individual values I used finalDrawId (as seen from using MaterialIdx). Lastly I have tried to imitate what is happening in fillBuffersFor:

Code: Select all

...
   //---------------------------------------------------------------------------
    //                          ---- VERTEX SHADER ----
    //---------------------------------------------------------------------------
    bool useIdentityProjection = queuedRenderable.renderable->getUseIdentityProjection();

    //uint materialIdx[]
    *currentMappedConstBuffer = datablock->getAssignedSlot();
    *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer+1 ) = datablock->
                                                                                mShadowConstantBias;
    *(currentMappedConstBuffer+2) = useIdentityProjection;
    currentMappedConstBuffer += 4;
    
    // START CUSTOM CODE
    Ogre::Renderable* renderable = queuedRenderable.renderable;
    for (size_t planeIndex = 0; planeIndex < TOTAL_PLANE_COUNT; planeIndex++)
    {
        if (renderable->hasCustomParameter(planeIndex))
        {
            Ogre::Vector4 plane = renderable->getCustomParameter(planeIndex);
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.x;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.y;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.z;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.w;
        }
    }
    
    // Mat4 => 4x floats, move buffer 4x per float?
    currentMappedConstBuffer += TOTAL_PLANE_COUNT * 4;
    // END CUSTOM CODE

    //mat4 worldViewProj
 ...
The log says the instance buffer is too big so I seem to be putting too much stuff in there.
I changed the array sizes of worldMaterialIdx clipPlanes to 2048 (half each) and that seems to have fixed the buffer size error message (while I'm sure I've missed changing a bunch of other stuff that have to be tweaked as well).
I'm fine with it lowering the number of objects I can have in total, but I still haven't understood how to match that from fillBuffersFor.

I've also tried to figure out HlmsPbs::clipPlanes. Haven't found an example of how to use it yet, it seems connected to the rendersystem?
A quick example of how to create a HlmsPso (to be able to set clipPlanes) would also be great, but secondary.
gabbsson
Halfling
Posts: 65
Joined: Wed Aug 08, 2018 9:03 am
x 13

Re: [2.1][SOLVED] Per Object Clipping using Shader

Post by gabbsson »

So I finally figured out how to do what I wanted. Posting this for anyone curious.
Ultimately it lowered the amount of objects I could handle too much and I will be doing global clip planes instead (as opposed to per object).

I edited the Structs_piece_vs_piece_ps.glsl to add new uvec4s for my planes. Since the max is 64kb (originally wordlMateiralIdx has 4096 indexes, vec4 = 16 bytes => 64kb) splitting the array into others means reducing the max amount of objects (as mentioned earlier). I tried using 512 indexes which gave me 7 planes per object.

Code: Select all

...
@piece( InstanceDecl )
//Uniforms that change per Item/Entity
layout_constbuffer(binding = 2) uniform InstanceBuffer
{	
	//.x =
	//Contains the material's start index.
	//.y = 
	//shadowCOnstatnBias. Send the bias directly to avoid an unnecessary indirection
	//during the shadow mapping pass.
	//Must be loaded with uintBitsToFloat
	//
	//.z =
	//Contains 0 or 1 to index into passBuf.viewProj[] Only used if
	//hlms_identity_viewproj_dynamic is set.
	uvec4 worldMaterialIdx[512];
	uvec4 clipPlanes0[512];
	uvec4 clipPlanes1[512];
	...
	uvec4 clipPlanes6[512];
} instance;
...
To use the planes I updated the VertexShader_vs.glsl, this should probably be added via a custom piece but since its a custom Hlms I doubt it matters that much. I am by no means an expert when it comes to GLSL, so theres probably a better way to write most of this!

Code: Select all


main()
{
	...
	uvec4 bufferedVec4 = instance.clipPlanes0[finalDrawId];
	vec4 plane = vec4(uintBitsToFloat(bufferedVec4.x), uintBitsToFloat(bufferedVec4.y), uintBitsToFloat(bufferedVec4.z), uintBitsToFloat(bufferedVec4.w));
	
	gl_ClipDistance[0] = dot(vertex.xyzw, plane.xyzw);
	
	// Repeat for each plane
	...
}

To get my data into the new arrays in the shader I updated fillBuffersFor, the trick was realizing each array follows monotonically (which seems obvious now in retrospect). So for each new plane I move the buffer by 512.

Code: Select all

...
   //---------------------------------------------------------------------------
    //                          ---- VERTEX SHADER ----
    //---------------------------------------------------------------------------
    bool useIdentityProjection = queuedRenderable.renderable->getUseIdentityProjection();

    //uint materialIdx[]
    *currentMappedConstBuffer = datablock->getAssignedSlot();
    *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer+1 ) = datablock->
                                                                                mShadowConstantBias;
    *(currentMappedConstBuffer+2) = useIdentityProjection;

    
    // START CUSTOM CODE
    Ogre::Renderable* renderable = queuedRenderable.renderable;
    for (size_t planeIndex = 0; planeIndex < TOTAL_PLANE_COUNT; planeIndex++)
    {
    		// Move the buffer into the "next" array
    	     currentMappedConstBuffer += 512 * 4 * planeIndex;
    	     
            Ogre::Vector4 plane = renderable->getCustomParameter(planeIndex);
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.x;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.y;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.z;
            *reinterpret_cast<float * RESTRICT_ALIAS>( currentMappedConstBuffer) = plane.w;
    }
    
    // Move the buffer back to the original array for the next renderable (next filbuffersfor call)
    currentMappedConstBuffer -= 512 * 4 * TOTAL_PLANE_COUNT;
    
    // Move the buffer the 4 for the vec4 written to the original array (to get the correct index in the shader)
    currentMappedConstBuffer += 4;
    
    // Mat4 => 4x floats, move buffer 4x per float?
    currentMappedConstBuffer += TOTAL_PLANE_COUNT * 4;
    // END CUSTOM CODE

    //mat4 worldViewProj
 ...
Which results in per object clipping for 512 objects with 7 planes for each object.

EDIT:

I'm guessing I can avoid the limiting of objects by creating my own constBuffers and binding them to new uniforms, but I think global planes will suffice.
Don't want to make the impression that it's impossible to solve in a different way. But to me this was the "easiest" without creating a bunch of custom stuff. Basically everything is based on adding some more data to what already exists.

EDIT2:

After looking into the HlmsPbs version of fillBuffersFor I noticed it is quite different and does not use the InstanceDecl by default. In other words the above does not work "right away" for Pbs unfortunately.
Post Reply