Triangle(s) clipped after geometry shader Topic is solved

[bishopnator]

Hi, probably the question should go to another board, but it is too technical, so I put it here within the dev forum.

I trying to convert a solid geometry to wireframe, filtering the edges and applying some thickness to them in geometry shader:

It seems to work as expected, however if I come with the camera closer to the thick lines, some of the triangles are clipped completely:

(notice the missing 2 triangles from the violet thick lines)

It is surely related to the generated z-values of the triangles, but with the camera seeing all the boxes, the z-buffer works correctly (resolving the visibility as expected). In the geometry shader I compute the positions of vertices for the thick line with following code snippet:

Code: Select all

	// take projected positions
	const float4 p0 = input[i0].position;
	const float4 p1 = input[i1].position;
	// uv axis of the edge after projecting the vertices - u is along the edge, v is perpendicular to the edge
	const float4 u = float4(normalize(p1.xy / p1.w - p0.xy / p0.w), 0, 0);
	const float4 v = float4(-u.y, u.x, 0, 0);

// main line segment body
emitVertexData(output, p0 + v * r0, float4(d0,  halfw, d0, d1), drawId);
emitVertexData(output, p0 - v * r0, float4(d0, -halfw, d0, d1), drawId);
emitVertexData(output, p1 + v * r1, float4(d1,  halfw, d0, d1), drawId);
emitVertexData(output, p1 - v * r1, float4(d1, -halfw, d0, d1), drawId);
output.RestartStrip();

The emitVertexData outputs the second parameter as position of the vertex. The p0 and p1 are actually the transformed positions of the vertices with model-view-projection matrix and as you can see, the v vector has both z and w values set to 0.0 so the output z and w values are the same for the whole triangle as for the particular line. I expect that the triangles should be clipped per pixel basis and not as whole.

Is there any setting in Ogre to influence the clipping? Note that I copied the code from my othe OpenGL application (which doesn't use Ogre) and there I don't see such strange clipping. The same behavior is actually with both Ogre renderers - GL3Plus and D3D11.

Just for completeness, here is the screenshot from another app which shows the expected clipping by near clip plane. but still showing both triangles of a thick edge of the box:

[bishopnator]

It ghosted me for long time before I posted the question here and somehow it helped me to realize what was the problem. When one vertex is inside the view frustum and another is outside, the p0.w and p1.w can have different signs so the generated triangles are not properly ordered. The emitted triangle strip in this case doesn't form a rectangle but actually a "ribbon" - one triangle is then not clipped, but it has wrong winding order - with the help of renderdoc I was able to identify the problem.

[dark_sylinc]

Unrelated to your specific problem but relevant to your overall problem:

If you are using GS, have you considered using barycentrics?

See this example.

The technique consist in adding 3 interpolators for each vertex in a single triangle, one for the barycentric of each vertex and its 2 siblings:

Code: Select all

struct PS_INPUT
{
    float bary_v0;
    float bary_v1;
    float bary_v2;
};

From this, you take the minimum value of them all, and thus you will have the distance to the border of the triangle. You can use discard or alpha blending to prevent rendering the rest of the triangle (only render when min( v0, v1, v2 ) > threshold).

This technique is a bit fillrate intensive because the whole triangle must be rendered (rather than just the edges); instead of rendering triangles that pretend to be lines.

But its simplicity is incredible. This also means you don't get to benefit from MSAA though, however you can use A2C (alpha to coverage) to tell the GPU how to blend the border.

This technique requires a Geometry Shader so that you can add the barycentrics. In theory you can prebake the barycentrics to the vertex buffer in order to avoid the GS, but that increases the complexity of the technique.

If using Vulkan with extensions or D3D12 SM6.1 (note: OgreNext has no D3D12 backend), you can use barycentrics provided by the HW.

Edit: I forgot. In order to add barycentrics, by hand, you need to do the following in the geometry shader:

Code: Select all

// Vertex 0:
geom_out[0].bary_v0 = 0; // Or is it -1?
geom_out[0].bary_v1 = 1;
geom_out[0].bary_v2 = 1;

// Vertex 1:
geom_out[1].bary_v0 = 1;
geom_out[1].bary_v1 = 0; // Or is it -1?
geom_out[1].bary_v2 = 1;

// Vertex 2:
geom_out[2].bary_v0 = 1;
geom_out[2].bary_v1 = 1;
geom_out[2].bary_v2 = 0; // Or is it -1?

And in the pixel shader:

Code: Select all

float v = min( ps_input.bary_v0, ps_input.bary_v1, ps_input.bary_v2 );
if( v > threshold )
     discard;

// You can get fancy and instead of discard, use smoothstep and then A2C:
return float4( colour, smoothstep( v, min_threshold, max_threshold ) );

[bishopnator]

Yes, I already played with it and unfortunately the results are not very acceptable in a lot of cases. The main problem is that the triangles are still used from the original mesh - if you consider a box, the triangles are not going outside like you see in my screenshots so in certain cases you actually get only half-width on the edges. Only if you see both triangles connected to the same edge, you can render the edge with proper width.

Another problem is with flatness - if you consider just a square (2 triangles) and for the sake of this example, the triangles are also bigger by half-width of the desired thickness so when camera looks directly on the rectangle, its 4 edges are possible to render properly with desired thickness. But if the camera starts rotating so the 2 triangles are more and more parallel with camera view direction, there is no chance to render the edges properly with desired thickness.

Similar technique like using barycentric coordinates is for rendering a thick circle using just 2 triangles, where in pixel shader, a distance between a pixel and circle center is calculated. It looks amazing, when camera looks perpendicular to the circle, but worse when it bends so the circle is parallel with view direction.

When I saw those artifacts, I abandoned such ideas I didn't find anything better than doing the real conversion of mesh triangles to lines and within the same GS back to triangles which are oriented towards the camera.

Some useful links:
https://gamedev.stackexchange.com/quest ... oth-circle
https://wunkolo.github.io/post/2022/07/ ... wireframe/

[bishopnator]

I am actually implementing this technique: https://jcgt.org/published/0002/02/08/paper.pdf
I am trying to encode all the information in the same mesh so it is not necessary to switch the mesh when it is rendered solid or wireframe, with solid lines or with pattern, thick or thin lines etc. My goal is to always use the same mesh. Encoding the distance along the connected visible edges is the next challenge.

In the link (pdf) you can see the triangles needed for the thick line and the caps - it is not so hard to imagine all the details, but I am posting it for completeness. Maybe somebody will be also interested in the implementation so he/she can reimplement it to his/her needs.