OpenGL render to texture flipped Topic is solved

[bishopnator]

Hi, in my testing app, I am drawing couple of boxes and converting them in geometry shader to wireframe (also with some additional filtering of the edges). I am trying to achieve some kind of caching of the rendered "static" scene so e.g. mouse movement over the scene if always fast regardless of the complexity of the scene (so with "cached" content, there is just 2 triangles + mouse cursor instead of redrawing whole the scene). I am constantly testing my shaders in D3D11 RS and also in GL3Plus RS. In GL3Plus I notices through render doc, that the cached content rendered in a texture is flipped and then when rendering full-screen quad to the window, it is correctly output (so the texture is flipped again). However this flipped rendering in the texture in GL3Plus seems to make problem in my geometry shader if I activate rendering of "thick" lines where my geometry shader expands the edges to the triangles. I suspect that due to this flipped rendering, the vertex order is flipped as well and all my triangles are clipped.

Some screenshots to give some insights from OpenGL rendering.

Render to texture:

Render to window - full screen quad as seen on the left side in the Event Browser:

Now with the thick lines activate in D3D11 render system (scene and full-screen quads have same orientation, no flipping):

In GL3Plus render system:

note: The displayed box has set minimum thickness so it is rendered as lines and hence the geometry is not clipped, rest is clipped

In the renderdoc it is possible to nicely visualize the output of the geometry shaders in the mesh view:

Is there is trick in Ogre how to force non-flipped rendering in the texture? Maybe there is a problem with my setup, but it looks strange as also rendering in GL3Plus directly to window is correct.

[paroj]

https://chat.deepseek.com/share/hgjak75m23ouwlzutn

[dark_sylinc]

This issue was addressed in your other post.

You can't force the render texture/window to not be flipped because that's an API quirk. As said in that other forum post, the ProjectionMatrix we send is altered so that final result never looks flipped. The most likely issue is that your Geometry Shader is working on vertices post ProjectionMatrix and not accounting for RenderPassDescriptor::requiresTextureFlipping.

Alternatively you could work pre-ProjectionMatrix and then apply the projection matrix in the GS.
Or you could build the ProjectionMatrix on your own without accounting for texture flipping. And then account for the flipping somewhere else.

Or you can simply do:

Code: Select all

if( flipped )
   vertex.y = 1.0 - vertex.y;
thick_lines( vertex );
if( flipped )
   vertex.y = 1.0 - vertex.y;

Sometimes these API details are hard to hide. For example see ForwardClustered::fillConstBufferData has to manually account for texture flipping as well:

Code: Select all

// The way ogre represents viewports is top = 0 bottom = 1. As a result if 'texture flipping'
// is required then all is ok. However if it is not required then viewport offsets are
// actually represented from the bottom up.
// As a result we need convert our viewport height offsets to work bottom up instead of top down;
// This is compounded by OpenGL standard being different to DirectX and Metal
if( !bRequiresTextureFlipping && shaderSyntax == "glsl" && !instancedStereo )
{
	viewportHeightOffset = static_cast<float>(
    ( 1.0 - ( viewport->getTop() + viewport->getHeight() ) ) * renderTargetHeight );
}

[bishopnator]

I was hoping for some kind of "non-hacky" solution With the quad rendering it was possible to solve just by checking the requiresTextureFlipping in my HLMS implementation and using pRenderSystem->_convertProjectionMatrix to prepare the projection matrix for the quad pass. It looks like this one is not enought regarding the geometry shaders if they operate on the already projected vertices (like it is in my case). The Hlms class already setups the hlms_forwardplus_flipY property and together with syntax property (not sure whether I need to use this as well, but in the HlmsPbs template files they are usually checked together). I will still try to look for some more systematic solution, but I will keep in mind, that something like this could probably work. My problem is just with code duplication in the templates as the block for emitting the triangles is already longer. If this code would be the only one to duplicate, it is ok, but I have just a fear at the moment, that starting using the property hlms_forwardplus_flipY will increase the complexity (and decrease the readability) of the templates.

What I don't understand yet is, that I am actully porting the rendering technique from another app, which is purely OpenGL and there I didn't have any problems with this winding order. The shader is exactly the same used whether it is rendered to the window or to the texture. Actually also the projection matrices are exactly the same and there is no flipping at all.

[dark_sylinc]

I was hoping for some kind of "non-hacky" solution

I'm quite convinced is that at some point your code does v.y += offset where now should be doing v.y -= offset. Once you realize what's going on, you can think of what's the least hacky way to do it.

This is because we don't just flip the projection matrix, we also flip the culling mode (this happens in Hlms::applyStrongMacroblockRules see HlmsMacroblock::InvertCullingMode).

That means that if you're offsetting in the right direction, the winding order should not flip. But if you're offsetting in the same direction you would in a non-flipped target, the winding order will flip (and the results will be wrong and look different, but the difference may be slight and easy to miss).

What I don't understand yet is, that I am actully porting the rendering technique from another app, which is purely OpenGL and there I didn't have any problems with this winding order. The shader is exactly the same used whether it is rendered to the window or to the texture. Actually also the projection matrices are exactly the same and there is no flipping at all.

That, I cannot help you with
RenderDoc has source level debugging, so I guess you can just single step through every line and spot the difference?

If both outputs the exact same vertices, then the obvious suspect is that one uses triangle culling while the other is set to none.

[bishopnator]

Looking again in renderdoc, I found following difference:

Rendering in window directly has cull mode set to back:

Rendering in texture has cull mode set to front:

This somehow doesn't sound good. Anyway, I tried to disable the culling in my HLMS and it outputs the geometry, but with "default" culling, I couldn't influence the order of emitted vertices in geometry shader to see the output correctly. I sounds strange, but however I changed the order, there is still nothing on the screen.

Code: Select all

	// Main line segment body.
	emitVertexData(output, p0 + v * r0, float4(d0,  halfw, d0, d1), drawId);
	emitVertexData(output, p0 - v * r0, float4(d0, -halfw, d0, d1), drawId);
	emitVertexData(output, p1 + v * r1, float4(d1,  halfw, d0, d1), drawId);
	emitVertexData(output, p1 - v * r1, float4(d1, -halfw, d0, d1), drawId);
	output.RestartStrip();

The emitVertexData is my local function in the GS just to emit all vertex attributes properly. The p0, p1 are positions after applying MVP matrix and vr0 or vr1 is just a vector calculated in projection space to achieve the desired line width. It should be enough just to swap middle 2 emitted vertices to invert the winding order, but homehow it doesn't work (I actually tried then I think all possible combinations, but without any success )

[bishopnator]

I tried in my GS also to invert the signs when I am applying the +v or -v, but the result is same. Somehow the emitted triangles are always culled.

[bishopnator]

With a huge shame I have to admit, that I edited wrong GS I have a GS for lines which converts them to thick lines (line --> triangle strip) and another GS for faces which converts them to thick lines (triangles -> triangle string). I can now properly integrate the changes there using hlms_forwardplus_flipY.