Thin lines on an RTT

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Thin lines on an RTT

Post by Shem »

Hi,

Something new I'm trying to solve is aliasing of thin lines on an RTT. On low settings (AA = x4) a primitive white line of width 1 is showing oscillation of color from lighter to darker and width from thinner to thicker. I almost got it fixed by pushing my card all the way to AA x64 and increasing the line width to 2. After this change, it looks much better but the line is still slightly oscillating from thinner thicker. Dumping the alpha in both cases I can see that that's the reason for this oscillation. I know there are different and maybe even better strategies for AA lines. But I need to know why I'm seeing this artifact. If it's a known issue with graphic cards (or their drivers), or if it's an indication of a problem with my rendering pipe. Or maybe a limitation of the AA algo used. My scene doesn't have any lights and the materials have the lights turned off. Same for shadows. On a big display most people wouldn't even notice this issue. But I have a display that lets me zoom in on a subset of the full screen at x16 so that's when I see these issues. This becomes slightly more noticeable when the camera start moving and the line looks like it's shimmering a bit.

Code: Select all

	m_RTTexture =
		TextureManager::getSingleton().createManual(
			"RttTex",
			ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME,
			TEX_TYPE_2D,
			m_Width, m_Height,
			0,
			PF_A8R8G8B8,
			TU_RENDERTARGET,
			NULL,
			false,
			64
			//, "Quality"
		);
		
Is there anything I should be doing with fsaaHint?

Thanks in advance

Win10 , OGL 3+, v1.10, Quadro K4000

Thanks in advance
Shem
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

I understand this is something that has been discussed a lot in many places. Yet I fail to understand what could be causing something like this:



This is with AA set to 4
paroj
OGRE Team Member
OGRE Team Member
Posts: 1993
Joined: Sun Mar 30, 2014 2:51 pm
x 1073
Contact:

Re: Thin lines on an RTT

Post by paroj »

this does not look like an AA artifact. Can you also post the material and code that creates the lines?
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

It appears to be a classic case of not handling pre-multiplied alpha correctly.
Once I added

Code: Select all

ppass->setSceneBlending(SBF_ONE, SBF_ONE_MINUS_SOURCE_ALPHA);
it was sorted.
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

It appears I did not solve the issue after all. The result looked good because I also changed the viewport background color to match my line (transparent white). So when the low alpha values blended with the view port background color, the color blended nicely. This blew in my face the minute I drew a line using a different color (e.g. black) which resulted in a line which oscillates from white to black. Analyzing the RTT (using writeContentsToFile) I can see that the AA is both adding transparency and blending the color with the view port background color. I think it would have worked if the AA was only changing the alpha and not the color as that would mean the viewport background could be of any color. Not sure what I'm missing here.

@paroj The line drawing is super basic:

Code: Select all

	
	manual->begin("BaseWhiteNoLighting", RenderOperation::OT_LINE_STRIP);

	manual->position(-1.3f, -1.3f, 0.0f);
	manual->position(1.3f, -1.3f, 0.0f);
	manual->position(1.3f, 1.3f, 0.0f);
	manual->position(-1.3f, 1.3f, 0.0f);

	manual->index(0);
	manual->index(1);
	manual->index(2);
	manual->index(3);
	manual->index(0);

	manual->end();
I am not using a material for the lines. Maybe I should look there next...
Thanks in advance
Shem
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Thin lines on an RTT

Post by dark_sylinc »

The problem with a line of thickness = 1 pixel is that if a pixel is partially covered by the line, then it is either covered or not covered, causing lots of aliasing.
Untitled.png
Untitled.png (919 Bytes) Viewed 12494 times
When you use MSAA, the same problem applies: if a pixel is partially covered by the line, then anything between 0 to 4 subpixels will be covered or not covered.
After averaging the 4 subpixels, you get 4 shades of grey: 0% (black) 25% 75% and 100% (white)

This is all there is to it. That's how MSAA works (by rendering to bigger screens then averaging down) and this is what you're witnessing.

You can increase the resolution further and then downscale (i.e. supersampling) so that you get more shades of grey and the solution approaches the coverage (i.e. most pixels are either 0% or > 75% coverage) and/or make the line width bigger.

For example if you render to a 1920x1080 target, rendering to a 3240x2100 4xMSAA and then downsampling will get you 16x of effective SSAA.

However this solution is essentially the same as this:
I almost got it fixed by pushing my card all the way to AA x64 and increasing the line width to 2
Just Cause games addressed this problem differently using a cylinder and a custom vertex shader (PDF) which grows or shrinks the geometry dynamically so that the covered area is always at least 1 pixel. See page 26 "Phone-wire AA"

Cheers
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Thin lines on an RTT

Post by dark_sylinc »

To expand further on the problem, this is what happens when you render to 4xMSAA:
Untitled02.png
Untitled02.png (1.17 KiB) Viewed 12492 times
The resolution has been increased by 2x on both width and height, hence 4x more pixels (4 MSAA subpixels per 1 pixel)

When we analyze how the subpixels are averaged together:
Untitled03.png
Untitled03.png (3.61 KiB) Viewed 12492 times
You will see that 3 some pixels have 1 subpixel covered (25% grey), 4 pixels (two pairs) have 2 subpixels covered (50% grey), 4 pixels have 3 subpixels covered (75% grey) and only 2 pixels have 4 subpixels (hence will be rendered as 100% black)

You may get better results by increasing the line width to something like 1.5, but you can't make the full problem go away.

Complementary you may use a custom MSAA resolve which may look into neighbour pixels and apply an exponent (i.e. pow) to the final result, so that pixels are shifted towards 100% or 0% if we get e.g. 3 subpixels covered but turns out our neighbour has 1 subpixel covered (although this assumes you know how to distinguish uncovered pixels from covered ones, which is something you need to do yourself using something else, e.g. a stencil buffer or storing data into the alpha channel to say 'this subpixel was touched by a line')
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

Thanks for the reply dark_sylinc,

And here is where my problem lies:
After averaging the 4 subpixels, you get 4 shades of grey: 0% (black) 25% 75% and 100% (white)
When I use a transparent viewport background color it mean I don't want the AA result to be influenced by that color so grey is not something that should show up for a white line if the final image has a white background. The AA fails (me) because it blends my line color with a different color. I need to keep whatever color I have for that line intact. The alpha channel on the other hand can & is blending fine as far as I can tell but since the color is already corrupted no matter what type of pass\texture blending is used it's already too late.

The only way I can think of achieving this is to have each line (or group of lines) render to a separate RTT with its viewport background set to these line's color so the AA doesn't produce these color artifacts. Then use that RTT as a texture in my scene.

Thanks for the reply
Shem
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

Below are two samples of the same line going through the the same RTT AA. One line is white and the other is black. The viewport background is transparent white. You can see the difference.
I'll write back again after I'm done doing my workaround.

White line:


Black Line:
Shem
Halfling
Posts: 55
Joined: Fri Feb 16, 2018 9:32 am
x 8

Re: Thin lines on an RTT

Post by Shem »

Yup, that did the trick. My 1 pixel lines look silky smooth. Added another RTT layer with a viewport background color that matches the line color and Ta-dah - I can now change the line color on the fly without any color artifacts. Hope this helps someone.

Thank you Ogre team for taking the time and trying to help a noob out
Shem
Post Reply