Persistent 1-pixel gap on right/bottom edge of the render window (windowed mode, ultrawide monitor) — not fixed by disab

[Lax]

Hi @dark_sylinc ,

I'm running into a rendering artifact I can't explain and haven't found reported elsewhere, so I wanted to describe it here in case it rings a bell or points to something in the RenderSystem I should be looking at.

Setup: Direct3D11 RenderSystem, windowed mode (Full Screen=No), running on an ultrawide monitor at 2560x1080. UI is MyGUI (Ogre2Platform), rendered through a compositor pass into the render window's texture.

Symptom: A full-screen quad/overlay that is created to cover the entire render window's client area still leaves a persistent 1 pixel wide sliver uncovered along the right edge and the bottom edge. Whatever is behind that overlay (the 3D scene, or the previous frame's content) shows through in that thin strip. This is completely consistent and reproducible, not a one-off flicker, and it happens regardless of what is actually being drawn — I get the identical gap with a plain black fullscreen quad used purely as a fade-to-black overlay.

What I've already ruled out, in order:

1. MyGUI/application-side coordinate math. I first suspected MyGUI::CoordConverter::convertFromRelative(), which truncates towards zero (int(relative * viewSize)) instead of rounding. That truncation is real and does lose up to 1 px per conversion, and I fixed it. I also switched the fullscreen overlay widgets from being created via relative coordinates (0,0,1,1) to being created directly in absolute pixels from MyGUI::RenderManager::getViewSize(), to remove the conversion entirely. I added logging around widget creation/resize that confirms the widget's absolute pixel rectangle matches RenderWindow::getWidth()/getHeight() exactly, e.g. (0, 0, 2560, 1080), on every occurrence. Despite that exact match, the 1 px gap is still visible on screen. So the overlay's own geometry, and MyGUI's relative-to-pixel conversion, are not the cause — the gap happens after MyGUI has already produced a geometrically correct, edge-to-edge rectangle.

2. Using full screen mode: Does not fix it.

3. Flip-model swapchain + MSAA resolve mismatch. My next suspicion was useFlipMode=Yes together with FSAA=4x MSAA, since flip-model swapchains can't present a multisampled backbuffer directly and need an internal resolve step, and a size mismatch there would produce exactly this kind of edge artifact. I tested both independently:

Code: Select all

Setting FSAA=None — gap still there.
Setting useFlipMode=No (Blt-model swapchain) — gap still there.

So neither the UI layer nor the MSAA/flip-model resolve path seems to be responsible. Given it survives both of those changes, I'm now wondering whether this is tied to the window/backbuffer size calculation itself on an ultrawide resolution in windowed mode — maybe something in how the client area size is queried versus how the swapchain/backbuffer is actually sized. I don't have a lead on which file to look at from here, so any pointer to where in the D3D11 window creation/resize code this discrepancy might originate would be very helpful. I'm happy to add more logging or put together a minimal repro if that's useful.

Best regards,
Lax

[dark_sylinc]

I can think of three reasons:

First, Viewport mismatch. Like, we call SetViewport( 2559, 1049 ). This can be confirmed in RenderDoc.

Second, Top-Left rule. This is a rule meant for the case that when you have two adjacent triangles touching the same pixels (a fairly common case), only one triangle is in charge of rendering those pixels and not both. But it also means that the bottom & right edges of a triangle will never rasterize those pixels even if it's touching them. This is by design and the solution is to enlarge the quad to cover one extra pixel (or was it half a pixel? I often confuse that).

Guardband woes. If you're using our method of using a fullscreen triangle (instead of fullscreen triangle), if the resolution is too high, it may fall outside the guardband. But I don't think this is the case because the guardband on modern GPUs should be enough to reach either 16384x16384 or 32768x32768, and at 2560x3 = 7680, 7680 falls well within the guardband.

Thus, you should check the calls in RenderDoc as a precaution, but it sounds like the problem is the top left rule.