[SOLVED][2.1]Black reflection map in Windows (porting issue)

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

[SOLVED][2.1]Black reflection map in Windows (porting issue)

Post by zxz »

Hello all,

We are in the process of porting our application to Windows. After getting everything to build properly (no small task), I ran into a strange problem with Ogre.

The symptom of the problem is that our PBS reflection maps are drawn black. The actual PBS rendering is working properly if I set a fixed cubemap texture for my datablocks. However, for time-of-day reasons, we sometimes render a dynamic cubemap with a separate compositor workspace. On Windows, this cubemap texture ends up black, while it gets properly rendered on Linux. The fragment shader used when drawing the cubemap does work, as the same shader is used to draw the environment onscreen with quad pass.

Naturally, I fired up RenderDoc to see what is going on. Running under RenderDoc, the rendering is still broken (black reflection cubemap), but the capture playback actually shows the correct result (with cubemap reflection in objects) in the backbuffer, different from the application itself.

Does anybody have any experience with such discrepancies between onscreen rendering and capture playback? What could be going wrong that results in such a difference in results? The main RenderDoc developer mentioned that it ususally happens when you run into undefined behaviour situations, but couldn't offer much more guidance.

Some info:
OS: Windows 10
Rendersystem: GL3Plus
Card: NVIDIA 1060
Driver: 385.xx
Last edited by zxz on Tue Oct 03, 2017 1:43 pm, edited 1 time in total.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [2.1] Black reflection map in Windows (porting issue)

Post by al2950 »

I have had different but similar confusing issues. Basically when checking in RenderDoc check all the mipmaps, as if they have not been updated the top level mipmap be fine but the rest will be black
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Black reflection map in Windows (porting issue)

Post by dark_sylinc »

Are you per chance reading from or writing to textures via the CPU? (i.e. via HardwarePixelBuffer::lock)

You may be assuming the pixels are flat, ignoring the rowPitch value, while RenderDoc always return a flat buffer, which could explain these discrepancies.

Have you ran your process in full Debug? The dx11 debug layer will complain if there's something weird going on.
The most reasonable explanation is that the reflection texture was still bound for rendering as an RTT while it was set as a texture which is invalid and D3D11 will automatically set the texture to null, causing black (while RenderDoc, if it uses a copy, it would render fine)
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: [2.1] Black reflection map in Windows (porting issue)

Post by zxz »

There are no textures being locked or interacted with on the CPU. The dynamic cubemap is rendered with quad passes, with "camera_cubemap_reorient true", using the same shader used to draw the "sky" on screen (like SkyPostprocess in samples), and then mipmaps are generated for it.

Code: Select all

abstract target cubemap_target
  {
    pass clear
    {
      colour_value 0 0 1 1
    }
  
    pass render_quad
    {
      quad_normals camera_direction
      material SkyPostprocess
      camera_cubemap_reorient true
    }
  }
  
  compositor_node CubemapRendererNode
  {
    in 0 cubemap
  
    target cubemap +X : cubemap_target { }
    target cubemap -X : cubemap_target { }
    target cubemap +Y : cubemap_target { }
    target cubemap -Y : cubemap_target { }
    target cubemap +Z : cubemap_target { }
    target cubemap -Z : cubemap_target { pass generate_mipmaps {} }
  }
  
  workspace DynamicCubemapWorkspace
  {
    connect_output CubemapRendererNode 0
  }
Note that all this is working properly in Linux, so it can't be completely wrong, but might be depending on some undefined behaviour or some uninitialized bindings or whatnot.

I am using the GL3Plus rendering system, so I am not sure how D3D11 comes into play. Do you suggest that I use D3D11 instead? I might be able to try using that eventually, if I rewrite some shaders which currentely are written in GLSL. I was hoping to run OpenGL for simplicity and reduced porting workload. Unfortunately our large dependency chain is not yet setup for full debug as an alternative on Windows. Getting things to build in one mode has been enough work at the moment. We will get there eventually..

A tiny RenderDoc dump is available at https://ufile.io/v37zg (3.8Mb). In this dump, a cubemap is rendered into the dynamic cubemap with a trivial shader (which would normally be replaced with a more dynamic sky shader, but this is simpler to show the issue). Then, a single draw call attempts to draw a cube, with a PBS material, using the dynamic cubemap as the PBS reflection map.

In the dump screenshot, you can see that this cube shows up completely black, while in RenderDoc, the backbuffer shows the perfectly reflecting cube when the backbuffer is swapped.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Black reflection map in Windows (porting issue)

Post by dark_sylinc »

Oh, this was GL3Plus in Windows?

What card and drivers were you using in both Linux and Windows? I'm willing to bet it's a driver bug.
And yeah, you should try D3D11. Sometimes hidden bugs are revealed that GL3+ was accepting just fine by accident.

I'll take a look at your capture when I get some time
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: [2.1] Black reflection map in Windows (porting issue)

Post by zxz »

The information is in the original post ;)

EDIT: I was wrong about the card model. It's not running on my regular machine, so my assumption was wrong. Thanks for making me double check :)

NVIDIA driver 385.41, NVIDIA 970 card. Same issue on an older NVIDIA 760 as well.

Cheers
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Black reflection map in Windows (porting issue)

Post by dark_sylinc »

Ooops, sorry. I should read better.

Wow... just took a look at your capture... it is waaaaay too simple.
Looks like a driver bug. GL driver quality in NVIDIA has declined this last year.
It could be a mipmapping problem like al2950 said.

I noticed you're using MSAA. Does the problem go away if you disable MSAA? It would indicate a problem when resolving.
Also could you post the screenshot? I can't get RenderDoc to show me the dump screenshot.

Thanks
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: [2.1] Black reflection map in Windows (porting issue)

Post by zxz »

The screenshot should be shown as the icon of the dump. It''s small, but there's not much to see anyhow. It shows the proper environment map in the background, but the cube (seen in the dump) is pitch black.

The issue persists without MSAA enabled. It also persists if I set the number of mipmaps to 0 during texture creation, and disable the mipmap generation pass in the compositor.
zxz
Gremlin
Posts: 184
Joined: Sat Apr 16, 2016 9:25 pm
x 19

Re: [2.1] Black reflection map in Windows (porting issue)

Post by zxz »

Hi,

I managed to solve this issue. The issue reproduced while running through apitrace (not in renderdoc), and I was able to get much more helpful error messages from there. Apparently the objects were owned by another context, due to our setup with a hidden renderwindow during startup. In Linux this was properly setup to reuse the context when the proper render windows are created, while this was missing in our codepath for windows.

I suppose that the cubemap was created by the initial context, which yielded the errors when it was bound to the framebuffer for rendering with the second context active.
Post Reply