[4.0.0] Mesh rendering issues on certain OSes

[haloman30]

Ogre Version: 4.0.0 (commit 8b7b3d17310c0243c264706cfb9f66424fbaf877)
Operating System: Windows 10 x64
Render System: Direct3D 11

When having some friends of mine (and myself) testing a build of my project, we ran into a rendering issue that seems operating system specific. On Windows 7 and Windows 11, rendering a basic mesh is fine - but on 10, when any lighting (in this case a directional light) is present, the entire mesh renders with a bunch of holes, that seem to follow the edges of any triangles on the mesh. On other OSes, or when no light is present, everything is fine.

The same issue happens with other meshes which aren't using PBS materials as well, ones with Unlit materials or custom shaders also have the same issue - and similarly, it goes away once the light is removed.

Seems like a deeper OGRE issue as opposed to a project issue (or a Windows 10 issue) given it works fine on older and newer systems. If there is something on my end that would cause this however, I'd appreciate any info!

The visual issue, with a light placed:

The same mesh, with no light placed:

[rpgplayerrobin]

I think dark_sylinc will most likely know what this is, but here are some things that I can think of:

• Do you use some kind of antialiasing technique? In that case, try turning it off and see if it fixes anything.
• Make sure your specific Visual C++ redist is installed on the target computer.
• Have you check that the mesh has valid tangents and normals? Because if tangents OR normals are 0, I get very strange and buggy results in my game as well, and those have to do with lighting. I have a function that goes through new meshes and checks that all its tangents and normals are actually non-zero.

[haloman30]

• Do you use some kind of antialiasing technique? In that case, try turning it off and see if it fixes anything.

No anti-aliasing at the moment.

• Make sure your specific Visual C++ redist is installed on the target computer.

I'm not sure if the exact specific one was used, given the testing was done by others - but I'd imagine it'd be the same ones tested across the board, as they also tested on their own Windows 7 and Windows 11 OSes (which both worked fine).

• Have you check that the mesh has valid tangents and normals? Because if tangents OR normals are 0, I get very strange and buggy results in my game as well, and those have to do with lighting. I have a function that goes through new meshes and checks that all its tangents and normals are actually non-zero.

Importing models is handled via assimp under the hood, which is set to automatically generate tangents as long as normals are present. The game engine will also check flags on the imported model, and will omit normals and tangets from the vertex layout entirely if they aren't present in the mesh. Not sure if that would be enough to avoid this, or if you're saying they need to always be included and be set to something?

[dark_sylinc]

This sort of artifacts are more common when the load action or store action in the Compositor was set to dont_care; which manifests as a bug on more recent NVIDIA (post GeForce 1080) and AMD (post Vega) while rendering fine in older hardware.

Did you try running your app and OgreNext built in Debug? There are more validations and asserts going on that may detect the problem.

RenderDoc may also have something to say. You didn't post Ogre.log, or information about the Hardware it was run on. So there's little to work with.

[rpgplayerrobin]

• Make sure your specific Visual C++ redist is installed on the target computer.

I'm not sure if the exact specific one was used, given the testing was done by others - but I'd imagine it'd be the same ones tested across the board, as they also tested on their own Windows 7 and Windows 11 OSes (which both worked fine).

I would make sure to actually get the correct one, it should exist on your computer where your visual studio version is installed. The Windows 7 and 11 machines might have it installed from some other software, so this does not completely solve this possible problem.

• Have you check that the mesh has valid tangents and normals? Because if tangents OR normals are 0, I get very strange and buggy results in my game as well, and those have to do with lighting. I have a function that goes through new meshes and checks that all its tangents and normals are actually non-zero.

Importing models is handled via assimp under the hood, which is set to automatically generate tangents as long as normals are present. The game engine will also check flags on the imported model, and will omit normals and tangets from the vertex layout entirely if they aren't present in the mesh. Not sure if that would be enough to avoid this, or if you're saying they need to always be included and be set to something?

It doesn't matter if you generate tangents in Blender or by using C++ later on to generate tangents for a mesh, since the tangents generated can be invalid for specific sections of meshes, which is in most cases the 3D artists fault for making geometry that does not make sense.
That is why I in my game validate each mesh that gets created in debug to see if they contain even a single tangent or normal that is 0,0,0, because if they do, that mesh will create visual artifacts because of shaders later on.

Also, as dark_sylinc mentioned RenderDoc, I would if I were you go into Visual Studio Graphics Debugger and check exactly what happens at the pixel where this error happens. Because then you will be able to see exactly what shader is causing what problem for that pixel.
Without that graphics debugger, it would be impossible for me to fix certain shaders issues that I got in my game, so I would say it is time for you to learn it.

Also, check this:
"DontCare: The cache is not initialized. This is the fastest option, and only works glitch-free if you can guarantee you will render to all the pixels in the screen with opaque geometry."
https://ogrecave.github.io/ogre-next/ap ... anges.html
So if you do use that, just make sure to use Clear or Store instead of DontCare.