Shader - detect back face facing camera

[kubatp]

Hi everybody,
my problem might be very simple for someone experienced in shaders, but I am fighting with it already for half a day:)
Basically I have a manualObject, which works as a fog above terrain. The fog material has scene_blend alpha_blend and depth_check off.
It all works fine but there are cases (specific camera angles) when I can see the faces of the fog which I shouldnt see. I need to somehow detect that these triangles should not be rendered and set its alpha to 0 in shader.

Do you have any ideas how to do this?

As far as I understand it, these triangles are rendered because the depth_check is off (and I need it off) and they are less transparent (more distinct) because the pixel is rendered with the fog triangles which are on the other side and should be rendered (that is why it is less transparent than the rest of the fog). I think that this means that these are actually the back faces of the triangle? How to detect that the rendered triangle faces the camera the other way and should not be rendered?

Thank you for any advice

[xrgo]

cull_hardware <clockwise|anticlockwise|none> should be enough but if you use none you can detect the backface in pixel shader by using gl_FrontFacing in openGL

[kubatp]

cull_hardware <clockwise|anticlockwise|none> should be enough

I tried that. When I tried clockwise or none, the result is the same, however when I set it to anticlockwise, it did the exact opossite - it renders only those faces I would like to hide. Any ideas why the clockwise does not work?

[xrgo]

then... I think the problem is that those are not backfaces, they are actually frontfaces and the just show because are behind another frontface, so its opacity adds up. If that's the case try disabling the transparency and use scene_blend add

Its a bit hard to see without moving the camera.. if you can post a video that would be great =)

edit: I read wrong, it seems that indeed are backfaces.. mmmm... its really strange

[kubatp]

edit: I read wrong, it seems that indeed are backfaces.. mmmm... its really strange

Ok, I tried the material setting with depth_check on and without cull_hardware setting (so it is default) and it looks like in the attachment. I am surprised that it is transparent when the depth_check and depth_write is turned on?

Any ideas?

[kubatp]

It seems like the problem is not in the back faces rendering but in rendering of two front faces. Basically when camera is in certain angles, it renders the upper part of the fog closer to the camera and it goes further to the lower part of the fog further from the camera. The back faces are not rendered however because two front faces are rendered and blended together the fog is less transparent.

I was trying to play with scene_blend, separate_scene_blend and separate_scene_blend_op as well, but it didnt fix the problem.

Is there a way how to avoid/set scene blending for certain render group but leave it the same for the rest of the scene? Or maybe reset the colour when the view "ray" goes through the back face?

[xrgo]

I think the problem is that those are not backfaces, they are actually frontfaces and the just show because are behind another frontface, so its opacity adds up

It seems like the problem is not in the back faces rendering but in rendering of two front faces

lol I was right...
the only solution that come to my mind is to render that mesh without any transparency in another texture and later in post process blend it to the rest of the scene (which was rendered in another texture), but it doesn't seems like a good solution (performance wise mostly)

maybe someone else has another idea?

[al2950]

If you are trying to create a ground fog effect, I would use a post process effect. I did a very quick google and came across this;
http://in2gpu.com/2014/07/22/create-fog-shader/

It seems to have a nice explanation of how to do fog as a post process and mentions ground fog at the very end.

[kubatp]

If you are trying to create a ground fog effect, I would use a post process effect.

Hi,
thank you for that. I found this already months ago, but it is something else than I need:-/

[boyamer]

Hi,
Try to digg around with depth bias values, maybe that would fix the issue.

[kubatp]

Try to digg around with depth bias values, maybe that would fix the issue.

Hi,
thank you for the advice, but depth_bias probably cannot help here, because the depth_check is turned off

[dark_sylinc]

It took me a while to understand what you were doing and trying to do.

The solution you're trying is not going to work. Alpha blending in order to work properly needs to be rendered back to front.
Right now the problem you have is that when you have the camera like this (ASCII art):

Code: Select all

    /\
C->/  \
  /    \
 /      \
 

Where C is the camera looking over a small mountain of fog, all if fine because it's rendereded back to front.
However when you move the camera to the other side of this mountain, unless you reorder every single triangle, now it's rendered front to back:

Code: Select all

    /\
   /  \ <-C
  /    \
 /      \

You could try to sort each triangle, but this could be expensive. It would be wise to change the approach or flatten the heights to prevent this from happening as much as possible, or better yet, force the camera to look always from above, not from oblique angles.

Alternatively, instead of alpha blending, you could try multiplication (scene_blend modulate) which should be commutative (a * b == b * a)

[kubatp]

Hi Dark Sylinc,
thank you for joining this.

However when you move the camera to the other side of this mountain, unless you reorder every single triangle, now it's rendered front to back:

Would it change anything when I tell you that the scene (and camera movement) is done the way that viewer can never see it from the other side (only from top)?

flatten the heights

Yes, it would fix the problem. It was actually the initial approach, but it didnt look that nice and created some anomalies together with the terrain itself.

[kubatp]

However when you move the camera to the other side of this mountain, unless you reorder every single triangle, now it's rendered front to back:

Would it change anything when I tell you that the scene (and camera movement) is done the way that viewer can never see it from the other side (only from top)?

Are you saying if I reorder the triangles the way that the further ones (from camera) are added to manual object first, it would fix the problem (if I cannot move the camera the other way, which would reverse the distance of triangles from the camera view)?

[kubatp]

Is it a dumb question? Or I am wrong?

[dark_sylinc]

You're not wrong. But I cannot say you'll get what you want. You're trying to solve a problem with a very specific approach. I cannot help you there.

If the camera rotates even a bit, there may be changes in how triangle order should be processed. It's very tricky and complicated.

Furthermore I noticed it looks like there's lighting applied to that fog, which could further explain the problems you're encountering.

[kubatp]

The best would probably be to reorganize it and try it, right?

I hope I dont have the lightning applied. I have in my material/technique/pass this

Code: Select all

lighting off

Why do you think there is lightning applied to the fog?

[kubatp]

I was thinking about another solution to this.
I could detect in vertex shader, if camera is facing back face and if so, change the position of vertex (lower the y position).

Has anyone tried to detect triangle facing camera in shader? I am not a pro in geometry, so any help is appreciated.

[paroj]

I think your mesh just have the wrong vertex winding for some of the triangles. It determines front/ backface and is readable in the fragment shader as
https://www.khronos.org/registry/OpenGL ... cing.xhtml

however you should be able to just use "cull_hardware". If anticlockwise gives you the backfaces, then they must not be visible with clockwise.

[kubatp]

I think your mesh just have the wrong vertex winding for some of the triangles. It determines front/ backface and is readable in the fragment shader as
https://www.khronos.org/registry/OpenGL ... cing.xhtml

Hi Paroj,
thank you for your answer. I am aware of gl_FrontFacing however I use CG shader language. Despite the fact that CG should support this function with the parameter type FACE, it doesnt. It is also reported here https://forums.ogre3d.org/viewtopic.php ... CE#p365728
The error is "error C5108: unknown semantics "FACE".

That is the reason why I ask how to detect this in shader by calculation.
I do not have a problem with culling. I need to modify vertices when the triangle is back faced.

[kubatp]

There is no easy way to find this out?

What I try to find - I have a "terrain" mesh and once it detects (in vertex shader) that the camera is facing the back face of the mesh, it should change the height of the vertex so it doesnt face the back face (I attach screenshot for easier understading).


What I need is to know how to calculate that face is backfacing the camera. I am aware that I need camera_position, normal of the face and probably also view_direction however I dont know how to combine these to find the information I need.
Thank you for any advices...

[kubatp]

The solution I was thinking about (in previous post) also doesnt work in such a way that it would improve the visual result.

I was wondering if any kind of usage of separate_scene_blend and separate_scene_blend_op could help here?

What I basically try to achieve is limitation of alpha channel of the blending mesh when the ray from camera crosses the mesh more than once.

[dark_sylinc]

That's not the problem you're having, as that gets easily fixed with cull_hardware clockwise (or counterclockwise)

This is the problem you're having:
It's two front faces. The only solutions to that, as I already said is:

1. Use an OIT algorithm (hard, You need shader knowledge)
2. Ensure triangles are always drawn back to front (difficulty depends on how your camera plays, you may need to reorder the index buffer every frame)
3. Use a blending mode that is order independent, such as scene_blend modulate or scene_blend add (easy)

[kubatp]

That's not the problem you're having, as that gets easily fixed with cull_hardware clockwise (or counterclockwise)
This is the problem you're having:

Hi Dark Sylinc,
you are right (and I already knew that this is the problem), but it doesnt mean that the solution I drafted is wrong. If you look at your draft, the problem would simply be solved if you lower the right peak (that is the drafted solution I was thinking about) - it would "remove" one front face from the rendering of those texels which were initially rendering two front faces.

Use an OIT algorithm (hard, You need shader knowledge)

Would you mind to tell me more about it (or send a link with description)? I was googling that but it seems like OIT is a set of different algorithms rather than one and you mind have one particular in mind?

Ensure triangles are always drawn back to front (difficulty depends on how your camera plays, you may need to reorder the index buffer every frame)

This is interesting, because I already have that. Basically I create the manual object from the most distant vertices to the closest ones (on X and Z axes). When camera doesnt rotate and look directly to NEGATIVE Z, it should always be rendered from back to front. But it doesnt help or maybe I need to specify something else?

Use a blending mode that is order independent, such as scene_blend modulate or scene_blend add (easy)

Yes, this is easy but how would I make the result transparent to see the meshes below?

[paroj]

Would you mind to tell me more about it (or send a link with description)? I was googling that but it seems like OIT is a set of different algorithms rather than one and you mind have one particular in mind?

I guess the easiest to implement is https://docs.nvidia.com/gameworks/conte ... sample.htm

Yes, this is easy but how would I make the result transparent to see the meshes below?

for instance use RENDER_QUEUE_SKIES_LATE