Page 1 of 1

Ogre post processing fog question for a mod

Posted: Tue Jan 29, 2019 10:23 pm
by disi
I created a shader for SSAO in Kenshi within the RTT post processing pipeline.
The problem I have is, the atmosphere fog is also in post processing and started by the main compositor in a late render queue for the scene, before any other post processing.

My effect is therefore applied and can be seen through the fog, which does not look so bad if I clip the shader near around the camera. The fog is more or less in certain areas and the SSAO should cut off with the fog-line.

I see parameters like fog-density which I could grab from the engine. For example change the alpha level in regard to the "foglevel" and then blend my shader in (I tried this, but with more or less effect because fog level seems to be hard to apply and seems sometimes random). My guess is they are particles and not fog?

Is this the way to go or is there a fancy Ogre option to just put everything 'fog' in front of my shader? Like the opposite of fog_override? Does this even apply if the fog is post processing?
Default is "false" and the engine does not seem to recognise the option.

I also tested culling and hoped the fog would be seen as something in front of my pixels... nope.

Since this is a mod, I have no access to change anything outside the pipeline. I also don't want to change the fog shader or touch the main compositor.

p.s. picture of the problem:
There should be fog in the background, where only the AO effect is visible

here my compositor:

Code: Select all

compositor_node SSSAO {
	in 0 rt_output
	in 1 rt_input
    
            // Temporary textures as buffer
            texture rt0 target_width_scaled 0.5 target_height_scaled 0.5 PF_A8R8G8B8
            texture rt1 target_width_scaled 0.5 target_height_scaled 0.5 PF_A8R8G8B8
            texture rt2 target_width target_height PF_A8R8G8B8
            
            // create AO
	        target rt0 {
                pass clear {
                    colour_value  1 1 1 1
                }
                pass render_quad {
                    material SSSAO
                    input 1 global_gbuffer 1	// normal
                    input 2 global_gbuffer 2	// depth
                }
            }
            
            // blur the AO
            target rt1 {
                pass clear {
                    colour_value  1 1 1 1
                }
                pass render_quad {
                    material SSAOB
                    input 0 rt0                 // pure ssao output
                }
            }

            // put original texture back in buffer
            target rt_output {
                pass clear {}
                pass render_quad {
                    material SSAOC
                    input 0 rt_input            // original texture
                }
            }            
            
            // combine rt_input and rt1
            target rt_output {
                pass render_quad {
                    material SSAOF
                    input 0 rt1                 // pure ssao output
                }
            }
            
	out 0 rt_output
	out 1 rt_input
}
And the material (the fragprogram only returns the colour)

Code: Select all

fragment_program ssaof_ps hlsl
{
	source SSAOF.hlsl
	target ps_4_0
	entry_point main
    enable_backwards_compatibility true
    default_params {
        shared_params_ref SharedDeferredParams
    }
}

material SSAOF
{
	technique
	{
		pass
		{
        	cull_hardware clockwise                                  // this is post processing, we want everything rendered
			cull_software none                                  // this is post processing, we want everything rendered
			depth_check on                                     // this is post processing, we want everything rendered
            scene_blend modulate                                // multiply the colours
            depth_func less                                    // less = only terrain, equal = OK, greater = off
			depth_write off                                     // this is post processing, we want everything rendered
            //alpha_rejection greater_equal 128                 // here you could filter for bright textures, use with care or stuff may not be drawn at all :)
			vertex_program_ref Posteffects/StdQuad_vp{}
			fragment_program_ref ssaof_ps {}
            texture_unit {
				tex_address_mode clamp
				filtering none 
                colour_op modulate
			}
		}
	}
}

Re: Ogre post processing fog question for a mod

Posted: Thu Jan 31, 2019 6:20 pm
by rpgplayerrobin
I have the same problem in my own game.

Even if you had the whole chain of events and the entire code in front of you, it would still be hard to deal with.
Not only does the fog make it challenging, but ALL transparent objects will also have the same problem.

Would it be bad to touch the main compositor as you mentioned? If you could do this effect before the fog is applied (if it even is in that main compositor), it would probably solve your issue.

is there a fancy Ogre option to just put everything 'fog' in front of my shader?
If the fog is done in a shader or a compositor, you are out of luck. Things as fog override only works for fixed-function as stated here:
https://www.ogre3d.org/docs/manual17/ma ... html#SEC67

If the fog is done in a compositor, it would be possible to just put that compositor after the SSAO compositor.
If the fog is done individually when rendering the objects, I would think it would be impossible to get SSAO working properly with fog.
If the fog is actually a particle effect as you mentioned it could be, it belongs with the transparent objects.

If you would make a compositor chain like this, it would probably work:
1. Render only opaque objects (and objects that use alpha rejection).
2. Render a depth buffer for effects such as SSAO.
3. Render the SSAO compositor.
4. Render only transparent objects.
5. Render the fog compositor.
6. Render all other compositors, such as Bloom, HDR, Gamma, Distortion etc.

Even if I write this, I have no clue on how to do this in an elegant manner in my game.
If I just skip the fog (which is done per object in my project, since some objects should not be affected by fog), I still don't know how to do even simple things such as particle effects over the SSAO.
I feel your struggle.

Re: Ogre post processing fog question for a mod

Posted: Sat Feb 02, 2019 8:01 pm
by disi
Thanks for the reply.
I just went with strength/distance in the pixel shader now. The effect becomes less, the more depth to just about end with the fog.
There seems to be no way to squeeze post effects in between.