[2.2] Mix Compositor Postprocessing effects with Hdr

[Lax]

Hi,

is it possible to mix Mix compositor postprocessing effects like glass etc. with Hdr?
I tried to mix that and got hdr working, but as soon, as I connect glass effect to HdrRenderUi, I get errors, because some channels are not connected.
Its hard to keep an overview over all node connections channels etc. Because hdr already uses some channels and when looking at the compositor postprocress Ogre sample, it also uses some channels for the post processing.

Best Regards
Lax

[rujialiu]

is it possible to mix Mix compositor postprocessing effects like glass etc. with Hdr?

I don't follow your explanations... We're using HDR with 3 post processing effects: Bloom, SMAA and SSAO easily. The only tricky thing I'm aware of, is you need to somehow reverse-tonemap Unlit materials if you render them before tone mapping. I never ran into problems like "some channels are not connected".

[Lax]

Ok maybe I need to be more specific.

I would like to use Hdr but also all the nice compositor effects like glass, old tv, motion blur, ... like in the "Sample_Postprocessing.exe" demo.
The issue, I'm having is that I cannot have both, Hdr and Postprocessing compositors, because the compositor nodes are not compatible.
All my workspaces (sky, background etc. are based on the compositor nodes from the Postprocessing demo.

Now I tried to use Hdr, which works, but then Its no more possible to use the compositor effects.

Here a minimal example, what I mean:

Code: Select all

compositor_node NOWAPbsRenderingNode
{
	texture rt0 target_width target_height PFG_RGBA8_UNORM_SRGB msaa_auto
	texture rt1 target_width target_height PFG_RGBA8_UNORM_SRGB msaa_auto

	target rt0
	{
		pass render_scene
		{
			load
			{
				all				clear
				clear_colour	0.2 0.4 0.6 1
			}
			store
			{
				colour	store_or_resolve
				depth	dont_care
				stencil	dont_care
			}

			overlays			off
			shadows				NOWAShadowNode
		}
	}
	
	out 0 rt0
	out 1 rt1
}

compositor_node NOWAPbsFinalCompositionNode
{
	in 0 rt_output
	in 1 rtN

	target rt_output
	{
		pass render_quad
	        {
			load { all dont_care }
			//Ignore the alpha channel
			material 	Ogre/Copy/4xFP32
	    	        input 0 	rtN
		}
		
		pass render_scene
		{
			lod_update_list	off

			//Render Overlays
			overlays		on
			rq_first		254
			rq_last		255
		}
		
		pass custom MYGUI
		{
		   
		}
	}
}

workspace NOWAPbsWorkspace
{
	connect_output NOWAPbsFinalCompositionNode 0
	connect NOWAPbsRenderingNode 0 NOWAPbsFinalCompositionNode 1
}

Note: The "NOWAPbsFinalCompositionNode" is similiar to the "FinalComposition" from the Postprocessing sample. But not compatible with "HdrRenderUi" from Hdr sample.

With this code, I can switch on/off compositor effects (which are located in another compositor file).

But now when I want to use Hdr, I cannot use "NOWAPbsFinalCompositionNode", because the scene is black. I tried several approaches, but it seems, that the Hdr sample scenario from Ogre is not compatible with compositor effects. In order to get Hdr working, I must change "NOWAPbsFinalCompositionNode" to:

Code: Select all

compositor_node NOWAHdrFinalCompositionNode
{
	in 0 rt_output

	target rt_output
	{
		pass render_scene
		{
			lod_update_list	off

			//Render Overlays
			overlays	on
			rq_first	254
			rq_last	255
		}
		
		pass custom MYGUI
		{
		   
		}
	}
	out 0 rt_output
}

But now the input texture 1 (rtN) and the pass render_quad is missing, which is necessary for compositor effects.

I think, I do not understand the whole compositor nodes system correctly...

So to come back to my question. I would like to mix the compositors from "Sample_Hdr.exe" with "Sample_Postprocessing.exe". How can I accomplish that?

Best regards
Lax

[Lax]

Ok, maybe my description is to fuzy. Here a simple scenario I want to accomplish with compositor nodes:

I tried in a simple scenario via following code:

HdrRenderingNode-->Glass-->HdrPostprocessingNode-->HdrRenderUi

Code: Select all

workspaceDef->connect("HdrRenderingNode", 0, "Glass", 0);
workspaceDef->connect("HdrRenderingNode", 1, "Glass", 1);
workspaceDef->connect("Glass", 0, "HdrPostprocessingNode", 0);
workspaceDef->connect("Glass", 1, "HdrPostprocessingNode", 1);
workspaceDef->connectExternal(0, "HdrPostprocessingNode", 2);
workspaceDef->connect("HdrPostprocessingNode", 0, "HdrRenderUi", 0);

Does not work! I get no error, but just a black screen.

So I tried a different approach:
HdrRenderingNode-->HdrPostprocessingNode-->Glass-->HdrRenderUi

Code: Select all

workspaceDef->connect("HdrRenderingNode", 0, "NOWAHdrPostprocessingNode", 0);
workspaceDef->connect("HdrRenderingNode", 1, "NOWAHdrPostprocessingNode", 1);
workspaceDef->connectExternal(0, "NOWAHdrPostprocessingNode", 2);
workspaceDef->connect("NOWAHdrPostprocessingNode", 0, "Glass", 0);
workspaceDef->connect("Glass", 0, "NOWAHdrFinalCompositionNode", 0);

--> Does not work! Because HdrPostprocessingNode has 3 inputs but only one output

I painted the scenarios on paper in order to better understand the connections, but still have no idea to get it working

Best Regards
Lax

[rujialiu]

Code: Select all

workspaceDef->connect("HdrRenderingNode", 0, "Glass", 0);
workspaceDef->connect("HdrRenderingNode", 1, "Glass", 1);
workspaceDef->connect("Glass", 0, "HdrPostprocessingNode", 0);
workspaceDef->connect("Glass", 1, "HdrPostprocessingNode", 1);
workspaceDef->connectExternal(0, "HdrPostprocessingNode", 2);
workspaceDef->connect("HdrPostprocessingNode", 0, "HdrRenderUi", 0);

Does not work! I get no error, but just a black screen.

Your code looks weird to me... I guess you've misunderstood how the whole compositor system works....
Let's try to understand Postprocessing.compositor first. The start node is PostprocessingSampleStdRenderer, which defines two textures rt0 and rt1. rt0 is the texture containing the (original) rendered scene, and rt1 is just for pingpoing so that other postprocessing nodes don't need to define extra textures if rt0+rt1 is enough for them (that's explained in the compositor's comment).

Then look at the final FinalComposition node. it's input 1 is the texture with rendered scene with all applied effects, we just copy it to input 0 and render overlays on top of it, then it's input 0 becomes the final output of the whole workspace.

When it comes to your use case, you're connecting HdrRenderingNode's output 0 and 1 to Glass's input 0 and 1, respectively. That's wrong. We just saw that Glass's two textures are the actual rendering rtt AND the extra one for pingponging. However, HdrRenderingNode's output 1 is NOT the extra one for pingponging, it's oldLumRt that's intended to hold last frame's luminance in order to do auto-exposure. It's a 1x1 texture and other special properties that should NOT be fed to "Glass" node. You should define another texture, as in PostprocessingSampleStdRenderer, which has exactly the same parameters as rt0, to be used for pingponging by Glass and other postprocessing nodes.

Then, the oldLum texture should be connected to HdrPostprocessingNode as in Hdr's sample, because HdrPostprocessingNode is expecting oldLum rather than a texture for pingpong. Here is my modified code (I didn't test it!):

Code: Select all

workspaceDef->connect("HdrRenderingNode", 0, "Glass", 0);
workspaceDef->connect("HdrRenderingNode", 1, "HdrPostprocessingNode", 1); // this is the oldLum
workspaceDef->connect("HdrRenderingNode", 2, "Glass", 1); // this is the new extra texture for Glass node
workspaceDef->connect("Glass", 0, "HdrPostprocessingNode", 0);

Note that Glass's output 1 is NOT used by HdrPostprocessingNode.

[Lax]

Hi rujialiu,

thanks for the clarification!
I did as you proposed. But still something is wrong. Also what I need to know.
From your code I do not see, where FinalComposition comes into play?

And what is also missing, where the connect external takes place. Because HDrPostprocessingNode's RenderWindow must be connected to HDrPostprocessingNode's input channel 0.

Here is what I tried (I set the whole code to one compositor file for better overview and connected everything in the compositor instead of code):

Code: Select all

compositor_node HdrRenderingNode
{
	texture rt0			target_width target_height PFG_RGBA16_FLOAT msaa_auto explicit_resolve
	texture oldLumRt	           1             1 PFG_R16_FLOAT depth_pool 0
	// This is new:
	texture rt1 		target_width target_height PFG_RGBA8_UNORM_SRGB msaa_auto

	target rt0
	{
		pass render_scene
		{
			load
			{
				all				clear
				//clear_colour	0.2 0.4 0.6 1
				clear_colour	6.667 13.333 20 1
				//clear_colour	13.333 26.667 40 1
			}
			store
			{
				depth			dont_care
				stencil			dont_care
			}
			//lod_update_list	off					//Turn Lod off?
			shadows				ShadowNode
			overlays			off
		}
	}

	//Initialize lumRt1 at startup, to avoid NaNs.
	target oldLumRt
	{
		pass clear
		{
			num_initial 1
			colour_value 0.01 0.01 0.01 1
		}
	}
	
	// This is new:
	target rt1
	{
		//Render opaque stuff
		pass render_scene
		{
			load
			{
				all				clear
				clear_colour	0.2 0.4 0.6 1
			}
			store
			{
				colour	store_or_resolve
				depth	dont_care
				stencil	dont_care
			}
			// 0 - 2 transparent objects would be rendered through the ground and sky would be visible!
			rq_first 	10
			rq_last 	20
			overlays	off
			shadows		ShadowNode
		}

		//Render sky after opaque stuff (performance optimization)
		pass render_quad
		{
			quad_normals	camera_direction
			material 		SkyPostprocess
		}
		
		//Render transparent stuff after sky
		pass render_scene
		{
			overlays	off
			shadows		ShadowNode reuse
		}
	}

	out 0 rt0
	out 1 oldLumRt
	// This is new:
	out 2 rt1
}

compositor_node HdrMsaaResolve
{
	in 0 hdr_msaa_rt
	in 1 oldLumRt

	texture resolvedRt target_width target_height PFG_RGBA16_FLOAT depth_pool 0

	target resolvedRt
	{
		pass render_quad
		{
			load { all dont_care }
			material HDR/Resolve_4xFP32_HDR_Box
			input 0 hdr_msaa_rt
			input 1 oldLumRt
		}
	}

	out 0 resolvedRt
}

compositor_node HdrPostprocessingNode
{
	in 0 rt0
	in 1 lumRt1
	in 2 rt_output

	//For maximum quality use these as FLOAT16_RGBA, however they're slower on
	//AMD cards (0.748ms slower on AMD Radeon HD 7770 at 1920x1080).
	//texture rtBlur0	256 256 PFG_RGBA16_FLOAT	depth_pool 0
	//texture rtBlur1	256 256 PFG_RGBA16_FLOAT	depth_pool 0

	texture rtBlur0	256 256 PFG_R10G10B10A2_UNORM	depth_pool 0
	texture rtBlur1	256 256 PFG_R10G10B10A2_UNORM	depth_pool 0

	texture lumRt0	  1   1 PFG_R16_FLOAT	depth_pool 0
	//texture lumRt1  1   1 PFG_R16_FLOAT	depth_pool 0
	texture rtIter0	 64  64 PFG_R16_FLOAT	depth_pool 0
	texture rtIter1	 16  16 PFG_R16_FLOAT	depth_pool 0
	texture rtIter2	  4   4 PFG_R16_FLOAT	depth_pool 0

	//Calculate luminosity
	target rtIter0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/DownScale01_SumLumStart
			input		0	rt0
		}
	}

	target rtIter1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/DownScale02_SumLumIterative
			input		0	rtIter0
		}
	}

	target rtIter2
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/DownScale02_SumLumIterative
			input		0	rtIter1
		}
	}

	target lumRt0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/DownScale03_SumLumEnd
			input		0	rtIter2
			input		1	lumRt1
		}
	}

	target lumRt1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	Ogre/Copy/1xFP32
			input		0	lumRt0
		}
	}

	//Perform Bloom
	target rtBlur0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BrightPass_Start
			input		0	rt0
			input		1	lumRt0
		}
	}

	target rtBlur1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur0
		}
	}

	target rtBlur0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurV
			input		0	rtBlur1
		}
	}

	target rtBlur1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur0
		}
	}

	target rtBlur0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurV
			input		0	rtBlur1
		}
	}

	target rtBlur1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur0
		}
	}

	target rtBlur0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur1
		}
	}

	target rtBlur1
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur0
		}
	}

	target rtBlur0
	{
		pass render_quad
		{
			load		{ all dont_care }
			material	HDR/BoxBlurH
			input		0	rtBlur1
		}
	}

	target rt_output
	{
		pass render_quad
		{
			load		{ colour dont_care }
			store
			{
				//Required by Metal as the overlays in the next pass will break the render encoder,
				//causing this pass to be resolved and then the overlays would render over a black
				//background (unfortunately, this also causes a performance hit).
				colour store
			}
			material	HDR/FinalToneMapping
			input		0	rt0
			input		1	lumRt0
			input		2	rtBlur0
		}
		
		pass render_quad
	    {
			load { all dont_care }
			//Ignore the alpha channel
			material 	Ogre/Copy/4xFP32
	    	input 0 	rt0
		}
	}

	out 0 rt_output
}

compositor_node Glass
{
	in 0 rt_input
	in 1 rt_output

	custom_id Ogre/Postprocess

	target rt_output
	{
		pass render_quad
		{
			material Postprocess/Glass
			input 0 rt_input
		}
	}
	
	out 0 rt_output
	out 1 rt_input
}

compositor_node FinalComposition
{
	in 0 rt_output
	// Take input texture #1 and use the local name "rtN" for reference in this scope
	in 1 rtN

	target rt_output
	{
		pass render_quad
	        {
			load { all dont_care }
			//Ignore the alpha channel
			material 	Ogre/Copy/4xFP32
	    	        input 0 	rtN
		}
		
		pass render_scene
		{
			lod_update_list	off

			//Render Overlays
			overlays		on
			rq_first		254
			rq_last			255
		}
		
		// Render MYGUI in pass here, so that no effect will be placed on the GUI!
		pass custom MYGUI
		{
		   
		}
	}
}

workspace HdrWorkspace
{
	connect HdrRenderingNode 0		Glass 0
	connect HdrRenderingNode 1		HdrPostprocessingNode 1
	connect HdrRenderingNode 2		Glass 1
	connect Glass 0					HdrPostprocessingNode 0
	connect_output					HdrPostprocessingNode 2
	
	connect	HdrPostprocessingNode 0	FinalComposition 1
}

Unfortunately hdr is not working anymore, the scene is rendered as usual, also with sky, no glass is working and no overlays are visible. The performance is not the best, because I introduced rt1 in HdrRenderingNode, as you proposed.

I think we are close in getting working it, but still something is wrong here.

[hedphelym]

The only tricky thing I'm aware of, is you need to somehow reverse-tonemap Unlit materials if you render them before tone mapping.

I'm wondering if you have some more info around this? We have this issue with unlit materials and HDR, they are not as brightly colored as we expect. Always a bit 'dull' when HDR is enabled, I suspect it is what you mention here.

[rujialiu]

performance is not the best, because I introduced rt1 in HdrRenderingNode, as you proposed.

You don't need to render to rt1 (i.e. no "targer rt1{} part) .... Just declare it
Also, I didn't notice that... are you using MSAA or not? Your script contains a node HdrMsaaResolve but you didn't use it.
There are two compositors in Ogre's sample, one with MSAA explicit resolve and one without. I can't remember the details but I believe my script was written BEFORE the HDR_MSAA variant appeared so I think I didn't use explicit resolve? I don't have much time now but if I were you, I would check if I missed up somethings with HDR.compositor and HDR_MSAA.compositor and then do a capture in RenderDoc. You can easily see whether the glass postprocessing never happend, or it is done, but discarded.

I never used "Glass" node (I even don't remember what it's for) so I don't know whether it's intended to be working before or after tonemapping...

[rujialiu]

The only tricky thing I'm aware of, is you need to somehow reverse-tonemap Unlit materials if you render them before tone mapping.

I'm wondering if you have some more info around this? We have this issue with unlit materials and HDR, they are not as brightly colored as we expect. Always a bit 'dull' when HDR is enabled, I suspect it is what you mention here.

I don't if you're treating unlit materials the same as we do.When we set a Red color in unlit material, we want it to be Red on screen, not affected by HDR's exposure and tonemapping. But the objects are drawn BEFORE tone mapping. So we have to make our own version of "setting unlit material" which reverse-tonemap the color before sending to Ogre, so after tonemapping, it becomes Red again!

But that's the "theoritical solution"
Our use case is very simple we did something simpler instead: we have a carefully fixed exposure and disabled tone mapping, so we just need to adjust unlit material's color multiplier... if you have a complex tonemapper AND use textures for unlit materials... maybe it's easier to draw them AFTER tone mapping?

[hedphelym]

if you have a complex tonemapper AND use textures for unlit materials... maybe it's easier to draw them AFTER tone mapping?

We use unlit with just color, but then also the older 'material' type with custom hlsls, this problem also affect them.
So if we for example set white on them - then it appears gray. I have to fix this on our end. Not entirely sure how yet, but this gives me some pointers on what to look for. Not sure how to draw them after tonemapping though, since the effect is done after all objects are drawn.
I noticed the same behaviour with overlays, but then I found the 'include overlay' in the hdr effect, created a last pass which included them (if I remember correct), then the overlays got drawn with proper color. (you see this in the sample project's hdr script).

[Lax]

Hi rujialiu,

thanks a lot. I got it working now! And by the way understood better how compositors do work in Ogre, so mission accomplished

Best Regards
Lax

[Lax]

Hi dark_sylinc,

In the Hdr Sample the ambient light is set with lower and upper hemisphere. When you look at the 'setAmbientLight' function, upper- and then lower hemisphere should be set:

Code: Select all

sceneManager->setAmbientLight(preset.ambLowerHemisphere, preset.ambUpperHemisphere, sceneManager->getAmbientLightHemisphereDir(), preset.envmapScale);

Is this intended or an error?

Correct should be:

Code: Select all

sceneManager->setAmbientLight(preset.ambUpperHemisphere, preset.ambLowerHemisphere, sceneManager->getAmbientLightHemisphereDir(), preset.envmapScale);

Best Regards
Lax

[hedphelym]

The only tricky thing I'm aware of, is you need to somehow reverse-tonemap Unlit materials if you render them before tone mapping.

I'm wondering if you have some more info around this? We have this issue with unlit materials and HDR, they are not as brightly colored as we expect. Always a bit 'dull' when HDR is enabled, I suspect it is what you mention here.

I don't if you're treating unlit materials the same as we do.When we set a Red color in unlit material, we want it to be Red on screen, not affected by HDR's exposure and tonemapping. But the objects are drawn BEFORE tone mapping. So we have to make our own version of "setting unlit material" which reverse-tonemap the color before sending to Ogre, so after tonemapping, it becomes Red again!

But that's the "theoritical solution"
Our use case is very simple we did something simpler instead: we have a carefully fixed exposure and disabled tone mapping, so we just need to adjust unlit material's color multiplier... if you have a complex tonemapper AND use textures for unlit materials... maybe it's easier to draw them AFTER tone mapping?

I found what's causing it now.
it's this line in the tonemapper hlsl:

Code: Select all

vSample.xyz  = FilmicTonemap( vSample.xyz ) / FilmicTonemap( W );

This causes the unlit \ custom materials to get this 'grey' color - when I want them to be white.
If you comment out this line you will see it looks more 'crisp'.

[dark_sylinc]

I found what's causing it now.
it's this line in the tonemapper hlsl:

Code: Select all

vSample.xyz  = FilmicTonemap( vSample.xyz ) / FilmicTonemap( W );

This causes the unlit \ custom materials to get this 'grey' color - when I want them to be white.
If you comment out this line you will see it looks more 'crisp'.

It's done on purpose.

HDR is meant for "real life" values, and so far there is no known "I-am-not-affected-by-light" object in real life. However there are emissive surfaces (i.e. they emit light) which resemble what unlit does.

But a light-emitting surface in the range of 0-1 isn't very bright; thus you need to make them brighter (i.e. use colour values much above > 1.0).

But this induces other problems, such as these unlit surfaces suddenly becoming bloomed or blowing auto exposure.
Which is why the best solution (specially if these unlit surfaces are meant for debugging or must be rendered "as is in LDR") is to draw them after tonemapping.

Anyway, tonemapping is both an objective and highly subjective matter, thus if you like how the render looks without the division, I am not one to stop you.