[GSoC 2012] Off-Screen Particles project - continuation

Threads related to Google Summer of Code
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Missing shader implemented alredy ;).

Code: Select all

void OSP_DepthTestForParticles_vp(
        float4 inputPosition      : POSITION,
		float2 texCoord0          : TEXCOORD0,
        out float4 outputPosition : POSITION,
		out float2 outputTexCoord : TEXCOORD0,
        out float outputDepth     : TEXCOORD1, //colour returned if depth test passes

        uniform float4x4 worldViewProjection,
		uniform float4 depthRange
        )
{
        outputPosition = mul(worldViewProjection, inputPosition);
		outputDepth = (outputPosition.z - depthRange.x) * depthRange.w;
		outputTexCoord = texCoord0;
}

void OSP_DepthTestForParticles_fp(
		float2 viewportPosition  : POSITION,
		float2 texCoord          : TEXCOORD0,
        float inputDepth         : TEXCOORD1,
        out float4 outValue      : COLOR,   //linear depth [0, 1]
		
		uniform sampler valueTexture           : register(s0),
		uniform sampler backgroundDepthTexture : register(s1)
		)
{
		float backgroundDepth = tex2D(backgroundDepthTexture, viewportPosition).r;
		if(inputDepth > backgroundDepth){
			discard;
		}
		else{
			outValue = tex2D(valueTexture, texCoord);
		}
}
Where first input texture contains (color+transparency) and second one contains depth.
Now I'm testing it.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

I am finally at the solution (I have just to reverse the y value)
My depth mask needed much more tranformations in case of particles :).
Image
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

YeS!!!
This is the solution (recepie for modifying particle effect to work with compositor:

Code: Select all

void OSP_DepthTestForParticles_vp(
        float4 inputPosition      : POSITION,
		float2 texCoord0          : TEXCOORD0,
        out float4 outputPosition : POSITION,
		out float2 outputTexCoord : TEXCOORD0,
        out float2 outputDepth    : TEXCOORD1,
		out float4 screenPosition : TEXCOORD2,

        uniform float4x4 worldViewProjection,
		uniform float4 depthRange
        )
{
        outputPosition = mul(worldViewProjection, inputPosition);
		screenPosition = outputPosition;
		outputDepth.x = (outputPosition.z - depthRange.x) * depthRange.w;
		outputDepth.y = outputDepth.x;
		outputTexCoord = texCoord0;
}

void OSP_DepthTestForParticles_fp(
		//float2 viewportPosition  : ,
		float2 texCoord          : TEXCOORD0,
        float2 inputDepth        : TEXCOORD1,
		float4 screenPosition    : TEXCOORD2,
        out float4 outValue      : COLOR,
		
		uniform sampler valueTexture           : register(s0),
		uniform sampler backgroundDepthTexture : register(s1)//,
		//uniform	float4 viewportSize
		)
{
		screenPosition /= screenPosition.w;
		float2 depthTexCoord = float2(screenPosition) * float2(0.5f, 0.5f) + float2(0.5f, 0.5f);
		
		float backgroundDepth = tex2D(backgroundDepthTexture, depthTexCoord).x;
		if(inputDepth.x > backgroundDepth){
		    //outValue = float4(1,0,0,0);
			discard;
		}
		else{
			outValue = tex2D(valueTexture, texCoord);//float4(inputDepth.x,0,1-inputDepth.x,0.1);//backgroundDepth,1);//
		}
}
Summary recipe:
To the original material of blended particles, we add 3 items, as in template:

Code: Select all

material MagicSmoke
{
	technique
	{
		pass
		{
			lighting off
			
			separate_scene_blend src_alpha one_minus_src_alpha one one_minus_src_alpha
			
			//scene_blend alpha_blend
			depth_write off
			diffuse vertexcolour
			
			vertex_program_ref OSP_DepthTestForParticles_vp 
            {
					param_named_auto worldViewProjection worldviewproj_matrix   //worldViewProj
					param_named_auto depthRange scene_depth_range  
            }
            fragment_program_ref OSP_DepthTestForParticles_fp    
            {
            }

			texture_unit 0
			{
				//filtering linear linear linear linear
				texture magicsmoke.png
				tex_address_mode clamp
			}
			
			texture_unit 1
			{
				filtering none
				content_type compositor OSP_AllInOne depthDownsampledTexture
				tex_address_mode clamp
			}
			
		}
	}
}
1. separate_scene_blend src_alpha one_minus_src_alpha one one_minus_src_alpha - provides correct alpha modification and alpha-blending on external texture
2. texture_unit 1 with property content_type compositor OSP_AllInOne depthDownsampledTexture gets the depth texture
3. we call vertex shader and pixel shader, passing 2 automatically generated values param_named_auto (this calling will always look the same)

Image
(warning about particle scripts: when we add our own shader, the colour modifier passed from particle script is lnot active (the one that modifies colour with passed time)) - I can write extension that would manage to transfer that animation of colour into shader.
Like I said it was just matter of few hours. Right now I'll clean this script and upload as a template under different name of material (so it does not interfere with original material that is designed for rendering out of the compositor).

This is how rendering looks like with colour background (we can compare it with version from a month ago - maxofdepth downsample is much better than previous (halo effects disappear)).

Image
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Pictures with end of url from 50.JPG to 57.JPG, for example
http://www.bmm.yoyo.pl/everyday_screens ... reen54.JPG, show sligtly altered shaders / compositors:

-With and without trilinear filtering, with simple depth test and with maxofdepth (we can see which halos are less harmfull).
-with sobel filter executed on particle system (this time there ARE sharp edges - ce can even see edges where sprite intersects solid objects - not only whe it is behind these objects).

Worst halo would be in depth test + no filters:
Image

Maxofdepth+linear filtering
Image

View of sobel filter exactly at the moment when particle sprite intersects solid object:
Image

The other pictures on these topics:
link
link
link
link
link

Update and zips with altered versions of compositor all aviable on https://bitbucket.org/KarolBadowski/ogr ... tidyupcode
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Spacegaier asked me to write article on the project summary.

I'm gathering information and soltions I used (and one I am about to use now for extending project with proposed second pass of full resolution render), and other parts that Spacegaier asked me to describe.

Since long time ago I was not able to access wiki-page (error is displayed when I enter the url). I would like to ask if it could be restored under that original adress.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

Watch out for my OGRE related tweets here.
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 136

Re: [GSoC 2012] Off-Screen Particles project

Post by spacegaier »

I just edited your signature to point to the correct wiki page that Assaf posted faster than I could :) .
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

Haha, I edited the signature slower then you... :-)
Watch out for my OGRE related tweets here.
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

This is other topic:
//----------------------------------------------------------------------------------------------------------------------
In this kind of work, finding out how to do something is alwas the bigger (longer time) part than actually type down the result on keyboard (second part is just blink of an eye).
Many of answers could be found after long search (forums, wiki, documentation, code of script parsers).

So I think that It would be nice if I write also second (tutorial) wiki article for people who will look for answers while writing simmmilar projects - to help them find these answers quicker (even total beginners). I've seen on formus many people looking for the same answers - so hopefully it will help someone.
After the summary is done I would like to write new tutorial on separate wiki page and give a link there from original article.

My work is nearly finished (it is after pencil-down for GSoC, but I'll still do some changes to project to adjust and improve it outside GSoC - so article part about solutions will be adjusted too (for example I've just finished the mising shader that was still mising yesterday) ).


There will be 2 articles. That is the reason:
For the project I've been doing, the many solutions were just difficult to find (There were many forum topics about needed techniqes, with more questions about given problem than answers (sometimes only questions without answer) - or just link to advanced application with answer difficult to deduce) - so I think it would be good for community If after this article I would write a second one tutorial article to gather needed information to write similar projects and explain especially this solutions and techniques that were the most hard to find - step afer step with simple examples (extended tutorial for total beginners).

(When I started the project I had only very beginner knowledge about ogre and spent most time on finding how to do something (mainly technical problems with ogre-specific script language that does not let us do everything and needed some ways around)).
Now I feel I can give answers for some harder topics to help other users.
What are adjustments I would like to add after the project and describe also in tutorial:
-Use prepared sobel filter for writing mixed-resolution pass extension (there will be button to turn it on/off).
-Automatic generation of particle material shader.
When someone adds own vertex shader (like I needed to) - particle script colour modifier is deactivated, etc. - the more techniques are supported, the better (for example SSAO does not support colour generation, Deffered Shading supports some of them for solid materials). Here it would be about inheritin some colour modifiers that come from *.particle script.

I would like to ask if someone has intereting link to article about original particle script parser (what is the original structe of vertex/pixel shader generated from cript) - it will help me write this extension to support all modifiers.
Last edited by Karol Badowski 1989 on Tue Aug 21, 2012 1:21 pm, edited 2 times in total.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
duststorm
Minaton
Posts: 921
Joined: Sat Jul 31, 2010 6:29 pm
Location: Belgium
x 80

Re: [GSoC 2012] Off-Screen Particles project

Post by duststorm »

This really is a nice feature. It looks really promising the way it is explained on the class diagrams on the wiki.
I hope you can still get it in that state, even if you will need more time.
Developer @ MakeHuman.org
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

spacegaier wrote:I just edited your signature to point to the correct wiki page that Assaf posted faster than I could :) .
Thanks, both of you :)
Now I'll eat breakfast and spend an hour to slightly transform shader template that was prepared tonight (little change is needed for DX9).
Then immediately move to writing summary and article - I'll try to write as big part as I can till tomorrow morning.
Tutorial and some other parts will be extended gradually with every upgrade and optimalisation.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

I got idea about another extension, not described in the article. NVIDIA only mentioned that it is some interesting issue. I just figured out solution (perhaps this solution could be also used in real-time calculation of shadows created by volume clouds even intersected with light sources like levels of sandwich).

They told they were nearly sure it is not possible to mix additive and blended particles to one off-screen target, but it perhaps might be possible to mix off-screen particles effects with use more than one off-screen terget.

I think I deduced now the multi pass (or MRT) way to do that that. It needs only 2 passes (2-target MRT) of rendering both kinds of particles.

I have one of them ready:
Lets' call first target "blend target"
-every blend particle uses technique:

Code: Select all

separate_scene_blend src_alpha one_minus_src_alpha one one_minus_src_alpha
-every additive particle uses technique:

Code: Select all

separate_scene_blend one one zero one
so it does not change transparency,

What did jut happen? We calculated a texture that will be alpha-blended on the background. Also for additive ones, we calculate how much they shine on further and closer blended particles - it alters the colour of "blend target".

This is how that layer look like (brightness particles influence ONLY off-screen target),
Image
i put it on the background so we could see what's happening on the picture.


Second target for that extension could be rendered also in downsampled mode (and stenciled too):
Lets' call it "brightness target".
Every blend particle decreases brightness influence of next (closer) particles the more oblique it is (the difference is alredy included in first target, so do not worry ;) - it is ok) - lets say that we calculate only impact on background - then blend particle is also an "obstacle" between background and additive particles closer to camera.
Every additive particle increaes this brightness - how much? Well, this depends on current alpha level. of "blend target" alpha channel on this level of depth (we could calculate this alpha again if we use multiple passes instead of MRT).

I will have to test this idea, but initially:
-every blend particle uses technique:

Code: Select all

separate_scene_blend zero zero one one_minus_src_alpha
-every additive particle uses technique:

Code: Select all

separate_scene_blend one_minus_dest_alpha one zero one
How do we put these two together?
First we add "brightness target" to background (actually, we have just calculated shadow casted by particle effect!) then we alpha blend the "blend target" on that surface.


Alternative one-pass way would not be as accurete, but it would use a little cheat for additive particles (transforming their alpha channel based on brightness - but it has to be done on level of shader) - I will check that.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

I've tested this calculated hypothesis - it seems that it is good configuration.
This is how "additive target" would look like in the proposed solution (i put it here without depth test - background is displayed only to see that fumes hide the impact of additive effect on background).
Image
Image
Last edited by Karol Badowski 1989 on Wed Aug 22, 2012 12:45 am, edited 1 time in total.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

I waited for good moment in animation to take this picture: it shows the best comparison of two render targets (additive and blended one).

First is "additive target". Compare it now with first layer "blend target" to see the resemblance :)
ImageImage
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
duststorm
Minaton
Posts: 921
Joined: Sat Jul 31, 2010 6:29 pm
Location: Belgium
x 80

Re: [GSoC 2012] Off-Screen Particles project

Post by duststorm »

Compositing seems to work quite well now. There are no apparent borders on the particle effects.
Developer @ MakeHuman.org
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Last one is with depth test turned off (point of last set of pictures is to show the strategy to mix additive and blended particles).

I'm still transforming depth test to work in dx9 too - somehow in log file i see a notice about *.cg shader that " 'if()' statements are not allowed in current mode" ) - I'm checking if it is really a matter (I am using "If()" statements in some post effects - but this one is render_scene pass, not render_quad pass - also some notice that "discard" command is not allowed in dx9).
I'll update it a sooon as I find the real matter ant fix it ;).
Last edited by Karol Badowski 1989 on Wed Aug 22, 2012 1:34 pm, edited 1 time in total.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

What shader model are you using?
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

https://bitbucket.org/KarolBadowski/ogr ... SP.program

Many of them are using ps_2_x - add ps_3_0 v- and make it the first
Watch out for my OGRE related tweets here.
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Assaf Raman wrote:What shader model are you using?
Do you mean profiles attribute in vertex_program script, or model supported by my graphic card/installed version of directX (if that second one how easiest to check that) - sorry if i sound dumb.

Code: Select all

vertex_program OSP_DepthTestForParticles_vp_cg cg            
{
	source OSP.cg  
    entry_point OSP_DepthTestForParticles_vp    
	
    profiles vs_4_0 vs_2_x vs_1_1 arbvp1
}
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

yes. But you can't use vs_4_0 - shader model 4 is only supported for D3D10 and up, use shader model 3.
Watch out for my OGRE related tweets here.
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Assaf Raman wrote:yes. But you can't use vs_4_0 - shader model 4 is only supported for D3D10 and up, use shader model 3.
Thanks :) I'll start with that.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

Not sure if it will help, but easy to try out.
Watch out for my OGRE related tweets here.
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

Assaf Raman wrote:Not sure if it will help, but easy to try out.
If() statements are enabled again :). It helped.

It seems that in render_quad pass, first profile was rejected and second one (3_0) was chosen so it worked.
However in render_scene pass this happeneded differently (perhaps some mistake in function that chooses best profile). I swapped in fragment program this profile to first place and if() statements are enabled again :)

I think that by accident we discovered a thing that could be fixed (it is supported already for render_quad pass) - i suspect that the code is parsed and profile is rejected for (earlier profiles than 3_0) if there is if() statement (or other unsupported things) in shader and then the next profile is chosen?

Lets put the same in render_scene pass parser too :)
Last edited by Karol Badowski 1989 on Wed Aug 22, 2012 4:37 pm, edited 1 time in total.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: [GSoC 2012] Off-Screen Particles project

Post by Assaf Raman »

Yes, I think if() is not supported in shader model 2.
Watch out for my OGRE related tweets here.
Karol Badowski 1989
Google Summer of Code Student
Google Summer of Code Student
Posts: 185
Joined: Fri Apr 06, 2012 3:04 pm
x 18

Re: [GSoC 2012] Off-Screen Particles project

Post by Karol Badowski 1989 »

To be specific, it was in fragment program, not in vertex program :).

My profile configuration looked like this:

Code: Select all

fragment_program OSP_DepthTestForParticles_fp_cg cg            
{
	source OSP.cg  
    entry_point OSP_DepthTestForParticles_fp
	
    profiles ps_2_0 ps_3_0 ps_1_1 arbfp1                   
}
Worked in render_quad, but not in render_scene ;)

Now it looks like that:

Code: Select all

fragment_program OSP_DepthTestForParticles_fp_cg cg            
{
	source OSP.cg  
    entry_point OSP_DepthTestForParticles_fp
	
    profiles ps_3_0 ps_2_0 ps_1_1 arbfp1                   
}
Chooses right profile in both kinds of pass :)

I hope it helps to find potential bug (/possibility to make profile chooser even better :) ). So it happens in fragment programs.
Google Summer of Code 2012 Student
Topic: "Implementation of Off-Screen Particles"
Project links: Project thread, WIKI page, Code fork for the project
Mentor: Assaf Raman