RT Shader System Component (RTSS)

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Tue Oct 27, 2009 8:49 am

But at least in my case its an NVidia card (Nvidia Quadro FX 1500M).
Was too tired when I wrote that... :lol:
No chance for me to get a Quadro :(
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Tue Oct 27, 2009 1:00 pm

Regarding to Compacting:

For me it makes sense to compact vars which get passed from the vertex shader to pixelshader (combine them).
like compact two vec2 into on vec4 and use it in the pixelshader trough masking (.xy or .zw).
I think this way of compacting can be done outside the shaderwriter class(so we have to write it only once).
Have you any other compacting usecase other that what i described above which i overlook?
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Tue Oct 27, 2009 3:22 pm

I agree that it has to be done outside the writer context.

I guess I'll add some preGpuProgramsCompile method to the ProgramManager and within that context I'll do the compacting which will cause some parameters relocations etc. In future we could add to this preCompile method context some other optimizations to the programs beside the compact operation.
Regarding the use case - there is of course the 3 and 1 usage and also maybe splitting an output vertex shader parameter and re-combines it in the pixel shader using local variable. I have to do some testing to see if it's worth it - but I'll start with the simple cases that need no splitting.
Future compacting may refer to other shaders in the pipeline - geometry, tessellation etc - but that will happen when D3D10/11 RS will be stable enough...
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Tue Oct 27, 2009 6:27 pm

I think i got you.
You mean to pack the normal vector on the vertex shader and for example unpack the normal vector in pixel shader like this Normal.z = sqrt(1.0f - Normal.x^2 - Normal.y^2) to save a field?
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Tue Oct 27, 2009 8:04 pm

I think i got you.
You mean to pack the normal vector on the vertex shader and for example unpack the normal vector in pixel shader like this Normal.z = sqrt(1.0f - Normal.x^2 - Normal.y^2) to save a field?
Not exactly – although this kind of encode decode will be possible by writing custom shader code.
I’ll give an example for what I mean.
Say you’ll have three output slots from the VS –
Two of them are normal and 3d texcoord that needs 3 float components and the other one is another texcoord but this time with 2 floats.
Totally you need 3+3+2 = 8 floats so you can save one output slot by splitting the output in the following way:
Out0.xyz = normal.xyz;
Out0.w = texCoord2d.x;
Out1.xyz = texCoord3d.xyz;
Out1.w = texCoord2d.y;

Then in the pixel shader the 2d texCoord will be assembled again -
lLocalTexCoord2d = float2(In0.w, In1.w).

As I told you before – I have to do some tests and research if this kind of thing worth it.
I also think that the compacting algorithm should be executed only when the original naïve arrangement exceeded the maximum output slot count…
0 x

User avatar
nullsquared
Old One
Posts: 3245
Joined: Tue Apr 24, 2007 8:23 pm
Location: NY, NY, USA

Re: RT Shader System Component

Post by nullsquared » Tue Oct 27, 2009 8:37 pm

Nir Hasson wrote: As I told you before – I have to do some tests and research if this kind of thing worth it.
I also think that the compacting algorithm should be executed only when the original naïve arrangement exceeded the maximum output slot count…
Yeah, don't do that unless you actually run out of slots. There's no point in compacting things if there are plenty of unused slots.
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Wed Oct 28, 2009 12:53 am

I added the Operand class and the ConstantParamters

Code: Select all

			curFuncInvocation = new FunctionInvocation(FFP_FUNC_LIGHT_POINT_DIFFUSE, groupOrder, internalCounter++); 			
			curFuncInvocation->getOperandList().push_back(Operand(mWorldViewMatrix, Operand::OPS_IN));			
			curFuncInvocation->getOperandList().push_back(Operand(mVSInPosition, Operand::OPS_IN));
			curFuncInvocation->getOperandList().push_back(Operand(mWorldViewITMatrix, Operand::OPS_IN));
			curFuncInvocation->getOperandList().push_back(Operand(mVSInNormal, Operand::OPS_IN));
			curFuncInvocation->getOperandList().push_back(Operand(curLightParams->mPosition, Operand::OPS_IN, (Operand::OPM_X |  Operand::OPM_Y | Operand::OPM_Z)));
			curFuncInvocation->getOperandList().push_back(Operand(curLightParams->mAttenuatParams, Operand::OPS_IN));
			curFuncInvocation->getOperandList().push_back(Operand(curLightParams->mDiffuseColour, Operand::OPS_IN, (Operand::OPM_X |  Operand::OPM_Y | Operand::OPM_Z)));					
			curFuncInvocation->getOperandList().push_back(Operand(mVSOutDiffuse, Operand::OPS_IN, (Operand::OPM_X |  Operand::OPM_Y | Operand::OPM_Z)));	
			curFuncInvocation->getOperandList().push_back(Operand(mVSOutDiffuse, Operand::OPS_OUT, (Operand::OPM_X |  Operand::OPM_Y | Operand::OPM_Z)));	
			vsMain->addAtomInstace(curFuncInvocation);	
Color constructing looks like this now

Code: Select all

		curFuncInvocation = new FunctionInvocation(FFP_FUNC_CONSTRUCT, FFP_VS_COLOUR, internalCounter++);
		curFuncInvocation->getOperandList().push_back(Operand(ParameterFactory::createConstParamFloat(0.0), Operand::OPS_IN));
		curFuncInvocation->getOperandList().push_back(Operand(ParameterFactory::createConstParamFloat(0.0), Operand::OPS_IN));
		curFuncInvocation->getOperandList().push_back(Operand(ParameterFactory::createConstParamFloat(0.0), Operand::OPS_IN));
		curFuncInvocation->getOperandList().push_back(Operand(ParameterFactory::createConstParamFloat(0.0), Operand::OPS_IN));
		curFuncInvocation->getOperandList().push_back(Operand(vsSpecular, Operand::OPS_OUT));
		vsMain->addAtomInstace(curFuncInvocation);
I converted everything and i am alomost done except a small bug. All shaders work except when i hover over the GUI then the transperncy is wrong do you know which shader is responsible for the gui hover effect?
this patch is also huge like the last one :)
Attachments
error.jpg
error.jpg (43.58 KiB) Viewed 4321 times
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 28, 2009 7:09 am

WOW - thats great news !!!
Looking forward to merge your patch ...

Regarding the hover thing - I don't remeber which one was it - you know just in the samples we easely cross the 100 count.
What you can do in this situations is clear the cache folder, run the app, wait a little bit and the hover the UI. Now you can go to the cache folder, sort by date modified and the last one is probably yours.
To be sure you can alter the cached shader – add something like modulate with red to the end of the PS and you’ll able to identify where it goes in the scene.
0 x

dermont
Orc Shaman
Posts: 780
Joined: Thu Dec 09, 2004 2:51 am
x 9

Re: RT Shader System Component

Post by dermont » Wed Oct 28, 2009 10:52 am

Samples/ShaderSystem.cpp has a few references to "Knot.mesh" instead of "knot.mesh".

On Linux I'm running the ShaderSample from the sdk installed in a non-standard location and it crashes with the following error:

Code: Select all

17:36:53: OGRE EXCEPTION(6:FileNotFoundException): Cannot locate resource ../share/OGRE/media/RTShaderLib/3664747110_VS.cg in resource group General or any other group. in ResourceGroupManager::openResource at /media/sda2/Development/OGRE/svn-trunk/OgreMain/src/OgreResourceGroupManager.cpp (line 752)
17:36:53: High-level program 3664747110_VS encountered an error during loading and is thus not supported.
OGRE EXCEPTION(6:FileNotFoundException): Cannot locate resource ../share/OGRE/media/RTShaderLib/3664747110_VS.cg in resource group General or any other group. in ResourceGroupManager::openResource at /media/sda2/Development/OGRE/svn-trunk/OgreMain/src/OgreResourceGroupManager.cpp (line 752)
17:36:53: OGRE EXCEPTION(2:InvalidParametersException): Could not create gpu programs from render state  in ProgramManager::acquireGpuPrograms at /media/sda2/Development/OGRE/svn-trunk/Components/RTShaderSystem/src/OgreShaderProgramManager.cpp (line 107)


If I update the RTShaderLib media to an absolute path in resources.cfg, then the demo runs fine, e.g:

Code: Select all

FileSystem=/media/sda2/Development/OGRE/sdk/svn-trunk/share/OGRE/media/RTShaderLib
#../share/OGRE/media/RTShaderLib
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 28, 2009 10:57 am

Thanks for sharing that info -

Regarding the knot.mesh I'll fix it.
Regarding the resource path - if you'll use the cmake it will build an absoulute path for you... so that should be solved.
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Wed Oct 28, 2009 1:37 pm

I can see a significant difference in the skeletal animation demo when switching between RTSS and FFP. The main light in the center seem to change its colour...

xad
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 28, 2009 2:01 pm

Yeap - there is.

There are two lights in this demo - both of them spot lights.
Now - when you toggle the per pixel lighting - pressing F3 - you'll see the spots in a clear way.

When you use the per-vertex lighting you may expect to get the same results as the FFP but since I've changed the default target profiles of the RT Shader System to include PS3.0 you get this odd result.

I didn't explore it too much but it seems that the interpolators of the COLOR slot under PS3.0 works a bit different than their brothers from older versions.
Add this line of code to setupContent method of the skeletal sample and see it yourself.

Code: Select all

 mShaderGenerator->setFragmentShaderProfiles("ps_2_0");
The same shader code looks significantly different using different target profiles – that fact led me to the conclusion I mentioned above.
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Thu Oct 29, 2009 11:31 am

Hi xad –

I did some digging regarding that lighting results differences.
I found that spec of Nvidia –
ftp://download.nvidia.com/developer/pre ... odel_3.pdf

On pages 27 and 34 you can see at the row of Interpolated registers that previous pixel shader models used 8+2 and in ps3 there are 10.
Now – the RTSS as well as the FFP uses the two first slots COLOR0 and COLOR1 to output lighting factors – diffuse and specular in respective.
In the older versions these two regs where interpolated using a different scheme and I guess different precision as well.
In the PS3 model – all outputs interpolated in the same way so we get this differences.
The different is for good – we get better precision and better interpolated scheme. :D
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Thu Oct 29, 2009 2:43 pm

Ah, Ok! Thanks for clarification!
Then I propose to change the light colour in the demo. It looks pretty scary now. :lol:

xad
0 x

User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel

Re: RT Shader System Component

Post by Assaf Raman » Thu Oct 29, 2009 2:55 pm

Can you post a scary screenshot?
0 x
Watch out for my OGRE related tweets here.

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Thu Oct 29, 2009 3:02 pm

Sure! Please don´t tell me its just me...
Attachments
rtss.jpg
rtss.jpg (109.18 KiB) Viewed 4217 times
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Thu Oct 29, 2009 6:13 pm

Xad – you really forced me to dig into it - :D
This is what I found:

The original sample light colors where something like (0.5, 0.5, 0.5) for ambient + (0.5, 1.0, 0.5) of the green light and (0.5, 0.5, 1.0) for the blue light. You can easily see that the sum of these three colors is (1.5, 2.0, 2.0).
Now, since the interpolators are different I guess that under PS2 the channels where clamped so at max you got 1.0 for each channel – that is the center where both lights affects. Under PS3 there is no such a clamp and there you get these results.
Anyway I modified this sample a bit and now the light colors are in sync with maximum values… - check it out

BTW: Liked your Photoshop work :wink:
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Fri Oct 30, 2009 2:39 pm

Ah, so finally its the clamping strategy. Nice catch!

I hope it was more motivation then force which kept you digging ;-)

xad
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Fri Oct 30, 2009 4:16 pm

Pure motivation :D
0 x

dermont
Orc Shaman
Posts: 780
Joined: Thu Dec 09, 2004 2:51 am
x 9

Re: RT Shader System Component

Post by dermont » Mon Nov 02, 2009 7:18 am

I think there may be a small typo in OgreShaderParameter.cpp, build fails on Linux:

Code: Select all

/home/dermont/Development/OGRE/svn-trunk/Components/RTShaderSystem/src/OgreShaderParameter.cpp: In member function ‘virtual Ogre::String Ogre::RTShader::ConstParameterFloat::toString() const’:
/home/dermont/Development/OGRE/svn-trunk/Components/RTShaderSystem/src/OgreShaderParameter.cpp:141: error: expected primary-expression before ‘.’ token
make[2]: *** [Components/RTShaderSystem/CMakeFiles/OgreRTShaderSystem.dir/src/OgreShaderParameter.cpp.o] Error 1
make[1]: *** [Components/RTShaderSystem/CMakeFiles/OgreRTShaderSystem.dir/all] Error 2
make: *** [all] Error 2

Code: Select all

Index: OgreShaderParameter.cpp
===================================================================
--- OgreShaderParameter.cpp	(revision 9248)
+++ OgreShaderParameter.cpp	(working copy)
@@ -138,7 +138,8 @@
 			String val = Ogre::StringConverter::toString(mValue);
 
 			// Make sure that float params have always this representation #.#
-			if(val.find(".") == String.npos)
+
+			if(val.find(".") == String::npos)
 			{
 				val += ".0";
 			}
@@ -682,4 +683,4 @@
 }
 
 }
-}
\ No newline at end of file
+}
0 x

User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
Contact:

Re: RT Shader System Component

Post by masterfalcon » Mon Nov 02, 2009 7:56 am

dermont wrote:I think there may be a small typo in OgreShaderParameter.cpp, build fails on Linux:

Code: Select all

/home/dermont/Development/OGRE/svn-trunk/Components/RTShaderSystem/src/OgreShaderParameter.cpp: In member function ‘virtual Ogre::String Ogre::RTShader::ConstParameterFloat::toString() const’:
/home/dermont/Development/OGRE/svn-trunk/Components/RTShaderSystem/src/OgreShaderParameter.cpp:141: error: expected primary-expression before ‘.’ token
make[2]: *** [Components/RTShaderSystem/CMakeFiles/OgreRTShaderSystem.dir/src/OgreShaderParameter.cpp.o] Error 1
make[1]: *** [Components/RTShaderSystem/CMakeFiles/OgreRTShaderSystem.dir/all] Error 2
make: *** [all] Error 2

Code: Select all

Index: OgreShaderParameter.cpp
===================================================================
--- OgreShaderParameter.cpp	(revision 9248)
+++ OgreShaderParameter.cpp	(working copy)
@@ -138,7 +138,8 @@
 			String val = Ogre::StringConverter::toString(mValue);
 
 			// Make sure that float params have always this representation #.#
-			if(val.find(".") == String.npos)
+
+			if(val.find(".") == String::npos)
 			{
 				val += ".0";
 			}
@@ -682,4 +683,4 @@
 }
 
 }
-}
\ No newline at end of file
+}
I had the same failure on OS X and iPhone. Checking in the fix.
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Mon Nov 02, 2009 8:08 am

bah f*#* typo do not checked OSX this time thx
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Tue Nov 03, 2009 8:12 am

I have some problems with glsl in OGRE:
I am trying to use as much vertex attributes i can ( i mean instead the built in ones like gl_Vertex because they already depreacated in new versions) but yesterday i found out that the attribute colour works not for overlays and billboards (it was always something like this (1.0,0,0,0)) when i switched the GLSLWriter to use gl_Color it works correctly.
Also i have a problem where i have 2 samplers textures 2d in GLSLbecause they have same content (Panel_Diffuse twice and the cubemap is bounded correctly) i already debugged in the GLRenderSystem and it seems that different texture where bound in the tex2d stages so it looks ok.(the problem is the Panels_refmask which is used by the envmapping example )
Ah and the normalmap is also not bound ( i mean its bound to the same texture the diffuse one) could that be a problem with PF_8 ?
This are the two main issuses they are left (i mean there some calc diffs between cg and glsl i want to bring in sync ;) )
Any ideas how i can track this issuses better down?
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Tue Nov 03, 2009 8:45 am

Any ideas how i can track this issuses better down?
Yeap - use the cache trick again - you'll have to track down the shaders you wanna debug (its easy in the Shader System Sample), then you can
alter them to whatever you want. In your case - if you wanna make sure a certain texture is bound, you can add to the last PS operation a modulation between the
suspected texel and the output color...
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Tue Nov 03, 2009 8:56 am

I already debugged the shaders this way thats why i conclude that in the two samplers have the same texture bounded ( i assgined each time the different sampler to glfrag and it was the same texture, diffuse base, except for the cubemap)

Btw since yesterday its a little bit harder to find the corresponding VS-FS files :D
0 x

Post Reply