RT Shader System Component (RTSS)

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Tue Oct 20, 2009 11:50 pm

Code: Select all

Index: Samples/Browser/include/SampleBrowser.h
===================================================================
--- Samples/Browser/include/SampleBrowser.h	(revision 9187)
+++ Samples/Browser/include/SampleBrowser.h	(working copy)
@@ -60,6 +60,7 @@
 #include "Terrain.h"
 #include "TextureFX.h"
 #include "Transparency.h"
+#include "ShaderSystem.h"
 
 typedef std::map<std::string, OgreBites::SdkSample *> PluginMap;
 
@@ -727,6 +728,7 @@
             mPluginNameMap["Sample_Terrain"]            = (OgreBites::SdkSample *) OGRE_NEW Sample_Terrain();
             mPluginNameMap["Sample_TextureFX"]          = (OgreBites::SdkSample *) OGRE_NEW Sample_TextureFX();
             mPluginNameMap["Sample_Transparency"]       = (OgreBites::SdkSample *) OGRE_NEW Sample_Transparency();
+			mPluginNameMap["Sample_ShaderSystem"]		= (OgreBites::SdkSample *) OGRE_NEW Sample_ShaderSystem();
 #endif
             
 			createWindow();
There is something missing for the static builds under mac (the above patch shows it).
Nice work :)
0 x

User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
Contact:

Re: RT Shader System Component

Post by masterfalcon » Wed Oct 21, 2009 12:20 am

Good catch wolfman!
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 8:44 am

Thanks guys - I knew I missed something :lol:
0 x

User avatar
boyamer
Orc
Posts: 459
Joined: Sat Jan 24, 2009 11:16 am
Location: Italy

Re: RT Shader System Component

Post by boyamer » Wed Oct 21, 2009 8:59 am

Tried the sample,looks very nice! Would be nice if you create a documentation/tutorials on how to use
it,maybe on how to create out own shader extensions!

Thanks for your hard work.
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 9:05 am

Tried the sample,looks very nice! Would be nice if you create a documentation/tutorials on how to use
it,maybe on how to create out own shader extensions!
Forgot to mention the following -
Regarding the sample itself I'm going to add some custom effect code - I think I'll go for reflection map + mask.
I also intend to add code that shows usage of extended material script loading/serializing.

And I will certainly write detailed documentation in the wiki.
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Wed Oct 21, 2009 10:15 am

For docu in the wiki could you maybe describe which steps do we have to take to make e.g. a GLSL shader writer :)
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 10:19 am

For docu in the wiki could you maybe describe which steps do we have to take to make e.g. a GLSL shader writer
Sure, I'll add general porting notes for other target languages.
I think it will be usefull to do a port for HLSL4 as well..
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Wed Oct 21, 2009 10:23 am

I think it belongs here: When starting the new ShaderSystem sample and then going back to the menu the highlight texture of the buttons has no transparency (at the edges) anymore.
The shader system is activated...

xad
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 10:37 am

I think it belongs here: When starting the new ShaderSystem sample and then going back to the menu the highlight texture of the buttons has no transparency (at the edges) anymore.
The shader system is activated...
Sharp eye :D

Thanks for that - I'll check it out
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 11:20 am

I'd like to thank xad for his sharp eye - you actually found a bug in the FFP Texturing stage :)
I fixed it now - you have to get the latest RT Shader System code...
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Wed Oct 21, 2009 1:01 pm

Hmm, I cannot confirm that this bug is solved with rev. 9189.
I still see this bug. Did you check that on your system?

xad
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Wed Oct 21, 2009 1:02 pm

Have u deleted ur shader cache? I think RTShader sys has to recreate the shaders based on the hash.
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Wed Oct 21, 2009 1:16 pm

Have u deleted ur shader cache? I think RTShader sys has to recreate the shaders based on the hash.
Wolfmanfx is right – every time the system code changes in a way that it affect the output shader sources, the cache should be deleted.
In practice you will manage the cache folder yourself and you may delete it or not every run.
I may add the option that tells the system to use cached files or not. Sometimes - especially when debugging - it is useful thing that the system uses the cached files, you can simply modify them and see the results in the next run...
0 x

User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany

Re: RT Shader System Component

Post by xadhoom » Wed Oct 21, 2009 2:22 pm

Yup, it works! I wasn´t aware of the caching folder...

Thanks Wolf! Nice work Nir! :D
0 x

User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel

Re: RT Shader System Component

Post by Assaf Raman » Wed Oct 21, 2009 10:49 pm

You should add a version number to the cache and add to the version number each time you change the code - this way - you will get a new cache key for the shaders every time the version changes.
0 x
Watch out for my OGRE related tweets here.

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Thu Oct 22, 2009 8:58 am

You should add a version number to the cache and add to the version number each time you change the code - this way - you will get a new cache key for the shaders every time the version changes.
It sounds like good idea ..
But here are some thoughts about it –

-It will add some maintenance overhead to the system itself – you'll have to remember to modify this number each time you modify the code.
-Sometimes the problem will be caused by external shader extension – so where this cache version number supposed to be held? –
Internally in the shader system or outside by the user?
In case of 3d party extension it is obvious that it can be held internally and all responsibility goes for the user and we miss the whole point.

I agree that there is a problem, but it only occurs during development process – as the system become stable the output sources will be exactly the same as the cached files so this whole issue won't be matter.
0 x

User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel

Re: RT Shader System Component

Post by Assaf Raman » Thu Oct 22, 2009 10:22 am

What about this idea - if the exe date have changed - then delete the cache, or - have the exe date as the cache "version value".
Or just delete the cache as a post build event.
0 x
Watch out for my OGRE related tweets here.

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Thu Oct 22, 2009 10:28 am

What about to build the version number into the hashing algorithm so that external extension have implcit a version number inside.

I dont like the idea with exe because its plattform specific to get such informations from files in general.
Btw nir have u see my request on msn to add you because i would ask if you already started with the documenation maybe i can help you out with something.
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Thu Oct 22, 2009 10:33 am

What about this idea - if the exe date have changed - then delete the cache, or - have the exe date as the cache "version value".
Or just delete the cache as a post build event.
I guess it will fit for solutions that links statically both to the RT Shader System and their own custom shader extension code, but what if the modified code come from some dynamic library… Besides – erasing it every time you re-compile for 1000 different reasons will make it somewhat useless.
I think we should leave it to our users choice – they have full control over the cache path settings, they can erase it or create whatever mechanism they want to handle it…
Btw nir have u see my request on msn to add you because i would ask if you already started with the documenation maybe i can help you out with something.
I probably missed it - send me your address in PM and i'll add you..
0 x

al2950
OGRE Expert User
OGRE Expert User
Posts: 1198
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 76

Re: RT Shader System Component

Post by al2950 » Thu Oct 22, 2009 3:34 pm

Just to let you know in case its a bug, I have been playing around with the RT Shader System and the new DeferredShading demo;

The application crashes occasionally when you enable the RT Shader System and Per Pixel Lighting. This crash is due to a CG compilation problem because the number of contant registers is exceeded;

Ogre log...
.............
15:30:01: RTShader::ProgramManager: Can not bind auto param named derived_ambient_light_colour to program named 967295939_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_diffuse_colour to program named 967295939_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_specular_colour to program named 967295939_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_emissive_colour to program named 967295939_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_shininess to program named 967295939_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named derived_ambient_light_colour to program named 2181009168_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_diffuse_colour to program named 2181009168_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_specular_colour to program named 2181009168_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_emissive_colour to program named 2181009168_PS
15:30:01: RTShader::ProgramManager: Can not bind auto param named surface_shininess to program named 2181009168_PS
15:30:01: OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 2179463537_PS: CG ERROR : The compile returned an error.
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
in CgProgram::loadFromSource at ..\..\..\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
15:30:01: High-level program 2179463537_PS encountered an error during loading and is thus not supported.
OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 2179463537_PS: CG ERROR : The compile returned an error.
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
(0) : error C6007: Constant register limit exceeded; more than 32 constant registers needed to compiled program
in CgProgram::loadFromSource at ..\..\..\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
15:30:01: OGRE EXCEPTION(2:InvalidParametersException): Could not create gpu programs from render state in ProgramManager::acquireGpuPrograms at ..\..\..\Components\RTShaderSystem\src\OgreShaderProgramManager.cpp (line 107)
As I said this not does happen all the time and you can and generally only happens when it has to re-compile the shaders when looking at the lights around the athena statue.

PS. (There is a very noticable delay as it recompiles shaders when you look at the statue, should this happen every time you look away and look back?)
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Thu Oct 22, 2009 4:24 pm

The problem might be that the generated shader exced the supported Instructions on your card.

[edit] wait i do not know if RT shader works at all with that demo
0 x

al2950
OGRE Expert User
OGRE Expert User
Posts: 1198
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 76

Re: RT Shader System Component

Post by al2950 » Thu Oct 22, 2009 5:08 pm

True, however i thought the number of shader 'constant registers' where controlled by which profile you use. In this case it looks like the RT Shader is using PS 2.0/a/b. I was just wondering if the RT Shader should handle such a situation a bit more gracefully. Also i was interested as to why it sometimes worked, ie the the RT Shader created a shader that had a sensible number of 'constant registers' and other times it tried to create a shader with too many...

Just curious! :D

PS. This is awesome addition to Ogre, really looking forward to its official release and some instructions on how to integrate your own shader code, ie texture shadows :D
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Thu Oct 22, 2009 6:17 pm

Hi -

Thanks for letting me know - I actually didn't tried it with the DeferredShading demo.
Regarding the target profiles - the default is to use all profiles from higher to lower.
You can also change it using the setVertexShaderProfiles/setFragmentShaderProfiles methods of the ShaderGenerator.
The constant issue depends on your system capabilities - but you got a point there regarding the way compilation errors handled.
One simple solution is to to a fallback to the original FFP material. Another solution I thought about is to remove SubRenderStates by priority -
I.E cancel custom effects try re-compile, in case of no success start to remove FFP stages etc...
What other ways do you suggest to handle this situation ?

Anyway -In my todo list I plan to add some compact mechnism in order to take advantage of output parameters - same problems occur there when you exceed the number of output regs....
PS. This is awesome addition to Ogre, really looking forward to its official release and some instructions on how to integrate your own shader code, ie texture shadows
Yep - that's one useful thing - I did it with my own PSSM system and it works great...
I'll try to work on some docs in the next days before version release - and also add some custom reflection map SubRenderState to the demo so you can see how to work with the system...
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: RT Shader System Component

Post by Wolfmanfx » Thu Oct 22, 2009 10:17 pm

Making some progress

Code: Select all

23:19:04: GLSL compiling: 3184089375_VS
23:19:04: GLSL compiled : 3184089375_VS
vertex shader is compiling but there a lot of things which is a little bit cumbersome in GLSL iam trying to make it work first and write the things down which i am not sure how todo it right.
0 x

User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
Contact:

Re: RT Shader System Component

Post by Nir Hasson » Fri Oct 23, 2009 12:49 am

Wow !! That was fast !! -

I'll talk with you tommorow - will help you with the GLSL progress ...
I hope someone will find time to do HLSL4 port as well - it will be more complicated since VS input must match the VB layout... But as Assaf Raman suggest me - one can solve it using some adapter entry method that will produce dummy parameters for the entry point generated by the system..
0 x

Post Reply