RT Shader System Component (RTSS)

[Nir Hasson]

Will the RT shader system be able to handle things like HardwareSkinning which need a special 'shadow_caster_vertex_program_ref' in material file for shadows??

**EDIT** Thinking about it, the skeleton demo works with the RT shader system on (Although the shadows dont work perfectly)

Well – actually when a certain technique is not pure FFP technique the RTSS won’t generate shaders for it – it will fail in the createShaderBasedTechnique method. This behavior is the desgined behaviour of the system.

In case of the SkeletalAnimation sample the technqiue used for rendering the animated charater contains vertex program for the doing the animation on the GPU (Ogre/HardwareSkinningTwoWeights) – so the RTSS doesn’t create a shader based technique for it.

Regarding the shadows - it works as separate modulative pass using ogre core shadow caster/receiver materials and agasin - both don’t go thru the RTSS pipe.
One point you raised here is the need of shader based extension for hardware skining – I’ll add it to the task list…

[al2950]

Thank you!!

One last thing, for now , will it be able to support MRT's!?

[Nir Hasson]

will it be able to support MRT's!?

I haven’t actually tested it yet but I guess this just requires providing SubRenderState extension that creates multiple fragment shader colour outputs
and assigning the matching values for each one of them… So basically you can do it right now.

[Nir Hasson]

I extended the RTSS to support multiple scene managers.
The interface changes are quite minor – I replaced the setSceneManager with addSceneManager and removeSceneManager methods.
In order to reduce the overhead of GPU program switching that will caused due to different scene managers settings it is highly recommended to create
different target schemes names for each scene manager.

I.E – say I have two scene managers A and B.
When I want to create shader based technique for some material I will call the createShaderBasedTechnique twice -

Code: Select all

// Create shader based technique for scene manager A.
RTShader::ShaderGenerator::getSingleton().createShaderBasedTechnique(“MyMaterial”, MaterialManager::DEFAULT_SCHEME_NAME, mSceneManagerA->getName() + RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

// Create shader based technique for scene manager B.
RTShader::ShaderGenerator::getSingleton().createShaderBasedTechnique(“MyMaterial”, MaterialManager::DEFAULT_SCHEME_NAME, mSceneManagerB->getName() + RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

To switch to shader generated techniques I’ll simply change to material scheme name of each of my viewports to the namesI set in the previous calls.

Code: Select all

// Switch viewport of scene manager A to use shader based techniques.
mViewportA->setMaterialScheme(mSceneManagerA->getName() + RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

// Switch viewport of scene manager B to use shader based techniques.
mViewportB->setMaterialScheme(mSceneManagerB->getName() + RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

I didn’t test it in multiple scene manager scenario so I’d like to get some feedback from those of you who work with multiple managers…

[CABAListic]

There currently is a fundamental issue with the Shader component. The problem is that the component contains a Singleton, but is built as a static library. However, the library is linked into every sample DLL, so that each sample has its own instance of the Shader component which can cause all sorts of problems. On Linux it produces segfaults when ending the SampleBrowser.
The question is how to resolve it. The simplest answer is that the Shader component is changed to build as a shared library. Another solution would be to remove the Shader component from the samples and only include in the one sample dedicated to the component. The third way would be to get rid of the singleton, but I don't know if this is easily doable.

(BTW, as far as I'm aware if we enable a static build for the whole of Ogre, then the samples are still built as dynamic libraries, therefore this same problem then probably also applies to OgreMain...)

[Assaf Raman]

What about linking it to OGRE and not the samples in this case?

[Nir Hasson]

I don’t have Linux machine – so I can’t check it out at my place
Can you provide more details – what exactly causes this problem?

Regarding the suggested solutions -
It is very important to keep the RTShader System active in all samples because it is the best way to check its FFP emulation caps.
Getting rid of the singleton will disable the material parsing capabilities – and beside I think this system perfectly fits to the singleton pattern – there is no need of another instance of it.
Making it a shared library – infect it was a shared library until some point where all the components turned into static library – as far as I remember the reason was something related to Mac OS issues – so Sinbad change all components to be static libs.

Bottom line – I suggest to trace what exactly cause this issue – maybe it’s related to the way dynamic libs are handled in Linux – I really don’t know. But we should provide a focused fix after we isolate the problem. As you said this issue might raise when using OgreMain as static lib as well – so my guess it’s some kind of platform issue rather than specific component code..

On windows I see that each sample DLL perform the initialization and finalization of the system in its own context – and there is no problem upon browser exit.
You can create a small test library that will implement a dummy singleton – then use it in one of the samples and see if it causes any problem…
I don’t have much experience with the Linux debugger – but in the worst case you can add bunch of log traces in the suspected places in the RTSS (Initialize/Finalize).

Hope this issue will be fixed soon …

[CABAListic]

The debugger doesn't help because what happens is some kind of heap corruption. I'm getting segmentation faults in random places.
Basically what happens is that each sample has its own RT Shader singleton. If this is not in itself a problem, then why is it a singleton at all? But if that's not the issue, then I'll have to look deeper.

[Nir Hasson]

It is a singleton since it is perfectly fits to this design pattern.

There is no need in multiple instances of it. You can think of it as the little brother of MaterialManager or other resource manager which they are all singletons. In practice – getting rid of it will introduce bunch of other logic issues – for example creating shader based technique on multiple shader generators will cause internal material clashes, another thing is the material script parsing logic that requires accessing the Shader generator instance – in case of multiple instances which one should it be??
My point is that the current state of the shader generator being a singleton was a design choice and it should stay like that
Hope you can track the problem ... from what you describe it sounds like a memory overrun that happens somewhere...
Is it happen even if you don't turn on the system ? I mean run the sample browser, choose a sample that is not the Shader System and don't press F2 and quit.
If this scenario doesn't cause crashes the problem lay inside the RTSS otherwise it is something else (I guess...).

[CABAListic]

No, I mean that due to the RT Shader being a static library, every sample has its own instance of the Singleton! Therefore in fact there exists more than one instance of the Shader component. I would assume that this is in fact the problem (because the library is designed to only have one single, global instance), unless you think that this should do no harm.

[Nir Hasson]

Ah – Ok , missed your point.

As far as I know only one sample runs in a given time and since the RTSS cleans every technique it created upon finalization
step that occurs on sample shutdown there is no harm in that sense that every sample owns instance of the shader generator.

[dermont]

(BTW, as far as I'm aware if we enable a static build for the whole of Ogre, then the samples are still built as dynamic libraries, therefore this same problem then probably also applies to OgreMain...)

I think you are mistaken on this, for a static build the samples are built as static libraries. Also on Linux they appear not to be used, rather you appear to be build the SampleBrowser against the Sample headers. Subsequently the build fails since not all Samples are header based. This issue has already been raised on the tracker:
http://sourceforge.net/tracker/?func=de ... tid=302997

On the other issue with Shader Component please can we have all components built dynamically for a dynamic build and statically for a static build, then all these problems will be resolved.

[Nir Hasson]

Regarding the Linux crash -

I just found out in the Samples CMakeLists.txt the following

Code: Select all

# Need to include resource files so that icons are linked
	if (WIN32)
		set(RESOURCE_FILES
    		${OGRE_SOURCE_DIR}/OgreMain/src/WIN32/resource.h
	    	${OGRE_SOURCE_DIR}/OgreMain/src/WIN32/OgreWin32Resources.rc
    		${OGRE_SOURCE_DIR}/OgreMain/src/WIN32/winres.h
		)
		source_group(Resources FILES ${RESOURCE_FILES})
	endif ()
  else ()
	# Link to RT Shader System if enabled.
	if (OGRE_BUILD_COMPONENT_RTSHADERSYSTEM)	
  		set(OGRE_LIBRARIES ${OGRE_LIBRARIES} OgreRTShaderSystem)
	endif () 
  endif ()

It seems that the RTShader System is being appended to Ogre libraries.
Maybe it is the cause to the crash on exit...
I think that in this case the sample browser is linked to the RTSS as well as each sample.

[CABAListic]

Yes, but since all the samples and the Browser make use of the RTShader's source, they need to be linked against the component, otherwise linking would fail.

[Vectrex]

Do you think RT Shader is now ready to get it's own Addons forum? I see the conversation starting to split into many bits especially with people doing shader subsystems.

[CABAListic]

RT Shader is a core component, not an addon. Simply open another thread in the developer forum if necessary

[Vectrex]

RT Shader is a core component, not an addon

Pedantic?.. I?
Rename Addons to 'Addons and Components' then

[CABAListic]

The point is, we don't need a separate forum for every single thing. There is one single thread here in the developer forum. Open some more here, if necessary. Only if there suddenly is a flood of threads about the RT shader component, then it's worth thinking about a separate forum.

[sinbad]

Seconded. If we had a separate forum for every feature we'd have about 200 of them. Just open a new thread.

[CABAListic]

Note: I have for the moment set the RTShader component to be compiled as a shared library on Linux, unless the whole build is declared static. This solves the issues on Linux, but it's not an ideal fix because I haven't figured out why static linking causes the problems in the first place. The way that the samples are using the component, it shouldn't really matter.

[Assaf Raman]

I just reviewed the files in the trunk folder: Samples\Media\RTShaderLib
Well, it seems that all the cg files have exactly the same code as the hlsl files, what do you think about deleting all the hlsl files and renaming all the cg files to cg_hlsl or something of the soft? Do we really need this multiple code?

[handcircus]

Hello,

First time playing around with RTSHader System component (R9471), look great! Couple of small issues that I found though - when doing a fresh build (OSX 10.6), I'm getting failed assets from RTShader::ProgramManager::getSingletonPtr and RTShader::ProgramWriterManager::getSingletonPtr as ms_Singleton remains null. I see that instances are being created in "ShaderGenerator::_initialize" but couldn't actually see where the ms_Singleton value was meant to be initialised? I I've been setting "ms_Singleton=this;" within the constructor for both classes in the meantime and the sample now run great. Apologies if I've missed something obvious here!

Cheers,
Simon

[Nir Hasson]

@Simon

I can’t test it on Mac platform – but it sounds strange.
The singleton attribute ms_Singleton is initialized in the Singleton constructor.
Both ProgramManager and ProgramWriterManager derives from the Singleton and therefore this shouldn’t happen. Try to place a break point at the allocation lines of these instances and follow up the code to see where it breaks.
In worst case – try rebuilding the RTSS, anyways you really should have to use the manual setting solution, it should be automated in the code path...

[masterfalcon]

@Simon

I can’t test it on Mac platform – but it sounds strange.
The singleton attribute ms_Singleton is initialized in the Singleton constructor.
Both ProgramManager and ProgramWriterManager derives from the Singleton and therefore this shouldn’t happen. Try to place a break point at the allocation lines of these instances and follow up the code to see where it breaks.
In worst case – try rebuilding the RTSS, anyways you really should have to use the manual setting solution, it should be automated in the code path...

I played around with this on OS X tonight. I'm also seeing the failed assertions. GDB refuses to resolve symbols in the RTShader code. Oddly, this is the only Ogre component(built as a static library) that has classes that derive from Singleton. I kinda wonder if that has something to do with the issue.

[Nir Hasson]

I played around with this on OS X tonight. I'm also seeing the failed assertions. GDB refuses to resolve symbols in the RTShader code. Oddly, this is the only Ogre component(built as a static library) that has classes that derive from Singleton. I kinda wonder if that has something to do with the issue.

Maybe this is the issue that causes the troubles on Linux upon the samples exist.
CABAListic solved it by turning it into a shared library under that platform.
I think this is subject worth exploring and solve.
I wonder if that issue can be isolated to the Singleton inheritance only.
I.E – if you create static library that contains only one class that derives from the Singleton, would it resolve the static member?