RT Shader System Component (RTSS)

[Praetor]

Static libraries will be problematic. Their symbols are pulled into each system that uses them. I believe I've run into this problem before where an initialized singleton in one compiled object (say OgreMain) was not the same as that in your application that must link with the same static lib.

[Assaf Raman]

I am working on the d3d11 render system - I want to use the RT shader system (RTSS) in a mode that it runs all the time, this requires changes in the sample browser.
Correctly - the RTSS is only created when a sample is loaded - I need a different behavior - I need the RTSS creating shaders from the first frame.
I suggest moving the system to the sample browser - this is what I need and may also solve the static linkage singleton issues.

One other issue I saw is that the RTSS convert the material only after it was rendered once - this is not good for me - I need it to replace the material before that.

[Nir Hasson]

Regarding your first suggestion – you may link the Sample Browser with the RTSS and pass the RTSS instance to the samples upon creation.
That way only one instance of the Shader Generator will live during the Browser lifetime and as you suggested it may solve the other issues under Linux/Mac.

Regarding your second remark – I will modify the ShaderGeneratorTechniqueResolverListener code in order to force shader generation in the calling context.

[CABAListic]

That way only one instance of the Shader Generator will live during the Browser lifetime and as you suggested it may solve the other issues under Linux/Mac.

Unlikely, because all the samples still need to link against the RTShader library. It might, in fact, make the problems worse because the samples might now be fooled into thinking the Singleton is not created. But since RTShader is now built as a dynamic library, anyway, it won't matter.

[Assaf Raman]

Why does all the samples still need to link against the RTShader library?
How can all the samples link to OGRE without a problem - what is different in the RTSS?
BTW: Why not link dynamically if that solves everything?

[Nir Hasson]

The original compiling target of the components was shared/dynamic library.
At some point they were modified to compile as static libs - you can search the change log for the reason - I don't remember what it was exactly. Anyways - since now the RTSS is static library - each sample that wants to use it required to be statically linked to it. OgreMain is compiled as shared lib and that's the difference. In order to make the RTSS compile as dynamic library I have to restore some code – mainly the class export declarations – that was discarded when it was turned into a static library.

[CABAListic]

Why does all the samples still need to link against the RTShader library?
How can all the samples link to OGRE without a problem - what is different in the RTSS?
BTW: Why not link dynamically if that solves everything?

Because OgreMain is built dynamically. If you compile Ogre statically, then all the samples are also compiled statically, in which case you have no problem. It's only when you have multiple shared libraries and/or executables which all link the same static library containing singletons when the trouble starts.

[Assaf Raman]

thanks for the replay.
The solution is clear then - have the same linkage strategy as OGRE for the RTSS.

[dermont]

(BTW, as far as I'm aware if we enable a static build for the whole of Ogre, then the samples are still built as dynamic libraries, therefore this same problem then probably also applies to OgreMain...)

I think you are mistaken on this, for a static build the samples are built as static libraries. Also on Linux they appear not to be used, rather you appear to be build the SampleBrowser against the Sample headers. Subsequently the build fails since not all Samples are header based. This issue has already been raised on the tracker:
http://sourceforge.net/tracker/?func=de ... tid=302997

On the other issue with Shader Component please can we have all components built dynamically for a dynamic build and statically for a static build, then all these problems will be resolved.

Still the same problem exists with a static build and the OceanDemo/ShaderSystem Samples not being header based. Also, looking at SamplePlugin.h:

Code: Select all

// Export macro to export the sample's main dll functions.
#if (OGRE_PLATFORM == OGRE_PLATFORM_WIN32) && !defined(__MINGW32__)
#	define _OgreSampleExport __declspec(dllexport)
#	define _OgreSampleClassExport
#elif defined ( OGRE_GCC_VISIBILITY )
#   define _OgreSampleExport  __attribute__ ((visibility("default")))
#   define _OgreSampleClassExport  __attribute__ ((visibility("default")))
#else
#	define _OgreSampleExport
#	define _OgreSampleClassExport
#endif

Shouldn't the above include a check if OGRE_STATIC_LIB is defined and on Windows not export/import _OgreSampleExport and on Linux (set _OgreSampleExport visibility to hidden - probably not needed since the Sample build is header based).

[sinbad]

Yes, this needs to be fixed. Logged a bug: http://www.ogre3d.org/mantis/view.php?id=259

[trilobite]

I rarely ask for it, but I need help. This error has been posted before in this thread, and some have solved it by fixing paths to the RTShaderLib. I do not think that this is my problem as the path to RTShaderLib is discovered when it is initialized in my app.

Here is what ogre.log reports...

OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 1964735936_FS: CG ERROR : "The compile returned an error."
../../media/RTShaderLib/1964735936_FS.cg(259) : error C1115: unable to find compatible overloaded function "FFP_Construct(float, float4)"
in CgProgram::loadFromSource at ..\..\..\ogre_source\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
13:25:08: High-level program 1964735936_FS encountered an error during loading and is thus not supported.
OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 1964735936_FS: CG ERROR : "The compile returned an error."
../../media/RTShaderLib/1964735936_FS.cg(259) : error C1115: unable to find compatible overloaded function "FFP_Construct(float, float4)"
in CgProgram::loadFromSource at ..\..\..\ogre_source\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
13:25:08: OGRE EXCEPTION(2:InvalidParametersException): Could not create gpu programs from render state in ProgramManager::acquireGpuPrograms at ..\..\..\ogre_source\Components\RTShaderSystem\src\OgreShaderProgramManager.cpp (line 117)
13:25:08: OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 1655149828_VS: CG ERROR : "The compile returned an error."
../../media/RTShaderLib/1655149828_VS.cg(561) : error C1008: undefined variable "worldview_matrix"
../../media/RTShaderLib/1655149828_VS.cg(563) : error C1008: undefined variable "worldview_matrix"
in CgProgram::loadFromSource at ..\..\..\ogre_source\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
13:25:08: High-level program 1655149828_VS encountered an error during loading and is thus not supported.
OGRE EXCEPTION(7:InternalErrorException): Unable to compile Cg program 1655149828_VS: CG ERROR : "The compile returned an error."
../../media/RTShaderLib/1655149828_VS.cg(561) : error C1008: undefined variable "worldview_matrix"
../../media/RTShaderLib/1655149828_VS.cg(563) : error C1008: undefined variable "worldview_matrix"
in CgProgram::loadFromSource at ..\..\..\ogre_source\PlugIns\CgProgramManager\src\OgreCgProgramManagerDll.cpp (line 65)
13:25:08: OGRE EXCEPTION(2:InvalidParametersException): Could not create gpu programs from render state in ProgramManager::acquireGpuPrograms at ..\..\..\ogre_source\Components\RTShaderSystem\src\OgreShaderProgramManager.cpp (line 117)

The shader example that accompanies Ogre 1.7 works flawlessly.

Win7
Ogre 1.7
NVidia 9800 GTX+

Thanks for this wonderful new feature, and any help in getting it installed correctly in my app.

[trilobite]

I solved this mystery, and I am very pleased with the result...

I almost wept tears of joy when my app didn't crash and I saw those smooth spotlights on my terrain. Aren't they lovely?

Here was the problem -- the use of colour_op_ex and alpha_op_ex in one of my materials. Here is what I had...

Code: Select all

//	Using the RTShaderLib,
//	This material causes an OGRE_EXCEPT( Exception::ERR_INVALIDPARAMS) in...
//	void ProgramManager::acquirePrograms(Pass* pass, RenderState* renderState)
//	in OgreShaderProgramManager.cpp
//	Note the use of colour_op_ex and alpha_op_ex
material pb_tbiteship_canopy
{
	technique
	{
		pass
		{
			scene_blend alpha_blend
			depth_write off
			specular 0.501961 0.501961 0.501961 32

			texture_unit
			{
				colour_op_ex source1 src_manual src_current 0.219608 0.439216 0.439216 

				alpha_op_ex source1 src_manual src_current 0.5
			}
		}
	}
}


//	Using the RTShaderLib,
//	This material works well.
//	Note that colour_op_ex and alpha_op_ex are commented out
material pb_tbiteship_canopy
{
	technique
	{
		pass
		{
			scene_blend alpha_blend
			depth_write off
			specular 0.501961 0.501961 0.501961 32

			texture_unit
			{
			//	colour_op_ex source1 src_manual src_current 0.219608 0.439216 0.439216 

			//	alpha_op_ex source1 src_manual src_current 0.5
			}
		}
	}
}

HUGE THANKS TO YOU, NIR!

[Nir Hasson]

mmm... Thanks for that observesion -
I guess I'll have to add the support in these new extensions.
(I worked on 1.6 when developing the RTSS core and they were introducted in 1.7 ... )

[sinbad]

I guess I'll have to add the support in these new extensions.
(I worked on 1.6 when developing the RTSS core and they were introducted in 1.7 ... )

Do you mean colour_op_ex and alpha_op_ex? We've had those since the very first version of Ogre

[Nir Hasson]

No - I'm just confused ,
I already deal with these in the FFP Texturing stage.
I'll have to take this material and see what's wrong with it...

[Nir Hasson]

@trilobite - I found this bug and commited to the 1.7 branch -
The manual blend should work for you now..

[trilobite]

@trilobite - I found this bug and commited to the 1.7 branch -
The manual blend should work for you now..

Acknowledged. I will try it out...
Thank you.

[trilobite]

I downloaded the trunk from svs today and built everything new. Working with all of the latest libraries. Rebuilt app. No success
I am experiencing the same crash with the Direct3D 9 and OpenGL rendersystems.

I did narrow the problem down to only ONE of the two previously mention manual methods... alpha_op_ex.
It is possible that my material script may be written with errors. Perhaps you or others here can verify before you set out on a red herring.
Here are the details:

Code: Select all

//	Using the RTShaderLib downloaded from svn on 01-11-10
//	This material causes an Ogre Exception in
//	void ProgramManager::acquirePrograms(Pass* pass, TargetRenderState* renderState)
//	at:
//		ProgramSet* programSet = renderState->getProgramSet();
//		// Create the GPU programs.
//		if (false == createGpuPrograms(programSet))
//		{
//			OGRE_EXCEPT( Exception::ERR_INVALIDPARAMS, 
//				"Could not create gpu programs from render state ", 
//				"ProgramManager::acquireGpuPrograms" );
//		}
//
//	in OgreShaderProgramManager.cpp	
//	Note that alpha_op_ex is NOT commented

material pb_tbiteship_canopy
{
	technique
	{
		pass
		{
			scene_blend alpha_blend
			depth_write off
			specular 0.501961 0.501961 0.501961 32

			texture_unit
			{
//				colour_op_ex source1 src_manual src_current 0.219608 0.439216 0.439216 
			
				alpha_op_ex source1 src_manual src_current 0.5
			}

		}
	}
}

//	Using the RTShaderLib downloaded from svn on 01-11-10
//	This material behaves without error. The color is expressed as expected. (There is no alpha blending)
//	Note that alpha_op_ex is commented.

material pb_tbiteship_canopy
{
	technique
	{
		pass
		{
			scene_blend alpha_blend
			depth_write off
			specular 0.501961 0.501961 0.501961 32

			texture_unit
			{
				colour_op_ex source1 src_manual src_current 0.219608 0.439216 0.439216 
			
//				alpha_op_ex source1 src_manual src_current 0.5
			}

		}
	}
}

I have had success in my app with the RTShaderLib with per-pixel shading and bump mapping. Haven't tried the reflection mapping. And this is in a multidocument interface that loads, unloads, and reloads the library as scene elements are modified or whole documents are switched. So, from this experience, the library has a great deal of stability. And I might add that the code itself elegant and beautiful, if complex.

Thank you

[Nir Hasson]

Hi - Thanks for that,

I commited to the 1.7 branch, didn't merged with the trunk yet - so you don't have it yet....
Will do it later.

[sinbad]

It was already merged yesterday: http://ogre.svn.sourceforge.net/ogre/?rev=9588&view=rev

[Nir Hasson]

Wow - That merge system is great…

Anyways – I checked it here again in the Bezier sample – just replaced the material with the one you specified and it works fine.
Note that in the bug fix I also added new shader functions to the core shader libs files,
so If you work in another scope of media you might have to update these files with the new ones.
If it still doesn't work - try to re-create it in the sample application scope.

[Assaf Raman]

If I will add the photoshop blending modes - do you want it in 1.7 or in the trunk?
If I will do it - I will want also to add a new sample to demonstrate it.

[Nir Hasson]

Great - add to 1.7 and it will be merged to the trunk

[sinbad]

Just be careful not to destabilise anything - new features should really be going into trunk now.

[xadhoom]

What kind of path information do I need to provide in mShaderGenerator::setShaderCachePath? I tried some obvious pathes ("D:\\", "..\\..\\") but all lead to gpu program compile exceptions.
Tracking it down it excepts first in ResourceGroupManager::openResource called in HighLevelGpuProgram::loadHighLevelImpl.

BTW: There is a minor typo in ShaderGenerator: "mCoreScriptTranslaotr"

xad