RTSS PSSM shadows on terrain

[duststorm]

I'm having problems with my PSSM on terrain setup.
I have it set up with three textures, using mostly the same code as in the Terrain sample and RTSS sample for entities shadows.
The problem is that when the camera is farther away, there is a shadow on the terrain, but when getting closer to the terrain the shadow becomes very light and eventually dissappears. You can see that the high resolution shadow texture stays white.

My issue is with shadows on terrain (the fortress should cast shadows on the terrain). Shadows on entities are working.
Also note that the square in the middle of the fortress is NOT terrain. Look at the shadows on the outsides of the walls. (for example to the left of the fort)

screenshot07132012_020727163_s.jpg

screenshot07132012_020757898_s.jpg

screenshot07132012_022800556_s.jpg

Now, when I descend under the terrain suddenly the 1st shadow texture gets filled. It's as if there is a wrong scaling of the y coordinate to the shader.
I first thought it might be because I had moved the center position of the terrain, but setting the terrain center to ZERO made no difference.


As a reference, this is my shadow code:

Code: Select all

void OgreTestApplication::setupShadows()
{
   #ifdef RTSHADER_SYSTEM_BUILD_EXT_SHADERS
        const Ogre::RTShader::SubRenderStateList& subRenderStateList = mSchemRenderState->getTemplateSubRenderStateList();
        Ogre::RTShader::SubRenderStateListConstIterator it = subRenderStateList.begin();
        Ogre::RTShader::SubRenderStateListConstIterator itEnd = subRenderStateList.end();

        for (; it != itEnd; ++it)
        {
            Ogre::RTShader::SubRenderState* curSubRenderState = *it;

            // This is the pssm3 sub render state -> remove it.
            if (curSubRenderState->getType() == Ogre::RTShader::IntegratedPSSM3::Type)
            {
                mSchemRenderState->removeTemplateSubRenderState(*it);
                break;
            }
        }
#endif
    }

#ifdef RTSHADER_SYSTEM_BUILD_EXT_SHADERS
    // Integrated shadow PSSM with 3 splits. (no depth shadow)
    else if (enableShadows)
    {
        mSceneMgr->setShadowTechnique(Ogre::SHADOWTYPE_TEXTURE_MODULATIVE_INTEGRATED);
        mSceneMgr->setShadowFarDistance(3000);

        // 3 textures per directional light (see http://www.stevestreeting.com/2008/08/21/parallel-split-shadow-maps-are-cool/)
        mSceneMgr->setShadowTextureCountPerLightType(Ogre::Light::LT_DIRECTIONAL, 3);
        mSceneMgr->setShadowTextureSettings(512, 3, Ogre::PF_FLOAT32_R);
   
        mSceneMgr->setShadowTextureSelfShadow(true);
        mSceneMgr->setShadowCasterRenderBackFaces(true);

        // Set up caster material - this is just a standard depth/shadow map caster
        mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster");

        // Disable fog on the caster pass.
        Ogre::MaterialPtr passCaterMaterial = Ogre::MaterialManager::getSingleton().getByName("PSSM/shadow_caster");
        Ogre::Pass* pssmCasterPass = passCaterMaterial->getTechnique(0)->getPass(0);
        pssmCasterPass->setFog(true);

        // shadow camera setup
        Ogre::PSSMShadowCameraSetup* pssmSetup = new Ogre::PSSMShadowCameraSetup();
        if(mPssmSetup)
            delete mPssmSetup;
        mPssmSetup = pssmSetup;

        pssmSetup->calculateSplitPoints(3, 5, mSceneMgr->getShadowFarDistance());
        pssmSetup->setSplitPadding(10);
        pssmSetup->setOptimalAdjustFactor(0, 2);
        pssmSetup->setOptimalAdjustFactor(1, 1);
        pssmSetup->setOptimalAdjustFactor(2, 0.5);

        mSceneMgr->setShadowCameraSetup(Ogre::ShadowCameraSetupPtr(pssmSetup));


        // Set up terrain shadowing
        Ogre::TerrainMaterialGeneratorA::SM2Profile* matProfile;
        matProfile = static_cast<Ogre::TerrainMaterialGeneratorA::SM2Profile*>(
               mTerrainGlobals->getDefaultMaterialGenerator()->getActiveProfile()
            );

        matProfile->setReceiveDynamicShadowsEnabled(true);
        matProfile->setReceiveDynamicShadowsPSSM(mPssmSetup);
        matProfile->setReceiveDynamicShadowsLowLod(false);
    }
#endif

    // Set split points
    Ogre::RTShader::SubRenderState* subRenderState = mShaderGenerator->createSubRenderState(Ogre::RTShader::IntegratedPSSM3::Type);
    Ogre::RTShader::IntegratedPSSM3* pssm3SubRenderState = static_cast<Ogre::RTShader::IntegratedPSSM3*>(subRenderState);
    const Ogre::PSSMShadowCameraSetup::SplitPointList& srcSplitPoints = mPssmSetup->getSplitPoints();
    Ogre::RTShader::IntegratedPSSM3::SplitPointList dstSplitPoints;

    for (unsigned int i=0; i < srcSplitPoints.size(); ++i)
    {
        dstSplitPoints.push_back(srcSplitPoints[i]);
    }

    pssm3SubRenderState->setSplitPoints(dstSplitPoints);
    mSchemRenderState->addTemplateSubRenderState(subRenderState);

    // Invalidate the scheme in order to re-generate all shaders based technique related to this scheme.
    mShaderGenerator->invalidateScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);
}

The scene is simply one terrain tile (same parameters as the sample, 12000 size, 513 resolution).
There is one directional light.

Anyone an idea what might be wrong?

[duststorm]

I can fix it if I leave out this line:

Code: Select all

mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster");

That makes shadows on terrain work properly, but entities are completely in shadow.

screenshot07132012_040338740_s.jpg

I also think this thread is related: http://www.ogre3d.org/forums/viewtopic.php?f=2&t=68947
But I don't fully understand yet what is happening.

Does this mean that shadowing approaches of RTSS and terrain are incompatible?
How can I solve the problem so that I have both terrain and entity shadowing? (preferably using only one set of shadow textures)
I presume the mSceneMgr->setShadowTextureCasterMaterial() call sets the default shadow caster vertex program for all materials. So would it work if I could just modify the generated material for the terrain in code?


This post explains exactly the problem I'm having:
http://www.ogre3d.org/forums/viewtopic. ... 94#p443594

[scrawl]

I never got PSSM to work properly with the default terrain material. I always end up making my own terrain material (mostly copy&pasting TerrainMaterialGeneratorA) and removing all the shadow code / inserting the code that I use for my normal entities. But this is necessary anyway, if you start using more advanced things (point light support for terrain, multi render targets or even deferred shading)

[bstone]

The terrain shaders are not compatible with PSSM/shadow_caster because they expect the shadow caster to account for shadow_scene_depth_range which PSSM/shadow_caster obviously ignores. I believe that's why they don't quite work together.

Try switching PSSM/shadow_caster to Ogre/shadow/depth/caster. You'll have to update the castle to use Ogre/shadow/depth/integrated/pssm as well but don't forget to init pssmSplitPoints for that material when you do.

[duststorm]

Thanks for the great answer. I think you summed up the best two solutions, and I'm starting to get a feel of how it works.
As I'd really love to use the RTSS (at least for everything except the terrain) I think I will create a custom terrainMaterialGenerator.
Ideally I would create a terrain generator that is just a wrapper for RTSS, but given how green I still am in this area I think my best bet is just modifying the terrainGenerator until it works with my PSSM setup.

Integrating the terrain component with RTSS would be really nice, though I feel it's something that's really missing.

[bstone]

Well, I'm not a fan of RTSS and to me basing the terrain component on RTSS would have been a waste. But since it employs material generators I believe a RTSS based one will appear sooner or later.

[duststorm]

See it like this: terrain component uses a material generator. RTSS is exactly that: a configurable material generator with reusable shader techniques. Once someone writes a shader technique for RTSS it can be reused everywhere.
The problem with terrain having its own material generator (which is because RTSS didn't exist yet back when it was created) is that incompatibilities might arise. Imagine that there are a lot of ogre components each using their own material generator, it would become increasingly difficult to integrate things with each other, and all the material generators would contain a lot of duplicated code.

In my opinion it would be better to have one centralized shader repository in Ogre that is guaranteed to be compatible. Something as omnipresent as the material system, but for shaders.
Ofcourse you should always have the option of writing shaders manually, as Ogre is all about customizability. I'm just talking about providing good defaults.

[bstone]

Well, the material generator abstraction in the terrain system is different from the RTSS abstraction. So other components having their own material generators for the purposes similar to the terrain component's would be perfectly legit to me.

[duststorm]

The terrain shaders are not compatible with PSSM/shadow_caster because they expect the shadow caster to account for shadow_scene_depth_range which PSSM/shadow_caster obviously ignores.

I still have a question about this:
I read up a bit on the depth range here
http://www.ogre3d.org/forums/viewtopic.php?f=4&t=52157
http://www.ogre3d.org/forums/viewtopic.php?f=2&t=54614
and am assuming that is used to dynamically set the depth range of the shadow textures only to what is needed to render a frame, possibly increasing the shadow resolution in tighter scenes.

Is this technique not better than the RTSS default PSSM/shadow_caster which always uses a fixed depth? Or is this technique outdated and does it not really result in better results? I wonder whether there is a reason why the simpler version was included in RTSS, as Sinbad added the improved sample with depth range to the playpen already some years ago (before RTSS was conceived).

[bstone]

That technique is better, no doubt here. It offers increased precision for the depth maps and lets you operate on linearized depth. The latter helps with biasing which can be then done using world coordinate units and be more precise as well.

I guess PSSM/shadow_caster uses a simpler method because it was... err... simpler?

[duststorm]

Apparrently the default terrain material generator only generates a shader that contains shadow_scene_depth_range parameters when setReceiveDynamicShadowsDepth() is true, which it is not by default. (and not in my code either).
If setReceiveDynamicShadowsDepth() is false it creates a shader that does what it calls "simple PSSM shadow calculation".
So I assume the technique with variable depth range is what is referred to as "PSSM depth shadows" (I was kind of wondering what the difference was with general PSSM shadows, guess that answers my question then).

But this means that the reason why RTSS shadow caster is not compatible with terrain shadow receiver is something else than the depth_range. I'll try to customize the terrain material generator so that it produces shadow code similar to the RTSS shader and see where I end up.

[bstone]

Well, that actually explains your issue even better

You use the shadow caster that renders depth maps but you told your terrain component that your shadows are "simple" (just black and white). Sorry but I would have never guessed you tried to use it like that. So in the end your terrain multiplies the diffuse component by the shadow depth value instead of comparing it to terrain's depth.

So the quick remedy would be to actually enable dynamic shadows depth for your terrain or disable depth shadows in your RTSS materials. If you opt for the depth shadows then you'll have to deal with the linear<->homogeneous depth inconsistency described above.

[duststorm]

I think I can't quite follow you there.
This far I assumed that my RTSS shadow setup code generated simple non-depth shadowing PSSM shaders (the Ogre::RTShader::IntegratedPSSM3 one). Am I correct here?
This does not mean black-and-white binary shadows (as far as I know), just texture shadowing with fixed depth range, as opposed to dynamic.

Terrain material generator allows to generate three types of shadows:
- simple (shadow texture map, lightmap I presume, multiply some pixel value of a shadow texture with diffuse component)
- simple PSSM
- depth PSSM

I'm using the second, by enabling terrain PSSM shadowing (and supplying the same pssm setup also given to RTSS), and setting depth shadows to false.

I tried setting terrain depth shadows to true (the third option), as per your suggestion, and the result is completely no shadow on the terrain.

I'm kind of confused here. I thought my RTSS shadow setup (the caster and receiver for regular meshes, RTSS generated) was simple PSSM, no depth. Is this assumption wrong?
Because of this assumption I thought the way to fix it for terrain was to create a matching shadow receiver (also PSSM, no depth).
If my RTSS setup does generate a PSSM depth shader then the problem of not receiving terrain shadows is not that there is a varying depth range, but that the conversion between depth ranges differs between RTSS and Terrain shaders.

Also: do I need to do something special to set the terrain shadow caster? I currently do mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster"); in my application. Will this also set the shadow caster material for terrain or not?

[bstone]

I think I can't quite follow you there.
This far I assumed that my RTSS shadow setup code generated simple non-depth shadowing PSSM shaders (the Ogre::RTShader::IntegratedPSSM3 one). Am I correct here?
This does not mean black-and-white binary shadows (as far as I know), just texture shadowing with fixed depth range, as opposed to dynamic.

No. There's no such thing as fixed depth range shadows as far as I know. It's either depth or multiplicative mask. Look at the PSSM/shadow_caster_psCg:

Code: Select all

void shadow_caster_ps(
    float2 depth        : TEXCOORD0,

    out float4 oColour  : COLOR,

    uniform float4 pssmSplitPoints)
{
    float finalDepth = depth.x / depth.y;
    oColour = float4(finalDepth, finalDepth, finalDepth, 1);
}

Clearly it casts the depth.

Terrain material generator allows to generate three types of shadows:
- simple (shadow texture map, lightmap I presume, multiply some pixel value of a shadow texture with diffuse component)
- simple PSSM
- depth PSSM

The first option has nothing to do with lightmaps. Simple shadows mean just one shadow texture with multiplicative mask (the color component is multiplied by the shadow texel to produce output). Simple PSSM means 3 (usually but could be 2 or more) shadow textures with multiplicative masks (no depth). Depth PSSM means 3 (in the default setup) depth map shadow textures.

One important thing here to avoid confusion: the terrain material generator does not generate shadows - it generates code for receiving shadows whereas generating shadows (casting them) is assumed to be done by the default shadow caster material.

I tried setting terrain depth shadows to true (the third option), as per your suggestion, and the result is completely no shadow on the terrain.

That's because the depth values expected by terrain (linearized depth that is normalized with respect to scene depth) does not match those generated by PSSM/shadow_caster (not normalized with respect to scene depth).

I'm kind of confused here. I thought my RTSS shadow setup (the caster and receiver for regular meshes, RTSS generated) was simple PSSM, no depth. Is this assumption wrong?

I'm not sure about your RTSS setup on the shadow receiving end as that depends on how you set that up. But if you use PSSM/shadow_caster as your default shadow casting material then you obviously casting depth shadows. What's your shadow texture format by the way?

If my RTSS setup does generate a PSSM depth shader then the problem of not receiving terrain shadows is not that there is a varying depth range, but that the conversion between depth ranges differs between RTSS and Terrain shaders.

Exactly.

Also: do I need to do something special to set the terrain shadow caster? I currently do mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster"); in my application. Will this also set the shadow caster material for terrain or not?

Yes, it will.

[duststorm]

What's your shadow texture format by the way?

Code: Select all

mSceneMgr->setShadowTextureSettings(512, 3, Ogre::PF_FLOAT32_R);

Thanks for taking the time to help me here
I'm obviously still green when it comes to shader work. I think with your help I will be able to get it working now.

[bstone]

Well, that's one of the things about RTSS that I don't like. It hides all the details behind a nice shell and it's hard to see what is going on unless you dig very deep. For example, I'm not even sure if it's legit to use PSSM/shadow_caster with RTSS' implementation of PSSM because it's not clear what depth range it expects. Furthermore, it seems that the only shadow caster provided by RTSS itself doesn't do any depth mapping at all. It simply returns 0.5 no matter what. So it might not be the end of your quest yet

Code: Select all

mSceneMgr->setShadowTextureSettings(512, 3, Ogre::PF_FLOAT32_R)

Yeah, that's what you need for depth shadow maps. I just thought that you might have used an RGB texture since you were talking about using "simple" shadows.

[duststorm]

I think it should work with PSSM/shadow_caster. I started from the RTSS sample in the samplebrowser, and that is the shadow caster that it used.
In my application shadows work on entities so I presume that is fine.

The only issue is getting those shadows to play nice with terrain.

[bstone]

All right. I've made myself to dig into RTSS and I can confirm that its PSSM implementation indeed deals with plain view space depth. So here's what you could do:

1) Copy OgreTerrainMaterialGeneratorA.* to MyTerrainMaterialGenerator.*
2) Replace TerrainMaterialGeneratorA with MyTerrainMaterialGenerator in MyTerrainMaterialGenerator.*
3) Go to line 1484 of MyTerrainMaterialGenerator.cpp and comment out this block:

Code: Select all

            if (prof->getReceiveDynamicShadowsDepth())
            {
                // make linear
                outStream <<
                    "oLightSpacePos" << i << ".z = (oLightSpacePos" << i << ".z - depthRange" << i << ".x) * depthRange" << i << ".w;\n";

            }

4) Add MyTerrainMaterialGenerator.* to your project.
5) Configure your terrain to use an instance of MyTerrainMaterialGenerator instead of the default one (perhaps with a call to TerrainGlobalOptions::setDefaultMaterialGenerator())

That should marry your terrain with RTSS materials well enough. If you're satisfied with the results then it will certainly pay off to figure out the minimal subset of the changes required to disable the depth linearization, like inheriting from OgreTerrainMaterialGeneratorA and overriding only the relevant shader generation code. This way you'll save yourself from dealing with potential updates to OgreTerrainMaterialGeneratorA in future Ogre versions, if there are any.

[duststorm]

Go to line 1484 of MyTerrainMaterialGenerator.cpp and comment out this block

That's in fact exactly what I was up to at this moment
I also compared both shaders generated from RTSS and terrain (with PSSM and depth shadowing enabled), and I believe they are pretty much similar apart from the depth linearization part you pointed to.
They also both use PCF (Percentage-Closer Filtering) for smoothing out the shadows.

it will certainly pay off to figure out the minimal subset of the changes required to disable the depth linearization, like inheriting from OgreTerrainMaterialGeneratorA and overriding only the relevant shader generation code. This way you'll save yourself from dealing with potential updates to OgreTerrainMaterialGeneratorA in future Ogre versions, if there are any.

Good idea. I'll post my result here in case others want to do the same thing.
There's not much to change but It's kind of tricky, the methods to override are buried deep in protected nested classes.

Really too bad addTechnique() is not virtual. It would have made the specialized materialgenerator very compact.

[bstone]

The PCF implementation in the terrain shader looks better to me. It's faster and handles projection in a way that avoids gradients (according to the comments).

[duststorm]

I'm glad to say it works now

screenshot07152012_235012936_s.jpg

Thanks a lot bstone. I have learned a lot about shadowing, RTSS and the terrain material generator (subjects I have up till now shied away from).

Unfortunately minimizing the replicated code wasn't a complete succes. I had to copy a lot of methods before calls were delegated properly to overridden functions, because a lot of helper methods were not declared virtual (and I had troubles overriding some pure virtual functions).

Actually these are the only methods that have to be overridden:
SM2Profile::addTechnique
ShaderHelper::defaultVpParams
ShaderHelperCg::generateVpDynamicShadowsParams
ShaderHelperCg::generateVpDynamicShadows


Anyway, here is the terrain material generator I used:
OgreTerrainRTSSMaterialGenerator.h

Code: Select all

#ifndef OgreTerrainRTSSMaterialGenerator_H
#define OgreTerrainRTSSMaterialGenerator_H

#include <Terrain/OgreTerrainMaterialGeneratorA.h>

namespace Ogre
{
    /**
      * Custom terrain material generator compatible with the RTSS PSSM shadow
      * technique. A TerrainMaterialGenerator which can cope with normal mapped, specular mapped
      *  terrain.
      *  @note Requires the Cg plugin to render correctly
      **/
    class TerrainRTSSMaterialGenerator : public TerrainMaterialGeneratorA
    {
    public:
        TerrainRTSSMaterialGenerator();
        ~TerrainRTSSMaterialGenerator();

        /** Shader model 2 profile target.
        */
        class SM2Profile : public TerrainMaterialGeneratorA::SM2Profile
        {
        public:
            SM2Profile(TerrainMaterialGenerator* parent, const String& name, const String& desc);
            ~SM2Profile();

            MaterialPtr generate(const Terrain* terrain);
            MaterialPtr generateForCompositeMap(const Terrain* terrain);

        protected:
            virtual void addTechnique(const MaterialPtr& mat, const Terrain* terrain, TechniqueType tt);

            /// Utility class to help with generating shaders for Cg / HLSL.
            class ShaderHelperCg : public TerrainMaterialGeneratorA::SM2Profile::ShaderHelperCg
            {
            public:
                virtual HighLevelGpuProgramPtr generateVertexProgram(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt);
            protected:
                virtual void generateVpHeader(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream);
                virtual void generateVpFooter(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream);
                virtual void generateVertexProgramSource(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream);

                virtual void defaultVpParams(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, const HighLevelGpuProgramPtr& prog);
                virtual uint generateVpDynamicShadowsParams(uint texCoordStart, const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream);
                virtual void generateVpDynamicShadows(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream);
            };
        };

    };

}

#endif // OgreTerrainRTSSMaterialGenerator_H

OgreTerrainRTSSMaterialGenerator.cpp

Code: Select all

#include "OgreTerrainRTSSMaterialGenerator.h"

#include <Terrain/OgreTerrain.h>
#include <OgreMaterialManager.h>
#include <OgreTechnique.h>
#include <OgrePass.h>
#include <OgreTextureUnitState.h>
#include <OgreGpuProgramManager.h>
#include <OgreHighLevelGpuProgramManager.h>
#include <OgreHardwarePixelBuffer.h>
#include <OgreShadowCameraSetupPSSM.h>

namespace Ogre
{
    TerrainRTSSMaterialGenerator::TerrainRTSSMaterialGenerator()
        : TerrainMaterialGeneratorA()
    {
        // Add custom SM2Profile specialisation
        mProfiles.clear();  // TODO - This will have to be changed if TerrainMaterialGeneratorA ever supports more profiles than only CG
        mProfiles.push_back(OGRE_NEW SM2Profile(this, "SM2", "Profile for rendering on Shader Model 2 capable cards (RTSS depth shadows compatible)"));
        // TODO - check hardware capabilities & use fallbacks if required (more profiles needed)
        setActiveProfile(mProfiles[0]);

    }
    //---------------------------------------------------------------------
    TerrainRTSSMaterialGenerator::~TerrainRTSSMaterialGenerator()
    {

    }
    //---------------------------------------------------------------------
    //---------------------------------------------------------------------
    TerrainRTSSMaterialGenerator::SM2Profile::SM2Profile(TerrainMaterialGenerator* parent, const String& name, const String& desc)
        : TerrainMaterialGeneratorA::SM2Profile::SM2Profile(parent, name, desc)
    {

    }
    //---------------------------------------------------------------------
    TerrainRTSSMaterialGenerator::SM2Profile::~SM2Profile()
    {
        // Because the base SM2Profile has no virtual destructor:
        OGRE_DELETE mShaderGen;
    }
    //---------------------------------------------------------------------
    void TerrainRTSSMaterialGenerator::SM2Profile::addTechnique(
        const MaterialPtr& mat, const Terrain* terrain, TechniqueType tt)
    {
        // Initiate specialized mShaderGen
        GpuProgramManager& gmgr = GpuProgramManager::getSingleton();
        HighLevelGpuProgramManager& hmgr = HighLevelGpuProgramManager::getSingleton();
        if (!mShaderGen)
        {
            bool check2x = mLayerNormalMappingEnabled || mLayerParallaxMappingEnabled;
            if (hmgr.isLanguageSupported("cg"))
                mShaderGen = OGRE_NEW TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg();
            else if (hmgr.isLanguageSupported("hlsl") &&
                ((check2x && gmgr.isSyntaxSupported("ps_4_0")) ||
                (check2x && gmgr.isSyntaxSupported("ps_2_x")) ||
                (!check2x && gmgr.isSyntaxSupported("ps_2_0"))))
                mShaderGen = OGRE_NEW ShaderHelperHLSL();
            else if (hmgr.isLanguageSupported("glsl"))
                mShaderGen = OGRE_NEW ShaderHelperGLSL();
            else if (hmgr.isLanguageSupported("glsles"))
                mShaderGen = OGRE_NEW ShaderHelperGLSLES();
            else
            {
                // todo
            }

            // check SM3 features
            mSM3Available = GpuProgramManager::getSingleton().isSyntaxSupported("ps_3_0");
            mSM4Available = GpuProgramManager::getSingleton().isSyntaxSupported("ps_4_0");

        }

        // Unfortunately this doesn't work :(
        /*
        // Default implementation
        TerrainMaterialGeneratorA::SM2Profile::addTechnique(mat, terrain, tt);
        */

        // So we have to replicate the entire method:
        Technique* tech = mat->createTechnique();

        // Only supporting one pass
        Pass* pass = tech->createPass();

                                        // Doesn't delegate to the proper method otherwise
        HighLevelGpuProgramPtr vprog = ((TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg*)mShaderGen)->generateVertexProgram(this, terrain, tt);
        HighLevelGpuProgramPtr fprog = mShaderGen->generateFragmentProgram(this, terrain, tt);

        pass->setVertexProgram(vprog->getName());
        pass->setFragmentProgram(fprog->getName());

        if (tt == HIGH_LOD || tt == RENDER_COMPOSITE_MAP)
        {
            // global normal map
            TextureUnitState* tu = pass->createTextureUnitState();
            tu->setTextureName(terrain->getTerrainNormalMap()->getName());
            tu->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);

            // global colour map
            if (terrain->getGlobalColourMapEnabled() && isGlobalColourMapEnabled())
            {
                tu = pass->createTextureUnitState(terrain->getGlobalColourMap()->getName());
                tu->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);
            }

            // light map
            if (isLightmapEnabled())
            {
                tu = pass->createTextureUnitState(terrain->getLightmap()->getName());
                tu->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);
            }

            // blend maps
            uint maxLayers = getMaxLayers(terrain);
            uint numBlendTextures = std::min(terrain->getBlendTextureCount(maxLayers), terrain->getBlendTextureCount());
            uint numLayers = std::min(maxLayers, static_cast<uint>(terrain->getLayerCount()));
            for (uint i = 0; i < numBlendTextures; ++i)
            {
                tu = pass->createTextureUnitState(terrain->getBlendTextureName(i));
                tu->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);
            }

            // layer textures
            for (uint i = 0; i < numLayers; ++i)
            {
                // diffuse / specular
                pass->createTextureUnitState(terrain->getLayerTextureName(i, 0));
                // normal / height
                pass->createTextureUnitState(terrain->getLayerTextureName(i, 1));
            }

        }
        else
        {
            // LOW_LOD textures
            // composite map
            TextureUnitState* tu = pass->createTextureUnitState();
            tu->setTextureName(terrain->getCompositeMap()->getName());
            tu->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);

            // That's it!

        }

        // Add shadow textures (always at the end)
        if (isShadowingEnabled(tt, terrain))
        {
            uint numTextures = 1;
            if (getReceiveDynamicShadowsPSSM())
            {
                numTextures = getReceiveDynamicShadowsPSSM()->getSplitCount();
            }
            for (uint i = 0; i < numTextures; ++i)
            {
                TextureUnitState* tu = pass->createTextureUnitState();
                tu->setContentType(TextureUnitState::CONTENT_SHADOW);
                tu->setTextureAddressingMode(TextureUnitState::TAM_BORDER);
                tu->setTextureBorderColour(ColourValue::White);
            }
        }
    }

    /**
      * generate() and generateForCompositeMap() are identical to TerrainMaterialGeneratorA implementation,
      * the only reason for repeating them is that, unfortunately, addTechnique() is not declared virtual.
      **/
    //---------------------------------------------------------------------
    MaterialPtr TerrainRTSSMaterialGenerator::SM2Profile::generate(const Terrain* terrain)
    {
        // re-use old material if exists
        MaterialPtr mat = terrain->_getMaterial();
        if (mat.isNull())
        {
            MaterialManager& matMgr = MaterialManager::getSingleton();

            // it's important that the names are deterministic for a given terrain, so
            // use the terrain pointer as an ID
            const String& matName = terrain->getMaterialName();
            mat = matMgr.getByName(matName);
            if (mat.isNull())
            {
                mat = matMgr.create(matName, ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
            }
        }
        // clear everything
        mat->removeAllTechniques();

        // Automatically disable normal & parallax mapping if card cannot handle it
        // We do this rather than having a specific technique for it since it's simpler
        GpuProgramManager& gmgr = GpuProgramManager::getSingleton();
        if (!gmgr.isSyntaxSupported("ps_4_0") && !gmgr.isSyntaxSupported("ps_3_0") && !gmgr.isSyntaxSupported("ps_2_x")
            && !gmgr.isSyntaxSupported("fp40") && !gmgr.isSyntaxSupported("arbfp1"))
        {
            setLayerNormalMappingEnabled(false);
            setLayerParallaxMappingEnabled(false);
        }

        addTechnique(mat, terrain, HIGH_LOD);

        // LOD
        if(mCompositeMapEnabled)
        {
            addTechnique(mat, terrain, LOW_LOD);
            Material::LodValueList lodValues;
            lodValues.push_back(TerrainGlobalOptions::getSingleton().getCompositeMapDistance());
            mat->setLodLevels(lodValues);
            Technique* lowLodTechnique = mat->getTechnique(1);
            lowLodTechnique->setLodIndex(1);
        }

        updateParams(mat, terrain);

        return mat;

    }
    //---------------------------------------------------------------------
    MaterialPtr TerrainRTSSMaterialGenerator::SM2Profile::generateForCompositeMap(const Terrain* terrain)
    {
        // re-use old material if exists
        MaterialPtr mat = terrain->_getCompositeMapMaterial();
        if (mat.isNull())
        {
            MaterialManager& matMgr = MaterialManager::getSingleton();

            // it's important that the names are deterministic for a given terrain, so
            // use the terrain pointer as an ID
            const String& matName = terrain->getMaterialName() + "/comp";
            mat = matMgr.getByName(matName);
            if (mat.isNull())
            {
                mat = matMgr.create(matName, ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME);
            }
        }
        // clear everything
        mat->removeAllTechniques();

        addTechnique(mat, terrain, RENDER_COMPOSITE_MAP);

        updateParamsForCompositeMap(mat, terrain);

        return mat;

    }
    //---------------------------------------------------------------------
    //---------------------------------------------------------------------
    void TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::defaultVpParams(const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, const HighLevelGpuProgramPtr& prog)
    {
        GpuProgramParametersSharedPtr params = prog->getDefaultParameters();
        params->setIgnoreMissingParams(true);
        params->setNamedAutoConstant("worldMatrix", GpuProgramParameters::ACT_WORLD_MATRIX);
        params->setNamedAutoConstant("viewProjMatrix", GpuProgramParameters::ACT_VIEWPROJ_MATRIX);
        params->setNamedAutoConstant("lodMorph", GpuProgramParameters::ACT_CUSTOM,
            Terrain::LOD_MORPH_CUSTOM_PARAM);
        params->setNamedAutoConstant("fogParams", GpuProgramParameters::ACT_FOG_PARAMS);

        if (prof->isShadowingEnabled(tt, terrain))
        {
            uint numTextures = 1;
            if (prof->getReceiveDynamicShadowsPSSM())
            {
                numTextures = prof->getReceiveDynamicShadowsPSSM()->getSplitCount();
            }
            for (uint i = 0; i < numTextures; ++i)
            {
                params->setNamedAutoConstant("texViewProjMatrix" + StringConverter::toString(i),
                    GpuProgramParameters::ACT_TEXTURE_VIEWPROJ_MATRIX, i);
                // Don't add depth range params
                /*
                if (prof->getReceiveDynamicShadowsDepth())
                {
                    params->setNamedAutoConstant("depthRange" + StringConverter::toString(i),
                        GpuProgramParameters::ACT_SHADOW_SCENE_DEPTH_RANGE, i);
                }
                */
            }
        }

        if (terrain->_getUseVertexCompression() && tt != RENDER_COMPOSITE_MAP)
        {
            Matrix4 posIndexToObjectSpace;
            terrain->getPointTransform(&posIndexToObjectSpace);
            params->setNamedConstant("posIndexToObjectSpace", posIndexToObjectSpace);
        }
    }
    //---------------------------------------------------------------------
    void TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVpDynamicShadows(const SM2Profile *prof, const Terrain *terrain, TechniqueType tt, StringUtil::StrStreamType &outStream)
    {
        uint numTextures = 1;
        if (prof->getReceiveDynamicShadowsPSSM())
        {
            numTextures = prof->getReceiveDynamicShadowsPSSM()->getSplitCount();
        }

        // Calculate the position of vertex in light space
        for (uint i = 0; i < numTextures; ++i)
        {
            outStream <<
                "	oLightSpacePos" << i << " = mul(texViewProjMatrix" << i << ", worldPos); \n";

            // Don't linearize depth range: RTSS PSSM implementation uses view-space depth
            /*
            if (prof->getReceiveDynamicShadowsDepth())
            {
                // make linear
                outStream <<
                    "oLightSpacePos" << i << ".z = (oLightSpacePos" << i << ".z - depthRange" << i << ".x) * depthRange" << i << ".w;\n";

            }
            */
        }


        if (prof->getReceiveDynamicShadowsPSSM())
        {
            outStream <<
                "	// pass cam depth\n"
                "	oUVMisc.z = oPos.z;\n";
        }
    }
    //---------------------------------------------------------------------
    uint TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVpDynamicShadowsParams(uint texCoord, const SM2Profile *prof, const Terrain *terrain, TechniqueType tt, StringUtil::StrStreamType &outStream)
    {
        // out semantics & params
        uint numTextures = 1;
        if (prof->getReceiveDynamicShadowsPSSM())
        {
            numTextures = prof->getReceiveDynamicShadowsPSSM()->getSplitCount();
        }
        for (uint i = 0; i < numTextures; ++i)
        {
            outStream <<
                ", out float4 oLightSpacePos" << i << " : TEXCOORD" << texCoord++ << " \n" <<
                ", uniform float4x4 texViewProjMatrix" << i << " \n";

            // Don't add depth range params
            /*
            if (prof->getReceiveDynamicShadowsDepth())
            {
                outStream <<
                    ", uniform float4 depthRange" << i << " // x = min, y = max, z = range, w = 1/range \n";
            }
            */
        }

        return texCoord;
    }
    //---------------------------------------------------------------------

    /**
      * This method is identical to TerrainMaterialGeneratorA::SM2Profile::ShaderHelperCg::generateVpHeader()
      * but is needed because generateVpDynamicShadowsParams() is not declared virtual.
      **/
    void TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVpHeader(
        const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream)
    {
        outStream <<
            "void main_vp(\n";
        bool compression = terrain->_getUseVertexCompression() && tt != RENDER_COMPOSITE_MAP;
        if (compression)
        {
            outStream <<
                "float2 posIndex : POSITION,\n"
                "float height  : TEXCOORD0,\n";
        }
        else
        {
            outStream <<
                "float4 pos : POSITION,\n"
                "float2 uv  : TEXCOORD0,\n";

        }
        if (tt != RENDER_COMPOSITE_MAP)
            outStream << "float2 delta  : TEXCOORD1,\n"; // lodDelta, lodThreshold

        outStream <<
            "uniform float4x4 worldMatrix,\n"
            "uniform float4x4 viewProjMatrix,\n"
            "uniform float2   lodMorph,\n"; // morph amount, morph LOD target

        if (compression)
        {
            outStream <<
                "uniform float4x4   posIndexToObjectSpace,\n"
                "uniform float    baseUVScale,\n";
        }
        // uv multipliers
        uint maxLayers = prof->getMaxLayers(terrain);
        uint numLayers = std::min(maxLayers, static_cast<uint>(terrain->getLayerCount()));
        uint numUVMultipliers = (numLayers / 4);
        if (numLayers % 4)
            ++numUVMultipliers;
        for (uint i = 0; i < numUVMultipliers; ++i)
            outStream << "uniform float4 uvMul_" << i << ", \n";

        outStream <<
            "out float4 oPos : POSITION,\n"
            "out float4 oPosObj : TEXCOORD0 \n";

        uint texCoordSet = 1;
        outStream <<
            ", out float4 oUVMisc : TEXCOORD" << texCoordSet++ <<" // xy = uv, z = camDepth\n";

        // layer UV's premultiplied, packed as xy/zw
        uint numUVSets = numLayers / 2;
        if (numLayers % 2)
            ++numUVSets;
        if (tt != LOW_LOD)
        {
            for (uint i = 0; i < numUVSets; ++i)
            {
                outStream <<
                    ", out float4 oUV" << i << " : TEXCOORD" << texCoordSet++ << "\n";
            }
        }

        if (prof->getParent()->getDebugLevel() && tt != RENDER_COMPOSITE_MAP)
        {
            outStream << ", out float2 lodInfo : TEXCOORD" << texCoordSet++ << "\n";
        }

        bool fog = terrain->getSceneManager()->getFogMode() != FOG_NONE && tt != RENDER_COMPOSITE_MAP;
        if (fog)
        {
            outStream <<
                ", uniform float4 fogParams\n"
                ", out float fogVal : COLOR\n";
        }

        if (prof->isShadowingEnabled(tt, terrain))
        {
            texCoordSet = generateVpDynamicShadowsParams(texCoordSet, prof, terrain, tt, outStream);
        }

        // check we haven't exceeded texture coordinates
        if (texCoordSet > 8)
        {
            OGRE_EXCEPT(Exception::ERR_INVALIDPARAMS,
                "Requested options require too many texture coordinate sets! Try reducing the number of layers.",
                __FUNCTION__);
        }

        outStream <<
            ")\n"
            "{\n";
        if (compression)
        {
            outStream <<
                "	float4 pos;\n"
                "	pos = mul(posIndexToObjectSpace, float4(posIndex, height, 1));\n"
                "   float2 uv = float2(posIndex.x * baseUVScale, 1.0 - (posIndex.y * baseUVScale));\n";
        }
        outStream <<
            "	float4 worldPos = mul(worldMatrix, pos);\n"
            "	oPosObj = pos;\n";

        if (tt != RENDER_COMPOSITE_MAP)
        {
            // determine whether to apply the LOD morph to this vertex
            // we store the deltas against all vertices so we only want to apply
            // the morph to the ones which would disappear. The target LOD which is
            // being morphed to is stored in lodMorph.y, and the LOD at which
            // the vertex should be morphed is stored in uv.w. If we subtract
            // the former from the latter, and arrange to only morph if the
            // result is negative (it will only be -1 in fact, since after that
            // the vertex will never be indexed), we will achieve our aim.
            // sign(vertexLOD - targetLOD) == -1 is to morph
            outStream <<
                "	float toMorph = -min(0, sign(delta.y - lodMorph.y));\n";
            // this will either be 1 (morph) or 0 (don't morph)
            if (prof->getParent()->getDebugLevel())
            {
                // x == LOD level (-1 since value is target level, we want to display actual)
                outStream << "lodInfo.x = (lodMorph.y - 1) / " << terrain->getNumLodLevels() << ";\n";
                // y == LOD morph
                outStream << "lodInfo.y = toMorph * lodMorph.x;\n";
            }

            // morph
            switch (terrain->getAlignment())
            {
            case Terrain::ALIGN_X_Y:
                outStream << "	worldPos.z += delta.x * toMorph * lodMorph.x;\n";
                break;
            case Terrain::ALIGN_X_Z:
                outStream << "	worldPos.y += delta.x * toMorph * lodMorph.x;\n";
                break;
            case Terrain::ALIGN_Y_Z:
                outStream << "	worldPos.x += delta.x * toMorph * lodMorph.x;\n";
                break;
            };
        }


        // generate UVs
        if (tt != LOW_LOD)
        {
            for (uint i = 0; i < numUVSets; ++i)
            {
                uint layer  =  i * 2;
                uint uvMulIdx = layer / 4;

                outStream <<
                    "	oUV" << i << ".xy = " << " uv.xy * uvMul_" << uvMulIdx << "." << getChannel(layer) << ";\n";
                outStream <<
                    "	oUV" << i << ".zw = " << " uv.xy * uvMul_" << uvMulIdx << "." << getChannel(layer+1) << ";\n";

            }

        }


    }
    //---------------------------------------------------------------------

    /**
      * This method is identical to TerrainMaterialGeneratorA::SM2Profile::ShaderHelperCg::generateVpFooter()
      * but is needed because generateVpDynamicShadows() is not declared virtual.
      **/
    void TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVpFooter(
        const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream)
    {

        outStream <<
            "	oPos = mul(viewProjMatrix, worldPos);\n"
            "	oUVMisc.xy = uv.xy;\n";

        bool fog = terrain->getSceneManager()->getFogMode() != FOG_NONE && tt != RENDER_COMPOSITE_MAP;
        if (fog)
        {
            if (terrain->getSceneManager()->getFogMode() == FOG_LINEAR)
            {
                outStream <<
                    "	fogVal = saturate((oPos.z - fogParams.y) * fogParams.w);\n";
            }
            else
            {
                outStream <<
                    "	fogVal = 1 - saturate(1 / (exp(oPos.z * fogParams.x)));\n";
            }
        }

        if (prof->isShadowingEnabled(tt, terrain))
            generateVpDynamicShadows(prof, terrain, tt, outStream);

        outStream <<
            "}\n";


    }
    //---------------------------------------------------------------------
    void TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVertexProgramSource(
        const SM2Profile* prof, const Terrain* terrain, TechniqueType tt, StringUtil::StrStreamType& outStream)
    {
        generateVpHeader(prof, terrain, tt, outStream);

        if (tt != LOW_LOD)
        {
            uint maxLayers = prof->getMaxLayers(terrain);
            uint numLayers = std::min(maxLayers, static_cast<uint>(terrain->getLayerCount()));

            for (uint i = 0; i < numLayers; ++i)
                generateVpLayer(prof, terrain, tt, i, outStream);
        }

        generateVpFooter(prof, terrain, tt, outStream);

    }
    //---------------------------------------------------------------------
    HighLevelGpuProgramPtr
        TerrainRTSSMaterialGenerator::SM2Profile::ShaderHelperCg::generateVertexProgram(
            const SM2Profile* prof, const Terrain* terrain, TechniqueType tt)
    {
        HighLevelGpuProgramPtr ret = createVertexProgram(prof, terrain, tt);

        StringUtil::StrStreamType sourceStr;
        generateVertexProgramSource(prof, terrain, tt, sourceStr);
        ret->setSource(sourceStr.str());
        ret->load();
        defaultVpParams(prof, terrain, tt, ret);
#if OGRE_DEBUG_MODE
        LogManager::getSingleton().stream(LML_TRIVIAL) << "*** Terrain Vertex Program: "
            << ret->getName() << " ***\n" << ret->getSource() << "\n***   ***";
#endif

        return ret;

    }
    //---------------------------------------------------------------------


}

And this is how you use it in your application:

Code: Select all

void OgreTestApplication::createScene()
{
    // Setup lighting (directional light or spotlight for PSSM) ...
    // Setup camera, sky dome, ...

    initRTShaderSystem();

    // Setup PSSM shadowing for entities
    setupShadows(mShadows);

    // Create terrain (with shadowing)
    createTerrain(1, 1, Ogre::Vector3::ZERO, mSceneMgr->getLight("DirectionalLight"), true);

    // Create scene entities ...

    // Set geometry split points for shadowing (of all entities and terrain)
    assignSplitPoints();

    // Add all entities for which RTSS will generate materials to std::vector<Ogre::Entity*> mTargetEntities
    // eg.
    // mTargetEntities.push_back(ent1);
    // mTargetEntities.push_back(ent2);
    updateSystemShaders();
}

void OgreTestApplication::setupTerrainShadows(bool enableShadows)
{
    // Assume we get a shader model 2 material profile
    Ogre::TerrainMaterialGeneratorA::SM2Profile* matProfile;

   // RTSS PSSM shadows compatible terrain material
   Ogre::TerrainRTSSMaterialGenerator *matGen = new Ogre::TerrainRTSSMaterialGenerator ();
   Ogre::TerrainMaterialGeneratorPtr ptr = Ogre::TerrainMaterialGeneratorPtr();
   ptr.bind(matGen);
   mTerrainGlobals->setDefaultMaterialGenerator(ptr);
   matProfile = static_cast<Ogre::TerrainRTSSMaterialGenerator::SM2Profile*>( matGen->getActiveProfile() );
   
   if (enableShadows) {
        // Make sure PSSM is already setup
        matProfile->setReceiveDynamicShadowsEnabled(true);
        matProfile->setReceiveDynamicShadowsPSSM(mPssmSetup);  // PSSM shadowing
        matProfile->setReceiveDynamicShadowsDepth(true);            // with depth
        matProfile->setReceiveDynamicShadowsLowLod(false);
   } else {
        matProfile->setReceiveDynamicShadowsPSSM(NULL);
   }
}

/**
  * For a more detailed version, look at the RTShader sample code
  * from the SampleBrowser.
  **/
void OgreTestApplication::setupShadows(bool enableShadows)
{
    // No shadow
    if (! enableShadows)
    {
        mSceneMgr->setShadowTechnique(Ogre::SHADOWTYPE_NONE);

        const Ogre::RTShader::SubRenderStateList& subRenderStateList = mSchemRenderState->getTemplateSubRenderStateList();
        Ogre::RTShader::SubRenderStateListConstIterator it = subRenderStateList.begin();
        Ogre::RTShader::SubRenderStateListConstIterator itEnd = subRenderStateList.end();

        for (; it != itEnd; ++it)
        {
            Ogre::RTShader::SubRenderState* curSubRenderState = *it;

            // This is the pssm3 sub render state -> remove it.
            if (curSubRenderState->getType() == Ogre::RTShader::IntegratedPSSM3::Type)
            {
                mSchemRenderState->removeTemplateSubRenderState(*it);
                break;
            }
        }
    }

    // Integrated shadow PSSM with 3 splits. (depth shadow)
    else if (enableShadows)
    {
        mSceneMgr->setShadowTechnique(Ogre::SHADOWTYPE_TEXTURE_MODULATIVE_INTEGRATED);
        mSceneMgr->setShadowFarDistance(3000);

        // 3 textures per directional light (see http://www.stevestreeting.com/2008/08/21/parallel-split-shadow-maps-are-cool/)
        mSceneMgr->setShadowTextureCountPerLightType(Ogre::Light::LT_DIRECTIONAL, 3);
        mSceneMgr->setShadowTextureSettings(512, 3, Ogre::PF_FLOAT32_R);  // Uses three 512x512 shadow textures
       // You can also do a more detailed setup, for example:
        /*
              mSceneMgr->setShadowTextureCount(3);
              mSceneMgr->setShadowTextureConfig(0,2048,2048, Ogre::PF_FLOAT16_RGB);
              mSceneMgr->setShadowTextureConfig(1,1024,1024, Ogre::PF_FLOAT16_RGB);
              mSceneMgr->setShadowTextureConfig(2,512,512, Ogre::PF_FLOAT16_RGB);
        */

        mSceneMgr->setShadowTextureSelfShadow(true);
        mSceneMgr->setShadowCasterRenderBackFaces(true);  // Good for shadow casters like tree leaves

        // Set up caster material - this is just a standard (RTSS compatible) depth/shadow map caster
        mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster");

        // Disable fog on the caster pass.
        Ogre::MaterialPtr passCaterMaterial = Ogre::MaterialManager::getSingleton().getByName("PSSM/shadow_caster");
        Ogre::Pass* pssmCasterPass = passCaterMaterial->getTechnique(0)->getPass(0);
        pssmCasterPass->setFog(true);

        // shadow camera setup
        Ogre::PSSMShadowCameraSetup* pssmSetup = new Ogre::PSSMShadowCameraSetup();
        if(mPssmSetup)
            delete mPssmSetup;
        mPssmSetup = pssmSetup;

        pssmSetup->calculateSplitPoints(3, 5, mSceneMgr->getShadowFarDistance());    // Calculate 3 split points (PSSM 3)
                // Increase near distance when experiencing artifacts
        pssmSetup->setSplitPadding(10);
        pssmSetup->setOptimalAdjustFactor(0, 2);
        pssmSetup->setOptimalAdjustFactor(1, 1);
        pssmSetup->setOptimalAdjustFactor(2, 0.5);

        mSceneMgr->setShadowCameraSetup(Ogre::ShadowCameraSetupPtr(pssmSetup));

        // Set up terrain shadowing after this
    }

    // Invalidate the scheme in order to re-generate all shaders based technique related to this scheme.
    mShaderGenerator->invalidateScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);
}

bool OgreTestApplication::initRTShaderSystem()
{
    // Set these to the proper paths. RTShader libs can be found in the media folder of the Ogre SDK
    Ogre::String shaderLibPath = "resources/RTShaderLib";
    Ogre::String shaderCachePath = "cache/RTShader";

    if (Ogre::RTShader::ShaderGenerator::initialize())
    {
        mShaderGenerator = Ogre::RTShader::ShaderGenerator::getSingletonPtr();

        // Add the shader libs resource location.
        Ogre::ResourceGroupManager::getSingleton().addResourceLocation(shaderLibPath, "FileSystem");

        // Set shader cache path.
        mShaderGenerator->setShaderCachePath(shaderCachePath);

        // Set the scene manager.
        mShaderGenerator->addSceneManager(mSceneMgr);

        mSchemRenderState = mShaderGenerator->getRenderState(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

       // Make this viewport work with shader generator scheme.
       // This will use the RTShaderSystem generated equivalents of all materials in this viewport
       mViewport->setMaterialScheme(Ogre::RTShader::ShaderGenerator::DEFAULT_SCHEME_NAME);

       return true;
    }

    return false;
}

/**
  * Generate shader materials from regular entity materials.
  **/
void OgreTestApplication::updateSystemShaders()
{
    EntityListIterator it = mTargetEntities.begin();
    EntityListIterator itEnd = mTargetEntities.end();

    for (; it != itEnd; ++it)
    {
        generateShaders(*it);
    }
}

/**
  * This is more or less the code illustrated in Basic Tutorial 3 
  * (http://www.ogre3d.org/tikiwiki/tiki-index.php?page=Basic+Tutorial+3&structure=Tutorials)
  **/
void OgreTestApplication::createTerrain(size_t xTiles, size_t yTiles, Ogre::Vector3 origin, Ogre::Light *light, bool shadows)
{
    mTerrainGlobals = OGRE_NEW Ogre::TerrainGlobalOptions();
    mTerrainGroup = OGRE_NEW Ogre::TerrainGroup(mSceneMgr, Ogre::Terrain::ALIGN_X_Z, TERRAIN_TILE_RESOLUTION, TERRAIN_TILE_SIZE);
    mTerrainGroup->setFilenameConvention(Ogre::String("OgreTerrain"), Ogre::String("dat"));
    mTerrainGroup->setOrigin(origin);
    configureTerrainDefaults(light);

    // This part is important! Do it before actually loading any terrain.
    setupTerrainShadows(shadows);

    for (long x = 0; x < xTiles; ++x)
        for (long y = 0; y < yTiles; ++y)
            defineTerrain(x, y);

    // sync load since we want everything in place when we start
    mTerrainGroup->loadAllTerrains(true);   // Terrain shadow material needs to be setup before this call!

    if (mTerrainsImported)
    {
        Ogre::TerrainGroup::TerrainIterator ti = mTerrainGroup->getTerrainIterator();
        while(ti.hasMoreElements())
        {
            Ogre::Terrain* t = ti.getNext()->instance;
            initBlendMaps(t);
        }
    }

    mTerrainGroup->freeTemporaryResources();
}

[duststorm]

I'm back with some more questions

Of course I had to try and get shadow casters working with alpha textures.
So I made a variant of PSSM/shadow_caster (called PSSM/shadow_caster_alpha) that basically adds this:

Code: Select all

if (tex2D(pDiffuseMap, uv).a < 0.5) // check for alpha value of material's texture and discard of pixel is transparent
   {
      discard;
   }

(of course I also added the necessary extra parameters to the header declarations)
and applied it using mSceneMgr->setShadowTextureCasterMaterial("PSSM/shadow_caster_alpha");

The result is that it works for semi-transparent textures, but that all solid textures don't cast shadows anymore.

screenshot07162012_024819545_s.jpg

Any idea why this might happen? How come the alpha color values of non-see through textures is 0? Has this something to do with the texture format that is input to the shader? (and if this is the case, is there an easy way to verify it?)
I assume the input texture format for the shadow caster is not PF_FLOAT32_RGB (this is only the output format)?
I also hit this topic which might be related.

The texture for the tree leaves is a .tga with alpha rejection enabled. The other (solid) textures in the scene are DDS1 and png.

I tried to sidestep the issue by assigning the regular shadow caster to the scene manager, and only adding the alpha caster to the material file of the tree leaves using the shadow_caster_material statement, like so:

Code: Select all

import * from "pssm_alpha.material"

material TreeLeaves
{
	technique
	{
		shadow_caster_material "PSSM/shadow_caster_alpha"
		
		pass
		{
			cull_hardware none
			cull_software none

			ambient 0.5 0.5 0.5 1
			diffuse 0.6 0.6 0.6 1
			
			alpha_rejection greater_equal 192
			
			texture_unit
			{
				texture leafs.tga alpha
			}
		}
	}
}

The shadows for solid objects are back, the problem is that now the leave shadows are completely gone.

screenshot07162012_031035265_s.jpg

Very strange. I'm using the same material (and shaders), only added in the material script instead of applied to the scene manager.

Because of what I read in the topic I linked to (that when referencing to a shadow caster from a material file the caster might inherit the alpha rejection), I tried forcing alpha rejection off for the PSSM/shadow_caster_alpha material with alpha_rejection always_pass. But that didn't help.

[bstone]

Dump the definition of your PSSM/shadow_caster_alpha so I could have a look.

[duststorm]

pssm_alpha.cg

Code: Select all

// PSSM shadow caster with support for alpha textures (eg. trees)
// Supports RTSS shadow receivers

void shadow_caster_alpha_vs(
   float4 position         : POSITION,
   float2 uv         : TEXCOORD0, // get UV of material's texture

   out float4 oPosition   : POSITION,
   out float2 oDepth      : TEXCOORD0,
   out float2 ouv   :TEXCOORD1, // translate UV to pixel shader

   uniform float4x4 wvpMat)
{
   // this is the view space position
   oPosition = mul(wvpMat, position);

   // depth info for the fragment.
   oDepth.x = oPosition.z;
   oDepth.y = oPosition.w;

   // clamp z to zero. seem to do the trick. :-/
   //oPosition.z = max(oPosition.z, 0);

   ouv = uv;
}

void shadow_caster_alpha_ps(
   float2 depth      : TEXCOORD0,
   float2 uv : TEXCOORD1,
   uniform sampler2D    pDiffuseMap    : register(s0),

   out float4 oColour   : COLOR,

   uniform float4 pssmSplitPoints)
{
   if (tex2D(pDiffuseMap, uv).a < 0.5) // check for alpha value of material's texture and discard of pixel is transparent
   {
      discard;
   }

   float finalDepth = depth.x / depth.y;
   oColour = float4(finalDepth, finalDepth, finalDepth, 1);
}

pssm_alpha.material

Code: Select all

// declare the vertex shader (CG for the language)
vertex_program PSSM/shadow_caster_alpha_vs cg
{
	// source file
	source pssm_alpha.cg
	// will run on vertex shader 1.1+
	profiles vs_1_1 arbvp1
	// entry function
	entry_point shadow_caster_alpha_vs

	default_params
	{
		param_named_auto wvpMat worldviewproj_matrix
		// this is the scene's depth range
		//param_named_auto depthRange			scene_depth_range
		//param_named_auto optimalAdustFactor	custom 0
	}
}

// declare the fragment shader (CG for the language)
fragment_program PSSM/shadow_caster_alpha_ps cg
{
	// source file
	source pssm_alpha.cg
	// will run on pixel shader 2.0+
	profiles ps_2_0 arbfp1
	// entry function
	entry_point shadow_caster_alpha_ps

	default_params
	{
	}
}

material PSSM/shadow_caster_alpha
{
	technique
	{
		// all this will do is write depth and depth² to red and green
		pass
		{
                        // Disable backface culling
			cull_hardware none
			cull_software none
			
			// Make sure that when including this material in another with 'shadow_caster_material' that it does not inherit alpha_rejection from it
			alpha_rejection always_pass
		
			vertex_program_ref PSSM/shadow_caster_alpha_vs
			{
			}

			fragment_program_ref PSSM/shadow_caster_alpha_ps
			{
			}
		}
	}
}

[bstone]

Yeah, that's what I thought. The shadow caster material doesn't bind any texture units to the shadow casting pass so whatever you get in r0 sampler is undefined. You will have to bind the proper texture (the one that provides the alpha channel) to the shadow casting material pass. The best way I can think of to do that is to create an abstract shadow caster material for all transparent objects and then inherit it for each specific texture employing set_texture_alias to pass it down to the base material.

Another thing I didn't like was the "alpha" option for your .tga texture. I'm not 100% sure but I think it doesn't do what you think it does. My understanding is that you should use the "alpha" option to load a single channel image into the .a component (as opposed to loading it into the .r color component by default). If your .tga image has an alpha channel then you don't need the "alpha" option as full RGBA data will be copied into the texture directly from the image.