Page 1 of 1

Shadow system: enhancements

Posted: Thu Oct 11, 2012 6:18 pm
by Xavyiy
Hi folks!

These last couple of weeks I've been working on the Paradise Engine shadow system, so I've been stressing out the Ogre3d shadow system a lot ;)
First of all I've to say that I've been very impressed by how easy and well the Ogre built-in shadow system works even with complex scenes.

Due to flexibility reasons, I'm using the SHADOWTYPE_TEXTURE_MODULATIVE technique(the whole post applies to SHADOWTYPE_TEXTURE_ADDITIVE, but not to integrated shadows), so Ogre is responsible from render the shadows in an extra pass sorting the geometry as needed (also, I allow the user to use custom casters/receivers if needed and also some kind of a additive shadows too, but that's not this post bussines).

Of course, I'm using custom shadow casters/receivers, set via SceneManager::setShadowTextureCaster/ReceiverMaterial.

Problems have arrived when I've implemented Cascaded shadow mapping, which uses more than one shadow map per light. Intuitively I thought that ogre would fill the "content_type shadow" texture unit states with the correct sahdow render target, BUT NOT. Note: I'm using Ogre 1.7.4.

After digging deeper into the matter, I've remarked one very ugly thing: the scene manager, in renderModulative/AdditiveTextureShadowedQueueGroupObjects, was removing all receiver pass texture unit states and just keeping the first one, which was filled with the first shadow map. Here is the relevant part of the code:

Code: Select all

for (i = mLightsAffectingFrustum.begin(), si = mShadowTextures.begin();
            i != iend && si != siend; ++i)
        {
            Light* l = *i;

            if (!l->getCastShadows())
                continue;

			// Store current shadow texture
            mCurrentShadowTexture = si->getPointer();
			// Get camera for current shadow texture
            Camera *cam = mCurrentShadowTexture->getBuffer()->getRenderTarget()->getViewport(0)->getCamera();
            // Hook up receiver texture
			Pass* targetPass = mShadowTextureCustomReceiverPass ?
				mShadowTextureCustomReceiverPass : mShadowReceiverPass;
			targetPass->getTextureUnitState(0)->setTextureName(
				mCurrentShadowTexture->getName());
			// Hook up projection frustum if fixed-function, but also need to
			// disable it explicitly for program pipeline.
			TextureUnitState* texUnit = targetPass->getTextureUnitState(0);
			texUnit->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
			// clamp to border colour in case this is a custom material
			texUnit->setTextureAddressingMode(TextureUnitState::TAM_BORDER);
			texUnit->setTextureBorderColour(ColourValue::White);

            mAutoParamDataSource->setTextureProjector(cam, 0);
            // if this light is a spotlight, we need to add the spot fader layer
			// BUT not if using a custom projection matrix, since then it will be
			// inappropriately shaped most likely
            if (l->getType() == Light::LT_SPOTLIGHT && !cam->isCustomProjectionMatrixEnabled())
            {
				// remove all TUs except 0 & 1 
				// (only an issue if additive shadows have been used)
				while(targetPass->getNumTextureUnitStates() > 2)
					targetPass->removeTextureUnitState(2);

                // Add spot fader if not present already
                if (targetPass->getNumTextureUnitStates() == 2 && 
					targetPass->getTextureUnitState(1)->getTextureName() == 
						"spot_shadow_fade.png")
				{
					// Just set 
					TextureUnitState* t = 
						targetPass->getTextureUnitState(1);
					t->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
				}
                else
				{
					// Remove any non-conforming spot layers
					while(targetPass->getNumTextureUnitStates() > 1)
						targetPass->removeTextureUnitState(1);

                    TextureUnitState* t = 
                        targetPass->createTextureUnitState("spot_shadow_fade.png");
                    t->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
                    t->setColourOperation(LBO_ADD);
                    t->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);
                }
            }
            else 
            {
				// remove all TUs except 0 including spot
				while(targetPass->getNumTextureUnitStates() > 1)
	                targetPass->removeTextureUnitState(1);

            }
			// Set lighting / blending modes
			targetPass->setSceneBlending(SBF_DEST_COLOUR, SBF_ZERO);
			targetPass->setLightingEnabled(false);

            targetPass->_load();

			// Fire pre-receiver event
			fireShadowTexturesPreReceiver(l, cam);

            renderTextureShadowReceiverQueueGroupObjects(pGroup, om);

            ++si;

        }// for each light
So, I've modified it in order to:
  • Don't remove any texture unit state
  • Iterate over each light's shadow map and bind it to the correct texture unit state
  • Also, set the texture projection for each shadow map in mAutoParamDataSource
That way, we're able to use shadow techniques which uses more than one simple shadow map and also we can use other textures(like noise textures for filtering, etc) in the receiver shader(not before since all were removed).

Here is the modified code:

Code: Select all

void SceneManager::renderModulativeTextureShadowedQueueGroupObjects(
	RenderQueueGroup* pGroup, 
	QueuedRenderableCollection::OrganisationMode om)
{
    /* For each light, we need to render all the solids from each group, 
    then do the modulative shadows, then render the transparents from
    each group.
    Now, this means we are going to reorder things more, but that it required
    if the shadows are to look correct. The overall order is preserved anyway,
    it's just that all the transparents are at the end instead of them being
    interleaved as in the normal rendering loop. 
    */
    // Iterate through priorities
    RenderQueueGroup::PriorityMapIterator groupIt = pGroup->getIterator();

    while (groupIt.hasMoreElements())
    {
        RenderPriorityGroup* pPriorityGrp = groupIt.getNext();

        // Sort the queue first
        pPriorityGrp->sort(mCameraInProgress);

        // Do solids
        renderObjects(pPriorityGrp->getSolidsBasic(), om, true, true);
        renderObjects(pPriorityGrp->getSolidsNoShadowReceive(), om, true, true);
    }


    // Iterate over lights, render received shadows
    // only perform this if we're in the 'normal' render stage, to avoid
    // doing it during the render to texture
    if (mIlluminationStage == IRS_NONE)
    {
        mIlluminationStage = IRS_RENDER_RECEIVER_PASS;

        LightList::iterator i, iend;
        ShadowTextureList::iterator si, siend;
        iend = mLightsAffectingFrustum.end();
        siend = mShadowTextures.end();
        for (i = mLightsAffectingFrustum.begin(), si = mShadowTextures.begin();
            i != iend && si != siend; ++i)
        {
			std::cout << "Current ST: " << si->getPointer() << std::endl;
			std::cout << "STs: " << mShadowTextures.size() << std::endl;

            Light* l = *i;

            if (!l->getCastShadows())
                continue;

			// Hook up receiver texture
			Pass* targetPass = mShadowTextureCustomReceiverPass ?
				mShadowTextureCustomReceiverPass : mShadowReceiverPass;

			// Set-up shadow maps
			for(size_t shadowMapNumber = 0; shadowMapNumber < mShadowTextureCountPerType[l->getType()]; shadowMapNumber++)
			{
				// Store current shadow texture
				mCurrentShadowTexture = si->getPointer();

				// Bind the shadow map render target to the correct dest texture unit
				size_t shadowTexIndex = 0;
				TextureUnitState* currentTUS = 0;

				for(size_t k = 0; k < targetPass->getNumTextureUnitStates(); k++)
				{
					// Note: the shadow textures must be the first ones to be declared in the .material file!!!

					if (shadowTexIndex == shadowMapNumber)// && 
					//	targetPass->getTextureUnitState(k)->getContentType() == TextureUnitState::CONTENT_SHADOW)
					{
						currentTUS = targetPass->getTextureUnitState(k);
						//currentTUS->_setTexturePtr(*si);
						currentTUS->setTextureName(mCurrentShadowTexture->getName());
						break;
					}

					//if (targetPass->getTextureUnitState(k)->getContentType() == TextureUnitState::CONTENT_SHADOW)
					{
						shadowTexIndex++;
					}
				}

				// Get camera for current shadow texture
				Camera *cam = mCurrentShadowTexture->getBuffer()->getRenderTarget()->getViewport(0)->getCamera();

				// Hook up projection frustum if fixed-function, but also need to
				// disable it explicitly for program pipeline.
				currentTUS->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
				// clamp to border colour in case this is a custom material
				currentTUS->setTextureAddressingMode(TextureUnitState::TAM_BORDER);
				currentTUS->setTextureBorderColour(ColourValue::White);

				mAutoParamDataSource->setTextureProjector(cam, shadowMapNumber);
				// if this light is a spotlight, we need to add the spot fader layer
				// BUT not if using a custom projection matrix, since then it will be
				// inappropriately shaped most likely
				if (l->getType() == Light::LT_SPOTLIGHT && !cam->isCustomProjectionMatrixEnabled())
				{
					// remove all TUs except 0 & 1 
					// (only an issue if additive shadows have been used)
					while(targetPass->getNumTextureUnitStates() > 2)
						targetPass->removeTextureUnitState(2);

					// Add spot fader if not present already
					if (targetPass->getNumTextureUnitStates() == 2 && 
						targetPass->getTextureUnitState(1)->getTextureName() == 
							"spot_shadow_fade.png")
					{
						// Just set 
						TextureUnitState* t = 
							targetPass->getTextureUnitState(1);
						t->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
					}
					else
					{
						// Remove any non-conforming spot layers
						while(targetPass->getNumTextureUnitStates() > 1)
							targetPass->removeTextureUnitState(1);

						TextureUnitState* t = 
							targetPass->createTextureUnitState("spot_shadow_fade.png");
						t->setProjectiveTexturing(!targetPass->hasVertexProgram(), cam);
						t->setColourOperation(LBO_ADD);
						t->setTextureAddressingMode(TextureUnitState::TAM_CLAMP);
					}
				}
				else // POINT, DIRECTIONAL
				{
					// remove all TUs except 0 including spot
				//	while(targetPass->getNumTextureUnitStates() > 1)
				 //       targetPass->removeTextureUnitState(1);
				}

				++si;
			}
			// Set lighting / blending modes
			targetPass->setSceneBlending(SBF_DEST_COLOUR, SBF_ZERO);
			targetPass->setLightingEnabled(false);

            targetPass->_load();

			// Fire pre-receiver event
			//fireShadowTexturesPreReceiver(l, cam);
			// Hack! Each shadow texture will have a custom camera, but it makes no much sense to fire multiple events per light...
			fireShadowTexturesPreReceiver(l, mCurrentShadowTexture->getBuffer()->getRenderTarget()->getViewport(0)->getCamera());

            renderTextureShadowReceiverQueueGroupObjects(pGroup, om);

        }// for each light

        mIlluminationStage = IRS_NONE;

    }

    // Iterate again - variable name changed to appease gcc.
    RenderQueueGroup::PriorityMapIterator groupIt3 = pGroup->getIterator();
    while (groupIt3.hasMoreElements())
    {
        RenderPriorityGroup* pPriorityGrp = groupIt3.getNext();

        // Do unsorted transparents
        renderObjects(pPriorityGrp->getTransparentsUnsorted(), om, true, true);
        // Do transparents (always descending)
        renderObjects(pPriorityGrp->getTransparents(), 
			QueuedRenderableCollection::OM_SORT_DESCENDING, true, true);

    }// for each priority

}
Of course, this is not a patch, I've just modified it for modulative shadows and there're some remaining issues:
  • Spot lights must be broken. To be honest I don't think the "spot_shadow_fade.png hack" is a good idea, we should think in a better alternative.
  • All shadow texture unit states must be declared at first since when we invoke TUS->setTextureName(...) the content type is changed to "CONTENT_NAMED" so we can't use this content data to know which TUS must be filled with the shadow map. I've tried to use _setTexturePtr(...), since it does not modify the TUS content type, but doesn't work with more than one light, dunno why.
  • fireShadowTexturesPreReceiver must be invoked for each shadow map caster, but it's not a good idea to invoke it n types per light where n is the number of shadow textures. So or we just remove the frustum param from it or we create another one listener function.
If would be nice to address this problem in Ogre 1.9, some kind of approach similar to the one I've implemented should be enough since now I've pretty nice cascaded shadows :D

Btw, I've remarked that the OgreSceneManager.cpp code is a mess... it would be very worth reviewing it: fixing comments typos and also adding some others in the difficult parts, at least.

Xavier

Re: Shadow system: enhancements

Posted: Thu Oct 11, 2012 7:06 pm
by bstone
Xavyiy wrote:it would be very worth reviewing it: fixing comments typos and also adding some others in the difficult parts, at least.
Why would you want to add other comment typos in the difficult parts? :D

Kidding! Good catch indeed but I'm surprised that you decided to go for non-integrated shadows. That would be a performance hit compared to any integrated implementation.

Re: Shadow system: enhancements

Posted: Thu Oct 11, 2012 7:34 pm
by Xavyiy

Code: Select all

Good catch indeed but I'm surprised that you decided to go for non-integrated shadows. That would be a performance hit compared to any integrated implementation.
Yeah, it's a performance hit compared to render lighting&shadows in the same pass, but that's the only way to allow people get shadows without having to deal with the shadowing part in the shaders and also, it's the only way for getting an arbitrary number of lights casting shadows and also being able to use different shadowing techniques using the same materials/shaders(without using a shader generator system, of course).

But, of course, the user is able to declare materials as "shadow_handler"(I'm using a custom material system, which at the end it just generates/handle ogre materials) and he's responsible from render shadows in their custom shaders, accessing to the caster matrices, textures, etc. That way the user is able to use "integrated" shadows too.

Re: Shadow system: enhancements

Posted: Sun Dec 02, 2012 3:40 pm
by Xavyiy
I've just upgrade the Paradise Engine to Ogre 1.8.1 (from 1.7.4) and I've detected a little bug related to non-integrated shadows(at least modulative shadows): them ignore Material::getReceiveShadows(), shadows were always rendered even if getReceiveShadows was false.

Fix:
File: OgreRenderQueueSortingGrouping.cpp
Replace this:

Code: Select all

if (mSplitNoShadowPasses &&
                mParent->getShadowsEnabled() &&
				((!pTech->getParent()->getReceiveShadows() ||
				rend->getCastsShadows()) && mShadowCastersNotReceivers))
by:

Code: Select all

if (mSplitNoShadowPasses &&
                mParent->getShadowsEnabled() &&
				(!pTech->getParent()->getReceiveShadows() ||
				rend->getCastsShadows() && mShadowCastersNotReceivers))
Xavier

Re: Shadow system: enhancements

Posted: Sun Dec 02, 2012 3:50 pm
by bstone
Err. I think I reported that a while back and the issue has been marked as resolved. Interesting.

Re: Shadow system: enhancements

Posted: Sun Dec 02, 2012 3:56 pm
by Xavyiy
Mmm maybe it's fixed in the trunk (I've upgraded to 1.8.1, not to the trunk). But your mantis entry is from June and 1.8.1 was released in September so... something weird here, apparently not fixed.

Re: Shadow system: enhancements

Posted: Sun Dec 02, 2012 4:06 pm
by bstone
Yep.

Re: Shadow system: enhancements

Posted: Mon Dec 24, 2012 12:05 pm
by Wolfmanfx
Yeah this were not fixed i added to 1.9

Re: Shadow system: enhancements

Posted: Mon Dec 24, 2012 4:04 pm
by bstone
Thanks Wolfmanfx.