Assert fail on Linux with Texture Shadows

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
LAva
Kobold
Posts: 27
Joined: Mon Mar 05, 2007 3:05 pm
Location: Styria, Austria
Contact:

Assert fail on Linux with Texture Shadows

Post by LAva »

I have some problems with texture shadows on Linux (Ubuntu 7.04).
The program throws an assert_fail in some situations which I can reproduce but I don't know why the program doesn't crash in other situations.

One of the situations where the program crashes is if I load a specific model twice in the scene (I didn't test this in an empty scene, but I think it is a combination of the whole scene and this two models). But there are other constellations where this error is thrown.

I tested my project on windows without any problems!

I'm using the Ogre CVS version but this behavior was also there on earlier releases (1.4.1).

Maybe somebody has a clue if this is a bug or if I'm doing something wrong.

Here is the call stack:

Code: Select all

#0 0xffffe410	__kernel_vsyscall() (??:??)
#1 0xb729a875	raise() (/lib/tls/i686/cmov/libc.so.6:??)
#2 0xb729c201	abort() (/lib/tls/i686/cmov/libc.so.6:??)
#3 0xb7293b6e	__assert_fail() (/lib/tls/i686/cmov/libc.so.6:??)
#4 0xb7dc32ef	Ogre::TextureUnitState::operator=() (../lib_linux/libOgreMain-1.4.4.so:??)
#5 0xb7d4f158	Ogre::SceneManager::deriveShadowCasterPass() (../lib_linux/libOgreMain-1.4.4.so:??)
#6 0xb7d4ffec	Ogre::SceneManager::_setPass() (../lib_linux/libOgreMain-1.4.4.so:??)
#7 0xb7d49127	Ogre::SceneManager::BRAND NAME::visit() (../lib_linux/libOgreMain-1.4.4.so:??)
#8 0xb7d13224	Ogre::QueuedRenderableCollection::acceptVisitorGrouped() (../lib_linux/libOgreMain-1.4.4.so:??)
#9 0xb7d13b51	Ogre::QueuedRenderableCollection::acceptVisitor() (../lib_linux/libOgreMain-1.4.4.so:??)
#10 0xb7d49711	Ogre::SceneManager::renderObjects() (../lib_linux/libOgreMain-1.4.4.so:??)
#11 0xb7d4bc7f	Ogre::SceneManager::renderTextureShadowCasterQueueGroupObjects() (../lib_linux/libOgreMain-1.4.4.so:??)
#12 0xb7d49017	Ogre::SceneManager::_renderQueueGroupObjects() (../lib_linux/libOgreMain-1.4.4.so:??)
#13 0xb7d4ac5d	Ogre::SceneManager::renderVisibleObjectsDefaultSequence() (../lib_linux/libOgreMain-1.4.4.so:??)
#14 0xb7d49778	Ogre::SceneManager::_renderVisibleObjects() (../lib_linux/libOgreMain-1.4.4.so:??)
#15 0xb777a34d	Ogre::TerrainSceneManager::_renderVisibleObjects() (../lib_linux/Plugin_OctreeSceneManager.so:??)
#16 0xb7d4a930	Ogre::SceneManager::_renderScene() (../lib_linux/libOgreMain-1.4.4.so:??)
#17 0xb777a706	Ogre::TerrainSceneManager::_renderScene() (../lib_linux/Plugin_OctreeSceneManager.so:??)
#18 0xb7b8d76e	Ogre::Camera::_renderScene() (../lib_linux/libOgreMain-1.4.4.so:??)
#19 0xb7dd05b0	Ogre::Viewport::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#20 0xb7d1e593	Ogre::RenderTarget::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#21 0xb7d4ca28	Ogre::SceneManager::prepareShadowTextures() (../lib_linux/libOgreMain-1.4.4.so:??)
#22 0xb7d4aa37	Ogre::SceneManager::_renderScene() (../lib_linux/libOgreMain-1.4.4.so:??)
#23 0xb777a706	Ogre::TerrainSceneManager::_renderScene() (../lib_linux/Plugin_OctreeSceneManager.so:??)
#24 0xb7b8d76e	Ogre::Camera::_renderScene() (../lib_linux/libOgreMain-1.4.4.so:??)
#25 0xb7dd05b0	Ogre::Viewport::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#26 0xb7d1e593	Ogre::RenderTarget::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#27 0xb7d225d9	Ogre::RenderWindow::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#28 0xb7d2250c	Ogre::RenderWindow::update() (../lib_linux/libOgreMain-1.4.4.so:??)
#29 0xb7d181db	Ogre::RenderSystem::_updateAllRenderTargets() (../lib_linux/libOgreMain-1.4.4.so:??)
The code where I activate the shadows:

Code: Select all

      mSceneMgr->setShadowTechnique(SHADOWTYPE_TEXTURE_MODULATIVE);
    mSceneMgr->setShadowTextureSize(512);
    mSceneMgr->setShadowTextureCount(10);
    mSceneMgr->setShadowColour(ColourValue(0.7,0.7,0.7));
    mSceneMgr->setShadowFarDistance(50);
    mSceneMgr->setShadowTextureSelfShadow(false);

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

What is needed is more where the assert is raised... did you compile with debug symbols ? can't you read the assert line and see what it's saying ?
That's what assert are for... precise information on requisites to make code working.

I tested my project on windows without any problems!
using opengl too ?
Anyway linux driver are not same quality as windows one...
mSceneMgr->setShadowTextureSize(512);
mSceneMgr->setShadowTextureCount(10);
10 shadow texture means 10 rendertarget of 512x512... that's huge... and I'm not sure it's that supported either by hardware or Ogre (check Ogre log "max render target" for instance...)

LAva
Kobold
Posts: 27
Joined: Mon Mar 05, 2007 3:05 pm
Location: Styria, Austria
Contact:

Post by LAva »

There are just two asserts in this method where the error happens.

Code: Select all

    TextureUnitState & TextureUnitState::operator = ( 
        const TextureUnitState &oth )
    {
        assert(mAnimController == 0);
        assert(mEffects.empty());
I'm not sure which one causes the program stop.
I always thought, if I compile Ogre on Linux there are already debug symbols in the libs, but I also tried to set the debug flag manually.
But if I try to debug my program I never get the line number in the Ogre libs. Do the source files have to be on a specific place for this to work?
I tested my project on windows without any problems!
using opengl too ?
Anyway linux driver are not same quality as windows one...
Yes! I tried my program using opengl in Windows as well. Worked without any problems.
10 shadow texture means 10 rendertarget of 512x512... that's huge... and I'm not sure it's that supported either by hardware or Ogre (check Ogre log "max render target" for instance...)
I'm using just one shadow texture now, but the error stays the same.

User avatar
tuan kuranes
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 2653
Joined: Wed Sep 24, 2003 8:07 am
Location: Haute Garonne, France
x 4
Contact:

Post by tuan kuranes »

run it using GDB or any front end to gdb, and you'll get much more information, stack/local var/etc...

LAva
Kobold
Posts: 27
Joined: Mon Mar 05, 2007 3:05 pm
Location: Styria, Austria
Contact:

Post by LAva »

The failed assert happens on mEffects.empty() in the TextureUnitState.

Code: Select all

TextureUnitState & TextureUnitState::operator = (
        const TextureUnitState &oth )
    {
        assert(mAnimController == 0);
        assert(mEffects.empty());

User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19265
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

We'll need the context in the method that's calling this operator.

User avatar
SuperMegaMau
Greenskin
Posts: 104
Joined: Mon Mar 21, 2005 3:31 am
Location: Portugal

Post by SuperMegaMau »

hi,

I had the exact same problem, and found out that disabling TextureUnitState effects (transforms) solved the problem. Dont know if this is the issue, but works for me.

TMT
Halfling
Posts: 59
Joined: Thu Aug 23, 2007 8:09 pm

Post by TMT »

I have begun to see this assert also. Mine asserts when being called from SceneManager::deriveShadowCasterPass(). I am studying the code to see exactly what is the cause, but I already see an inconsitency that is suspect:

In SceneManager::deriveShadowCasterPass() you have:

Code: Select all

			// Copy texture state, shift up one since 0 is shadow texture
			unsigned short origPassTUCount = pass->getNumTextureUnitStates();
			for (unsigned short t = 0; t < origPassTUCount; ++t)
			{
				TextureUnitState* tex;
				if (retPass->getNumTextureUnitStates() <= t)
				{
					tex = retPass->createTextureUnitState();
				}
				else
				{
					tex = retPass->getTextureUnitState(t);
				}
				// copy base state
				(*tex) = *(pass->getTextureUnitState(t));
				// override colour function
				tex->setColourOperationEx(LBX_SOURCE1, LBS_MANUAL, LBS_CURRENT,
					isShadowTechniqueAdditive()? ColourValue::Black : mShadowColour);
			}
but in SceneManager::deriveShadowReceiverPass() you have:

Code: Select all

            // Copy texture state, shift up one since 0 is shadow texture
            unsigned short origPassTUCount = pass->getNumTextureUnitStates();
            for (unsigned short t = 0; t < origPassTUCount; ++t)
            {
                unsigned short targetIndex = t+1;
                TextureUnitState* tex;
                if (retPass->getNumTextureUnitStates() <= targetIndex)
                {
                    tex = retPass->createTextureUnitState();
                }
                else
                {
                    tex = retPass->getTextureUnitState(targetIndex);
                }
                (*tex) = *(pass->getTextureUnitState(t));
				// If programmable, have to adjust the texcoord sets too
				// D3D insists that texcoordsets match tex unit in programmable mode
				if (retPass->hasVertexProgram())
					tex->setTextureCoordSet(targetIndex);
            }
These appear to be doing roughly the same thing, but note he shift in the index in the latter. The latter sure seems more correct according to the comment. Anyone familiar with this code know if the former is incorrect or not?

TMT
Halfling
Posts: 59
Joined: Thu Aug 23, 2007 8:09 pm

Post by TMT »

Well, I thought someone would comment. I did change the loop in SceneManager::deriveShadowCasterPass() to be like SceneManager::deriveShadowReceiverPass() and I no longer get the assert. I'm assuming this is a bug that's been there for a while. I'm using 1.4.9, but it appears to be in all subsequent versions as well.

BTW, I'm using Windows and not Linux as the original creator of this thread was using.

User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19265
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Only just saw this. The 2 blocks of code are supposed to be different, the comments have just been inappropriately copied.

In the caster pass, there is in fact no shadow texture, because you're rendering to the shadow texture. Therefore, no offset. There is only an extra texture unit in the receiver pass, since that's where you're pulling in the shadow texture.

I still haven't seen enough detail about this problem to be able to propose a cause, and I haven't come across it yet.

TMT
Halfling
Posts: 59
Joined: Thu Aug 23, 2007 8:09 pm

Post by TMT »

Thanks sinbad for looking at it. I can give some more information in hopes it can clue you or someone in to the specific problem. The material it is asserting on is a flat scrolling (scroll_anim 0.0 0.002) water surface. The receiving TextureUnitState already has the ET_VSCROLL in its mEffects map and hence the assert in the assignment operator fails. None of the other textures have problems...just this one and it's the only one with an effect on it.

As I write this I suddenly am thinking that I can avoid the assert by making sure this water surface doesn't cast a shadow as it really shouldn't anyway, but that wouldn't resolve the issue for a case where an animated texture is on an object that should cast a shadow.

Perhaps the TextureUnitState assignment operator shouldn't care if there are already things in mEffects and simply clear it before copying them.

Shadow code is not my area of expertise, so I'm trying to resolve this somewhat blindly. I apologize for being somewhat dense in this area.

User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2702
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 47

Re: Assert fail on Linux with Texture Shadows

Post by syedhs »

Just a note, I have hit the same exact error (assert on mEffects.empty() for material with scroll_anim in it) and the workaround is also to disable shadow casting for the entities.
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me

scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 214

Re: Assert fail on Linux with Texture Shadows

Post by scrawl »

5 years later, the bug is still unsolved. I wonder why it was never reported to the issue tracker?

I have the same crash in my project and it occurs 100% of the time when using shadows and a material with a spherical environment map.

Here is my fix attempt: https://bitbucket.org/sinbad/ogre/pull- ... unitstates

Post Reply