visibility masks permanently set?

[Rak'kar]

I'm having a problem where I cannot change visibility masks while using a compositor.

I have visibility masks as follows:

Code: Select all

VISIBILITY_MASK_DEFAULT=1<<0,
	VISIBILITY_MASK_RADAR=1<<1,
	VISIBILITY_MASK_ROOFS=1<<2,
	VISIBILITY_MASK_TRIGGER_VOLUME=1<<3,

What I've been doing, which works fine without a compositor, is I set the visibility mask depending on if I want to show the roof or not. This changes as the player moves around.

Code: Select all

int visibilityMask = GetApp()->GetSceneManager()->getVisibilityMask();
GetApp()->GetSceneManager()->setVisibilityMask(visibilityMask | VISIBILITY_MASK_ROOFS);

However, if I add a compositor, roofs no longer show up. At first I thought this was a bug with roofs. However, I found the scene manager visibility flags become fixed forever at the first setting that I ever call the compositor with, and it just so happens that with the first setting roofs were never visible. If I force the first time I call it to show roofs, roofs are always visible.

Code: Select all

unsigned int oldVisibilityMask = sceneManager->getVisibilityMask();

// BUG - Proves that the visibility masks are fixed at whatever I set it to be the first time I ever render it
// Changing visibility masks during runtime has no effect!
static bool firstCall=true;
if (firstCall)
	oldVisibilityMask=(1<<2) | (1<<0);

Ogre::ShadowTechnique oldShadowTechnique = sceneManager->getShadowTechnique();
if (oldShadowTechnique!=Ogre::SHADOWTYPE_NONE)
	sceneManager->setShadowTechnique(Ogre::SHADOWTYPE_NONE);

unsigned i;

for (i=0; i < ti->getNumTargetPasses(); i++)
	ti->getTargetPass(i)->setVisibilityMask(oldVisibilityMask);

shieldGroup->update(curTimeMS);
distortionRenderTarget->update();
shieldGroup->hideGroup();
Ogre::MaterialManager::getSingleton().setActiveScheme(currentScheme);
if (oldShadowTechnique!=Ogre::SHADOWTYPE_NONE)
	sceneManager->setShadowTechnique(oldShadowTechnique);
sceneManager->setVisibilityMask(oldVisibilityMask);

The scene manager visibility masks input to the function is correct and changes as the player moves around, it's something in Ogre that becomes fixed like a pitbull that won't let go.

Is there some function I need to call, or maybe I am calling these things out of order?

[sinbad]

I presume you have a render_scene pass in your compositor? If so, it is the visibility mask that the compositor pass has which applies, not any other mask you may have set.

You might want to consider removing the render_scene pass from the compositor and using 'input previous' instead. Either that or alter the visiblity mask on your compositor.

[Rak'kar]

We are using input_previous

Code: Select all

compositor ShieldDistort
{
	technique
	{
		//original scene
		texture rtScene target_width target_height PF_R8G8B8
		
		//render the original scene (full)
		target rtScene
		{
			input previous
		}
		
		//combine the original scene with the distortion texture (pre generated)
		target_output
        	{
            		input none
           		pass render_quad
           		{
				input 0 rtScene
              			material Render/DistortionCombine
            		}
       		}
	
	}
}

The compositor itself doesn't have any visibility masks.

Will input previous use the global visibility masks?

[sinbad]

Ah ok, it's because during the compilation stage the visibility mask is copied and the implied render_scene still does that. You can force it to recompile when you change the setting by calling CompositorChain::_markDirty.

[Rak'kar]

Works now, thanks a lot! Maybe it would be a good feature to automatically check the visibility masks to see if they were different from last time.

[sinbad]

Yeah, working on it

[fractile]

I know this is an old thread, but I'm facing the exact same problem and I just can't use either of the proposed solutions. I have a split-screen setup, with two viewports and I need to hide some objects from the other viewport. I have visibility flags set on the objects, but I just can't get the mask set properly.

Only way I have found to have anti-aliasing work with compositors requires using render_scene, which makes using setVisibilityMask() in viewports impossible. More on this in my posts here: http://www.ogre3d.org/forums/viewtopic.php?f=2&t=77938

The other proposed way of setting visibility mask on compositor doesn't work well either. It seems that both viewports share the same compositor pass instances, so if I set the visibility mask on render_scene pass for the other viewport, it affects the other one too.

Any ideas on how to resolve this?

[fractile]

I tried registering a RenderTargetListener to the window and setting visibility mask for compositors in preViewportUpdate() for every viewport. It seemed to work for a while, but it broke down when enabling/disabling any compositors. Presumable due to compositor chain getting "recompiled".

Only remaining option I can think of is duplicating compositor scripts for every viewport. That should result in separate compositor pass instances and let me set different visibility masks for different viewports. Far from ideal, but it should work.