Render image in compositor

[Lax]

Hi,

I want to create a keyhole compositor effect, in which a keyhole will grow or shrink (fade in scene, fade out scene).

Its already working, if use calculate a growing/shrinking circle via distance calculation in the shader.

Now I wanted to let the user choose a custom image mask to grow/shrink, like this one:

I prepared everything. The shaders are working, but no matter what I do. I do not see the mask as compositor.

This is my reduced code so far:

Create a TextureGPU out of the Image and set for the material texture unit:

Code: Select all

		Ogre::Image2 shapeImage;
		shapeImage.load(this->shapeMask->getString(), ResourceGroupManager::AUTODETECT_RESOURCE_GROUP_NAME);

	this->destroyShape();

	//const uint8 numMipmaps = image.getNumMipmaps();

	TextureGpuManager* textureManager = Ogre::Root::getSingleton().getRenderSystem()->getTextureGpuManager();

	Ogre::PixelFormatGpu pixelFormat = shapeImage.getPixelFormat();
	this->shapeTexture = textureManager->createTexture(this->shapeMask->getString() + "_" + Ogre::StringConverter::toString(this->gameObjectPtr->getId()),
													   GpuPageOutStrategy::SaveToSystemRam, TextureFlags::RenderToTexture | TextureFlags::AllowAutomipmaps, TextureTypes::Type2D);
	this->shapeTexture->setResolution(shapeImage.getWidth(), shapeImage.getHeight());
	this->shapeTexture->setPixelFormat(pixelFormat);
	this->shapeTexture->scheduleTransitionTo(GpuResidency::Resident);

	if (nullptr == this->shapeStagingTexture)
	{
		this->shapeStagingTexture = textureManager->getStagingTexture(shapeImage.getWidth(), shapeImage.getHeight(), 1u, 1u, pixelFormat);
	}
	this->shapeStagingTexture->startMapRegion();
	TextureBox texBox = this->shapeStagingTexture->mapRegion(shapeImage.getWidth(), shapeImage.getHeight(), 1u, 1u, pixelFormat);

	//for( uint8 mip=0; mip<numMipmaps; ++mip )
	texBox.copyFrom(shapeImage.getData(0));
	this->shapeStagingTexture->stopMapRegion();
	this->shapeStagingTexture->upload(texBox, this->shapeTexture, 0, 0, 0);

	if (nullptr != this->pass)
	{
		Ogre::TextureUnitState* texUnitState = this->pass->getTextureUnitState("Keyhole_Mask");
		texUnitState->setTexture(this->shapeTexture);
	}

The keyhole material:

Code: Select all

// GLSL shaders
fragment_program Postprocess/Keyhole_ps_GLSL glsl
{
	source Keyhole_ps.glsl
	
default_params
{
	param_named Keyhole_Mask int 0
}
}

// HLSL shaders
fragment_program Postprocess/Keyhole_ps_HLSL hlsl
{
	source Keyhole_ps.hlsl
	entry_point main
	target ps_5_0 ps_4_0 ps_4_0_level_9_1 ps_4_0_level_9_3
}

fragment_program Postprocess/Keyhole_ps unified
{
	delegate Postprocess/Keyhole_ps_HLSL
	delegate Postprocess/Keyhole_ps_GLSL
}

material Postprocess/KeyholeEffect
{
    technique
    {
        pass
        {
            depth_check off
            depth_write off
            cull_hardware none

        // Use the default compositor vertex shader
        vertex_program_ref Ogre/Compositor/Quad_vs
        {
        }

        // Reference the unified keyhole fragment shader
        fragment_program_ref Postprocess/Keyhole_ps
        {

        }

        texture_unit Keyhole_Mask
        {
            tex_address_mode clamp
            filtering trilinear
	texture Keyhole_Mask.png 2d
        }
    }
}
}

The hlsl shader:

Code: Select all

Texture2D Keyhole_Mask : register(t0);  // Keyhole mask texture
SamplerState samplerState : register(s0);

float4 main
(
    float2 texCoord  : TEXCOORD0
) : SV_Target
{
    // Just output the sampled texture color for debugging
	float4 image = Keyhole_Mask.Sample(samplerState, texCoord);
    return image;
}

What could be wrong here? I get no error nothing.

Best Regards
Lax

[dark_sylinc]

I do not see the mask as compositor.

This is a bit vague. I assume you mean you're using a pass_quad but it doesn't show up at all?

Have you tried looking at RenderDoc to see what the quad pass is actually doing? (e.g. what texture is actually using, where in the screen the rectangle is, the values from the UVs. etc)