Page 1 of 1

[solved] Compositor is black with radeon 5700 xt

Posted: Sun Feb 14, 2021 2:51 pm
by suny
Ogre Version: :?: 1.12.5
Operating System: :?: windows10
Render System: :?: D9D

Hi,

I have several compositors in the SHMUP Creator, and they work with most PCs.
But with some gfx cards, the result of those compositors is black, and I don't know why.
For instance, with the radeon 5700 XT... There is no error message during the shaders compilation.

I'm doing this for the compositor:

Code: Select all

compositor pixelateFX
{
    technique
    {
        texture scene target_width target_height PF_BYTE_RGB
		
        // the scene we want to modulate
        target scene
        {
            input previous
        }



        target_output
        {
            input none
			shadows off

            pass render_quad
            {
                material modulatePixelate_out
                input 0 scene
            }
        }
    }
}
My material:

Code: Select all

vertex_program pixelate_out_vp hlsl
{
	source outline.hlsl
	entry_point outline_out_vp
    target vs_1_1 vs_2_0
    default_params
   	{
		param_named_auto worldViewProj worldviewproj_matrix
   	}
}

fragment_program pixelate_out hlsl
{
	source pixelate.hlsl
	entry_point pixelate_out
    target ps_3_0
    default_params
   	{
		 param_named pixelSize float 1
		 param_named_auto resoWidth viewport_width
		 param_named_auto resoHeigth viewport_height
   	}
}


material modulatePixelate_out
{
    technique
    {
        pass
        {
			vertex_program_ref pixelate_out_vp
			{
			}
			
			fragment_program_ref pixelate_out
			{
				param_named pixelSize float 1
				param_named_auto resoWidth viewport_width
				param_named_auto resoHeigth viewport_height
			}

            texture_unit
            {
				tex_address_mode border
                texture white.bmp
                filtering bilinear
				tex_address_mode clamp
            }
        }
    }
}
And my shader:

Code: Select all

struct VIn
{
    float4 p   : POSITION;
    float2 uv  : TEXCOORD0;
};

struct VOut
{
    float4 p   : SV_POSITION;
    float2 uv  : TEXCOORD0;
};

struct PIn
{
	float4 p   : SV_POSITION;
    float2 uv  : TEXCOORD0;
};

VOut pixelate_out_vs(VIn IN, uniform float4x4 wvp)
{
    VOut OUT;
    OUT.p = mul(wvp, IN.p);
    // clean up inaccuracies for the UV coords
    float2 uv = sign(IN.p);
    // convert to image space
    uv = (float2(uv.x, -uv.y) + 1.0) * 0.5;
    OUT.uv = uv;
    return OUT;
}


//-------------------------------
//Pixelate
//-------------------------------
float4 pixelate_out(float2 texCoord: TEXCOORD0,
		uniform float pixelSize,
		uniform float resoWidth ,
		uniform float resoHeigth ,
		uniform sampler RT: register(s0)
		) : COLOR {
	
	half dx = pixelSize*(1.0/resoWidth);
	half dy = pixelSize*(1.0/resoHeigth);
	texCoord = half2(dx*floor(texCoord.x/dx), dy*floor(texCoord.y/dy));
	
	float3 finalcol = tex2D(RT, texCoord).xyz;
		
	return float4(finalcol,1);
}
If anybody has an idea, I would be glad to know why!
S.

Re: Compositor is black with radeon 5700 xt

Posted: Sun Feb 14, 2021 3:49 pm
by paroj
- maybe PF_BYTE_RGB is not supported. Try PF_BYTE_RGBA.
- SV_POSITION is technically D3D10+. Try POSITION.

Re: Compositor is black with radeon 5700 xt

Posted: Thu Apr 08, 2021 9:02 pm
by suny
I tried your ideas, and still black with all my compositosr, on every AMD cards :/
What should I do to pinpoint what is wrong?

In my code I do this:

Code: Select all

postPIXELATE = Ogre::CompositorManager::getSingleton().addCompositor(playStatePtr->mviewport2, "pixelateFX");
postPIXELATE->setEnabled(level->IsPostPixel);
postPIXELATE->notifyResized();	
S.

Re: Compositor is black with radeon 5700 xt

Posted: Fri Apr 09, 2021 2:18 pm
by dark_sylinc
  1. Enable debug layer from DX9 panel. It may reveal the error
  2. Try changing OUT.p = mul(wvp, IN.p); for OUT.p = mul(wvp, float3(IN.p.xyz,1.0f));
  3. Get rid of the half, change it to float. RDNA cards actually support half, and that may be causing trouble

Re: Compositor is black with radeon 5700 xt

Posted: Mon Apr 12, 2021 9:36 am
by suny
I tried 2 and 3 with no luck.

I newbie question: I don't know how to enable debug layer, and I can't find information about a DX9 panel.
How can I use this?
S.

Re: Compositor is black with radeon 5700 xt

Posted: Mon Apr 12, 2021 10:57 pm
by dark_sylinc
Mmmm, it's hard to find info now... I think it came with the Directx june 2010 sdk. I think it was named dxcpl.exe or dxpanel.cpl or something like that; somewhere in the installation folder (just look for .exe and .cpl files in the installed folder). If you're lucky it will install itself in the Control Panel.

It's also quite possible the debug runtimes will only work in Windows 7 (without KB 2370838) or XP. Update: Confirmed. It won't work in newer versions of Windows.

The panel looked like this.

Re: Compositor is black with radeon 5700 xt

Posted: Fri Apr 16, 2021 8:58 pm
by paroj
yeah the best thing about D3D11 and GL3+ is that you get proper debug layers for them (e.g. renderdoc). If your app supports one of them, that might be worth giving a shot.

Otherwise you can always resort to poor-mans debugging: comment everything out until you get a trivial (working!) shader and start commenting-in things line-by-line.

Re: Compositor is black with radeon 5700 xt

Posted: Wed Apr 21, 2021 7:53 am
by suny
I found a way to fix it:
I was using target ps_3_0 in the compositor's fragment shaders, and I changed it to target ps_2_b.

I'm happy, but I still don't know why those shaders were not working on AMD...
S.

Re: Compositor is black with radeon 5700 xt

Posted: Wed Apr 21, 2021 2:25 pm
by dark_sylinc
Ohhh. Now I see it. You must pair ps 3.0 with vs 3.0
You were using VS 2.0

That's why it failed