[Solved] Custom moveable textbox trouble (shading)

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
hedphelym
Gremlin
Posts: 180
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 23
Contact:

[Solved] Custom moveable textbox trouble (shading)

Post by hedphelym »

We have a implementation of moveabletext (billboard with text) that we've used in earlier ogre.
Now I'm working on getting this to work with the latest ogre, and I've been debugging a lot, but have not found a solution yet.

It seems to be as if the mesh object itself, and mesh for 'letters' are working, they show up looking as it should, and gets updated when the
text is changed, like this:
Image

Also the font used is loaded fine.

We set the vertex\fragment program through code when the moveable text is created:

Code: Select all

mpMaterial->getBestTechnique()->getPass(0)->setVertexProgram(vertexProgramName);
mpMaterial->getBestTechnique()->getPass(0)->setFragmentProgram(fragmentProgramName);
The problem seems to the the last part of it all - the pixel shader \ program.
This is what I know the least about in the whole process, so I wanted to ask here, and maybe some one of you who knows more about it can
enlighten me if there is something wrong with it (that has to do with ogre 2.1).

TextBox.program :

Code: Select all

vertex_program TextBox_vs hlsl
{
    target vs_4_0
    source TextBox.hlsl
    entry_point TextBox_vs

    default_params
    {
		param_named_auto wvpMat worldviewproj_matrix
		param_named_auto wMatInv inverse_world_matrix
		param_named arrowTargetWS float4 0.0 0.0 0.0 0.0
    }
}

fragment_program TextBox_ps hlsl
{
    target ps_4_0
    source TextBox.hlsl
    entry_point TextBox_ps

    default_params
    {
		param_named rowsAlphaOffset float 0.0
    }
}
TextBox.hlsl:

Code: Select all

float4 TextBox_vs
(
	in float4 			inPosOS			: POSITION,
    in float2 			inUV			: TEXCOORD0,
	in float4 			inColor			: COLOR,
	uniform float4x4 	wvpMat,
	uniform float4x4	wMatInv,
	uniform float4 		arrowTargetWS,
    out float2 			outUV			: TEXCOORD0,
	out float4 			outPosOS		: TEXCOORD1,
	out float4 			outColor 		: COLOR
) 
: SV_Position
{
	// If this is the arrow target (uv == 3.5) and we have a target (arrowTargetWS.w != 0.0), move this point to the target position.
	if(inUV.x > 3.4 && inUV.x < 3.6 && arrowTargetWS.w != 0.0)
	{
		inPosOS = mul(wMatInv, arrowTargetWS); //float4(0.0, 5.0, 0.0, 1.0));
	}
	outPosOS = inPosOS;
	outUV = inUV;
	outColor = inColor;
	float4 outPosSS = mul(wvpMat, inPosOS);	
	return outPosSS;
}


Texture2D fontMap : register(t0);
SamplerState fontSampler : register(s0);

float4 TextBox_ps
(
	in float2 			inUV			: TEXCOORD0,
	in float4 			inPosOO			: TEXCOORD1,
	in float4 			inColor			: COLOR,
	uniform float 		rowsAlphaOffset
)
: SV_TARGET
{
	float4 outColor = float4(0, 0, 0, 1);
	
	if(inUV.x >= 1.0 && inUV.y >= 1.0)		// This is the background if UV is outside range [0, 1)
	{	
		outColor = inColor;		
		
		// Rows with slightly different color
		if(rowsAlphaOffset > 0.0)
		{
			if((int)((inPosOO.y + 1000.0) * 4.0) % 2 == 0)	
			{
				outColor.a += rowsAlphaOffset;
			}
			else
			{
				outColor.a -= rowsAlphaOffset;
			}	
		}
	}
	else									// Else, it is font
	{
		outColor = fontMap.Sample(fontSampler, inUV) * inColor;
	}
	
	return outColor;
}
The result in Ogre viewport looks like this:
Image



And when I use 'RenderDoc' to debug, I see that the fontmap is loaded in and referenced in the mesh's material.
It looks like this:
(This is why I suspect that there is something wrong in the shader code above causing the uv's not to align the way they should.)
In the debugger I see the uv's for each letter is set, so the mesh itself has them.
Image
Last edited by hedphelym on Fri Apr 13, 2018 4:56 pm, edited 1 time in total.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Custom moveable textbox trouble (shading)

Post by dark_sylinc »

I don't see anything 2.x-related that could be wrong.

I'd suggest to look at RenderDoc's:
  • Mesh Output: See that the input UVs are right; and check that the output UVs coincide with their input and that they make sense for what you're trying to do
  • Pipeline State: Check that the shaders being used in both VS & PS are actually the ones you want, and check the parameters, that their values are what you set
hedphelym
Gremlin
Posts: 180
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 23
Contact:

Re: Custom moveable textbox trouble (shading)

Post by hedphelym »

Hi, thank you for the reply.

The uv in the VS Input, and VS Output shows the same values in renderdoc.

I now manually 'plotted' a set of uv's of a character I know (,), then plot that on top of the image,
it then lands directly on the character it's supposed to be at (green box is uv coordinates for the two faces that makes up the letter.

Image

I do not know if this has something to do with it, but the format in the texture viewer in 'RenderDoc' is: R8G8_UNORM

I set the vertex \ fragment by name:

Code: Select all

const std::string vertexProgramName = "TextBox_vs";
const std::string fragmentProgramName = "TextBox_ps";
			
if(!Ogre::HighLevelGpuProgramManager::getSingleton().getByName(vertexProgramName).isNull() && !Ogre::HighLevelGpuProgramManager::getSingleton().getByName(fragmentProgramName).isNull())
{
	mpMaterial->getBestTechnique()->getPass(0)->setVertexProgram(vertexProgramName);
	mpMaterial->getBestTechnique()->getPass(0)->setFragmentProgram(fragmentProgramName);
}else
{
	Ogre::LogManager::getSingleton().logMessage("MovableText::setFontName(...) - Could not find shaders (\"" + vertexProgramName + "\" and \"" + fragmentProgramName + "\"). Fixed function shader will be used instead, causing some features to be disabled.", Ogre::LML_CRITICAL);
}
And the shader that is loaded is the correct one (I have checked that).
hedphelym
Gremlin
Posts: 180
Joined: Tue Nov 25, 2008 10:58 am
Location: Kristiansand, Norway
x 23
Contact:

Re: Custom moveable textbox trouble (shading)

Post by hedphelym »

Found the problem.
The UV's where located in 'inPosOO', and not 'inUV'.
When I use that it works as intended.
Post Reply