Shader Issue - TEXCOORD changes my value

Problems building or running the engine, queries about how to use features etc.
Post Reply
MattStevens
Goblin
Posts: 239
Joined: Mon Apr 07, 2008 10:27 pm

Shader Issue - TEXCOORD changes my value

Post by MattStevens » Wed Nov 18, 2009 9:35 pm

Hello,
I have an issue with my shader. It is very simple in fact, I just want to make a check and see if the vertex that is rendering is a front face or a backface (The material has both cull_software and cull_hardware set to none). Both my vertex / fragment programs are cg and uses the vs_2_0 & ps_2_x profiles.

An easy way I thought I could use is a semantic from CG shader language. The CG Users Manual says that the FACE can do this automaticly. However it does not work.
CG Users Manual wrote: The FACE semantic can be applied to a varying parameter to a program. The value of such a parameter has a value less than zero if the fragment being rendered is back facing, greater than zero if it is front facing, and zero if the fragment was from a line or a point.
I get the following error log when trying to use FACE semantic in either of the shaders.

Code: Select all

float4 CartoonPS(PS_INPUT Input,
					float testFace : FACE,
					[other params, samplers, etc...]
					) : COLOR
{...}

error C5108: unknown semantics "FACE" specified for "testFace"
So then I decided to calculate it myself, by using the dot product of the face normal and the view vector in the vertex program. Here is the .cg and .program file.

Code: Select all

float IsBackFace(float3 worldNormal, float3 viewVector)
{
	if(dot(worldNormal, viewVector) >= 0.0)
		return 0.0;
	else
		return 1.0;
}

struct VS_INPUT
{
	float4 Position		: POSITION;
	float3 Normal		: NORMAL;
};

struct PS_INPUT
{
	float4 Position					: POSITION;

	float3 WViewDirection			: TEXCOORD0;
	float3 WNormal					: TEXCOORD1;
	float Backface					: TEXCOORD2;
};
					
PS_INPUT CartoonVS(VS_INPUT Input,
		uniform float4		eyePosition,
		uniform float4x4	matWorld,
		uniform float4x4	matWorldViewProj)
{
	PS_INPUT Output;

	Output.Position = mul( matWorldViewProj, Input.Position );

	float4 worldPosition = mul(matWorld, Input.Position );

	Output.WNormal = normalize(mul((float3x3)matWorld, Input.Normal));
	Output.WViewDirection = normalize(eyePosition - worldPosition).xyz;
	
	Output.Backface = IsBackFace(Output.WNormal, Output.WViewDirection);

	return Output;
}

float4 CartoonPS(PS_INPUT Input) : COLOR
{
	float4 outColor;
	if(Input.Backface == 0.0)
	{
		outColor = float4(1.0, 0.0, 0.0, 1.0);
	}
	else if(Input.Backface == 1.0)
	{
		outColor = float4(0.0, 1.0, 0.0, 1.0);
	}
	else
	{
		outColor = float4(0.0, 0.0, Input.Backface, 1.0);
	}

	return outColor;
}

Code: Select all

vertex_program CartoonVS cg
{
	source Cartoon.cg
	entry_point CartoonVS
	profiles vs_2_0

	default_params
	{
		param_named_auto eyePosition camera_position
		param_named_auto matWorld world_matrix
		param_named_auto matWorldViewProj worldviewproj_matrix
	}
}

fragment_program CartoonPS cg
{
	source Cartoon.cg
	entry_point CartoonPS
	profiles ps_2_x

	default_params
	{
	}
}
Basically I check if it is a front or a back face and I pass the value 1 or 0 to the fragment program. Depending on that value, I render the red or green color. If the value isn't 1 or 0, I output shades of blue.

Now, you think I should only see red or green, right ? WRONG. I have blue, and a lot of blue actually. I really don't understand why my value would change between the vertex and fragment program. I know that the graphic card extrapolation some values between the vertex and fragment program, for example the uv mapping, position, and maybe some other stuff, but it shouldn't change my Backface value.

The first picture shows my character from the outside.
The second picture is taken from INSIDE the character. What you see are the backfaces of character's body.

Image
Image

- Matt
0 x

User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US

Re: Shader Issue - TEXCOORD changes my value

Post by xavier » Wed Nov 18, 2009 10:26 pm

MattStevens wrote:I really don't understand why my value would change between the vertex and fragment program. I know that the graphic card extrapolation some values between the vertex and fragment program, for example the uv mapping, position, and maybe some other stuff, but it shouldn't change my Backface value.
It interpolates anything that is in a TEXCOORDx semantic.
0 x
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.

MattStevens
Goblin
Posts: 239
Joined: Mon Apr 07, 2008 10:27 pm

Re: Shader Issue - TEXCOORD changes my value

Post by MattStevens » Wed Nov 18, 2009 10:39 pm

xavier wrote:
MattStevens wrote:I really don't understand why my value would change between the vertex and fragment program. I know that the graphic card extrapolation some values between the vertex and fragment program, for example the uv mapping, position, and maybe some other stuff, but it shouldn't change my Backface value.
It interpolates anything that is in a TEXCOORDx semantic.
Based on what exactly? I'll try to find some informations on the subject but I thought it had to know what it is working with to be able to interpolate the TEXCOORD.

And if I want to pass a value without changing it, what do I use ? There's always the COLOR semantic, but it is usually used for other stuff.

Also I tried doing the calculation in the fragment program, but the Normal happens to be interpolated also and the edges are of the wrong color.

- Matt
0 x

User avatar
nullsquared
Old One
Posts: 3245
Joined: Tue Apr 24, 2007 8:23 pm
Location: NY, NY, USA

Re: Shader Issue - TEXCOORD changes my value

Post by nullsquared » Thu Nov 19, 2009 1:08 am

MattStevens wrote: Based on what exactly? I'll try to find some informations on the subject but I thought it had to know what it is working with to be able to interpolate the TEXCOORD.
Based on distance to the parenting vertices and other factors. There's no way to keep any parameters unchanged since there is no 1:1 ratio between vertices and fragments, any 3 vertices making up a triangle can take any number of fragments.
And if I want to pass a value without changing it, what do I use ?
You can't.
0 x

User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US

Re: Shader Issue - TEXCOORD changes my value

Post by xavier » Thu Nov 19, 2009 1:26 am

MattStevens wrote:
xavier wrote:
MattStevens wrote:I really don't understand why my value would change between the vertex and fragment program. I know that the graphic card extrapolation some values between the vertex and fragment program, for example the uv mapping, position, and maybe some other stuff, but it shouldn't change my Backface value.
It interpolates anything that is in a TEXCOORDx semantic.
Based on what exactly?
Based on the values at the vertices between which it is interpolating.
And if I want to pass a value without changing it, what do I use ? There's always the COLOR semantic, but it is usually used for other stuff.
You can't use certain semantics as they are not available to the fragment shader -- POSITION and COLOR are two of those. That's why TEXCOORDx is used for everything else -- but those are hardwired to interpolate between the values at the vertices in the triangle.

So you have a face normal, and a view vector. One is not changing per triangle, and the other is not changing over the whole primitive render. In other words, your dot product of the two should be constant over the triangle face, and no interpolation should be happening between the two (since they are identical). Agreed that's odd. I would recommend running the app in PIX (assuming you are using Windows and Direct3D of course) and debug the shaders to see that you are generating the values you think you are.
0 x
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.

MattStevens
Goblin
Posts: 239
Joined: Mon Apr 07, 2008 10:27 pm

Re: Shader Issue - TEXCOORD changes my value

Post by MattStevens » Thu Nov 19, 2009 4:45 pm

EDIT: Oups, I didn't see nullsquared's post when replying. Thanks for the extra infos.
xavier wrote:Based on the values at the vertices between which it is interpolating.
Wow, thanks a lot. This little sentence just made me understand the whole interpolating process. If I understand well, the vertex program runs for the 3 vertices of a triangle, and then it runs the fragment program. Since it got the values for all vertex, it interpolates the TEXCOORD semantic without worrying what kind of values are inside. It's just mindless computation. Am I right ?
xavier wrote:So you have a face normal, and a view vector. One is not changing per triangle, and the other is not changing over the whole primitive render. In other words, your dot product of the two should be constant over the triangle face, and no interpolation should be happening between the two (since they are identical). Agreed that's odd. I would recommend running the app in PIX (assuming you are using Windows and Direct3D of course) and debug the shaders to see that you are generating the values you think you are.
Actually, the view vector should be slightly different for each vertex, since the camera position does not change but the vertex position does. However it should not matter, the result of the dot product should always be > 0 or < 0 for all 3 vertices.

I just did a little test for the normals. I've just output the Normal as the color by putting this line in the above Fragment Program

Code: Select all

outColor = float4(Input.WNormal, 1);
If the Normals were effectively identical for every vertex of a triangle I expected to clearly see every one of them, but it is not the case, the colors are smooth. This makes me believe that the normals are not identical, and it would explain the problems that I have.
The following picture has both wireless and solid rendering of a characters face.
Image

Also I'm looking into PIX right now, I've never used anything to debug shaders so far but it looks quite useful.

Thanks for your help xavier,
- Matt
0 x

User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US

Re: Shader Issue - TEXCOORD changes my value

Post by xavier » Thu Nov 19, 2009 6:57 pm

MattStevens wrote:EDIT: Oups, I didn't see nullsquared's post when replying. Thanks for the extra infos.
xavier wrote:Based on the values at the vertices between which it is interpolating.
Wow, thanks a lot. This little sentence just made me understand the whole interpolating process. If I understand well, the vertex program runs for the 3 vertices of a triangle, and then it runs the fragment program. Since it got the values for all vertex, it interpolates the TEXCOORD semantic without worrying what kind of values are inside. It's just mindless computation. Am I right ?
Correct -- simple linear interpolation of the vector components (it interpolates x, y, z and w -- aka rgba -- independently). This, incidentally, is why you need to re-normalize in the fragment shader, things like normals, light vectors, etc, that you pass through the interpolator.

Also I'm looking into PIX right now, I've never used anything to debug shaders so far but it looks quite useful.

Thanks for your help xavier,
- Matt
Glad I could assist. :)
0 x
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.

nbeato
Gnome
Posts: 372
Joined: Thu Dec 20, 2007 1:00 am
Location: Florida
x 1
Contact:

Re: Shader Issue - TEXCOORD changes my value

Post by nbeato » Thu Nov 19, 2009 11:02 pm

http://en.wikipedia.org/wiki/Barycentri ... thematics)

Read the section called "Interpolation on a triangular unstructured grid".
0 x

MattStevens
Goblin
Posts: 239
Joined: Mon Apr 07, 2008 10:27 pm

Re: Shader Issue - TEXCOORD changes my value

Post by MattStevens » Fri Nov 20, 2009 7:13 pm

Thanks for the link nbeato, very interesting.

Also I have more infos on my "normal problem". Since there is only 1 normal / vertex and most of the vertex of my characters are shared between 2 or more triangles, their normals is rarely, if ever, identical to a face's normal. Because of that I don't think I can use the dot(ViewVector, Normal) to find the side of the face.

So to come back to my original problem, I'm trying to know if I am rendering a front face or a back face in my vertex program. Is there any way to get this ?

Also, I am wondering, since my topic title isn't revelant anymore, should I rename it, or start a new thread ?

- Matt
0 x

User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US

Re: Shader Issue - TEXCOORD changes my value

Post by xavier » Fri Nov 20, 2009 7:18 pm

MattStevens wrote:Thanks for the link nbeato, very interesting.

Also I have more infos on my "normal problem". Since there is only 1 normal / vertex and most of the vertex of my characters are shared between 2 or more triangles, their normals is rarely, if ever, identical to a face's normal. Because of that I don't think I can use the dot(ViewVector, Normal) to find the side of the face.
It depends on whether you are using "smooth" normals (averaged among all the faces that share the vertex) or not. It sounds like you are, in which case you probably need to send the face normal in an additional element in your vertex decl if you want access to the per-face normals.
0 x
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.

Post Reply