ASM to HLSL conversion

[stoneCold]

Hi all,
I know it's very seldom that I post in the artists forum, but when I do it, it's really urgent as you might guess

For a week or so I have been learning HLSL now and I'm really happy to do so
But yesterday I found something very interesting about offsetmapping at the forums and had a look on it...

http://www.ogre3d.org/phpBB2/viewtopic. ... setmap+zip

I recognised that the shader code of this example is completely written in ASM (which I really never used till yesterday), so with the help of M$DN's ASM command list, I started to convert the ASM code to HLSL.

Here's the whole code of the OpenGL example:

Code: Select all

!!ARBfp1.0

PARAM light0color = state.light[0].diffuse;
PARAM light1color = state.light[1].diffuse;
PARAM ambient = state.lightmodel.ambient;

TEMP eyevects;
TEMP rgb, height, temp, bump, total;
TEMP light0tsvec, light1tsvec;
TEMP newtexcoord;

# normalize tangent space eye vector
DP3 temp, fragment.texcoord[3], fragment.texcoord[3];
RSQ temp, temp.x;
MUL eyevects, fragment.texcoord[3], temp;

# calculate offset
TEX height, fragment.texcoord[0], texture[2], 2D;
MAD height, height, 0.03, -0.015;  # scale and bias
MAD newtexcoord, height, eyevects, fragment.texcoord[0];

# get texture data
TEX rgb, newtexcoord, texture[0], 2D;

# normalize the light0 vector
DP3 temp, fragment.texcoord[1], fragment.texcoord[1];
RSQ temp, temp.x;
MUL light0tsvec, fragment.texcoord[1], temp;

# add light0 color
MUL_SAT total, light0tsvec.z, light0color;

# normalize the light1 vector
DP3 temp, fragment.texcoord[2], fragment.texcoord[2];
RSQ temp, temp.x;
MUL light1tsvec, fragment.texcoord[2], temp;

# add light1 color
MUL_SAT temp, light1tsvec.z, light1color;
ADD_SAT total, total, temp;

# add ambient lighting
ADD_SAT total, total, ambient;

MUL_SAT result.color, rgb, total;

END

And here is my code with the problems in it:

VS:

Code: Select all

float4x4 matViewProjection;
float4 vViewPosition;

struct VS_INPUT 
{
   float4 Position : POSITION0;
   float2 Texcoord : TEXCOORD0;
   
};

struct VS_OUTPUT 
{
   float4 Position : POSITION0;
   float2 Texcoord : TEXCOORD0;
   float3 viewPosition : TEXCOORD1;   
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
   VS_OUTPUT Output;

   Output.Position = mul( matViewProjection, Input.Position );
   Output.Texcoord = Input.Texcoord;
   Output.viewPosition = normalize(vViewPosition - Input.Position);

   return( Output );
   
}

PS:

Code: Select all

sampler2D RGB_map;
sampler2D HEIGHT_map;

struct PS_INPUT 
{
   float2 Texcoord : TEXCOORD0;
   float3 viewPosition : TEXCOORD1;
   
};

float4 ps_main( PS_INPUT Input ) : COLOR0
{
   float2 height;
   float2 new_TexCoords;
   
   height = tex2D ( HEIGHT_map, Input.Texcoord) * 0.03 -0.015;
   new_TexCoords = height * Input.viewPosition + Input.Texcoord;
   return tex2D( RGB_map, new_TexCoords );
   
}

with this approach I get a crazy result... (I'll upload a image when I'm @ home later this day)

[edit]: don't mention that I left the light part away

many thanks

[klauss]

Noble enterprise, but have in mind that it's written in assembler because it's the only "quite portable" way. HLSL works only on DirectX, and GLSL needs the equivalent of ps_2_0 or above to work (besides being OpenGL-only). Cg has no ps_1_4 profile, which is required for that shader, so the only option left is using assembler which will hopefully work on many ps_1_4-capable cards thanks to the atifs extension.

If anyone knows a better (portable) option, I'm open to suggestions, since I'm facing that same portability problem myself.

[stoneCold]

Here's the promised screen:

[stoneCold]

...So there's no way to get this shader into Ogre via HLSL? ...but can I use ASM shaders as well in ogre, because I never saw a (material) example which did it.

thanks,
stoneCold

[Deranged]

...So there's no way to get this shader into Ogre via HLSL? ...but can I use ASM shaders as well in ogre, because I never saw a (material) example which did it.

thanks,
stoneCold

Yes, you can use ASM shaders and it really isn't hard, not sure if it's still done this way, but it used to be

Code: Select all

fragment_program name asm
{
  source file.asm
  syntax ps_1_1
}

or something like that.

[stoneCold]

OMG, I'm damn happy that I didn't leave my fingers off this shader...

@ klauss: I think you missunderstood me, or maybe I missunderstood you, nevertheless, you wrote that this is not possible in HLSL, but you are wrong, I (nearly) did it now, the only thing I was missing, was a filter option in RenderMonkey for the height map and now it (nearly) works fine.

There's just a problem with the view direction calculation left (just a guess of me)

Here's the openGL ASM example (with BumpMapping to see things better):


...and here's my RenderMonkey HLSL work, where you see the crazy UVs (pixels are stretched where the displacement should take place):


Is anyone able to figure out what I did wrong in the above code?
I would be very pleased for any help.

PS: I LOVE HLSL damn, what do I say......I LOVE ALL SHADING LANGUAGES

greetings & thanks,
stoneCold

[klauss]

@ klauss: I think you missunderstood me, or maybe I missunderstood you, nevertheless, you wrote that this is not possible in HLSL, but you are wrong, I (nearly) did it now, the only thing I was missing, was a filter option in RenderMonkey for the height map and now it (nearly) works fine.

Oh, no... I said it will only work under the DX9 renderer, and not others (like DX7, or OpenGL).
...or did you get it working under OpenGL?

[stoneCold]

Nope, I didn't even try it, so I can't say if it will work.

I think you missunderstood me, because I never said that I want to bring the code into GLSL but I want to bring it into HLSL which works very nice (except the problem above) as you say it.

Any solution for the problem, anyone?

[klauss]

Hm... you didn't normalize the eye vector. That surely will bring some trouble. Also, you didn't project it into tangent space:

Code: Select all

height = tex2D ( HEIGHT_map, Input.Texcoord) * 0.03 -0.015; 
new_TexCoords = height * normalize(Input.viewPosition).xy + Input.Texcoord; 
return tex2D( RGB_map, new_TexCoords );

I'm not sure that'll work (I can't test it right now), but it should.
Also, I'm not sure it will even compile, I couldn't ever manage to get a normalize() in HLSL/ps_1_4 without going over the instruction limit, and you need three of them. You'll have to use normalization cubemaps.

You should be fine, though, if you select ps_2_0 as a target.

EDIT: Input.viewPosition isn't even in tangent-space... you could pass a tangent-space vector and ease some work on the fragment shader, but let's see how it would work without it.

EDIT: Nah... use tangentspace. The original shader uses it... you have to modify your vertex shader to pass a tangentspace eye vector.

[stoneCold]

Hm... you didn't normalize the eye vector....

I did it...but in the vertex shader as you might see.

But what you say about the tangent space could be true, because I'm not sure if I did it right.....(but it's in the VS too) coud you give me a tip?

[klauss]

Vertex shader normalizations won't work... because you have to normalize after interpolation as well.

But usually, normalizing in the vertex shader is also sone to prevent register overflow with big numbers (some implementations have less precision on the registers passed to fragment shaders, so it's a good idea to normalize vectors before sending them).

Suppose you have

Code: Select all

float3 normal;
float3 binormal;
float3 tangent;

(for instance, as part of the input to your vertex shader)
Make sure those three are normalized. Then, to generate a tangentspace version of vector viewVector:

Code: Select all

float3 viewVector = ...;
float3 viewVectorts;
viewVectorts.x = dot(viewVector,tangent);
viewVectorts.y = dot(viewVector,binormal);
viewVectorts.z = dot(viewVector,normal);

You can exchange tangent and binormal, I'm not sure what's the standard ordering.

[stoneCold]

Yeah now I see, I changed my shader code to fit but now it gets just worse, the problem with the stretched UVs remains, plus a kind of lense is floating arround on the model when i move the camera.

VS:

Code: Select all

float4x4 matViewProjection;
float4 vViewPosition;

struct VS_INPUT 
{
   float4 Position : POSITION0;
   float2 Texcoord : TEXCOORD0;
   float3 Normal  : NORMAL0;
   float3 Tangent : TANGENT0;
   float3 Binormal: BINORMAL0;
};

struct VS_OUTPUT 
{
   float4 Position : POSITION0;
   float2 Texcoord : TEXCOORD0;
   float3 viewPosition : TEXCOORD1;
};

VS_OUTPUT vs_main( VS_INPUT Input )
{
   VS_OUTPUT Output;

   Output.Position = mul( matViewProjection, Input.Position );
   Output.Texcoord = Input.Texcoord;
   
   float3 viewVec = vViewPosition - Input.Position;
   viewVec.x = dot(viewVec, Input.Tangent);
   viewVec.y = dot(viewVec, Input.Binormal);
   viewVec.z = dot(viewVec, Input.Normal);
   
   Output.viewPosition = normalize(viewVec);

   return( Output );
   
}

PS:

Code: Select all

sampler2D RGB_map;
sampler2D HEIGHT_map;

float scale;
float bias;

struct PS_INPUT 
{
   float2 Texcoord : TEXCOORD0;
   float3 viewPosition : TEXCOORD1;
   
};

float4 ps_main( PS_INPUT Input ) : COLOR0
{
   float2 height;
   float2 new_TexCoords;
   
   height = tex2D ( HEIGHT_map, Input.Texcoord) * scale - bias;
   new_TexCoords = height * Input.viewPosition + Input.Texcoord;
   return tex2D( RGB_map, new_TexCoords );
   
}

many thanks, klauss
[EDIT]: do you have ICQ or MSN? it will save us a lot of time

[klauss]

PMed you.

Plus... did you normalize? It should look like:

Code: Select all

float4 ps_main( PS_INPUT Input ) : COLOR0 
{ 
   float2 height; 
   float2 new_TexCoords; 
    
   height = tex2D ( HEIGHT_map, Input.Texcoord) * scale - bias; 
   new_TexCoords = height * normalize(Input.viewPosition).xy + Input.Texcoord; 
   return tex2D( RGB_map, new_TexCoords ); 
}

(Frankly, it shouldn't even compile without the .xy ... but I guess it does?)