[SOLVED]Need help with HLSL shader and reading PF_L8 Texture

Problems building or running the engine, queries about how to use features etc.
Post Reply
User avatar
petrocket
Gremlin
Posts: 178
Joined: Tue Mar 20, 2007 3:29 am
x 10
Contact:

[SOLVED]Need help with HLSL shader and reading PF_L8 Texture

Post by petrocket »

I've spent a lot of time trying to debug this HLSL shader/pixel format problem.

I've created a dynamic texture PF_L8 like this:

Code: Select all

        heightMapTex = TextureManager::getSingleton().createManual(
	        "rttHeightMap",
	        "General", 
	        TEX_TYPE_2D, 
                33, 
	        33, 
	        0, 
	        PF_L8, 
	        TU_DYNAMIC_WRITE_ONLY
        );
Then I filled it like this:

Code: Select all

    TexturePtr heightMapTex = TextureManager::getSingleton().getByName("rttHeightMap");
    HardwarePixelBufferSharedPtr heightPBuf = heightMapTex->getBuffer();

    // lock the buffer
    heightPBuf->lock(HardwareBuffer::HBL_DISCARD); 
    const PixelBox& heightPBox = heightPBuf->getCurrentLock();

    // case to uint8
    uint8* heightDest = static_cast<uint8*>(heightPBox.data);

    for(unsigned int i = 0; i < heightPBox.getWidth(); ++i) {
        for(unsigned int j = 0; j < heightPBox.getHeight(); ++j) {
             *heightDest++ = 255;
        }
     }

     heightPBuf->unlock();
Then my simple HLSL shader does this:

Code: Select all

void main_vp( float4 position      : POSITION,
			  float2 uv		       : TEXCOORD0,						  
			  uniform float4x4 worldViewProj,
			  
			  
			  out float4 oPosition : POSITION,
			  out float2 oUv	   : TEXCOORD0
			  )
{
	oPosition = mul(worldViewProj, position);
	oUv = uv;
}

void main_fp(
	float2 uv : TEXCOORD0,
	uniform sampler2D heightMap : register(s0),  // <-- this is the dynamic texture that should be white
	out float4 outcolour : COLOR0
)
{
	float altitude = tex2D(heightMap,uv).r;

        // outcolour should be the height - all white for now because we set it to 255 for every pixel
	outcolour = float4(altitude,altitude,altitude,1.0f);
}
The problem is that this code doesn't output an all-white texture! It outputs a texture that the top half is white and the bottom half is a solid grey.
My gfx card is a 8600M GT and it supports non-power of 2 textures (From the log -> 22:12:12: * Non-power-of-two textures: yes)

In OpenGL with a GLSL shader that does the same thing this works fine, but in DirectX it doesn't work. Funny thing is if I use tex2D(heightMap, uv).a the texture is all white.

Should I be using a different pixel format? I tried PF_FLOAT16_R and PF_FLOAT_32R and changed the cast from uint8 to uint16 and uint32, but both of those made the texture appear completely black.

*so confused* :(
Last edited by petrocket on Wed Jul 22, 2009 4:25 am, edited 1 time in total.
Ogre API & Manual | Ogre Wiki

blog | website | work

Follow me on twitter @ twitter.com/petrocket
User avatar
petrocket
Gremlin
Posts: 178
Joined: Tue Mar 20, 2007 3:29 am
x 10
Contact:

Re: Need help with HLSL shader and reading PF_L8 Dynamic Texture

Post by petrocket »

Well I figured out the solution but I don't know why it works:

My original code for writing to the pixel buffer was:

Code: Select all

    uint8* heightDest = static_cast<uint8*>(heightPBox.data);

    for(unsigned int i = 0; i < heightPBox.getHeight(); ++i) {
        for(unsigned int j = 0; j < heightPBox.getWidth(); ++j) {
           *heightDest++ =255;
        }
    }
And I changed it to this:

Code: Select all

    uint8* heightDest = static_cast<uint8*>(heightPBox.data);
    const int hPitch = (int)heightPBox.rowPitch;

    for(unsigned int i = 0; i < heightPBox.getHeight(); ++i) {
        for(unsigned int j = 0; j < heightPBox.getWidth(); ++j) {
            heightDest[(hPitch * i) + j] = 255;
        }
    }
Ogre API & Manual | Ogre Wiki

blog | website | work

Follow me on twitter @ twitter.com/petrocket
Post Reply