Using a fragment shader to convert YUV to RGB

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.
Post Reply
JeDi
Gnome
Posts: 351
Joined: Thu Oct 21, 2004 1:34 pm
Location: Diepenbeek, Belgium
x 3
Contact:

Using a fragment shader to convert YUV to RGB

Post by JeDi »

Hi,

We have some video textures that were H264 encoded, and thus in YUV420 pixel format. As Ogre/graphics hardware can't natively use this pixel format, we have to convert it to RGB(A). I want to do (part of) this in a fragment shader, but as I have no experience in this area, I have some questions.

I've found some cg source of a possible shader in this source file. However, I'm not sure how to convert this to the Ogre world.
I've put the cg source itself (without the quotes and newlines off course) in YUVtoRGB.cg, and created this Ogre material:

Code: Select all

uniform sampler2DRect Ytex;
uniform sampler2DRect Utex,Vtex;

void main(void) {
  float nx,ny,r,g,b,y,u,v;
  vec4 txl,ux,vx;
  nx=gl_TexCoord[0].x;
  ny=576.0-gl_TexCoord[0].y;
  y=texture2DRect(Ytex,vec2(nx,ny)).r;
  u=texture2DRect(Utex,vec2(nx/2.0,ny/2.0)).r;
  v=texture2DRect(Vtex,vec2(nx/2.0,ny/2.0)).r;

  y=1.1643*(y-0.0625);
  u=u-0.5;
  v=v-0.5;

  r=y+1.5958*v;
  g=y-0.39173*u-0.81290*v;
  b=y+2.017*u;

  gl_FragColor=vec4(r,g,b,1.0);
}

Code: Select all

fragment_program YUVtoRGB cg
{
  source YUVtoRGB.cg
  entry_point main
  profiles ps_2_0 arbfp1

  //default_params
  //{
  //}
}

material test/video
{
  technique
  {
    pass
    {
      texture_unit
      {
        texture test_video
        tex_address_mode clamp
      }

      fragment_program_ref YUVtoRGB
      {
      }
    }
  }
}
I have no idea how to continue from here. I'm searching for some good reference/tutorial on using shaders in Ogre, but no success so far.

I don't think the shader is very complex, so the profile I've chosen is probably way too heavy. Which parameters should I bind? Where do the gl_TexCoord and gl_FragColor variables come from? Are they automatically accessible?

What I want to do is feed the YUV data to a dynamic texture and let the fragment shader to the conversion to RGB(A).


Any help appreciated!

Greetz,
JeDi
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

That code you have there is GLSL, not cg. Also, it's expecting the 3 YUV channels as 3 separate textures rather than one texture with 3 components.

A slightly adapted version from this site leads to

Code: Select all

float3 main(
    in      float3    texcoords0 : TEXCOORD0,  
    uniform sampler yuvImage : register(s0),
) : COLOR

{

    float3 yuvcolor; // f(xyz) -> yvu

    float3 rgbcolor;

    yuvcolor.x = tex2D(yuvImage, texcoords0).x;
    yuvcolor.z = tex2D(yuvImage, texcoords0).y-0.5;
    yuvcolor.y = tex2D(yuvImage, texcoords0).z-0.5;  

    rgbcolor.r = 2*(yuvcolor.x/2 + 1.402/2 * yuvcolor.z);
    rgbcolor.g = 2*(yuvcolor.x/2 - 0.344136 * yuvcolor.y/2 - 0.714136 * yuvcolor.z/2);
    rgbcolor.b = 2*(yuvcolor.x/2 + 1.773/2 * yuvcolor.y);  

    return rgbcolor; 

} 

Not sure if that's right, just quickly thrown together!
JeDi
Gnome
Posts: 351
Joined: Thu Oct 21, 2004 1:34 pm
Location: Diepenbeek, Belgium
x 3
Contact:

Post by JeDi »

Thanks! After a bit of research, I came up with about the same shader as you posted. The numbers are a bit different though, and that may be why my results are weird. I'll try this one as soon as possible.
pragnesh_58
Gnoblar
Posts: 1
Joined: Fri Jul 14, 2017 3:23 pm

Re: Using a fragment shader to convert YUV to RGB

Post by pragnesh_58 »

1) I have used this link : http://www.fourcc.org/source/YUV420P-OpenGL-GLSLang.c and convert my YUV420 image to rgb image now i want to read back my processed RGB data back into host memory from CPU memory, how can i do this?

2) It will be more helpful if you provide me a full source code for below code (yuv -> rgb )of yours:

float3 main(
in float3 texcoords0 : TEXCOORD0,
uniform sampler yuvImage : register(s0),
) : COLOR

{

float3 yuvcolor; // f(xyz) -> yvu

float3 rgbcolor;

yuvcolor.x = tex2D(yuvImage, texcoords0).x;
yuvcolor.z = tex2D(yuvImage, texcoords0).y-0.5;
yuvcolor.y = tex2D(yuvImage, texcoords0).z-0.5;

rgbcolor.r = 2*(yuvcolor.x/2 + 1.402/2 * yuvcolor.z);
rgbcolor.g = 2*(yuvcolor.x/2 - 0.344136 * yuvcolor.y/2 - 0.714136 * yuvcolor.z/2);
rgbcolor.b = 2*(yuvcolor.x/2 + 1.773/2 * yuvcolor.y);

return rgbcolor;

}
Post Reply