[2.1] Hlms - Create basic vertex shader

Problems building or running the engine, queries about how to use features etc.
Post Reply
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

[2.1] Hlms - Create basic vertex shader

Post by Herb »

Ok, now that my app is ported to 2.1, I'm taking a look at shaders. I've done some basic vertex shaders (via Cg) with Ogre 1.9, so that's my frame of reference.

Reading through forum posts and the Ogre porting doc, I'm still a bit confused.... :? What is the workflow for this?

It seems like to implement my own GLSL vertex shader, I would have to implement a custom HLMS? Like something different than Pbs or Unlit?

I keep thinking there would be a way to use Pbs and add some other block type to do a custom vertex shader? I'm used to adding the Cg shader in the material and pushing any needed variables in code.

I can't seem to find a basic "hello world" of tying a basic OpenGL shader to the HLMS. Could anyone give me some tips or point me at an example that would help?
xrgo
OGRE Expert User
OGRE Expert User
Posts: 1148
Joined: Sat Jul 06, 2013 10:59 pm
Location: Chile
x 168

Re: [2.1] Hlms - Create basic vertex shader

Post by xrgo »

Most of the stuffs you'll ever need can be possible with the default pbs and unlit implementations and those are very easy to use.
But sometimes one needs to create something different and sadly working with the Hlms to create completely new stuffs is very lowlevelish... but very powerful!
Kinslore wrote this basic example:
http://www.ogre3d.org/forums/viewtopic. ... 79#p519340
I have used it to create my Hlms Sky material and works great =)
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Hlms - Create basic vertex shader

Post by dark_sylinc »

Hi, depends on what you want to do:
  • If it's just for one or a few objects, or for prototyping, you may still find using the old material system (now called "low level materials") will just work, as usual (except you'll need to use HLSL and/or GLSL instead of Cg)
  • If you just need to add some extra functionality to our existing shaders (i.e. I want the trees to "wave" and "shake" with my custom vertex shader code, but I still want PBS lighting) then the way to go is described in "8.6.5 Customizing an existing implementation" from the manual, where you can insert your own code. There's no sample currently showing how to do it but our users have grasped the interface quite quickly.
  • If you want something radically different, with the flexibility and performance of the Hlms (i.e. I wrote my own sky shader with this), then you need to create your own Hlms implementation.
Recommended reading:
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

ok, that makes sense. Thanks for the clarifications. Sounds like customizing might be where I'll start and we'll see where it goes from there.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Hlms - Create basic vertex shader

Post by dark_sylinc »

I'd personally suggest making your own Hlms first.
Something very simple:

Code: Select all

#version 330 core
#extension GL_ARB_shading_language_420pack: require
out gl_PerVertex
{
	vec4 gl_Position;
};
in vec4 position;
void main()
{
     gl_Position = position.xyzw;
}

Code: Select all

#version 330 core
#extension GL_ARB_shading_language_420pack: require
layout(location = FRAG_COLOR, index = 0) out vec4 outColour;
void main()
{
     outColour = vec4( 1, 1, 1, 1 );
}
Just to get used to how it works. You don't have to use the Hlms template keywords (like @property). They're optional.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

Ok, I think a few more comments and i'll get a break through on this....

Ok, let's say I'm starting off with my own Hlms. I have a basic shader like the simple one dark_sylinc pasted and I'm trying to run it against an mesh. So to do that, I need to create a "Hlms implementation" class like the Pbs one that inherits from HlmsBufferManager and then create a custom DataBlock class inheriting from HlmsDataBlock. Then in the initialization code for the app, set the GLSL shaders (like the pasted one) as the archive of shaders to this new hlms type that I register. Is this "roughly" the process?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Hlms - Create basic vertex shader

Post by dark_sylinc »

Yes. Beware I typed it with no world or projection matrix, so it will just render "as is". Rendering a triangle with vertices [-1;-1;0] [1;-1;0] [1;1;0] will cover half of the screen.

Then you add a const or tex buffer so you can pass the worldviewproj matrix, and work progressively from there.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

gotcha, makes sense. Thanks dark_sylinc!
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

I'm grabbing the example code for a simple Hlms I saw in this post: http://www.ogre3d.org/forums/viewtopic.php?f=25&t=83763

A few things I'm scratching my head at.... This example's new Hlms inherits from Ogre::Hlms, but Pbs and Unlit inherit from both HlmsBufferManager and ConstBufferPool. Is there a reason why, and what one should I use?

Using the example code I saw in the post I referenced above, I'm working through a few compile issues, but I'm getting TONS of incomplete types and undeclared types from OgreHlms.h and OgreHlmsCommon.h and a few other files.... :? I find this strange.... Am I missing something? I'm putting this new Hlms code right in my Ogre application. I have the FindOgre script (latest version) for both includes and linking main and component libs (including Pbs and Unlit). Any ideas?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Hlms - Create basic vertex shader

Post by dark_sylinc »

Herb wrote:A few things I'm scratching my head at.... This example's new Hlms inherits from Ogre::Hlms, but Pbs and Unlit inherit from both HlmsBufferManager and ConstBufferPool. Is there a reason why, and what one should I use?
In one of those posts it's explained in detail, but the short story is those two classes are helper functions useful for desktop APIs (or even modern mobile APIs, i.e. not GLES2) that help managing reusable const & texture buffers per frame (HlmsBufferManager derives from Hlms btw).
While ConstBufferPool is also a helper function for maintaining an array of materials up to date GPU side; i.e. in the shader we do (simplified):

Code: Select all

struct Material
{
    float4 diffuse;
    float4 specular;
};
uniform Material materials[256];
uniform uint materialIdxBuffer[]; //Not valid syntax, for explanation purposes.
in uint drawId;
uint materialIdx = materialIdxBuffer[drawId]
Material material = materials[materialIdx];
This selection is done GPU side. In DX9 this didn't exist. In DX9 we would set materialDiffuse and materialSpecular from the CPU every time it needed to change, which added a lot of CPU overhead. In Ogre 2.1 we just swap materials[256]'s pointer every now and then. Note that this GPU-side selection isn't free (i.e. on GCN it requires more VGPR registers instead of SGPR register). But it's an acceptable trade off, at least for us.
ConstBufferPool is an utility class for keeping materials[256]'s GPU pointers up to date with the CPU data. That's basically the point of this thread.

You don't neeeeed them, but they're useful (particularly for Pbs and Unlit which need to support lots of different materials and work be applied to lots of different meshes). For example, I didn't need them for my sky shader. Nor for a terrain shader.

Coming from Cg/DX9/Ogre 1.x; you had a mentality where a shader was a "thing" where you set a couple of switches, and fired it off. If you needed a different setting, you flipped those switches again, and launched another shader. And do so on.

DX11/GL3+/Ogre 2.1 treats shaders with the mentality that a shader is just a program running in GPU, and manipulates pointers to GPU memory. Instead of flipping switches each time we needed to make a change, we just write all our parameters to a pointer in memory, and when we've ran out of memory, we change the pointers.
I explained this in detail in Performance Questions thread (note: I'm adding this to recommended reading post).
but I'm getting TONS of incomplete types and undeclared types from OgreHlms.h and OgreHlmsCommon.h and a few other files.... :? I find this strange.... Am I missing something?
C++ Forward declarations. You're probably missing one or two include headers, once you include them these errors will be gone. Without some hint of the errors I can't guess which files you're missing to include.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

Ok, making more sense. yes, I will commit to doing that reading. :)

As for the errors, here's some snippets below...

Number of errors on ptrdiff_t

Code: Select all

/usr/local/include/OGRE/OgreHlmsCommon.h:113:13: error: 'ptrdiff_t' was not declared in this scope
I added reference in my HLMS and datablock class for <cstddef> where I know it's defined but still upset.

Several more on shared pointer GpuProgramPtr.

Code: Select all

 
/usr/local/include/OGRE/OgreHlmsCommon.h: At global scope:
/usr/local/include/OGRE/OgreHlmsCommon.h:183:25: error: field 'vertexShader' has incomplete type
         GpuProgramPtr   vertexShader;
                         ^
/usr/local/include/OGRE/OgreHlmsCommon.h:184:25: error: field 'geometryShader' has incomplete type
         GpuProgramPtr   geometryShader;
                         ^
/usr/local/include/OGRE/OgreHlmsCommon.h:185:25: error: field 'tesselationHullShader' has incomplete type
         GpuProgramPtr   tesselationHullShader;
                         ^
/usr/local/include/OGRE/OgreHlmsCommon.h:186:25: error: field 'tesselationDomainShader' has incomplete type
         GpuProgramPtr   tesselationDomainShader;
                         ^
/usr/local/include/OGRE/OgreHlmsCommon.h:187:25: error: field 'pixelShader' has incomplete type
         GpuProgramPtr   pixelShader;   


I've added both

#include "OgreGpuProgram.h"
#include "OgreSharedPtr.h"

and it's still complaining, not sure what header references I'm missing.

Oh, things like "ArchiveVec not declared, several, like this example one below

Code: Select all

                        
/usr/local/include/OGRE/OgreHlms.h:303:15: error: 'ArchiveVec' has not been declared
               ArchiveVec *libraryFolders );


I added reference to OgreArchive.h where I see the typedefl for ArchiveVec, but still angry...

So, yeah, scratching my head a bit on this. :?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [2.1] Hlms - Create basic vertex shader

Post by dark_sylinc »

Don't include OgreHlmsCommon.h directly.

Code: Select all

//#include "OgreArchive.h" //Goes first! Needed? Not sure if OgrePrerequisites.h includes it.
#include "OgrePrerequisites.h"
#include "OgreHlms.h"

#include "OgreHighLevelGpuProgramManager.h"
#include "OgreHighLevelGpuProgram.h"
That should work.
User avatar
Herb
Orc
Posts: 412
Joined: Thu Jun 04, 2009 3:21 am
Location: Kalamazoo,MI
x 38

Re: [2.1] Hlms - Create basic vertex shader

Post by Herb »

Hmm..still having issues. Maybe it's since OgreHlms.h includes OgreHlmsCommon.h.... Most of the forward declare issues are coming from OgreHlmsCommon.h which the compiler is telling me it's from the including of OgreHlms.h in my custom HLMS class header. Anyways, I'll poke at for a bit. I need to read those posts referenced a little more first anyways. Sounds like using the HlmsBufferManager will be good to use anyways and I can copy the include structure for that based off Pbs and Unlit.
Post Reply