[Draft] Material system improvements

[scrawl]

Edit: Heavily updated version in GSOC forum: http://www.ogre3d.org/forums/viewtopic.php?f=13&t=80120

[Zonder]

Since there's no material generator that can be truly generic, it will be a pure virtual class in the Ogre core, left to be implemented by users (we will add a sample implementation to the sample browser, though).

I think this should be a supported component not a sample. But having it swappable is a very good idea.

• Add some test shaders to the sample. Add test cases for all the features of the new system (additional technique generation, changing settings, shader language switcher, etc)

It would be nice to scope in some actual shaders for the start of a ogre shader library.

I would love to implement these improvements in a GSOC next year. I would also love to start earlier, but unfortunately don't have time to dive into it now. In any case, it may be a good idea to wait until 2.0 has matured at least a bit, since the work is probably going to be based on that branch.

This defiantly needs to wait for 2.0 to mature a bit and also gives time to refine the system before coding!

[scrawl]

Since there's no material generator that can be truly generic, it will be a pure virtual class in the Ogre core, left to be implemented by users (we will add a sample implementation to the sample browser, though).

I think this should be a supported component not a sample. But having it swappable is a very good idea.

I think it's better if it's a sample. Components should be as generic as possible, and if most users are either not using it, or end up using a modified or completely from-scratch version of the generator, then it shouldn't be a component. Also, it's probably not enough code to justify a new Component.

• Add some test shaders to the sample. Add test cases for all the features of the new system (additional technique generation, changing settings, shader language switcher, etc)

It would be nice to scope in some actual shaders for the start of a ogre shader library.

Yeah sure! If we need test shaders anyway, might as well make them do something useful. Per-pixel, multilight, normal mapping, directional [PSSM] shadows would be easy enough to start with.

[Zonder]

Material generator

Since there's no material generator that can be truly generic, it will be a pure virtual class in the Ogre core, left to be implemented by users (we will add a sample implementation to the sample browser, though).

Material generators may be registered and unregistered at any time, which may be useful for swapping out entire pipelines (eg forward vs deferred)

I might be miss understanding this section, it reads to me there is no material generator in ogre so it won't function unless you add one?

[scrawl]

Material generator

Since there's no material generator that can be truly generic, it will be a pure virtual class in the Ogre core, left to be implemented by users (we will add a sample implementation to the sample browser, though).

Material generators may be registered and unregistered at any time, which may be useful for swapping out entire pipelines (eg forward vs deferred)

I might be miss understanding this section, it reads to me there is no material generator in ogre so it won't function unless you add one?

Yes, the Ogre core only contains the interface for the material generator, not the implementation. And there are good reasons for that.
First, not everyone may want to use material generators. If you use a pure fixed function pipeline, you won't need one.
If you only have very few shader effects, the O(2^n) problem doesn't really matter and you also don't need a generator.
Other users may already have their own material generator systems in place.
Second, a material generator for shader materials is pretty useless, unless it comes with shaders. And it's completely impossible to create an uber shader that will suit everyone's needs. Even if it was possible, we can not have that uber shader as a dependency for the Ogre core, because it would be far too huge.

[scrawl]

Added to draft: Macro to get the current shader type

[mkultra333]

One thing I've found over the past four years of writing shaders on my own project is that you really do run into driver bugs now and then, and you need to be able to work around them. For instance, I've had a single shader where "reflect" suddenly doesn't work and I have to actually calculate the reflection. Or where "frac" stopped working and had to be replaced with "modf", or somehow the shader compiler ignores some of the values you place in a float4 and decides to use some other value it "thinks" should be there. Weird stuff.

Working around those kinds of issues means you need to be looking at the real shader, so the above ideas probably need to have to option to output generated shaders as real HLSL or GLSL code if a workaround is needed on some driver bug.

Edit: To be honest, I myself probably wouldn't use most of this autogenerated shader stuff. I prefer just writing the shader variations myself. But as I mentioned in the other thread, I am interested in the "Shiny" idea of a meta-language that converts to either HLSL or GLSL as needed, provided it was very similar to HLSL/GLSL. Just for the cross platform capabilities. (I originally used CG but it didn't work out.) Provided I could save the generated HLSL or GLSL, the driver bug issue could be worked around too.

Is that metalanguage idea part of this proposal?

[scrawl]

I don't get what you mean. The shaders are written in HLSL or GLSL. As I said, plain old shader files, just with added macros (none of which affect the basic language syntax).

[mkultra333]

Sorry, I must have misunderstood.

[scrawl]

But as I mentioned in the other thread, I am interested in the "Shiny" idea of a meta-language that converts to either HLSL or GLSL as needed, provided it was very similar to HLSL/GLSL. Just for the cross platform capabilities. (I originally used CG but it didn't work out.) Provided I could save the generated HLSL or GLSL, the driver bug issue could be worked around too.

Is that metalanguage idea part of this proposal?

No, it is not. BUT, it will be very straightforward to implement by users. You just create a header with #defines for each target language. And with the OGRE_SHADER_LANG macro you can get the current target language.

[TheSHEEEP]

Do I understand it correctly that the macros such as @setting(foo) are written inside the actual shader code (so, HLSL/GLSL), thus serving as a kind of "extension" to the original shader language?

I also have troubles what such a Material Generator could look like, and what it would actually do.
What if I want to define the texture unit setup inside a material file and not in C++ code (so that I don't have to recompile with every change)?

[scrawl]

Do I understand it correctly that the macros such as @setting(foo) are written inside the actual shader code (so, HLSL/GLSL), thus serving as a kind of "extension" to the original shader language?

Exactly.

I also have troubles what such a Material Generator could look like, and what it would actually do.

The material generator is given a list of properties and then builds a suitable material. Here is how a very simplistic one might look.

Code: Select all

MyMaterialGenerator::generate(MaterialPtr mat, map<string,string> properties)
{
   Technique* tech = mat->getTechnique(0);
   tech->getPass(0)->createTextureUnitState(properties["diffuse_map"]);
   if (!properties["normal_map"].empty())
   {
         tech->getPass(0)->createTextureUnitState(properties["normal_map"]);
   }
   tech->getPass(0)->setVertexProgram (mVertexShaderGenerator->getShader(properties));
   tech->getPass(0)->setFragmentProgram (mFragmentShaderGenerator->getShader(properties));
}

What if I want to define the texture unit setup inside a material file and not in C++ code (so that I don't have to recompile with every change)?

The idea is to use properties for everything you would want to change in an individual material and have the material generator react to these properties.
We cannot use a regular material file because we may not know the structure of the material beforehand. Eg if a generated material has no "normal_map" property we do not want a texture unit for the normal map. Or think about doing 2-pass transparency if a certain property is true. This will radically change the structure of the material and we cannot use a predefined material file.

[Owen]

Material improvements for Ogre 2.0


Design proposal

Generated material

As we've seen, regular materials just don't cut it for many use cases. The main reason for that is that materials are a collection of very specific techniques, passes and texture units (along with their associated render states), rather than a set of physical properties used to describe a surface (as you'd expect by the term "material").
To fix this, we need to go one step higher and introduce generated materials.
A generated material has a set of properties attached to it. Its techniques, passes and texture units only get filled in once it's been generated at runtime by the material generator assigned to it.
Generated materials can also use inheritance, to avoid too much repetition of the same properties.

Why a distinct class of materials?

Why not just specify the properties on an "ordinary" material, and let Ogre do the generation?

Material generator

Since there's no material generator that can be truly generic, it will be a pure virtual class in the Ogre core, left to be implemented by users (we will add a sample implementation to the sample browser, though).

Material generators may be registered and unregistered at any time, which may be useful for swapping out entire pipelines (eg forward vs deferred)

Ogre already has a hook for this. In fact, RTSS already plugs into it!

Said hook is called whenever no viable technique exists for a material. If you didn't specify any techniques, then this hook would automatically apply.

Shader generator
The default shader generator will use plain old shader files (typically with GLSL or HLSL code) combined with a couple of preprocessing macros for added flexibility (see Macros section). Custom shader generators can also be implemented, allowing for fancier setups e.g. building shaders in Lua!

Again, there can be many shader generators being used at the same time, and it's up to the material generator to decide which shader generator(s) should be used. Example: you might have one shader generator for Deferred materials, one for regular Forward materials, one for Terrain, etc.

To reduce the amount of shaders compiled, we track the permutations of compiled shaders and just return an already existing shader if the same permutation is requested again.

This would be a functional regression over the existing RTSS.

I think any system which attempts to rejig our shader generation should at least aim for true comprehension of the shaders involved - yes, that involves parsing them.

I see two levels to this system: The low level one; a parse tree, walked by a compiler in order to build a shader for the target language (aiming for GLSL(ES)+HLSL initially; later we can perhaps move onto other outputs (someone crazy want to compile a shader to e.g. DXSI?)

Settings

Typically there are at least a couple of graphics options in a game, such as enable shadows, enable shaders, etc.
We can achieve this nicely by introducing Settings to the material manager and leveraging them in the material/shader generators.
Internally, settings are a map <string, int> belonging to the material manager. It's up to the shader and material generators to interpret the values.
Settings can be modified at runtime, and any affected materials/ shaders will be re-generated immediately.
Settings could also be used to customize shaders for specific scenes, such as creating the shaders for a different number of lights, etc.

Examples:

{ "normal_mapping" -> 1,
"shadows" -> 1,
"shadow_pcf" -> 0,
"shadow_cascades" -> 3 }

Perhaps its' not so succinctly expressed - but I think that schemes can already be used this way? (I think RTSS leverages them somehow?)

Default shader generator details

Macros that will be available with the default shader generator:

OGRE_SHADER_LANG

Gets expanded to the shader language currently being used. May be useful for handling GLSL and GLSLES in the same file since they're almost identical.

Example:
#if OGRE_SHADER_LANG == OGRE_LANG_GLSLES

OGRE_VERTEX_SHADER, OGRE_FRAGMENT_SHADER, ...

Expanded to "1" or "0" depending on the type of shader currently being compiled. This can be used to handle vertex and fragment shaders in the same file. I think this can be useful because VS and FS need to match with their inputs and outputs and this could make it easier.
Example:

#if OGRE_VERTEX_SHADER
...
#else
...
#endif

@property(propertyName)

Expanded to "0" if the property does not exist or is 0, otherwise expanded to "1".
Example:
#if @property(normal_map)

@setting(settingName)

Expanded to the (integer) value of settingName.
Example:
#if @setting(lighting_mode) == 2

#foreach (symbol, count)
... code ...
#endforeach

Code block gets repeated count times. Useful for multiple lights in single pass, multi-layer texture splatting, etc.
Current iteration can be retrieved via @(symbol).
#line directives will be inserted to keep the line numbers valid when debugging compile errors.

@param_named(...)
@param_named_auto(...)

Identical to param_named / param_named_auto in a regular shader definition. The reason we need this as macro is to not bind the parameter if we're in a dead branch.


Precedence:
foreach -> property -> setting -> [eliminate dead branches] -> param_named[_auto]

Any attempt to do this using the preprocessor will only end up, as I see it, a serious mess.

"Monster shaders" are truly awful things to develop. They become horrible twisty mazes.

Lets give Ogre the smarts. Simple compilers are easy! You could knock together a non-optimizing OgreSL to GLSL compiler in about a month. Optimizing compilers are hard, but as long as we keep the code in reasonable form then the platform shader compilers shouldn't have too much trouble with it.

If certain specific shaders prove problematic, the developer can always take the generated code and hand-tune it anyway.

[scrawl]

Why a distinct class of materials?

Why not just specify the properties on an "ordinary" material, and let Ogre do the generation?

Actually, that is what I have in mind. I should have clarified more - generated material is just a new term, on the C++ side it's an Ogre::Material instance just like regular materials.

Ogre already has a hook for this. In fact, RTSS already plugs into it!

Said hook is called whenever no viable technique exists for a material. If you didn't specify any techniques, then this hook would automatically apply.

I know about the MaterialManager::Listener::handleSchemeNotFound. But there is currently a problem with it - it only gets called once the material is rendered, to the point that it won't get called if the entity with that material is culled by the frustum. This causes huge lags when looking around after initially loading a scene.
I would much rather have the techniques be generated when the material is loaded.

This would be a functional regression over the existing RTSS.

I think any system which attempts to rejig our shader generation should at least aim for true comprehension of the shaders involved - yes, that involves parsing them.

I see two levels to this system: The low level one; a parse tree, walked by a compiler in order to build a shader for the target language (aiming for GLSL(ES)+HLSL initially; later we can perhaps move onto other outputs (someone crazy want to compile a shader to e.g. DXSI?)

I have to disagree there. Yes, it would be ideal to have a unified language for all targets. But there's nothing that we can reasonably do about that.

If we invent our own compiler, users won't be familiar it -> there won't be many examples or tutorials for it -> it won't see a lot of adoption. We have already seen the end of that road with the current state of the RTSS. Let's face it - we are one 3d engine out of many and if users see a very complex system that no one in the industry is really familiar with they will start looking to other engines.
Imo its a much better idea to start with the systems that people are familiar with - GLSL and HLSL - and add some preprocessing tricks to make it more flexible when using many permutations.

I would like to hear more opinions, especially from the Ogre team!

[Owen]

Inventing our own compiler doesn't involve creating a whole new language.

Theres nothing stopping the compiler develoepr from choosing to essentially clone GLSL or HLSL or Cg.

A new properly compiled shader language is going to have a lot more in common with the existing ones than any macro system abstracting them away can ever hope to.

[scrawl]

Inventing our own compiler doesn't involve creating a whole new language.

Theres nothing stopping the compiler develoepr from choosing to essentially clone GLSL or HLSL or Cg.

Still not keen on that idea. I would bet this will cause very difficult to track down bugs. Besides we are always going to lag behind in features if e.g. a new version of GLSL comes out. Essentially we will have the same problems that CG always struggled with. And CG was backed by NVIDIA - we are just an open source project and do not have the resources to maintain something like that.
Valve has used a similiar approach for the linux port of the source engine where they (basically) cross-compile HLSL shaders to GLSL and they said they would have chosen a different approach if they were to do it again - due to difficult bugs they were facing (it's somewhere in this talk)

A new properly compiled shader language is going to have a lot more in common with the existing ones than any macro system abstracting them away can ever hope to.

And that is not what I am proposing. The macro system does not abstract away the shader language that is used. Maybe I need to word this more clearly.

[Owen]

Inventing our own compiler doesn't involve creating a whole new language.

Theres nothing stopping the compiler develoepr from choosing to essentially clone GLSL or HLSL or Cg.

Still not keen on that idea. I would bet this will cause very difficult to track down bugs. Besides we are always going to lag behind in features if e.g. a new version of GLSL comes out.

We are always going to lag behind on features anyway (The engine itself needs updating to support them)

Essentially we will have the same problems that CG always struggled with. And CG was backed by NVIDIA - we are just an open source project and do not have the resources to maintain something like that.
Valve has used a similiar approach for the linux port of the source engine where they (basically) cross-compile HLSL shaders to GLSL and they said they would have chosen a different approach if they were to do it again - due to difficult bugs they were facing (it's somewhere in this talk)

Cg tries to do a lot more. The Cg compiler's HLSL/GLSL support is essentially just scratching the surface - it also supports compilation to dozens of NVIDIA assembly language variants, etc.

I just watched the shader portion of that talk. They compiled DXSI (DirectX Shader Intermediate - bytecode) to GLSL. They said that if they were doing it again... they'd use MojoShader or HLSLCrossCompiler. That doesn't really invalidate the idea (Note that their method and both MojoShader and HLSLCrossCompiler do the same)

And that is not what I am proposing. The macro system does not abstract away the shader language that is used. Maybe I need to word this more clearly.

So you end up having to maintain multiple variants of a shader anyway..

[scrawl]

We are always going to lag behind on features anyway (The engine itself needs updating to support them)

As far as I know, in GLSL, you use a #version directive in shaders and do not need any changes in your program.
And I've successfully used #version 130 (GL3) shaders with the GL2 RS, which has no support for GL3.

So you end up having to maintain multiple variants of a shader anyway..

Sadly, yes. The improvements suggested in this thread are mainly related to permutation management. Language abstraction is an entirely different issue, and a very complicated one at that. But once we are going to tackle that, it should be dealt with a the GpuProgram level, so it has nothing to do with the design I am proposing here. I think that we should open a new thread

just watched the shader portion of that talk. They compiled DXSI (DirectX Shader Intermediate - bytecode) to GLSL.

woops - been a while since I watched that. Sorry for mixing things up.

[dark_sylinc]

I'm starting to believe we're seeing this backwards.

We're trying to generate a very flexible system that will let the user do anything he wants and write his own shaders.

So flexible, that there has to be a lot of features to support how the shader "subsystems" have to interact with each other; some of these features won't even be used (or won't be understood) and we'll have a giant flexible God system with a steep learning curve. It will let you do everything you want, but because of that very reason, it will also be hard to use.

And the steep learning curve is the problem, which is essentially what happened with RTSS.

That's why I think we're seeing it backwards: I'm now advocating for not-so-flexible materials. A higher level system that supports generating the materials based on a few parameters.

On C++ it would look like this:

Code: Select all

Entity *entity; //Valid ptr
String diffuseMap = "path/to/diffuse.dds"; //Or leave blank
String normalMap = "path/to/normal.dds"; //Or leave blank
String specularMap = "";
String envMap = "";
PBS_BDRF settings; //contains fresnel, diffuse colour, etc;
MaterialLibrary::generate( entity, diffuseMap, normalMap, specularMap, envMap, settings );

By having access to the entity, we can find out if it is skeletally animated, instanced, if shadows are enabled etc. Settings and texture maps can be loaded from a script. The system would generate a low level material (if not done already for another mesh with same params) and set it to the entity.

"PBS_BDRF" is the Physically Based Shading system that will be used and implements a shader generator. And would derive from a base class.
"CelShading_BDRF" could derive from the same base class but implement a cel shading approach. And so forth. In other words, you can choose a limited variation of material types. Just like CryEngine.
MUCH more manageable because we took away the flexibility.

PBS_BDRF could generate the shaders either by joining a bunch of HLSL/GLSL files programatically, or by generating the code from C++ (like the Terrain component, ugh), that is implementation defined.
For the latter, a pre-processor is still nice (particularly for the TEXCOORD numbers and the texture register numbers)

A few bunch of material types should be enough for the vast majority of users: They get nice looking materials, which are as flexible as Maya materials are, and they don't have to deal with HW skinning, shadows or instancing.

For the minority of users who want to use their own stuff, i.e. because they have stylized aesthetics like Prince of Persia 2008, they can either:

• Deal with low materials directly (i.e. the current Ogre 1.x way)
• Look how the Material generator works and modify an existing BDRF implementation to fit their needs. They can work at the C++ level and/or shader level
• Implement their own BDRF generator
• Use alternative systems like RTSS

Because this component is less intrusive than RTSS (only need to call MaterialLibrary::generate on the Entity) if you want to use a different system (i.e. directly low level materials) just don't call MaterialLibrary::generate and do it the old way.
The system would be entirely optional, but if it's good & easy to use; Samples will be build around it and community will follow.

Btw, because BRDFs generators would make shaders programatically, there's no limit to say a BRDF generator can delegate the task to an alternate language (Lua is in popular demand)

My 2 cents.

[drwbns]

Sounds completely different, but like you said, much more manageable and honestly looks a little more Ogre-like. I think I like it better tbh

[TheSHEEEP]

Dark_sylinc has a point.
Ease to use > greatest possible flexibility.

Of course any system needs to be flexible, but what he is proposing sounded intuitive to me instantly. And at least at the moment I can not think of anything that it would not be able to do.
Except the name "PBS_BDRF", I really don't like class names like that, where you have to search on Google to find out what the name of a class means

[dark_sylinc]

Except the name "PBS_BDRF", I really don't like class names like that, where you have to search on Google to find out what the name of a class means

Hehe, yeah, that's a really bad name.
Though "PhysicallyBasedShadingBidirectionalReflectanceDistributionFunction" looks like a long name for a class

[spacegaier]

Perhaps PhysicalShading_BRDF, then the context of BRDF it clear (physical shading) and the appendix lists which kind of shading. The long form should then of course be the first thing in the class comment .

[scrawl]

I'm starting to believe we're seeing this backwards.

We're trying to generate a very flexible system that will let the user do anything he wants and write his own shaders.

So flexible, that there has to be a lot of features to support how the shader "subsystems" have to interact with each other; some of these features won't even be used (or won't be understood) and we'll have a giant flexible God system with a steep learning curve. It will let you do everything you want, but because of that very reason, it will also be hard to use.

I think you got the wrong idea from my post.
It's not complicated or huge at all. Almost all of the ideas I posted are not actually in Ogre, they were just some ideas what cool things you can do with user-defined material generators. I can see why it was confusing, so I removed the "Settings" "LOD techniques" and "Material schemes" sections since they have nothing to do with the changes that will be made to Ogre. Let me know if it's clearer (and easier to use) now.
That said - now the only difference to your post is whether we allow actually hooking up material generators to the material manager, or if we require the user to generate low-level materials from within his own code.
I can see several advantages with the former approach.
First, I think its very nice to generate materials on demand, i.e. when they are actually being loaded (which happens when they get added to the scene). I have a project where I need lots of shader variations and loading them all at startup actually takes a lot of time (10-20 seconds). By hooking the generator to Material::loadImpl we can nicely load the shaders on-demand. (Again, the material listener is not really suitable for that at the moment, due to the lag problem I mentioned earlier in this thread)
We also need to think about what kind of message we are sending to users. By saying "you can generate materials/shaders, but you're on your own then" you're not really advocating it. By having a material generator interface (not implementation!) inside Ogre core, its going to be official, it's going to be supported, and it will be documented properly.
Lastly, it's much nicer to let Ogre's existing MaterialSerializer deal with loading properties for generated materials, rather than imposing that on the user as well.

PBS_BDRF could generate the shaders either by joining a bunch of HLSL/GLSL files programatically, or by generating the code from C++ (like the Terrain component, ugh), that is implementation defined.

If, in Ogre 2.0, anyone still decides to use that approach, then I think we will have done something wrong.
It's the worst way to do shaders. Period. You don't get syntax highlighting. It's extremely frustrating to debug - change, recompile, change, recompile - error on line X? Have fun figuring out which line that is. And don't get me started on trying to figure out what the produced shader is actually doing.
It's not just Terrain component that does this - take a look at Hydrax's shaders for example... we really need a decent shader preprocessor so that we can get rid of these abominations once and for all.
That's also why I think its very important to port the Terrain component shaders to the shader generator. And with shiny, I've already shown that it's possible to do just that in a regular shader file with only very few macros.

Regarding entities - here's another idea: Derive Material from Any, so you can save the entity in the material as user data. Then when it's being generated, you can check what entity it belongs to. I like this idea better, simply because generation shouldn't be specific to Entity. Should I not be able to generate materials just because I'm using custom renderables?
Or if the extra memory footprint from Any is too high, you can just create a global map<Material, Entity> to look up the entity in your own code. Not a big deal.

[nickG]

so maybe procedural material a like UE style?