Specular IBL

[TaaTT4]

There's, in my opinion, a problem related to how OGRE handles specular IBL component in lighting. I believe the issue is present since the introduction of ibl_specular pass. With artists' help, I've tried to counteract/bypass it, but in characters' clothes is too much noticeable to simply ignore it.

Since images worth thousands words (Marmoset 3.x on the left, OGRE on the right):

In the scene above there is neither ambient light not direct light. The lighting contribution is only provided by the reflection map.

Trousers' material uses metallic workflow, default_uncorrelated BRDF and has perceptual roughness enabled. Roughness value is 0.8 and metalness value is 0.0. Diffuse and normal maps are the ones here below:

The "specular highlights" (pardon me, I don't know how to properly indicate them...) in the trousers are cleary introduced by the specular IBL component. Indeed, removing the Rs term from the final sum in @piece( BRDF_EnvMap ) makes them vanish:

Code: Select all

...

midf3 Rd = pixelData.envColourD * pixelData.diffuse.xyz * 3.141592654 * fresnelD;
midf3 Rs = pixelData.envColourS * pixelData.specular.xyz * ( fresnelS * envBRDF.x + envBRDF.y );

...

finalColour += Rd + Rs;

Assuming code of @piece( DeclareBRDF ) and @piece( BRDF_EnvMap ) is correct (I really hope so!), I'm wondering whether the issue isn't related to the brdfLUT map. According to this old post, it seems the brdfLUT map has been generated with an external tool.

• Is the brdfLUT map generation code compatible with OGRE (and thus data embedded in that map coherent with what OGRE is expecting)? E.g.: the generation algorithm takes roughness values in input; does it make some difference whether perceptual roughness is enabled?

• Is 64x64 an enough size for the map? If I have correctly understood the algorithm, it means that both NdotV and roughness can assume "only" 64 possible values (discretization step 0.015625; probably enough for roughness, don't know for NdotV)

• Are 16 bits an enough precision for the map? I believe so, but I've also seen tools exporting with 32 bit precision.

[dark_sylinc]

Hi!

RenderDoc now supports line stepping debugging. You'll quickly find the divergence that way.

Just compare both side by side in the debugger

As for the 64x64 is that enough? It's interpolated. Obviously more values = more accurate. It's just like a circle being approximated with an hexagon vs decahedron. More tesellated means the more it resembles the circle.

[TaaTT4]

Just compare both side by side in the debugger

I'm sorry, but I haven't understood: compare what with what?

[dark_sylinc]

Marmoset vs Ogre, both in renderdoc.

[TaaTT4]

Marmoset seems to use a different rendering approach compared to what OGRE does (no split sum approximation and thus no BRDF LUT map). So, I've abandoned the trying to mimic its visual look and I've started instead to re-study PBR from scratch to see whether OGRE lacks something.

Before going further, let me recap some context infos:

there is neither ambient light not direct light. The lighting contribution is only provided by the reflection map.

In the images below, from left to right:

• IBL diffuse + specular contribution:

  Code: Select all

  finalColour += Rd + Rs;

• IBL diffuse contribution only:

  Code: Select all

  midf3 Rd = pixelData.envColourD * pixelData.diffuse.xyz * 3.141592654 * fresnelD;

• IBL specular contribution only:

  Code: Select all

  midf3 Rs = pixelData.envColourS * pixelData.specular.xyz * ( fresnelS * envBRDF.x + envBRDF.y );

Character materials:

metallic workflow, default_uncorrelated BRDF and has perceptual roughness enabled. Roughness value is 0.8 and metalness value is 0.0.

Test spheres have roughness value 0.0 in the topmost row (which increases until 1.0 going down) and metalness value 0.0 in the leftmost column (which increases until 1.0 going right).

And now, let me show the discrepancies that I've discovered.

In SpecularIblIntegrator_piece_cs.any (the compute shader that pre-filters and convolutes the reflection map) there's a function (taken from IBLBaker, I suppose) that resemble the GGX (Trowbridge-Reitz) NDF:

Code: Select all

// Smoth GGX
INLINE float specularD( float roughness, float NoH )
{
	float NoH2 = NoH * NoH;
	float r2 = roughness * roughness;
	return r2 / pow( NoH2 * ( r2 - 1.0 ) + 1.0, 2.0 );
}

Compared to what Epic Games does in UE4, OGRE doesn't square the roughness value and there's a missing PI in the denominator. If using squared roughness could be a stylistic choice (linear vs perceptual roughness), the lack of PI sounds like an error to me. Or is there an (unknown to me) reason behind it?

Anyway, this is how I've changed the function:

Code: Select all

// Smoth GGX
INLINE float specularD( float roughness, float NoH )
{
	float NoH2 = NoH * NoH;
	float a = roughness * roughness;
	float a2 = a * a;
	return a2 / PI * pow( NoH2 * ( a2 - 1.0 ) + 1.0, 2.0 );
}

I'm wondering whether the issue isn't related to the brdfLUT map.

My supposition was correct: OGRE BRDF LUT map is crappy. I don't know where it has been taken (my guess: saved from a RenderDoc capture of IBLBaker, downscaled and then converted) or how it has been generated, but the values stored in it are totally wrong. Changing IBLBaker code to directly generate a 64x64 FLOAT16 texture (to avoid downscaling which I believe is what corrupts data) returns the expected texture. To verify this I've changed LearnOpenGL code in the same way and the texture returned is pretty much the same as before. If you're afraid in playing with RenderDoc, shaders and changing 3rd party code, here you can find a C++ BRDF LUT generator (Windows only unfortunately; it relies on DirectXMath and DirectXTex) I've written translating some LearnOpenGL shader code.

Is 64x64 an enough size for the map?

Are 16 bits an enough precision for the map?

Yes and yes. Increasing BRDF LUT map size and/or floating point precision doesn't produce any appreciable visual difference.

Last but not least: fresnel! I believe the following lines of code in 200.BRDFs_piece_ps.any hasn't been adapted when perceptual roughness has been introduced:

Code: Select all

@piece( getSpecularFresnelWithRoughness )pixelData.F0 + pow( _h( 1.0 ) - pixelData.NdotV, _h( 5.0 ) ) * (max( make_float_fresnel( _h( 1.0 ) - pixelData.roughness ), pixelData.F0 ) - pixelData.F0)@end
@piece( getDiffuseFresnelWithRoughness )max( make_float_fresnel( _h( 1.0 ) - pixelData.roughness ), pixelData.F0 ) - pixelData.F0 + pow( _h( 1.0 ) - NdotL, _h( 5.0 ) ) * pixelData.F0@end

In fact they still use pixelData.roughness which, when perceptual roughness is enabled, is equal to material squared roughness:

Code: Select all

@property( perceptual_roughness )
	pixelData.roughness = max( pixelData.perceptualRoughness * pixelData.perceptualRoughness, _h( 0.001f ) );
@else
	pixelData.roughness = max( pixelData.perceptualRoughness, _h( 0.001f ) );
@end

Summarizing:

• Modification of GGX NDF function in reflection map pre-filtering and convolution;

• OGRE BRDF LUT map replacement;

• Substitution of pixelData.roughness with pixelData.perceptualRoughness in get*FresnelWithRoughness pieces;

Gives me the following visual results:

As you can see, the specular highlights have been greatly reduced (if not vanished...):


This visual look is a way more compliant to what me and the artists are expecting due to the high roughness value of the materials.

[Semi-OT remark]

If you paid (a lot of!) attention you should have noticed that there is an extra PI multiplication in IBL diffuse contribution:

Code: Select all

midf3 Rd = pixelData.envColourD * pixelData.diffuse.xyz * 3.141592654 * fresnelD;

In my opinion this is necessary, other than a math point of view (which I explained why in this issue), also because otherwise the diffuse contribution is clearly too low:

[zxz]

Great detective work, TaaTT4!

We have also had issues where rather rough materials look way too shiny and have too strong fresnel reflections. It would be great to get fixes like these ones into Ogre, to better match the expected output of artist tools.

[dark_sylinc]

Yes indeed thanks for all detective work!!!

My supposition was correct: OGRE BRDF LUT map is crappy. I don't know where it has been taken (my guess: saved from a RenderDoc capture of IBLBaker, downscaled and then converted) or how it has been generated,

I swear honest to God I don't remember.

But based on your link to my own post it's quite possible I used that compute shader linked there to generate the texture, which should've generated the same texture (emphasis on SHOULD).

This link of yours isn't public. Is there a reason or an oversight? (this link is also non public)

Anyway, I'm old, grumpy and tired. You took a lot of effort to fix this so if you say it works, it works .

Can I get the generated BRDF file so we can upload it to the repo? (and fix the formula).

I hope the 16-bit half format isn't the cause of the problem. I do remember converting it to 16-bit half to let the texture be packed together with the other two LTC textures.

[TaaTT4]

This link of yours isn't public. Is there a reason or an oversight? (this link is also non public)

My fault, the sharing wasn't public. Should be fixed now.

Anyway, I'm old, grumpy and tired. You took a lot of effort to fix this so if you say it works, it works .

Can I get the generated BRDF file so we can upload it to the repo? (and fix the formula).

I'm taking a look both at direct and indirect lighting. Give me some more days and I'll come back with a report (and hopefully some improvements).

I hope the 16-bit half format isn't the cause of the problem.

Correct, 16-bit precision seems enough.

[KammutierSpule]

Code: Select all

// Smoth GGX
INLINE float specularD( float roughness, float NoH )
{
	float NoH2 = NoH * NoH;
	float r2 = roughness * roughness;
	return r2 / pow( NoH2 * ( r2 - 1.0 ) + 1.0, 2.0 );
}

Compared to what Epic Games does in UE4, OGRE doesn't square the roughness value and there's a missing PI in the denominator. If using squared roughness could be a stylistic choice (linear vs perceptual roughness), the lack of PI sounds like an error to me. Or is there an (unknown to me) reason behind it?

There is a bug I rise in 2017 related to this:
https://github.com/OGRECave/ogre-next/issues/5
and
viewtopic.php?f=25&t=93654&sid=e95213f3 ... 74af6b6e0c

Regarding the Pi, I don't remember but you may want to check on other place "later in the code or early before" that it may be applied.

[TaaTT4]

I'm taking a look both at direct and indirect lighting. Give me some more days and I'll come back with a report (and hopefully some improvements).

So the "days" turned into months... but hey, better late than never!

I'll use this post to present all the fixes and changes I've made to OGRE to make it closer/similar to the other rendering engines. This was all necessary because our artists weren't entirely happy with the way OGRE rendered (especially in borderline situations); partly due to bugs and partly due to some (peculiar) approaches taken in the OGRE rendering pipeline. To do this work I studied (again...) the PBR theory from scratch, especially the papers by Sébastien Lagarde and Brian Karis as well as the Filament documentation; I also used Learn OpenGL, Marmoset Toolbag (version 3) and UE4 for code references and comparisons.

For clarity and readability, I will divide this post into three main sections:

• Direct lighting;

• IBL;

• Clear coat;

In the links above you can find all the source code (three branches I created in my fork of OGRE; I'm not sure, but they should be easily cherry pickable to a vanilla OGRE without conflicts). I won't detail all the code (the commit messages should be self explanatory, I hope), but I will focus instead on explaining the more subtle aspects. In any case, feel free to ask any questions or request further explanations.

Direct lighting

The first discrepancy concerns the reflectance value for dielectric surfaces. A hardcoded value of 4% is "traditionally" used in the literature (see: https://seblagarde.files.wordpress.com/ ... br_v32.pdf, pag. 13); I don't know why, but OGRE uses 3% instead. Since there are no appreciable visual differences, I've only corrected it for consistency.
Another inconsistency concerns the parametrization of the term k in the computation of the geometric shadowing (specular G). OGRE uses k = roughness⁴, Marmoset uses k = roughness² / 2, but the "standard" (or at least, the most used value) seems to be k = (roughness + 1)² / 8 (see: https://blog.selfshadow.com/publication ... tes_v2.pdf, pag. 3).

From left to right (direct lighting only): OGRE default BRDF (height uncorrelated); w/ 4% reflectance for dielectric surfaces; w/ k = (roughness + 1)² / 8.

And now the tricky part (probably what causes the most visual difference between OGRE and other rendering engines). In both the direct lighting pass and the IBL pass, OGRE "weights" the diffuse lighting contribution by a factor of 1 - fresnelS. None of the rendering engines I've reviewed do this; all use the diffuse lighting contribution as-is, without any weighting factors. Based on a response Sébastien Lagarde gave to a user who commented on a post on his blog, it appears that 1 - fresnelS is a better approximation, mathematically speaking, than doing nothing.

The "downside" of this increased mathematical accuracy is that it leads to a visual appearance that is not what artists expect or are used to.

From left to right (direct lighting only): OGRE default BRDF; w/o any weighting factor on the diffuse lighting contribution.

IBL (diffuse)

Unlike Filament and UE4 (which use the Lambertian model for diffuse lighting), OGRE uses the Disney model. Sébastien Lagarde paper states that, as is done for the specular part, the DFG term must also be considered in diffuse preintegration (see: pag 58-73; especially pag. 67 and listing 18, 24). Similarly to the specular part, also in this case the DFG term depends only on the viewing angle and the roughness and therefore can be precalculated. I then modified Filament cmgen (the program used by OGRE to generate its BRDF LUT map) to also insert the diffuse DFG term into the map (blue channel) containing the precalculated DFG terms. The application of the diffuse DFG term then becomes straightforward.

Here you can find the maps (please ignore the .txt extension, it's just to overcome the limitations of the OGRE forum with attachments) with which to overwrite the brtfLutDfg.dds file:

• DFG_Smith.dds.txt
• DFG_Schlick.dds.txt

Ideally, you should use the same GGX formulation in geometric shadowing (specular G) calculation, for both direct lighting and IBL. The DFG_Smith.dds file was generated using the Smith GGX formulation which, in OGRE terms, corresponds to the default BRDF; on the contrary, the DFG_Schlick.dds file was generated using the Schlick approximation of the Smith GGX formulation which, in OGRE terms, is the default BRDF with no height correlation. I haven't checked if there's any difference (and how big) in being consistent vs mixing GGX formulations; I've attached both files just for completeness.

As I did in direct lighting pass, also in IBL I removed the weighting of the diffuse lighting contribution.

From left to right (IBL diffuse only): OGRE IBL; w/ diffuse DFG term applied; w/o any weighting factor on the diffuse lighting contribution.

IBL (specular)

There is a bug in the computation of the normal distribution function (specular D) during prefilter convolution. If you compare the theory with OGRE code, you can clearly see that a PI is missing in the denominator. Also, OGRE still use linear roughness while in the theory perceptual roughness (alpha = roughness²) is used. I replaced linear roughness with perceptual roughness as the latter seems to be the de facto standard, but a better (and more flexible) approach would have been to use something like a @property( perceptual_roughness ) as OGRE supports both workflows. Finally, the probability distribution function can be simplified as pdf = specular_d / 4 (see: https://seblagarde.files.wordpress.com/ ... br_v32.pdf, listing 19).
Another problem with roughness is that in get(Diffuse|Specular)FresnelWithRoughness pieces OGRE always uses linear roughness (ignoring if perceptual roughness is enabled).
The last quirk is that Schlick fresnel (fresnelS) is used instead of f0 (amount of light reflected back at normal incidence). I have no idea of the reasons behind this choice; my guess is that it's taken from Learn OpenGL since it's the only rendering engine that does the same thing.

Besides all this, I made two more cosmetic changes both taken from Filament. They don't fix any bugs, but our artists prefer the way Filament works over the behaviour of OGRE. The first modification is to use a different perceptual roughness to LOD mapping. Filament describes it as:

The mapping below is a quadratic fit for log2(perceptualRoughness) + iblRoughnessOneLevel when iblRoughnessOneLevel is 4. We found empirically that this mapping works very well for a 256 cubemap with 5 levels used. But also scales well for other iblRoughnessOneLevel values.

The second change is to allow some overlap between samples when convoluting the environment map.


From left to right, top to bottom (IBL specular only): OGRE IBL; w/ NDF fixed; w/ linear/perceptual roughness fixed; w/ Filament perceptual roughness to LOD mapping; w/ samples overlapping; w/ Schlick fresnel replaced by f0.

The final result of all this work.

From left to right: OGRE (IBL only); w/ all the above fixes/changes; OGRE (direct lighting and IBL); w/ all the above fixes/changes.

Clear coat

Since I'm the one who added the clear coat feature... shame on me! Seriously, I don't know if I made a mistake then or Filament (where have I looked to implement this feature) has made some improvements over time. The bug was that specular lighting contribution of IBL in clear coat layer was not treated as a "common" IBL (but with different NdotV, perceptual roughness and f0 values).
For consistency, I've added the same perceptual roughness to LOD mapping in the clear coat layer as well.

Remarks

I keep insisting: I'm pretty sure there is a missing multiplication by PI in diffuse lighting contribution of IBL. All screenshots were taken with that fix in place.
I worked on this stuff months ago so my memories might be a little faded, but at that time I tried to be super meticulous and to double check everything at least twice. Please let me know if you find errors or something is not clear.

[Crystal Hammer]

Kudos. I'm glad someone understands this theory and equations.
As a game developer and creator I just can't even .
Man I still remember the times when specular had just one exponent power and that was it.
How much more complicated can this even get in 10 years
I wonder if that bug changes anything for my setup, or is it just my lack of using proper values for material parameters.

[TaaTT4]

Kudos. I'm glad someone understands this theory and equations.
As a game developer and creator I just can't even .

Well the math is not that hard, but for sure you have to study it carefully and pay attention because it is easy to miss something when you translate it into code. In my opinion, Sébastien Lagarde's paper and Filament documentation are the best online resources if you want to go deeper.

I wonder if that bug changes anything for my setup, or is it just my lack of using proper values for material parameters.

Other than the missing multiplication by PI bug, to be honest I didn't notice any problems at first (but I lack an artistic eye). It was one of our artist (used to commercial engines) who started to notice some visual artifacts (especially with some normal maps). Trying to solve those problems, he then pushed me to investigate and what I discovered is reported in my previous post.
Real-time graphics programming is an "approximation game" and therefore there is no one-size-fits-all solution. Unfortunately, the big players tend to make the "rules" and, to keep the artists from complaining, I had to chase the visual appearance of the commercial engines as much as possible.

[dark_sylinc]

This is wonderful!

Thanks!

I've already created a ticket. I'll integrate your changes in January.

You've tidied everything up for easy inclussion! Thanks a lot! It seems like it will be easy to integrate back.

It's a shame to see you go

BTW, I will copy your post (possibly with some rephrasing or not) explaining these changes in the manual. Is that ok?

I fear forum posts go down, get lost, uploaded picture services go down, etc. This is too useful information to leave it out of the manual.

The last quirk is that Schlick fresnel (fresnelS) is used instead of f0 (amount of light reflected back at normal incidence). I have no idea of the reasons behind this choice; my guess is that it's taken from Learn OpenGL since it's the only rendering engine that does the same thing.

I don't think there's a hard reason. IBL is so much of an approximation it hurts (specially when you're not at extreme roughness values like 0.01 or 1; but rather in-between). I think I felt fresnelS was more accurate. But if everyone else "does it the other way", then we'll have to do it that way.

[TaaTT4]

BTW, I will copy your post (possibly with some rephrasing or not) explaining these changes in the manual. Is that ok?

I fear forum posts go down, get lost, uploaded picture services go down, etc. This is too useful information to leave it out of the manual.

Be my guest .

Since you'll be getting your hands on PBR, a nice and easy addition you can add to my work is multiscattering. It "only" requires you to account for an energy compensation factor and use a different brtfLutDfg.dds (with a different reconstruction formula).

For your convenience, I'm attaching here the maps containing the precalculated DFG terms for multiscattering:

• DFG_Smith_multiscatter.dds.txt
• DFG_Schlick_multiscatter.dds.txt

In the end I didn't add multiscattering because our artists weren't 100% convinced about it (I assume it was a matter of habit), but I still have some screenshots from when I was experimenting.





From left to right: OGRE; w/ all my fixes/changes; w/ multiscattering. From top to bottom: direct lighting only; IBL diffuse only; IBL specular only; IBL only; direct lighting and IBL.

I also found some screenshot with a little more context (and with real-world materials):


From left to right: OGRE; w/ all my fixes/changes.

[dark_sylinc]

I've added your changes to OgreNext 3.0

Thank you for your contributions!!! You will be missed!

I've documented your changes.

Of all your changes, the diffuse fresnel change was the hardest to swallow. Though I kept diffuse fresnel (you had removed it completely) but it is now BRDF PbsBrdf::DefaultHasDiffuseFresnel.

However now the default settings do not have diffuse fresnel. Personally, I think it looks worse, is less accurate, and gives less flexibility to the artist; and can represent fewer materials.

However in OgreNext we strive that, out of the box, when an artist exports its PBR content out of the tool of their choice (e.g. Marmoset, UE5, Blender, etc) we should render as close as possible as how it looked in the tool. And in order to do that, we must follow standard practice and conventions.

This is why the defaults do no longer use diffuse fresnel, but can be enable if the user so desires (i.e. it's porting its 2.4 materials to 3.0 and they should keep looking they way they used to or just likes it better)

I replaced linear roughness with perceptual roughness as the latter seems to be the de facto standard, but a better (and more flexible) approach would have been to use something like a @property( perceptual_roughness ) as OGRE supports both workflows.

You're overthinking it. We added HlmsPbs::setPerceptualRoughness to have backwards compatibility with old materials (i.e. users coming from 2.2.0 upgrading into 2.2.1+). But this level of flexibility is too expensive to maintain.

We now have a deprecation mechanism which is why I just deprecated it in 3.0; and it will not be available in 4.0

[TaaTT4]

I've added your changes to OgreNext 3.0

Thank you for your contributions!!! You will be missed!

I've documented your changes.

I saw it! And thank you for crediting me.

Of all your changes, the diffuse fresnel change was the hardest to swallow. Though I kept diffuse fresnel (you had removed it completely) but it is now BRDF PbsBrdf::DefaultHasDiffuseFresnel.

However now the default settings do not have diffuse fresnel. Personally, I think it looks worse, is less accurate, and gives less flexibility to the artist; and can represent fewer materials.

However in OgreNext we strive that, out of the box, when an artist exports its PBR content out of the tool of their choice (e.g. Marmoset, UE5, Blender, etc) we should render as close as possible as how it looked in the tool. And in order to do that, we must follow standard practice and conventions.

This is why the defaults do no longer use diffuse fresnel, but can be enable if the user so desires (i.e. it's porting its 2.4 materials to 3.0 and they should keep looking they way they used to or just likes it better)

I'm not a huge fan of using a worst-approximate solution either just because "big players" use it, but you known, unfortunately, they dictate the rules. But I can understand artists who are used to a particular visual look and expect it to be pretty much the same across different engines. In our case in particular, the artist went crazy (unsuccessfully) trying to remove/mitigate the "specular reflection highlights" in dielectric materials (metalness = 0, roughness = around 1) in some scenarios (see the character images in my first messages); I don't know if it's a limitation of the metallic workflow (but again: this seem the standard workflow).

I encourage artists and OGRE users to play and experiment with this new "render model" in order to validate it. It would be great If someone was be able to do a comparison with other engines using more realistic (than debug spheres) models and materials.

I then modified Filament cmgen (the program used by OGRE to generate its BRDF LUT map) to also insert the diffuse DFG term into the map (blue channel) containing the precalculated DFG terms.
...
Ideally, you should use the same GGX formulation in geometric shadowing (specular G) calculation, for both direct lighting and IBL.

I haven't documented how I modified Filament cmgen to also generate the diffuse DFG term. The patch below (for Filament v1.21.3; don't know if it works with recent versions) contains all the changes:

DFG.diff.txt

In addition to putting the diffuse DFG term in the blue channel of the map, the patch contains a #define Visibility that allows you to switch between Smith (default) and Schlick GGX formulations. E.g. you can use it to regenerate the brtfLutDfg.dds map in case you use the default BRDF with no height correlation and want to be consistent in geometric shadowing (specular G) calculation for both direct lighting and IBL.

[zxz]

This work seems like a great improvement. Switching to the new implementation has improved a lot of our shading issues. Thanks for the investigative work, and the effort to implement everything in Ogre!

A remaining question is how to best match the "blurriness" of environment reflections and the light spotlights across the roughness range. It seems to depend on the initial cubemap resolution and the number of mip levels. A comment in the shader code seems to suggest that it is tweaked for 256 pixel cubemaps. Are there any formulas for how this should be adjusted for other cubemap resolutions?