Normal offset bias is negatively affected by object scale

[zxz]

Hello!

After upgrading to Ogre 2.3 I noticed severe shadow acne on some meshes. Only meshes with an applied scaling factor were affected (Node::setScale). The scaling ends up being applied to the normals used for calculating the normal offset bias, through the world/worldView matrices. In my case, the normals ended up being 100 times shorter, which severely reduced the effect of the biasing. This is a problem because the bias settings would have to vary wildly between different objects to compensate for this effect. I think that the biasing should be independent of the scale. I identified the following lines as relevant:

https://github.com/OGRECave/ogre-next/b ... s.any#L193
https://github.com/OGRECave/ogre-next/b ... s.any#L266

Normalizing these two normals after the matrix multiplication makes the biasing work as expected for scaled objects as well.

Is there any clever, more efficient way of solving the same problem?

Thanks

[zxz]

Well.. I figured out that normalization won't give the right result for non-uniform scalings. This seems to be a more general problem, not only affecting the shadow biasing, but the normals used for all lighting themselves.

The correct transformation to apply to normals seems to be the transpose of the inverse of the transformation matrix.

The inverse-transpose seems to remain in some older parts of Ogre, e.g. the inverse_transpose_* parameters listed here: https://www.ogre3d.org/docs/manual18/manual_23.html . HlmsPbs doesn't seem to provide these matrices to the shaders though.

This should probably be fixed in order to correctly transform the normals. It requires providing a couple more matrices to the shader, but is required for correctness. In this case the normal is transformed by worldMat in one place, and worldViewMat in the other. So the inverse-transpose of each should be made available to the shaders in order to get the right result.

Opinions?

Reference:
https://paroj.github.io/gltut/Illuminat ... ation.html

[dark_sylinc]

More than a decade doing graphics and I never knew about this.

GRAAAAAAAAAAHHHHHH.

Calculating & sending another (3x3) matrix per object would be expensive.

My best proposal so far is to use a CMake option like we already do. When enabled, accurate normalization is applied. When disabled, normalizing the normals is used as a quick hack.

However I can't help like we're missing something. Like, undoing that operation shouldn't require 9 floats (well, I guess we could send a quaternion and multiply the normals against that, in order to get proper rotations without scale, but still it wouldn't be cheap as it would kick the alignments out of whack)

Edit: Oh we don't have quaternion data readily available because the world matrix is calculated out of compound operations based on inheritance.

Edit 2: An easy alternative would be to calculate the inverse-transpose in the vertex shader. This is expensive, but due to how the Hlms works, we can toggle it per object or per material (i.e. as an exception).

Edit 3: For skeletal animation it would be costly to do, at any point (CPU or GPU)

[zxz]

More than a decade doing graphics and I never knew about this.

GRAAAAAAAAAAHHHHHH.

I used to know this many years ago, but had managed to forget it until I rediscovered it now. There used to be a built-in uniform called gl_NormalMatrix that contained transpose(inverse(gl_ModelViewMatrix)), which you had to use instead of the model-view matrix for transforming normals.

As for some basic scaling discussion I see a few different use-cases (for regular meshes shown in the scene), each of which might have different requirements and solutions. In our case right now, the reason why we need to scale some objects is that some meshes are designed with a centimeter scale, while others have a meter scale. Right now, we apply a (uniform) scaling factor to the various scene nodes to cancel that difference. This kind of use-case could be solved by requiring more uniform input data, or some kind of load-time unit conversion. In practice, this turns out to be a bit difficult logistically, and requiring complete content uniformity also comes with a cost (just not in runtime), while the scene-node scaling is very convenient. There is also the use case where a single mesh is presented with various scalings to add some amount of variety to a set of objects. This use-case requires proper scaling support, and sometimes even non-uniform scaling. There is of course a third case where some completely dynamic scaling is used for whatever reason, where no amount of pre-processing would do any good. I don't know how common that is.

Calculating & sending another (3x3) matrix per object would be expensive.

Do you have any estimation of the cost of providing the extra matrix/matrices? I can't easily judge how expensive it would be relative to all the existing work being done. I suppose that having to compute at least one inverse for each object is not great (plus the cost of the extra uniform data). In the current normal offset bias calculation, the normal is required both in view space and world space, requiring yet another matrix. Maybe there is a way to avoid this by moving calculations to different vector spaces.

However I can't help like we're missing something. Like, undoing that operation shouldn't require 9 floats (well, I guess we could send a quaternion and multiply the normals against that, in order to get proper rotations without scale, but still it wouldn't be cheap as it would kick the alignments out of whack)

The problem with this alternative is that a simple rotation can't describe the transformation caused by non-uniform scaling. The inverse-transpose ends up applying the inverse of the original scaling operation (as well as rotations etc).

Edit: Oh we don't have quaternion data readily available because the world matrix is calculated out of compound operations based on inheritance.

Edit 2: An easy alternative would be to calculate the inverse-transpose in the vertex shader. This is expensive, but due to how the Hlms works, we can toggle it per object or per material (i.e. as an exception).

Edit 3: For skeletal animation it would be costly to do, at any point (CPU or GPU)

What is your opinion on the simplified case of always handling uniform scaling correctly? I feel like this question might benefit from being answered separately, while the non-uniform case might need more discussion, and possibly a more complicated solution. The solution here is as simple as adding a couple of normalization calls.

When it comes to non-uniform scaling, I am not sure which variant strikes the best balance between correctness, flexibility and performance.

A compilation option is the least flexible variant, and feels quite heavy handed. Even in most situations where non-uniform scaling is present, I would guess that a majority of the objects would not be scaled non-uniformly, in which case such a coarse option would pessimize all other objects. Secondly, it would prevent the same Ogre build from being used in projects where some have a need for correct non-uniform scaling behavior, and others prioritize performance. It would double the number of required build variants. I can't say that I'm very enthusiastic about this solution.

A more fine-grained approach would be preferable when it comes to ease of use and flexibility.

There seems to be two basic ways of solving the non-uniform case. Doing the calculation on the CPU (per object) or the GPU (per vertex). Both of these variants would benefit from an optimization where the cost is only borne by objects that require it. Both the CPU and GPU variants would have to choose different shaders depending on whether or not non-uniform scaling is present in the transformation. For the CPU case, the difference would be if extra uniforms are provided to the shader (containing the inverse-transpose matrix), or not, while the GPU case would switch between shaders where the transpose-inverse is performed per-vertex or not. However, there is also a cost to determining whether or not a non-uniform scale is present in the final transformation matrix (which is affected by the whole chain of parent nodes). I guess it could be handled a user-provided flag to avoid that cost, but that also feels rather ugly. How feasible do you think it is to make a dynamic choice per object, deciding if non-uniform scaling is required or not?

I can't easily judge the relative costs between these various solutions, and none of them feels all that great.

TLDR (only questions):
Do you have any estimation of the cost of providing the extra matrix/matrices?
What is your opinion on the simplified case of always handling uniform scaling correctly? (while solving the non-uniform case separately)
How feasible do you think it is to make a dynamic choice per object, deciding if non-uniform scaling is required or not?

[dark_sylinc]

Uniform scaling case:
It would seem a simple normalize() is the solution. Easy to do. Cheap. Necessary.

Non-uniform scaling case:
There are various solutions. We have two audiences: game devs who wants the most performance. Non game devs who usually need accuracy and performance is a nice to have.

While a CMake option would probably try to satisfy both, there are a few aspects (such as skeletal animation) that wouldn't be addressed. And like you said, it's an added complexity (i.e. make sure your build option is enabled).

And even users who are not gamedevs may not be using non-uniform scaling at all, thus it's an unwanted cost.

Therefore I believe the best solution is to calculate this on the vertex shader with an Hlms option. e.g.

Code: Select all

@property( accurate_non_uniform_scaled_normals )
   normalMat = transpose( inverse( worldMat ) );
@end

This is expensive on the shader side but:

• API is easy and simple

• Those who need accuracy can get it

• Those who don't care can leave this option disabled (default)

• Low maintenance

Considering all options, I think that's the most viable solution in terms of simplicity and keeping everyone happy.

[zxz]

Uniform scaling case:
It would seem a simple normalize() is the solution. Easy to do. Cheap. Necessary.

Agreed. I think it's probably reasonable to add this unconditionally.

Non-uniform scaling case:
There are various solutions. We have two audiences: game devs who wants the most performance. Non game devs who usually need accuracy and performance is a nice to have.

While a CMake option would probably try to satisfy both, there are a few aspects (such as skeletal animation) that wouldn't be addressed. And like you said, it's an added complexity (i.e. make sure your build option is enabled).

And even users who are not gamedevs may not be using non-uniform scaling at all, thus it's an unwanted cost.

I agree that the best option would be if it's there's no added cost when the functionality isn't used.

Therefore I believe the best solution is to calculate this on the vertex shader with an Hlms option. e.g.

Code: Select all

@property( accurate_non_uniform_scaled_normals )
   normalMat = transpose( inverse( worldMat ) );
@end

This is expensive on the shader side but:

• API is easy and simple

• Those who need accuracy can get it

• Those who don't care can leave this option disabled (default)

• Low maintenance

Considering all options, I think that's the most viable solution in terms of simplicity and keeping everyone happy.

It is my (limited) understanding that Hlms properties can be set at various granularities. Correct me if I am wrong.

At which granularity is accurate_non_uniform_scaled_normals configurable in your suggestion above? Globally? Per material? Per object?

A global option isn't great in situations where the majority of the meshes are not scaled (or only uniformly), and only a minority require correctness. It is the simplest solution though.

A a material option is more flexible, and should be simple to implement, but requires a flag in every HlmsPbs datablock. However, logically it is fundamentally the "wrong" place for such an option, since the scaling is determined per-object. For example, one would sometimes have to unnecessarily duplicate materials and reassign materials to objects just to change this option.

A per-object granularity is the most flexible and practically the most useful one. However, it either comes with the cost of determining at runtime whether or not a non-uniform scale is present (undesirable extra computation), or a new per-object flag (also not great).

[dark_sylinc]

The original issue in the topic is fixed.
And the feature is implemented in OgreNext 4.0.

Cheers and happy new year!