Next bunch of fixes

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Next bunch of fixes

Post by Eugene »

Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HLSL11

Facts:
Now samples media have two folders with shader code - Cg and HLSL.
1. HLSL9 shaders are usually copied from Cg folder (code duplication), extension changed from .cg to .hlsl [hard to diff folders] and usually no additional tweaks are needed as HLSL9 is almost strict subset of Cg.
2. HLSL11 shaders are sometimes copied from HLSL9 shaders by adding suffix "_sm4" or "vs40" or "ps40" (code duplication)[hard to diff folders].
3 Resulting files are tweaked as:
a) HLSL11 has stricter rules about data types conversion, and while corresponded changes could be backported to HLSL9 and Cg they usually are not [unnecessary differences]
b) HLSL11 aka D3D11 requires strictly matched signatures between VS_OUTPUT and PS_INPUT. These changes could be backported to HLSL9 and Cg (no penalty as both eliminates unused parameters), but usually are not [unnecessary differences]
c) HLSL11 has another method of texture access - declares two variables, for example "Texture2D tx" and "SamplerState ss" rather than "sampler2D s" as do HLSL9 and Cg, and has different function for sampling, "tx.Sample(ss, xy)" vs "tex2D(s, xy)". This difference can be hidden with D3DCOMPILE_ENABLE_BACKWARDS_COMPATIBILITY flag in many cases except sampling 1D textures and shadow maps.
d) if data types in vertex declaration differs from shader parameter, than HLSL11 reinterprets data rather than convert it as do HLSL9 and Cg. For example, BLENDINDICES are VET_UBYTE4 in Ogre and D3DDECLTYPE_UBYTE4 in D3D9. Cg and HLSL9 accepts them as float4 parameter, conversion is done by D3D9 runtime. But in D3D11 VET_UBYTE4 became DXGI_FORMAT_R8G8B8A8_UINT and bits are copied to register as is. Shader should use uint4 type to reference them, otherwise bit patterns of uint values would be reinterpreted as float4, causing error in logic. In other words, BLENDINDICES could be float4 in HLSL9 and Cg but should be uint4 in HLSL11.

Problem:
We often have 3 very similar but different versions of one shader [Cg/HLSL9/HLSL11] per Ogre branch. Fixes in one version often not propagated to other, including logic fixes and introduction of additional parameters. Merging changes from other Ogre branch became nightmare.

Solution:
1. Share shaders code for [Cg/HLSL9/HLSL11] languages family whenever possible in one shared folder, named HLSL_Cg.
2. Use HLSL9 syntax variation as a base. Cg would accept it as is, HLSL11 would accept it if compiled with D3DCOMPILE_ENABLE_BACKWARDS_COMPATIBILITY flag.
3. Declare one Cg and one HLSL program in material script - for Cg with as many profiles as you wish and for HLSL with vs_2_0 profile. Both should reference the same shared source file, with .cg extension.
4. Handle cases when HLSL11 differs from HLSL9/Cg via #ifdef SHADER_MODEL_4 in shader code.
5. Modify D3D11 render system so that vs_2_0 programs are accepted, change default for enable_backward_compatibility setting from false to true (exposes D3DCOMPILE_ENABLE_BACKWARDS_COMPATIBILITY flag), pass additional SHADER_MODEL_4 define to shader compiler.
6. Merge as much as possible separate [Cg/HLSL9/HLSL11] shaders removing accident differences and fixing D3D11 compatibility issues.
7. Propagate result to other Ogre branches.

Look at https://bitbucket.org/sinbad/ogre/pull- ... in/commits where this solution is used to fix some samples that does not work on D3D11 render system.
Last edited by Eugene on Wed Oct 22, 2014 11:13 am, edited 4 times in total.
User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7157
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 534

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Kojack »

My proposal would be to drop CG altogether. It causes problems with ogre's shader delegate system (a lot of previous hard to track down sample browser bugs were due to CG being used).
Ogre 2.0 is dropping dx9 as well. That means all we really need is HLSL DX11 and GLSL (and GLSLES I guess).

Even Nvidia has dropped CG. The version from 2012 (CG 3.1) is the last version that will be released, it has been end-of-life'd.

CG is also missing the following modern features (found on stack overflow):
compute shaders
atomic counters
shader-writeable storage blocks (shader storage blocks)
shader-writable textures (image load / store)
runtime shader function selection (shader subroutines)
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Eugene »

Now Cg allows to run the same HLSL9-compatible shader on OpenGL render system, and on platforms other than Windows. But there would be no reasons to use it with D3D9 and D3D11 render systems, and it is not allowed on D3D11 WinRT - after all Cg programs would additionally be declared as HLSL.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by dark_sylinc »

I'll be brutally honest here.

Yes, you're right. The data folder is a mess. We have one folder for GLSL, another for GLSL150 which is an almost copy paste, a GLSLES folder (which is a copy-paste), a Cg folder, an HLSL folder (if it's not an almost copy-paste from Cg!), and an HLSL4 folder.
What's worst, sometimes it's not even in folders, so they're mixed in the same folder.

On top of that, we have like 5 vertex shaders that do exactly the same (i.e. Compositor's quad pass vertex shader); and we have really old sample code (VS 1.4 hardware skinning, really? "Bump mapping" PS 2.0 shader, really?)

BUUUUUUUT:
1. Cg is dying. More like anihilated. It's main selling point was being the only cross platform (cross vendor) shader authoring tool with a nice language. Back when GLSL didn't exist (or was really, really bad) Cg allowed to target D3D and GL. And there was hope to target D3D10 too.
Today, the only useful output is for D3D9 really; and I'd rather use HLSL directly. At least PIX allows me to debug HLSL it with source code.
Cg is left as an afterthought in Ogre 2.0

2. Ogre 2.0 no longer supports D3D9, so one less API & language to worry about.

3. The Hlms prevents the data folder explosion of files, as it doesn't require one set of shaders per sample (or per mesh!). The only shaders that would be manually written are compositors and a very few specific to make some mesh have a specific look with lots artistic freedom.

4. Oddities like strict signature matching, sm5 vs sm4, etc. are left to the Hlms.

5. Ogre 2.0 doesn't suffer any of the other problems you mention. The Hlms has two templates folders, one for OpenGL and one for D3D11. The GL path is both used for GLSL and GLSLES (*), the Hlms automatically resolves the small differences between the languages. No code duplication required. The other template folder is for HLSL.
Keeping two sets of templates is much more maintainable than the hellish nightmare in 1.x's data folder.

(*) There are two Hlms implementations though, one called PbsMobile and another simply Pbs (PBS as in physically based shading). The mobile version works on GLES2 and GL3+. The Desktop version only works on GL3+ and D3D11 (and with some hope, in GLES 3 too; but the desktop version is wip yet so I don't know for sure). The reason for having two implementations is that GLES2 is just too different (the lack of constant buffers and dynamic array indexing in the pixel shader in GLES2 is utterly annoying and wastes precious cycles) and its market is too big to ignore.


So, frankly, dedicating time on tidying 1.x samples feels like a complete waste of time. I realized this problem long ago and decided to fix it by the root.
The 2.0 branch wipes the core problem. There is a very low degree of code duplication in the templates (i.e. the hw skinning math in GLSL vs HLSL), but is nowhere near to 1.x where we have 60+ files to handle per platform.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by dark_sylinc »

One more thing @eugene:
I am told imminently Assaf is going to be doing a massive merge with lots of D3D11 fixes; which is why I've been reluctant to accept your pull request. I'm quite sure there are going to be lots of merge conflicts between his and your changes.

You two may want to get in touch.
User avatar
lunkhound
Gremlin
Posts: 169
Joined: Sun Apr 29, 2012 1:03 am
Location: Santa Monica, California
x 19

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by lunkhound »

Eugene wrote:Now Cg allows to run the same HLSL9-compatible shader on OpenGL render system, and on platforms other than Windows. But there would be no reasons to use it with D3D9 and D3D11 render systems, and it is not allowed on D3D11 WinRT - after all Cg programs would additionally be declared as HLSL.
Isn't it true that on non-Windows platforms Cg only works with Nvidia hardware? That seems pretty useless if you ask me.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by dark_sylinc »

lunkhound wrote:Isn't it true that on non-Windows platforms Cg only works with Nvidia hardware? That seems pretty useless if you ask me.
That's a bit of a missunderstanding.

Cg supports outputting to arbvp1 and arbfp1 which is assembly that works on almost all GPUs. However it's quite limited (roughly between shader model 1.x and 2.x); thus NVIDIA released his own GL extensions vp30, fp30, vp40, fp40; being 30 roughly similar to Shader Model 2.0, and 40 similar to SM 3.0

However these extensions were never picked up by the other vendors who favoured GLSL (and eventually, even NVIDIA abandoned them too).

Essentially, any decent Cg shader requires at least vp30/fp30 which is only supported by NVIDIA in both Windows and Linux; unless you use the glsl backend, which sucks anyway.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Eugene »

I`ll be honest too.

I don`t care about samples. At all. I use D3D11 render system for WinRT version of Live Interior 3D, there are bugs that are absent on Mac version of Live Interior 3D, and that is important for me.

Update 1.3.1 of LI3D has description: "Fixed rendering with lights and shadows on ARM devices". It was bug in Ogre, that I found and fixed. I decided to share this fix with community, and it was initial commit of famous Pull request #385 https://bitbucket.org/sinbad/ogre/pull- ... in/commits.

Than I saw the problem with code of finite extrusion shadow volumes of directional lights - direction vector was not normalized, and shadow volume could be clipped as a result. Seven years ago I fixed this problem for point lights, but not for directional. When you do tiled rendering you need custom projection matrix, and this is not compatible with infinite far plane, so even in modern times finite extrusion is required, and should not contain bugs in it. This fix is #4 in Pull request #385.

Than I found a problem with reflections in LI3D, and decide to check could it be seen in Fresnel reflection sample in Ogre. But first that I saw when run Sample Browser using D3D11 render system - was black buttons and samples previews. It was regression since my 22 months old version of Ogre that I use in Live Interior 3D. I tracked the problem down and it was related with broken generation of MipMap levels in D3D11 RS. It`s fix #8 in Pull request #385.

Fresnel reflection sample was really broken - reflection was not identical to the original. The fix was obvious, I just do what was described in comment // TODO: invert culling mode based on mInvertVertexWinding && mActiveRenderTarget->requiresTextureFlipping() that I added some time ago in two places, D3D11RenderSystem::_setCullingMode(...) and D3D11RenderSystem::setStencilBufferParams(...). This fix is in Pull request #385.

BTW, setStencilBufferParams() actually requires more complex fix, that I done in https://bitbucket.org/eugene_gff/ogre-p ... 03f83f8896, this fix is already pulled into Ogre v1.9, but not yet merged into Ogre v2.0.

The Fresnel reflection sample was the first where I added HLSL variant to Cg shader using method described in first post of this topic. I just need working sample to check for bug and to test bugfix. But reflections in Live Interior 3D after this fix disappeared at all. The reason was disabled near clip plane culling in D3D11 RS. Fix for that problem is in Pull request #385, named "[D3D11] Near clip plane should be enabled for reflection to work properly".

Than I got crash due to NULL reference, fix in Pull request #385 is named "[D3D11] Fix for access violation - errors variable is filled only if compilation failed, and is NULL otherwise".

Than I think: "Hey, Ogre`s samples are excellent test suite, I can use them to find problems in D3D11 RS much more quickly". And I run sample "Facial animation", that was near "Fresnel reflection", and got terrible results - random geometry. After tracking it down the problem was in D3D11 RS - D3D11HardwareBuffer::copyData() erroneously ignores shadow buffer, and subsequent locking it with read semantic provides trash as input to animation algorithm. Two commits named "[D3D11] D3D11HardwareBuffer::copyData() erroneously ignore shadow buffer, causing broken FacialAnimation sample." and "[D3D11] Use GPU to do D3D11HardwareBuffer::_updateFromShadow, also fixing crash due to CopyResource called while HW buffers are mapped." are dedicated to this problem in Pull request #385.

Than I recall, that Character sample crashed when swords are drawn - that problem was at times when I ported Ogre to WinRT, something like 2 years ago, but at those times I have no time to investigate it further. The bug was still here, so I tracked it down to the shader code that RTSS generates to handle 1D textures that were used by Ogre::RibbonTrail - it was incompatible with D3D11. Fix is named "[D3D11] RTSS generates wrong HLSL SM4 shader code for 1D textures, causing crash in Ogre::RibbonTrail (Character sample, press 'Q')", and is in Pull request #385

Than I start to port and fix samples one by one in alphabet order, as 23 from 46 were broken on D3D11 RS, due to the missing HLSL shaders or bugs in D3D11 RS. Cell shading, Bump mapping, Grass, and voila - Dual quaternion skinning sample was broken due to D3D11 RS passes BLENDINDICES as uint4, RTSS generated code references them as float4, and D3D11 doing reinterpret_cast rather than conversion as D3D9 does. Fix is named "[RTSS, D3D11] Use uint4 type for VET_UBYTE4 aka DXGI_FORMAT_R8G8B8A8_UINT on D3D11 to avoid reinterpreting uint as float." and is in Pull request #385.

And now you say, that my pull request should be ignored due to the possible conflicts? It would be better that all those conflicts would be resolved in my way, and all samples were fixed not waiting HLMS, so that bugs in D3D11 RS would be revealed and fixed too. IMHO.
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 99
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Wolfmanfx »

No thats a misunderstanding - nobody is ignoring your PR. We just mean that assaf needs todo the merge manual because the worked the last couple of months to make it working with a real product (like yours). Hopefully both products will benefit from the merged fixes.
Thx for the massive fixes :)
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by dark_sylinc »

Nooooo!! We're NOT going to ignore your pull requests.

Your D3D11 commits are more than welcomed. What I said is that merge conflict resolution is most almost sure to happen because both improvements (yours and Assaf) are big and I'm sure some of the bugs were fixed by both you and him.
If you two get in contact things could go smoother. In fact the description of your reasoning is great for knowing what was broken and how it was fixed.

I'll propose you something: If by the end of the week there are no news regarding your PR (neither from Assaf or you), I'll merge it.
And now you say, that my pull request should be ignored due to the possible conflicts? It would be better that all those conflicts would be resolved in my way, and all samples were fixed not waiting HLMS, so that bugs in D3D11 RS would be revealed and fixed too. IMHO.
Ok, we're mixing things here. Your original post was about a proposal/idea to clean the samples. My reply regarding that specific idea is that I, personally, don't have much interest in cleaning the current samples (because new ones will be on their way) just for the sake of cleaning them.
Your pull requests are fixes to the D3D11 RS and they happen to also include fixes for samples because it helps finding and fixing bugs; then they're welcomed.
What I said is that if you want to clean the samples for the sake of making it clean then you're wasting your time. But if the reason is that it helps troubleshooting current D3D11 problems (or any other RS); then obviously it's not wasted time.

Please take my apologies if I offended you.
User avatar
lunkhound
Gremlin
Posts: 169
Joined: Sun Apr 29, 2012 1:03 am
Location: Santa Monica, California
x 19

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by lunkhound »

Glad to hear all of these fixes are coming to DX11 RS. :D

Kinda surprised so much work is being done on the 1.9 version of it though, especially since a whole GSOC project worth of DX11 fixes is absent from 1.9.
It'll be nice to see these make it into 1.10 along with all the other DX11 fixes.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Eugene »

dark_sylinc wrote: I'll propose you something: If by the end of the week there are no news regarding your PR (neither from Assaf or you), I'll merge it.
OK, let's do it.

One question. Seems that 'default' branch is v1.10, not v2.0, am I correct? If so, I can use it for my own development, and can rebase my pull request onto it, this would reduce merging conflicts due to the differences in blanks.
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 99
Contact:

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Wolfmanfx »

Yes 1.10 is default.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Idea: Share shader code for Cg/HLSL9/HLSL11

Post by Eugene »

OK, I encounter conflict during merge with political consequences.

HLSL11 has another method of texture access - declares two variables, for example "Texture2D tx" and "SamplerState ss" rather than "sampler2D s" as do HLSL9 and Cg, and has different function for sampling, "tx.Sample(ss, xy)" vs "tex2D(s, xy)". This difference can be hidden with D3DCOMPILE_ENABLE_BACKWARDS_COMPATIBILITY flag in many cases except sampling 1D textures and shadow maps - demonstrated by crash in Character sample, when swords are drawn. Reason - SM2.0 emulates 1D textures using 2D, and sampler1D actually emits Texture2D reference, that is incompatible with Texture1D that is used by D3D11.

I solved this issue, fix named "D3D11] RTSS generates wrong HLSL SM4 shader code for 1D textures, causing crash in Ogre::RibbonTrail (Character sample, press 'Q')" is in pull request #385, can be seen here https://bitbucket.org/eugene_gff/ogre-p ... 114314a168
Idea is that resource MUST be declared using SM4 rules, i.e. Texture1D + SamplerState rather than sampler1D.

The problem is that in v1.10 during last GSoC was made incomplete attempt to solve this problem too, but generated shader code is overcomplicated and is not even compilable.

And I need to undo these changes to apply my fix.
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HL

Post by Zonder »

I see no problem with undoing another piece of work if yours is complete and working and the other still needs further work.

But I haven't looked at both implementations to determine which is the best approach though. Maybe the mentor/student could give some information.
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HL

Post by Eugene »

Zonder wrote:I see no problem with undoing another piece of work if yours is complete and working and the other still needs further work.

But I haven't looked at both implementations to determine which is the best approach though. Maybe the mentor/student could give some information.
Both are similar in spirit. Idea is to use struct to wrap two parameter, and pass it to RTSS that expects one parameter. But there are numerous differences too.

1. Such struct can not be used to declare those parameters (compiler restriction). Parameters should be declared separately and gathered to struct on demand. This is where GSoC attempt is very complex and not completed and requires numerous changes in RTSS. I solved this problem with 10 lines of C++ code in one place that generates 3 lines of HLSL.
2. I prefer approach where well-known sampling instructions are overloaded for such struct. This, together with #define sampler2D SamplerStruct2D allows to not change RTSS library - function that take sampler2D and use tex2D to sample continue to work unchanged. It is inline with an attempt to share Cg/HLSL9/HLSL11 shader code - only #include "ShaderModel4_Prerequisites.hlsl" is required. Moreover, this "ShaderModel4_Prerequisites.hlsl" could be used to port other SM2.0 shaders to SM4.0 without code modification. GSoC approach requires that every place that uses tex2D would be rewritten using library function FFP_SampleTexture(), which body has #ifdef-ed SM2.0 and SM4.0 implementations - not only in generated code where FFP_SampleTexture is method of abstraction, but inside RTShaderLib too.
3. I prefer the macro that is predefined when compiling shaders as HLSL11 shaders to be named SHADER_MODEL_4 rather than D3D11. If you think about D3D12 - you will understand. Unity3D for example uses SM4_PROFILE macro for the same purposes.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HL

Post by Eugene »

OK, I ported fixes to v1.10 branch - pull request #391.
https://bitbucket.org/sinbad/ogre/pull- ... 10/commits

This pull request does not contain shared shaders for samples. Dead code in D3D11 RS is not removed - this code is really unnecessary and even confusing, but this removal could be delayed till Assaf merge his own changes. There are some old fixes that were accepted to v1.9 but lost in previous merging from 1.9 to default - were repeated. Issue with RTSS generating wrong code for SM4 was probably caused by RTSS misconfiguration - results were plain wrong, but seems that code that should do the right work was present, just not invoked. I still replaced this code with much simpler version for the reasons described above.

Pull request #385 is still valid - it fixed the same bugs but in branch 1.9
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HL

Post by Eugene »

So, here I am again.

Next pull request is PR #404, https://bitbucket.org/sinbad/ogre/pull- ... ny/commits
It has bunch of fixes for D3D11 and several fixes for OS X. I`m afraid, that nobody would be brave enough to accept such pull request with motivation like "Part about D3D11 is OK, but who knows about those Mac stuff" or "Part about Mac is OK, but who knows about those D3D11 stuff". Therefore I want to escalate this issue, so that all questions could be asked and answered here.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Bunch of fixes for D3D11 + Share shaders for Cg/HLSL9/HL

Post by dark_sylinc »

All that you've touched was code I'm familiar with (except Mac). And all looks OK on my end. I want to merge this.
The only thing that I didn't like was that NSight checking is now a function rather than being enveloped in a class. But this is just syntactic sugar.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Build Ogre for WinStore and WinPhone without custom steps

Post by Eugene »

So, here I am again.

Next pull request is PR #410, https://bitbucket.org/sinbad/ogre/pull- ... ne/commits
Several years ago I adapted Ogre and D3D11 RS to work on WinRT. I send patches to CMake so that it was able to generate library projects compatible with WinRT, but executable projects were still not supported. SampleBrowserWinRT project was created by me and placed into the Ogre repository as is, and it requires numerous manual steps to be linked with the rest of Ogre. Then similar SampleBrowserWinRT.Xaml appeared, than SampleBrowserWP8 for WinPhone 8.0
The good news - Microsoft forked CMake and made it able to generate WinRT compatible projects of any kind, i.e. dll and exe, WinStore 8.0 and 8.1 and even WinPhone 8.0 and 8.1. CMakeMS is located here http://cmakems.codeplex.com/ Even better news - this support was reintegrated into the original CMake, and is available in development builds of CMake 3.0.2, and would be released in CMake 3.1. The main reason of existence of CMakeMS now - it could be installed side-by-side with any version of original CMake. And the best news - with this support I teached Ogre to build itself for WinStore 8.0/8.1 and WinPhone 8.0/8.1 without all those numerous manual steps required before.
Last edited by Eugene on Fri Oct 03, 2014 10:41 am, edited 1 time in total.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Bunch of fixes for SampleBrowser on HiDPI devices

Post by Eugene »

So, here I am again.

Next pull request is PR #414, https://bitbucket.org/sinbad/ogre/pull- ... pi/commits

Problem: SampleBrowser show own cursor not on the spot where hardware cursor located on HiDPI devices (WinStore, WinPhone, OS X Retina).
Additionally, cursor can not be moved to bottom or right part of screen of WinStore or WinPhone emulator, mouse is slower (2x slower on OS X retina with content scaling factor = 2.0)


Many windowing systems that support HiDPI displays use special points to specify
size of the windows and controls, so that windows and controls with hardcoded
sizes does not become too small on HiDPI displays. Such points have constant density
~ 100 points per inch (probably 96 on Windows and 72 on Mac), that is independent
of pixel density of real display, and are used through the all windowing system.

Sometimes, such view points are chosen bigger for output devices that are viewed
from larger distances, like 30" TV comparing to 30" monitor, therefore maintaining
constant points angular density rather than constant linear density.

In any case, all such windowing systems provides the way to convert such view points
to pixels, be it DisplayProperties::LogicalDpi on WinRT or backingScaleFactor on MacOSX.
We use pixels consistently through the Ogre, but window/view management functions
should take view points for convenience, as does the rest of windowing system. Such parameters
should not be mixed with pixels without being converted using appropriate scale.

Sometimes such scale factor can change on-the-fly, for example if window is dragged
to monitor with different DPI. In such situation, window size in view points is usually
preserved by windowing system, and Ogre should adjust pixel size of RenderWindow.

This is the reason of creating RenderWindow::getViewPointToPixelScale() function, that can be used to convert from windowing system points to pixels. Note, that conversion from view points to pixels is already done by Ogre in WinRT and OS X Cocoa render windows, so the scale factor just was made available externally. And as input is coming to us from windowing system too - it should be adjusted by this function as well, if hit-testing logic expect pixels - that part was missing in SampleBrowser and causes problems described at the start of this post.
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Problem in use FlipSequential mode, FSAA and sRGB gamma toge

Post by Eugene »

I implemented FlipSequential mode for Win32, but then was stopped by problem occured when it is activated together with FSAA and gamma

1) Render Target View should have DXGI_FORMAT with _SRGB suffix for gamma correction and be multi sampled for FSAA
2) Swapchain in FlipSequential mode does not support FSAA and _SRGB suffix - creation is just failed

Therefore I create intermediate texture, that is multi sampled and is actual render target, data from transferred to swap chain back buffer using ResolveSubresource. So far, so good.

If gamma correction is not enabled - all works as a charm, rendering directly to back buffer of the swap chain if no FSAA, or through intermediate texture, everything is created without SRGB suffix.

Swapchain(FSAA=0, SRGB=0) <= RenderTargetView(FSAA=0, SRGB=0)
Swapchain(FSAA=0, SRGB=0) <-ResolveSubresource- Buffer(FSAA=4x, SRGB=0) <= RenderTargetView(FSAA=4x, SRGB=0)

If gamma correction is enabled and there are no FSAA - swap chain is created without SRGB suffix (is not supported for swap chains in flipSequential mode), RenderTargetView is created with it for gamma correction, special relaxed rule allow to use SRGB RTV with non-SRGB buffer if it is back buffer from swap chain.

Swapchain(FSAA=0, SRGB=0) <= RenderTargetView(FSAA=0, SRGB=1)

But in last situation, gamma correction + FSAA, SRGB suffix of interim render target should be absent so that ResolveSubresource be able to copy data to swap chain, and should be present so that SRGB RenderTargetView could be created to perform gamma correction, as relaxed rule about SRGB RTV for non-SRGB buffer is not applicable for buffers that are not back buffers of the swap chain.

Swapchain(FSAA=0, SRGB=0) <-ResolveSubresource- Buffer(FSAA=4x, SRGB= ??? ) <= RenderTargetView(FSAA=4x, SRGB=1)

Problem could be solved by second interim buffer, i.e.

Swapchain(FSAA=0, SRGB=0) <-CopyResource- Buffer2(FSAA=0, SRGB=1) <-ResolveSubresource- Buffer(FSAA=4x, SRGB=1) <= RenderTargetView(FSAA=4x, SRGB=1)

but I really don`t like this solution, for performance reasons too. Situation is even worse as FSAA is a must have feature, FlipSequential mode is forced on WinRT and hardware gamma correction is very important for realistic rendering.


Code is at https://bitbucket.org/sinbad/ogre/pull- ... pi/commits , pull request #414
User avatar
lunkhound
Gremlin
Posts: 169
Joined: Sun Apr 29, 2012 1:03 am
Location: Santa Monica, California
x 19

Re: Bunch of fixes for SampleBrowser on HiDPI devices

Post by lunkhound »

Is there any way to use a pixel shader to handle the resolve and gamma conversion in one step?
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Bunch of fixes for SampleBrowser on HiDPI devices

Post by Eugene »

lunkhound wrote:Is there any way to use a pixel shader to handle the resolve and gamma conversion in one step?
I think that sequence of ResolveSubresource + CopyResource would be more effective. Actually I decide to use this approach, if user really want FSAA + gamma correction on FlipSequential swapchain. Commit would be pushed soon.

P.S. Pull request PR #415, https://bitbucket.org/sinbad/ogre/pull- ... gamma/diff
User avatar
Eugene
OGRE Team Member
OGRE Team Member
Posts: 185
Joined: Mon Mar 24, 2008 4:54 pm
Location: Kraków, Poland
x 41

Re: Next bunch of fixes

Post by Eugene »

So, here I am again.

Next pull request is PR #429, https://bitbucket.org/sinbad/ogre/pull- ... d0/commits

•[MeshLod sample] Preserve edge list if it was present before LOD manipulations
•Wrong range locked for index buffers with nonzero indexStart (potential data corruption)
•Fixed loading meshes with shared index buffers (crash on switching to Lod#2 for meshes loaded from disk with Lods)
•[MeshLod sample] Added ability to strip Lod0 with Lod0-only vertices, i.e. perform irreversible mesh simplification
•Fix for compilation under Win7 SDK - thanks to transporter for discovering
Post Reply