RTSS Improvements

[syedhs]

There is another performance issue/bug which I have run into and I believe its the same as syedhs has a while back. I did solve it in my own pipeline, and was something to do with shadow casting renders changing the number of lights that were used on the material, meaning as soon as you used more than 1 light the RTSS would compile new shaders twice every frame because the number of lights changed. Now the RTSS does use a hash system to see if a shader config has already been compiled to prevent stalls like this, but to be honest its not working correctly. Actually it is but its in the wrong place IMHO.

Does this mean the bug I reported (Or what you believed to be similar one) is solved? Because I cannot test the scenario which repro the bug easily - it is difficult because it needs 5 or more computers to get the bug reproduced.

Thank you.

[TheSHEEEP]

@Assaf: I think most of your points are valid and corrcet, but I also think that some of them are rather biased (people tend to be biased about their children - huh ), so I'm going to comment only on those I disagree with and silently nod in agreement to all other points

2. The RTSS isn't that hard to learn

Bringing in single persons as examples for a generalized statement is not really a good idea. I guess that Lior was already an Ogre, or at least shader expert to begin with, probably both. That makes it far easier to learn anything Ogre or shader related. Also "in a few days" IMO is a lot of time for something that should (again, IMO) be learnable in a few hours. Really, RTSS does not do much more than tacking together shader code depending on varying conditions. That is the basic idea, and it should not take days to be able to do that.
Also, people learn differently. Take me, for example, I learn by reading, copying and changing example code. There are no good examples for RTSS, to be honest, or too few of them. Others learn by reading tutorials, mostly. There are no good tutorials for RTSS, or too few of them. Others learn by reading documentation.
Others (like Lior, it seems) learn by reading the code of the component itself. That only works if it is written in an understandable manner. Now each coder has a different opinion on "understanable", so this point is mood already. It might work like that for some, and not for others.
Having no documentation and/or good samples is a stunt one could never pull if he wanted to sell something. Yeah, this is open source, and having a RTSS system without docs or good samples is better than having none at all, but I guess you still see my point

5. Cg is not dead and actually the best way I know not to write shaders code twice

I would add that it is not dead, yet. With the increasing rise of OpenGL (and for good, it is IMO far more lean than DirectX, and in many cases produces faster results, not being supported as well by drivers is also a myth from the past) most projects simply don't need DirectX any more. And no, I do not see Windows Phone as an argument here, it's spread is close to non-existant. For the very reason that it does not have much software to offer. Which, in turn, is for the reason that it does not support OpenGL. Which is something MS will change in the future - or Windows Phone will be another MS failure (like Win 8 or Silverlight). In both cases, DirectX will not stay an argument in favor of CG. Consoles would be an argument, but Ogre does not target consoles. In our project using Ogre, we only use OpenGL. And we do not need anything else, as all platforms are covered (desktop game). We do not need to write shader code twice, while covering all relevant platforms, so why would we need Cg?

8. Some people wrote that the RTSS\OGRE wasn't built for artists

Well, the point (I think) was that it was not built to be used by artists. Artists obviously benefit from it.

9.There is more then one sample in the sample browser that is dedicated for it.

Yes, but it is one of those samples (like the Deferred Shading sample) that show so many things at once it becomes hard to follow them. Having more samples with less sample code in each would be far better. Especially for something very flexible.
So I'm all for a GSoC that reworks the samples to be less a "show what Ogre can do" and more a "show how to it". Could be coupled with improving the documentation, maybe?

10.The RTSS makes OGRE truly cross platform – as it makes the same code, with the same models – run on different platforms that support totally different shader languages.

I really don't mean this personally, so don't take it as an insult, but you really seem to be stuck in the "Windows = DirectX" theory. This is not true any more, not by any means (again, except Windows Phone, but then you might as well count PS Vita as an important target). OpenGL can do everything DirectX can, and will be at the same speed, or even faster. I know this is not true for the OpenGL renderer of Ogre, but that is being reworked at this very moment (and not the fault of OpenGL in the first place). And I'm very positive that this will end the slowness of Ogre's OpenGL renderer. Again, our prioject uses only OpenGL, and there is no reason why it should be any different. Not since a few years. As much as I dislike gaming on mobile devices, it has definitely pushed OpenGL everywhere, and that is for the good of every coder.

[Assaf Raman]

Let me answer some of your points.

1. You think that DirectX is obsolete and could be replaced by OpenGL. For my uses - DirectX get much better performance then OpenGL - I worked hard for years to try to make OpenGL run as fast as DirectX and in the end you just can't make it happen, you will have to believe me that is a fact - you can also find multiple threads on this forum to support my claim. Also - OpenGL doesn't support more then one GPU and DirectX does. There are extensions for both NVIDIA and AMD to support multi GPU rendering - but this only works on their high end GPUs that are 4 times more expansive (Quadro vs Gforce). And the latest thing that I can only do on DirectX (an issue I only solved in my product in the last few months) - in Windows you can have true V-Sync in windowed mode (meaning V-Sync without any V-Blank misses in a window that is not full screen) - only using DirectX9Ex - meaning it is not supported by GL or even Direct9c. So - for large scale multi-GPU projects like I do - DirectX is the only solution at this point in time - for performance, multi-GPU support, and for v-sync in a window. So sadly, "Windows = DirectX" is a fact and not just a theory for me. BTW - on the XBox you also don't have OpenGL as far as I know, I never work on it - so I can't tell, but searching Google may give you results to support my claim, so there may be at least one other popular platform except windows that supports DirectX only. And according to this - Cg is also supported on the Xbox.

2. You are missing a simple tutorial how to extend the RTSS, and perhaps also how to to add it to a project. Well, this is something that is easy to resolve I guess.

3. The use by artists point - without the RTSS if an artist wants to use a shader - he need to learn a shader language, with it - he just need to add some lines to the material. I think it improves his life, don't you?

4. Lior didn't have background in OGRE and little knowledge in shaders, but he is a very smart guy, so I guess we can't assume anything based on his performance. (Guys - don't tell him about this post as I will never hear the end of it)

[al2950]

Does this mean the bug I reported (Or what you believed to be similar one) is solved? Because I cannot test the scenario which repro the bug easily - it is difficult because it needs 5 or more computers to get the bug reproduced.
Thank you.

Well I am afraid I did not get round to fixing it in RTSS, as I only had 2 days to debug, fix & test! My solutions was to make sure RTSS did not auto update the light count

Code: Select all

Ogre::RTShader::RenderState::setLightCountAutoUpdate(false)

The light count was then set manually from the materials using something like;

Code: Select all

iteration 1 per_n_lights 4 point/spot

@Everyone
I appreciate everyone has got some strong opinions on this topic, including me , but I think we need to try and start forming a plan for a flexible yet easy to use material system for Ogre 2.x, I personally think RTSS was a good start and I will take some time to look at Shiny as well. I am happy to start putting some hours into this especially in the winter, BUT I MUST HAVE (constructive!) INPUT/BUY IN FROM YOU, AS I DONT WANT TO FINISH AND THEN FOR IT TO BE THROWN AWAY! A decent material system is going to need momentum for it to take off.

Gus

[mkultra333]

Just a side note to TheSheep

I really don't mean this personally, so don't take it as an insult, but you really seem to be stuck in the "Windows = DirectX" theory. This is not true any more, not by any means (again, except Windows Phone, but then you might as well count PS Vita as an important target). OpenGL can do everything DirectX can, and will be at the same speed, or even faster.

I dunno... almost every Steam game I have is DirectX, and I have a lot of them. Personally I use DX because of the stereoscopic support too, something lacking in OpenGL for normal PC cards. So I'd hate to see a sudden change from being both DX and OGL compatible to a sudden emphasis on OGL.

On another note, I'm no longer as opposed to an Ogre specific shader language as I was before, as long as it was very similar to HLSL and GLSL. My rough understanding is that this is more or less what Shiny does. It would be neat to be able to write in either HLSL, GLSL or OgreShinyShaderLanguage. Once I was confident that the OgreShinyShaderLanguage worked properly, and that my OgreShinyShaderLanguage skills could be easily translated to HLSL or GLSL if I had to move on, I'd probably write all my code in OgreShinyShaderLanguage for the huge simplification it would allow with making my game cross platform. (As for CG, I don't like it much... I used it before and found it buggy and limiting. Things went much better once I changed to HLSL.)

Regarding other uses of real time shader creation though, apart from simulating the Fixed Function Pipeline, they really mean nothing to me. I don't mind writing different versions of shaders at all, it seems easy. I don't IFDEF stuff, I just make another shader and change a few things. Most the differences between my own shaders come about from the textures being used, so a core of about 10-15 templates used in various materials covers 90% of my shader needs. That doesn't seem that unmanageable to me. And my rendering setup is far from simple.

So apart from the use in an OgreShinyShaderLanguage and Fixed Function Pipeline simulation, real time material generation is something I don't think about or feel like I need.

[TheSHEEEP]

1. You think that DirectX is obsolete and could be replaced by OpenGL. (....)

Okay, you have some very, very specific needs. Multi-GPU support and full VSync in windowed mode is something I would call not required for anything. So, OpenGL is behind DirectX on bleeding edge stuff, I give you that. But I really can't agree with the performance, as I have made the exact opposite experiences. So, let's just leave the performance question out. Xbox does not have OpenGL, yes, but as Ogre is not targeted at consoles, I think it does not matter here.

I should probably rephrase: You can do anything with OpenGL that you can do with DirectX, that you actually need for a game these days. On my two years old graphics card, I can play any game on the highest settings with a good frame rate. I really see no need for multiple graphic cards here. Maybe in some years, who knows, but by then OpenGL will very likely have that, too.

I dunno... almost every Steam game I have is DirectX, and I have a lot of them. Personally I use DX because of the stereoscopic support too, something lacking in OpenGL for normal PC cards. So I'd hate to see a sudden change from being both DX and OGL compatible to a sudden emphasis on OGL.

That includes stereoscopic vision. That is only a question of shaders (you can find multiple tutorials for that for OpenGL). Maybe DirectX has that as a built-in feature, I don't know. But it's certainly no limitation on OpenGL.
And the reason for most games on Windows being DirectX is easy to explain. OpenGL being on par (for games) with DirectX is only the case since a few years, many devs still think OpenGL to be inferior. And for bleeding edge, that even seems to be the case. But how many people need bleeding edge? < 1%, I'd say. The OpenGL renderer for Ogre being slower than the DirectX one doesn't help, either, but I hope this is resolved soon. Also most developers started with DirectX and just kept their preference. And DirectX = MS, Windows = MS, so it seems only logical to use it.
But as soon as you want to go cross/platform, you have to make a decision. Either do work twice or use one ring to rule them all. And to me, that ring clearly is OpenGL as it targets all platforms relevant for games (that are not consoles, but even there only Xbox falls flat). And it does not have the extra step that CG needs.

I'm absolutely convinced that 100% of all games you can find on Steam that use DirectX, could also use OpenGL and the result would be the same game, except that it would also work on other platforms (well, the renderer at least).

And don't get me wrong, I'm not suggesting Ogre should focus on OpenGL. I would like that, but most people would disagree. But if I did my own rendering engine for games, I wouldn't even think about using DirectX. Not because I think DirectX is bad, it clearly isn't. But because I know that OpenGL is just as good (if not better) for my purposes and runs on almost every platform.

Regarding other uses of real time shader creation though, apart from simulating the Fixed Function Pipeline, they really mean nothing to me. I don't mind writing different versions of shaders at all, it seems easy. I don't IFDEF stuff, I just make another shader and change a few things. Most the differences between my own shaders come about from the textures being used, so a core of about 10-15 templates used in various materials covers 90% of my shader needs. That doesn't seem that unmanageable to me. And my rendering setup is far from simple.

So apart from the use in an OgreShinyShaderLanguage and Fixed Function Pipeline simulation, real time material generation is something I don't think about or feel like I need.

Yep, same here. I don't like having my shader code fragmented over dozens of files (which are in the worst case in C++ so I'd have to recompile to see changes) and not having the possibility to view one full shader in one file. And I really haven't encountered a situation that could not be solved by using #ifdefs in a vertex or fragment shader.
So maybe we are just the wrong guys in this discussion

[Zonder]

Xbox does not have OpenGL, yes, but as Ogre is not targeted at consoles, I think it does not matter here.

I disagree here completely. OGRE is a cross platform rendering engine so It should be full capable of rendering on consoles, though not as a core feature. With the new PS and XBOX OGRE probably will work fairly easily with the PS needing more work as they have a custom OpenGL implementation (previously) saying that though there is every chance they will use AMD's GL implementation as it already exists for there hardware.

[TheSHEEEP]

Edited my post, don't worry I'm not suggesting removing DirectX or anything like that
If Ogre would also officially support the major consoles, I'd also vote either for CG or for a custom Ogre language.

Also, we're derailing this, which certainly wasn't my intention, so I'll just keep anything non-RTSS related to myself in this thread from now on

[scrawl]

I told you I'd write a proprosal, and here it is: http://www.ogre3d.org/forums/viewtopic.php?f=25&t=79006
Looking forward to your feedback.

Also, we're derailing this, which certainly wasn't my intention, so I'll just keep anything non-RTSS related to myself in this thread from now on

Feel free to continue in my new thread!

[Owen]

Xbox does not have OpenGL, yes, but as Ogre is not targeted at consoles, I think it does not matter here.

I disagree here completely. OGRE is a cross platform rendering engine so It should be full capable of rendering on consoles, though not as a core feature. With the new PS and XBOX OGRE probably will work fairly easily with the PS needing more work as they have a custom OpenGL implementation (previously) saying that though there is every chance they will use AMD's GL implementation as it already exists for there hardware.

PSGL is irrelevant. Of the games which shipped using it, the majority are "HD remakes", where performance is much less critical and they're mostly a texture-detail update of PS2/XBox/GameCube era games.

(PSGL's performance is low, therefore I assume a vicious cycle of "Its' slow so nobody uses it" and "Nobody uses it so why work on it" occurred)

Most PS3 games use a mixture of Sony's libGCM (Which does low-level buffer/command management) and direct hardware access. In this regard, perhaps Mantle will prove to be the most comparable thing in PC space (perhaps at a similar level to libGCM, except without the direct hardware access escape hatches)

[TheSHEEEP]

In this regard, perhaps Mantle will prove to be the most comparable thing in PC space (perhaps at a similar level to libGCM, except without the direct hardware access escape hatches)

It would be wonderful.
*Enables realism* But what I actually is expect is one more minor-to-medium-spread language that we (and other engines) will possibly have to cover. Ugh.

[Zonder]

Mantel is interesting guess another renderer will be needed anyway this is getting very OT