Keeping up with V1

[andrecaldas]

A few days ago, I bumped into a "bug". Then, I realized (@paroj told me) that the bug was fixed in V1, but it still existed in Ogre-Next.

Today, I have spent a couple of hours implementing Wayland windowing for Vulkan in Ogre-Next. In the process, I have found out that there already is a Wayland for GLX in V1 (committed precisely in my birthday!), but not in Ogre-Next.

Although the Ogre website states that Ogre-Next and Ogre3D are different "sister projects", I do have this idea that Ogre-Next is supposed to supersede V1 some day. Maybe not everyone is so excited about Ogre-Next... I don't know...

Shouldn't Ogre have some sort of policy to warrant that improvements done in V1 shall be done in Ogre-Next whenever it makes sense?

Sorry if this has already been debated somewhere else.

[paroj]

it used to be worse https://www.rojtberg.net/1360/on-ogre-versions/

the problem with policies is that it also takes someone to act on them..

btw. Ogre1 also has Wayland windowing on Vulkan:
https://github.com/OGRECave/ogre/commit ... 456a0b4bad

[andrecaldas]

it used to be worse https://www.rojtberg.net/1360/on-ogre-versions/

Very interesting and very informative!

the problem with policies is that it also takes someone to act on them..

Yes... I thought so. I suppose Ogre and OgreNext are "too much" for the project to handle.
I am really new here and I don't know much of the history... or what the other developers think about Ogre-Next. I mean, even if people don't need it, we should care for it. I think...

Maybe, I good policy for Ogre-Next would be if it was designed to replace a few directories in Ogre3D. In Ogre-Next branch, it would be forbidden to touch any directories besides xxx, yyy and zzz and any files besides aaa, bbb and ccc. The directories xxx, yyy and zzz would not exist in Ogre3D and merging would only need to deal with files aaa, bbb and ccc that would certainly raise conflicts. Ogre-Next would regularly merge changes from Ogre3D.

In this case, Ogre-Next would simply be a branch. The policy would simply be something to make it easy to regularly merge from 3D to Next.

btw. Ogre1 also has Wayland windowing on Vulkan:
https://github.com/OGRECave/ogre/commit ... 456a0b4bad

I do not really need Ogre-Next. I was going for it because my software will probably not be released in the next 10 years. So, I use C++26, SDL3, etc. I worry about Ogre-Next's future... looks like not every one really cares for it.

But if you guys do have a "merge strategy" for when it is "ready", then it's okay. If you do not, I bet it will be a lot of work to reintegrate all those years of progress in Ogre3D that Ogre-Next is missing...

[paroj]

well, there are several problems with merging one way or the other:

• Ogre-Next is quite conservative with modern C++ features and will only adopt C++11 with the 3.0 release, while Ogre1 was using C++11 since 2018 (currently on C++14)
• related to the above, Ogre1 tends to embrace the STL containers, while Ogre-Next tends to avoid them (e.g FastArray)
• Ogre-Next likes to use Ogre::IdString all over the place, which is not compatible with language bindings (Python, C#) that Ogre1 provides
• Ogre1 uses ogre script exclusively, while Ogre-Next uses JSON for materials
• I do not like the streaming texture introduced in Ogre-Next 2.2, as in forcing the async API onto all users
• I do not like Ogre-Next of duplicating stuff into the v1:: namespace instead of providing backwards compatible API
• I generally do not think that breaking API that much is justified. Obsoleting years of documentation, books etc. (hence me maintaining Ogre1 I guess ^^)
• the shader generator (RTSS) in Ogre1 is still completely optional, while the one in Ogre-Next (HLMS) is used throughout the codebase

given that the above has been this way for years now, it is quite hard to identify code that can be shared - it certainly will not work on per-folder level.

Actually, Ogre-Next used to be the v2-1 branch of the Ogre repository. We did the repo-split after realizing that merging will not be feasible anytime soon.

Back then I was suggesting to call it Ogre-HLMS instead of Ogre-Next to emphasize the focus of the development. While the HLMS brings very significant performance gains for large scenes, it also comes with many tradeoffs when it comes to flexibility in my biased opinion.

[dark_sylinc]

It is as Paroj says.

Originally OgreNext was just Ogre 2.x (2.0 -> 2.1 -> ...) with full 1.x removal.

But OgreNext changed too much how things were being done and some parts of the community did, understandably, not like or need that direction.

Thus Pavel (paroj) stepped as maintainer for 1.x, who was already doing his own fork. And eventually the split into Ogre and OgreNext happened since like Pavel says, there were too many differences.

On the other hand there are still many similarities:

1. IIRC Pavel backported some of my Vulkan OgreNext stuff back into Ogre

2. OgreNext changed how Windowing worked but many things were still retained so they remain more or less compatible, and I also even looked at how Ogre is doing some windowing as they had some fixes

3. Eventually a user notices Ogre fixed <something> and submits a ticket or PR to port it to OgreNext

4. The concept of SceneNodes, MovableObjects & Renderables have changed a bit but remain largely similar

5. ResourceGroupManager remains largely compatible

6. What OgreNext calls v1 / low level materials is what Ogre just calls materials; and for certain stuff (like postprocessing effects) very little porting effort is needed

Thus someone who is used to Ogre will feel familiar with OgreNext, yet at the same time get angry that certain things that used to be <X> way now need to be done quite differently.

While the HLMS brings very significant performance gains for large scenes, it also comes with many tradeoffs when it comes to flexibility in my biased opinion.

I agree 100% percent.

Of course my bias is towards HLMS, but the analysis is spot on: It brings performance gains for large scenes but tradeoffs\* flexibility as a result.

The flexibility problem would be easier to fix from a game engine approach, rather than graphics engine.
It is only very recently that I added tutorials on how to customize the Hlms. Nonetheless what was just a few lines of material scripts in Ogre, is more C++ code in OgreNext.

\* Although in part of that flexibility loss does not have to do with performance, but rather that customizing an Hlms will let you integrate your code with existing features (shadows, skeletal animation, etc).

[andrecaldas]

well, there are several problems with merging one way or the other:
[...]

Thank you very much for your time!!! I really appreciate all this careful explanation!

I was really misled by the "next". I was under the impression that there would be a merge in the near future. From all you have exposed, now I think Ogre1 looks more like me. I hadn't even tried it.

given that the above has been this way for years now, it is quite hard to identify code that can be shared - it certainly will not work on per-folder level.

I hope part of the nice stuff from Ogre-Next can be backported to Ogre1. I wish things in Ogre-Next are loosely coupled so parts of it can be separated, like the resource part management. I love it when bells and whistles are optional.

Eventually a user notices Ogre fixed <something> and submits a ticket or PR to port it to OgreNext

I guess you really need a strategy to not waste all improvements that are done in Ogre1. Ogre-Next seems to be missing lots of improvements.

If we were Genies and not Ogres, I would make a wish...

I am sure Ogre-Next is doing a great job in innovation. I hope both ogres can "converge" in a near future.

[andrecaldas]

[...] was using C++11 since 2018 (currently on C++14)

Since I have never managed a big project, I really do not understand why this strong requirement of not using things above C++11 or C++14. Other projects do just like you do.

Is it because MSVC users do not have the money to buy the new version?

Or is it because people's code will stop compiling if they start using a newer compiler? But if I use Ogre and my project uses C++11 or C++14, can't I link to code compiled with C++17 or C++23???? I really do not understand that.

Couldn't we restrict only the ".h" files and leave ".cpp" free to use newer features?

[dark_sylinc]

Companies move very slowly. Setting aside MSVC licenses (which used to be a problem in the past, but less so now), migrating to a newer compiler means everyone in a project must move in tandem (all users who can compile, all CI build systems).

It also means that slight behavior changes need to be thoroughly tested and nobody wants to tackle that.

Couldn't we restrict only the ".h" files and leave ".cpp" free to use newer features?

In some cases this is possible but in other cases it produces binary incompatibilities either at link-time or run-time, so it's risky (unless you use a strictly abstracted API, like COM or a C API where the header is completely independent of how the C++ implementation was written, which Ogre and OgreNext do not follow).

I really do not understand why this strong requirement of not using things above C++11 or C++14

From a personal opinion (and I am the one who was deciding stuff for OgreNext), benefits of new C++ features need to outweight their disadvantages.

Disadvantages include:

1. Stack traces are more difficult to debug.

2. Error output from compiler is more difficult to understand.

3. Build time performance is slower.

The last one, I believe is crucial for gamedev, where iteration speed matters a lot. The following program:

Code: Select all

#include <algorithm>
#include <cmath>
#include <cstdio>
#include <memory>
#include <string>

int main()
{
	return 0;
}

Takes different time to compile based on what std is used, regardless of whether it is actually used:

Code: Select all

#
# GCC
#

g++ --version
g++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0

time g++ --std=c++98 main.cpp 
real	0m0,091s
user	0m0,068s
sys	0m0,023s

time g++ --std=c++11 main.cpp 

real	0m0,194s
user	0m0,164s
sys	0m0,030s

time g++ --std=c++17 main.cpp 
real	0m0,306s
user	0m0,284s
sys	0m0,022s

#
# CLANG
#

clang++-12 --version
Ubuntu clang version 12.0.0-3ubuntu1~20.04.5

time clang++-12 --std=c++98 main.cpp 

real	0m0,141s
user	0m0,093s
sys	0m0,047s

time clang++-12 --std=c++11 main.cpp 

real	0m0,219s
user	0m0,191s
sys	0m0,028s

time clang++-12 --std=c++17 main.cpp 

real	0m0,326s
user	0m0,296s
sys	0m0,029s

time clang++-12 --std=c++20 main.cpp 

real	0m0,351s
user	0m0,318s
sys	0m0,026s

You can read more about my stance here.

However as CPUs become faster and projects migrate to a minimum of C++11, it became more difficult to ignore some of the advantages of C++11 features, such as constructors invoking constructors, override and final (virtual), static_assert; which are now used everywhere in OgreNext >= 3.0.

Thanks to Eugene, we also take optional advantage of C++14/17 features such as std::from_chars which is an excellent function. We also take optional advantage of other C++14/17 features like [[deprecated]].

I would use lambdas more often if it weren't for the problem that std::function in a header destroys build-time performance and stack traces on lambdas often become a mess.

Unfortunately C++ versions keep getting bloated instead of becoming more isolated. It would be great to use several actually good functionallity without all the baggage that increases build time so much. Build times impact productivity a lot.

[andrecaldas]

The last one, I believe is crucial for gamedev, where iteration speed matters a lot. The following program:

[...]

Unfortunately C++ versions keep getting bloated instead of becoming more isolated. It would be great to use several actually good functionallity without all the baggage that increases build time so much. Build times impact productivity a lot.

For that I give you C++20: modules.

So, when you ogres decide to push it a little further... almost everything implemented! For gcc, clang, msvc, intel...
https://en.cppreference.com/w/cpp/20

I would love to loop like this:

Code: Select all

  for(const auto& [idx, value]: myMap) {
    ...
  }

But I think that more then anything, I think the BIG problem is that even after you adopt some newer standard, the code will not magically upgrade. So, you will be stuck with all stuff you have implemented yourself instead of using the standard stuff and newer libraries. For example, I think Ogre-Next does not use smart-pointers... I never know if I am supposed to delete things or not. Even though shared_ptr and unique_ptr are C++11. And you cannot use a std::shared_lock, or STL semaphores. When you start adopting C++17 or 20, the code will still not use STL shared mutexes.

But I do understand you.
We have different opinions. And you want to play safe. But the world is changing fast...

[paroj]

Is it because MSVC users do not have the money to buy the new version?

I would argue linux users are even a bigger problem. e.g. RHEL7 (supported until 2028) uses GCC 4.8 as the default compiler. However, full C++14 support only comes with GCC 4.9..

So using certain C++14 features means that Ogre will not be compileable on that platform any more. Or maybe some other embedded platform. So the new features we get by bumping C++ levels better outweight that.

On the other hand some of our dependencies, notably glslang, already use C++17. So it is about balancing whether a sensible feature-set is still available on older platforms.

[andrecaldas]

Is it because MSVC users do not have the money to buy the new version?

I would argue linux users are even a bigger problem. e.g. RHEL7 (supported until 2028) uses GCC 4.8 as the default compiler. However, full C++14 support only comes with GCC 4.9..

What is the "default compiler" in Windows 10?

That is not a problem of linux users. It is a cultural problem. I fail to understand why a game developer would use RHEL. From wikipedia:

The first version of Red Hat Enterprise Linux to bear the name originally came onto the market as "Red Hat Linux Advanced Server". In 2003, Red Hat rebranded Red Hat Linux Advanced Server to "Red Hat Enterprise Linux AS" and added two more variants, Red Hat Enterprise Linux ES and Red Hat Enterprise Linux WS.

Because "debian" is slow in releasing new software versions, people prefer "ubuntu". But debian is not slow. Debian has a very well defined release cycle. The "rock stable" one is called "stable". Other versions have misleading names, like "testing" and "unstable". People did not want "stable". They wanted "testing", but they wanted it to be called "stable". They were using the wrong version! Later, Ubuntu did the same and created the LTS (long term support). Debian should have called their versions something like "rock solid, but born outdated", "stable" and "almost stable".

At the university I work, we had an LTS server. The support ended. Absolutely NOBODY has the courage to upgrade it. The changes would be too radical!!! So, we use an unsupported extremely outdated system that has no updates and not even security updates. All that because culturally we got very scared when there is the LTS. We are afraid of not choosing the "safe choice". So, we prefer to have LTS and never worry about upgrading. 10 years later you realize it was a bad idea. The only warranty you had was that your system would NEVER EVOLVE! I suppose it is just people's mindset.

If you have a critical server that you absolutely do not want to touch, you might have a good reason to stick with some 10 year old system. I do think this is a symptom of a bigger problem, because you are not confident enough to TEST before going into production. It is the engineering motto: DO NOT TOUCH ANYTHING!!! IT IS WORKING!!!

Even if you do not want to use C++20, I see no good reason for sicking with GCC 4.8 when GCC 14.2 is already available. Debian has it. The longer you take to move, the harder it gets! If you cannot because you do not have confidence the process will be smooth, then you are not facing a problem. You are facing the symptoms of other many problems you do have.

So using certain C++14 features means that Ogre will not be compileable on that platform any more. Or maybe some other embedded platform. So the new features we get by bumping C++ levels better outweight that.

It is like saying Ogre will not compile on Windows 10 because it does not come with a C++ compiler. But I understand what you mean. There is no concept of "distro" in windows, so people do not mind installing MSVC. Although RHEL has a "repo", it does not tie you to the official one. You tie yourself because you want to.

On the other hand some of our dependencies, notably glslang, already use C++17. So it is about balancing whether a sensible feature-set is still available on older platforms.

I think this "still available on older platforms" has at least three meanings:

1. Do you think it is important to compile ogre in an old platform?
2. Do you think it is important to compile the app in an old platform?
3. Do you think it is important to run the app in an old platform?

Also, "old platform" has different meanings:
a. Old platform means old computer with old GPU?
b. Old platform means old system installed?
c. Old platform means old compiler installed?

For 3, you can compile in some newer platform (or newer compiler) or even cross-compile. You do not need to compile it in Android to use it in Android. You do not need one executable for each possible windows version. You can compile it in ubuntu and run it in debian. You can compile it in one ubuntu and run it in some other ubuntu. You can even compile it in debian and get it to run in windows! Etc, etc, etc.

For 2, from the point of view of Ogre, it is about the same. But you do not need to worry about the app, just the lib.

For 1, if by "old platform" you mean a system where you can install GCC 14, then you can still work with C++20. For this outdated Ubuntu LTS I told you about, I got the administrator to install some (old) libs and compiled GCC 14 myself. I did this because I wanted to do distributed compilation (distcc).

Disclaimer: All this is the opinion of someone who has never worked for a big company, and has never developed a game.

PS: When I say you, I do not mean "you". I mean the person in this situation.

[paroj]

I fail to understand why a game developer would use RHEL.

RHEL was just an example. Actually I tend to follow the software stack of Ubuntu LTS releases unless someone actually complains.

However, keep in mind that Ogre is not only used for games. E.g. ROS is using it and they are quite dependent on the distro provided Ogre:
https://github.com/ros-visualization/rviz

So, we prefer to have LTS and never worry about upgrading. 10 years later you realize it was a bad idea. The only warranty you had was that your system would NEVER EVOLVE! I suppose it is just people's mindset.

its not about never evolve, but rather about being a library. Ogre is never the project but rather part of a larger project, which does not want to get the C++ level dictated by one of many libraries. If we just go ahead and require C++20, those projects will simply stick to an outdated version of Ogre, which in turn reduces testing for us.

For 3, you can compile in some newer platform (or newer compiler) or even cross-compile. You do not need to compile it in Android to use it in Android. You do not need one executable for each possible windows version. You can compile it in ubuntu and run it in debian. You can compile it in one ubuntu and run it in some other ubuntu. You can even compile it in debian and get it to run in windows! Etc, etc, etc.

I mean this mostly. However things are not as easy as you say there
See https://peps.python.org/pep-0513/#rationale

And this is not theoretical as we do provide ogre-python and thus must adhere to some limits.

Before you complain about manylinux1 being way too old. We actually use manylinux_2_28, I just used referenced the old PEP as it is simpler.