Page 1 of 1

using C++11 for OGRE

Posted: Sat Aug 04, 2012 11:16 am
by Shtuka
VC++ 10/12 and especially gcc >= 4.6.0 already support some C++11 features. What about using them in OGRE?

Re: using C++11 for OGRE

Posted: Sat Aug 04, 2012 11:36 am
by CABAListic
We still have users on VC2005, and 2008 is still in wide-spread use. And support in 2010 is afaik not that great. So for the foreseeable future, using C++11 is not an option, because at the moment we would have to implement separate code paths with a fallback for those platforms/compilers that don't support the particular feature.

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 3:58 pm
by Klaim
Most projects in the same situation than Ogre regarding this point will switch to allow using C++11 in the code starting at a future version, when compilers are all almost equal on the features set and there is a strong interface breaking in the interface of the project.
As Ogre 2.0 will not arrive this year nor the next one (I think), I suggest you consider allowing it for 2.0, but not now, only starting from the point Microsoft provide a compiler that have most of C++11 features.

But I also think this should be discussed again in, say, one year from now. Maybe set a timer? :D

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 4:12 pm
by CABAListic
I'm not sure this really helps us. After all, we do want to start development on 2.0, and we have to base our decision on what we have now. Yes, its release is about two years away, but I am not convinced we'll get good enough coverage on all platforms at that point. Even if we do, companies tend to stick to working tools, so I reckon that e.g. VS2010 will still be in use in two years.

From a pragmatic point of view we can only start using the features when we are sure that the vast majority of our user base uses tools with good C++11 support. And right now, I can't make that prediction, so I'd rather err on the side of caution.

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 5:42 pm
by Cygon
I'm always confused why some shops supposedly make these decisions.

I started Windows development with Visual C++ 5 and apart from Visual Studio .NET, switching has never been a disruptive process. The companies I've worked for or with during the last 15 years all welcomed each new version and switched in the first 3-6 months usually.

The only blocker I could imagine would be off-the-shelf closed source middleware with a C++ interface where the developers have either abandoned the project or aren't providing new releases for another reason.

Personally, I've already dropped support for Visual Studio 2010 in all of my projects. I'm simply sick of Boost shooting my compile times through the roof and bloating my releases with several megabytes of utterly dead code and I feel that any time invested in writing alternative code paths for Visual Studio 2010 would be wasted :)

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 10:38 pm
by Klaim
CABAListic wrote:I'm not sure this really helps us. After all, we do want to start development on 2.0, and we have to base our decision on what we have now. Yes, its release is about two years away, but I am not convinced we'll get good enough coverage on all platforms at that point. Even if we do, companies tend to stick to working tools, so I reckon that e.g. VS2010 will still be in use in two years.
Well I think we cannot guess right now, that's why I think that should be discussed in at least one year from now (or maybe 2 yeah).
From a pragmatic point of view we can only start using the features when we are sure that the vast majority of our user base uses tools with good C++11 support. And right now, I can't make that prediction, so I'd rather err on the side of caution.
Pardon my questionning: I don't know if you have experienced most C++11 features, like on gcc, just to get a feel about the knew features? I'm asking because maybe you know something that I don't know about retro-compatibility issues?

My current understanding (from theory and practice of most first implemented features, using msvc, clang and gcc) is that changing compiler version of C++ and still using your C++03 codebase is not a problem (other than the compiler pointing more potential problems in your code base, but it's very rare), so my experience (I used C++11 coding some commercialized games on IPhone- for dayjob unfortunately) is that C++11 code can be use to write libraries without impacting C++03 client code. C++11 retro compatibility is big. Even when you use C++11 features in interfaces you can still use the interface with C++03.
In fact I suspect a lot of C++ developers to wish to upgrade compilers just because of some specific features that would allow easy refactoring of their codebase (like type traits, variadic templates, tuples, or more straight to the ground: lambdas). Even if they don't use C++11, they can still activate it and see no difference.

Obviously, that would mean that if you want to provide a C++11 library, you still have ton convince users to activate C++11 on gcc and clang... (it's by default and non-deactivable on msvc - I suppose they are confident in the retro compatibility?)

What would be the benefit for Ogre to use C++11?
I can only guess at the moment.
My first guess is that Ogre use a lot of shared pointers, sometimes where something less heavy than shared pointers (ref counting + necessary synchronisation on copy) could be used (like unique_ptr). That could do a speedup, but I don't think it's a bottleneck at the moment.
The conversion to a more data-oriented organization of Ogre 2.0 could benefit a lot from writing some of the types as value semantic instead of entiy semantic (like maybe nodes, but really I'm just saying what gets on top of my head, knowing I'm not a specialist about Ogre). This imply that some types are defined as non-copyable but movable, so C++11 features. Maybe Ogre would benefit from the new scoped allocators, as it uses a lot of the STL? Some listener interfaces could be simplified to just be represented by std::function objects?

Or maybe not. I think there is no way to be sure, without investing.

On a side note, if Visual Studio was up to date compared to clang and gcc, I would have started seriously to contribute to Ogre by proposing some enhancements related to C++11. But as you say, there is no point right now or in the next future. My code base using Ogre need to be compatible with MSVC at the moment, and Ogre too.

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 11:00 pm
by CABAListic
How does interface backwards-compatibility help if people can't compile Ogre because their compiler doesn't support language features that Ogre uses?

In any case, the most immediate area where C++11 support could help would be threading support, particularly in line with the roadmap towards parallelising the core. But given that this is meant to be ready for 2.0, implementation will ideally start this year or early next year, so right now I'd have to decide not to use C++11 for thread support because I can't predict if it's going to be widespread enough at the time of release of 2.0. Once it is, we can review thread support and decide if it's worth it to change to C++11, but right now it's simply not an option.

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 11:23 pm
by Klaim
CABAListic wrote:How does interface backwards-compatibility help if people can't compile Ogre because their compiler doesn't support language features that Ogre uses?
I understand that, what I'm saying is that we obviously cannot know if the users will switch to a compiler upgrade or not, because compatibility should be a no-brainer, but then maybe a lot of people will just stick to their compiler version anyway maybe not, but not for purely technical reasons.
So I agree with you it's not a choice for near future. I'm not giving counter arguments, I just say it's hard to say anything other than "let's discuss it in a few years ok?" right now.
CABAListic wrote: In any case, the most immediate area where C++11 support could help would be threading support, particularly in line with the roadmap towards parallelising the core. But given that this is meant to be ready for 2.0, implementation will ideally start this year or early next year, so right now I'd have to decide not to use C++11 for thread support because I can't predict if it's going to be widespread enough at the time of release of 2.0. Once it is, we can review thread support and decide if it's worth it to change to C++11, but right now it's simply not an option.

I agree with this.

Re: using C++11 for OGRE

Posted: Sun Aug 05, 2012 11:26 pm
by CABAListic
Well, then we are in agreement and simply missed each other's point ;)

Re: using C++11 for OGRE

Posted: Mon Aug 06, 2012 10:39 am
by Cygon
I'm still confused about who these developers running VS2005 and VS2008 in 2012 are and why they won't simply upgrade (and even if they had a valid reason, whether they're not just 1 or 2 people who can keep using Ogre 1.8 until they start their next project).

Would you accept a patch that adds C++11 threads as an option similar to Boost, TBB and POCO?

Despite the planned threading redesign and the fact that I would need to roll my own ReaderWriterLock on top of the std::thread facilities, I could use C++11 thread support right now. For WinRT, Boost is the only threading choice in Ogre that works and - Boost being Boost - it shoots my compile times through the roof and adds megabytes of dead code to my releases. C++11 threads have been a pleasure to use so far and Microsoft's <thread>, <mutex> and <atomic> headers are insanely fast (they don't even include Windows.h).

Re: using C++11 for OGRE

Posted: Mon Aug 06, 2012 10:53 am
by CABAListic
Would you accept a patch that adds C++11 threads as an option similar to Boost, TBB and POCO?
For 1.9, maybe. The plan for the 2.0 threading redesign is to get rid of external threading dependencies altogether, which would also cut the dependency on Boost. It might be that parts of that redesign, particularly our own threading lib, make it into 1.9. In that case your patch would become irrelevant. But it's by no means guaranteed, in which case I'd have no objections to such a patch, as long as it's implemented properly:
  • It needs to properly determine availability of std::thread. In case of MSVC a test for the header should suffice, in case of GCC we may need to test for the appropriate compile flag first. CMake has modules for both tests. These tests should probably go into Dependencies.cmake.
  • It must not change the default thread provider of Boost, so if both Boost and std::thread are detected, Boost is still the default choice.
  • If it is chosen as the thread provider, it must ensure that the proper compile flags are added to CMAKE_CXX_FLAGS to compile Ogre with C++11 support.

Re: using C++11 for OGRE

Posted: Mon Aug 06, 2012 11:04 am
by Klaim
Cygon wrote:I'm still confused about who these developers running VS2005 and VS2008 in 2012 are and why they won't simply upgrade (and even if they had a valid reason, whether they're not just 1 or 2 people who can keep using Ogre 1.8 until they start their next project).
Chaning tools means everybody have to be used to it, means it takes some time, means it cost money, so it's an investment and not all companies, in particular if they are in the middle of a (long term) project would even consider the option. I've been in one for 2.5 years, asking each month to upgrade the old gcc version. The problem they had with this is that they couldn't do it without hierarchichal decisions that had nothing to do with technical ones. The other problem was that they still investigated the switch to better compilers, but the drastique improvement in quality made that a big part of the code base had to be fixed before making it compile on latest gcc (4.5 at the time...). Also, they were late in their schedule (for a very important project that was in dev for years) and the most competent people in C++ were leaving.
In this kind of context, you don't change your tools easily.
Would you accept a patch that adds C++11 threads as an option similar to Boost, TBB and POCO?
CABAlistic said no, they can't until they are sure their user base can compile it. I think in some times from now some polling will be necessary, but frankly, as said before, I don't see it happening before an optimistic 1 year.

However, the nature of DSVC makes it interesting that you can expose your version and improve it through time and features additions until Ogre can accept the changes. So if you have resources to do it, why not? At least it will serve yourself apparently.
Despite the planned threading redesign and the fact that I would need to roll my own ReaderWriterLock on top of the std::thread facilities, I could use C++11 thread support right now. For WinRT, Boost is the only threading choice in Ogre that works and - Boost being Boost - it shoots my compile times through the roof and adds megabytes of dead code to my releases. C++11 threads have been a pleasure to use so far and Microsoft's <thread>, <mutex> and <atomic> headers are insanely fast (they don't even include Windows.h).
Actually I dont' think boost is killing your perfs if you are using only boost.thread (it really depends on what you use in boost). It is written by the same people who made just::thread that is the first implementation of C++11 thread and that is written by the main designer of C++11 thread.
In his book he says that some mutex types (that Ogre use at this moment) are not in the standard so using boost mutex using C++11 guards is the best option if you want to stick with as much C++11 as possible.
My personal belief is that whithout a task scheduler implementation, I don't see how boost or C++11 can greatly help Ogre integrate in other mutlithreaded applications.
I'm currently working on this kind of stuffs and I really think C++11 is very great, but a task scheduler is highly lacking to work with other libraries.

Re: using C++11 for OGRE

Posted: Mon Aug 06, 2012 2:25 pm
by Cygon
Klaim wrote:Actually I dont' think boost is killing your perfs if you are using only boost.thread (it really depends on what you use in boost). It is written by the same people who made just::thread that is the first implementation of C++11 thread and that is written by the main designer of C++11 thread.
I know it is Boost that is killing my build times.

I'm not just guessing, I'm intimately familiar with the internals of Boost.Thread. I have at various times put together trimmed down distributions of Boost (even going so far as to write a tool that invokes cl.exe, scans the compile errors for missing headers and copies those headers into a special include directory to build a minimalized subset of Boost). And I've recently ported Boost.Thread to WinRT. The dependencies are a small catastrophe.

I've got an older version of an open source library I'm maintaining here that still supports Visual Studio 2010 and can switch between Boost.Threads and C++11 threads (compiled on an SSD):
  • Compiling with <mutex>, <condition_variable> header: 1 minute, 5 seconds
  • Compiling with <boost/thread/mutex.hpp> and <boost/thread/condition_variable.hpp> header: 4 minutes, 20 seconds
Before doing that test, I thought I could just post a visualization of Boost's dependency mess to make my point (using this tool: C/C++ #include hierarchy viewer). It generated a nice tree with a dozen or so headers for Visual Studio 2012's <mutex> header, but I had to abort when, after churning for 2 minutes, it failed attempting to save a 157,595 x 3,590 pixel PNG of Boost's <boost/thread/mutex.hpp> header. Maybe that makes a point as well :mrgreen:
Klaim wrote:In his book he says that some mutex types (that Ogre use at this moment) are not in the standard so using boost mutex using C++11 guards is the best option if you want to stick with as much C++11 as possible.
I think the problematic part is boost::shared_mutex (used by Ogre as OGRE_RW_MUTEX, I referred to it as a ReaderWriterLock in my post above). If I find the time to put together a patch I would build such a lock from the C++11 threading primitives.

Re: using C++11 for OGRE

Posted: Mon Aug 06, 2012 3:47 pm
by Klaim
Nice that you measured this, it's good information.

Re: using C++11 for OGRE

Posted: Tue Dec 03, 2013 7:36 pm
by al2950
Well its over a year later, how about now!!!?? :) :?:

**EDIT**
I have not checked particularly thoroughly but are we not using C++11 in Ogre 2.x?

Re: using C++11 for OGRE

Posted: Tue Dec 03, 2013 8:06 pm
by AlexeyKnyshev
I think c++11 is not panacea but maybe we can implement optional c++11 theading backend for Ogre.

Re: using C++11 for OGRE

Posted: Tue Dec 03, 2013 9:26 pm
by Klaim
Until all these conditions are met:
- Ogre 2.x is stabilized;
- Visual Studio fixes some STL issues like chrono not being precise enough;
- Visual Studio fixing their compilers for C++11 features (there are huge bugs);

I don't see how C++11 can be begun to be used in Ogre source code other than in minor implementation files.

On the threading issues, with a bit more experience related to this, I now think that the best strategy for a library
wishing to be a good citizen in an application that might use several threads but you can't guess how they are organized
is to have an interface similar to the "executor" proposal and expose it in it's interface. Then it should be easier to change it to a standard one later (in something like 5+ years....)
or to use a boost or tbb based provided implementations following this interface (which is very simple but allow to build threading tools over it in a similarly simple way).

Anyway, there have been too much huge changes in Ogre right now for the team to give some focus on adding C++11 features in Ogre's interface.
I don't see that happening soon.

Here is a uncomplete unfocused and random though not completely related to this but I often get when working with Ogre: I now got a lot of experience with more "modern" styles (without going into hardcore C++ expert style that Boost sometime exposes) than what Ogre exposes and I've thought several times that some things could be enhanced in Ogre using more RAII related techniques. Not in fundamental things though but on some part of the interface clarity or "ergonomy" (like the type system forbidding me to do something stupid). That being said I didn't experiment see if my instinct would be correct so I'm just thinking that in a future 2.X or 3.X version of Ogre, it would be nice to try to focus more on Ogre's interface. Ogre 2.x focus on speed, which is exactly what it needs now. I hope it will help re-balance efforts later on the design of the interface.

Re: using C++11 for OGRE

Posted: Thu Dec 12, 2013 10:29 am
by TheSHEEEP
I agree with Klaim.

One big thing at a time :)