Future of threading in Ogre

[Klaim]

I don't know what all the fuss is about boost and std::atomic -- ogre has had atomic operations for 5 years now already...

I don't know about implementation details but maybe people around here just have more confidence into boost atomic because it's clearly more tested and have more eyes on the code than Ogre one? Also boost atomic have evolved a lot but I dont know the reasons exactly. Ogre don't reinvent containers but still reinvent atomics, which is apparently hard to get right.

[sparkprime]

If a portable set of operations were available in every compiler with no issue (like the situation with containers) then sure I wouldn't reinvent them. But they're not -- you're forced to either roll your own or drag in a library. I wouldn't say they are particularly hard to get right either. You just need to get at the low level cpu operation with the least amount of crap in between.

[Klaim]

If a portable set of operations were available in every compiler with no issue (like the situation with containers) then sure I wouldn't reinvent them. But they're not -- you're forced to either roll your own or drag in a library. I wouldn't say they are particularly hard to get right either. You just need to get at the low level cpu operation with the least amount of crap in between.

I wouldn't comment on the "get it right" point, I'm just pointing the fact that apparently depending on the architecture the best implementation can be very different, but I'm not an expert.

However, your assertion about portable set of operations is only correct in C++03. There IS a std::atomic in c++11 standard library (and std::thread but that's beside the point), so that would be a solution IF Ogre internal code was allowed to use C++11. I understand it will not happen soon, for good reasons, so I'm not suggesting switching to c++11, but it's important to note that info and keep it in mind.

[ricardog]

As I said in another post, I think OpenMP might be a nice cross-platform solution. It doesn't require a separate package such as Boost or TBB and, unlike C++11 threads, it's available on older compilers.

[Transporter]

However, your assertion about portable set of operations is only correct in C++03. There IS a std::atomic in c++11 standard library (and std::thread but that's beside the point), so that would be a solution IF Ogre internal code was allowed to use C++11. I understand it will not happen soon, for good reasons, so I'm not suggesting switching to c++11, but it's important to note that info and keep it in mind.

Maybe it's possible to create an other threading provider based on C++11 next to boost and TBB. Which can be used with appropriate available compilers.

[Klaim]

However, your assertion about portable set of operations is only correct in C++03. There IS a std::atomic in c++11 standard library (and std::thread but that's beside the point), so that would be a solution IF Ogre internal code was allowed to use C++11. I understand it will not happen soon, for good reasons, so I'm not suggesting switching to c++11, but it's important to note that info and keep it in mind.

Maybe it's possible to create an other threading provider based on C++11 next to boost and TBB. Which can be used with appropriate available compilers.

I already tried that and reported in a thread somewhere, and I was not the only one it seems. Then main problem was that if you keep the current way of using mutexes then there is a lack of shared_mutex in the C++11 standard library and some other constructs available in boost but not in c++11 (nor c++14, but maybe in future TS that should come in the same time).

Anyway, re-organizing completely Ogre's concurrency might allow use of C++11 but I suspect it will not be enough anyway if you want to use higher abstractions than threads and mutexes (and I think concurrent maps and queues would be just perfect for streaming resource system without locking). Atomics are find though, it's best to use the ones provided by compilers than the one with boost for example.
Also beware of the c++11 thread library current implementations: for example the current (and apparently next version) VS2012 (including the CTP) have a latency problem with it's chrono implementation which makes the highest performance clocks have a 8ms minimum tick. This impact all library tools which rely on time including condition_variable, wait_for member functions of futures etc. It makes basically impracticable high performance threading with this implementation.
I discovered that by implementing a game-specific engine and the solution was to use boost because it currently is more efficient (but I expect it will be different with years) and it provides some experimental high level constructs likely to go to the standard as it is just implementing new proposals. Also, tbb for containers and task executor. So far it worked well for me but it's a combination that is heterogenous:

- tbb provides higher level constructs but don't provide standard-like abstraction;
- boost don't have concurrent containers other than lockfree queues (Boost.LockFree) but have all the rest and a bit more ( like boost::future<>.then( []{ do_something_else(); } ) );
- c++11 don't privide high level constructs, c++14 adds shared_mutex if I remember correctly but it seems that normal mutexes should be used instead of shared_mutex until you have very high read counts compared to write counts.

All provide atomics.

(that's just a brain dump of what I know so far using first c++11 only then switched totally to boost+tbb)

[sparkprime]

for example the current (and apparently next version) VS2012 (including the CTP) have a latency problem with it's chrono implementation which makes the highest performance clocks have a 8ms minimum tick. This impact all library tools which rely on time including condition_variable, wait_for member functions of futures etc. It makes basically impracticable high performance threading with this implementation.

[Klaim]

for example the current (and apparently next version) VS2012 (including the CTP) have a latency problem with it's chrono implementation which makes the highest performance clocks have a 8ms minimum tick. This impact all library tools which rely on time including condition_variable, wait_for member functions of futures etc. It makes basically impracticable high performance threading with this implementation.

Yeah I got the same reactions when I discovered this. See: https://connect.microsoft.com/VisualStu ... ls/719443/#
After reading STL's comment (the guy maintaining the standard library for VS), I asked him some details by email:

> I suppose it is still deferred to post-VC12, which is unfortunate.

Yeah, I just don't have time to rewrite high_resolution_clock (and steady_clock) now.

> My question is: does the lack of precision of the high precision clock in VC implementation impact
> the other threading facilities related to time, like std::condition_variable::wait_for() ?

We're eventually calling GetSystemTimeAsFileTime there, so it looks like it's affected. I've added a note to the bug.

Thanks,
STL

That was on 25th of March this year.

I don't have hope for this to be fixed for VS2013 so I rely on Boost now, which indeed have very high precision with the high_resolution_clock because it uses QueryPerformanceThing() functions.

[Transporter]

I don't have hope for this to be fixed for VS2013 so I rely on Boost now, which indeed have very high precision with the high_resolution_clock because it uses QueryPerformanceThing() functions.

This are bad news, but I belive you are right. In this case thre should be a redesign of Ogre's threading structure (see http://www.ogre3d.org/forums/viewtopic.php?f=3&t=78190).

[Grognard]

On this subject, two things I'd like to have in ogre or at least just get a pointer to an open source place to grab the code.

1. Get the id of the current thread in cross platform fashion.

2. Get the CPU count in cross platform fashion.

I am sure some libs like boost must have this but I don't want to install any big library. Maybe someone knows where it is in a lib like that to just rip it out?

[Klaim]

Both information are available through C++11 standard library and boost.thread. See http://en.cppreference.com/w/cpp/thread ... ive_handle and http://en.cppreference.com/w/cpp/thread ... oncurrency

[Grognard]

Thanks, Klaim.