The roadmap to 1.9 and 2.0

[Wolfmanfx]

Agreed.

But what i mentioned is also mentioned by tuan here http://www.ogre3d.org/forums/viewtopic. ... 91#p453845

[Diamond145]

Parallelizing the scene graph:

Job copies the scenenode state to the stack and does the positioning, rotating, scaling, etc. (doing this, we could make the budd-ify the data with the cache ), passing (probably a copy off the stack) the results in the returned Ogre::WorkQueue::Response.
This could be done one hierarchy level at at time, starting (adding jobs) the next hierarchy level in ResponseHandler::handleResponse. This would avoid locks to individual nodes (don't need locks if we guarantee no write contention (by disallowing writes outside the work set). Plus any single node can only have one parent).

Additionally, we could possibly merge parts of the _findVisableObjects steps into the scene graph jobs.

Problems (supposition though):

• Deep scene graph hierarchies would hurt performance. It would work best for a large amount of children-less nodes attached directly to the root node.
• Combined with above, batching will be touch and go. Perhaps if we had an array of all the nodes (no hierarchy) we could order them such that parents get calc-ed before children. We'd have to batch all a parents children together, which might be problematic depending on the graph's layout (a parent with, like, a hundred children ).
• The AABB update moves opposite to the position/rotation/scale update (child to parent vs parent to child).

I admit I don't know much about the graphical API's (OpenGL, DirectX) and their respective multithreading behaviors, especially compared to you guys, but the above shouldn't (as far as I know) touch the render systems. Additionally, if I'm correct, parts of the above could be merged into Ogre without an external API change.

Thoughts?
Regards,
Richard Diamond

[CABAListic]

As I already said, the current WorkQueue will not be a part of the threading redesign as it's too complicated and uses heavy locking.

As far as handling SceneNode hierarchies goes, I'm not an expert on task-based multithreading, but one possible way that seems promising to me based on my preliminary research goes like this:
- Create tasks for all root SceneNodes and divide them evenly among the available threads.
- Finished tasks dynamically create new tasks for all child nodes on the local thread queue.
- If a thread runs out of work, it steals or requests tasks from other threads.

Obviously, that's only a superficial outlook, and care must be taken that creating tasks is very cheap. It's also not the only possibility. I'm happy for feedback from people who have some experience with such designs, but I'll eventually create a separate topic for threading discussions, so let's not go into details here.

[iFire]

Remember that old code is complicated because it had to fix bugs.

If you throw it away, you'll must find why the locks were there.. That is unless you can prove your design can't use locks (i.e. thorem proving..).

Then at some point.. someone ruins the design and uses locks as patch fixes >.<

Anyways, I remember a chart from http://software.intel.com/en-us/blogs/2 ... askell-01/ showing their speed.

Haskell is a functional language.

[CABAListic]

The current WorkQueue was designed as a general background task utility and could not assume much about the tasks it would be given. Sinbad chose a design that would ensure that a task's consumers would always be available as long as a task is still active. But this requires heavy and complicated locking - the WorkQueue uses reader/writer mutexes and additional event signalling. In fact, our use of threading primitives is so complicated that std::thread doesn't even provide all of the required primitives, and that's part of the reason why it has to go, imho.

Anyway, threading the core is very different to puttings tasks into the background. You can make a lot more assumptions, don't suffer the possibility of the consumer going away in the meantime, but instead you have to do it as fast as possible to reap the benefits. Locking or no, the current WorkQueue was simply not designed for this kind of microtasking, so sticking with it doesn't really help anything.

[iFire]

You replied before I edited:

I remember looking up http://software.intel.com/en-us/blogs/2 ... askell-01/.

They tested a series of concurrency algorithms. I suggest playing with each of them and see how they test when you transfer them to c++.

They also wrote a summary at some time. http://dspace.mit.edu/bitstream/handle/ ... ?sequece=1.

[Cooncat]

What's about CHC++?

[Programpriv]

I may say anything but, write an interface to create GUI in our application would can good.

[th3flyboy]

Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/

[syedhs]

Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/

Yes looks pretty good and simple. Thanks for the link.

[Klaim]

I think it lacks a lot of documentation in the interfaces, and a quick look at it's interface makes me think it need quite a refinement (mostly because of pointers used where there shouldn't).

That said I didn't try it yet. (also this kind of libraries are interesting to me)

[Kojack]

I may say anything but, write an interface to create GUI in our application would can good.

Ogre used to have a gui built in. It was removed years ago, since external libs (there's a HUGE number of them for ogre) do it better. The odds of one being added back in are very unlikely.

[CABAListic]

Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/

Yes, thanks, I already found this library. I'm beginning to believe that it's best to write our own threading library for Ogre, but cpptask is valuable as a reference and as proof that the required effort is not as big as writing our own threadingbuildingblocks

[bstone]

I would also take a look at yaTS. It has Apache License and therefore is not usable as is but as a source of inspiration and working solutions it's very good. The author has put a lot of thinking and expertise into it no doubt.

[masterfalcon]

I thought of something else that we should include for 1.9. Integrating the changes for sample media improvement from here: http://www.ogre3d.org/forums/viewtopic.php?f=3&t=67840

[Kojack]

1.9 or maybe 2.0
Since there's a lot of work involved in the asset overhaul, any incremental changes will bloat the repository and all samples and a heap of wiki pages will break, it might be better to target 2.0.

[RiderV]

I recommend you google on Data Oriented Design.

Pitfalls of Object Oriented Programming
http://harmful.cat-v.org/software/OO_pr ... CAP_09.pdf

Data-Oriented Design (Or Why You Might Be Shooting Yourself in The Foot With OOP)
http://gamesfromwithin.com/data-oriented-design

and finally you must read blog of Bitsquid company! It is very interesting for yours roadmap 2.0.
http://www.bitsquid.se/

Good luck.

[Transporter]

I'll like to point on Cygons work at the zip topic: http://www.ogre3d.org/forums/viewtopic. ... 43#p461631

Maybe this is an useful feature for the next release.

[dtecta]

Intel's tbb comes close, but its license is tainted.

Could you please explain to me how the TBB license is harmful for Ogre? TBB uses GPL v2 with runtime exception. Under these terms the libstdc++ is released. Surely, you must think it's OK to link with libstdc++, otherwise no GCC build would be possible. I am not a lawyer but I see no harm in the license terms of TBB.

Edit: My point was brought up earlier in this thread which I missed at first glance. I see your point on GPL being a scare, but the fact that TBB uses the same terms as libstdc++ to which every C++ application that is build with GCC must link is pretty big reassurance. Commercializing applications that are built with GCC is not debated AFAIK.

[Klaim]

Intel's tbb comes close, but its license is tainted.

Could you please explain to me how the TBB license is harmful for Ogre? TBB uses GPL v2 with runtime exception. Under these terms the libstdc++ is released. Surely, you must think it's OK to link with libstdc++, otherwise no GCC build would be possible. I am not a lawyer but I see no harm in the license terms of TBB.

My 2 cents,

Gino

The problem often mentionned is not that it is not permissive licence, it is that the way it is explained is not as clear as say MIT or BSD licence, so from none-lawyer perception it is suspicious (why not use a similar licence if it is that permissive?). Now, as it is suspicious, it also means that different lawyers might interpret it differently, or that there are possibilities for lawyers to bend the interpretation.

I personally don't know any case where this licence was a problem so I think it is ok to use it for myself without thinking more about it, but it is still a bit suspicious from some of us.

[CABAListic]

@dtecta: The vast majority of companies whose lawyers would take objection to GPL will not be using GCC in the first place. So that doesn't really help. (Unless perhaps they are developing for Mac, too. But Mac is switching to clang, I think.)

[ppClarity]

Edit: My point was brought up earlier in this thread which I missed at first glance. I see your point on GPL being a scare, but the fact that TBB uses the same terms as libstdc++ to which every C++ application that is build with GCC must link is pretty big reassurance. Commercializing applications that are built with GCC is not debated AFAIK.

There are non-GPL alternatives to libstdc++. There's only one TBB.

[lygyue]

As a ogre user and using ogre for years, I think mobile render system like Android and multi thread is most important. I think a few years later, Android will be the most important mobile system. so in my opinion, the two points need to modify in future.

sorry for my poor english.

[dtecta]

There are non-GPL alternatives to libstdc++. There's only one TBB.

With alternatives you mean use a different compiler from a commercial vendor? Well, from that perspective, there is an alternative to the GPL(with exception) TBB and that is the commercial TBB (shipped with Intel compiler and parallel studio).

[Zonder]

I do think the threading should just be abstracted as one threading solution won't necessarily work best on another platform or future platforms. And the internal ogre version should only act single threaded (no licensing problems) then I would say write a plug-in to give a threaded environment which the team back for the Intel platform only. And maybe an ARM plug-in in the future