The roadmap to 1.9 and 2.0

Discussion area about developing with Ogre2 branches (2.1, 2.2 and beyond)
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Wolfmanfx » Fri Jun 15, 2012 1:31 pm

Agreed.

But what i mentioned is also mentioned by tuan here http://www.ogre3d.org/forums/viewtopic. ... 91#p453845
0 x

Diamond145
Gnoblar
Posts: 1
Joined: Tue Jan 20, 2009 7:34 pm

Re: The roadmap to 1.9 and 2.0

Post by Diamond145 » Sun Jun 17, 2012 9:59 pm

Parallelizing the scene graph:

Job copies the scenenode state to the stack and does the positioning, rotating, scaling, etc. (doing this, we could make the budd-ify the data with the cache :P ), passing (probably a copy off the stack) the results in the returned Ogre::WorkQueue::Response.
This could be done one hierarchy level at at time, starting (adding jobs) the next hierarchy level in ResponseHandler::handleResponse. This would avoid locks to individual nodes (don't need locks if we guarantee no write contention (by disallowing writes outside the work set). Plus any single node can only have one parent).

Additionally, we could possibly merge parts of the _findVisableObjects steps into the scene graph jobs.

Problems (supposition though):
  • Deep scene graph hierarchies would hurt performance. It would work best for a large amount of children-less nodes attached directly to the root node.
  • Combined with above, batching will be touch and go. Perhaps if we had an array of all the nodes (no hierarchy) we could order them such that parents get calc-ed before children. We'd have to batch all a parents children together, which might be problematic depending on the graph's layout (a parent with, like, a hundred children :P ).
  • The AABB update moves opposite to the position/rotation/scale update (child to parent vs parent to child).
I admit I don't know much about the graphical API's (OpenGL, DirectX) and their respective multithreading behaviors, especially compared to you guys, but the above shouldn't (as far as I know) touch the render systems. Additionally, if I'm correct, parts of the above could be merged into Ogre without an external API change.

Thoughts?
Regards,
Richard Diamond
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Sun Jun 17, 2012 10:16 pm

As I already said, the current WorkQueue will not be a part of the threading redesign as it's too complicated and uses heavy locking.

As far as handling SceneNode hierarchies goes, I'm not an expert on task-based multithreading, but one possible way that seems promising to me based on my preliminary research goes like this:
- Create tasks for all root SceneNodes and divide them evenly among the available threads.
- Finished tasks dynamically create new tasks for all child nodes on the local thread queue.
- If a thread runs out of work, it steals or requests tasks from other threads.

Obviously, that's only a superficial outlook, and care must be taken that creating tasks is very cheap. It's also not the only possibility. I'm happy for feedback from people who have some experience with such designs, but I'll eventually create a separate topic for threading discussions, so let's not go into details here.
0 x

iFire
Gnoblar
Posts: 24
Joined: Mon Sep 26, 2005 6:39 am

Re: The roadmap to 1.9 and 2.0

Post by iFire » Tue Jun 19, 2012 11:22 am

Remember that old code is complicated because it had to fix bugs.

If you throw it away, you'll must find why the locks were there.. That is unless you can prove your design can't use locks (i.e. thorem proving..).

Then at some point.. someone ruins the design and uses locks as patch fixes >.<

Anyways, I remember a chart from http://software.intel.com/en-us/blogs/2 ... askell-01/ showing their speed.

Haskell is a functional language.
Last edited by iFire on Tue Jun 19, 2012 11:37 am, edited 1 time in total.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Tue Jun 19, 2012 11:34 am

The current WorkQueue was designed as a general background task utility and could not assume much about the tasks it would be given. Sinbad chose a design that would ensure that a task's consumers would always be available as long as a task is still active. But this requires heavy and complicated locking - the WorkQueue uses reader/writer mutexes and additional event signalling. In fact, our use of threading primitives is so complicated that std::thread doesn't even provide all of the required primitives, and that's part of the reason why it has to go, imho.

Anyway, threading the core is very different to puttings tasks into the background. You can make a lot more assumptions, don't suffer the possibility of the consumer going away in the meantime, but instead you have to do it as fast as possible to reap the benefits. Locking or no, the current WorkQueue was simply not designed for this kind of microtasking, so sticking with it doesn't really help anything.
0 x

iFire
Gnoblar
Posts: 24
Joined: Mon Sep 26, 2005 6:39 am

Re: The roadmap to 1.9 and 2.0

Post by iFire » Tue Jun 19, 2012 11:40 am

You replied before I edited:

I remember looking up http://software.intel.com/en-us/blogs/2 ... askell-01/.

They tested a series of concurrency algorithms. I suggest playing with each of them and see how they test when you transfer them to c++.

They also wrote a summary at some time. http://dspace.mit.edu/bitstream/handle/ ... ?sequece=1.
Last edited by iFire on Wed Jun 20, 2012 10:56 pm, edited 1 time in total.
0 x

User avatar
Cooncat
Gnoblar
Posts: 6
Joined: Thu May 03, 2012 12:14 am

Re: The roadmap to 1.9 and 2.0

Post by Cooncat » Tue Jun 19, 2012 1:19 pm

What's about CHC++?
0 x

Programpriv
Gnoblar
Posts: 1
Joined: Thu Jun 21, 2012 9:33 am

Re: The roadmap to 1.9 and 2.0

Post by Programpriv » Thu Jun 21, 2012 9:37 am

I may say anything but, write an interface to create GUI in our application would can good.
0 x

th3flyboy
Gnoblar
Posts: 10
Joined: Sun Aug 16, 2009 3:52 am

Re: The roadmap to 1.9 and 2.0

Post by th3flyboy » Fri Jun 22, 2012 3:44 am

Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/
0 x

User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2702
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 3

Re: The roadmap to 1.9 and 2.0

Post by syedhs » Fri Jun 22, 2012 5:44 am

th3flyboy wrote:Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/
Yes looks pretty good and simple. Thanks for the link.
0 x
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Klaim » Fri Jun 22, 2012 5:49 am

I think it lacks a lot of documentation in the interfaces, and a quick look at it's interface makes me think it need quite a refinement (mostly because of pointers used where there shouldn't).

That said I didn't try it yet. (also this kind of libraries are interesting to me)
Last edited by Klaim on Fri Jun 22, 2012 9:27 am, edited 1 time in total.
0 x

User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7152
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 19

Re: The roadmap to 1.9 and 2.0

Post by Kojack » Fri Jun 22, 2012 6:52 am

I may say anything but, write an interface to create GUI in our application would can good.
Ogre used to have a gui built in. It was removed years ago, since external libs (there's a HUGE number of them for ogre) do it better. The odds of one being added back in are very unlikely.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Fri Jun 22, 2012 9:53 am

th3flyboy wrote:Regarding threading, I feel obligated to link to this library, it's BSD licensed and looks pretty good: http://code.google.com/p/cpptask/
Yes, thanks, I already found this library. I'm beginning to believe that it's best to write our own threading library for Ogre, but cpptask is valuable as a reference and as proof that the required effort is not as big as writing our own threadingbuildingblocks ;)
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Fri Jun 22, 2012 12:04 pm

I would also take a look at yaTS. It has Apache License and therefore is not usable as is but as a source of inspiration and working solutions it's very good. The author has put a lot of thinking and expertise into it no doubt.
0 x

User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
Contact:

Re: The roadmap to 1.9 and 2.0

Post by masterfalcon » Sat Jun 23, 2012 5:41 pm

I thought of something else that we should include for 1.9. Integrating the changes for sample media improvement from here: http://www.ogre3d.org/forums/viewtopic.php?f=3&t=67840
0 x

User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7152
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 19

Re: The roadmap to 1.9 and 2.0

Post by Kojack » Sun Jun 24, 2012 4:36 am

1.9 or maybe 2.0
Since there's a lot of work involved in the asset overhaul, any incremental changes will bloat the repository and all samples and a heap of wiki pages will break, it might be better to target 2.0.
0 x

RiderV
Gnoblar
Posts: 3
Joined: Sun Jun 24, 2012 3:46 pm

Re: The roadmap to 1.9 and 2.0

Post by RiderV » Sun Jun 24, 2012 4:03 pm

I recommend you google on Data Oriented Design.

Pitfalls of Object Oriented Programming
http://harmful.cat-v.org/software/OO_pr ... CAP_09.pdf

Data-Oriented Design (Or Why You Might Be Shooting Yourself in The Foot With OOP)
http://gamesfromwithin.com/data-oriented-design

and finally you must read :) blog of Bitsquid company! It is very interesting for yours roadmap 2.0.
http://www.bitsquid.se/

Good luck.
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany
x 2

Re: The roadmap to 1.9 and 2.0

Post by Transporter » Mon Jun 25, 2012 7:18 pm

I'll like to point on Cygons work at the zip topic: http://www.ogre3d.org/forums/viewtopic. ... 43#p461631

Maybe this is an useful feature for the next release.
0 x

dtecta
Gnoblar
Posts: 14
Joined: Sun Sep 19, 2010 10:49 pm

Re: The roadmap to 1.9 and 2.0

Post by dtecta » Thu Jun 28, 2012 12:01 pm

CABAListic wrote:
Intel's tbb comes close, but its license is tainted.
Could you please explain to me how the TBB license is harmful for Ogre? TBB uses GPL v2 with runtime exception. Under these terms the libstdc++ is released. Surely, you must think it's OK to link with libstdc++, otherwise no GCC build would be possible. I am not a lawyer but I see no harm in the license terms of TBB.

Edit: My point was brought up earlier in this thread which I missed at first glance. I see your point on GPL being a scare, but the fact that TBB uses the same terms as libstdc++ to which every C++ application that is build with GCC must link is pretty big reassurance. Commercializing applications that are built with GCC is not debated AFAIK.
Last edited by dtecta on Thu Jun 28, 2012 12:11 pm, edited 2 times in total.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Klaim » Thu Jun 28, 2012 12:09 pm

dtecta wrote:
CABAListic wrote:
Intel's tbb comes close, but its license is tainted.
Could you please explain to me how the TBB license is harmful for Ogre? TBB uses GPL v2 with runtime exception. Under these terms the libstdc++ is released. Surely, you must think it's OK to link with libstdc++, otherwise no GCC build would be possible. I am not a lawyer but I see no harm in the license terms of TBB.

My 2 cents,

Gino

The problem often mentionned is not that it is not permissive licence, it is that the way it is explained is not as clear as say MIT or BSD licence, so from none-lawyer perception it is suspicious (why not use a similar licence if it is that permissive?). Now, as it is suspicious, it also means that different lawyers might interpret it differently, or that there are possibilities for lawyers to bend the interpretation.

I personally don't know any case where this licence was a problem so I think it is ok to use it for myself without thinking more about it, but it is still a bit suspicious from some of us.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Thu Jun 28, 2012 12:27 pm

@dtecta: The vast majority of companies whose lawyers would take objection to GPL will not be using GCC in the first place. So that doesn't really help. (Unless perhaps they are developing for Mac, too. But Mac is switching to clang, I think.)
0 x

ppClarity
Gremlin
Posts: 194
Joined: Sat Sep 02, 2006 12:27 am

Re: The roadmap to 1.9 and 2.0

Post by ppClarity » Thu Jun 28, 2012 5:06 pm

dtecta wrote:Edit: My point was brought up earlier in this thread which I missed at first glance. I see your point on GPL being a scare, but the fact that TBB uses the same terms as libstdc++ to which every C++ application that is build with GCC must link is pretty big reassurance. Commercializing applications that are built with GCC is not debated AFAIK.
There are non-GPL alternatives to libstdc++. There's only one TBB.
0 x

lygyue
Greenskin
Posts: 122
Joined: Wed Nov 29, 2006 4:07 pm

Re: The roadmap to 1.9 and 2.0

Post by lygyue » Fri Jun 29, 2012 5:16 am

As a ogre user and using ogre for years, I think mobile render system like Android and multi thread is most important. I think a few years later, Android will be the most important mobile system. so in my opinion, the two points need to modify in future.

sorry for my poor english.
0 x

dtecta
Gnoblar
Posts: 14
Joined: Sun Sep 19, 2010 10:49 pm

Re: The roadmap to 1.9 and 2.0

Post by dtecta » Fri Jun 29, 2012 12:43 pm

ppClarity wrote: There are non-GPL alternatives to libstdc++. There's only one TBB.
With alternatives you mean use a different compiler from a commercial vendor? Well, from that perspective, there is an alternative to the GPL(with exception) TBB and that is the commercial TBB (shipped with Intel compiler and parallel studio).
0 x

User avatar
Zonder
Ogre Magi
Posts: 1133
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 22

Re: The roadmap to 1.9 and 2.0

Post by Zonder » Fri Jun 29, 2012 2:32 pm

I do think the threading should just be abstracted as one threading solution won't necessarily work best on another platform or future platforms. And the internal ogre version should only act single threaded (no licensing problems) then I would say write a plug-in to give a threaded environment which the team back for the Intel platform only. And maybe an ARM plug-in in the future
0 x
There are 10 types of people in the world: Those who understand binary, and those who don't...

Post Reply