The roadmap to 1.9 and 2.0

Design / architecture / roadmap discussions related to future of Ogre3D (version 2.0 and above)
bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Fri Jun 29, 2012 5:52 pm

I hardly see a plug-in dealing with the threading options. Not feasible.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Fri Jun 29, 2012 6:05 pm

Well, somewhat feasible, perhaps. We do want to support a pluggable architecture so that people can integrate Ogre's tasks into their own threading system. Basically, this means that in the Ogre core the task architecture must be present, but not necessarily a multi-threaded implementation of a task manager. Ogre could initialise with a single-threaded default task manager that can be replaced at runtime. Then you could have a plugin that loads the multi-threaded implementation.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Fri Jun 29, 2012 6:43 pm

I'll stick with being realistic. Besides having a task based architecture with a single threaded model as the primary in mind... that sounds like some serious overhead to me.
0 x

th3flyboy
Gnoblar
Posts: 10
Joined: Sun Aug 16, 2009 3:52 am

Re: The roadmap to 1.9 and 2.0

Post by th3flyboy » Sat Jun 30, 2012 7:58 pm

CABAListic wrote:Well, somewhat feasible, perhaps. We do want to support a pluggable architecture so that people can integrate Ogre's tasks into their own threading system. Basically, this means that in the Ogre core the task architecture must be present, but not necessarily a multi-threaded implementation of a task manager. Ogre could initialise with a single-threaded default task manager that can be replaced at runtime. Then you could have a plugin that loads the multi-threaded implementation.
If you're thinking how I think you're thinking, you are implying create a threading system similar to how the existing pluggable render system setup is right now, am I correct?

Also, if I may suggest an option, would it be possible to move in 1.9 or 2.0 to a more data orientated approach for how data is handled? I would also suggest moving properties of data types, such as int, unsigned int, and so on, to a class that can be used as a template for those types in a data orientated fashion, acting as a interface to a data orientated template for that type of data.
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Sat Jun 30, 2012 8:40 pm

th3flyboy wrote: If you're thinking how I think you're thinking, you are implying create a threading system similar to how the existing pluggable render system setup is right now, am I correct?
Maybe not quite like that, but I realise that some people already have threaded task systems in place in their engines and want to integrate Ogre's tasks into their own solutions instead of having it run separately. The easiest solution appears to be to let the task manager be exchanged for a custom solution. Assuming, of course, that gives better results than just running Ogre's system in parallel and let the OS worry about scheduling.
Also, if I may suggest an option, would it be possible to move in 1.9 or 2.0 to a more data orientated approach for how data is handled? I would also suggest moving properties of data types, such as int, unsigned int, and so on, to a class that can be used as a template for those types in a data orientated fashion, acting as a interface to a data orientated template for that type of data.
There have already been ideas here to do this for SceneNodes and possibly other stuff. However, we cannot really afford to rebuild Ogre from the grounds up, so any such changes can only happen gradually and be focussed on performance critical interna. SceneNodes are a good candidate and may well see some changes in that regard.
0 x

User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Re: The roadmap to 1.9 and 2.0

Post by Mako_energy » Sun Jul 08, 2012 7:05 am

I brought this up earlier in the thread, but sadly haven't seen any talk about it. Is there any plans in these refactors to allow scene objects to be able to be moved from one SceneManager to another seemlessly(without destruction)? How complicated would this be to achieve?
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Sun Jul 08, 2012 8:42 am

I believe that was one of the central issues in sinbad's original plans for the scene manager redesign - basically allowing for multiple scene managers to cooperate seamlessly which requires that entities can be moved between them.

I can't currently say if we can realise this feature for 2.0. At the moment, entities are tied to their creating scene manager, but I don't know how deep the coupling goes. That's one of the things that we would need to find out in the redesign planning stage.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Klaim » Sun Jul 08, 2012 9:54 am

Wouldn't having a separate factory for the scene nodes (with a templated creation function for scene-specific nodes) that would be used by all scene managers be a good solution?

From what I understand, the problem is mostly to be able to keep a pointer to a specific node valid even if moved around in scenes. So it would solve the problem.

The factory would organize data in an optimized way to keep scene nodes from the same scene together, but would put nodes that would have been created with a special hint saying this node will move between scenes in a separate memory pool or something. If the user wants all nodes to be movable between scenes, they would be close in memory.


Another way would be to have scene nodes not be objects but just identifiers, like in (id-based) component systems in some games.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Sun Jul 08, 2012 11:07 am

The coupling is not deep: the node ownership and the ability to reference nodes by their names are the only road blocks here. As it's been suggested above a node pool could handle the ownership. The name<->node relationship is easy to maintain by keeping the respective map where it was - in SM. But I would make it optional, e.g. disabling it would never populate and reference the map to save some memory and make adding nodes to SM a bit faster. The nodes referenced by SM could be kept in a vector making scene walking more cache friendly (a tiny bit but still).
0 x

User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7142
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 9

Re: The roadmap to 1.9 and 2.0

Post by Kojack » Sun Jul 08, 2012 1:56 pm

Moving Entities between scene managers should be pretty easy, I don't believe the managers do anything specific to them.
Scene Nodes are a bit different, each scene manager implements a custom version of the scene node, these are not compatible and can't be moved between managers (you'd need to make a new one and copy the transforms into it). These custom scene nodes have extra data required for bsp and octree scenes.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Sun Jul 08, 2012 2:05 pm

Good point. I was talking about instances of one and the same scene manager class of course. Interoperability of different SM's is a whole other story and I think it's something that should fall on the shoulders of those who need it.
0 x

User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 4

Re: The roadmap to 1.9 and 2.0

Post by Mako_energy » Sun Jul 08, 2012 10:40 pm

I don't know about everyone else's use cases, but I don't think copying transform data and making another scene node, then detach/reattach a moveable object is unreasonable. Transform data is easy to understand and copy. Mesh, material, and animation data is a bit more complicated and is what I wouldn't want to duplicate, so as long as that kind of stuff is avoided I would be overjoyed.

If you really want to go crazy with making this easier, one thing that could be done is adding an additional construction function to the scenemanager base class for scenenodes, where it accepts a scenenode pointer and automatically copies all the relevant shared data between the implementations. So once the user gets it, it's already good to go.
0 x

User avatar
Chris Jones
Lich
Posts: 1742
Joined: Tue Apr 05, 2005 1:11 pm
Location: Gosport, South England

Re: The roadmap to 1.9 and 2.0

Post by Chris Jones » Mon Jul 09, 2012 9:41 am

I just thought i'd throw an idea out about multiple scene managers and the fact that they have specific scene node implementations. I don't know how well it will work in large applications (i used it in my degree's final year project but that was very small games).

Basically split out the spatial/occlusion part so that you never inherit from SceneNode, all it does is hold the transforms and the child scene nodes/objects. These scene nodes can then be referenced by a spatial manager that has a specific implementation (octree etc.). That way scene nodes can easily be shared/moved. Callbacks/events or whatever can be used to notify the spatial "owner" that it has moved for example so it can be moved between octree nodes (or whatever method you want to use to do this notification).

This was a bit rushed so i don't know if i've explained it well, it just seems that the scene node hierarchy/transforms can be separated out from the method of working out what is visible or not which would make it easier to support multiple implementations. It also won't ruin references to scene nodes that the user has stored. What do you think?
0 x

drwbns
Orc Shaman
Posts: 777
Joined: Mon Jan 18, 2010 6:06 pm
Location: Costa Mesa, California

Re: The roadmap to 1.9 and 2.0

Post by drwbns » Mon Jul 09, 2012 6:11 pm

I think there should be a wiki for this :)
0 x

Revelation
Gnoblar
Posts: 3
Joined: Tue Aug 14, 2012 4:41 pm

Re: The roadmap to 1.9 and 2.0

Post by Revelation » Tue Aug 14, 2012 4:50 pm

I know I'm about to make an idiot of myself. *sigh* Here we go anyways! Is there support planned for an x64 build as yet, or is it something we are just expected to recompile ourselves from source? I suspect there is a general performance improvement to be had there, although I'm unsure as to whether the source needs to be recompiled pointing to a different Directx API. I suspect to gain any distinct advantage, the code will need to be re-optimized for the 64-bit environment as well, so maybe my daft question isn't totally merit-less?
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Klaim » Tue Aug 14, 2012 5:07 pm

AFAIK you can already compile Ogre in 64 bit, but you have to make CMake generate it, obviously (see the choice of generator).

So there will certainly not be any distribution in binaries of x64 version before it's really everywhere.

Another thing to point: it looks like the performance side is totally dependent on your use. There is apparently no general rule about this, so you have to compare in your case.
0 x

Revelation
Gnoblar
Posts: 3
Joined: Tue Aug 14, 2012 4:41 pm

Re: The roadmap to 1.9 and 2.0

Post by Revelation » Wed Aug 15, 2012 7:48 pm

Klaim wrote:AFAIK you can already compile Ogre in 64 bit, but you have to make CMake generate it, obviously (see the choice of generator).

So there will certainly not be any distribution in binaries of x64 version before it's really everywhere.

Another thing to point: it looks like the performance side is totally dependent on your use. There is apparently no general rule about this, so you have to compare in your case.
I appreciate what you are saying, but isn't that sort of the backwards way of doing it?

I mean, from a developers point of view (my own) I want to jump right into a 64-bit dev environment... not worry about some bull##tags about hitting a 1GB data limit and make some ##taggingly decent software. I understand that there are situations that a 64-bit stack will advantage devs, and in other circumstances not benefit devs.... but I fail to see many use-case scenarios where having a 64-bit stack immediately available disadvantages 32-devs since there is a distinct mutual exclusion going on... and why limit the playing field to the lowest common denominator... I mean... there aren't a whole bunch of 16-bit devs up in arms about the lack of Win3.11 support....

Sorry, thats not meant as a rant at you, thats just a rant in general about *not* providing the most premium dev platforms by default. That said, I understand what you're saying, I'm probably in the minority. But, its always the minority jerks like me that force pricks like the devs to make more radical software, and I loves you all.
0 x

bstone
OGRE Expert User
OGRE Expert User
Posts: 1920
Joined: Sun Feb 19, 2012 9:24 pm
Location: Russia

Re: The roadmap to 1.9 and 2.0

Post by bstone » Wed Aug 15, 2012 7:55 pm

Compiling the x64 version would have taken about the same time as writing that rant :D (I know you've been carrying those dim thoughts for hours!)
0 x

CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
Contact:

Re: The roadmap to 1.9 and 2.0

Post by CABAListic » Wed Aug 15, 2012 7:57 pm

Our stance is that the provided prebuilt SDKs are only meant as a way to get introduced to Ogre quickly. Anyone doing serious Ogre development will, at some point, want to build the library themselves, if only for the full Visual Studio debug symbols we don't ship due to size. As such, providing the lowest common denominator actually makes sense because building these SDKs takes time, so reducing the number we have to build is a plus.
0 x

Revelation
Gnoblar
Posts: 3
Joined: Tue Aug 14, 2012 4:41 pm

Re: The roadmap to 1.9 and 2.0

Post by Revelation » Wed Aug 15, 2012 8:11 pm

CABAListic wrote:Our stance is that the provided prebuilt SDKs are only meant as a way to get introduced to Ogre quickly. Anyone doing serious Ogre development will, at some point, want to build the library themselves, if only for the full Visual Studio debug symbols we don't ship due to size. As such, providing the lowest common denominator actually makes sense because building these SDKs takes time, so reducing the number we have to build is a plus.
Yep, I guess thats a fair enough point of view. From a personal view point, I'd like to see these available by default, but I'll happily recompile myself if its simply not feasible due to time and/or bandwidth and/or cycle constraints. Thanks for your input all.

To tha Devs: From what I threw at Ogre's engine about a year ago (having a CSE degree, but never bothering to push data at a naked 3D engine before) I was very impressed, particularly with the quality of the tutorials and easy of approachability. Almost any idiot could program for this thing!! I think thats where Ogre really shines, despite its technologically advanced engine which should be able to stand on its own merits. If you need more volunteers to devote wiki time, say the word on your front page... in the mean-time, please keep it as accessible has it has been, or even more so, so that others are easily able to jump into a crazy-easy game builds.

That is all.
0 x

User avatar
Zonder
Gargoyle
Posts: 1098
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 9

Re: The roadmap to 1.9 and 2.0

Post by Zonder » Fri Aug 17, 2012 12:03 am

CABAListic wrote:Our stance is that the provided prebuilt SDKs are only meant as a way to get introduced to Ogre quickly. Anyone doing serious Ogre development will, at some point, want to build the library themselves, if only for the full Visual Studio debug symbols we don't ship due to size. As such, providing the lowest common denominator actually makes sense because building these SDKs takes time, so reducing the number we have to build is a plus.
I remember my first source build took 12 hours :S (long time ago)

But providing 64bit would maybe get people like these http://www.ogre3d.org/forums/viewtopic.php?f=11&t=70299 to make plugins for both, without a NDA.
0 x
There are 10 types of people in the world: Those who understand binary, and those who don't...
My Blog - http://www.This post is suspected as SPAM! If you feel otherwise contact a moderator.

User avatar
Mind Calamity
Ogre Magi
Posts: 1255
Joined: Sat Dec 25, 2010 2:55 pm
Location: Macedonia

Re: The roadmap to 1.9 and 2.0

Post by Mind Calamity » Fri Aug 17, 2012 6:38 am

Zonder, OGRE 1.8 builds in about 5-10 minutes max with Unity building enabled.
0 x
BitBucket username changed to iboshkov (from MindCalamity)
Do you need help? What have you tried?
- xavier
---------------------
HkOgre - a Havok Integration for OGRE | Simple SSAO | My Blog | My YouTube | My DeviantArt

User avatar
Zonder
Gargoyle
Posts: 1098
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 9

Re: The roadmap to 1.9 and 2.0

Post by Zonder » Fri Aug 17, 2012 7:32 am

Mind Calamity wrote:Zonder, OGRE 1.8 builds in about 5-10 minutes max with Unity building enabled.
it was 8 years ago I think :)
0 x
There are 10 types of people in the world: Those who understand binary, and those who don't...
My Blog - http://www.This post is suspected as SPAM! If you feel otherwise contact a moderator.

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: The roadmap to 1.9 and 2.0

Post by Transporter » Fri Aug 17, 2012 8:14 am

Mind Calamity wrote:Zonder, OGRE 1.8 builds in about 5-10 minutes max with Unity building enabled.
I need 80 minutes to build all configurations (54 projects: Debug, MinSizeRel, Release, RelWithDebInfo) for x86 and amd64 without unity build. I also created the doc and install the files in that time.

I have a new point for the roadmap: Code cleanup
It's hard to detect errors building ogre (and ogre applications) because there are thousands of warnings. It would be nice if they could be solved or disabled via cmake.
1>c:\ogre\ogremain\include\OgreMesh.h(309): warning C4251: 'Ogre::Mesh::sharedBlendIndexToBoneIndexMap' : class 'std::vector<_Ty,_Ax>' needs to have dll-interface to be used by clients of class 'Ogre::Mesh'
1> with
1> [
1> _Ty=Ogre::ushort,
1> _Ax=Ogre::STLAllocator<Ogre::ushort,Ogre::GeneralAllocPolicy>
1> ]

1>c:\ogre\ogremain\include\OgreHeaderSuffix.h(34): warning C4193: #pragma warning(pop) : no matching '#pragma warning(push)'

1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::ResourceGroupManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::ResourceGroupManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::ResourceGroupManager
1> ]

1>c:\ogre\ogremain\include\asm_math.h(183): warning C4244: 'return' : conversion from 'double' to 'float', possible loss of data
1>c:\ogre\ogremain\include\asm_math.h(213): warning C4244: 'return' : conversion from 'double' to 'float', possible loss of data
1>c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\utility(163): warning C4244: 'initializing' : conversion from 'int' to 'Ogre::Real', possible loss of data
1> c:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\utility(247) : see reference to function template instantiation 'std::_Pair_base<_Ty1,_Ty2>::_Pair_base<_Ty,int>(_Other1 &&,_Other2 &&)' being compiled
1> with
1> [
1> _Ty1=bool,
1> _Ty2=Ogre::Real,
1> _Ty=bool,
1> _Other1=bool,
1> _Other2=int
1> ]
1> c:\ogre\ogremain\\src\OgreMath.cpp(346) : see reference to function template instantiation 'std::pair<_Ty1,_Ty2>::pair<bool,int>(_Other1 &&,_Other2 &&)' being compiled
1> with
1> [
1> _Ty1=bool,
1> _Ty2=Ogre::Real,
1> _Other1=bool,
1> _Other2=int
1> ]

1>c:\ogre\ogremain\src\OgreProgressiveMesh.cpp(1795): warning C4267: 'initializing' : conversion from 'size_t' to 'unsigned int', possible loss of data
1>c:\ogre\ogremain\src\OgreProgressiveMesh.cpp(1802): warning C4267: 'argument' : conversion from 'size_t' to 'unsigned int', possible loss of data
1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::LodStrategyManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::LodStrategyManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::LodStrategyManager
1> ]
1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::ResourceGroupManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::ResourceGroupManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::ResourceGroupManager
1> ]
1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::MeshManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::MeshManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::MeshManager
1> ]
1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::HardwareBufferManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::HardwareBufferManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::HardwareBufferManager
1> ]
1>c:\ogre\ogremain\include\OgreSingleton.h(94): warning C4661: 'Ogre::LogManager *Ogre::Singleton<T>::msSingleton' : no suitable definition provided for explicit template instantiation request
1> with
1> [
1> T=Ogre::LogManager
1> ]
1> c:\ogre\ogremain\include\OgreSingleton.h(75) : see declaration of 'Ogre::Singleton<T>::msSingleton'
1> with
1> [
1> T=Ogre::LogManager
1> ]

3>c:\ogre\ogremain\include\OgreScriptTranslator.h(257): warning C4275: non dll-interface class 'Ogre::ScriptTranslatorManager' used as base for dll-interface class 'Ogre::BuiltinScriptTranslatorManager'
3> c:\ogre\ogremain\include\OgreScriptTranslator.h(100) : see declaration of 'Ogre::ScriptTranslatorManager'
3> c:\ogre\ogremain\include\OgreScriptTranslator.h(256) : see declaration of 'Ogre::BuiltinScriptTranslatorManager'
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: The roadmap to 1.9 and 2.0

Post by Klaim » Fri Aug 17, 2012 8:35 am

Transporter wrote: It's hard to detect errors building ogre (and ogre applications) because there are thousands of warnings. It would be nice if they could be solved or disabled via cmake.
If I remember correctly this have been discussed several times and the main argument was that some compiler warnings are different between compilers and when you fix one it add a warning to the other or something.

I don't know if it's a solvable problem. I guess it is because most warning I see are conversion warnings.
0 x

Post Reply