Automatic dependencies management with ExternalProject

What it says on the tin: a place to discuss proposed new features.
Post Reply
User avatar
Mikachu
Gnoll
Posts: 603
Joined: Thu Jul 28, 2005 4:11 pm
Location: Nice, France

Automatic dependencies management with ExternalProject

Post by Mikachu » Thu Nov 14, 2013 4:33 pm

Hi,

From it's 2.8 version, CMake supports the ExternalProject_Add command, which enables a project to automatically download/update/build its dependencies, without any manual intervention.
I think it could make building Ogre even easier and more straightforward... (of course, it should be possible to disable it, in case dependencies are already available on the PC).

I always considered C++'s dependency management rather tedious, I guess this feature could improve it (not as systematic as maven in the java world, of course, but still an improvement)

More documentation can be found here

Of course, if I find some time, I'll try to do this myself, starting with my own library...
0 x
OgreProcedural - Procedural Geometry for Ogre3D

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Thu Nov 14, 2013 11:30 pm

If the CMake version used by boost is upgraded, I suggest also adding include and definition informations so that using the Ogre targets implicitely generate include directory and compilation definition for the user. See target_include_directories and target_compile_definitions commands in the recent documentation.
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: Automatic dependencies management with ExternalProject

Post by Transporter » Fri Nov 15, 2013 10:35 am

This is an interesting option. But it's not usable for Ogre now, because the dependencies are using different build systems and not CMake.
0 x

User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 1
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Wolfmanfx » Fri Nov 15, 2013 10:36 am

Deps using CMake
0 x

User avatar
Mikachu
Gnoll
Posts: 603
Joined: Thu Jul 28, 2005 4:11 pm
Location: Nice, France

Re: Automatic dependencies management with ExternalProject

Post by Mikachu » Fri Nov 15, 2013 11:01 am

Transporter wrote:This is an interesting option. But it's not usable for Ogre now, because the dependencies are using different build systems and not CMake.
If I understand correctly, it could work with non-cmaked dependencies.

For example, I read about a 'addBoost.cmake' macro, which downloads the latest boost version and compiles it (never tried it, though)
0 x
OgreProcedural - Procedural Geometry for Ogre3D

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Fri Nov 15, 2013 2:51 pm

Transporter wrote:This is an interesting option. But it's not usable for Ogre now, because the dependencies are using different build systems and not CMake.
I build the dependencies using CMake. Maybe you don't know that there is a repository with all the dependencies gathered together and projects defined through CMake?
0 x

User avatar
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
Location: Germany
x 2

Re: Automatic dependencies management with ExternalProject

Post by scrawl » Fri Nov 15, 2013 2:57 pm

So this is like a primitive package manager? Maybe interesting on windows. On anything else, native package managers + pkgconfig should be far superior :)
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: Automatic dependencies management with ExternalProject

Post by Transporter » Fri Nov 15, 2013 11:08 pm

Klaim wrote:
Transporter wrote:This is an interesting option. But it's not usable for Ogre now, because the dependencies are using different build systems and not CMake.
I build the dependencies using CMake. Maybe you don't know that there is a repository with all the dependencies gathered together and projects defined through CMake?
I know that, but I don't want to use it, because they are not up to date.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Sat Nov 16, 2013 12:03 am

But that's your problem then :D I understand that some want to update the dependencies of their dependencies but personally I don't have time to do it. Also, Ogre is not tested against other versions of these dependencies, so I don't want to upgrade them. Finally, personally, except OIS, I don't use directly these Ogre dependencies, so for most of us I guess it's ok.

By the way, cmake have ways to bind non-cmake projects to cmake targets, so you can use (some of) the functions to associate compiler definitions and include directories.
That's what I do with TBB (which don't provide CMake files yet - I wanted to provide them some but didn't find the time in the end).
0 x

User avatar
c6burns
Beholder
Posts: 1511
Joined: Fri Feb 22, 2013 4:44 am
Location: Deep behind enemy lines

Re: Automatic dependencies management with ExternalProject

Post by c6burns » Sat Nov 16, 2013 12:13 am

I feel like the Ogre team does enough to ensure ease of use that I don't need them to bother figuring out how to build boost for me on every possible environment. Also for any users who actually struggle building dependencies they bundle everything together regularly into SDK downloads. Plus the dependencies are regularly built as a seperate download, so I think everything is top notch in the convenience department.
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: Automatic dependencies management with ExternalProject

Post by Transporter » Sat Nov 16, 2013 9:46 am

Klaim wrote:But that's your problem then :D I understand that some want to update the dependencies of their dependencies but personally I don't have time to do it. Also, Ogre is not tested against other versions of these dependencies, so I don't want to upgrade them.
I test it :D I have the time because I have a couple of batch scripts which are building all dependencies. I only have to do some work if there are bugs in the dependencies or the project files have been modified.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Sat Nov 16, 2013 11:57 am

Would you consider to provide PRs to the Ogre dependencies repo so that we all benefit from your upgraded dependencies?
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: Automatic dependencies management with ExternalProject

Post by Transporter » Sun Nov 17, 2013 9:00 am

Klaim wrote:Would you consider to provide PRs to the Ogre dependencies repo so that we all benefit from your upgraded dependencies?
I could do that, but I only test them on windows! I have no idea what happens on Linux etc.
0 x

User avatar
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
Location: Germany
x 2

Re: Automatic dependencies management with ExternalProject

Post by scrawl » Sun Nov 17, 2013 12:46 pm

Don't worry, no one uses that on linux. We have package managers :wink:
0 x

Transporter
Minaton
Posts: 933
Joined: Mon Mar 05, 2012 11:37 am
Location: Germany

Re: Automatic dependencies management with ExternalProject

Post by Transporter » Sun Nov 17, 2013 1:35 pm

scrawl wrote:Don't worry, no one uses that on linux. We have package managers :wink:
I can build the dependencies for windows package manager, too. :wink:
https://drive.google.com/folderview?id= ... sp=sharing
But I don't know if there are any package manager for Win8 RT, OSX, Android and iOS.
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Sun Nov 17, 2013 1:42 pm

scrawl wrote:Don't worry, no one uses that on linux. We have package managers :wink:
I will have to because I need the dependencies to be the same, with the same modifications, for my project.
0 x

User avatar
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
Location: Germany
x 2

Re: Automatic dependencies management with ExternalProject

Post by scrawl » Sun Nov 17, 2013 2:27 pm

What kind of modifications do you need?
0 x

User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
Contact:

Re: Automatic dependencies management with ExternalProject

Post by Klaim » Mon Nov 18, 2013 12:53 am

scrawl wrote:What kind of modifications do you need?
For example, until OIS is removed from dependencies: https://bitbucket.org/cabalistic/ogrede ... is-as/diff

Also when I upgrade Ogre or any other dependency of my project, I get the code source of the version I want, put it in my repo and check if everything works. If at some point something don't work and it's in Ogre, I try to fix it and make a pull requests. But sometime the fix is a quickfix which shouldn't appear in Ogre main base so I do the quickfix for me and talk about the problem on this forum. Ogre people are very reactive so a solution is often found and implemented quickly, but I can't say the same for all libraries I use (and I use a lot).
Same thing with Boost.

What I mean is that my code is supposed to compile with a specific version of my dependencies, and these (maybe modified versions of) dependencies needs to be shiped in the final product, so no package manager will help with that.
0 x

Post Reply