Page 1 of 20

CMake build system for Cthugha *looking for testers*

Posted: Thu Oct 16, 2008 7:05 pm
by CABAListic
[Update 09-01-21]
The system has come a fairly long way up to now and is in a state where it can do almost anything you might want it to do on Windows and Linux. It needs thorough testing so that it hopefully might become the official build system, provided that it's to the team's liking.
You can grab the latest copy here:
Instructions are included.

Reading some recent forum posts suggests that the non-VS build systems sometimes get missed and lack newly created files from the build process. So I thought this sounds like a job for a cross-platform build system. I know, there have been requests before for a CMake build system, and since I wanted to get some more practice with CMake anyway, I went ahead and created one for Shoggoth head.

Or rather, the basic foundation for it. Ogre's various platforms are not exactly coherent in their building behaviour in some regards, and getting those quirks right is not exactly trivial (read: it is easier to use CMake if you plan with it from the beginning rather than fit it in retroactively), so it needs to be considered a work in progress. It works fairly well on Linux, though (at least for me), and it's imho so much nicer than the autotools :)

Anyway, I've uploaded my current version, you can get it here: CMake build files
Please do take note of the issues below. If you will, you could consider it a case study, and I'd be interested to know if anyone thinks this might have some future with Ogre (after sorting out the quirks, of course). Or, more precisely: Should I try to improve it further, or is Ogre covered enough in terms of build systems?

For anyone not familiar with CMake, here are some of the more interesting features it could offer:
  • Single build system for all platforms. If this build system is modified (i. e. new source files added), the changes would reflect everywhere.
  • Coherent configuration utility. CMake builds can be configured via a GUI where you can set specific options. For example, you can specify which parts to build, and it can even set the OgreConfig.h options. You can configure static or dynamic builds, threading level etc.
  • Produces native build solutions for the targeted platform. On Windows you get Visual Studio solutions very similar to the custom-built Ogre ones, whereas it creates very nice Makefiles for Unix or mingw. It can also create CodeBlocks projects, though they are admittedly not as nice because it just throws everything into one big project at the moment. On Mac there's an XCode generator.
  • Builds usually happen out of source, i. e. you provide a work directory differing from your source dir, leaving the latter free from compiled clutter.
  • CMake is accompanied by CPack which is a package generator utility. It can package builds for the various platforms, for example it can generate NSIS installers on Windows or .debs and .rpms on Linux. If setup properly, this might make for a (semi-)automated build of official SDK distributions for various platforms fully integrated into the build system. But I haven't investigated this further.
I've tried to model the current primary build systems on Linux and Windows (I don't own a Mac, so there is no Mac support currently), and it's working in principle. But there are some restrictions, especially on Windows:
  • There's no make uninstall for Linux
  • No pkg-config description is generated for Linux
  • Precompiled headers on Windows are not working
  • Debug build of OgreMain on Windows is not quite working. The linking fails, claiming to have exceeded memory restrictions. There must be some compile option set in the original Ogre projects that I have missed, or maybe it's the use of a precompiled header, dunno. It's kind of hard to find out since each change results in a full rebuild of OgreMain.
  • DLLs on Windows are not copied to the Samples/Common/bin dir. Setting this up is slightly awkward, I'd actually prefer to build the samples in the top bin directory (the need for a separate samples bin dir never quite struck me).
And probably many more I didn't spot. So, as I said, it's more a case study or an experiment which is by no means meant to be complete or in a fully usable state. Nevertheless, I'd be interested to hear what you think. :)

Posted: Thu Oct 16, 2008 7:23 pm
by cdkeito
can i help (i've some hours this weekend) ?

you can copy desc and everything from my script...

Posted: Thu Oct 16, 2008 8:23 pm
by PolyVox
Great work! Working on build systems is rarely fun (they made me do it at work :evil: ) but it is very useful. Actually, milliams and I did briefly work a a CMake build script for Ogre but we didn't get very far. Think we did the main .dll and that was all... can't quite remember. So well done for persevering :)

One point, I've noticed in the past that the file hierarchy in the built SDK is different to the file hierarchy is the source. I've found it fiddly in the past, when working on my application, to make it build against both the Ogre SDK and the source tree. Maybe it's a good time to address that as well?

Posted: Thu Oct 23, 2008 10:57 am
by a4z
I have tried to user the cmake build under linux, there are a view problems

first 2 errors:

Code: Select all

CMake Error in PlugIns/OctreeZone/CMakeLists.txt:
  Cannot find source file "OgreOctreeZoneOctree.h".  Tried extensions .c .C
  .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx

CMake Error in PlugIns/OctreeZone/CMakeLists.txt:
  Cannot find source file "OgreOctreeZoneOctree.cpp".  Tried extensions .c .C
  .c++ .cc .cpp .cxx .m .M .mm .h .hh .h++ .hm .hpp .hxx .in .txx
these files do not exist

after commenting out OgreOctreeZoneOctree.h/cpp cmake ran fine, but the build gave me this error

Code: Select all

/worksoftware/ogre-v1-6-0RC1/ogre/PlugIns/PCZSceneManager/src/OgrePCZoneFactory.cpp:56: error: expected primary-expression before ‘const’
make[2]: *** [PlugIns/PCZSceneManager/CMakeFiles/Plugin_PCZSceneManager.dir/src/OgrePCZoneFactory.cpp.o] Error 1

after disabling the PCZSceneManager plugin build went trought until

Code: Select all

ogree-v1-6-0RC1/ogre/Samples/Isosurf/src/ProceduralTools.cpp:26: error: reference to ‘uint’ is ambiguous
/usr/include/sys/types.h:153: error: candidates are: typedef unsigned int uint
so I have disabled Isosurf demo
and also the RenderToTexture sample

Code: Select all

/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/src/RoomObject.cpp: In member function ‘Ogre::ManualObject* RoomObject::createRoom(Ogre::SceneManager*, const Ogre::String&, short int, bool, Ogre::Vector3, Ogre::Vector3)’:
/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/src/RoomObject.cpp:231: error: no matching function for call to ‘RoomObject::addMaterial(const std::basic_string<char, std::char_traits<char>, std::allocator<char> >&, Ogre::ColourValue, Ogre::SceneBlendType)’
/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/include/RoomObject.h:30: note: candidates are: void RoomObject::addMaterial(const Ogre::String&, Ogre::ColourValue&, Ogre::SceneBlendType)
next error in PCZTestApp

Code: Select all

/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/include/PCZTestApp.h:299: error: no matching function for call to ‘RoomObject::createTestBuilding(Ogre::SceneManager*&, Ogre::String)’
/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/include/RoomObject.h:65: note: candidates are: Ogre::PCZSceneNode* RoomObject::createTestBuilding(Ogre::SceneManager*, Ogre::String&)
/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/include/PCZTestApp.h:306: error: no matching function for call to ‘RoomObject::createTestBuilding(Ogre::SceneManager*&, Ogre::String)’
/worksoftware/ogre-v1-6-0RC1/ogre/Samples/PCZTestApp/include/RoomObject.h:65: note: candidates are: Ogre::PCZSceneNode* RoomObject::createTestBuilding(Ogre::SceneManager*, Ogre::String&)
I disabled it, the rest build fine.

what I miss in the CMakeLists.txt is something like

Code: Select all

SET (CMAKE_CXX_FLAGS "-Wall -DOGRE_GUI_GLX -DOGRE_CONFIG_LITTLE_ENDIAN" CACHE STRING "Flags used by the compiler during all build types." FORCE)
SET (CMAKE_CXX_FLAGS_DEBUG "-DDEBUG -ggdb" CACHE STRING "Flags used by the compiler during debug builds." FORCE)
SET (CMAKE_CXX_FLAGS_RELEASE "-DNDEBUG -DRELEASE -O2" CACHE STRING "Flags used by the compiler during release builds." FORCE )
(also for DebugRelease if needed, and the compiler specs are just samples, I do not know what ogre 1.6 expect to get)

usually cmake edit_cache let you edit/juse the build tye und the build flags, edititng the build flags is unaviable on my machin

and I think it would be a better idear to place the output (binaries and libs) into the build dir, not the project dir

and put

to the top of the top CMakeLists file so that the user has the alternative to view the full compiler output, but this is just cosmetic

finaly I have to say 'a big thank you' for that work, I find it really good!
I am very happy about that ogre cmake build, this will make things make more easey in future for me, and I think not just for me

Posted: Thu Oct 23, 2008 11:43 am
by CABAListic
I used the SVN head for 1.6, so I assume that your build problems are fixed there, as I definitely have the files you mention. It would be nice if you could try to use the SVN version just to make sure.

As for the CMAKE_CXX_FLAGS, they are available, but flagged as advanced, so you have to toggle advanced options in ccmake. Unfortunately, you'll also get a bunch of entries from the find_package commands, so they might be easy to miss. Far as I can tell, all the preprocessor defines are determined automatically inside the Ogre headers.

Where to place the output files is a matter of taste, I guess. For those that do not install the files via "make install", though, I find it more plausible to have them in the source directory (where they are entirely separate from the source files) than in some arbitrary build directory. But I'm not fixed on that :)

Finally, CMAKE_VERBOSE_MAKEFILE is also available under the advanced options. Use 't' to toggle them in ccmake.

Posted: Fri Oct 24, 2008 7:22 am
by volca
I think this is great, thanks for sharing!

I'd like to hear what Ogre team thinks about it - is there any chance this would replace the AutoTools/VS solution files?

Or be the replacement for at least the non-windows makefile generation?

- Maybe this is worth the trouble - after fixing all the minor issues this can really save time in the long run.

I'll try it on gentoo if I'll find the time.

Posted: Fri Oct 24, 2008 9:53 am
by Schmelly
I agree it would be great to have a common build system

CABAListic: I'm facing the exact same problems like a4z right now with the trunk version of ogre... (checked out yesterday evening)
I should have some time to give your configuration some more deeper tests at the weekend.

Posted: Fri Oct 24, 2008 1:48 pm
by sinbad
I keep meaning to try getting premake to work, since I've seen that used in other projects. Anyone got opinions on the relative merits?

This is useful though, thanks. Is it easy enough to create a 'configure' equivalent too? Any idea of XCode support? [edit]nm I see there is

Posted: Fri Oct 24, 2008 7:16 pm
by sam_baker

Thanks for the CMake files - I have had Ogre building with CMake for a few weeks but due to time constraints, without some of the plugins and the GL renderer. Your CMake files are a little more thorough and the GL one will be especially helpful.

I did fix the debug build memory issue in Visual Studio by adding this to my Ogre CMakeLists.txt:

# Disable incremental linking

It's the default attempt at incremental linking only present in debug builds, that causes MSVS to run out of memory, at least on my machine.

I really recommend CMake, it's been a joy to use, despite the fact that generally I don't consider writing build systems fun.

It's great to be able to use Visual Studio for building/debugging but to never have to touch the project files which are a nightmare to maintain in MSVS. CMake is simple to use but the quality of their Visual Studio integration is really outstanding.

The installation and the CPack commands are useful too. I have my game packaged in a Nullsoft installer using 10 lines of CMake script and rebuilding the installer executable is just a case of building the Visual Studio PACKAGE project created in the game's solution by CMake.


Posted: Fri Oct 24, 2008 7:26 pm
by CABAListic
sinbad wrote:I keep meaning to try getting premake to work, since I've seen that used in other projects. Anyone got opinions on the relative merits?
I haven't used premake, but I was always under the impression that premake is primarily a distributor's tool. You as the source distributor of the project would use premake to generate build scripts for various platforms from a single source, and your users would use the build scripts you generated. CMake, on the other hand, is meant be to used by the users entirely. In a way it fills the role that the configure script from the Unix autotools package has: It provides options to configure the build which the user can manipulate, and then it creates a set of individual build scripts for the user, for example Makefiles or Visual Studio projects.
Then again, premake does have support for options which can configure the build process as well, so my observation might be completely wrong.
Even so, it seems to me that CMake is the more mature alternative of the two, and it has significant backing by the KDE project to ensure that it remains so. Plus it has a GUI to configure the build, while premake seems to be an entirely console driven tool (which CMake can also be).
This is useful though, thanks. Is it easy enough to create a 'configure' equivalent too? Any idea of XCode support? [edit]nm I see there is
I'm not sure what you mean with 'configure' equivalent. Do you mean the Unix autotools package and its configure script? If so, then yes, CMake is, as I said above, pretty much the equivalent of the configure script, just with more comfort and a wider platform support. As for XCode support, CMake can do that, but as I said, I don't own a Mac, so there is no support in the build files I posted.

Before this CMake build system can seriously compete/replace one of the existing Ogre ones, there's still work to do. I outlined some of the more immediate problems above, but there are some minor details as well. I'd be happy to try to sort some of those things out, though preferably with the help of some more people (and at least one Mac expert). The question is whether that time is well spent - does a CMake build system have a future with the Ogre project?

Edit @sam_baker: I know that disabling incremental linking fixes the issue. However, the original Ogre VS build files have incremental linking enabled (what the heck does this do, by the way? :) ) and seem to be fine. That's what's puzzling me.

Posted: Sat Oct 25, 2008 4:54 pm
by sinbad
I would personally very much like to get rid of the manual updating of 6 separate build systems, as it's getting unwieldy. I've just never had the time to sit down and learn premake / CMake and get it working. So yes, if this can be made to work well I'd definitely support it.

I'm a little concerned about OS X since XCode is a bit on the fruity side compared to other tools, and there are some specific things we've had to do to make it work which I have no idea how difficult they will be to replicate. For example, our plugins are 'bundles' in Mac OS X, and on Windows and Linux you can link a dynamic library to another one at build time - Plugin_OctreeZone and Plugin_PCZSceneManager for example. However on the Mac you cannot link a bundle to another bundle to resolve symbols, it just doesn't work, so to get around it I've simply built one bundle containing everything (so there's just one plugin for PCZ on OS X, not 2). We also have some symlink trickery to get frameworks to link & package the way we want which is entirely specific to OS X.

But, if it can be made to work, and it saves time, and allows me to create more build variants in MSVC more easily (I'd love to include build variants for double precision, threading etc on top of the static linking we have now, but the cartesian product of all the potential settings becomes a nightmare to manage so I just tweak the OgreConfig.h manually, which isn't as slick), I would be very happy and assistance is very welcome.

Posted: Sat Oct 25, 2008 5:26 pm
by CABAListic
Actually, support for double precision and threading is already built into the system :) You specify the desired settings in the CMake GUI, and then it creates a corresponding OgreConfig.h and links in Boost.Thread when creating the Visual Studio project files. This is probably not quite error-proof yet, but it's principally working.

As for the XCode problems: Since you can write platform-specific instructions in CMake via platform checks, you can possibly adjust to any XCode quirks, it might just get a little ugly at times. If there are volunteers here who own a Mac and are familiar with XCode and CMake, we can probably sort it out in some way.

Posted: Sat Oct 25, 2008 5:59 pm
by sinbad
Awesome. Well, I'm willing to be 'the Mac guy' - I hope to take a closer look at this soon once I've nailed all the 1.6 things on my list.

Posted: Sat Oct 25, 2008 6:35 pm
by sam_baker
Switching to CMake is a good choice. I'm certain it'll save you guys a lot of maintenance time after initial setup. It'll also save me from modifying my CMake scripts when new Ogre builds are released...

I recommend this book:

and the online documentation:

The online docs are good but there's a decent amount of extra info in the book.

Regarding incremental linking, I believe that's just an optimization for faster linking when a full link isn't necessary. I'm happy to live without it - it may be more of a benefit for people actively updating Ogre code though. I don't know why the discrepency between the CMake project and the hand-build project - you would need to check the differences in the compile and link settings. It shouldn't be hard to find which one is causing that.


Posted: Sat Oct 25, 2008 6:46 pm
by CABAListic
The problem is that any change in the compile settings triggers a complete recompilation of OgreMain which takes time. Plus I already tried most of the differences without success. What's left is the precompiled header, don't know if this affects the linking. But I have not yet succeeded to get the precompiled header to work. Ah well, sooner or later we'll get to the bottom of this :)

Posted: Sat Oct 25, 2008 11:51 pm
by mirlix
One idea I had after looking over the script is, that large parts of the file list could be replaced by the file command, which would also make the whole buildsystem much more flexible because new files would be automaticly added and old wouldnt needed to be deleted by hand.

In the CMakeLists.txt in the OgreMain folder the two rather long file enumerations could be replaced by the following two lines

Code: Select all

#Files for OgreMain, searches for all  header files and cpp files in this path, but not in subfolders
file(GLOB HEADER_FILES include/*.h)
file(GLOB SOURCE_FILES src/*.cpp)
This will only work of course if there isnt a problem with adding all files in this folder, but I dont thing it shoudl. I testet it with the OgreMain and both RenderSystems and it compiles.
If there isnt some negative point i missed, i could modify all the files and upload the buildfiles for other people to test this approach

Posted: Sun Oct 26, 2008 9:42 am
by CABAListic
I know about that command, but there are two drawbacks:
It is tempting to use this command to avoid writing the list of source files for a library or executable target. While this seems to work, there is no way for CMake to generate a build system that knows when a new source file has been added. Normally the generated build system knows when it needs to rerun CMake because the CMakeLists.txt file is modified to add a new source. When the source is just added to the directory without modifying this file, one would have to manually rerun CMake to generate a build system incorporating the new file.
In short: The build systems automatically rerun CMake if the CMakeLists.txt are modified, so if you add new files there, the generated build systems will (more or less) automatically update. If you use this command, then that won't work because the build systems have no means to check for new files themselves.
Of course, this is not the most devastating drawback you could imagine and is more a matter of preference.

However, there are occasionally dead files or files for other platforms in Ogre's code. The command would collect them all which is incorrect and will result in compile errors. Therefore, explicit file lists are the only safe way to guarantee that the build will always work correctly.
(As an example, OgreMain still containts source files for DevIL which collide with the newer image handling code unless you take precautions. My CMakeLists.txt explicitly exclude them.)

Besides, creating the explicit file lists is not that much work if you use the right editor for the task ;) Vim's editing abilities come in handy here. You can start with ':r !ls include/*.h' and then use the visual block mode to remove any bogus information in front of the file list, and then manually remove any unwanted files from the list. That's how I did it, and it wasn't that much effort.

Posted: Sun Oct 26, 2008 11:27 am
by mirlix
Didnt know Ogre had dead files, this will be a problem.
One thing I like about this function is that is includes all file, which is true for dead files and new files. So if a developer adds a new file he needs to modify the CMakeLists.txt as well, with the functions he wouldnt have to. Isnt much of a advantage but it like this in my own projects.

And why does have OGRE deadfiles? Is there a reason or just some overlooked files which should have been deleted. If the second case is true, the function would help detect those :D

I have not much experience with cmake so I dont know which is the best way, for me it seems like both have their drawbacks.

Posted: Sun Oct 26, 2008 11:44 am
by CABAListic
Well, I guess "dead" files may have been an exaggeration on my part. I'm mostly referring to the DevIL image codec files which can still be used, especially on Linux. But they should be excluded from a normal build, and therefore a catch-all-files call doesn't quite cut it. Even more so with platform specific files which are not always in separate subfolders.

It's true you wouldn't have to edit the CMakeLists.txt files if you added new files. But you would have to rerun CMake manually which is otherwise done automatically. As I said, this is a matter of preference.

Posted: Sun Oct 26, 2008 4:12 pm
by sinbad
Yes, we'd definitely want to keep file inclusion specific rather than filter-based, since we often have build variants that use different source files, for different platforms and build options.

Posted: Wed Nov 05, 2008 8:13 pm
by a4z
I tested the cmake build with the 1.6.0 Ogre Release

build seems to work, but I have 2 problems

for debug version I find it not so good that following libs are searched with the _d postfix
because normaly I am not interested to debug those libs and there are no debug versions on the system aviable
(especially peopel who are interested in Ogre and install the dependency libs via package tool)

if I turn on threading

CMake Error at CMakeLists.txt:97 (message):
Could not find dependency: boost-thread. Can't enable threading

you will not get this error if you do not enable threading, but if you do so some of the examples ar not running
undefined symbol: pthread_mutexattr_init in DynLib::load
looks like if the -lpthread linker flag is missing

boost libs are found and shown correct , but in the root CMakeList.txt I think there sould be a use of the FindBoost module
maybe something like FIND_PACKAGE( Boost 1.36.0 COMPONENTS thread )
befor the "if (NOT BOOST_FOUND)" , line 96
could be the solution

or the "if (NOT BOOST_FOUND)" sould be placed somewhere else?

Posted: Wed Nov 05, 2008 8:36 pm
by CABAListic
The FindBoost package is already used, it's implicitly called in the FindDependencies script. If it cannot find boost, I'm afraid you'll have to provide the paths yourself in the CMake configuration.

Anyway, if you don't want to debug too deeply, you might want to use RelWithDebugInfo as your build type instead of Debug, I think this might suit you better. Debug is debug and will use debug libs if it can find them.

As for the pthread issue, that's a bit strange. Does this occur without threading? The flag is already added to threading builds, but since I don't fully know what it implies for code not using threading, I didn't add it as a default.

Posted: Wed Nov 19, 2008 10:26 pm
by milliams
I've had a play around with the CMake scripts you posted in the 1st post. As PolyVox said, he and I had a go a while back at creating our own CMake build system for OGRE and I actually had some success. However, yours seems to replicate the current system better.

I've taken the state you had it in and have been editing it. You can see the small amount of work I've done so far at Gitorious. I've only added SO versioning and a OGREConfig.cmake and OGREConfigVersion.cmake file (so that projects don't need a FindOGRE.cmake file to find OGRE) so far but I've got some ideas about what else needs doing.

One thing I don't like though is the building of the libraries and binaries in ${CMAKE_SOURCE_DIR}/lib/ or ${CMAKE_SOURCE_DIR}/bin since that completely breaks the whole "out-of-source" build ethic of CMake. Instead, I'm thinking that the demos can be build in a ${CMAKE_SBINARY_DIR}/Samples/ directory. On Windows, I can copy in the libraries needed to run them but on Linux it will work automatically once OGRE is installed.

Also I'm going to have a go at neatening up the FindDependencies bit by splitting the different packages out into separate FindFoo.cmake files to aid reuse.

Posted: Wed Nov 19, 2008 11:20 pm
by CABAListic
Heh, SO versioning and the OgreConfig.cmake were on my todo list, but I'm currently somewhat busy, so I never got to that. Anyway, I'm looking forward to your changes :)

Posted: Sat Nov 22, 2008 8:31 pm
by milliams
Okay, I've been doing some more work on this. For a detailed overview of the changes I've made, look at the commit history.

Basically, I've added the following features:
  • The ability to specify the location of the OGRE dependencies package on Windows with the OGRE_DEPENDENCIES_DIR variable
  • Output at the end of the CMake run to tell the user what was found and what wasn't
  • Install the OGRE.pc file so that people can use pkg-config if they want
  • Now nothing is changed in the source directory so that you can have multiple build directories for different build variants without them interfering with each other
  • Samples are built in the build-dir. Most work but some don't yet (due to *.cfg stuff) and probably none will work on Windows
  • Beginning of CPack integration (so that we can automatically generate source and SDK releases for all platforms)
If you want the Git repo then use

Code: Select all

git clone git:// ogre
or if you just want a tarball, you can get one from the link at the bottom of this page. Either way, just extract the OGRE 1.6 source tarball over the top of it. Then do:

Code: Select all

mkdir build
cd build
cmake ..
or use one on the CMake GUIs.

I would appreciate if people would give this a try. It completely builds for me on Linux and e.g. the Demo_Grass demo works. Windows should work but I can't guarantee it. MacOS X probably won't work but if anyone wants to try it or help get it working, just let me know.