Simpler Build System

[cadabra]

Here's a boring idea for 2.0: a simpler build system. (I'm not suggesting moving away from CMake.)

Over the years of using Ogre here and there, the biggest thing that stands out to me is all the issues I've had simply building and linking it. Every time I want to start a new project, it's a hurdle to figure out the right parameters I need to pass to CMake to build Ogre, discover its dependencies, and then actually make my own CMake project find Ogre. Every time I pull new Ogre changes from hg, I have to go through the same process because something changed and broke in Ogre's CMakeLists.txt files or FindOGRE.cmake.

At this very moment, I can't link Ogre because FindOgre is broken on OS X in the 1.9 branch. I'm guessing it's the usual "Ogre built Ogre.framework but FindOGRE is looking for libOgreMain". I've debugged FindOGRE.cmake before and it isn't pretty. Stepping through cmake --debug-output --trace and adding print statements to FindOGRE makes me want to cry.

Even looking at the results of `make install` after running CMake, it's clear that there's a bit of a mess here. There's an include directory with headers under OGRE and other prefixes. Makes sense. There's a lib directory with built dependencies but no Ogre libraries. Under lib/macosx/$BUILD_TYPE/, there's Ogre.framework with a Headers directory that has some headers but lacks OgrePrereqs and so doesn't actually work. Then we have a lib/pkgconfig/OGRE.pc that refers to libraries that weren't built. Etc, etc, etc.

I don't know how many times I've looked for alternatives to Ogre simply because of these build and integration issues.

What's the problem? I stepped inside Ogre source and counted the LOC in CMake files: 22214. Does Ogre really need 22 000 lines of code in build configuration? The problem that keeps breaking the build and FindOGRE is that Ogre goes through so much trouble to produce libraries in different ways based on the environment. I understand that the intent is to be helpful here, but it just makes things complicated.

- Did the user set OGRE_STATIC? Then we're going to append Static to all our library names. Unless we're on OS X.
- Are we on OS X? Let's build frameworks instead of libraries. What if the user set OGRE_STATIC? No idea.
- Let's put those frameworks in lib/macosx/$BUILD_TYPE instead of the standard location.
- Are we on OS X? Let's rename OgreMain to just Ogre.
- Let's use OGRE in some places and Ogre in others.
- Etc.

It's no wonder that FindOGRE is always breaking when there's so much logic to keep in sync.

My christmas wish list is for Ogre's build configuration to stop trying to be so helpful and instead just build standard CMake libraries and go with the established conventions of CMake. Don't customize anything. Don't try to be smart. KISS. Be dumb. Let CMake do all the heavy lifting.

- We don't need OGRE_STATIC: CMake already provides and respects BUILD_SHARED_LIBS.
- We don't need to add Static when we're building statically.
- We don't need to modify the install path based on the build type: lib/macosx/$BUILD_TYPE. Just let CMake pick the install path.
- On OS X, let's not build frameworks. Sure, frameworks are a little easier to drag-drop into Xcode, but users will have to figure out how to link Ogre's dependencies anyway, and iOS users will want to link statically (and possibly OS X users too), so why not keep it simple and just make standard CMake libraries?
- We don't even need FindOGRE.cmake, we can just generate and install an accurate OgreConfig.cmake file and let CMake load those (which it does automatically when the user tries to find something). Find$FOO files are intended for finding external non-CMake libraries. Making everyone copy FindOGRE.cmake into their source and keep it up to date guarantees it's always out of date and broken somewhere.

There's a few thoughts about ways we could make this better. Please forgive me if this is the wrong forum section. Let me know what you think!

[masterfalcon]

I'm going to start my response with a couple statements that will address many of your issues.
1. "Standard CMake libraries" do not exist. CMake is a build system, that's it. Development library formats vary across platforms(.a, .so, .dll, .framework, etc). And we aim to provide libraries that provide the best experience for the platform that you are developing on. Now if you are coming from Windows and aren't comfortable with frameworks, that is part of the learning curve of OS X, not Ogre.

2. Ogre is a large and complex codebase that supports many platforms. As such, project configuration is not extremely simple. Especially when you are trying to make things as easy as possible for users of varying degrees of experience.

3. CMake has come a long way in the last couple years but Apple keeps changing things. Because of changes that Apple has made and that we can't guarantee that everyone has the same version of CMake, a certain amount of hacks are necessary to create a homogenous experience.

You say that FindOgre is broken on OS X in the 1.9 branch. Care to elaborate on that? File a bug.

Now all that being said, of course there is room for improvement.

Using OGRE_STATIC may be not be necessary but replacing it with a builtin variable isn't going to reduce the complexity so let's just leave it.
We probably can remove the word Static from library names as the file extension will let you know. I'm not sure why it was added in the first place.
I don't know how that config file is supposed to be generated. It's been around longer than I have. I honestly didn't even realize that it existed.

[Kojack]

We probably can remove the word Static from library names as the file extension will let you know. I'm not sure why it was added in the first place.

With visual studio builds you can't tell static libs from dynamic import libs, both end with .lib.
So ogremain.lib could be a static library, or the import library for ogremain.dll.

[masterfalcon]

Ah ok. The naming does have a purpose then.

[sinbad]

You should have seen what it was like *before* CMake (many Visual Studio files, XCode files, autotool files, really hard to configure without changing files in-place, yadda yadda). Build systems across multiple platforms with tunable parameters (because people genuinely have different needs) is inherently complicated. I suggest that if you don't like CMake, don't use it for your project, just use a simpler direct reference system or file copying etc (that's what I used to do in other external projects). Ogre itself really does need this complexity, sorry.

[cadabra]

Hey, thanks for all your responses!

1. "Standard CMake libraries" do not exist. CMake is a build system, that's it. Development library formats vary across platforms(.a, .so, .dll, .framework, etc).

Here's what CMake.org's front page says: "CMake is used to control the software compilation process using simple platform and compiler independent configuration files."

The promise of CMake is that you can write highly generic configuration files which CMake transforms into something that ideally suits the user's target platform. There is such a thing as a standard CMake library, namely: add_library(name source123...). CMake will produce the right thing from that. Additionally, CMake allows your client to customize parts of your program using well-established hooks, e.g. BUILD_SHARED_LIBS, CMAKE_BUILD_TYPE, CMAKE_CXX_COMPILER, CMAKE_OSX_ARCHITECTURES, CMAKE_OSX_DEPLOYMENT_TARGET, etc. The less you specify in your CMakeLists.txt, the better CMake can target different platforms and accommodate your clients. Think of it as a high level language for building software.

Unfortunately, CMake gives CMakeLists.txt authors so many hooks into that process that it's easy to over-configure and inadvertently break things. Also, like you said, CMake isn't always perfect and sometimes requires specific hacks. More on that below.

And we aim to provide libraries that provide the best experience for the platform that you are developing on.

That's great. That said, few developers will ship products using the Ogre binaries you provide. They're great for getting started, but my impression is that almost everyone will want to build their dependencies from source. Am I wrong? That's why building and linking Ogre should be easy. If Ogre follows CMake conventions, that makes life easier for everyone.

Now if you are coming from Windows and aren't comfortable with frameworks, that is part of the learning curve of OS X, not Ogre.

I'm not coming from Windows and I have nothing against frameworks. The reasons I think Ogre shouldn't do frameworks are:

- Frameworks add complexity to Ogre's build system and FindOGRE.cmake. They also add complexity for users, because:

- There are huge numbers of (A) iOS only developers. There are loads of (B) iOS + OS X developers, especially in games. The number of (C) OS X only developers is comparatively tiny.
- It would be nice if the process for integrating Ogre in your Xcode project was the same for all three groups A, B, and C.
- It would be nice if your #include statements were the same for OS X and iOS code, especially if you have cross-platform code which uses Ogre.
- By producing frameworks on OS X but static libraries on iOS, you're catering to C but making things harder for B (B has to link Ogre differently (static library vs dynamic framework) and write their #includes differently (OGRE/OgreRoot.h vs Ogre/OgreRoot.h)). The group B is much larger than the group C.

2. Ogre is a large and complex codebase that supports many platforms. As such, project configuration is not extremely simple. Especially when you are trying to make things as easy as possible for users of varying degrees of experience.

Understood. Complexity follows size. Still, I maintain that project configuration could be simpler. That was the whole point of CMake, after all.

Here's a question: Why does Ogre need the separate concepts of Components and Plugins? Why aren't those just additional libraries which depend on the library OgreMain?

3. CMake has come a long way in the last couple years but Apple keeps changing things. Because of changes that Apple has made and that we can't guarantee that everyone has the same version of CMake, a certain amount of hacks are necessary to create a homogenous experience.

I understand that CMake doesn't support every target platform perfectly and sometimes lags behind Apple's changes. The counter argument is this: there are a million CMake projects. It makes more sense to fix CMake than to hack every project out there. CMake is open source, has an active mailing list, and has official releases every couple months. Sure, I know no-one wants to hack on CMake. But who wants to maintain 22 000 lines of CMake configuration files?

CMake allows you to require that users have a minimum version with cmake_minimum_required(VERSION 1.2.3 FATAL_ERROR). I noticed Ogre specifies 2.6.2, from 2008. Why? Are there Ogre users who cannot upgrade CMake?

You say that FindOgre is broken on OS X in the 1.9 branch. Care to elaborate on that? File a bug.

I'll file a bug or pull request and post it. The bug tracker changed and my account couldn't login.

With visual studio builds you can't tell static libs from dynamic import libs, both end with .lib. So ogremain.lib could be a static library, or the import library for ogremain.dll.

If I maintained a family of a dozen projects (e.g. Ogre) and someone told me that they:
- built my projects once statically,
- built my projects one more time dynamically, and
- they want to put all the build results into one tree - (so that some parts of their project can link Ogre statically and some parts dynamically?)
- and for them to do that I have to change all my projects to add Static to static library names or something along those lines
- and presumably everyone who maintains a CMake project has to do the same in order to accommodate them,

My first response to that would be: Don't do that. Why put those together? I like to apply a kind of categorical imperative here: should we modify every project in the world to add Static when building statically to accommodate this particular use case?
- Is it important for users to commingle the same static and dynamic build products?
- If we don't do this, is there a workaround for this user? E.g. could they just go ahead and put OgreDynamic and OgreStatic build results in different trees? Will Visual Studio fail to work like that?
- Is it common on Windows to build things statically and dynamically and put them all together?
- Is there precedent in other libraries?
- If this is a hack specifically tailored to support Windows, then can we limit it to Windows builds?
- Could the client just use CMAKE_STATIC_LIBRARY_SUFFIX for this? I.e. users should be able to specify `cmake -DCMAKE_STATIC_LIBRARY_SUFFIX=Static.lib` when they generate the Ogre build on Windows, and Ogre shouldn't have to know anything about it.

You should have seen what it was like *before* CMake (many Visual Studio files, XCode files, autotool files, really hard to configure without changing files in-place, yadda yadda). Build systems across multiple platforms with tunable parameters (because people genuinely have different needs) is inherently complicated. I suggest that if you don't like CMake, don't use it for your project, just use a simpler direct reference system or file copying etc (that's what I used to do in other external projects). Ogre itself really does need this complexity, sorry.

I do like CMake, or the idea of CMake. Like I said, I don't want to replace CMake. I just think we could clean up our logic and let CMake do more of what it does.

[spacegaier]

CMake is open source, has an active mailing list, and has official releases every couple months.

If you are aiming at getting in touch with CMake to push some patches: We or more specifically Assaf tried that a few times, but it took forever AFAIK (if it even was payed real attention to from the CMake devs).

I'll file a bug or pull request and post it. The bug tracker changed and my account couldn't login.

You need to create a new account on our new JIRA tracker.

[sparkprime]

At some point (pre-cmake) I decided to build Ogre using my own build scripts, and I haven't looked back.

* don't need samples
* don't need plugins
* don't need dynamic libraries
* only need 2 platforms (windows & linux)

Once you start simplifying it for your project then it becomes sane again. At the end of the day, it is the source code that matters.

[CABAListic]

Let's ignore all questions of necessary or unnecessary complexity, then we might agree that the build system has become quite a beast. But it's the "natural" order of things. When I wrote the original version, it was still relatively simple - already more complex than an average build system, but as sinbad mentioned, that can't be avoided. Then other developers began expanding on it, adding features and new platforms. At some point the build system passed the point where it would have been a good idea to document it and create some developer guidelines, particulary since not everyone in the Ogre team was quite familiar with the quirks of CMake.

So, one thing leads to another, and a somewhat complex system becomes more complex and more complicated. I did not foresee some of the things that we need CMake to handle, particularly in regards to dealing with the mobile platforms. As a consequence, there are some hacks and quite a bit of code duplication that in hindsight could have been designed much more elegantly without sacrificing features.

In the end, I believe the build system is very similar to a regular code base that is evolving. I think it does its job quite well overall, but every once in a while a little refactoring doesn't hurt to keep it that way.

[Klaim]

My (long now and very very hurtful) experience with CMake is that indeed it is far more useful to consider it as a language for a build framework specific for your project. So you have to consider the cmake scripts as a sub project which have to have specific and regular attention. This is very painful but so far we don't have better tools (with c++).

I'm not totally familiar with the CMake internals of Ogre, I only use a specific set of options and generate only for windows so far, but I was wondering if it's not just sign that there needs to be a progressive refactoring process? It looks like the hard part is to have someone both willing to do it and having enough knowledge about CMake to see how to fix most problem through high level organization.

[Kojack]

- Is there precedent in other libraries?

Boost uses a different naming scheme for static libs and dynamic import libs on windows.

[masterfalcon]

I'll reply more in depth later but it's getting late.

First I noticed that the number 22,000 was mentioned twice in regards to the number of lines in CMake scripts that we use. That caught my attention so I decided to test it.

Code: Select all

 find . -name '*.cmake' -o -name '*CMakeLists.txt' -o -name '*.cmake.in' -o -name '*CMakeLists.txt.in'| xargs wc -l
      78 ./CMake/CMakeLists.txt
     293 ./CMake/ConfigureBuild.cmake
     310 ./CMake/Dependencies.cmake
     212 ./CMake/FeatureSummary.cmake
     273 ./CMake/InstallDependencies.cmake
     241 ./CMake/InstallResources.cmake
      46 ./CMake/Packages/FindCarbon.cmake
      53 ./CMake/Packages/FindCg.cmake
      46 ./CMake/Packages/FindCocoa.cmake
      46 ./CMake/Packages/FindCoreVideo.cmake
      47 ./CMake/Packages/FindCppUnit.cmake
      71 ./CMake/Packages/FindDirectX.cmake
     113 ./CMake/Packages/FindDirectX11.cmake
     138 ./CMake/Packages/FindDoxygen.cmake
      49 ./CMake/Packages/FindFreeImage.cmake
      61 ./CMake/Packages/FindFreetype.cmake
      45 ./CMake/Packages/FindGLSLOptimizer.cmake
      45 ./CMake/Packages/FindHLSL2GLSL.cmake
      45 ./CMake/Packages/FindIOKit.cmake
     568 ./CMake/Packages/FindOGRE.cmake
      88 ./CMake/Packages/FindOIS.cmake
      87 ./CMake/Packages/FindOpenGLES.cmake
     160 ./CMake/Packages/FindOpenGLES2.cmake
     156 ./CMake/Packages/FindOpenGLES3.cmake
     101 ./CMake/Packages/FindPOCO.cmake
      80 ./CMake/Packages/FindSoftimage.cmake
     125 ./CMake/Packages/FindTBB.cmake
      26 ./CMake/Packages/FindTinyXML.cmake
      31 ./CMake/Packages/FindWix.cmake
      48 ./CMake/Packages/FindZLIB.cmake
      50 ./CMake/Packages/FindZZip.cmake
      32 ./CMake/Packaging.cmake
      79 ./CMake/PrepareThreadingOptions.cmake
      28 ./CMake/Templates/OGREConfig.cmake.in
      29 ./CMake/Templates/OGREConfigVersion.cmake.in
     302 ./CMake/Templates/SDK_CMakeLists.txt.in
      65 ./CMake/Templates/SDK_Samples_CMakeLists.txt.in
    1129 ./CMake/toolchain/android.toolchain.cmake
      97 ./CMake/toolchain/AndroidJNI.cmake
     163 ./CMake/Utils/FindPkgMacros.cmake
     134 ./CMake/Utils/MacroLogFeature.cmake
     122 ./CMake/Utils/OgreAddTargets.cmake
     462 ./CMake/Utils/OgreConfigTargets.cmake
      35 ./CMake/Utils/OgreFindFrameworks.cmake
      23 ./CMake/Utils/OgreGetVersion.cmake
      64 ./CMake/Utils/PrecompiledHeader.cmake
      60 ./CMake/Utils/PreprocessorUtils.cmake
     547 ./CMakeLists.txt
      37 ./Components/CMakeLists.txt
      73 ./Components/Overlay/CMakeLists.txt
      72 ./Components/Paging/CMakeLists.txt
      50 ./Components/Property/CMakeLists.txt
     111 ./Components/RTShaderSystem/CMakeLists.txt
      82 ./Components/Terrain/CMakeLists.txt
      78 ./Components/Volume/CMakeLists.txt
      14 ./CTestConfig.cmake
      43 ./Docs/CMakeLists.txt
     817 ./OgreMain/CMakeLists.txt
      54 ./PlugIns/BSPSceneManager/CMakeLists.txt
      43 ./PlugIns/CgProgramManager/CMakeLists.txt
      38 ./PlugIns/CMakeLists.txt
      47 ./PlugIns/OctreeSceneManager/CMakeLists.txt
      48 ./PlugIns/OctreeZone/CMakeLists.txt
      85 ./PlugIns/ParticleFX/CMakeLists.txt
      66 ./PlugIns/PCZSceneManager/CMakeLists.txt
      75 ./RenderSystems/CMakeLists.txt
      96 ./RenderSystems/Direct3D11/CMakeLists.txt
      97 ./RenderSystems/Direct3D9/CMakeLists.txt
     266 ./RenderSystems/GL/CMakeLists.txt
     184 ./RenderSystems/GL3Plus/CMakeLists.txt
     209 ./RenderSystems/GLES/CMakeLists.txt
     282 ./RenderSystems/GLES2/CMakeLists.txt
      20 ./Samples/AtomicCounters/CMakeLists.txt
      19 ./Samples/BezierPatch/CMakeLists.txt
     434 ./Samples/Browser/CMakeLists.txt
      22 ./Samples/BSP/CMakeLists.txt
      20 ./Samples/CameraTrack/CMakeLists.txt
      19 ./Samples/CelShading/CMakeLists.txt
      25 ./Samples/Character/CMakeLists.txt
     260 ./Samples/CMakeLists.txt
      35 ./Samples/Compositor/CMakeLists.txt
      19 ./Samples/CubeMapping/CMakeLists.txt
      56 ./Samples/DeferredShading/CMakeLists.txt
      20 ./Samples/Dot3Bump/CMakeLists.txt
      19 ./Samples/DualQuaternion/CMakeLists.txt
      20 ./Samples/DynTex/CMakeLists.txt
      26 ./Samples/EndlessWorld/CMakeLists.txt
      19 ./Samples/FacialAnimation/CMakeLists.txt
      20 ./Samples/Fresnel/CMakeLists.txt
      19 ./Samples/Grass/CMakeLists.txt
      33 ./Samples/Instancing/CMakeLists.txt
      34 ./Samples/Isosurf/CMakeLists.txt
      19 ./Samples/Lighting/CMakeLists.txt
      26 ./Samples/Media/CMakeLists.txt
      20 ./Samples/MeshLod/CMakeLists.txt
      21 ./Samples/NewInstancing/CMakeLists.txt
      35 ./Samples/OceanDemo/CMakeLists.txt
      20 ./Samples/ParticleFX/CMakeLists.txt
      36 ./Samples/ParticleGS/CMakeLists.txt
      35 ./Samples/PCZTestApp/CMakeLists.txt
      18 ./Samples/PNTrianglesTessellation/CMakeLists.txt
      32 ./Samples/ShaderSystem/CMakeLists.txt
      32 ./Samples/ShaderSystemMultiLight/CMakeLists.txt
      30 ./Samples/ShaderSystemTexturedFog/CMakeLists.txt
      31 ./Samples/Shadows/CMakeLists.txt
      19 ./Samples/SkeletalAnimation/CMakeLists.txt
      20 ./Samples/SkyBox/CMakeLists.txt
      20 ./Samples/SkyDome/CMakeLists.txt
      20 ./Samples/SkyPlane/CMakeLists.txt
      19 ./Samples/Smoke/CMakeLists.txt
      20 ./Samples/SphereMapping/CMakeLists.txt
      21 ./Samples/SSAO/CMakeLists.txt
      22 ./Samples/Terrain/CMakeLists.txt
      19 ./Samples/Tesselation/CMakeLists.txt
      19 ./Samples/TextureArray/CMakeLists.txt
      19 ./Samples/TextureFX/CMakeLists.txt
      20 ./Samples/Transparency/CMakeLists.txt
      34 ./Samples/VolumeCSG/CMakeLists.txt
      34 ./Samples/VolumeTerrain/CMakeLists.txt
      37 ./Samples/VolumeTex/CMakeLists.txt
      35 ./Samples/Water/CMakeLists.txt
     314 ./Tests/CMakeLists.txt
      39 ./Tests/PlayPen/CMakeLists.txt
      14 ./Tests/VisualTests/CheckResult.cmake
     237 ./Tests/VisualTests/CMakeLists.txt
     229 ./Tests/VisualTests/Context/CMakeLists.txt
      12 ./Tests/VisualTests/CTestCustom.cmake
      38 ./Tests/VisualTests/PlayPen/CMakeLists.txt
      26 ./Tests/VisualTests/PostTest.cmake
      31 ./Tests/VisualTests/RunVTests.cmake
      44 ./Tests/VisualTests/VTests/CMakeLists.txt
      16 ./Tools/CMakeLists.txt
      18 ./Tools/MeshUpgrader/CMakeLists.txt
      44 ./Tools/XMLConverter/CMakeLists.txt
      30 ./Tools/XSIExport/CMakeLists.txt
   13034 total

Still a lot, but far short of 22,000. Notice that over 1000 lines are in the android toolchain file alone. That probably cannot be helped. You can get somewhat close to that number if you include your build directory, but that's not part of the source repo so you need to make sure to count with a fresh checkout.

So I decided to look for places to trim down. The first thing is the long file lists that we maintain. Those are a pain so let's use file globbing instead. I'm not done yet but I already have it down to 11,557.

Code: Select all

      12 ./Tests/VisualTests/CTestCustom.cmake
      14 ./CTestConfig.cmake
      14 ./Tests/VisualTests/CheckResult.cmake
      16 ./Tools/CMakeLists.txt
      18 ./Samples/PNTrianglesTessellation/CMakeLists.txt
      18 ./Tools/MeshUpgrader/CMakeLists.txt
      19 ./Samples/BezierPatch/CMakeLists.txt
      19 ./Samples/CelShading/CMakeLists.txt
      19 ./Samples/CubeMapping/CMakeLists.txt
      19 ./Samples/DualQuaternion/CMakeLists.txt
      19 ./Samples/FacialAnimation/CMakeLists.txt
      19 ./Samples/Grass/CMakeLists.txt
      19 ./Samples/Lighting/CMakeLists.txt
      19 ./Samples/SkeletalAnimation/CMakeLists.txt
      19 ./Samples/Smoke/CMakeLists.txt
      19 ./Samples/Tesselation/CMakeLists.txt
      19 ./Samples/TextureArray/CMakeLists.txt
      19 ./Samples/TextureFX/CMakeLists.txt
      20 ./Samples/AtomicCounters/CMakeLists.txt
      20 ./Samples/CameraTrack/CMakeLists.txt
      20 ./Samples/Dot3Bump/CMakeLists.txt
      20 ./Samples/DynTex/CMakeLists.txt
      20 ./Samples/Fresnel/CMakeLists.txt
      20 ./Samples/MeshLod/CMakeLists.txt
      20 ./Samples/ParticleFX/CMakeLists.txt
      20 ./Samples/SkyBox/CMakeLists.txt
      20 ./Samples/SkyDome/CMakeLists.txt
      20 ./Samples/SkyPlane/CMakeLists.txt
      20 ./Samples/SphereMapping/CMakeLists.txt
      20 ./Samples/Transparency/CMakeLists.txt
      21 ./Samples/NewInstancing/CMakeLists.txt
      21 ./Samples/SSAO/CMakeLists.txt
      22 ./Samples/BSP/CMakeLists.txt
      22 ./Samples/Terrain/CMakeLists.txt
      23 ./CMake/Utils/OgreGetVersion.cmake
      25 ./Samples/Character/CMakeLists.txt
      26 ./CMake/Packages/FindTinyXML.cmake
      26 ./Samples/EndlessWorld/CMakeLists.txt
      26 ./Samples/Media/CMakeLists.txt
      26 ./Tests/VisualTests/PostTest.cmake
      28 ./CMake/Templates/OGREConfig.cmake.in
      28 ./Components/RTShaderSystem/CMakeLists.txt
      29 ./CMake/Templates/OGREConfigVersion.cmake.in
      29 ./PlugIns/CgProgramManager/CMakeLists.txt
      29 ./PlugIns/OctreeSceneManager/CMakeLists.txt
      29 ./PlugIns/PCZSceneManager/CMakeLists.txt
      30 ./PlugIns/BSPSceneManager/CMakeLists.txt
      30 ./Samples/ShaderSystemTexturedFog/CMakeLists.txt
      30 ./Tools/XSIExport/CMakeLists.txt
      31 ./CMake/Packages/FindWix.cmake
      31 ./PlugIns/ParticleFX/CMakeLists.txt
      31 ./Samples/Shadows/CMakeLists.txt
      31 ./Tests/VisualTests/RunVTests.cmake
      32 ./CMake/Packaging.cmake
      32 ./Samples/ShaderSystem/CMakeLists.txt
      32 ./Samples/ShaderSystemMultiLight/CMakeLists.txt
      33 ./Samples/Instancing/CMakeLists.txt
      34 ./Samples/Isosurf/CMakeLists.txt
      34 ./Samples/VolumeCSG/CMakeLists.txt
      34 ./Samples/VolumeTerrain/CMakeLists.txt
      35 ./CMake/Utils/OgreFindFrameworks.cmake
      35 ./Samples/Compositor/CMakeLists.txt
      35 ./Samples/OceanDemo/CMakeLists.txt
      35 ./Samples/PCZTestApp/CMakeLists.txt
      35 ./Samples/Water/CMakeLists.txt
      36 ./Samples/ParticleGS/CMakeLists.txt
      37 ./Components/CMakeLists.txt
      37 ./PlugIns/OctreeZone/CMakeLists.txt
      37 ./Samples/VolumeTex/CMakeLists.txt
      38 ./PlugIns/CMakeLists.txt
      38 ./Tests/VisualTests/PlayPen/CMakeLists.txt
      39 ./RenderSystems/Direct3D11/CMakeLists.txt
      39 ./Tests/PlayPen/CMakeLists.txt
      40 ./RenderSystems/Direct3D9/CMakeLists.txt
      41 ./Components/Paging/CMakeLists.txt
      42 ./Components/Overlay/CMakeLists.txt
      43 ./Components/Volume/CMakeLists.txt
      43 ./Docs/CMakeLists.txt
      44 ./Tests/VisualTests/VTests/CMakeLists.txt
      44 ./Tools/XMLConverter/CMakeLists.txt
      45 ./CMake/Packages/FindGLSLOptimizer.cmake
      45 ./CMake/Packages/FindHLSL2GLSL.cmake
      45 ./CMake/Packages/FindIOKit.cmake
      46 ./CMake/Packages/FindCarbon.cmake
      46 ./CMake/Packages/FindCocoa.cmake
      46 ./CMake/Packages/FindCoreVideo.cmake
      47 ./CMake/Packages/FindCppUnit.cmake
      48 ./CMake/Packages/FindZLIB.cmake
      49 ./CMake/Packages/FindFreeImage.cmake
      50 ./CMake/Packages/FindZZip.cmake
      50 ./Components/Property/CMakeLists.txt
      53 ./CMake/Packages/FindCg.cmake
      56 ./Samples/DeferredShading/CMakeLists.txt
      58 ./Components/Terrain/CMakeLists.txt
      60 ./CMake/Utils/PreprocessorUtils.cmake
      61 ./CMake/Packages/FindFreetype.cmake
      64 ./CMake/Utils/PrecompiledHeader.cmake
      65 ./CMake/Templates/SDK_Samples_CMakeLists.txt.in
      71 ./CMake/Packages/FindDirectX.cmake
      75 ./RenderSystems/CMakeLists.txt
      75 ./RenderSystems/GL3Plus/CMakeLists.txt
      78 ./CMake/CMakeLists.txt
      79 ./CMake/PrepareThreadingOptions.cmake
      80 ./CMake/Packages/FindSoftimage.cmake
      86 ./RenderSystems/GLES/CMakeLists.txt
      87 ./CMake/Packages/FindOpenGLES.cmake
      88 ./CMake/Packages/FindOIS.cmake
      90 ./RenderSystems/GL/CMakeLists.txt
      97 ./CMake/toolchain/AndroidJNI.cmake
     101 ./CMake/Packages/FindPOCO.cmake
     113 ./CMake/Packages/FindDirectX11.cmake
     122 ./CMake/Utils/OgreAddTargets.cmake
     122 ./RenderSystems/GLES2/CMakeLists.txt
     125 ./CMake/Packages/FindTBB.cmake
     134 ./CMake/Utils/MacroLogFeature.cmake
     138 ./CMake/Packages/FindDoxygen.cmake
     156 ./CMake/Packages/FindOpenGLES3.cmake
     160 ./CMake/Packages/FindOpenGLES2.cmake
     163 ./CMake/Utils/FindPkgMacros.cmake
     212 ./CMake/FeatureSummary.cmake
     229 ./Tests/VisualTests/Context/CMakeLists.txt
     237 ./Tests/VisualTests/CMakeLists.txt
     241 ./CMake/InstallResources.cmake
     260 ./Samples/CMakeLists.txt
     273 ./CMake/InstallDependencies.cmake
     293 ./CMake/ConfigureBuild.cmake
     302 ./CMake/Templates/SDK_CMakeLists.txt.in
     310 ./CMake/Dependencies.cmake
     314 ./Tests/CMakeLists.txt
     384 ./OgreMain/CMakeLists.txt
     434 ./Samples/Browser/CMakeLists.txt
     462 ./CMake/Utils/OgreConfigTargets.cmake
     547 ./CMakeLists.txt
     568 ./CMake/Packages/FindOGRE.cmake
    1129 ./CMake/toolchain/android.toolchain.cmake
   11557 total

[holocronweaver]

So I decided to look for places to trim down. The first thing is the long file lists that we maintain. Those are a pain so let's use file globbing instead.

Nice! I always assumed the file lists were there for either CMake performance or cross-platform compatibility reasons.

[CABAListic]

Careful with file globbing, while it has a certain elegance to it, it has some downsides. One is that if you add or remove source files you need to remember to manually rerun CMake, or else it won't pick up the changes. And the other thing is that we have several modules that add different source files for different platforms. Getting the glob pattern right in those instances might be very tricky.

If I remember correctly, the CMake docs generally don't recommend file globs for source file lists, although if you pay attention to the issues, I guess they work fine.

[cadabra]

Strange. This is the listing I got:

Code: Select all

$ f "[Cc][Mm]ake" | xargs wc -l
wc: ./CMake: read: Is a directory
      78 ./CMake/CMakeLists.txt
     293 ./CMake/ConfigureBuild.cmake
     310 ./CMake/Dependencies.cmake
     212 ./CMake/FeatureSummary.cmake
      32 ./CMake/FlashCC.make
     273 ./CMake/InstallDependencies.cmake
     241 ./CMake/InstallResources.cmake
wc: ./CMake/Packages: read: Is a directory
      46 ./CMake/Packages/FindCarbon.cmake
      53 ./CMake/Packages/FindCg.cmake
      46 ./CMake/Packages/FindCocoa.cmake
      46 ./CMake/Packages/FindCoreVideo.cmake
      47 ./CMake/Packages/FindCppUnit.cmake
      71 ./CMake/Packages/FindDirectX.cmake
     113 ./CMake/Packages/FindDirectX11.cmake
     138 ./CMake/Packages/FindDoxygen.cmake
      49 ./CMake/Packages/FindFreeImage.cmake
      61 ./CMake/Packages/FindFreetype.cmake
      45 ./CMake/Packages/FindGLSLOptimizer.cmake
      45 ./CMake/Packages/FindHLSL2GLSL.cmake
      45 ./CMake/Packages/FindIOKit.cmake
     568 ./CMake/Packages/FindOGRE.cmake
      88 ./CMake/Packages/FindOIS.cmake
      87 ./CMake/Packages/FindOpenGLES.cmake
     160 ./CMake/Packages/FindOpenGLES2.cmake
     156 ./CMake/Packages/FindOpenGLES3.cmake
     101 ./CMake/Packages/FindPOCO.cmake
      80 ./CMake/Packages/FindSoftimage.cmake
     125 ./CMake/Packages/FindTBB.cmake
      26 ./CMake/Packages/FindTinyXML.cmake
      31 ./CMake/Packages/FindWix.cmake
      48 ./CMake/Packages/FindZLIB.cmake
      50 ./CMake/Packages/FindZZip.cmake
      32 ./CMake/Packaging.cmake
      79 ./CMake/PrepareThreadingOptions.cmake
wc: ./CMake/Templates: read: Is a directory
      22 ./CMake/Templates/Android.mk.in
      34 ./CMake/Templates/Android_resources.cfg.in
      22 ./CMake/Templates/AndroidGLES1.mk.in
      27 ./CMake/Templates/AndroidManifest.xml.in
      22 ./CMake/Templates/AndroidManifest_JNI.xml.in
      16 ./CMake/Templates/DemoLicense.rtf
    1796 ./CMake/Templates/demomedia.wxi.in
     123 ./CMake/Templates/demos.wxs.in
    1877 ./CMake/Templates/html.cfg.in
      16 ./CMake/Templates/media.cfg.in
      10 ./CMake/Templates/OGRE-Overlay.pc.in
      11 ./CMake/Templates/OGRE-Paging.pc.in
      11 ./CMake/Templates/OGRE-PCZ.pc.in
      11 ./CMake/Templates/OGRE-Property.pc.in
      11 ./CMake/Templates/OGRE-RTShaderSystem.pc.in
      11 ./CMake/Templates/OGRE-Terrain.pc.in
      11 ./CMake/Templates/OGRE-Volume.pc.in
      12 ./CMake/Templates/OGRE.pc.in
      41 ./CMake/Templates/OGRE.props.in
      81 ./CMake/Templates/OgreBuildSettings.h.in
      28 ./CMake/Templates/OGREConfig.cmake.in
      29 ./CMake/Templates/OGREConfigVersion.cmake.in
      13 ./CMake/Templates/OGREStatic.pc.in
      18 ./CMake/Templates/plugins.cfg.in
      18 ./CMake/Templates/plugins_d.cfg.in
       2 ./CMake/Templates/quakemap.cfg.in
       2 ./CMake/Templates/quakemap_d.cfg.in
      40 ./CMake/Templates/resources.cfg.in
      40 ./CMake/Templates/resources_d.cfg.in
      46 ./CMake/Templates/samples.cfg.in
      45 ./CMake/Templates/samples_d.cfg.in
     301 ./CMake/Templates/SDK_CMakeLists.txt.in
      65 ./CMake/Templates/SDK_Samples_CMakeLists.txt.in
       4 ./CMake/Templates/tests.cfg.in
       4 ./CMake/Templates/tests_d.cfg.in
       9 ./CMake/Templates/vcrt_vc8.wxi.in
       9 ./CMake/Templates/vcrt_vc9.wxi.in
       0 ./CMake/Templates/version.txt.in
     121 ./CMake/Templates/VisualStudioUserFile.vcproj.user.in
      22 ./CMake/Templates/VisualStudioUserFile.vcxproj.user.in
wc: ./CMake/toolchain: read: Is a directory
    1129 ./CMake/toolchain/android.toolchain.cmake
      97 ./CMake/toolchain/AndroidJNI.cmake
wc: ./CMake/Utils: read: Is a directory
     163 ./CMake/Utils/FindPkgMacros.cmake
     134 ./CMake/Utils/MacroLogFeature.cmake
     122 ./CMake/Utils/OgreAddTargets.cmake
     462 ./CMake/Utils/OgreConfigTargets.cmake
      35 ./CMake/Utils/OgreFindFrameworks.cmake
      23 ./CMake/Utils/OgreGetVersion.cmake
      64 ./CMake/Utils/PrecompiledHeader.cmake
      60 ./CMake/Utils/PreprocessorUtils.cmake
     547 ./CMakeLists.txt
      37 ./Components/CMakeLists.txt
      73 ./Components/Overlay/CMakeLists.txt
      72 ./Components/Paging/CMakeLists.txt
      50 ./Components/Property/CMakeLists.txt
     111 ./Components/RTShaderSystem/CMakeLists.txt
      82 ./Components/Terrain/CMakeLists.txt
      78 ./Components/Volume/CMakeLists.txt
      14 ./CTestConfig.cmake
      43 ./Docs/CMakeLists.txt
wc: ./ogredeps/cmake: read: Is a directory
    1130 ./ogredeps/cmake/android.toolchain.cmake
     126 ./ogredeps/cmake/FindDirectX.cmake
     161 ./ogredeps/cmake/FindPkgMacros.cmake
      37 ./ogredeps/CMakeLists.txt
      53 ./ogredeps/src/Cg/CMakeLists.txt
     127 ./ogredeps/src/CMakeLists.txt
     549 ./ogredeps/src/FreeImage/CMakeLists.txt
     112 ./ogredeps/src/freetype/CMakeLists.txt
wc: ./ogredeps/src/freetype/src/tools/docmaker: read: Is a directory
       1 ./ogredeps/src/freetype/src/tools/docmaker/.gitignore
     584 ./ogredeps/src/freetype/src/tools/docmaker/content.py
     113 ./ogredeps/src/freetype/src/tools/docmaker/docbeauty.py
     106 ./ogredeps/src/freetype/src/tools/docmaker/docmaker.py
     188 ./ogredeps/src/freetype/src/tools/docmaker/formatter.py
     347 ./ogredeps/src/freetype/src/tools/docmaker/sources.py
     593 ./ogredeps/src/freetype/src/tools/docmaker/tohtml.py
     132 ./ogredeps/src/freetype/src/tools/docmaker/utils.py
     180 ./ogredeps/src/ois/CMakeLists.txt
      47 ./ogredeps/src/zlib/CMakeLists.txt
      63 ./ogredeps/src/zziplib/CMakeLists.txt
     817 ./OgreMain/CMakeLists.txt
      54 ./PlugIns/BSPSceneManager/CMakeLists.txt
      43 ./PlugIns/CgProgramManager/CMakeLists.txt
      38 ./PlugIns/CMakeLists.txt
      47 ./PlugIns/OctreeSceneManager/CMakeLists.txt
      48 ./PlugIns/OctreeZone/CMakeLists.txt
      85 ./PlugIns/ParticleFX/CMakeLists.txt
      66 ./PlugIns/PCZSceneManager/CMakeLists.txt
      75 ./RenderSystems/CMakeLists.txt
      96 ./RenderSystems/Direct3D11/CMakeLists.txt
      97 ./RenderSystems/Direct3D9/CMakeLists.txt
     266 ./RenderSystems/GL/CMakeLists.txt
     184 ./RenderSystems/GL3Plus/CMakeLists.txt
     209 ./RenderSystems/GLES/CMakeLists.txt
     282 ./RenderSystems/GLES2/CMakeLists.txt
      20 ./Samples/AtomicCounters/CMakeLists.txt
      19 ./Samples/BezierPatch/CMakeLists.txt
     434 ./Samples/Browser/CMakeLists.txt
      22 ./Samples/BSP/CMakeLists.txt
      20 ./Samples/CameraTrack/CMakeLists.txt
      19 ./Samples/CelShading/CMakeLists.txt
      25 ./Samples/Character/CMakeLists.txt
     260 ./Samples/CMakeLists.txt
      35 ./Samples/Compositor/CMakeLists.txt
      19 ./Samples/CubeMapping/CMakeLists.txt
      56 ./Samples/DeferredShading/CMakeLists.txt
      20 ./Samples/Dot3Bump/CMakeLists.txt
      19 ./Samples/DualQuaternion/CMakeLists.txt
      20 ./Samples/DynTex/CMakeLists.txt
      26 ./Samples/EndlessWorld/CMakeLists.txt
      19 ./Samples/FacialAnimation/CMakeLists.txt
      20 ./Samples/Fresnel/CMakeLists.txt
      19 ./Samples/Grass/CMakeLists.txt
      33 ./Samples/Instancing/CMakeLists.txt
      34 ./Samples/Isosurf/CMakeLists.txt
      19 ./Samples/Lighting/CMakeLists.txt
      26 ./Samples/Media/CMakeLists.txt
      20 ./Samples/MeshLod/CMakeLists.txt
      21 ./Samples/NewInstancing/CMakeLists.txt
      35 ./Samples/OceanDemo/CMakeLists.txt
      20 ./Samples/ParticleFX/CMakeLists.txt
      36 ./Samples/ParticleGS/CMakeLists.txt
      35 ./Samples/PCZTestApp/CMakeLists.txt
      18 ./Samples/PNTrianglesTessellation/CMakeLists.txt
      32 ./Samples/ShaderSystem/CMakeLists.txt
      32 ./Samples/ShaderSystemMultiLight/CMakeLists.txt
      30 ./Samples/ShaderSystemTexturedFog/CMakeLists.txt
      31 ./Samples/Shadows/CMakeLists.txt
      19 ./Samples/SkeletalAnimation/CMakeLists.txt
      20 ./Samples/SkyBox/CMakeLists.txt
      20 ./Samples/SkyDome/CMakeLists.txt
      20 ./Samples/SkyPlane/CMakeLists.txt
      19 ./Samples/Smoke/CMakeLists.txt
      20 ./Samples/SphereMapping/CMakeLists.txt
      21 ./Samples/SSAO/CMakeLists.txt
      22 ./Samples/Terrain/CMakeLists.txt
      19 ./Samples/Tesselation/CMakeLists.txt
      19 ./Samples/TextureArray/CMakeLists.txt
      19 ./Samples/TextureFX/CMakeLists.txt
      20 ./Samples/Transparency/CMakeLists.txt
      34 ./Samples/VolumeCSG/CMakeLists.txt
      34 ./Samples/VolumeTerrain/CMakeLists.txt
      37 ./Samples/VolumeTex/CMakeLists.txt
      35 ./Samples/Water/CMakeLists.txt
     314 ./Tests/CMakeLists.txt
      39 ./Tests/PlayPen/CMakeLists.txt
      14 ./Tests/VisualTests/CheckResult.cmake
     237 ./Tests/VisualTests/CMakeLists.txt
     229 ./Tests/VisualTests/Context/CMakeLists.txt
      12 ./Tests/VisualTests/CTestCustom.cmake
      38 ./Tests/VisualTests/PlayPen/CMakeLists.txt
      26 ./Tests/VisualTests/PostTest.cmake
      31 ./Tests/VisualTests/RunVTests.cmake
      44 ./Tests/VisualTests/VTests/CMakeLists.txt
      16 ./Tools/CMakeLists.txt
      18 ./Tools/MeshUpgrader/CMakeLists.txt
      44 ./Tools/XMLConverter/CMakeLists.txt
      30 ./Tools/XSIExport/CMakeLists.txt
   22272 total

Without ogredeps, I get 17623. Looks like the difference is in CMake/Templates. Not sure why I have a bunch of stuff there that you don't... hg seems to think it all came from the repo.

EDIT: Oh, my pattern accidentally included ogredeps/docmaker for an extra 2k loc. So 20 000 without that. 17 000 without ogredeps.

[masterfalcon]

I also didn't include almost all of the .in files. that is likely quite a bit of it. But some is unrelated to CMake. Like html.cfg.in. That's just for generating the doxygen docs.

[scrawl]

Yeah, the problem with file globbing is that it won't pick up changes until you run cmake again, resulting in undefined references when a new file was added/pulled.
Most users probably recreate their build folder anyway when doing a fresh build or having changed the branch, so I guess it's really up to the Ogre devs to decide whether it's an acceptable workflow or not.

[masterfalcon]

If you just add to the list don't you have to re-run CMake anyway? That's always been my experience.

[CABAListic]

If any of the CMake files are changed, CMake reruns automatically, so you can't forget. That's the difference basically

[scrawl]

Exactly; if a file is added to the list, cmake will re-run automatically the next time you type "make". That is not the case when using file globbing.

[masterfalcon]

Ah ok. My instinct has been to just manually re run it.

[Klaim]

Mine too now because the re-cemake thing don't seem to work anymore for me, but I have no idea why (for non-Ogre projects)

[Klaim]

Would the new CMake 2.8.11 features allow Ogre CMake scripts to be a bit lighter and manageable after some related refactoring? See:
http://www.gamedev.net/page/resources/_ ... rt-5-r3182

[themean]

Why don't you use scons. Its extremely powerful and easy for learn.
http://www.scons.org/

[PhilipLB]

Hm, would be uncomfortable for at least the Windows user as they had to install Python first.