Page 1 of 2

Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 4:53 am
by LabMonkey
Hello,

I have been experimenting with CoApp (http://coapp.org) autopackage powershell tools for creating native nuget packages. For the purpose of my own applications the use of nuget packages for ogre binaries could simplify the setting of Visual C++ project properties across multiple workstations without including the binaries themselves in source control. As a result of this experimentation it has occurred to me that the ogre project itself could potentially benefit from this endeavor and I would like to invite discussion here regarding the matter.

For those unfamiliar with nuget and the coapp tools, I will attempt to breifly describe their utility here.

NuGet functions as a package management utility for development projects. Most often this occurs in the form of packages containing .NET assemblies to be referenced by other .NET projects. The NuSpec NuGet package specification allows for packaging multiple runtime versions of the packaged assemblies (e.g. net40 for .NET 4.0, net45 for .NET4.5, wp7 for Windows Phone 7, etc.).

In short, when adding a dependency package to a project that package is extracted (.nupkg is basically a zip archive) to $(SolutionDir)\packages\packagename-version (e.g. $(SolutionDir)\packages\EntityFramework-6.0.2). Managed projects within the solution which must reference this library then have project references added by relative path to the platform-appropriate assemblies within that directory.

Additionally, the spec allows for packaging native assets. For Visual C++ projects it's not simply a matter of pointing the vcxproj to the DLL(s) you want to reference; you also need include path(s) and lib files for linking. For multiple platforms and build configurations these settings can be tedious to introduce and maintain on a single development machine, let alone some workstation-agnostic path specification for an entire team of developers. NuGet handles this by utilizing Visual Studio props and targets files for native packages. However, creating and maintaining these props and targets files can be quite tedious as well.

Enter CoApp...

It is my understanding that the CoApp project originally came to exist in response to a desire among certain parties within Microsoft to have Windows-compatible open source libraries available as NuGet packages. This would allow a developer using Visual C++ to consume openssl, zlib, etc in their projects with only a couple of mouse clicks, rather than the grind of manually configuring project properties.

The CoApp Developer Tools set of powershell utilities and the autopkg file specification greatly enhances the packaging process for native NuGet packages. With a single autopkg file we can create multiple packages for a library, separating the runtime DLLs, development headers and lib files, debugging symbols, sources, samples, and so on. Additionally, the autopkg format supports "pivoting" on conditions such as debug/release configuration, x86/x64/arm platform, desktop/winrt runtimes, Visual Studio version (for msvc runtime versions), and any custom pivots we can choose to define.

So how does this matter to Ogre?

I am currently working to produce these CoApp-built native NuGet packages to reference in my native projects. However, I do not believe that it would be appropriate for me or my organization to publish these packages to a publicly available NuGet package feed as I am not a member of the official Ogre Team. I would like to work with members of community and the team to create a process by which the packages can be produced and published automatically by the team, for use by Visual Studio users. Additionally, for those who modify and build their own customized ogre source locally, these autopkg scripts could easily be made configurable (as part of cmake perhaps) to enable publishing your own custom ogre packages to your own custom package feed.

What is required to implement all of this stuff?

I would like to create autopkg files for the complete Ogre feature set which is available on Windows. It is my belief that packages should be created on feature boundaries (e.g. OgreMain, OIS, RenderSystem_Direct3D11, Plugin_Foo, etc.) This will allow API consumers to install only the packages containing the features they want. Ultimately, it may be useful to have the cmake-generated visual studio projects consume at least some of the dependencies in ogredeps from packages already available on nuget.org or other such feeds.

Where I currently most need assistance in this endeavor is with:
[*] Establishing feature boundaries. Which includes, libs, DLLs belong to which features?
[*] Designing and creating build automation for packaging. This could be particularly challenging if the community desires packages to be built for multiple versions of visual studio. It is at least challenging to build for multiple platforms as even for a particular version of Visual Studio cmake only generates projects for one of x86/x64/arm. So packaging binaries for all three targets requires three passes of cmake and build (both debug and release).

Man that's a lot of words..

If you've gotten this far I appreciate your patience. This effort seems to me one which could provide considerable value to the community if implemented properly. Not only can arbitrary developers include ogre in their projects via the nuget package manager, but organizations customizing ogre could consume updates easily while avoiding sticky subjects like DLLs in source control or network drop locations.

At this time I would like to invite comments and discussion on the subject.

Thank you for your time and attention.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 2:14 pm
by jacmoe
Since Ogre uses CMake, perhaps that could be used?

You are welcome to embark on this endeavor but I don't think there are many Ogre developers actually messing around with Visual Studio solutions/projects.
And the .NET framework is not used by Ogre at all.

I am more interested in Biicode - considering that the build system Ogre use is CMake.

<edit>
I could be wrong, though. :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 2:54 pm
by c6burns
It almost seems like the same situation as asking for up to date packages for linux distro X, Y or Z. I'd rather see the team members' work go into the library itself rather than maintaining packages for package management system X.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 2:56 pm
by frostbyte
I am more interested in Biicode - considering that the build system Ogre use is CMake.
imho, ogre having it's own package manager could work wonders on the ogre eco system...( open source answer to assets store? )
already suggested to use somthing like this: https://github.com/iauns/cpm as this one looks like it requires minimum effort in changing cmake scripts

i bet there are many solutions, but i agree on one thing, downloading/cloning, configuring and building ogre/dependencies/plugins/apps should become dead-simple and error proof...

a bit about binary package managers( and why i don't approve the idea ):
while some people kindly make an effort and maintain public packages for the community
http://www.ogre3d.org/forums/viewtopic.php?t=69274
http://www.ogre3d.org/forums/viewtopic.php?f=1&t=81770
this solutions are not official and do not really solve the problem of easily building and maintaining what you want by yourself....
while prebuilds may off-load lots of work, they are kind of missing the whole point of open source which is building stuff from source-code ...

what i have in mind is some tool where i can just mark packages that i want from a list of updated packages, choose target arch, and then hit-"make"(my day)
this wonder tool will download all packages from source-control then build and install them for me with no errors...

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 3:56 pm
by frostbyte
short version: cmake-git/hg based package manager for ogre official and 3rd party side will be cool and cross platform 8)...

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:00 pm
by Zonder
jacmoe wrote: And the .NET framework is not used by Ogre at all.
.NET has nothing to do with the packages themselves you find packages on there which are just javascript as well for example. The whole point it it works great for visual studio and Microsoft backs it so it's not going anywhere. (It also works on linux and macos but that's more to do with the mono .net side of things and I haven't looked at that my self)

I have actually mentioned this when it first appeared a while back it seems a obvious way to go for the windows platform.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:04 pm
by jacmoe
Oh, I misunderstood what it was all about- kind of like automated package installation for Windows.
Like Vagrant and Docker and Puppet but for the Windows family?

Frostbyte: Biicode is like composer/bower/npm/gem but for C/C++ projects.
That would make sense for Ogre since it's middleware. Biicode would pull down Ogre and all of it's dependencies in one go.
And it's open source too.
I can see the benefit from that.
Especially since I am using CMake everywhere I go. :)
I would love to have Ogre and Ogre related stuff registered with Biicode or similar.
Because I have grown to like package managers for software development when doing web work: I just don't have worry about my dependencies. I don't even add them to my dev repository, only the composer/bower file. The rest is simply generated. And I don't have to worry about hunting down deps. :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:11 pm
by Zonder
jacmoe wrote:Oh, I misunderstood what it was all about- kind of like automated package installation for Windows.
Like Vagrant and Docker and Puppet but for the Windows family?
No it's more like bower and npm etc.

biicode does look interesting it's the IDE support that really matters with these things but it says VS is supported (and eclipse)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:13 pm
by jacmoe
NuGet is the package manager for the Microsoft development platform
I know that Microsoft has gotten romantically inclined towards Linux, but Redmond did indeed invent the NIH syndrome, and it will probably be a while before they stop being so proprietary. :)

@Zonder: Biicode is a layer on top of CMake so it supports everything that CMake does.
And I agree: I started my previous post week, but I managed to end it strong :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:24 pm
by jacmoe
You can actually do many of the same things using CMake alone, but Biicode automates it. :)

I do like the idea of Nuget packages for Ogre, but I don't know.. If you are using Visual Studio and Ogre with it professionally, then why not register and upload some packages to the Nuget repository?

The reason why the Ogre team decided to switch to CMake was to stop having to support a gazillion build scripts for a gazillion build tools and development environments..
I can't really see why they should start and officially support Nuget packages.

So, if you want it, you have to do it, LabMonkey.
That's the reason why we have open source projects and communities around them: to scratch our own itch :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 5:28 pm
by jacmoe
Microsoft is scratching their own itch by letting us have the full Visual Studio for free, and doing what they can to get us hooked.
Nuget is part of that sinister plan :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 6:27 pm
by frostbyte
Because I have grown to like package managers for software development when doing web work
well jacmoe, maybe you're back just in time to wrap ogre-emscripten into ogreJS
while ogre-emscripten is very very....very cool for c++ devs...
ogreJS can potentially open the doors to a whole new community of web devs...
i started building ogreJS repl... idea is similar to shaderTOY https://www.shadertoy.com/ but based on ogre, eclipse orion( java-script repl ) and github
i call it OgreTOY 8)
but being realisic i'm no web master, i have other project and i'm too lazy :lol:
so i'm sending this idea free...
You can actually do many of the same things using CMake alone, but Biicode automates it. :)

while i was touring around for cmake package manager - i discovered cmake external modules...but they seem to be a bit complicated...
i also stumbled upon Biicode but for some reason, don't realy remember why i decided that https://github.com/iauns/cpm is cooler 8)
That's the reason why we have open source projects and communities around them: to scratch our own itch :)
yeah, no need to wait for team approval - you can put t on the web today - whoever like it will use it...

regarding official nuget support, i don't think this will pull-off since its only one platform/ide while ogre is cross platfrom( mobile/web/osx/linux/amiga? )
multi ide( msvc/ eclipse/code-blocks/vim? ) but anyway you should wait for the official refusal and reasoning from ogre-team...

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 6:45 pm
by jacmoe
CPM looks really cool, I agree. :)
Even though I like CMake (no secret), I think Biicode sounds a bit too intrusive, since I prefer full control of my CMake scripts. However, they seem to be backed by some major parties, including the team behind CMake..
I like the spirit of CPM, though. Very Packagist-like. :)

What I am really looking forward to, though, is when KDevelop finally gets released for Windows. Soon: https://www.kdevelop.org/news/kdevelop-470-released
KDevelop 5 will probably make me abandon Visual Studio, even when they're giving VS away for free. ;)

The only thing I really want from Microsoft is their development tools (compiler/linker/debugger) - they are awesome, on Windows.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 6:51 pm
by jacmoe
I think Neoaxis would be a much, much better fit for a Nuget effort :)
It would mean that they don't have to work too hard to finally be able to go Linux.
Visual Studio for Linux is just a matter of time. Probably not a CE, though.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:07 pm
by LabMonkey
jacmoe wrote:You can actually do many of the same things using CMake alone, but Biicode automates it. :)

I do like the idea of Nuget packages for Ogre, but I don't know.. If you are using Visual Studio and Ogre with it professionally, then why not register and upload some packages to the Nuget repository?

The reason why the Ogre team decided to switch to CMake was to stop having to support a gazillion build scripts for a gazillion build tools and development environments..
I can't really see why they should start and officially support Nuget packages.

So, if you want it, you have to do it, LabMonkey.
That's the reason why we have open source projects and communities around them: to scratch our own itch :)
I'm not at all opposed to creating packages myself. In fact I'm already working on that for my own projects. :)

However, if I push my ogre nuget packages to the nuget.org feed I wouldn't expect anyone to actually use them in the same way that I would be *very* hesitant to use a package built by some other non-official source.

That said, even if the official team doesn't want to build and publish official packages, having the option available for the user to automatically build the packages and publish them to some custom/internal/local package feed as part of the build process could be useful to users. I know it is expected to be useful on my end.

As mentioned earlier, aside from feature boundaries (which is just a matter of determining which files apply to which features), I'm still scratching my head as to how to get cmake to configure for x86/x64/arm simultaneously so that we only need to run cmake once, compile the solution for all selected platforms and configurations, then autopackage once. Admittedly, I'm pretty new to cmake so this may just be a matter of reading more docs.

Thanks for all the feedback thus far.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:13 pm
by jacmoe
I am not aware of how much work supporting Nuget would be.

But the fact that Ogre packages for Debian and Redhat are maintained by the Ogre community, and not the Ogre team (AFAIK)..
Linux packages seem to be laborious to maintain.
How is Nuget in comparison?
If it is almost zero work, then chances are much better that someone from the Ogre Team decides to take on support for that. :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:15 pm
by jacmoe
The CMake scripts was initially a community effort - read: one guy (who later joined the team) and it could be the same with this, maybe.

But, the thing is: you can actually trust the Ogre Community. :)
Especially the Debian packages are rock solid.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:21 pm
by frostbyte
Visual Studio for Linux is just a matter of time. Probably not a CE, though.
i still remember telling people to invest in microsoft and their vision of a unified os for pc tablet and mobile...
we all saw how that turned out...so, i'm not holding my breath...
microsoft and open-source? :shock: ... come on...i'm sure they'll find a way to screw this over and over...

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:27 pm
by jacmoe
LabMonkey wrote:As mentioned earlier, aside from feature boundaries (which is just a matter of determining which files apply to which features), I'm still scratching my head as to how to get cmake to configure for x86/x64/arm simultaneously so that we only need to run cmake once, compile the solution for all selected platforms and configurations, then autopackage once. Admittedly, I'm pretty new to cmake so this may just be a matter of reading more docs.
If you fire up Cmake-GUI, then you probably need to run it 3 times, one for each build-chain.
I do this for 32bit and 64bit versions of Ogre on my machine.
Just make sure that you always use out-of-source build directories.
Like:
ogre
ogre_build
ogre32_build

The source directory stays the same. That's one of the main features that I really like: no pollution of my source. :)

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:31 pm
by jacmoe
frostbyte wrote:microsoft and open-source? :shock: ... come on...i'm sure they'll find a way to screw this over and over...
You could be equally shocked about KDE/KDevelop actually making an effort to support Microsoft Windows. :)
Times are changing.
Visual Studio for *nix is the next logical step now, after opening .NET - MS has realized that people are increasingly using other OSs than Windows in their daily living.
And developers are migrating.
It would be wise of them to meet the developers wherever they are, even if they are on Linux/Mac OS X..

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:33 pm
by N0vember
jacmoe wrote:CPM looks really cool, I agree. :)
Even though I like CMake (no secret), I think Biicode sounds a bit too intrusive, since I prefer full control of my CMake scripts. However, they seem to be backed by some major parties, including the team behind CMake..
I like the spirit of CPM, though. Very Packagist-like. :)
CPM looks cool, there is just one think I don't really understand in its documentation. It says it wraps the classes in some kind of namespace. What's about that ? I don't really understand, usually a well written library already uses a namespace.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:38 pm
by jacmoe
CPM seems to use the same PSR autoload mechanism like composer:
\<NamespaceName>(\<SubNamespaceNames>)*\<ClassName>
For example, if you were to submit a CPM package, it would be namespaced like november/wonderlib.
Kind of like Java libraries or Python packages, I guess.
Just to make sure that two people are not accidentally putting up a package by the same name. And also make it easier to determine what packages are related.

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:52 pm
by frostbyte
You could be equally shocked about KDE/KDevelop actually making an effort to support Microsoft Windows. :)
i am.... :shock:
i guess kde folks realized that people are increasingly using other OSs than Linux in their daily living.
It would be wise of them to meet the developers wherever they are, even if they are on Windows :lol:

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 7:55 pm
by N0vember
Jacmoe >
This snippet will automatically download, build, and link version 1.0.2 of a simple axis aligned bounding box implementation named aabb. A new namespace is generated for aabb and a preprocessor definition for this namespace is automatically added to your project. The preprocessor namespace definition always follows the form CPM_<NAME>_NS where <NAME> is the capitalized first argument of your call to CPM_AddModule.

For example, in the 'aabb' snippet above, the first argument to CPM_AddModule was "aabb" so the preprocessor definition CPM_AABB_NS would be added to our project. This declares the namepsace under which CPM has bound the 'aabb' module. For instance, aabb's only class is named AABB. So the fully qualified name for the AABB class would be: CPM_AABB_NS::AABB. You may want to rename the namespace to something more appropriate: namespace glm_aabb = CPM_AABB_NS;. But that's entirely up to you. Depending on your needs, using the CPM namespace as-is may be all you need.
That's from the documentation. I believe it is about the C++ classes, and wrapping the classes in a C++ namespace (CPM_AABB_NS::AABB).
This doesn't really make sense to me :( But that's probably because I'm a noob at package managers

Re: Visual Studio nuget support for ogre/ogredeps

Posted: Mon Jan 26, 2015 8:02 pm
by jacmoe
Ah, yes. I just read that.
One of the goals is to support using multiple variants of the same lib and the namespace thing will make that possible.
Because C/C++ is compiled and linked, not dynamic like Node, Python, etc..
I think that feature can be turned off.
I see CPM more like a hacker CMake toolkit for your own use, at least in it's current state.
I also have seen that the developer has been hi-jacked by a competing project - check out the closed issues :)