Ogre 1.10 fork on github

[paroj]

the ogre 1.10 branch currently seems to be very stale. Most of the commits happen on the 2.1 branch by dark_sylinc - even though he also accepts merge request to default from time to time.
However he does not maintain the mobile ports.

This lead to the situation where some of my patches are hanging around for 7 months already - without any progress happening.
Furthermore there was talk about setting up a CI build which would improve the code stability/ merge ratio drastically - but unfortunately it went nowhere.
Part of the problem here was that there CI solutions for Bitbucket are rare. Furthermore IMO Bitbucket is generally falling behind as a code hosting platform.

To generate some drive I created a Ogre fork on github which brings the following advantages:

• github being more popular should lead to more drive-by contributions
• better integration of issue tracking and the Wiki
• continuous integration using Travis. Currently only a Ubuntu 12.04 build is done but Android should be easily possible
• continuous testing: Tests are being run for every pull request. (fixed tests so everything is passing)
• github/ git is more flexible in regard to history rewriting which leads to cleaner pull requests

you can find the fork here: https://github.com/OGRECave/ogre.
My focus is the default branch right now - even though I also migrated v2-0 and v2-1 as a proof of concept.

To keep the spot on the above issues I plan on accepting pull requests for the fork and merging with Bitbucket.
However I would rather like the whole project being migrated to github. There could be a TorusKnot group consisting of the current maintainers which would serve as an umbrella for Ogre and Ogre related projects like Meshy and Ogitor.

The changes so far are:

• fixed all tests to pass on Linux
• switched documentation to markdown so we get a friendly landing page
• merged my non controversial PRs: #477, #520, #571, #573

[dark_sylinc]

Many of the PRs affect many files for me to checkout, that's why I didn't merge them. I did just merge one that was very trivial though. Like you said I'm focused on 2.x branches.

As regarding Github:

• We do have a build server. We were just lazy to keep it running. Masterfalcon was taking care of it, but we kinda got lost in other stuff. Don't know if github is any better at this. I don't know if it can test PRs though.
• I do like Github ticket systems more than JIRA/Bitbucket. But TBH I'm finding forums more versatile when addressing bugs.

My biggest problem with github is... git. I've had horrible experiences with it. The GUIs are really bad (except for SourceTree, but ST is slow; you're all alone when ST fails, and it doesn't allow to use all the tool's capabilities; also no decent Git GUI for Linux), Git often leaves the repo in a broken state for no reason, it exposes low level mechanisms of the versioning phases that are not necessary, deals with EOL very poorly, it is very slow for parsing large repos, and works like crap on Windows (the Windows bugs are countless!).

[paroj]

Many of the PRs affect many files for me to checkout, that's why I didn't merge them.

this is exactly where a automatic build for PRs would help. You could directly see whether the PR builds and whether the tests still pass.

My biggest problem with github is... git. I've had horrible experiences with it. The GUIs are really bad (except for SourceTree, but ST is slow; you're all alone when ST fails, and it doesn't allow to use all the tool's capabilities; also no decent Git GUI for Linux), Git often leaves the repo in a broken state for no reason, it exposes low level mechanisms of the versioning phases that are not necessary, deals with EOL very poorly, it is very slow for parsing large repos, and works like crap on Windows (the Windows bugs are countless!).

I have come to the conclusion that Eclipse (egit) is the best available Git GUI. Sure it is awkward to run eclipse just for this task, but it really exposes all of the git features (interactive rebase/ commiting only chunks of a file) through the GUI.
Furthermore being purely written in Java it works the same cross-platform. The operations are slower than using native git - but generally it is fast enough and sureley faster than ST.
http://eclipse.github.io/

[TheOnlyJoey]

I am personally opposed to forking.

Though I am personally not a fan of Mercurial and host a couple of projects on Github myself, I strongly disagree that Github generates more contributors.
If a working system is in place and people are genuinely interested in a project, they will use the system it is hosted on.
If signing up and using a system you are not already using is to much hassle, I don't think we have lost a contributor.

I have contributed a couple of commits myself to Ogre, had to learn to use Bitbucket, sign up for JIRA and I don't mind that, last thing you want to do is fragment your developers and community.

Though I prefer git and Github over Mercurial and Bitbucket, I rather see more people getting into the core Ogre team, more code reviewers, more branch maintainers, which is all doable without moving to a entire different hosting platform.
(Also I dislike the use of GUI tools for things like git/mercurial/svn but that is a different matter)

[scrawl]

I haven't used Ogre in a while, but I'm watching the development out of interest. I agree the maintenance of 1.10 and the general handling of pull requests has fallen into disrepair a bit. If the current Ogre team does not have the time or expertise to merge such requests, perhaps its time to take on new team members? paroj has been contributing for a long time and seems competent enough as a maintainer. If you do not do anything about the current situation, it's only natural that more forks start to pop up.

[dark_sylinc]

I haven't used Ogre in a while, but I'm watching the development out of interest. I agree the maintenance of 1.10 and the general handling of pull requests has fallen into disrepair a bit. If the current Ogre team does not have the time or expertise to merge such requests, perhaps its time to take on new team members? paroj has been contributing for a long time and seems competent enough as a maintainer. If you do not do anything about the current situation, it's only natural that more forks start to pop up.

That is a good point. Raised it to the other team members.

[paroj]

Though I am personally not a fan of Mercurial and host a couple of projects on Github myself, I strongly disagree that Github generates more contributors.

the point of github being more popular is probably the least important one.

the important ones are

1. you directly see which PRs build and pass all tests
2. you get cleaner PRs because of rebase/ squash/ fixup git history rewriting operations
3. the issue tracker is fast and right where you expect it

in other news I migrated all issues from JIRA to github. Furthermore there is http://hg-git.github.io/ if you like github, but prefere hg.

[mkultra333]

I really hope 1.10 is kept as up-to-date and bug fixed as possible. I'm very likely to update my own project from 1.8 to 1.10 soon, just for the DX11. I've been working on my project for 6 years part time, it would be way to much work and headache to update to 2.1.

It'd suck if there are a bunch of important fixes for 1.10 just laying around unused because there's no one to properly patch them in, especially if they improve or fix bugs in DX11 support. So I'll take 1.10 support wherever I can get it.

[TheOnlyJoey]

Though I am personally not a fan of Mercurial and host a couple of projects on Github myself, I strongly disagree that Github generates more contributors.

the point of github being more popular is probably the least important one.

in other news I migrated all issues from JIRA to github. Furthermore there is http://hg-git.github.io/ if you like github, but prefere hg.

All this time put in something that might not fall through, while it also could be spend on actual development.
The whole 'which hosting for our project' all seems bikeshedding to me, we have something that works, lets keep it working.

Again, as a maintainer of multiple Github and projects on different hosting sites I can conclude that there are no benefits of github being 'more popular', the same point I brought last post still stands, people who want to contribute, will contribute, regardless of where the code is hosted.

I really hope 1.10 is kept as up-to-date and bug fixed as possible. I'm very likely to update my own project from 1.8 to 1.10 soon, just for the DX11. I've been working on my project for 6 years part time, it would be way to much work and headache to update to 2.1.

Well 1.10 is basically a bugfixed 1.9 with a more up to date GL driver, better android support, basic HLMS implementation and is quite stable. All that 1.10 could benefit from is fixing papercuts (where papercuts are found) but mostly feature implementations.
We use 1.10 internally and has not given us any problems yet.

I am for allowing new team members and repository maintainers.

[al2950]

All this time put in something that might not fall through, while it also could be spend on actual development.
The whole 'which hosting for our project' all seems bikeshedding to me, we have something that works, lets keep it working.

Could not agree more.

Also although I have no opposition to Git, and I do use it a fair amount, Git on windows has annoyed me on too many occasions!

[paroj]

All this time put in something that might not fall through, while it also could be spend on actual development.
The whole 'which hosting for our project' all seems bikeshedding to me, we have something that works, lets keep it working.

part of this whole thread is that actual development is blocked by the pull request throughput. The pull request throughput is in turn limited by the maintainers checking out and testing stuff. And even then bad PRs come through because often the PRs are tested only on a single platform/ RS. (fixed a couple of those already)

github being more popular means that we can use travis-ci which simply does not work with bitbucket.

I also set up a MSVC build on appveyor, but because of MSVC awesomeness it hits the 60min quota just compiling ogre, so we really need travis to run the tests.

[TheOnlyJoey]

The maintainer thing can be solved, allow more maintainers!
There are a lot of people using Ogre for years (including me and others in my company) who are happy to help out with managing if asked.

github being more popular means that we can use travis-ci which simply does not work with bitbucket.

If the whole point is about continues testing, there are a lot of alternatives to travis to do building and unit testing.
I run buildbot's for our own software at the office and would be up for setting up a builtbot for Ogre if that would help development out.
Let me get in contact with dark or holocron and I will set this up soon.

[Wolfmanfx]

@TheOnlyJoey

Its kinda sad with CI / CD - we invested cash into dedicated machines for CI but its extermly time consuming. I already looked into payed systems (Jenkins + Docker for example) but for that we would need cash (For the hosting in the cloud) / time. Maybe you have interest to jump on that topic? Maybe you already have something like that for your project? Also bear in mind that David invested already many hours with bamboo (which did not worked out that great) and switched to Jenkins.
If we could automate let say 70 - 80% with automated test to catch fishy PR it would already help.

Regarding the fork - it would make sense to have a github mirror not a fork a 1:1 copy - than ppl could more easily integrate OGRE as subrepo when they are using git.
But there is a lot room of improvment.

[paroj]

@TheOnlyJoey

Its kinda sad with CI / CD - we invested cash into dedicated machines for CI but its extermly time consuming. I already looked into payed systems (Jenkins + Docker for example) but for that we would need cash (For the hosting in the cloud) / time. Maybe you have interest to jump on that topic? Maybe you already have something like that for your project? Also bear in mind that David invested already many hours with bamboo (which did not worked out that great) and switched to Jenkins.
If we could automate let say 70 - 80% with automated test to catch fishy PR it would already help.

the fork does CI on Windows/ Linux for free and already integrated into the PR workflow right now - just saying..

Regarding the fork - it would make sense to have a github mirror not a fork a 1:1 copy - than ppl could more easily integrate OGRE as subrepo when they are using git.
But there is a lot room of improvment.

you can also create git repositories on bitbucket if all you want is to allow a git mirror. The fork is there to demonstrate the advantages of moving away from bitbucket.

[Wolfmanfx]

As I read you are hitting time constraints with the servers (may I miss read it). Also I found some cloud soultions which provide a free tier until you hit a certain allowed porcessing time.

Got you now regarding bitbucket - I thought you want to move away from hg.

[c6burns]

Does Atlassian not provide free full licensing to open source projects still? https://www.atlassian.com/software/view ... e-request/

Obviously there is a problem, which has been brought up by paroj. It seems like everyone is in agreement that this problem exists; however, I think the proposed solution of forking is not a great solution. But this is exactly what happens to projects in this scenario.

The project is already using Atlassian for issue tracking and code versioning. Just get Atlassian to pony up Bamboo licensing for CI if you want CI. I don't understand why anyone from the Ogre team would be paying cash for licenses.

EDIT: Sorry I misread what wolfman was saying. It was machines for build agents that got bought I guess. Still I think the whole "bitbucket is falling behind" is not a useful train of thought. The problem as I understand it is that no one is looking out for 1.x branches ... the problem is not that the project uses bitbucket, though if dropping it makes sense then obviously that's fine

Man I have to stop editing this ... but final thought. I guess if paroj is volunteering to maintain 1.x then I have no opinion on where code is versioned, where issues are tracked, or anything else. I'm not willing to maintain 1.x branches and so if paroj is then that is super awesome and I am happy to hear it. Not that my opinion matters anyway ... I'm just an end user of the current default branch

[paroj]

As I read you are hitting time constraints with the servers (may I miss read it). Also I found some cloud soultions which provide a free tier until you hit a certain allowed porcessing time.

yes, this is a problem. However I was able to workaround it by building less of ogredeps so build times are around 40mins with MSVC now. (gcc build needs only 16min) Yet sometimes the build still fails because of timeout.

Got you now regarding bitbucket - I thought you want to move away from hg.

I indeed prefer git to hg because it encourages writing clean commits and commit messages. For example compare:

https://bitbucket.org/sinbad/ogre/pull- ... rs/commits and
https://github.com/paroj/ogre/pull/165

they contain exactly the same changes. However on git I could easily regroup and reorder the changes s.t. they are easier to follow.

[Kojack]

Hmm, that Appveyor website says "Blazingly fast builds on dedicated hardware with SSD drives". But my laptop can do a full 64bit release build (using visual studio 2013) of ogre 1.10, all deps, all samples (72 projects total) in just 5min 21sec.

[c6burns]

My 4 years old thinkpad can build everything in under 40 min so yeah ... it's hard to imagine what must be going on with that build hardware.

My new bottom of the line thinkpad builds everything in around 10 min in VS 2015

[TheOnlyJoey]

I have a buildbot script ready to do continues building and testing, my idea is to get a VPS to host the buildbots and integrate it with the Bitbucket.

For buildbot, there is almost zero to none maintenance required, we host buildbots for our own projects for 3 years now and I am ok with maintaining them.

[paroj]

Hmm, that Appveyor website says "Blazingly fast builds on dedicated hardware with SSD drives". But my laptop can do a full 64bit release build (using visual studio 2013) of ogre 1.10, all deps, all samples (72 projects total) in just 5min 21sec.

probably this is meant like: "you can get fast builds if you pay us". Appveyor might be slow, but it is free, which your laptop was not

meanwhile i could extend the build matrix to

• Ubuntu build: gcc4.6, GL Renderystem running the non-visual tests
• Ubuntu build: gcc4.6, GL3Plus and GLES2
• OSX build: Xcode (clang), GL3Plus
• MSVC 2013: D3D11, GL3Plus
• Android NDK10e: gcc4.9 GLES3

see the build results here

[Transporter]

Its kinda sad with CI / CD - we invested cash into dedicated machines for CI but its extermly time consuming. I already looked into payed systems (Jenkins + Docker for example) but for that we would need cash (For the hosting in the cloud) / time. Maybe you have interest to jump on that topic? Maybe you already have something like that for your project? Also bear in mind that David invested already many hours with bamboo (which did not worked out that great) and switched to Jenkins.

I've set up a Jenkins system a few month ago, but I had to cancel it because the server was not strong enough. But there is hope for 2016. If possible I'll set up a Jenkins for Ubuntu, VS2013 and VS2015. The main problem on a CI are the dependencies.

Regarding the fork - it would make sense to have a github mirror not a fork a 1:1 copy - than ppl could more easily integrate OGRE as subrepo when they are using git.
But there is a lot room of improvment.

That's a good idea!

[paroj]

updated build matrix:

• Ubuntu: gcc4.6, GL Renderystem running the non-visual tests
• Ubuntu: gcc4.6, GL3Plus and GLES2
• OSX: Xcode 7.2, GL3Plus
• iOS 9.2: Xcode 7.2, GLES3
• MSVC 2013: D3D11, GL3Plus
• Android NDK10e: gcc4.9 GLES3

see: https://travis-ci.org/OGRECave/ogre/builds/99829813

however the codebase has diverged now, as Assaf and Lior pushed crap and broke the GL Rendersystems in several ways.

[paroj]

related news: https://lwn.net/Articles/669924/rss

Obviously from these numbers Git is by far the more popular DVCS for open source projects and choosing the more popular VCS has a number of positive benefits.

For new contributors it increases the likelihood that they will have already learned the basics of Git as part of working with another project or if they are just now learning Git, that they'll be able to take that knowledge and apply it to other projects. Additionally a larger community means more people writing how to guides, answering questions, and writing articles about Git which makes it easier for a new user to find answers and information about the tool they are trying to learn.

[paroj]

after quite some effort, the PlayPen Tests are now running on CI:

https://travis-ci.org/OGRECave/ogre/builds/101256510

However the generated images are not checked yet as there is no persistent storage between CI runs. So this basically only checks that none of the VTests crashes.

Yet as the VTests are stable now we can assess the current state of the GL3+ RS:

https://ogrecave.github.io/ogre/gl3plus_status/

note that that some of reference images are broken in the GL RS.