Linux distro testing issues

[jacmoe]

I don't think it's such a big deal.
I mean: there's 3-4 flavors: Debian, RedHat, Arch and Gentoo.
If you want to support Ubuntu, perhaps it would be a good idea to first make a Debian compliant package install.
That could be ported easily to RedHat and customized for Fedora, OpenSuse.
Arch Linux is actually fairly straight forward as it uses simple build scripts. Gentoo probably the same (emerge scripts).

It would be foolish to not start with Ubuntu, of course. That's where the [s]people[/s] kool kids with more money than brains are.

I can't recommend actually using Ubuntu as your main development machine - but I probably shouldn't care less.

[bvanevery]

If you want to support Ubuntu, perhaps it would be a good idea to first make a Debian compliant package install.

Ogre already has an Ubuntu package created by somebody. Only a matter of time before someone creates one for 1.9. Not something I have to worry about right now; I'm not even committed to Ogre yet. I'll worry about installers for my actual game when I get closer to having one. Not like I haven't contended with 3 zillion build system horrors in the past....

It would be foolish to not start with Ubuntu, of course. That's where the [s]people[/s] kool kids with more money than brains are.

What money? I put it on a 6 year old laptop and it mostly worked. To the extent it didn't, I think it's a kernel problem with Dell laptop power management, not Ubuntu's fault. After pulling my hair out for a month and suffering serious productivity losses, wondering what the hell was wrong, I finally decided that hibernation problems were Linux's fault, not my HW's, not curable, and I'd just have to either sleep or shutdown. Sleep doesn't turn out to consume that much battery power overnight, it turns out.

I did settle on Lubuntu for performance reasons. If Canonical hadn't had the Lubuntu flavor available, I would have bolted for something else. Unity ran rather piggishly on my 6 year old HW. Lubuntu is crisp and I get better battery life than Vista. I did have to install the Jupiter power management thingy because LXDE didn't have anything good for managing a laptop that's typically being run off a deep cycle battery and a power inverter.

I can't recommend actually using Ubuntu as your main development machine - but I probably shouldn't care less.

I'm not seeing the problem. For package self-defense, I've learned to either download bleeding edge packages manually rather than enable untrustworthy repositories wholesale, or to compile sources myself.

[jacmoe]

I am basing my opinion on 3 years of running Ubuntu (after having been a Fedora/RedHat user before that), 2 years of Mint and 1 year with Arch and now - after a brief period where I used OpenSuse, I have been using Debian Sid for the last 2 years, and it simply is the most stable and most current system I've ever run.
Simply because I don't have to install/compile additional 'stuff' in order to be able to use the newest GCC, graphics drivers, Qt framework, latest CMake, Apache, PHP, ...
As a developer, I need the latest version of most things.
You simply can't get that with a stable/frozen edition. You need a cutting edge, rolling release distribution.
And kernels?
I get a kernel update almost weekly.
Also, 64-bit is important. Debian recently switched to multi-arch, which makes it so much easier.

If I want to test on a stable mass-consumer distribution (like Ubuntu), it's a virtual machine away.

That you are not seeing the problem, is not my problem.

[jacmoe]

Ogre has an official Debian package, which is current and actively maintained.
The Ubuntu package is not (AFAIK).

[bvanevery]

I think YDNK. The Debian package says 1.8.0 and so does Ubuntu. I see Ogre 1.8.1 coming from quantal-getdeb. So the latest is available, but not officially maintained or pushed in Debian or Ubuntu by default. Just a part of the Getdeb Games project.

[bvanevery]

I am basing my opinion on 3 years of running Ubuntu (after having been a Fedora/RedHat user before that), 2 years of Mint and 1 year with Arch and now - after a brief period where I used OpenSuse, I have been using Debian Sid for the last 2 years, and it simply is the most stable and most current system I've ever run.
Simply because I don't have to install/compile additional 'stuff' in order to be able to use the newest GCC, graphics drivers, Qt framework, latest CMake, Apache, PHP, ...
As a developer, I need the latest version of most things.
You simply can't get that with a stable/frozen edition. You need a cutting edge, rolling release distribution.

Maybe you can't for some reason, but I am not presently committed to big stacks of open source stuff where something breaks if someone sneezes. I just haven't seen any problem or crisis that requires the latest greatest for everything. When I've really needed something current, I've either found a PPA or compiled it myself. I did run into issues with various OpenGL wrapper and sample code projects, whose developers tend to hang out at opengl.org. I ended up giving up on all of them, as their latest greatest wasn't all that great. They just don't have enough labor going into those things to be reliable, and they don't fix the bugs I file. Checking out Ogre and ClanLib is something of a last hurrah. If they prove wanting, that will pretty much do it for me and open source. (Irrlicht has lost the OpenGL 3.x contract for now. They'll have something eventually, but presently they have nothing.)

I get a kernel update almost weekly.

Hasn't affected me much. Although, something weird is happening lately with sleep and shutdown. Sometimes my laptop won't terminate properly. Nothing that hitting the power off button won't cure. I chalk it up to hobbling along with a 6 year old Dell laptop. It'll be replaced sometime this year, so I'm not going to stress about it.

Also, 64-bit is important. Debian recently switched to multi-arch, which makes it so much easier.

That's not unique to Debian. Ubuntu does it as well. As a Debian derivative, why wouldn't it?

[holocronweaver]

Simply because I don't have to install/compile additional 'stuff' in order to be able to use the newest GCC, graphics drivers, Qt framework, latest CMake, Apache, PHP, ...
As a developer, I need the latest version of most things.
You simply can't get that with a stable/frozen edition. You need a cutting edge, rolling release distribution.
And kernels?
I get a kernel update almost weekly.
Also, 64-bit is important. Debian recently switched to multi-arch, which makes it so much easier.

I am genuinely surprised that a system that is updated so often remains stable. This sounds like what I was hoping to get out of Arch, with less configuring on my part. I am hesitantly intrigued.

As for keeping up with the latest greatest kernels, it is not clear to me why this is useful for game developers. I rarely experience kernel related bugs anymore and about the only time I get excited about a kernel update is when a new hardware platform is supported or updated (low-latency VR tech *drool*). Very, very rarely a kernel update will cause trouble with my AMD drivers, but usually this is fixed with a kernel update in Ubuntu within a few weeks. Would be nice if it was fixed in a few days though.

One of the reasons I like Ubuntu is because it keeps you reasonably up to date while witholding poorly tested patches. Major changes are saved for OS updates so they can mature and stabilize. Plus I can set aside one or two days a year to get all that busywork out of the way. Others bare the brunt of the technological turbulence and I hang a ways back to stroll through the gentle breeze.
The only exception to this is when I am working on patches for core desktop software (KDE and GNOME, primarily). In that case I will typically run a cutting edge distro in a virtual machine to do my work. Lately I have doing most of that in Fedora daily releases, but maybe I should consider Debian since Fedora daily is a bit unstable at times.

If I want to test on a stable mass-consumer distribution (like Ubuntu), it's a virtual machine away.

I prefer multiboot myself as I have encountered many game bugs that only pop up in a realworld hardware environment. Speaking of which, are you using VirtualBox? How is the hardware acceleration these days for game testing? Would be nice to not have to reboot so much during daily testing.

[jacmoe]

YDNK ? Tried to Google it, but it came up with nothing.
You Don't Know Nothing?

In my experience, the Debian package tends to be more current than the Ubuntu package.
However, none of them are official.

Ogre is one of the few things that I do build and install from source these days.

AFAIK, the Debian Ogre package is in the official channel. The Ubuntu package is not, IIRC.
So, if you are relying on the package being present in the package repositories of your users, Debian is going to be your best bet. Unless you're planning on shipping Ogre with your app.

[jacmoe]

I am genuinely surprised that a system that is updated so often remains stable. This sounds like what I was hoping to get out of Arch, with less configuring on my part. I am hesitantly intrigued.

I must admit that I am surprised as well.

But that is the truth.
I do a dist-upgrade at least once a week. After checking on the Siduction forum that there ain't any breaking changes. And, if there are, they are usually fixed within hours.
I think that's the reason: the Debian package maintainers are extremely active.

Arch is great, but gave me a lot of headaches, to be honest.

[bvanevery]

YDNK ? Tried to Google it, but it came up with nothing.
You Don't Know Nothing?

I created the acronym on the fly. You Do Not Know. In response to AFAIK.

AFAIK, the Debian Ogre package is in the official channel. The Ubuntu package is not, IIRC.

The Ubuntu package is in Universe, not Main. That's a subtle point I've forgotten about over time, as I enabled Universe pretty early on. So you're right, it wouldn't automatically show up in end users' package repositories. However, end users aren't all going to be Debian users either, so the point is moot. And if an Ubuntu user did want to receive it, it's in an official Ubuntu repository, just not Main. Of course as noted earlier, neither it or Debian is actually tracking the latest Ogre 1.8.1 version.

So, if you are relying on the package being present in the package repositories of your users, Debian is going to be your best bet. Unless you're planning on shipping Ogre with your app.

If present packaging realities hold, then I'd either have to ship Ogre with my game, or get involved with pushing current Ogre packages out to more places. There are also a gazillion underlying dependencies that could conceivably break. I am more inclined to perform the experiment of letting open source deal with that though, rather than pursue the Windows paranoia mentality of shipping everything myself. Hey, I built Ogre from source, but I did not build any dependencies. That's one thing I like about a Linux development environment compared to Windows!

[bvanevery]

I do a dist-upgrade at least once a week. After checking on the Siduction forum that there ain't any breaking changes. And, if there are, they are usually fixed within hours.
I think that's the reason: the Debian package maintainers are extremely active.

Um, I don't know what the underlying mechanism is on Ubuntu. Quite frequently, a little Software Updater dialog pops up. Something to do with updating my packages I think. Some of those are kernel updates. I rubber stamp whatever it's asking me to do, I never look at the details. This has never burned me. I think this pushing of updates, whatever the heck they are, is probably more reliable than what Microsoft does on Windows, and I did that blithely for years. Every once in a blue moon MS would burn me somehow.

There is simply not a stability or maintenance problem here. YMMV.

[jacmoe]

The reason why I would go with Debian first is that the Debian package guidelines are extremely strict and very detailed.
Dependencies must be installed separately, I think, or at least be prependended by your application/library name.
I am still looking into making Ogitor Debian ready. Everything must go into the right place in the file system.
In that regard, Windows is ridiculously easy. :p
So, if you are Debian ready, Ubuntu is going to be easier as it's not as strict (I think).

For example, with Ogitor, it's not all that easy to just install it's dependencies into the filesystem, if you are building a real package, that is.
Arch is much easier as you can instruct it to grab the dependencies, if not present, and compile them as part of the installation process..

[jacmoe]

Um, I don't know what the underlying mechanism is on Ubuntu. Quite frequently, a little Software Updater dialog pops up. Something to do with updating my packages I think. Some of those are kernel updates. I rubber stamp whatever it's asking me to do, I never look at the details. This has never burned me. I think this pushing of updates, whatever the heck they are, is probably more reliable than what Microsoft does on Windows, and I did that blithely for years. Every once in a blue moon MS would burn me somehow.

There is simply not a stability or maintenance problem here. YMMV.

I sense that you don't know the difference between dist-upgrade and a system update.
Yes, you will get maintenance updates, not upgrades. That is quite different.
Sid doesn't have updates, only upgrades.

You are lucky that updates haven't burned you yet in Ubuntu. It did that to me more often than what was comfortable. And then there is the inevitable distribution upgrades.. * shudder *.

[bvanevery]

I did Lubuntu 12.04 most of last year. Something happened around the time 12.10 came out, that for other reasons, caused me to nuke my system and start completely from scratch. I think I was still experimenting with various distros, and also hadn't settled on my Windows vs. Linux hard drive allocations. So, I ended up doing 12.10 on a clean install to my 2nd HD. I think that was 5 months ago.

[bvanevery]

One thing I'm realizing is it's hell in Linux-land to update drivers. Ubuntu is rather conservative in the drivers offered, with their NVIDIA "Experimental" driver being back at 310.14. The latest long term NVIDIA release is 310.40 and the latest short term release is 313.26. I can't just get these from the Ubuntu repository, I have to do the "official" NVIDIA shell script dance. I'm not sure how dangerous that is to my underlying system. It's not an operation I'd want to make a game customer perform, and it means I have to dust off my system imaging kung fu, which I've gotten rusty on over the past few months. I will not under any circumstances enable 3rd party repositories for mission critical components like video drivers, no matter how tempting from a convenience standpoint it may seem. The last time I did that, my whole system died. They were supposed to have some "back out" utility if things didn't work, and of course the utility failed, because all 3rd party repositories are run by rank amateurs.

I'm torn about trying to follow more recent drivers. They may fix problems, give me access to OpenGL 3.x features, and provide lead time for shaking out problems before I actually have a game to ship. But someday, when I get closer to shipping, I'm going to have to solve customer problems with already deployed drivers as a given. I can recommend a customer try an official Ubuntu driver, but I can't recommend they go through the NVIDIA shell script dance, let alone enable a 3rd party repository. I hope "something happens" in Linux gaming land between now and when I ship, that improves this infrastructure, because it's not a good situation and nothing like just blithely installing the "latest greatest" driver on Windows.

Some time later... got the system imaging done with Redo Backup and Recovery. I've tried several things in the past and this one seems to work. NVIDIA driver install wasn't so bad, just some command line magic. So now I have 313.26 to test.

[holocronweaver]

I can't just get these from the Ubuntu repository, I have to do the "official" NVIDIA shell script dance. I'm not sure how dangerous that is to my underlying system.

I personally have not had trouble with proprietary Nvidia driver installation since Ubuntu 10.04, both on my desktops and laptops. So long as you have a relatively recent card your experience should be smooth. Ironically, AMD proprietary drivers are easier to install (everything can be done using a graphical installer and your package manager - everything is updated with kernel updates automatically), but a bit buggier to use.

I'm torn about trying to follow more recent drivers. They may fix problems, give me access to OpenGL 3.x features, and provide lead time for shaking out problems before I actually have a game to ship.

I highly recommend keeping up with the latest, verified stable driver you can get. Recent attention Linux has recieved from game developers has resulted in a significant uptick in Linux driver performance competition between Nvidia and AMD. Hopefully we can contribute to the flame.

I hope "something happens" in Linux gaming land between now and when I ship, that improves this infrastructure, because it's not a good situation and nothing like just blithely installing the "latest greatest" driver on Windows.

This is mostly up to Nvidia and AMD rather than the Linux kernel developers, as detailed in this great article on improving driver installation in Linux. Both companies need to open source their drivers, or else the third-party open source drivers need to be given greater attention. Those are the two avenues forward and I agree with the Linux developers that supporting an ABI to ease closed source driver installs would be a bad idea. Linux is about open source and freedom - aiding those who want to build walls and turn Linux into yet another Windows clone is not conducive to progress in either direction.

Some time later... got the system imaging done with Redo Backup and Recovery. I've tried several things in the past and this one seems to work. NVIDIA driver install wasn't so bad, just some command line magic. So now I have 313.26 to test.

Glad everything is so far so good.

[bvanevery]

That article isn't great, it's from 2004. What are people talking about doing now? I'm going to have games to sell, and I don't care about trying to force AMD and NVIDIA to open source their drivers. They have their business reasons to want to keep them closed, to protect their advantages as they design 3d HW to compete with the other guy. If Intel makes progress getting the 3d job done with open source only drivers in the real world, that's great. That's called free market competition. But as long as AMD and NVIDIA provide substantially better performance with the tradeoff of being proprietary, the situation will persist. It would be nice if Nouveau and Wayland reverse engineered whatever AMD and NVIDIA have previously done, but they're playing catch-up and we all see they don't have the development resources to get there. Now Canonical has lost faith, if they ever had any, and are going their own way with Mir.

Generally speaking I don't believe in the GPL side of open source at all. I realize the FSF created a large body of software that was suitable for implementing a popular open source Unix, and that Linux with a GPL + linkage exception license became that Unix. But I do wonder if the same could have been achieved with MIT / BSD code. What gets people to do the work? The zealots believe people will only do the work if code is under the GPL. I wonder if we'll ever reach a "that was then, this is now" period of history where the GPL isn't so compelling anymore. Ogre for instance is the largest open source 3d engine project now, and is MIT licensed, but it started life as LGPL. Did the LGPL force things to keep getting plowed back into Ogre, or was it really about the people who organized the project and kept it going over the years? I suspect the latter. After all, Python and several other programming languages are perfectly big, and never embraced the GPL or LGPL as a development culture.

Meanwhile... just got my first unsolicited semi-crashes probably due to the 313.26 NVIDIA short term release driver. The window manager seemed to die but then recover. Happened twice in a row while using the Chromium browser, while I was typing an email in GMail. I switched to Firefox, don't know if that matters, but it hasn't died yet. I really don't feel like dealing with this right now, so if it keeps up, I'll try the 310.40 long term release driver. Both are still farther forwards than Ubuntu.

Well, I got impatient wondering if I'd screwed something else up, following various bits of bad advice about manually tinkering with stuff while in a hurry to look for solutions earlier. So, I thought I'd just try to revert back to Ubuntu's 310.14 Beta driver using the package manager. That didn't work because my kernel now has modules compiled against 313.26. I got a plain TTY login screen. Fortunately I remembered that's the way one installs from a .run file, so I just installed 313.26 again, going in a complete circle to accomplish absolutely nothing. This is the problem with all these different "masters to serve" for doing a driver install. Once you start doing .run installations, you have a lot of extra work and grief to get out of having to do .run installs anymore.

This is a consumer nightmare, and it's a PITA even for the tech savvy. How many times would I futz with a non-working system before I gave up and found something better to do with my time? That might be the end of selling games to that person on Linux. Depends on whether they were much committed to Linux in the 1st place.

[jacmoe]

The driver and kernel update race is handled automatically on my box.
My current driver is at version 313.26 from Debian experimental. The Siduction team does a great job of providing shims. No command-line magic here.
That was definitely one of the major head-aches that I had with Ubuntu..

[bvanevery]

Had any crashes attributable to 313.26? The grass isn't greener if it still crashes for the same reasons. I had another weird lockup, I say to hell with that driver. Dropped it back to 310.40, will see how that goes. Well, got one random lockup already, so I doubt the grass is much greener.

Hrm, a small issue. The NVIDIA .run installer is not multiarch-aware. It dumps libGL.so into /usr/lib instead of /usr/lib/x86_64-linux-gnu. This means my CMake configuration is invalid, which means I'll probably have to recompile everything, all from changing a driver. Oddly enough I bet this isn't a runtime dependency, but a compile time dependency. Probably can't wreck installed apps, but can wreck builds because CMake isn't being smart here. Wonder if that's CMake's fault or the CMakeLists.txt's fault.

[jacmoe]

No crashes.
I can't go back because of recent changes to glx and stuff.
No need to either.
However, I probably want to try out nouveau as some of you say that it's maybe faster than nVidia's proprietary driver.

[bvanevery]

However, I probably want to try out nouveau as some of you say that it's maybe faster than nVidia's proprietary driver.

Really? That would be news to me. A January Phoronix article says Nouveau is massively slower than NVIDIA proprietary.

[jacmoe]

The notion, I see now, is based on the benchmark from November last year, before Valve launched Steam for Linux and compelled the nVidia Linux crew to improve the driver.
So, I think I'll stick with the proprietary driver, for now. Thanks.

[bvanevery]

I have my doubts that the open source community can ever answer this problem unless NVIDIA or AMD willingly decide to go open source. I've had the 3d device driver job once upon a time. It's a boring job, to me personally, or at least the excitement ran out quite awhile ago. Looking at some job ads recently it was also a $140K/year job as an employee, nevermind the silly amounts of money one could conceivably make as a contractor. Given that level of money involved, for fairly dry work, how is open source volunteerism supposed to compete with that? A certain number of younger programmers may be willing to take it on, and they may make great strides for a year or two, but then the fun or challenge is likely to go out of it and there's nothing left but gruntwork. At which point they'll either find something more interesting to do, or go get paid those big bucks to do the dry stuff. There's just not much reason to live in career purgatory on something like this.

I don't think open source is capable of organizing itself in some amazingly competitive way absent large influxes of cash. I think the progress of Nouveau vs. the proprietary drivers is about what one would expect. There's progress, but the resources will never be at parity with the proprietary development. If there's some funding model for Nouveau or Nouveau-like projects that I've missed, well, that would be educational for me.

I can justify working on a 3D engine in an open source manner because I'm trying to be an indie game developer. I'll need some kind of engine anyways, and with my background, the temptation is to write a minimal one. But even so, I've been watching DX10, DX11, and OpenGL drag their feet for a good 6 years now. Thanks partly to the Vista fiasco it's taken a long time for better DX HW to make it into the wild. It took even longer for OpenGL standards to catch up, and open source is lagging even after that. Consider that OpenGL 3.0 was released 5 years ago and Ogre is only getting around to a renderer for it now. Not to cast aspersions on anyone's good work, but man, it's been a long time coming. You may notice my forum join date is from 2007 but I'm only talkative recently, well, there's a reason for that.

From this I think we can observe that open source lags seriously behind proprietary development. However in the 3D API arena it has not mattered in practice, since DX9 class HW has been a perfectly viable medium for games. The consoles have been holding things up, and much game development energy went into the mobile devices. The benefits of API modernization are not as straightforward as a driver benchmark. Can't just compare 2 numbers for framerates and call one product better than another.

Anyways, back to NVIDIA or AMD willingly deciding to go open source. Only thing I see making it happen is Intel, with their open source driver strategy. I don't know enough about Intel's 3D development to know that they can "come from behind" and reach parity with the proprietary drivers. It seems far more likely they'll follow the historical pattern, which is to serve the mass market of ICs. Maybe someday the ICs will be good enough for most people that the proprietary performance advantage doesn't matter anymore. I think that day is quite a ways off though.

[jacmoe]

The reason why the proprietary driver suddenly became so much better than Nouveau was because Valve software compelled the nVidia Linux team to improve their driver.
A neglected driver is much easier to catch up with.
Open source is great, but it is nVidia who makes the hardware, so unless they open source their driver, the open source community have to invent/guess/do_their_best. Actually, it's impressive that the performance was better than the proprietary in November last year. That is, before Valve needed a better driver for Linux.

[jacmoe]

And, no: it wasn't because Linus Torvalds told them to fuck off.