Smaller downloads with 7-zip

Lord LoriK · Post by **Lord LoriK** » Mon Mar 05, 2007 2:10 am

Well, we are in the 21st century, and ZIP files are a bit outdated considering the huge amount of more modern formats available: RAR, 7-zip, bzip2, LZX (cab files), etc. I think it wouldn't hurt anyone if the downloads were in one of those newer formats.

Pros:
* Smaller downloads for users means faster upgrading and less excuses for keeping outdated versions
* Less bandwith use (for the server)

Cons: none, AFAIK. 7-zip is free, and even if you had to download it, is less than 1MB. You can decompress RAR, bzip2 and cab files with it.

Consider this: Windows sources plus dependencies add up to 49MB for VC++. I recompressed them with 7-zip to distribute them on other machines with no internet access (my pen drive is small) and guess what: 25MB... Do the math, guys.

Post by **pjcast** » Mon Mar 05, 2007 2:32 am

No one disagrees that there are far better compression formats than zip. However, Ogre moved back to zip from tars (?) because many Window's users complained. Most of the problem stems from the fact that WinXP + natively supports zips, while it doesn't export other formats. Although, zip support in XP sucks as far as I am concerned (slow, and 7-zip handles it better). Perhaps we could distribute the source files in a self-extracting 7-zip file.. although, than users would complain about security concerns with executables.

Post by **Praetor** » Mon Mar 05, 2007 3:30 am

WinXP zip support is hilariously bad. WinRar often unzips things what seems like 10x faster.

Post by **xavier** » Mon Mar 05, 2007 3:50 am

ZIP files are the lowest-common-denominator on Windows -- that's why you find everything for Windows as ZIPs. Broad range of support -- it's what you have to do with software with wide appeal.

syedhs · Post by **syedhs** » Mon Mar 05, 2007 4:57 am

I think the ZIP file support in XP is there to demonstrate the Archive capability. In commercial application, you will want to customize this class yourself.

Edit: Totally off-mark. This is about changing file format download

I think providing the alternative is a good idea. It however should have much better compression (read: at least 20% smaller than zip counterpart) otherwise it gives more trouble than the benefit.

Lord LoriK · Post by **Lord LoriK** » Mon Mar 05, 2007 6:01 am

7-zip solid archives give A LOT MORE than 20%, and as I proved with my example, there's a huge gain, about 50%, at least for source packages. Beside that, you can extract them with WinRAR, and there is a port of 7-zip (p7zip) for Linux and other POSIX systems. It's worth a try, I think.

pjcast wrote:Perhaps we could distribute the source files in a self-extracting 7-zip file.. although, than users would complain about security concerns with executables.

Well, the SDK presents the very same problem, and that doesn't keep people from installing it. And at least, having 7-zip/WinRAR allows you to extract files manually from SFX archives, while the Nullsoft Installer, used in the VC++ SDK distribution, doesn't.

xavier wrote:ZIP files are the lowest-common-denominator on Windows

7-zip works even under Windows 95. And if you have to download a 50MB file, you won't mind downloading a program only once to decompress it. I'd really like to know how many of those who install Ogre don't have a program that handles .7z files already. Maybe we could make a poll with this?

As syedhs suggests, this could be an alternative. I would use it for sure, because my internet connections is really lousy and 50MB (or 35 if I don't need newer dependencies) takes about an hour to download. I just can't wait that long when I see a newer Ogre version.

Post by **genva** » Mon Mar 05, 2007 8:10 am

We encourage gather source from cvs, so you have no delay with Ogre update. And there have bonus: I don't think there have a compress algorithm can take bandwidth advantage than progressive update from CVS.

Wretched_Wyx · Post by **Wretched_Wyx** » Mon Mar 05, 2007 9:28 am

You wanna see some compression? Use the NSIS installer + 7zip plugin. Basically compress all your files with 7zip, make an install with NSIS with scripts for decompressing/installing those 7zip files. Oh, maybe perhaps hit that NSIS installer with UPX.

Post by **xavier** » Mon Mar 05, 2007 9:50 am

Lord LoriK wrote: 7-zip works even under Windows 95. And if you have to download a 50MB file, you won't mind downloading a program only once to decompress it.

But you should not have to force a user to get something just to get the software. It's the reason I don't really like what's up with OpenGUI -- I don't want to have to d**k with scons just to build the thing. I don't see any reason to add even more additional requirements to get Ogre, on top of the stuff you need to get just to build it or build with it.

As for alternatives...I don't think that we need to burden the dev team with more to manage than they already have. I'd rather they spend their time making Ogre better, rather than spend time making smaller downloads.

Wretched_Wyx · Post by **Wretched_Wyx** » Mon Mar 05, 2007 10:06 am

Good point there. Eh, screw point- it should be a commandment. Less aesthetic worries, and more hardcore programming

.

Game_Ender · Post by **Game_Ender** » Mon Mar 05, 2007 4:17 pm

xavier wrote:I don't want to have to d**k with scons just to build the thing

That's only because its a tool you don't use. If someone doesn't use VS then to build a VS based project they have to "d**k" with it too.

I would like to see the smaller file download option, but not at the cost of lots of developer time.

Post by **betajaen** » Mon Mar 05, 2007 4:23 pm

I think it's best just to use the de facto compression format for the operating system; zip for windows, tar.gz for linux and stuff it or dmg for OSX.

I really don't want to download 7-zip, or winrar or what ever the best is at the moment, and to me it seems a little unprofessional to do so.

Lord LoriK · Post by **Lord LoriK** » Mon Mar 05, 2007 4:27 pm

genva wrote:We encourage gather source from cvs, so you have no delay with Ogre update. And there have bonus: I don't think there have a compress algorithm can take bandwidth advantage than progressive update from CVS

xavier wrote:
Lord LoriK wrote: 7-zip works even under Windows 95. And if you have to download a 50MB file, you won't mind downloading a program only once to decompress it.
But you should not have to force a user to get something just to get the software.

Hmm, interesting. You encourage to gather from CVS, and Windows has no built-in CVS service. What's the difference? Both programs are quite useful. I'm not talking of "forcing users", just giving a good alternative. I can use your arguments and say: "you force users to download huge packages or install CVS services to get smaller distributions". What's better? You force them either way.

Be open-minded, please. Maybe you, citizens of the first world and developed countries, have fast and reliable internet connections, but that's not true for the rest of the world...

xavier wrote:I'd rather they spend their time making Ogre better, rather than spend time making smaller downloads.

Me too, but come on... It takes 10 minutes being very pessimistic. How much does it affect the project, considering there is about one update every 2 weeks or so...

Game_Ender · Post by **Game_Ender** » Mon Mar 05, 2007 4:31 pm

Gziped tars are standard but you can also do bziped tars just as easilly on Linux just use "tar -jxf" instead of "tar -zxf". If having the user download something else is really that big a deal, we are developers after all, you can just use something like this it looks like it only adds an extra 23kb to the download (free for non-commerical use).

Really Off Topic-
Using CVS is also more bandwidth intensive that SVN, but due to projecct management issues Ogre has not made the transistion to that either.

Post by **betajaen** » Mon Mar 05, 2007 4:32 pm

Lord LoriK wrote:Be open-minded, please. Maybe you, citizens of the first world and developed countries, have fast and reliable internet connections, but that's not true for the rest of the world...

Then wouldn't it be better to download just the changes of code from the previous release, and not 90% of the same files each time?

The CVS to me, serves that purpose beautifully.

Post by **pjcast** » Mon Mar 05, 2007 4:34 pm

No one forces one to download Ogre. So your argument that one is forced to download a huge zip file (which it is not that bad, I've downloaded a lot bigger projects lately) is wrong. Nor does one force you to download a CVS client - that is just a preferred way. Heck, some people like me still use Win2000, where it does not handle zips... so I still need to get something to Open Ogre with. but guess what... I rarely need to download Ogre. I just grab a small CVS client, and checkout Ogre - then I can keep updated without having to grab large zip files at all.

So, if you are complaining about fast internet connection, just do yourself a favor and grab a CVS client. The zip file situtation is unlikely to change, for several reasons already mentioned. And, there is little reason to provide Ogre releases in multiple compression formats. All you have to do, is hit download, and get a cup of coffee (or two). If your internet connection is that bad where it bothers you, I think other requirements will bother you more (DXSDK, dependencies, Sevice pakcs, IDEs, etc).

Lord LoriK · Post by **Lord LoriK** » Mon Mar 05, 2007 4:38 pm

betajaen wrote:I think it's best just to use the de facto compression format for the operating system; zip for windows, tar.gz for linux and stuff it or dmg for OSX.

ZIP is no way the "de facto" standard for Windows. It's just a hackish and very poor add-on that MS added to Windows XP. Before that, you had to get your own (de)compressor. Gee, you cannot even run (or use) things inside "compressed folders" transparently. They force you to decompress it...

betajaen wrote:I really don't want to download 7-zip, or winrar or what ever the best is at the moment, and to me it seems a little unprofessional to do so.

I never suggested that. Just stick to a new format, thats it. You all seem to forget what SFX files were invented for... We already have SDKs that are executable installers. What's the difference?

Well, it was an idea. It seems you don't like it. A shame. More options means wider audience, AFAIK.

Lord LoriK · Post by **Lord LoriK** » Mon Mar 05, 2007 4:44 pm

pjcast wrote:No one forces one to download Ogre. So your argument that one is forced to download a huge zip file (which it is not that bad, I've downloaded a lot bigger projects lately) is wrong

Well, I give up. No more arguments from me. I was just defending my idea, but it's pointless as far as I have not even one more user on my side

. I'll keep reading, though.

Post by **sinbad** » Mon Mar 05, 2007 4:55 pm

As pjcast says, we deviated from this before and got a shedload of complaints from new users. Yes, there are much better alternatives - I use 7z and .tar.bz2 when I have the chance, but that's not what we're talking about here. We're talking about mass palatability and like it or not, that's Zip on windows.

CVS is the most efficient way to get updates - that's the option for the more discerning user. Average Joe Public wants ZIP, so that's what he gets. We're not going to distribute zipped packages in more than one format so it has to be the one with the largest appeal. The only other option I would consider is self-extracting 7-Zip but I can't build those automatically on my autobuild server box (it's Linux) and I already have enough manual tasks to do at release time (my Windows box is tied up building the SDK) so don't need another one, thanks

And hey, if you don't like downloading a slightly larger package, bear in mind that I have to upload 3 or more of them at crappy ADSL upstream rates. You think I would do that if I didn't think it was necessary?

Let's just say I'm not feeling your pain

Wretched_Wyx · Post by **Wretched_Wyx** » Mon Mar 05, 2007 5:00 pm

Lord Lorik wrote:Well, it was an idea. It seems you don't like it. A shame. More options means wider audience, AFAIK.

To be more specific, it was *your* idea, and it got shot down. Nothing to get bent about, it's just the reasoning behind it wasn't all that convincing. And it's not as if Ogre devs and the community aren't keen on cutting edge distribution techniques. See, if suddenly Ogre were to be distributed in another compression format, you'd have X amount of users going nuts asking how to open this new format. That amounts to X amount of time and effort that others have to waste explaining what program to get, where to get it, etc. That's X amount of time not going to Ogre.

I'm certain though, that in time there will be new and exciting features to the Ogre website, and it's features. Most likely including an overhauled download system. Thing is- right now Ogre is banging, and what's there now works just fine. To step away from more important things, and fix something that isn't broke, or even working at half-capacity, would be bad managment of time.

I was on 56k when I first started toying with Ogre. If I were downloading an updated SDK release, I'd just que it up in a download manager and go clean my toenail clipping collection, or pretend to throw rocks at birds that I thought were living in my kitchen. But I never once thought about how the evil well developed countries that banded together to develop Ogre, were supressing my right to download faster.

Now don't get me wrong, more compressed Ogre releases would be cool. And 7zip would be the format to use for that purpose. Instead of trying to force an overhaul on the current system, maybe a more trivial route, that wouldn't leave a footprint here (ogre3d.org) would be to start a site "OgreLite". Perhaps some sort of "pick and choose" system, where you pick the components of the latest release you want (no demos, no media, etc), and a nifty script zips up a file for you. But that would be a lot of effort, considering that you can... Like a few stated before, simply just keep up to date via CVS. You'll have the latest features and fixes, and your download times will be kept at a minimum.

Bah. I just realized that was all rambling, repetitive, crap. Zip is fine.

ChalkTrauma · Post by **ChalkTrauma** » Mon Mar 05, 2007 5:50 pm

I agree that I like 7zip better that zip these days, but the choice of apps that work with it is more limited.

I happen to think that the 7zip File Manager is great, and I use it for all the archive formats. I also use the 7zip command line version a lot. I mainly use 7zip for archiving and backing up. But it is also great for moving stuff around.

I just think that zip is so ubiquitous that it is hard to change. As Sinbad said.. it's what Joe Q Public wants.

I really understand what you are saying.. I lived though the days of 16kb or expanded memory on my C64 and kinda have a fetish for space optimization ( or optimisation ). And I can squeeze 38% off of the ogre-win32-v1-4-0RC2.zip with the right magic command line params with 7zip, but I think it is more of a usage thing where the benefits of the availability of Zip are greater than the bandwidth savings of 7zip..