A kernel.org update

Posted Oct 31, 2013 9:59 UTC (Thu) by mgedmin (guest, #34497)
Parent article: A kernel.org update

Is that a single 30 gig repository, or a set of repositories that total 30 gigs?

What's the story behind the phasing out of bz2?

to post comments

A kernel.org update

Posted Oct 31, 2013 13:05 UTC (Thu) by dlang (guest, #313) [Link] (2 responses)

> What's the story behind the phasing out of bz2?

If you are asking why:

IIRC, .bz2 is slower and larger than .xz (lzma), so it has no advantage

If you are asking about when, I can't answer

A kernel.org update

Posted Oct 31, 2013 13:58 UTC (Thu) by Jonno (guest, #49613) [Link]

> IIRC, .bz2 is slower and larger than .xz (lzma), so it has no advantage
While there might be some pathological cases, in the general case:
xz -2 compresses faster, and usually better, than bzip2 -9.
xz -3 compresses better, and usually faster, than bzip2 -9.
xz always decompresses faster than bzip2.

The decompression memory requirement of xz -2 (3 MiB) and -3 (5 MiB) are comparable to bzip2 -9 (4 MiB), but the compression memory requirement are slightly higher (17M / 32M compared to 8M).

xz -6 (the default) compresses even better, but compress is usually about an order of magnitude slower than bzip2 -9, and it requires a lot more memory (94 MiB to compress, 9 MiB to decompress). So xz -6 is not really a direct replacement for bzip2, but can make sense when distributing files that are compressed once but downloaded and decompressed many times.

A kernel.org update

Posted Oct 31, 2013 14:32 UTC (Thu) by mgedmin (guest, #34497) [Link]

Thanks, that answers my question perfectly.

A kernel.org update

Posted Nov 7, 2013 15:08 UTC (Thu) by mricon (subscriber, #59252) [Link]

Entirety of kernel.org repositories take up about 30GB on the mirrors due to most of them being cloned via "git clone -s" and thus benefiting from object sharing between Linus's tree and every other clone of it (it's safe since Linus never rebases his tree or performs similar unsafe operations that would break shared repos). If repos are cloned without sharing, such as to our long-term backups, they take up about 250GB.