patch for compression type priority (gz, bz2, possibly others)

Bug #71746 reported by Jyrki Muukkonen
4
Affects Status Importance Assigned to Milestone
apt (Debian)
Invalid
Undecided
Unassigned
apt (Ubuntu)
Fix Released
Wishlist
Unassigned

Bug Description

The current apt version defaults to {Packages,Sources,Whatever}.bz2 and
fallbacks to .gz. The .bz2 files are around 20% smaller than .gz files, but
uncompression takes about five times longer. That's something you don't want
on slow machines with fast network connections.

This little patch adds Acquire::CompressionTypes configuration list to define
the compression extension order. Also, this patch makes any future compression
formats a little bit easier to implement.

Tagging this beast with "worksforme" and "experimental". Also, my C++ is a
little bit rusty. It might make your system unusable or send froobazillions of
requests to .deb mirror you're using.

Sample apt.conf:

Acquire::CompressionTypes {
 ".gz";
 ".bz2";
}

A patch against upstream 0.6.46.3 can be found at https://no.spoon.fi/~jvtm/patches/apt/apt-compression-priority-0.6.46.3_001.patch

Debug::Acquire::http is quite helpful when testing this.

Using this on Edgy/amd64 and after the compilation finishes on Etch/ixp4xx (NSLU2).

Not sure if this qualifies as a "bug report", but didn't find any feature request / implementation posting. I'm also too lazy to create a Debian specific bugzilla/whatever account.

Tags: patch
Revision history for this message
Jyrki Muukkonen (jvtm) wrote :

the patch as attachment

Revision history for this message
Michael Vogt (mvo) wrote :

Removing the debian task, its enough to have one task.

Changed in apt:
importance: Undecided → Wishlist
status: Unconfirmed → Confirmed
status: Unconfirmed → Rejected
Michael Vogt (mvo)
Changed in apt:
milestone: none → later
Revision history for this message
Scott Ritchie (scottritchie) wrote :

What's the current behavior of APT these days? I notice we provide Packages.bz2 for Intrepid Universe, but it's not obvious to me that we're using it. Recompressing this with lzma shows another 10% savings over Packages.bz2

Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (10.0 KiB)

This bug was fixed in the package apt - 0.7.23.1ubuntu1

---------------
apt (0.7.23.1ubuntu1) karmic; urgency=low

  [ Matt Zimmerman ]
  * apt-pkg/deb/dpkgpm.cc:
    - Suppress apport reports on dpkg short reads (these I/O errors are not
      generally indicative of a bug in the packaging)

  [ Loïc Minier ]
  * cmdline/apt-key:
    - Emit a warning if removed keys keyring is missing and skip associated
      checks (LP: #218971)

  [ Brian Murray ]
  * cmdline/apt-get.cc:
    - typo fix (LP: #370094)

  [ Michael Vogt ]
  * apt-pkg/deb/dpkgpm.cc:
    - when tcgetattr() returns non-zero skip all pty magic
      (thanks to Simon Richter, closes: #509866)
  * apt-inst/contrib/arfile.cc:
    - show propper error message for Invalid archive members
  * apt-pkg/acquire-worker.cc:
    - show error details of failed methods
  * apt-pkg/contrib/fileutl.cc:
    - if a process aborts with signal, show signal number
  * methods/http.cc:
    - ignore SIGPIPE, we deal with EPIPE from write in
      HttpMethod::ServerDie() (LP: #385144)
  * debian/apt.cron.daily:
    - if the timestamp is too far in the future, delete it
      (LP: #135262)

  [ Merge ]
  * merged from debian, reverted the libdlopen-udev branch
    because its too late in the release process for this now
  * not merged the proxy behaviour change from 0.7.23 (that will
    be part of lucid)

apt (0.7.23.1) unstable; urgency=low

  [ Michael Vogt ]
  * apt-pkg/pkgcache.cc:
    - do not set internel "needs-configure" state for packages in
      triggers-pending state. dpkg will deal with the trigger and
      it if does it before we trigger it, dpkg will error out
      (LP: #414631)
  * apt-pkg/acquire-item.cc:
    - do not segfault on invalid items (closes: #544080)

apt (0.7.23) unstable; urgency=low

  [ David Kalnischkies ]
  * cmdline/apt-get.cc:
    - add APT::Get::HideAutoRemove=small to display only a short line
      instead of the full package list. (Closes: #537450)
    - ShowBroken() in build-dep (by Mike O'Connor, Closes: #145916)
    - check for statfs.f_type (by Robert Millan, Closes: #509313)
    - correct the order of picked package binary vs source in source
    - use SourceVersion instead of the BinaryVersion to get the source
      Patch by Matt Kraai, thanks! (Closes: #382826)
    - add pkg/archive and codename in source (Closes: #414105, #441178)
  * apt-pkg/contrib/strutl.cc:
    - enable thousand separator according to the current locale
      (by Luca Bruno, Closes: #223712)
  * doc/apt.conf.5.xml:
    - mention the apt.conf.d dir (by Vincent McIntyre, Closes: #520831)
  * apt-inst/contrib/arfile.cc:
    - use sizeof instead strlen (by Marius Vollmer, Closes: #504325)
  * doc/apt-mark.8.xml:
    - improve manpage based on patch by Carl Chenet (Closes: #510286)
  * apt-pkg/acquire-item.cc:
    - use configsettings for dynamic compression type use and order.
      Based on a patch by Jyrki Muukkonen, thanks! (LP: #71746)
  * apt-pkg/aptconfiguration.cc:
    - add default configuration for compression types and add lzma
      support. Order is now bzip2, lzma, gzip, none (Closes: #510526)
  * ftparchive/writer.cc:
    - add lzma support also here, patch for this (and...

Changed in apt (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.