"Size mismatch" caused by writing wrong data

Bug #1234946 reported by Willem de Groot
38
This bug affects 6 people
Affects Status Importance Assigned to Milestone
apt-cacher-ng (Ubuntu)
Confirmed
Undecided
Unassigned

Bug Description

5-15% of packages downloaded through apt-cacher-ng are corrupt with a "Size mismatch" or "Hash Sum mismatch" error, reported by the client. Apparently, the corrupted files on the cache host in /var/cache/apt-cacher-ng are not corrupt but contain a different package than the filename suggests. For example, python-apt_0.8.3ubuntu7.1_amd64.deb is in fact xz-lzma_5.1.1alpha+20110809-3

Not sure how to debug this further.

I use apt-cacher-ng on various hosts on various isolated environments. Some apt-cacher-ng installs yield a higher "size mismatch" frequency than others, which might indicate a race condition. The ones with lower error frequency have higher uplink speeds and higher cpu speeds.

ProblemType: Bug
DistroRelease: Ubuntu 13.10
Package: apt-cacher-ng 0.7.18-1
ProcVersionSignature: Ubuntu 3.11.0-9.16-generic 3.11.2
Uname: Linux 3.11.0-9-generic x86_64
NonfreeKernelModules: nvidia
ApportVersion: 2.12.5-0ubuntu1
Architecture: amd64
Date: Thu Oct 3 22:44:42 2013
EcryptfsInUse: Yes
InstallationDate: Installed on 2013-09-03 (30 days ago)
InstallationMedia: Xubuntu 13.10 "Saucy Salamander" - Alpha amd64 (20130723.2)
MarkForUpload: True
SourcePackage: apt-cacher-ng
UpgradeStatus: No upgrade log present (probably fresh install)
modified.conffile..etc.apt.cacher.ng.backends.ubuntu: [modified]
modified.conffile..etc.apt.cacher.ng.security.conf: [inaccessible: [Errno 13] Permission denied: '/etc/apt-cacher-ng/security.conf']
mtime.conffile..etc.apt.cacher.ng.backends.ubuntu: 2013-09-03T16:38:18.300177

Revision history for this message
Willem de Groot (gwillem) wrote :
Revision history for this message
Eduard Bloch (edi-gmx) wrote :

Well... could you tell me (exactly) which version of acng is in use there?

Revision history for this message
Willem de Groot (gwillem) wrote : Re: [Bug 1234946] [NEW] "Size mismatch" caused by writing wrong data

Hi Eduard, all installs use apt-cacher-ng 0.7.18-1. Will run it with debug
mode on when I find a spare hour. Cheers!

On Saturday, October 5, 2013, Eduard Bloch wrote:

> Well... could you tell me (exactly) which version of acng is in use
> there?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1234946
>
> Title:
> "Size mismatch" caused by writing wrong data
>
> Status in “apt-cacher-ng” package in Ubuntu:
> New
>
> Bug description:
> 5-15% of packages downloaded through apt-cacher-ng are corrupt with a
> "Size mismatch" or "Hash Sum mismatch" error, reported by the client.
> Apparently, the corrupted files on the cache host in /var/cache/apt-
> cacher-ng are not corrupt but contain a different package than the
> filename suggests. For example, python-apt_0.8.3ubuntu7.1_amd64.deb is
> in fact xz-lzma_5.1.1alpha+20110809-3
>
> Not sure how to debug this further.
>
> I use apt-cacher-ng on various hosts on various isolated environments.
> Some apt-cacher-ng installs yield a higher "size mismatch" frequency
> than others, which might indicate a race condition. The ones with
> lower error frequency have higher uplink speeds and higher cpu speeds.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 13.10
> Package: apt-cacher-ng 0.7.18-1
> ProcVersionSignature: Ubuntu 3.11.0-9.16-generic 3.11.2
> Uname: Linux 3.11.0-9-generic x86_64
> NonfreeKernelModules: nvidia
> ApportVersion: 2.12.5-0ubuntu1
> Architecture: amd64
> Date: Thu Oct 3 22:44:42 2013
> EcryptfsInUse: Yes
> InstallationDate: Installed on 2013-09-03 (30 days ago)
> InstallationMedia: Xubuntu 13.10 "Saucy Salamander" - Alpha amd64
> (20130723.2)
> MarkForUpload: True
> SourcePackage: apt-cacher-ng
> UpgradeStatus: No upgrade log present (probably fresh install)
> modified.conffile..etc.apt.cacher.ng.backends.ubuntu: [modified]
> modified.conffile..etc.apt.cacher.ng.security.conf: [inaccessible:
> [Errno 13] Permission denied: '/etc/apt-cacher-ng/security.conf']
> mtime.conffile..etc.apt.cacher.ng.backends.ubuntu:
> 2013-09-03T16:38:18.300177
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/apt-cacher-ng/+bug/1234946/+subscriptions
>

Revision history for this message
Willem de Groot (gwillem) wrote :

Set debug to 10, it doesn't give more output. I guess I need a recompile with debugging enabled.

apt-cacher-ng is pretty much unusable right now. Sometimes, the failure rate is 100%. In that case, every cached deb files is stored with the filename of the previously requested package.

Will switch over to squid-deb-proxy as it does pretty much the same (http://www.garron.me/en/blog/ubuntu-deb-proxy-cache.html)

Cheers!

Revision history for this message
Seth Arnold (seth-arnold) wrote :

I have the same issue with apt-cacher-ng version 0.7.18-1 as packaged in Ubuntu 13.10.

I've attached pieces of today's apt-cache.err and apt-cache.log files. I hope they are useful.

(Willem, thanks so much for filing this bug report. I've spent two hours going around in circles trying to discover this problem. I've seen dozens, if not hundreds, of size mismatches while trying to build some software today and none of my debugging attempts would fix the issue for good.)

Revision history for this message
Seth Arnold (seth-arnold) wrote :
Revision history for this message
Seth Arnold (seth-arnold) wrote :
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apt-cacher-ng (Ubuntu):
status: New → Confirmed
Revision history for this message
Seth Arnold (seth-arnold) wrote :

This is how apt inside a chroot saw the failure to download and install packages.

Revision history for this message
James Kyle (jkyle) wrote :

Seeing the same error patterns here on 0.7.11-1~ubuntu12.04.1

Revision history for this message
Michael Tibben (michael-tibben) wrote :

Seeing the same issues on Trusty with 0.7.18-1

Revision history for this message
Michael Tibben (michael-tibben) wrote :

Ah sorry I'm actually on Saucy.

Revision history for this message
Rolf Leggewie (r0lf) wrote :

does this ever occur in trusty or later?

Changed in apt-cacher-ng (Ubuntu):
assignee: nobody → Rolf Leggewie (r0lf)
status: Confirmed → Incomplete
Revision history for this message
Rolf Leggewie (r0lf) wrote :

I believe I'm seeing this with acng on a jessie host. And indeed that host has patchy and slow internet connectivity.

Changed in apt-cacher-ng (Ubuntu):
assignee: Rolf Leggewie (r0lf) → nobody
status: Incomplete → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.