Ubuntu archive server returning incorrect content-encoding - content-header incorrect type

I am seeing the same symptom trying to upgrade from Gutsy to Hardy using apt-cacher. Is there a work-around or do I have to connect directly (re-downloading all packages)?

Hi, I have a similar problem and reported it to ubuntuforums installation and upgrade section but still could not find a workaround.

I am trying to upgarde gutsy to hardy behind a proxy server that requires authentication username and password. I can update the packages using the global proxy setting of gnome and even can wget in the terminal with that setting. Even the synaptic is working fine with its own proxy configuration. But when attempted the upgarde process it says Failed to fetch some packages -- proxy authorization problem.

I see the symptoms while trying to upgrade from 8.04 to 8.10 beta.

root@white:~# update-manager --devel-release &
[1] 6790
root@white:~# extracting 'intrepid.tar.gz'
authenticate 'intrepid.tar.gz' against 'intrepid.tar.gz.gpg'
exception from gpg: GnuPG exited non-zero, with code 131072
Debug information:

gpg: WARNING: unsafe permissions on homedir `/tmp/tmp83J8IK'

gpg: can't open `/tmp/tmp83J8IK/intrepid.tar.gz.gpg'
gpg: verify signatures failed: file open error

[1]+ Done update-manager --devel-release
root@white:~#

On a successful run, I do not get the permissions message.

update-manager-1:0.87.30--all
update-manager-core-1:0.87.30--i386

I removed a local apt-cacher-ng proxy from my configuration, and got past the problem.

Acquire::http { Proxy "http://192.168.1.30:3142"; };

I have not verified the wireshark analysis.

How do you remove local apt-cacher-ng proxy from my configuration?

Thank you for taking the time to report this bug and helping to make Ubuntu better. You reported this bug a while ago and there hasn't been any activity in it recently. We were wondering is this still an issue for you? Can you try with latest Ubuntu release? Thanks in advance.

Dear Teej,

This problem still exists because the problem is with the configuration of Ubuntu's
servers (such as "archive.ubuntu.com"), not the distribution it produces. To summarize,
Ubuntu's archive servers return wrong headers for gpg files, and this causes errors
when there is a proxy between the computer running Ubuntu's upgrade program and
Ubuntu's archive servers.

I encountered the same problem when I tried to upgrade from 6.06 to 8.04 a few
days ago. When I disabled the proxy configuration for apt by modifying apt's
configuration, the upgrade worked. This is expected, as can be understood from
the description of the bug.

I would appreciate it if the configuration of Ubuntu's archive servers could be fixed.
Please read the description of this bug to learn about the problem in the configuration
of Ubuntu's archive servers. (I did not actually let the upgrade begin, so I can still test
the server if the configuration of the server is eventually changed.)

Regards,

M. Vefa Bicakci

Sorry I never got back to you on this. I don't suppose this is still a problem in Karmic 9.10? Thank you.

Teej, as M. Vefa pointed out, the problem is not in the distribution but on the Ubuntu HTTP servers.

The Content-Encoding header mentioned by Bob Drzyzgula in the original report seems to be gone now.

However, Content-Type is now application/x-gzip which is incorrect for a PGP signature. I haven't tested upgrading through a proxy recently, but it's still possible that this header could break that.

It looks like mod_mime matches extensions in the middle of a file name if the final extension is unknown. This explains why for files with a .tar.gz.gpg extension, the Ubuntu servers used to indicate a gzip encoding and a tar content type, and why they now return a gzip content type.

Here is what the Ubuntu servers currently reply:

$ wget -q --no-cache --save-headers http://archive.ubuntu.com/ubuntu/dists/hardy-proposed/main/dist-upgrader-all/0.87.31/hardy.tar.gz.gpg
$ cat hardy.tar.gz.gpg
HTTP/1.1 200 OK
Date: Sun, 17 Jan 2010 13:00:31 GMT
Server: Apache/2.2.8 (Ubuntu)
Last-Modified: Thu, 29 Jan 2009 15:38:41 GMT
ETag: "bd-461a0e1fda240"
Accept-Ranges: bytes
Content-Length: 189
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/x-gzip

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQBJgc2BQJdur0N9BbURAnhDAJwNh53DLg9jF8c3+dcjTKM+CnpK6gCggG9e
2Fi2CguhyGmEktTiVEQT4Hk=
=5Ug/
-----END PGP SIGNATURE-----

$ wget -q --no-cache --save-headers http://archive.ubuntu.com/ubuntu/dists/lucid/main/dist-upgrader-all/0.131.3/lucid.tar.gz.gpg
akaihola@morris:/tmp$ cat lucid.tar.gz.gpg
HTTP/1.1 200 OK
Date: Sun, 17 Jan 2010 13:04:35 GMT
Server: Apache/2.2.8 (Ubuntu)
Last-Modified: Tue, 12 Jan 2010 19:27:03 GMT
ETag: "bd-47cfca3780fc0"
Accept-Ranges: bytes
Content-Length: 189
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: application/x-gzip

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQBLTM0HQJdur0N9BbURAgoQAJ9jwi/0EBi+Hd5VWivJ90SYhRFwsQCggxR/
Y2Z4yunLdiqmR9x2DrOUxM4=
=tv7O
-----END PGP SIGNATURE-----

I think this should be added to Apache's /etc/apache2/mods-available/mime.conf:
AddType application/pgp-signature .gpg

I tried this on my Debian Squeeze -based server and sure enough, .tar.gz.gpg files are now served with the application/pgp-signature content type. This needs to be tested though by creating an Ubuntu repository mirror and upgrading from it through a proxy.

On the other hand, maybe a wrong Content-Type doesn't matter and the removal of the Content-Encoding header already fixed the issue.

Thanks for updating this. Marking this Triaged, Medium, as there is more than enough information for someone to see what is going on here.

I also notified Debian's apache2 bug tracker about the issue:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=565626

Brilliant! Thank you. I will add a watch for this. For future reference you can do this easily by choosing "Also affects project" and pasting the link to the other Bug Tracking System. :)

"Also affect distribution", sorry.

No, I was right the first time, don't worry, it's set now, and Launchpad will keep track of it. Remember if this affects you to add yourself as affected, currently there is only 1 affected user, and if you can provide any more relevant information to this bug report, or the Debian bug, please add it, perhaps on both to make it easier for developers. Thank you.

Unassigning ubuntu-archive - we manage the contents of the archive but in general have no control over the configuration of the servers that host it, so there isn't much point in us being assigned to this.

OK

Very old bug. I just tried wgetting http://archive.ubuntu.com/ubuntu/dists/disco-proposed/main/dist-upgrader-all/current/disco.tar.gz.gpg with and without a proxy, and it worked just fine, even though it's still being identified as x-gzip:
andreas@nsnx:~/x$ http_proxy=http://squid-ds216.lxd:3128/ wget --no-cache http://archive.ubuntu.com/ubuntu/dists/disco-proposed/main/dist-upgrader-all/current/disco.tar.gz.gpg
--2019-02-06 14:57:16-- http://archive.ubuntu.com/ubuntu/dists/disco-proposed/main/dist-upgrader-all/current/disco.tar.gz.gpg
Resolving squid-ds216.lxd (squid-ds216.lxd)... 10.0.100.206
Connecting to squid-ds216.lxd (squid-ds216.lxd)|10.0.100.206|:3128... connected.
Proxy request sent, awaiting response... 200 OK
Length: 1554 (1,5K) [application/x-gzip]
Saving to: ‘disco.tar.gz.gpg’

disco.tar.gz.gpg 100%[================================================================================================================================>] 1,52K --.-KB/s in 0s

2019-02-06 14:57:17 (75,9 MB/s) - ‘disco.tar.gz.gpg’ saved [1554/1554]

andreas@nsnx:~/x$ file disco.tar.gz.gpg
disco.tar.gz.gpg: PGP signature Signature (old)

andreas@nsnx:~/x$ cat disco.tar.gz.gpg
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAABCgAGBQJcRf7TAAoJEDtP5qzAsh8yFaMQANW6mTi+bCHp08NeFC7wcSal
z2xtiNxm2FYIJ9BkZZGvS1YI5rQsS8X0x1auU21mCus39anOUp5XaWUDPVMfjuZO
RxajHohB1YUAiLMi7zY86LkpEMSYOMknwhH57ZZSMlHELyqSb/hKzawPfrBKTnWo
N0b0+3hbpwi7tRZu0Y3Y3cooeH6C6lgFJEAPkFXcgdFw72OtGs1ByOIkRlr4AUb7
...

I'm going to mark this bug as incomplete, and it will expire automatically unless someone has something to add.

I'm real sorry to waste bandwidth like this, but I just want to let y'all know that I'm LOLing at this point. :-)

The bug 1888058 is somewhat related to the current bug.

I can't upgrade 19.10 to 20.04 LTS and 18.04 LTS to 20.04 LTS using squid-deb-proxy.

I'm not sure why this is a problem now as the old archive servers were doing the same thing. In any case, I've added the following to apache2 configs:

| AddType application/pgp-signature .gpg

Testing without:

> GET /ubuntu/dists/focal-updates/main/dist-upgrader-all/current/focal.tar.gz.gpg HTTP/1.1
> Host: archive.ubuntu.com
> ...
< Last-Modified: Thu, 16 Jul 2020 08:58:10 GMT
< ETag: "612-5aa8b3d596c80"
< Accept-Ranges: bytes
< Content-Length: 1554
< Content-Type: application/x-gzip

Testing with:

> GET /ubuntu/dists/focal-updates/main/dist-upgrader-all/current/focal.tar.gz.gpg HTTP/1.1
> Host: archive.ubuntu.com
>...
< Last-Modified: Thu, 16 Jul 2020 08:58:10 GMT
< ETag: "612-5aa8b3d596c80"
< Accept-Ranges: bytes
< Content-Length: 1554
< Content-Type: application/pgp-signature