default PPAs to HTTPS

This is non-trivial due to the arbitrary content served on the domain. We would do better to move it away from ppa.launchpad.net entirely.

Right now, PPAs don't even support HTTPS at all (I just tried to switch an APT entry pointing to "http://ppa.launchpad.net/yubico/stable/ubuntu xenial main" over to HTTPS, and got a 404)?

What would it take to at least serve a copy of the HTTP content, over HTTPS?

It would require deciding on and acquiring a suitable domain for this that isn't under launchpad.net (compare e.g. launchpadlibrarian.net, which exists for the same reason), since otherwise an XSS attack by user-supplied content on ppa.launchpad.net would be able to steal cookies from its launchpad.net superdomain. We're currently saved by the Secure attribute, but that would become ineffective if ppa.launchpad.net were served over HTTPS. It would of course then require reconfiguring various things to point to that, and I would expect quite a bit of miscellaneous fallout; it's the sort of task that requires budgeting a lot of time for debugging.

(The alternative would be moving the main webapp to a subdomain such as www.launchpad.net and resetting all session cookies, but that's much more work and I think that ship has long since sailed.)

The usual strategy of redirecting HTTP to HTTPS may or may not be a good idea here, as the client situation isn't the same as with browsers. apt gained support for following redirects in 2009, which is long enough ago that we can probably assume it's present, but it's also configurable and it's hard to know how many people will have disabled it.

As illustrated by comment #2, people will probably expect to just be able to switch the scheme and not the host. It's possible that we could safely have an HTTPS virtual host that just redirects everything to whatever the new domain is.

It seems like all the package files are already on launchpadlibrarian.net through https, eg https://launchpad.net/~fnu/+archive/ubuntu/main-fnu/+files/libcurl3_7.36.0-1fnu0~trusty_amd64.deb which redirects to https://launchpadlibrarian.net/173431189/libcurl3_7.36.0-1fnu0~trusty_amd64.deb

So isn't it just a case of changing the urls in the relevant file in /etc/apt/sources.list.d/ ?

Bob, I'm afraid not, because (a) that doesn't include the indexes that apt uses to find those files, and (b) the librarian stores files in a generic layout which isn't directly suitable for apt.

CVE-2019-3462 puts this topic back into focus. It would be great to have PPAs support https.

I (barely) understand the XSS issue - is this because all the launchpad.net subdomains share the launchpad.net certificate vs having a different certificate per subdomain?

I assume the latter is not an option because of the sheer volume of certificates required...

No, it has nothing to do with certificates.

Launchpad needs to set the domain of its session cookies to ".launchpad.net" so that they're visible by its other virtual hosts (code.launchpad.net, bugs.launchpad.net, etc.), but this also causes them to be visible to all other subdomains of launchpad.net. We also set the "secure" flag on those cookies, which mitigates this by ensuring that browsers at least only send them over HTTPS. However, if ppa.launchpad.net were accessible over HTTPS, then any page there would be able to exfiltrate your session cookie. While I don't currently know of a way to get ppa.launchpad.net to serve arbitrary content with Content-Type: text/html, it's a service that hosts a great deal of user-generated files, and it wouldn't take that much to construct abuses of it that would confuse browsers. We need to move it before using HTTPS is safe.

Using apt with http returns a "500 software caused connection abort" error when going through Sophos XG, but work well when going through https. Until launchpad supports https, I'm stuck using the inferior, slow, and slow to update https mirrors sparsely scattered throughout the internet.

I've filed https://portal.admin.canonical.com/C133313 (internal link only) asking our sysadmins to register another domain so that we can start making progress on this.

The alternate domain in question has been registered, and I've filed https://portal.admin.canonical.com/C133345 (again, internal link only) to get it configured on the relevant host.

ppa.launchpadcontent.net and private-ppa.launchpadcontent.net now exist, and are both HTTPS-capable. It's possible to use https://ppa.launchpadcontent.net/ today in place of http://ppa.launchpad.net/ in sources.list.

We still have some work to do to make the Launchpad UI default to this, as well as tools like add-apt-repository.

The Launchpad web UI now advertises the new https://ppa.launchpadcontent.net/ URLs (and https://private-ppa.launchpadcontent.net/ for private PPAs). As previously mentioned, the old URLs will also continue working indefinitely.

I've filed bug 1959015 for the add-apt-repository changes.