SSL trust not system-wide

It does seem that p11-kit-trust.so is working correctly. If I just make a symlink from libnssckbi.so to it, corporate trust installed by update-ca-certificates *does* work in Firefox.

Hi dwmw2,
thank you for your bug report and your help to make Ubuntu better.

I beg a pardon as I'm clearly not an expert on this particular area, but I try to sort out the details of this bug report to understand what has to be done.

Currently I understand this as feature request to make update-ca-certificates (almost?) all certificate users in one shot.

The current default config doesn't do that

Thanks for pointing out the links and background to this.

The answer on this thread is what I think the current state is http://superuser.com/questions/437330/how-do-you-add-a-certificate-authority-ca-to-ubuntu
and I understand and agree that to get this as "one shot accept this CA" is a valid feature-request-bug.

I happened to find various similar/related on other projects like firefox for example:
https://bugzilla.mozilla.org/show_bug.cgi?id=620373
https://bugzilla.mozilla.org/show_bug.cgi?id=449498
https://bugzilla.mozilla.org/show_bug.cgi?id=454036
There might be more for others, but it seems to fix the whole thing a Distribution would need to modify all consuming packages to agree on sort of a shared path and mechanism.

Ok, so far I was just trying to wrap my head around this a bit, I guess the next step clearly is the security Teams position on this in general - so I subscribe them for a statement.
Maybe they also know on past or existing approaches to this.

@Security Team - do you happen to know about this overall topic and could you share either whatever was the outcome of such discussions in the past or OTOH what you assert on this as a feature request would be?

The Mozilla bugs you link are a bit of a red herring. They refer to an abortive attempt by Mozilla/NSS to have a 'shared system database' in sql:/etc/pki/nssdb. The idea is that applications specify that as their NSS database and although it's obviously read-only, it automatically adds the user's database from ~/.pki/nssdb as a writeable token. This gets a step towards consistency for all NSS-using applications — but as those bugs note, not even Mozilla's own products are actually using it. You should support that anyway, but it isn't the focus of this bug.

The fix here (which has been working in Fedora for years, since you ask for existing approaches) is to replace NSS's built-in trust root module libnssckbi.so with a symlink to p11-kit-trust.so. Then you get the system's configured trust roots, instead of whatever's hard-coded into that particular instance of libnssckbi.so (and you're shipping multiple potentially different ones of those!)

I believe we need to update p11-kit to v0.23.4 to make the key pinning work correctly in the recommended configuration, by adding the CKA_NSS_MOZILLA_CA_POLICY attribute.

https://bugs.freedesktop.org/show_bug.cgi?id=99453
https://bugzilla.mozilla.org/show_bug.cgi?id=1324096

I believe NSS wants these patches backported from 3.30:
https://bugzilla.mozilla.org/show_bug.cgi?id=1334976

Firefox has its own copy of NSS which I think as of Firefox 54 should be fine.
Thunderbird also needs fixing, I think.

cf. https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741005
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=704180
https://lists.freedesktop.org/archives/p11-glue/2013-June/000331.html

Status changed to 'Confirmed' because the bug affects multiple users.

Any progress on fixing this?

Wow, unified CA management would be awesome. No more fiddling around with (and forgetting to correctly install/remove certificates in) various applications (most notably in Firefox, Chromium, wget).

No progress on this yet, afaik it is just not high up on anyone's personal task list :-/

I'm trying to make use of this in Ubuntu 14.04 with p11-kit 0.23.2-5~ubuntu16.04.1, but get the following error:

# trust list
p11-kit: ca-certificates.crt: BEGIN ...: pem block before p11-kit section header
p11-kit: ca-certificates.crt: BEGIN ...: pem block before p11-kit section header

Is p11-kit just too old there?

This isn't "just" a bug, it's a roadmap item in my view, as many products are affected. It needs a spec, like in the fedora case. I agree that it would be awesome to have this.

@dwmw2

Were you able to make this work by doing this for firefox?

sudo mv /usr/lib/firefox/libnssckbi.so /usr/lib/firefox/libnssckbi.so.bak
sudo ln -s /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/firefox/libnssckbi.so

https://askubuntu.com/questions/244582/add-certificate-authorities-system-wide-on-firefox/1036637#1036637

I wasn't able to get it to work. Doing the above doesn't work at all for me. I'm on 18.04.3

should this be marked as something to fix in focal for the next LTS?

@kvasko yes, it works here. Are you sure that's the version of libnssckbi.so that is being used? There are lots; I've replaced them all...

@dwmw2,

I figured out the issue. Long story short, freeipa (which is our CA), when we enroll a PC into the realm, it adds the freeIPA cert to /etc/ssl/certs/ca-certificates.crt like it should, however it also adds other information that it shouldn't.

This results in p11-kit-trust.so blowing parsing errors.

You can read the entire bug report here if you want.

https://pagure.io/freeipa/issue/8106

Like others, I'm manually symlinking .so files on all of my interactive hosts and hoping updates don't break it. IMO this is not a valid workaround.

@ahasenack - I understand this is a roadmap item that would ideally resolve for multiple packages, but it seems that the Mozilla products are the worst offenders at the moment. I don't see anyone requesting anything else in this bug. Would it be possible to at least resolve it for Firefox and Thunderbird? What would it take to get this looked at for the next LTS?

For now, Thunderbird needs this too (and works for me on 18.0.3 LTS):

sudo mv /usr/lib/firefox/libnssckbi.so /usr/lib/firefox/libnssckbi.so.bak
sudo ln -s /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/firefox/libnssckbi.so
sudo mv /usr/lib/thunderbird/libnssckbi.so /usr/lib/thunderbird/libnssckbi.so.bak
sudo ln -s /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-trust.so /usr/lib/thunderbird/libnssckbi.so

nss should have everything on focal

p11-kit too

according to #4 nss should still symlink libnssckbi.so to p11-kit-trust.so

It looks like symlinking firefox and thunderbird's own copies of libnssckbi.so to the system-wide p11-kit-trust.so is the proper way to fix this bug, as far as Mozilla's products are concerned.

Before I proceed to doing this, I'd welcome comments from the security team on this approach though, as I suspect I don't understand all the implications.

(an alternative would be building firefox/thunderbird against the system-wide nss, but firefox currently requires 3.50, which isn't yet in focal, and I suspect that requirement is being bumped often, so that wouldn't really work with our distribution model)

On Thu, 2020-03-19 at 09:44 +0000, Olivier Tilloy wrote:
> It looks like symlinking firefox and thunderbird's own copies of
> libnssckbi.so to the system-wide p11-kit-trust.so is the proper way to
> fix this bug, as far as Mozilla's products are concerned.
>
> Before I proceed to doing this, I'd welcome comments from the security
> team on this approach though, as I suspect I don't understand all the
> implications.
>
> (an alternative would be building firefox/thunderbird against the
> system-wide nss, but firefox currently requires 3.50, which isn't yet in
> focal, and I suspect that requirement is being bumped often, so that
> wouldn't really work with our distribution model)

Right, don't bother trying to replace NSS just for this (although
really, having a single version of NSS on the system *would* be nice).

The interface to libnssckbi.so is a standard PKCS#11 library, and it's
perfectly reasonable to replace that in each of
firefox/thunderbird/chromium individually.

Before we switch any software to using p11-kit-trust.so, we need to fix our ca-certificates package to properly handle untrusted or blacklisted certificates. At the moment, I believe they are simply skipped when generating the contents of /usr/share/ca-certificates.

so what does it require to fix ca-certificates?

Looks like Fedora substantially modified the scripts used by ca-certificates to extract untrusted and blacklisted certs. We should probably start by investigating how their package is handling this, what files they are generating, and if they are being properly handled by p11-kit-trust.

So for the avoidance of doubt, every independent distro has its own custom ca-certificates package with no shared history. I know Debian, Fedora, and openSUSE all have their own completely separate upstreams. Looking at what Fedora does is probably a good idea indeed, just keep in mind it has no shared history with Debian's package. I took a quick look at openSUSE's package and it looks like it has good p11-kit integration as well. Arch uses Fedora; not sure about other independent distros. They all use Mozilla's certificates, but Mozilla doesn't release a package in a way that's directly usable by distros.

Debian's ca-certificates implements certificate blacklisting by putting a ! character at the start of a line in /etc/ca-certificates.conf (which doesn't exist on other distros). Once a certificate is removed, it stays removed, see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=743339 which was never fixed.

Unfortunately, the ! character at the beginning the the line in ca-certificates.conf is just for blacklisting ca certificates from being imported into the system store, it's not really a backlist that can be used by a crypto library.

Also adding SSSD here, would be easy enough to make its default PAM CA ring to point to /etc/ssl/certs/ca-certificates.crt by default (and change-able in settings) but not sure if we want to go this route as it may make SSSD documentation confusing (as it everywhere mentions /etc/sssd/pki/sssd_auth_ca_db.pem or /etc/sssd/pki/sssd_auth_ca_db.pem).

Maybe a nice way would be to provide a default sssd.conf file that explicitly set that instead of hard-coding it, so we won't break current installations.

Status changed to 'Confirmed' because the bug affects multiple users.

Related: https://bugs.launchpad.net/ubuntu/+source/crypto-policies/+bug/1926664

(I might create a task here for crypto-policies and close the bug above)