gnutls regression: failure in certificate chain validation

It turns out that debian/patches/91_CVE-2008-4989.diff is missing an "add" chunk. Compare it with the gnutls advisory at
http://article.gmane.org/gmane.comp.encryption.gpg.gnutls.devel/3215

I rebuilt it locally with that missing chunk added to the patch and then it started to work.

The patch (ebian/patches/91_CVE-2008-4989.dif) is consistent with upstream's 2.6.2. The 2.6.1 fix for this CVE introduced a regression (see Debian http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=505279). Upstream's response (and patch) can be read about here:
http://article.gmane.org/gmane.comp.encryption.gpg.gnutls.devel/3248

Ubuntu never suffered from this regression, and has the full patch as seen in 2.6.2. Could gnutls have been too lenient in the past? If the certificate is self-signed, you should be able to use '--insecure' to connect to staging.

I should also mention that Jaunty doesn't trust the certificate either-- and it is the version from Debian sid (2.4.2-3) with no Ubuntu patches.

For a detailed discussion of the 2.6.2 fix, see http://thread.gmane.org/gmane.comp.encryption.gpg.gnutls.devel/3216/focus=3230

Upstream's 2.6.2 fails in the way.

So, here is a quick list of sites that "break":
- staging.landscape.canonical.com
- www.verisign.com
- www.citibank.com

These Debian bugs seem related:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=506853
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=507633

Upstream is discussing this here:
http://lists.gnu.org/archive/html/gnutls-devel/2008-12/msg00006.html

For completeness, 2.6.1 is also broken with www.verisign.com, so this affects all vendors who pushed out an update for CVE-2008-4989, no matter which patch they chose.

Upstream has a patch that I am currently testing.

This is fixed in 2.4.2-2 on Jaunty.

I can confirm Jaunty is OK with libgnutls26-2.4.2-4. The sites with problems are now working again, and the Landscape client is happy.

Oh yes, I meant to say libgnutls26-2.4.2-4. Packages are being prepared for the other releases and should be released soon.

This bug was fixed in the package gnutls13 - 1.6.3-1ubuntu0.3

---------------
gnutls13 (1.6.3-1ubuntu0.3) gutsy-security; urgency=low

  * Fix for regression where some valid certificate chains would be untrusted
    - Update debian/patches/91_CVE-2008-4989.diff to check if last certificate
      is self-signed and prevent verifying self-signed certificates against
      themselves. Patch from upstream.
    - http://lists.gnu.org/archive/html/gnutls-devel/2008-12/msg00008.html
    - LP: #305264

 -- Jamie Strandboge <email address hidden> Fri, 05 Dec 2008 14:49:34 -0600

This bug was fixed in the package gnutls13 - 2.0.4-1ubuntu2.3

---------------
gnutls13 (2.0.4-1ubuntu2.3) hardy-security; urgency=low

  * Fix for regression where some valid certificate chains would be untrusted
    - Update debian/patches/91_CVE-2008-4989.diff to check if last certificate
      is self-signed and prevent verifying self-signed certificates against
      themselves. Patch from upstream.
    - http://lists.gnu.org/archive/html/gnutls-devel/2008-12/msg00008.html
    - LP: #305264

 -- Jamie Strandboge <email address hidden> Fri, 05 Dec 2008 14:47:31 -0600

This bug was fixed in the package gnutls26 - 2.4.1-1ubuntu0.2

---------------
gnutls26 (2.4.1-1ubuntu0.2) intrepid-security; urgency=low

  * Fix for regression where some valid certificate chains would be untrusted
    - Update debian/patches/20_CVE-2008-4989.diff to check if last certificate
      is self-signed and prevent verifying self-signed certificates against
      themselves. Patch from upstream.
    - http://lists.gnu.org/archive/html/gnutls-devel/2008-12/msg00008.html
    - LP: #305264

 -- Jamie Strandboge <email address hidden> Fri, 05 Dec 2008 14:36:48 -0600

http://www.ubuntu.com/usn/usn-678-2

After installing 2.0.4-1ubuntu2.3 on hardy our ldap client setup stops working.

We get "pam_ldap: ldap_simple_bind Can't contact LDAP server" in syslog

and when using the ldap command line utility ldapsearch we get (with debug level 1)

TLS: peer cert untrusted or revoked (0x82)
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

using the gnutls command line utility gnutls-cli to verify the certificates works fine and in fact fails with when using 2.0.4-1ubuntu2.2 (but works with 2.0.4-1ubuntu2)

So, I'm a bit confused with the upstream discussion that happened around this bug and some proposed patches.

It seems that CAs self-signed with RSA-MD2 are "bad" and not supported, or something like that. I checked Ian's certificate chain and the last CA in the chain uses RSA-MD5 from what I could tell, so that doesn't seem to be the problem.

Maybe it's a specific bad interaction the openldap libraries have with this new gnutls version.

Ian, did the openldap server also get upgraded with this gnutls package?

We have 3 ldap servers. 2 running on ubuntu hardy with the gnutls version 2.0.4-1ubuntu2.2 (so not the latest update) and one running under debian etch using openssl.

I've tried using only the debian etch/openssl server and we still have the same problem.

I've tried connecting in TLS and SSL mode - both with the same results.

I think the problem is related to the way the openldap uses gnutls since the gnutls utility gnutls-cli thinks that the certificate is valid.

If you want me to try updating one of the ubuntu hardy ldap servers to use the latest update to see if it works then I can do that tomorrow at 8am GMT.

Ian, please make sure your Ubuntu clients and servers have the latest gnutls packages, and restart any services that use these libraries. 2.0.4-1ubuntu2.3 specifically has a fix for a problem in 2.0.4-1ubuntu2.2 which caused certain valid certificates to be considered untrusted.

I have now updated both our hardy servers and even rebooted one - this made no difference to the problem.

The openldap client libraries with the updated gnutls thinks our valid ssl key is invalid.

Ian, can you downgrade to 2.0.4-1ubuntu2 (or 2.0.4-1ubuntu2.1) on your clients and servers, restarting any necessary services and confirm that it works again. If it does, can you privately email me your certificate?

This bug may not be completely fixed, though I'm not sure -- check this out:

gnutls-cli -V --x509cafile /etc/ssl/certs/ca-certificates.crt login.launchpad.net

that tells me that the "Peer's certificate is NOT trusted", but with the "staging.landscape.canonical.com" in the original description it's working now (up-to-date intrepid).

I just tried 'login.launchpad.net' on unpatched and patched hardy and intrepid systems, and all fail. This certificate not working is not a regression, but may be a new bug.

I have finally been able to reproduce this with ldapsearch.

After performing:
$ sudo apt-get install ca-certificates ldap-utils

I tried to do on unpatched hardy:
$ LDAPTLS_CACERT=/etc/ssl/certs/ca-certificates.crt ldapsearch -ZZ -H ldaps://<Ian's public ldap server>:636/ -d 1
...
ldap_open_defconn: successful
...

and then on patched hardy:
$ LDAPTLS_CACERT=/etc/ssl/certs/ca-certificates.crt ldapsearch -ZZ -H ldaps://<Ian's public ldap server>:636/ -d 1
...
TLS: peer cert untrusted or revoked (0x82)
ldap_err2string
ldap_start_tls: Can't contact LDAP server (-1)

All patched versions of gnutls on Hardy, Intrepid, Jaunty and Debian Sid
are affected (Dapper and Gutsy ldap-utils use openssl and are not
affected).

I cannot reproduce this with the gnutls tools. I have Ian's certificate
and the result of:
$ certtool -e --infile <Ian's certificate>

is the same for unpatched and patched versions of gnutls on hardy and
intrepid, and also jaunty.

I then did:
$ gnutls-cli -V --x509cafile /etc/ssl/certs/ca-certificates.crt -p 636 \
<Ian's public ldap server>

and it works fine on patched and unpatched versions of gnutls on hardy
and intrepid, and also on jaunty.

The ldap issue has been reported in Debian bug http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=509593.

Ian, would you mind adding your ldap server URL to the bug? This way other developers can test against it. If not, I mentioned in the Debian report that I would give the URL to the maintainer privately.

I compiled 2.6.3 on Jaunty and it also gives the same error.

For clarity: Dapper, Gutsy, Hardy and Intrepid have the same patch as Jaunty (and Sid), which is the same as upstream 2.6.3.

The Debian gnutls maintainer points to
<http://news.gmane.org/find-root.php?message_id=%3c49654581.3020505%40anl.gov%3e>, which shows how this is a gnutls bug rather than an openldap one. Reopening the gnutls tasks and closing the openldap tasks.

The upstream commit is given here.
http://git.savannah.gnu.org/gitweb/?p=gnutls.git;a=commitdiff;h=423fc8b82f2b9aa3ea820cd5cf75d5813dffbbf0

Note, however, that this commit only fixes the problem when passing certain non-default options to gnutls, which are not passed by openldap, to enable use of V1 SSL certificates. Ultimately, these certificate chains worked with OpenLDAP+GnuTLS by accident, not design, as a result of the bug fixed in this security update.

Upstream is opposed to changing the default flags to enable V1 certificates because V1 certs are vulnerable to various sorts of attack and GnuTLS is documented to not support these by default. I think it's inappropriate to change the default flags in OpenLDAP for the same reason. If it's determined that enabling V1 certs is the lesser evil, I think it makes more sense to enable them globally than to enable them just in OpenLDAP, since this potentially affects all consumers of libgnutls.

As for whether enabling them is the lesser evil, note that the attacks V1 certs are subject to are not a strict subset of the attacks GnuTLS was subject to prior to this security update, so there's no easy choice here.

Commenting per request in #ubuntu-meeting.

It is a really unfortunate situation that these certificates unintentionally passed verification before the updates. IMO, the security fix (that is also in other distributions now) is needed and should not be backed out. Without it, man-in-the middle attacks against certificate chains are much easier to conduct. From a security perspective, the patch needs to stay and the gnutls defaults of disabling V1 certificates need to stay the same.

I am well aware that the current situation breaks certain configurations, and do not feel I can make the final decision.

There is also the patch in bug #314915, also discussed upstream, that may be an option. AFAICT, this patch has not been applied upstream yet and I feel uncomfortable applying it without more Debian and Gnutls feedback (lately, each time this section of code has been touched another bug in the certificate chain verification popped up).

I believe applying the propsed patch is becoming increasingly urgent. It is obviously in the 2.6.4 and 2.4.3 releases of GnuTLS and AFAIK, it didn't break anything.

Pinning down on 2.0.4-1 of libgnutls13 on is not a long-term solution, especially not for an LTS system. The patch has been verified as working in staging environments, and I believe we have to come to a decision.

Maintaining my own version of gnutls for the next 4 years doesn't really appeal to me either..

Also, in my experience it is not uncommon to use home brewed root certificates without the basicConstraints extension, i.e. for authentication of the directory service. This configuration fails with the current ubuntu version.

Further discussion led to the observation that OpenLDAP's gnutls support is a port of the existing OpenSSL handling, and it's therefore reasonable for openldap itself to enable the V1 CA cert option in order to provide feature parity when building with GnuTLS vs. OpenSSL, even if this is not altogether desirable from a security POV. I'm therefore reopening the openldap tasks for those releases where openldap is linked against GnuTLS.

The upstream discussion also points to regressions in behavior that are side effects of the change, rather than deliberate security enhancements, which should therefore be fixed in the gnutls26 package still - so leaving those tasks open also.

Upstream released 2.4.3 to address both the vulnerability and the known regressions. Reviewing upstream's mailing list shows no regressions so far with this version. I've sync'd Jaunty with 2.4.2-6, which brings its patches in line with upstream 2.4.3, so I am marking Jaunty as 'Fix Released'.

I have backported the relevant patches to Dapper through Intrepid, and am testing them now. I will upload them shortly for testing.

Thanks.

Jamie Strandboge wrote:
> Upstream released 2.4.3 to address both the vulnerability and the known
> regressions. Reviewing upstream's mailing list shows no regressions so
> far with this version. I've sync'd Jaunty with 2.4.2-6, which brings its
> patches in line with upstream 2.4.3, so I am marking Jaunty as 'Fix
> Released'.
>
> I have backported the relevant patches to Dapper through Intrepid, and
> am testing them now. I will upload them shortly for testing.
>
> ** Changed in: gnutls26 (Ubuntu Jaunty)
> Status: Triaged => Fix Released
>

--

  Douglas E. Engert <email address hidden>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois 60439
  (630) 252-5444

Dapper - Intrepid have been uploaded to the ubuntu-security-proposed ppa (https://launchpad.net/~ubuntu-security-proposed/+archive/ppa). Once they have finished building, they can be pocket copied to -proposed and people can use https://wiki.ubuntu.com/Testing/EnableProposed. Please leave feedback here so they can be moved to -updates.

Dapper through Intrepid have been copied to -proposed now.

Tried the Intrepid version, looks like it works. Thanks.

Jamie Strandboge wrote:
> Dapper through Intrepid have been copied to -proposed now.
>
> ** Tags added: verification-needed
>

--

  Douglas E. Engert <email address hidden>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois 60439
  (630) 252-5444

I was able to reproduce the libldap client bug:

0. Need two versions of openldap : one compiled with gnutls, the other with
openssl.

1. Create a V1 CA.
2. Create a certificate to be used by slapd and sign it with the V1 CA.
3. Configure a slapd+openssl system with certificates issues above.
4. Try to connect to the slapd+openssl system with a libldap+gnutls client:

mathiaz@t-slapd-gnutls:~$ ldapsearch -b "dc=vmnet" -D "cn=admin,dc=vmnet" -x -w
mypwd -H ldaps://t-slapd-openssl./ -d 1
ldap_url_parse_ext(ldaps://t-slapd-openssl./)
ldap_create
ldap_url_parse_ext(ldaps://t-slapd-openssl.:636/??base)
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP t-slapd-openssl.:636
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 172.19.42.220:636
ldap_pvt_connect: fd: 3 tm: -1 async: 0
TLS: peer cert untrusted or revoked (0x82)
TLS: can't connect: (unknown error code).
ldap_err2string
ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)

I've filed a bug in openldap bug tracker:
http://www.OpenLDAP.org/its/index.cgi?findid=5992

Note that a possible workaround is to put *all* of the CA certificates from the chain in the TLS_CACERT file. If one of the intermediate CA certs is a V3 certificate gnutls shouldn't fail and the ldap connection should proceed.

This is actually mentioned in the Admin guide:

16.2.2.1. TLS_CACERT <filename>

  This is equivalent to the server's TLSCACertificateFile option. As noted in the TLS Configuration section, a client typically may need to know about more CAs than a server, but otherwise the same considerations apply.

which points to section 16.2.1.1. TLSCACertificateFile <filename>:

  If the signing CA was not a top-level (root) CA, certificates for the entire sequence of CA's from the signing CA to the top-level CA should be present.

libldap is now patched in OpenLDAP cvs HEAD. We anticipate releasing a bugfix-only 2.4.16 release very soon, with this fix included.

One workaround is to put all of the CA certs in the trusted CA certificate file.

If the system running slapd is on hardy (or intrepid or jaunty) you should also add all of the CA certificates to the server certificate file - this is to workaround a bug where the slapd daemon doesn't send all of the CA certificates to the client.

This bug was fixed in the package openldap - 2.4.15-1ubuntu1

---------------
openldap (2.4.15-1ubuntu1) jaunty; urgency=low

  [ Steve Langasek ]
  * Update priority of libldap-2.4-2 to match the archive override.
  * Add the missing ldapexop and ldapurl tools to ldap-utils, as well as the
    ldapurl(1) manpage. Thanks to Peter Marschall for the patch.
    Closes: #496749.
  * Bump build-dependency on debhelper to 6 instead of 5, since that's
    what we're using. Closes: #498116.
  * Set the default SLAPD_SERVICES to ldap:/// ldapi:///, instead of using
    the built-in default of ldap:/// only.

  [ Mathias Gug ]
  * Merge from debian unstable, remaining changes:
    - Modify Maintainer value to match the DebianMaintainerField
      speficication.
    - AppArmor support:
      - debian/apparmor-profile: add AppArmor profile
      - debian/slapd.postinst: Reload AA profile on configuration
      - updated debian/slapd.README.Debian for note on AppArmor
      - debian/control: Recommends apparmor >= 2.1+1075-0ubuntu6
      - debian/control: Conflicts with apparmor-profiles << 2.1+1075-0ubuntu4
        to make sure that if earlier version of apparmour-profiles gets
        installed it won't overwrite our profile.
      - follow ApparmorProfileMigration and force apparmor compalin mode on
        some upgrades (LP: #203529)
      - debian/slapd.dirs: add etc/apparmor.d/force-complain
      - debian/slapd.preinst: create symlink for force-complain on pre-feisty
        upgrades, upgrades where apparmor-profiles profile is unchanged (ie
        non-enforcing) and upgrades where apparmor profile does not exist.
      - debian/slapd.postrm: remove symlink in force-complain/ on purge
    - debian/control:
      - Build-depend on libltdl7-dev rather then libltdl3-dev.
    - debian/patches/autogen.sh:
      - Call libtoolize with the --install option to install config.{guess,sub}
        files.
    - Don't use local statement in config script as it fails if /bin/sh
      points to bash (LP: #286063).
    - Disable the testsuite on hppa. Allows building of packages on this
      architecture again, once this package is in the archive.
      LP: #288908.
    - debian/slapd.postinst, debian/slapd.script-common: set correct ownership
      and permissions on /var/lib/ldap, /etc/ldap/slapd.d (group readable) and
      /var/run/slapd (world readable). (LP: #257667).
    - Enable nssoverlay:
      - debian/patches/nssov-build, debian/rules: Build and package
        the nss overlay.
      - debian/schema/misc.ldif: add ldif file for the misc schema
        which defines rfc822MailMember (required by the nss overlay).
    - debian/{control,rules}: enable PIE hardening
    - Use cn=config as the default configuration backend instead of
      slapd.conf. Migrate slapd.conf file to /etc/ldap/slapd.d/ on upgrade
      asking the end user to enter a new password to control the access to the
      cn=config tree.
  * Dropped:
    - debian/patches/corrupt-contextCSN: The contextCSN can get corrupted at
      times. (ITS: #5947) Fixed in new upstream version 2.4.15.
    - debian/patches/fix-ucred-libc due to changes how newer glibc handle
      the ucred struct...

I've attached the specific patch that enable V1 Certs to be trusted.

Mathias Gug wrote:
> One workaround is to put all of the CA certs in the trusted CA
> certificate file.

Yes, that is what we have had to do.

The real fix is to get the gnutls people to support certificate
directories, like OpenSSL. Why the rush to convert to gnutls
when it has so many issues. (Licencing issues are low on my list of
reasons.)

>
> If the system running slapd is on hardy (or intrepid or jaunty) you
> should also add all of the CA certificates to the server certificate
> file - this is to workaround a bug where the slapd daemon doesn't send
> all of the CA certificates to the client.

All or just the intermediate certificates?

Another issue with gnutls, no intermediate file (or directory) of
certificates.

>

--

  Douglas E. Engert <email address hidden>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois 60439
  (630) 252-5444

On Mon, Mar 09, 2009 at 02:21:58PM -0000, Doug Engert wrote:
> The real fix is to get the gnutls people to support certificate
> directories, like OpenSSL. Why the rush to convert to gnutls
> when it has so many issues. (Licencing issues are low on my list of
> reasons.)

Licensing was the main motivation to move to 2.4 and GnuTLS. The other
option was to keep the client libraries to 2.1.

> > If the system running slapd is on hardy (or intrepid or jaunty) you
> > should also add all of the CA certificates to the server certificate
> > file - this is to workaround a bug where the slapd daemon doesn't send
> > all of the CA certificates to the client.
>
> All or just the intermediate certificates?
>

The intermediate certificates should be enough. If not all of them
should work.

> Another issue with gnutls, no intermediate file (or directory) of
> certificates.
>

Please open a new bug to track this specific issue.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

I've noticed strange behaviour which could be related to this bug.

#certtool -i < ldap-cert.pem | grep -i issu
        Issuer: C=RU,ST=State,L=City,O=company,OU=SysAdmin,CN=ca.domain.my,<email address hidden>

#certtool -e --load-ca-certificate cacert.pem < ldap-cert.pem
        Issued by: C=RU,ST=State,L=City,O=company,OU=SysAdmin,CN=ldap.florist.my,<email address hidden>
certtool: Error: The last certificate is not self signed.

Notice that in verify mode it things tha issuer is the same as CN of ldap-cert.pem, while in information mode it show issuer correctly.

Of course openssl verify verifies ldap-cert.pem seamlessly.

Even though the issue has been reported as 'fixed' I am still facing this problem with an OpenBSD OpenLDAP server:

# ldapsearch -x -b 'dc=ini,dc=uzh,dc=ch' uid=stephan -H ldap://ldap.ini.uzh.ch -ZZ -d1
...

TLS: peer cert untrusted or revoked (0x42)
ldap_err2string
ldap_start_tls: Connect error (-11)

I've tried on Hardy and Intrepid with the same results. I tried specifing TLS_CACERT with no luck.

A openSUSE 11.0 clients works out of the box, though.

On Thu, Mar 26, 2009 at 04:35:38PM -0000, star26bsd wrote:
> Even though the issue has been reported as 'fixed' I am still facing
> this problem with an OpenBSD OpenLDAP server:
>
> # ldapsearch -x -b 'dc=ini,dc=uzh,dc=ch' uid=stephan -H ldap://ldap.ini.uzh.ch -ZZ -d1
> ...
>
> TLS: peer cert untrusted or revoked (0x42)

@Stephan: Please include the information requested at
https://wiki.ubuntu.com/DebuggingOpenldap#ssl-client-failure.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Mathias, in regards to the wiki you linked above, my preference when debugging these issues is to recommend debug level 7, which includes packet traces, instead of debug 1. It's much better (to me) to be able to see all the traffic, which includes the raw transfer of certificates and their DER DNs, when tracking down TLS problems.

Accepted openldap into intrepid-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Accepted openldap into hardy-proposed; please test and give feedback here. Please see https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

@Martin Pitt: Ok, here's all the stuff:

$ ldapsearch -x -b 'dc=ini,dc=uzh,dc=ch' uid=stephan -H ldap://ldap.ini.uzh.ch -ZZ -d7

ldap_url_parse_ext(ldap://ldap.ini.uzh.ch)
ldap_create
ldap_url_parse_ext(ldap://ldap.ini.uzh.ch:389/??base)
ldap_extended_operation_s
ldap_extended_operation
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP ldap.ini.uzh.ch:389
ldap_new_socket: 3
ldap_prepare_socket: 3
ldap_connect_to_host: Trying 172.16.3.220:389
ldap_pvt_connect: fd: 3 tm: -1 async: 0
ldap_open_defconn: successful
ldap_send_server_request
ber_scanf fmt ({it) ber:
ber_scanf fmt ({) ber:
ber_flush2: 31 bytes to sd 3
  0000: 30 1d 02 01 01 77 18 80 16 31 2e 33 2e 36 2e 31 0....w...1.3.6.1
  0010: 2e 34 2e 31 2e 31 34 36 36 2e 32 30 30 33 37 .4.1.1466.20037
ldap_write: want=31, written=31
  0000: 30 1d 02 01 01 77 18 80 16 31 2e 33 2e 36 2e 31 0....w...1.3.6.1
  0010: 2e 34 2e 31 2e 31 34 36 36 2e 32 30 30 33 37 .4.1.1466.20037
ldap_result ld 0x6121b0 msgid 1
wait4msg ld 0x6121b0 msgid 1 (infinite timeout)
wait4msg continue ld 0x6121b0 msgid 1 all 1
** ld 0x6121b0 Connections:
* host: ldap.ini.uzh.ch port: 389 (default)
  refcnt: 2 status: Connected
  last used: Wed Apr 1 10:27:29 2009

** ld 0x6121b0 Outstanding Requests:
 * msgid 1, origid 1, status InProgress
   outstanding referrals 0, parent count 0
  ld 0x6121b0 request count 1 (abandoned 0)
** ld 0x6121b0 Response Queue:
   Empty
  ld 0x6121b0 response count 0
ldap_chkResponseList ld 0x6121b0 msgid 1 all 1
ldap_chkResponseList returns ld 0x6121b0 NULL
ldap_int_select
read1msg: ld 0x6121b0 msgid 1 all 1
ber_get_next
ldap_read: want=8, got=8
  0000: 30 0c 02 01 01 78 07 0a 0....x..
ldap_read: want=6, got=6
  0000: 01 00 04 00 04 00 ......
ber_get_next: tag 0x30 len 12 contents:
read1msg: ld 0x6121b0 msgid 1 message type extended-result
ber_scanf fmt ({eAA) ber:
read1msg: ld 0x6121b0 0 new referrals
read1msg: mark request completed, ld 0x6121b0 msgid 1
request done: ld 0x6121b0 msgid 1
res_errno: 0, res_error: <>, res_matched: <>
ldap_free_request (origid 1, msgid 1)
ldap_free_connection 0 1
ldap_free_connection: refcnt 1
ldap_parse_extended_result
ber_scanf fmt ({eAA) ber:
ldap_parse_result
ber_scanf fmt ({iAA) ber:
ber_scanf fmt (}) ber:
ldap_msgfree
tls_write: want=82, written=82
  0000: 16 03 02 00 4d 01 00 00 49 03 02 49 d3 25 71 2b ....M...I..I.%q+
  0010: 1e 5d fd 39 7b 4b 7f 7e 6a ac 75 04 40 44 e5 db .].9{K.~j.u.@D..
  0020: 94 7a e8 71 c5 8f 15 3b 21 e9 16 00 00 18 00 39 .z.q...;!......9
  0030: 00 33 00 16 00 38 00 32 00 13 00 66 00 35 00 2f .3...8.2...f.5./
  0040: 00 0a 00 05 00 04 02 01 00 00 07 00 09 00 03 02 ................
  0050: 00 01 ..
tls_read: want=5, got=5
  0000: 16 03 01 00 4a ....J
tls_read: want=74, got=74
  0000: 02 00 00 46 03 01 49 d3 25 6c 7c 54 63 e1 09 a6 ...F..I.%l|Tc...
  0010: 4d 29 bc 73 64 5c 63 38 ce fe 5e 54 59 16 e1 2c M).sd\c8..^TY...

@Stephan:

Could you provide the output of the following command run on the system where the ldap failure happens:

dpkg-query -W -f='${Package} ${Version} ${Source} ${Status}\n' | egrep 'slapd|ldap|gnutls'

The 18 month support period for Gutsy Gibbon 7.10 has reached its end of life -
http://www.ubuntu.com/news/ubuntu-7.10-eol . As a result, we are closing the
Gutsy task.

I guess we are having the same problem authenticating against a sun open directory server. I use intrepid-proposed on my client:

root@client:~# dpkg-query -W -f='${Package} ${Version} ${Source} ${Status}\n' | egrep 'slapd|ldap|gnutls'
gnutls-bin 2.4.1-1ubuntu0.3 gnutls26 install ok installed
ldap-auth-client 0.5.2 install ok installed
ldap-auth-config 0.5.2 ldap-auth-client install ok installed
ldap-utils 2.4.11-0ubuntu6.2 openldap install ok installed
libcurl3-gnutls 7.18.2-1ubuntu4.3 curl install ok installed
libgnutls26 2.4.1-1ubuntu0.3 gnutls26 install ok installed
libldap-2.4-2 2.4.11-0ubuntu6.2 openldap install ok installed
libnss-ldap 260-1ubuntu2 install ok installed
libpam-ldap 184-4ubuntu2 install ok installed

The rest is attached:
ldap.conf
output of gnutls-cli -p636 --x509cafile CAFILE srv.obf.obf.ob
output of ldapsearch -x -ZZ -d7

If you need anything else, please ask.

Doug Engert wrote:

>The real fix is to get the gnutls people to support certificate
>directories, like OpenSSL. Why the rush to convert to gnutls
>when it has so many issues. (Licencing issues are low on my list of
>reasons.)

Indeed, for a security tool you want a package written by experienced security developers, not a science project. This isn't a game after all. GnuTLS doesn't even merit a version number greater than 0.5, IMO.

http://www.openldap.org/lists/openldap-devel/200802/msg00072.html

Sorry for my report, it was out of confusion between /etc/ldap.conf and /etc/ldap/ldap.conf. I think their names are rather unfortunate, but this is another issue.

Mika

For the gnutls/hardy SRU:

I have reproduced the acceptance of rsa/md2 v1 certificates by the version of gnutls13 in hardy-updates, 2.0.4-1ubuntu2.3, and can confirm that the version of gnutls13 in hardy-proposed does not accept rsa/md2 certificates. I have added a testcase for this situation in the gnutls test script in the lp:qa-regression-testing bzr tree. The package passes the rest of the regression test in the testsuite, with the exception of known bug 292604 which is not a regression (2.0.4-1ubuntu2.3 also fails this test and it looks like it won't get fixed in hardy).

For the openldap/hardy SRU:

I have:

  (1) reproduced the acceptance of the v1 certificates as
      outlined in Mathias' test case by the ldap clients with ldap
      2.4.9-0ubuntu0.8.04.2 and gnutls13 2.0.4-1ubuntu2.

  (2) reproduced the rejection of v1 certificates by the ldap clients
      with ldap 2.4.9-0ubuntu0.8.04.2 and gnutls13 versions
      2.0.4-1ubuntu2.3 and 2.0.4-1ubuntu2.5.

  (3) confirmed that the with the version of the ldap packages in
      hardy-proposed, 2.4.9-0ubuntu0.8.04.3, that the v1 certificates
      are once again accepted when using gnutls13 versions
      2.0.4-1ubuntu2.3 and 2.0.4-1ubuntu2.5.

I have also run the ldap packages in hardy-proposed through the
openldap testcases in the lp:qa-regression-testing bzr tree with all
three available versions of gnutls13 (gnutls13 2.0.4-1ubuntu2,
2.0.4-1ubuntu2.3, 2.0.4-1ubuntu2.5) and confirmed that there were no
introduced regressions seen there as well.

I consider the openldap packages verification-done for hardy.

This bug was fixed in the package gnutls13 - 2.0.4-1ubuntu2.5

---------------
gnutls13 (2.0.4-1ubuntu2.5) hardy-security; urgency=low

  * Fix for certificate chain regressions introduced by fixes for
    CVE-2008-4989
  * debian/patches/91_CVE-2008-4989.diff: updated to upstream's final
    2.4.2 - 2.4.3 patchset for lib/x509/verify.c to fix CVE-2008-4989 and
    address all known regressions. To summarize from upstream:
    - Fix X.509 certificate chain validation error (CVE-2008-4989)
    - Fix chain verification for chains that end with RSA-MD2 CAs (LP: #305264)
    - Deprecate X.509 validation chains using MD5 and MD2 signatures
    - Accept chains where intermediary certs are trusted (LP: #305264)

 -- Jamie Strandboge <email address hidden> Fri, 20 Feb 2009 13:02:36 -0600

I'm seeing problems with the new version.

Tests with either SSH or sudo, my first password attempt is rejected, yet the second attempt succeeds.
I get this in the logs:
pam_ldap: ldap_starttls_s: Connect error

Setting tls_checkpeer to no in /etc/ldap.conf makes things work fine again.

Reverting to 2.0.4-1ubuntu2.3 correct the issue as well.

@Andy:

Could you describe the X509 certs and CA you're using?

Mathias Gug wrote:
> @Andy:
>
> Could you describe the X509 certs and CA you're using?
>

We were using ldap and Verisign, and the root CA was a V2 from 1999
which signed an intermediate cert that signed the server certs.

I submitted to gnutls a few changes to allow for stoping at the
intermediate cert which I believe they added.

In the meantime, we turned off cert checking, and have now
replaced LDAP Verisign certs with certs issued localy.

I will send you a copy of the note to gnutls from 1/8/2009
which has the certs.

--

  Douglas E. Engert <email address hidden>
  Argonne National Laboratory
  9700 South Cass Avenue
  Argonne, Illinois 60439
  (630) 252-5444

Copy of note sent on 1/8/2009:

Attached are the server cert (auth2.it.anl.gov), the intermediate cert (f0a38a80.0)
and the CA self signed cert (7651b327.0) a debug version of verify.c
and partial output of an ldapsearch using the debug.c

My patch has been #if 0'ed out at line 151.

   Lets refer to the cert chain as A, B and C. The OpenLDAP server (using OpenSSL)
sends server cert A, intermediate cert B, and CA cert C.

The TLS_CACERT file has B and C.

The clist_size is then 3, and the code in _gnutls_x509_verify_certificate
around lines 443 drop it to 2, leaving the chain as A, B.

The tcase_size is 2.

_gnutls_verify_certificate2 at line 452 is called with cert B and
tcas with B and C and flags 0.

At line 265, find_issuer is called with B. It returns C.
check_is_ca is called at line 297, which fails
because there is no BasicConstraint. The if at 293 looks correct too.

*BUT* if one trusts both B and C, do we need to verify C?
Why does the code arount line 265 not stop after finding that B is in the tcas,
rather then looking for C, and then verifying it?

If I try it again with the TLS_CACERT file with only B,
it also fails because it can not find the issuer of B.
If the code around line 265 was modified if B was found in the tcas,
this shopuld also work.

Simon Josefsson wrote:
> "Douglas E. Engert" <email address hidden> writes:
>
>> This is also being submitted to https://bugs.launchpad.net/bugs
>>
>> Using the Ubuntu version of libgnutls13_2.0.4-1ubuntu2.3 on Hardy 8.04.1,
>> ldaps: has stopped working. This looks like it is related to
>> the December changes that are also in gnutls-2.6.3. See attached
>> patch that should work in both.
>>
>> ldapsearch -d 1 -H ldaps://...
>>
>> TLS: peer cert untrusted or revoked (0x82)
>> ldap_err2string
>> ldap_sasl_bind(SIMPLE): Can't contact LDAP server (-1)
>>
>>
>> The OpenLDAP ldap server certificate issued by Verisign is signed by:
>>
>> Verisign_Intermediate-Secure_Site_Managed_PKI_for_SSL_Standard_Certificates.pem
>>
>> which is signed by:
>> Verisign_Class_3_Public_Primary_Certification_Authority.pem
>>
>> Both of these are in /etc/ssl/certs as 7651b327.0 and f0a38a80.0
>>
>> Verisign_Class_3_Public_Primary_Certification_Authority.pem
>> is a self signed version 1 cert issued in 1996, with no extensions.
>
> Do you have a complete chain that triggers this? It will help our
> regression test suite.
>
> I don't have f0a38a80.0 on my debian lenny system. Does it also lack a
> basicConstraint? Does it use RSA-MDx? If yes, that would explain the
> problem.
>
>> In lib/x509/verify.c gnutls_x509_crt_get_ca_status is called
>> but returns GNUTLS_E_REQUESTED_DATA_NOT_AVAILABLE as there is no
>> Basic Constraint.
>>
>> The attached patch (to gnutls13_2.0.4-1ubuntu2.3) checks for
>> this return and if it is a self signed cert, will treat it as a CA.
>> The patch looks like it can be applied to 2.6.3 as well.
>
> The patch seems too permissive to me: the intention is that V1 certs
> should be rejected by GnuTLS unless GNUTLS_VERIFY_ALLOW_X509_V1_CA_CRT
> and/or GNUTLS_VERIFY_ALLOW_ANY_X509_V1_CA_CRT are passed as flags.
>
> Internally, GnuTLS by default enables the
> GNUTLS_VERIFY_ALLO...

I am using a self created CA with certificates signed by it.

I used this command to create it:
openssl req -config openssl.cnf -new -x509 -extensions v3_ca -keyout physicsCA/private/cakey.pem -out physicsCA/cacert.pem -days 2190

I create and sign the certificates with these commands:

openssl req -config openssl.cnf -new -nodes -keyout $1.key -out $1.csr -days 1065
openssl ca -config openssl.cnf -policy policy_anything -out $1.crt -infiles $1.csr

The CA certificate file is distributed to all of my machines and is specified in the ldap.conf.

If you want me to attach the openssl.cnf let me know.

Hi Andy,

On Thu, Jul 09, 2009 at 03:51:04PM -0000, Andy Wettstein wrote:
> If you want me to attach the openssl.cnf let me know.

Could you please attach your openssl.cnf file so that it's easier to
reproduce your environment?

Thank you,

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Attaching my openssl.cnf

I ran into the same problem (the update to libgnutls13 2.0.4-1ubuntu2.5 broke LDAP auth, due to the certificate chain no longer validating). The quick fix was to set TLS_REQCERT to allow in /etc/ldap/ldap.conf, but that is just a temporary workaround.

Indeed, using gnutls-cli to connect to server:636 fails verification with this version, whereas after rolling back to -ubuntu2.3 it succeeds. 'openssl s_client' is still happy. I discovered that this is because both my CA cert and the server cert were using md5/rsa, which are no longer supported by gnutls. Note the changelog entry " - Deprecate X.509 validation chains using MD5 and MD2 signatures"; apparently by "deprecate" they mean "no longer support at all".

Apparently gnutls doesn't mind the self-signed CA cert being MD5, but it's not happy with the server cert:
andrew@shiny:~$ echo|gnutls-cli --print-cert --x509cafile isc-ca.crt -p 636 ldap.server | certtool -e
Certificate[0]: C=US,ST=Texas,L=College Station,O=TAMU,OU=VPR,CN=ldap.server,<email address hidden>
 Issued by: C=US,ST=Texas,L=College Station,O=TAMU,OU=VPR,CN=ISC CA,<email address hidden>
 Verifying against certificate[1].
 Verification output: Not verified, Insecure algorithm.

Certificate[1]: C=US,ST=Texas,L=College Station,O=TAMU,OU=VPR,CN=ISC CA,<email address hidden>
 Issued by: C=US,ST=Texas,L=College Station,O=TAMU,OU=VPR,CN=ISC CA,<email address hidden>
 Verification output: Verified.
[names/emails sanitized somewhat]

After generating a new server cert using sha1/rsa and plugging it into slapd (but still using the same md5/rsa CA cert), gnutls is now happy, and in turn, so is pam_ldap. I suppose the better solution would be to create a new sha1 CA cert also, but that would require copying it to all the clients, which is a lot more work.

Andy, I notice you have default_md=md5 in your openssl.cnf. You should change this to sha1 or something else not considered broken, and generate a new server cert, and that should solve your problem.

In summary, the recent gnutls update broke MD5 certs, but this was intentional. It would've been nice to announce this more loudly though. (To me, "deprecate" means "discourage future use", not "remove support".)

We need to push gnutls12 in Dapper and gnutls26 in Intrepid in -proposed to -security since these fix CVE-2009-2409. Dapper should not be a problem with openldap since openldap uses libssl0.9.8 on Dapper. For Intrepid, openldap will need to be copied as was done with Hardy.

Hardy openldap2.3 was fixed awhile ago, but didn't auto-close:

openldap2.3 (2.4.9-0ubuntu0.8.04.3) hardy-proposed; urgency=low

  * debian/patches/gnutls-enable-v1-ca-certs: Enable V1 CA certs to be
      trusted (LP: #305264).

 -- Mathias Gug < <email address hidden> (mathiaz: 10900) [universe-contributors] [ubuntumembers] [ubuntu-dev] [ubuntu-bugcontrol] [ubuntu-core-dev] > Wed, 25 Mar 2009 14:30:35 -0400

Copied gnutls12 1.2.9-2ubuntu1.5 to dapper-security and dapper-updates

This bug was fixed in the package gnutls26 - 2.4.1-1ubuntu0.3

---------------
gnutls26 (2.4.1-1ubuntu0.3) intrepid-security; urgency=low

  * Fix for certificate chain regressions introduced by fixes for
    CVE-2008-4989
  * debian/patches/20_CVE-2008-4989.diff: updated to upstream's final
    2.4.2 - 2.4.3 patchset for lib/x509/verify.c to fix CVE-2008-4989 and
    address all known regressions. To summarize from upstream:
    - Fix X.509 certificate chain validation error (CVE-2008-4989)
    - Fix chain verification for chains that end with RSA-MD2 CAs (LP: #305264)
    - Deprecate X.509 validation chains using MD5 and MD2 signatures
    - Accept chains where intermediary certs are trusted (LP: #305264)

 -- Jamie Strandboge <email address hidden> Fri, 20 Feb 2009 12:24:43 -0600

Copied gnutls26 2.4.1-1ubuntu0.3 from -proposed to -security and -updates.

This bug was fixed in the package openldap - 2.4.11-0ubuntu6.2

---------------
openldap (2.4.11-0ubuntu6.2) intrepid-proposed; urgency=low

  * debian/patches/gnutls-enable-v1-ca-certs: Enable V1 CA certs to be
    trusted (LP: #305264).

 -- Mathias Gug <email address hidden> Wed, 25 Mar 2009 12:52:23 -0400

Copied openldap 2.4.11-0ubuntu6.2 from -proposed to -updates.

Just noting for posterity, as of GnuTLS 2.8.0 (released 2009-05-27) you can use %VERIFY_ALLOW_X509_V1_CA_CRT in the TLSCipherSuite options to enable V1 CA certs. I will probably #ifdef the current OpenLDAP patch to turn it off for GnuTLS >= 2.8.0. (Haven't decided on best course of action yet, given http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=541256 )