olcAccess are options broken on upgrade in {-1}frontend.ldif

I've reviewed that code and came up with a much simpler to handle root access. That should make upgrade more robust the root olcAccess is just stuck at the beginning of the olcAccess list. Existing olcAccess from upgrades are still applied as the inserted line ends with "by *break".

I also got ready of the cn=localroot,cn=config mapping which could be exist on systems. It makes the whole thing simpler and more robust.

I took a quick look through the new slapd.postinst script found in:
 lp:~mathiaz/ubuntu/lucid/openldap/fix-root-olcaccess-upgrade

Am I correct that you no longer attempt to delete the
  olcAccess: {0}to * by * none
line from the olcDatabase={0}config.ldif file (i.e the line that is generated automatically by the slapd.conf -> slapd.d conversion, e.g. during a Hardy->Lucid upgrade)?

In my quick testing, I found that having that line still in the file prevented me from accessing that part of the tree (even though it appeared after the new gidNumber=0 line). For example, when "grep olcAccess olcDatabase\=\{0\}config.ldif" returned these two lines:

  olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break
  olcAccess: {0}to * by * none

, then an ldapsearch returned:
=====================
[...]
# LDAPv3
# base <olcDatabase={0}config,cn=config> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# search result
search: 2
result: 32 No such object

# numResponses: 1
=====================

But, when I stopped slapd, removed the "olcAccess: {0}to * by * none" line by hand, and restarted slapd, then the exact same ldapsearch command returned data:
=====================
# LDAPv3
# base <olcDatabase={0}config,cn=config> with scope subtree
# filter: (objectclass=*)
# requesting: ALL
#

# {0}config, config
dn: olcDatabase={0}config,cn=config
objectClass: olcDatabaseConfig
olcDatabase: {0}config
olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external
 ,cn=auth manage by * break
[...]

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1
=====================

(On the other hand, I didn't find much explanation about using the "gidNumber=0" form of authentication, other than the very brief mention of the switch to it in the openldap 2.4.17-1ubuntu3 release notes entry, so perhaps I missed something when running these tests.... The command line I ended up using was
  # ldapsearch -Y EXTERNAL -Hldapi:/// -b "olcDatabase={0}config,cn=config"
, run as root... but let me know if that wasn't actually testing what I should have been testing....)

Also, should the edits made to the the olcDatabase={-1}frontend.ldif file include granting
  to dn.base="" by * read'
permissions, too? It appears that that statement exists in (for example) the Hardy version of slapd.conf, but the slapd.conf -> slapd.d conversion migrates it to the olcDatabase={1}hdb.ldif file only.

In slapd 2.4.21-0ubuntu4, the slapd.init.ldif f was edited to include that access in the dn: olcDatabase={-1}frontend,cn=config section (LP#427842), but no attempt was made to get that permission fixed up after a the slapd.conf -> slapd.d conversion.

(LPb#427842 also includes a
  to dn.base="cn=subschema" by * read
permission line. I don't see that line in my old slpad.conf file, but based on the discussion in that bug, I'm wondering if the postinst script should be adding it to the {-1}frontend.ldif file as well?)

Nathan

(Obviously, that should be LP: #427842 .)

+ sed -i 's/^\(olcDatabase: {-1}frontend\)/\0\nolcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break/' "${SLAPD_CONF}/cn=config/olcDatabase={-1}frontend.ldif"
+ sed -i 's/^\(olcDatabase: {0}config\)/\0\nolcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break/' "${SLAPD_CONF}/cn=config/olcDatabase={0}config.ldif"

I know this isn't new to this patch, but I think I need to teach someone about the sed 'a' command :)

This patch doesn't appear to clean up any previously-added olcAuthzRegexp lines, or previously-added olcAccess lines referencing localroot, so the resulting config will be different for users upgrading to lucid final from an earlier release, vs. users who have upgraded to lucid RC. Is this ok?

This bug was fixed in the package openldap - 2.4.21-0ubuntu5

---------------
openldap (2.4.21-0ubuntu5) lucid; urgency=low

  * Fix local root connection access: replace olcAuthzRegexp mapping to
    cn=localroot,cn=config with using the SASL dn directly in olcAccess.
    Makes upgrades much simpler and robust (LP: #563829).
 -- Mathias Gug <email address hidden> Fri, 23 Apr 2010 00:23:31 -0400

On Mon, Apr 26, 2010 at 10:53:29AM -0000, Steve Langasek wrote:
>
> This patch doesn't appear to clean up any previously-added
> olcAuthzRegexp lines, or previously-added olcAccess lines referencing
> localroot, so the resulting config will be different for users upgrading
> to lucid final from an earlier release, vs. users who have upgraded to
> lucid RC. Is this ok?
>

The issue with deleting the old configuration is that it's hard (if not
impossible) to figure out if the olcAuthzRegexp and relevant olcAccess options
have been added by the package or manually by the local sysadmin.

Having the old rules doesn't break the new configuration either. So I'd rather
keep them around.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

THANK YOU for breaking my ldap install. THANK YOU!! THANK YOU!!

zoolook@venkman:~$ dpkg -l | grep slap
ii slapd 2.4.21-0ubuntu5 OpenLDAP server (slapd)

before this (broken) version:

zoolook@venkman:~$ ldapsearch uid=zoolook
SASL/GSSAPI authentication started
SASL username: <email address hidden>
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <dc=bensa,dc=ar> (default) with scope subtree
# filter: uid=zoolook
# requesting: ALL
#

# zoolook, users, bensa.ar
dn: uid=zoolook,ou=users,dc=bensa,dc=ar
objectClass: top

with this version:

zoolook@venkman:~$ ldapsearch uid=zoolook
ldap_sasl_interactive_bind_s: No such object (32)

after I fixed YOUR broken acls:

zoolook@venkman:~$ ldapsearch uid=zoolook
SASL/GSSAPI authentication started
SASL username: <email address hidden>
SASL SSF: 56
SASL data security layer installed.
# extended LDIF
#
# LDAPv3
# base <dc=bensa,dc=ar> (default) with scope subtree
# filter: uid=zoolook
# requesting: ALL
#

# zoolook, users, bensa.ar
dn: uid=zoolook,ou=users,dc=bensa,dc=ar
objectClass: top

DO NOT MESS WITH ACL EVER AGAIN BEFORE TELLING YOUR USERS EXACTLY WHAT YOU ARE GOING TO DO!

NEVER!

Yes. I'm VERY angry, and I think I have the right to be. These changes are expected on alphas, maybe betas, BUT NEVER on RCs!

On Tue, Apr 27, 2010 at 02:40:11 -0000, Mathias Gug wrote:
> The issue with deleting the old configuration is that it's hard (if not
> impossible) to figure out if the olcAuthzRegexp and relevant olcAccess options
> have been added by the package or manually by the local sysadmin.
>
> Having the old rules doesn't break the new configuration either. So I'd rather
> keep them around.

I think it would be helpful if there were some explaination somewhere
(NEWS.Debian, README.Debian, or perhaps some new file with "Ubuntu" in
the name?) spelling out exactly what the "expected" configuration is, so
that users have some idea how to properly clean things up manually if
necessary....

      Nathan

On Tue, Apr 27, 2010 at 02:49:11AM -0000, zoolook wrote:
> THANK YOU for breaking my ldap install. THANK YOU!! THANK YOU!!
>

Could you please open a new bug outlining the configuration of your
infrastructure:
 * SASL mechanism used
 * local ACLs
 * which queries were working before
 * how you fixed your configuration after the upgrade

Thanks,

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

It seems like the new slapd.postinst in 2.4.21-0ubuntu5 will cause a configuration error for upgrades from previous Lucid versions of the package.

Specifically, up through 2.4.21-0ubuntu4, the postinst script added the following line:
  olcAccess: to * by dn.exact=cn=localroot,cn=config manage by * break
to the /etc/ldap/slapd.d/cn=config/olcDatabase={-1}frontend.ldif file (when upgrading from a previous version older than 2.4.17-1ubuntu3).

The new version of the script will add the following line instead (and will do so when upgrading from any prior version of the package):
  olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break

But since there is no attempt to remove the old non-indexed line while the indexed one is added, slapd will refuse to start with the resulting frontend.ldif file.

I can't easily run an actual test of this upgrade path myself, but bug 570657 and bug 570533 seem to be consistent with this scenario.

On a separate note, I did a test upgrade from a stock hardy install directly to lucid/slapd 2.4.21-0ubuntu5. The package upgrade completed successfully, but I can confirm that

  $ ldapvi --discover -h ldap://testhost/

does not work until I manually add the
  olcAccess: {1}to dn.base="" by * read
line into the olcDatabase\=\{-1\}frontend.ldif file (as discussed in LP: #427842).

(I mention this here because it would seem to fall into the topic of "olcAccess are options broken on upgrade in {-1}frontend.ldif", but let me know if you would like me to open a new bug for this aspect of the access permissions configuration.)

We should definitely add a release notes, as this will affect every OpenLDAP server users that upgraded to pre-release lucid.

Please open new bugs.

Nathan you're analysis in comment 11 is correct.

When you say "bugs", would you like two separate new bugs, one for the slapd-won't-start-after-upgrading issue and the other about the dn.base="" permissions?

(Or do you just need a new bug related to the permissions issue?)

On Tue, Apr 27, 2010 at 05:38:25PM -0000, Nathan Stratton Treadway wrote:
> When you say "bugs", would you like two separate new bugs, one for the
> slapd-won't-start-after-upgrading issue and the other about the
> dn.base="" permissions?
>

A bug for each separate problem as it makes things simpler to track and to
focus on.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

On Tue, Apr 27, 2010 at 19:10:03 -0000, Mathias Gug wrote:
> A bug for each separate problem as it makes things simpler to
> track and to focus on.

I guess my question is whether you consider the issue raised in
comment 11 to be a separate problem from this bug (LP#563829),
thus requiring a newly-created bug for that, too.

(I will go ahead and create a new bug for the other permission
issues.)

      Nathan

Moving the release notes item to bug 571057, where it belongs.

To follow up on my comment #2: I did some more testing and determined that the behavior I was seeing related to the olcAccess lines in the olcDatabase={0}config.ldif file was due to the "localroot"-related lines left over from earlier versions of the slapd.posting script. Once I removed all those references, then everything worked as expected even when the two lines
  olcAccess: {0}to * by dn.exact=gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth manage by * break
  olcAccess: {0}to * by * none
were both found in the config.ldif file.

I will add a comment on bug 571057 related to the manual cleanup steps that should be mentioned in the release notes.

(In case any else is following this trail of crumbs, the issue I had was that the olcAuthzRegexp line that mapped the UID=0 user to "cn=localroot,cn=config" was still found in my slapd.d/cn=config.ldif file. This meant that the "dn.exact=gidNumber=0" line mentioned above was not matched. Thus, the permission check would fall to the "olcAccess: {0}to * by * none" line and access would be denied.

When the "olcAccess: {0}to * by * none" line was removed from the {0}config.ldif file, the access control search continued on through to the olcAccess lines found in the olcDatabase={-1}frontend.ldif file... and that file still contained a line granting "localroot" access, so my ldapsearch succeeded.)

I have opened Bug #571752 for the issue related to missing ACLs for the frontend database after upgrading from earlier versions of slapd (discussed in comments 3 & 12 here).

(Obviously, the discussion related to the issue mentioned in comment 11 here has moved to Bug #571057.)