OpenLDAP default search base

Although it's possible to specify a search base on the client when doing an ldapsearch, it's often nicer if the server can have it set correctly already. I noticed there's an olcDefaultSearchBase attribute for olcDatabase entries, however you can only use it on entry -1, the frontend database. This makes sense, because for one LDAP server instance you can only have a single default search base.

The following LDIF will set the default search base to dc=denc,dc=nl:

dn: olcDatabase={-1}frontend,cn=config
changetype: modify
add: olcDefaultSearchBase
olcDefaultSearchBase: dc=denc,dc=nl

Works like a charm for me!

Recovering from glue objects in OpenLDAP

After some syncing issues and a few transfers of /var/lib/ldap between servers, our company LDAP database had lost it's root organization entry. Doing a slapcat resulted in the entry listed with objectClass glue and all of it's attributes gone. However, this was the same at all of our servers.

The first thing that came to mind to fix this issue was doing an ldapmodify on the entry, however ldapmodify would return ldap_modify: No such object (32). The logical next step would then be to add the object, since ldapmodify complains it's not there... However, that would result in ldap_add: Already exists (68)! Amazing, one program telling me the object can't be modified because it's not there, the other telling me I can't add it because it exists.

I did some searching, but couldn't find a proper solution or anyone with a similar issue. I could of course start from scratch, but that would destroy the sync status, modified timestamp, modifier's name, create timestamp and creators name and perhaps even more, so that wouldn't really be an option in my humble opinion.

During my (re)search I did come across slapadd. slapadd can be used to do offline database edits (at least additions to the database). So I stopped slapd, and fired up slapadd and entered my LDIF... Same issue! The entry exists, so it can't be added. slapadd doesn't seem to support modify either (I'm not complaining, just stating the facts), so I had to figure out something else...

Suddenly I had it all figured out. slapadd and slapcat are similar tools in that they operate directly on the database instead of talking to slapd. Thus if you slapcat your database you can give the output back to slapadd!

# slapcat -n 1 > entries.ldif
# slapadd -n 1 -l entries.ldif

Of course this very simple code example will result in similar errors, because all your entries are already there. Besides, it would also be nice to edit the broken entry while we're at it, which will result in the following list of commands to complete it all (code assumes broken tree is database number 1, replace with your database index if it's not the first database):

1. # cp -ar /var/lib/ldap{,.bak}
2. # slapcat -n 1 > entries.ldif
3. # rm -r /var/lib/ldap
4. # mkdir -p /var/lib/ldap/bdb
   This line assumes a BDB database, you can probably replace bdb with hdb if you're using HDB
5. Now edit entries.ldif so your entry makes sense again. Just fix the objectClass (be sure to create a correct objectClass chain, i.e. top, dcObject, organization), structuralObjectClass and attributes required by the newly set objectClasses (i.e. dc, o).
6. # slapadd -n 1 -l entries.ldif

Now your entry should be back again, with a proper objectClass and related attributes. If you get errors along the way, make sure there aren't more entries with attributes that aren't available in the schema files. Just remove the incorrect attributes (and probably incorrect objectClasses accompanying the attributes) from the LDIF and repeat the database delete and add steps (or remove everything earlier in the LDIF and just add the new entries using slapadd, of course!)

The last step would be to index the database. I don't know if it's required (slapd will run fine without), but before starting slapd run the following:

# slapindex -n 1

Now your LDAP tree should be back to a proper state again!

There's just one issue left... If you didn't change contextCSN attributes, slapd won't sync the entry to other servers because they will all think the entry never changed (and thus the other servers will keep the broken entry). There's an easy solution: just use ldapmodify to change an attribute and the contextCSN will update and the change will propagate to the other servers. The real fix would be to change the contextCSN for the rid of the server you're editing to the current time, however this is more prone to mistakes and the result should be the same (unless using delta syncrepl, where it is possible that only the change will get propagated.)

This was my not-so-short introduction to LDAP disaster recovery without losing contextual information. I'm hoping you enjoyed reading this post and that it helped you to recover from long-standing errors.

Kerberos SSH logins on Mac OS X

As a testing step of our Kerberos / Mac OS X integration I was testing SSH using a Kerberos ticket. At first it didn't seem to work. However, SSH can easily provide some more detailed debugging information, which I could compare with debugging information from a Linux machine which would successfully login with a Kerberos ticket. Turned out GSSAPI authentication is disabled by default for SSH on Mac OS X, you can enable it by editing /etc/ssh_config:

Host *
GSSAPIAuthentication yes

or by passing the option to SSH on every connection:

$ ssh -o GSSAPIAuthentication=yes <host>

Mac OS X and OpenLDAP

At work we had some issues trying to join Mac OS X machines into our Samba Windows domain. Turned out Mac OS X was doing a search with scope base and empty base, which is meant to return some information that can be used for compatibility or some global knowledge about the LDAP tree. This object is the RootDSE object. In our case that search would return nothing, instead of the descriptive entry.

After quite a while we noticed closed bug #427842 on Launchpad. The bug describes some missing access control rules that can lead to this problem. Although this bug is closed, it can still show up when migrating data from an older release, which was also the case for us. The bug also has the required ldif, which I'll copy here for future reference:

dn: olcDatabase={-1}frontend,cn=config
changetype: modify
add: olcAccess
olcAccess: to dn.base="" by * read
olcAccess: to dn.base="cn=subschema" by * read

You can feed this to ldapmodify or ldapadd (yes, ldapadd can also do modifies). A quick ldapsearch will reveal if it worked:

$ ldapsearch -x -b '' -s base

This should return an object of the OpenLDAPRootDSE objectClass (and empty distinguished name).

Now we're at it, let's add another useful gem for Mac OS X: altServer attributes. Mac OS X searches for altServer attributes in order to find other servers that should provide the same data, in case the server is down (although I don't know when this data is cached).

It's possible to add attributes to the OpenLDAPRootDSE object by creating an LDIF file and pointing the olcRootDSE attribute on the config object to the created LDIF file. Create the following file, place it at /etc/ldap/rootdse.ldif:

dn:
altServer: ldap://server2.domain.tld/dc=domain,dc=tld
altServer: ldap://server3.domain.tld/dc=domain,dc=tld

Now add the following LDIF to OpenLDAP:

dn: cn=config
changetype: modify
add: olcRootDSE
olcRootDSE: /etc/ldap/rootdse.ldif

You can add this one using ldapmodify again.

Another quick ldapsearch will verify the attributes are really there:

$ ldapsearch -x -b '' -s base "+"

This should present quite a list detailing some support, including the just added altServer attributes.

Now there's one last thing that we should add to offer our Mac OS X users (or better, ourselves as sys admins!) a more pleasant experience: an Avahi (bonjour/zeroconf) entry for our OpenLDAP server. This will make the server show up as an option in some dialogs, for instance when adding an LDAPv3 directory server for authentication or contacts. To do this, add the following service file to avahi, for instance as /etc/avahi/services/slapd.service:

<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">

<service-group>
    <name replace-wildcards="yes">%h</name>
    <service>
        <type>_ldap._tcp</type>
        <port>389</port>
        <host-name>atlas.denc.nl</host-name>
    </service>
</service-group>

The only additional step to integrating OpenLDAP even more with Mac OS X would be by adding the Apple schemas and providing OpenDirectory support using OpenLDAP on Linux. I'll probably come to that later, but one thing I'll definitely post about is authentication against our existing OpenLDAP user tree.