DHCP for PXE booting only

At work we use a Novell Netware 5 server (don't worry, it will be replaced) for, amongst others, DHCP. Not much of an issue, but I wanted to netboot clients so I could do easy operating system installations. When I wanted to add the appropriate options to the DHCP server, I noticed it wasn't possible. Some searching on the internet revealed some hacks, but nothing you'd easily try on a server in use.

So then what? I noticed before when I was trying to boot PXE clients and they were attached to the main network (instead of my private network) that they wouldn't get (accept?) DHCP leases, so there was my solution: add a DHCP server that gives leases to PXE clients only.

Although this may sound hard, it's actually pretty easy. PXE clients send along a so-called vendor class identifier containing the string PXEClient. Using the ISC DHCP server we can easily check for this string, and then hand out a lease to those clients only. One last thing to keep in mind: don't hand out leases in the same range as the authorative DHCP server.

Finally, here's a sample config (/etc/dhcp3/dhcpd.conf):

ddns-update-style none;
option domain-name "mycroes.nl";
option domain-name-servers 192.168.5.1;

default-lease-time 600;
max-lease-time 7200;
log-facility local7;

class "pxeclients" {
 match if substring(option vendor-class-identifier, 0, 9) = "PXEClient";
 filename "pxelinux.0";
}

shared-network 5 {
 subnet 192.168.5.0 netmask 255.255.255.0 {
 }
 pool {
  allow members of "pxeclients";
  range dynamic-bootp 192.168.5.201 192.168.5.240;
 }
}

Updating Ubuntu without removing grub-efi

Ubuntu still is trying to remove grub-efi everytime a new kernel arrives. I have a Mac Mini without a display, so grub-pc is useless for me, so how do I prevent this grub-efi removal all the time?

Simple solution, just tell apt you also want to install grub-efi, regardless of the availability of a new version:

$ sudo apt-get install linux-generic-pae grub-efi
Reading package lists... Done
Building dependency tree       
Reading state information... Done
grub-efi is already the newest version.
The following extra packages will be installed:
  linux-image-2.6.35-22-generic-pae linux-image-generic-pae
Suggested packages:
  fdutils linux-doc-2.6.35 linux-source-2.6.35 linux-tools
Recommended packages:
  grub-pc grub lilo
The following NEW packages will be installed:
  linux-image-2.6.35-22-generic-pae
The following packages will be upgraded:
  linux-generic-pae linux-image-generic-pae
2 upgraded, 1 newly installed, 0 to remove and 3 not upgraded.
Need to get 34.1MB of archives.
After this operation, 107MB of additional disk space will be used.
Do you want to continue [Y/n]?

And voila, grub-pc is a suggested package and no longer forced upon me! Thanks go out to Frank Groeneveld for suggesting the solution!

Installing Samba 4 on Ubuntu Maverick (10.10)

Samba 4 is currently able to serve as a active directory domain controller for both Windows XP and Windows 7 (as tested by me) and probably for other Windows versions too. With Ubuntu 10.10 there finally is a recent enough version to make use of all the current Samba 4 functionality, however some issues still remain. This post will provide a short guide to setting up Samba 4 on your Ubuntu Maverick system, but it won't go into more advanced Samba topics. At first I wanted this to be a full step-by-step guide, however I can't find the time to complete it as such (I started writing when Maverick was in beta). I welcome comments adding more details and I hope everyone will be able to follow this howto.

Let's start by updating the system.

$ sudo apt-get update

Next add a PPA which includes a more recent Bind 9 version. I believe this is mainly needed so your Windows clients can send DNS updates to the domain controller, but I can't say I thoroughly tested with the Ubuntu Maverick distributed version.

Personally I used bind9 from Hauke Lampe's PPA (BIND 9 Updates : Hauke Lampe).

Install samba4 and bind9:

$ sudo apt-get install samba4 samba4-clients bind9

Move existing smb.conf:

$ sudo mv /etc/samba/smb.conf{,.old}

Create a samba 4 config and provision the database:

$ sudo LD_PRELOAD=/usr/lib/libdcerpc.so.0.0.1 /usr/share/samba/setup/provision --realm=samdom.example.com --domain=SAMDOM --adminpass=SOMEPASSWORD --server-role='domain controller'

You might be wondering what this LD_PRELOAD is about, well it's needed because some stuff is missing the link to the dcerpc library.

Now we want to start samba, there's another issue ahead. The samba4 init script doesn't check for the existence of the samba directory in /var/run, so let's add that ourselves.

# /etc/init.d/samba4
...
  log_daemon_msg "Starting Samba 4 daemon" "samba"

  if [ ! -d $(dirname $SAMBAPID) ]; then
   mkdir -p $(dirname $SAMBAPID)
  fi

  if !...

We're still not there yet... Remember the missing library link? It will also return while running Samba, so let's work around it by creating local versions of the samba programs that will load the library:

Create /usr/local/sbin/samba:

#!/bin/sh
LD_PRELOAD=/usr/lib/libdcerpc.so.0.0.1 /usr/sbin/$(basename $0)

Now symlink samba_dnsupdate and samba_spnupdate to the same file:

$ sudo ln -s /usr/local/sbin/samba{,_dnsupdate}
$ sudo ln -s /usr/local/sbin/samba{,_spnupdate}

Now start samba:

$ sudo /etc/init.d/samba4 start

Let's do a quick test if it's working:

$ smbclient -UAdministrator -Llocalhost
Password for [SAMDOM\Administrator]:

 Sharename       Type       Comment
 ---------       ----       -------
 netlogon        Disk       
 sysvol          Disk       
 IPC$            IPC        IPC Service (Samba 4.0.0alpha12-GIT-UNKNOWN)
 ADMIN$          Disk       DISK Service (Samba 4.0.0alpha12-GIT-UNKNOWN)
REWRITE: list servers not implemented

Seems to be working!

Now let's get DNS working too. Start by editing named.conf.local:

// /etc/bind/named.conf.local
...
//include "/etc/bind/zones.rfc1918";

include "/var/lib/samba/private/named.conf";

Thought we were done? Think again! AppArmor is protecting our samba4 files from bind, I'd rather have bind read them though...

# /etc/apparmor.d/usr.sbin.named
...
/var/lib/samba/private/* rw,
/var/lib/samba/private/dns/* rw,
}

Reload AppArmor profiles and restart bind:

$ sudo /etc/init.d/apparmor reload
$ sudo /etc/init.d/bind9 restart

Bind should now start without any issues. Next is to actually use bind for DNS:

# /etc/resolv.conf
nameserver 127.0.0.1

You can verify it's working by querying dns for kerberos:

$ host -t SRV _kerberos._udp.samdom.example.com

This should return an SRV record, if not, something's broken!

Now let's move the Kerberos config into place:

$ sudo cp /var/lib/samba/private/krb5.conf /etc/

You can verify it's working by installing krb5-user and doing a kinit Administrator, but since Kerberos comes out of the box with samba, I'm assuming it's working (it always did for me).

If you chose to add a PPA with a recent Bind version, you can enable Kerberized DNS updates by pointing named to the correct principal and keytab. More details on this can be found on the Samba 4 howto, I will add my own details here later.



You should now be able to administer your Samba 4 domain controller using the microsoft utilities for windows server management, the Samba net tool or direct LDAP queries.

Updates

dec 8 2010, 22:56

Added missing apparmor policy changes

Rsync and remote sudo

Running rsync with superuser privileges can be hard at times, but here's an easy solution works on Ubuntu 10.04 (some other solutions failed to work):

$  echo "password" | ssh sudo -S -v
$ sudo rsync -a -e ssh --rsync-path="sudo rsync"

The first line will touch the timestamp for sudo, the second line will really sync. Keep in mind that this doesn't take care of credentials for ssh, so you will need to take care of this using keys, agents or some external authentication mechanism like Kerberos.

rsync with --delete-excluded

While setting up daily (offsite) automated backups I ran into a few issues. First of all backups didn't complete before people were getting to work again, so I had to manually stop them and start them at a lower transferrate. This is easily done by passing rsync the --bwlimit=<kbps> option.

Next I often want to sync just part of the tree, so I would add --exclude=/<folder> to the options to exclude all folders I don't want. However, I also exclude some files and I use --delete, which has the nasty side-effect of not deleting the excluded files on the receiving end (if they were deleted on the sender), thus leaving non-empty folders on the receiver and generating errors because the non-empty folders aren't deleted. There's an option that 'fixes' this, and that's --delete-excluded. This option will delete excluded files on the receiving end. You can guess that combined with my --exclude=/<folder> this would result in deleting an entire branch of the tree that should not be removed... The solution is to specify that the exclude is a receiving side exclude, because excludes are server side exclude by default when --delete-excluded is also provided. This can be done by using a filter rule instead of an exclude rule, resulting in the following option: --filter=-r_/<folder>. The - is to specify it's an exclude, the r specifies it's for the receiving side and the _ seperates the modifiers from the path (space is also allowed, but using an underscore prevents the need for quoting or even double-quoting). Now there's one nasty issue remaining: the excluded folder will still be parsed on the sender, so let's make it an exclude for both sender and receiver: --filter=-rs_/<folder>.

Using the above it's now possible to exclude files from an rsync transfer, without removing them on the receiving side, but with deletion of exclude files on the receiving end. In short: rsync --exclude='*.tmp' --filter='-rs_/important/' --delete --delete-excluded <source> <dest> will leave the important folder alone on the destination, but will remove all .tmp files in the destination.

OpenLDAP default search base

Although it's possible to specify a search base on the client when doing an ldapsearch, it's often nicer if the server can have it set correctly already. I noticed there's an olcDefaultSearchBase attribute for olcDatabase entries, however you can only use it on entry -1, the frontend database. This makes sense, because for one LDAP server instance you can only have a single default search base.

The following LDIF will set the default search base to dc=denc,dc=nl:

dn: olcDatabase={-1}frontend,cn=config
changetype: modify
add: olcDefaultSearchBase
olcDefaultSearchBase: dc=denc,dc=nl

Works like a charm for me!

Recovering from glue objects in OpenLDAP

After some syncing issues and a few transfers of /var/lib/ldap between servers, our company LDAP database had lost it's root organization entry. Doing a slapcat resulted in the entry listed with objectClass glue and all of it's attributes gone. However, this was the same at all of our servers.

The first thing that came to mind to fix this issue was doing an ldapmodify on the entry, however ldapmodify would return ldap_modify: No such object (32). The logical next step would then be to add the object, since ldapmodify complains it's not there... However, that would result in ldap_add: Already exists (68)! Amazing, one program telling me the object can't be modified because it's not there, the other telling me I can't add it because it exists.

I did some searching, but couldn't find a proper solution or anyone with a similar issue. I could of course start from scratch, but that would destroy the sync status, modified timestamp, modifier's name, create timestamp and creators name and perhaps even more, so that wouldn't really be an option in my humble opinion.

During my (re)search I did come across slapadd. slapadd can be used to do offline database edits (at least additions to the database). So I stopped slapd, and fired up slapadd and entered my LDIF... Same issue! The entry exists, so it can't be added. slapadd doesn't seem to support modify either (I'm not complaining, just stating the facts), so I had to figure out something else...

Suddenly I had it all figured out. slapadd and slapcat are similar tools in that they operate directly on the database instead of talking to slapd. Thus if you slapcat your database you can give the output back to slapadd!

# slapcat -n 1 > entries.ldif
# slapadd -n 1 -l entries.ldif

Of course this very simple code example will result in similar errors, because all your entries are already there. Besides, it would also be nice to edit the broken entry while we're at it, which will result in the following list of commands to complete it all (code assumes broken tree is database number 1, replace with your database index if it's not the first database):

1. # cp -ar /var/lib/ldap{,.bak}
2. # slapcat -n 1 > entries.ldif
3. # rm -r /var/lib/ldap
4. # mkdir -p /var/lib/ldap/bdb
   This line assumes a BDB database, you can probably replace bdb with hdb if you're using HDB
5. Now edit entries.ldif so your entry makes sense again. Just fix the objectClass (be sure to create a correct objectClass chain, i.e. top, dcObject, organization), structuralObjectClass and attributes required by the newly set objectClasses (i.e. dc, o).
6. # slapadd -n 1 -l entries.ldif

Now your entry should be back again, with a proper objectClass and related attributes. If you get errors along the way, make sure there aren't more entries with attributes that aren't available in the schema files. Just remove the incorrect attributes (and probably incorrect objectClasses accompanying the attributes) from the LDIF and repeat the database delete and add steps (or remove everything earlier in the LDIF and just add the new entries using slapadd, of course!)

The last step would be to index the database. I don't know if it's required (slapd will run fine without), but before starting slapd run the following:

# slapindex -n 1

Now your LDAP tree should be back to a proper state again!

There's just one issue left... If you didn't change contextCSN attributes, slapd won't sync the entry to other servers because they will all think the entry never changed (and thus the other servers will keep the broken entry). There's an easy solution: just use ldapmodify to change an attribute and the contextCSN will update and the change will propagate to the other servers. The real fix would be to change the contextCSN for the rid of the server you're editing to the current time, however this is more prone to mistakes and the result should be the same (unless using delta syncrepl, where it is possible that only the change will get propagated.)

This was my not-so-short introduction to LDAP disaster recovery without losing contextual information. I'm hoping you enjoyed reading this post and that it helped you to recover from long-standing errors.