Automating server installations: Part 1; PXE–Boot to puppet

So, as part of my toe in the water of VMWare, I’m looking at ways of provisioning servers more efficiently.

Here’s where I’d like to get to:

I’d like to reach a stage where, when I want a new server setting up, I do the following:

• Allocate an IP address for it’s MAC in DHCP
• Register the hostname in DNS
• Describe its role in puppet
• Rack it (physical) or create the blank VM
• Boot it

  - and have the rest of the process be automated.

At the moment, I’m leaning towards a strategy of ‘netboot-install the OS, then use Puppet to build it’.

By running a PXE-installer on each VM host, I should be able to get a standard OS image (Ubuntu, in my case) onto each box with no manual intervention. Assuming I can then wedge the puppet packages anjd config on, I can then have a single puppetmaster server take over from there and customize the new server for whatever it’s intended role might be.

There’s an additional slight complication; what I want to do in production is to have two networks – a private one which is only used for pxe-booting, and the “main” network, which connects to the internet. I don’t have any control over our ‘main’ network’s DHCP server, and I don’t really want to run an non-authoritative server on the same network; hence the appeal of the private net.

After some faffing, it turns out that by far the easiest way for this to work is to have the private network do nothing except respond to the DHCP request and supply the tftp boot image. The preseed and the packages are fetched over the production network. I suspect that someone with better networking/debian install skills than I could probably make it all work over the private network, but getting diagnostics out of the installer is beyond my feeble capabilities.

So, step 1: Get a PXE-Boot environment working. For added complexity, I’m testing this out on VirtualBox rather than VMWare. However, the process seems to be pretty similar.

This how-to is the first port of call. It walks you through the process of installing tftp and dhcpd, and configuring them appropriately. By the end of page 2, you should be at the point where you can PXE-boot into the graphical installer menu. Note that, in VirtualBox, you need to enable network booting (settings->advanced). Don’t make the network be the first boot device though, otherwise it’ll re-install on every reboot.

Next step is to install a proxy for apt to use, so we don’t thrash the network too much. I went for apt-cacher-ng, largely because it’s simpler to set up and doesn’t require faffing with CGI scripts.

Now, we need to have some way of automatically answering all the questions that the installer will want to ask. This is where a ‘seed’ file comes in – basically just a list of answers to questions the installer might ask.

I started with the debian example, and modified it like this:

$ curl http://www.debian.org/releases/stable/example-preseed.txt | sdiff -s - preseed

d-i debian-installer/locale string en_US              |    d-i debian-installer/locale string en_GB
d-i console-keymaps-at/keymap select us                  |    d-i console-keymaps-at/keymap select gb
d-i netcfg/get_hostname string unassigned-hostname          |    d-i netcfg/get_hostname string ubuntu
d-i mirror/http/hostname string http.us.debian.org          |    d-i mirror/http/hostname string xxxx.warwick.ac.uk:3142
d-i mirror/http/directory string /debian              |    d-i mirror/http/directory string /ubuntu
#d-i mirror/suite string testing                  |    d-i mirror/suite string jaunty
#d-i partman-auto/disk string /dev/sda                  |    d-i partman-auto/disk string /dev/sda
#d-i passwd/user-fullname string Debian User              |    d-i passwd/user-fullname string Build User
#d-i passwd/username string debian                  |    d-i passwd/username string xxxxx
#d-i passwd/user-password password insecure              |    d-i passwd/user-password password xxxx
#d-i passwd/user-password-again password insecure          |    d-i passwd/user-password-again password xxxx
                                  >    d-i apt-setup/multiverse boolean true
                                  >    d-i apt-setup/universe boolean true
                                  |    tasksel tasksel/first multiselect standard
#d-i pkgsel/include string openssh-server build-essential     |    d-i pkgsel/include string openssh-server puppet

Now, we just need to tell the PXE-installer to use our new preseed file. Note that the layout of the tftpd config in Ubuntu Jaunty has changed quite a bit, with everything split over multiple files. I chose to override the ‘text only’ config, and make it the default. There are an almost infinite number of possible ‘append’ lines scattered around the internet; this one worked for me…

label cli
        menu label ^Command-line install
        kernel ubuntu-installer/i386/linux
        append ramdisk_size=14984 locale=en_GB console-setup/layoutcode=gb netcfg/choose_interface=eth0 url=http://xxxx.warwick.ac.uk/preseed netcfg/get_hostname=ubuntu priority=critical vga=normal initrd=ubuntu-installer/i386/initrd.gz  

a point to note here: If your VM boots, and looks OK, but when you type you just see multiple diamonds printed for each keystroke, it’s because you haven’t got console-setup/layoutcode defined properly in the “append” line. I used en_GB at first, which isn’t valid, and caused the console to default to Afghanistan instead. You can’t override this in your preseed file either; it has to be passed to the kernel

- and that’s enough to get me to a point where I have a system I can log into. Now, I need to get puppet to be running on first boot. I’ve not yet looked into pre-allocating certificates, so I’m going to get it to start with -w 0, to make it wait for the puppetmaster to sign a cert for it.

First, I made a custom version of /etc/default/puppet. Borrowing heavily from this example, but with a few adaptations so it works with dash (ubuntu’s version of /bin/sh).

# Defaults for puppet - sourced by /etc/init.d/puppet

# Start puppet on boot? 
START=yes
# Puppet doesn't like starting before it can contact the puppetmaster.
# Wait up to 30 seconds before continuing.

for n in `seq 1 30`; do
  ping -c 1 xxxx.warwick.ac.uk > /dev/null 2> /dev/null && break
  echo -n " $n" 
  sleep 1
done

# Startup options
DAEMON_OPTS="-w 0 --server xxxx.warwick.ac.uk"

Now, I just need to tell the netboot installer to put this file into place, which I do by adding this to the end of the preseed file:

d-i preseed/late_command string wget -O /target/etc/default/puppet http://xxxxx.warwick.ac.uk/puppet

And it works! I get a system which boots, starts puppet, and then waits politely for the puppetmaster to sign its cert and send it a config. From here on in, I can use standard puppetry to build up the server to whatever configuration it needs.

Chris May : 27 Apr 2009 16:12 |  Comments (2) |  Report a problem