Following the awful time I had setting up a working preseed configuration for Debian installer I am exploring the alternative method of pre-building the disk image using debootstrap.

I am starting with the process described by Diogo Gomes on his blog.

Like the preseed method, this took a lot more effort than anticipated to get it working. This time it was getting grub to install a UEFI loader into the disk image when the host has a BIOS and persuading systemd to play nicely while building in the chroot.

Pre-requisites

In order to build a working disk image suitable for use with QEMU/KVM, I need debootstrap (assuming kvm is already installed) and dosfstools for the UEFI system partition:

apt-get install debootstrap dosfstools

We will also need qemu-img from qemu-utils but that packages should already be installed.

Preparing to install

Create the image

This is straight-forward, for example to create a 10G qcow2 format image:

qemu-img create -f qcow2 debian-buster.qcow2 10G

This can be done as any user - we will move it to the right place and set permissions latter - however from now on almost everything needs to be run as root.

Mount the image

First I need to load the nbd (Network Block Device) kernel module, which will create the /dev/nbd[0-9] devices:

modprobe nbd

Then the disk can be exposed as a network block device using qemu-tool’s server:

qemu-nbd -c /dev/nbd0 debian-buster.qcow2

Partition

The virtual disk can then be partitioned using your favourite tool:

parted -s -a optimal -- /dev/nbd0 \
  mklabel gpt \
  mkpart primary fat32 1MiB 270MiB \
  mkpart primary linux-swap 300MiB 1GiB \
  mkpart primary ext4 1GiB -0 \
  name 1 uefi \
  name 2 swap \
  name 3 root \
  set 1 esp on

This:

  • Non-interactively (-s)
  • Aligning for best performance (-a optimal)
  • Makes a new GPT partition table (mklabel gpt)
  • Makes a 269 EFI system partition (mkpart primary fat32 1MiB 270MiB)
  • Makes a 754MiB swap partition (mkpart primary linux-swap 270MiB 1GiB)
  • Makes a partition, as large as possible in the remaining space (mkpart primary ext4 1GiB -0)
  • Sets the name of the EFI system partition (1) to “efi” (name 1 efi)
  • Sets the name of the new swap partition (2) to “swap” (name 1 swap)
  • Sets the name of the new partition (3) to “root” (name 2 root)
  • Sets the EFI system partition flag on the UEFI partition (1) (set 1 esp on)

Format

Using the usual tools:

mkfs -t fat -F 32 -n EFI /dev/nbdp1
mkswap -L swap /dev/nbd0p2
mkfs -t ext4 -L root /dev/nbd0p3

Mount

Now we have a filesystem, we can mount it and continue with the Debian installation instructions for installing a new Debian system with debootstrap.

Using the UUIDs to mount will help grub configure itself correctly (or it will add root=/dev/.. to the kernel options, and Linux will not be able to find its root filesystem as the device names will be different inside the VM. Finding the UUIDs can be easily scripted (remember I used generic but specific labels for the filesystems?), since I know which NBD mount I used:

swap_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="swap" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )"
root_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="root" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )"
efi_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="EFI" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )"

mount $root_uuid /mnt

Installing the OS

I am now in familiar territory for building Debian chroots.

Bootstrapping the basic system

Begin with bootstrapping the install. Note I am installing my configuration management tool of choice (SaltStack) immediately:

debootstrap --arch amd64 --include=salt-minion buster /mnt http://ftp.uk.debian.org/debian

Mount proc and dev

Deviating slightly from the Debian manual, to do it the way I used to in my Gentoo days:

mount -o bind,ro /dev /mnt/dev
mount -t proc none /mnt/proc
mount -t sysfs none /mnt/sys # Grub will need this to install

The bind mount is read-only to address this note in the Debian manual:

There are different ways to go about this and which method you should use depends on the host system you are using for the installation, on whether you intend to use a modular kernel or not, and on whether you intend to use dynamic (e.g. using udev) or static device files for the new system.

A few of the available options are: … bind mount /dev from your host system on top of /dev in the target system; note that the postinst scripts of some packages may try to create device files, so this option should only be used with care

chroot into the new system

LANG=C.UTF-8 chroot /mnt /bin/bash

Once in the chroot, set the terminal to something sensible:

export TERM=xterm-color

Configuring fstab

Despite what the manual says, /proc does not need to be specified in fstab (and is not on the host system) these days.

See this answer from StackExchange:

The claimed statement is generally true:

It is not necessary to list /proc and /sys in the fstab unless some special options are needed. The boot system will always mount them. (Debian Wiki)

So we only need to list the disks we have created. I have used the UUID method, as is current good practice (header taken from host system’s fstab, substituting in the UUIDs looked up from earlier):

# /etc/fstab: static file system information.
#
# Use 'blkid' to print the universally unique identifier for a
# device; this may be used with UUID= as a more robust way to name devices
# that works even if disks are added and removed. See fstab(5).
#
# <file system> <mount point>   <type>  <options>       <dump>  <pass>
$swap_uuid none swap sw  0       0
$root_uuid / ext4 errors=remount-ro 0 1
$efi_uuid /boot/efi vfat defaults 0 1

We now need to mount the extra filesystems:

# Make sure the efi mount point exists
[[ -d /boot/efi ]] || mkdir /boot/efi
mount -a

Configuring the timezone

This can be done interactively (dpkg-reconfigure tzdata) or by preseeding the answers to debconf then non-interactively letting dpkg reconfigure it. The latter is more scriptable (which it should be obvious is where this blog post is heading by now):

debconf-set-selections <<EOF
tzdata tzdata/Areas select Europe
tzdata tzdata/Zones/Europe select London
EOF
# This is necessary as tzdata will assume these are manually set and override the debconf values with their settings
rm -f /etc/localtime /etc/timezone
DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -f noninteractive tzdata

Networking

When the VM is configured and boots we will want networking to come up automatically. With Debian’s splitting of /etc/interfaces preparing the system for that is easy by creating /etc/network/interfaces.d/enp1s0

allow-hotplug enp1s0
iface enp1s0 inet dhcp

The debootstrapped system also has no loopback (lo) interface configured, so I added it to /etc/network/interfaces directly:

auto lo
iface lo inet loopback

The debootstrap install manual said this about loopback, but I am yet to come across a real system without it configured so I have put it in anyway:

The loopback interface isn’t really required any longer, but can be used if needed

Since the system will be configured by DHCP, there is not point editing /etc/resolv.conf as it will be rebuilt when the VM reconfigures its network.

Put the correct hostname into /etc/hostname:

echo "debian-buster" > /etc/hostname

And configure /etc/hosts:

127.0.0.1       localhost
127.0.1.1       debian-buster.my.domain.tld    debian-buster

# The following lines are desirable for IPv6 capable hosts
::1     localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Apt sources

debootstrap will have created a very minimal /etc/apt/sources.list but we also want to get source and security packages:

deb http://ftp.uk.debian.org/debian/ buster main
deb-src http://ftp.uk.debian.org/debian/ buster main

deb http://security.debian.org/debian-security buster/updates main
deb-src http://security.debian.org/debian-security buster/updates main

# buster-updates, previously known as 'volatile'
deb http://ftp.uk.debian.org/debian/ buster-updates main
deb-src http://ftp.uk.debian.org/debian/ buster-updates main

Don’t forget to refresh the cache:

apt-get update

Locales and keyboard layout

Preseed the answers before installing the packages:

debconf-set-selections <<EOF
locales locales/locales_to_be_generated multiselect en_GB.UTF-8 UTF-8
locales locales/default_environment_locale select en_GB.UTF-8
keyboard-configuration keyboard-configuration/layoutcode string gb
keyboard-configuration keyboard-configuration/variant select English (UK)
keyboard-configuration keyboard-configuration/model select Generic 105-key PC (intl.)
EOF
# Stop anything overriding debconf's settings
rm -f /etc/default/locale /etc/locale.gen /etc/default/keyboard
apt-get install locales console-setup

Kernel

As this will be a bootable VM image, we need a kernel installed:

apt-get install linux-image-amd64

Bootloader

I just used the usual grub install commands:

apt-get install grub-efi-amd64
# Add console=ttyS0 so we get early boot messages on the serial console.
sed -i -e 's/^\\(GRUB_CMDLINE_LINUX="[^"]*\\)"$/\\1 console=ttyS0"/' /etc/default/grub
# Tell GRUB to use the serial console
cat - >>/etc/default/grub <<EOF
GRUB_TERMINAL="serial"
GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600 --stop=1"
EOF
grub-install --target=x86_64-efi
update-grub

In the chroot, grub is using the disk path (e.g. /dev/nbd0s3) rather than the UUID - presumably because this is how it was mounted.

I also copied the fallback bootloader to the default bootloader location, as this is the only OS and when we first boot our VM we will need this to initialise the boot options in the nvram:

mkdir /boot/efi/EFI/BOOT
cp /boot/efi/EFI/debian/fbx64.efi /boot/efi/EFI/BOOT/bootx64.efi

Tidying up

Enable the serial console:

systemctl enable serial-getty@ttyS0.service

Set a password for root (this can be automated by passing a ‘username:encrypted_password’ to chpassed -e on stdin):

passwd

Delete the cached base packages:

apt-get clean

Exit the chroot:

exit

Unmount the partitions:

umount /mnt/dev /mnt/proc /mnt/sys /mnt/boot/efi /mnt

Disconnect the image from the NBD:

qemu-nbd -d /dev/nbd0

Moving the image

When building a VM around an existing image virt-install does not move it, so ideally we need to put it in the same place a new images would be created and with the same permissions:

mv debian-buster.qcow2 /var/lib/libvirt/
chown libvirt-qemu:libvirt-qemu /var/lib/libvirt/debian-buster.qcow2
chmod 600 /var/lib/libvirt/debian-buster.qcow2

Scripting the whole thing

The build-debian-image script, which automates all of these steps, is available in my GitHub repository. At the time of writing, it supported these options:

Usage: build-debian-image [-hSDM] [-s suite] [-f file] [-z size] [-r passwd] [name]

-h:        Display this message and exit
-S:        Skip debootstrap initialising a blank image and go stright to
           mounting.  Requires an already setup image file as target.
-D:        Do debootstrap only and exit without chrooting and doing the
           stage 2 process.  As the debootstrap is the most time-consuming
           element, this can be helpful combined with -S during debugging and
           development.  The stage 2 script will still be written out.
-M:        Leave the NBD attached and chroot filesystems mounted at the end.
-s suite:  the Debian suite to build, defaults to buster (taken from
           first line matching /^deb / in /etc/apt/sources.list).
-f file:   filename for the image, defaults to <name>.qcow2
-z size:   Size of the image, in a format understood by the qemu-img command.
           Defaults to 5G, first 1GiB is entirely consumed by boot and swap
           partitions.
-w size:   Size of the image's swap partition in MiB (set to 0 to disable),
           defaults to 753
-r var:    Use passwd in var environment variable as the encrypted root
           password in the built image (avoids exposing the password via the
           command-line, which any user on the system can see).
-d domain: Use domain as the new hosts domain, defaults to my.domain.tld (
           taken from this host's 'hostname -d')
name:      hostname to configure the image to, defaults to debian-<suite>

Building a virtual machine around the image

The key to this is to specify --import to virt-install. Note that I am now using EFI for the VM’s boot, not the default BIOS emulation.

virt-install --virt-type kvm --name debian-buster \
  --vcpus 2 \
  --memory 2000 \
  --os-variant debian10 \
  --disk /var/lib/libvirt/debian-buster.qcow2 \
  --import \
  --network bridge=br0 \
  --boot uefi \
  --graphics none \
  --noautoconsole \
  --console pty,target_type=serial

Performance comparison (Mythbusters’ style)

As I was going through this, and discovering that both methods (vanilla install with preseed vs build disk image directly via debootstrap) have their pain points I did wonder which method was quickest to deploy a new VM. Clearly neither will be as fast as taking a “golden image”, changing the host-specific features (pretty much just hostname in a disk-image-only, although it might be advantageous to regenerate the virtual disk, partition and filesystem UUID to be unique) and booting a new VM around it.

In true Mythbusters’ style I timed each method once, so one sample datapoint.

debootstrap / direct disk image creation

$ time ( sudo --preserve-env=my_root_pass build-debian-image -z 10G -r my_root_pass && virt-install --connect qemu:///system --virt-type kvm --name debian-buster \
  --vcpus 2 \
  --memory 2000 \
  --os-variant debian10 \
  --disk ./debian-buster.qcow2 \
  --import \
  --network bridge=br0 \
  --boot uefi \
  --graphics none \
  --noautoconsole \
  --console pty,target_type=serial )
...
Domain creation completed.

real    13m35.923s
user    1m59.798s
sys     0m26.988s

Full install / preseed

Note that the preseed file has been updated for an efi install - mainly the partitioning and I also added console=ttyS0 to the kernel options (d-i debian-installer/add-kernel-opts string console=ttyS0).

$ time virt-install --connect qemu:///system --virt-type kvm --name debian-buster \
  --vcpus 2 \
  --memory 2000 \
  --location http://deb.debian.org/debian/dists/buster/main/installer-amd64/ \
  --os-variant debian10 \
  --extra-args "auto-install/enable=true netcfg/get_hostname=debian-buster netcfg/get_domain=$( hostname -d ) preseed/url=http://debian-preseed.$( hostname -d )/d-i/buster/./preseed-vm.cfg console=ttyS0" \
  --disk size=10 \
  --network bridge=br0 \
  --boot uefi \
  --graphics none \
  --console pty,target_type=serial
...
real    10m22.966s
user    0m3.136s
sys     0m1.782s

Conclusion

While this was a fun excursion that reminded me of installing Gentoo, in the end it was as painful as figuring out the preseed configuration file (albeit in different ways) and scripting it is reinventing a wheel that is already provided by virt-install and preseed. The preseeded install is also almost 3 minutes faster (approx. 22% faster).

The only remaining deficiency in that method is I have still not managed to get the scripting hooks work in a way that allows me to dynamically configure the partitioning based on kernel options (e.g. auto-install/classes). With Red Hat’s kickstart this is easy with a pre script to generate a partition scheme into a temporary file, and then including that file back in the kickstart. This StackExchange answer looks like it might help with the Debian version of this.

It is important to note that in order to remove a UEFI VM, I have to give the --nvram option (presumably to confirm I know I am deleting the VM’s persistent nvram storage):

virsh --connect qemu:///system undefine --nvram debian-buster