Following the awful time I had setting up a working preseed configuration for Debian installer I am exploring the alternative method of pre-building the disk image using debootstrap.
I am starting with the process described by Diogo Gomes on his blog.
Like the preseed method, this took a lot more effort than anticipated to get it working. This time it was getting grub to install a UEFI loader into the disk image when the host has a BIOS and persuading systemd to play nicely while building in the chroot.
apt-get install debootstrap dosfstools
We will also need
qemu-img from qemu-utils but that packages should already be installed.
Preparing to install
Create the image
This is straight-forward, for example to create a 10G qcow2 format image:
qemu-img create -f qcow2 debian-buster.qcow2 10G
This can be done as any user - we will move it to the right place and set permissions latter - however from now on almost everything needs to be run as root.
Mount the image
First I need to load the
nbd (Network Block Device) kernel module, which will create the
Then the disk can be exposed as a network block device using qemu-tool’s server:
qemu-nbd -c /dev/nbd0 debian-buster.qcow2
The virtual disk can then be partitioned using your favourite tool:
parted -s -a optimal -- /dev/nbd0 \ mklabel gpt \ mkpart primary fat32 1MiB 270MiB \ mkpart primary linux-swap 300MiB 1GiB \ mkpart primary ext4 1GiB -0 \ name 1 uefi \ name 2 swap \ name 3 root \ set 1 esp on
- Non-interactively (
- Aligning for best performance (
- Makes a new GPT partition table (
- Makes a 269 EFI system partition (
mkpart primary fat32 1MiB 270MiB)
- Makes a 754MiB swap partition (
mkpart primary linux-swap 270MiB 1GiB)
- Makes a partition, as large as possible in the remaining space (
mkpart primary ext4 1GiB -0)
- Sets the name of the EFI system partition (1) to “efi” (
name 1 efi)
- Sets the name of the new swap partition (2) to “swap” (
name 1 swap)
- Sets the name of the new partition (3) to “root” (
name 2 root)
- Sets the EFI system partition flag on the UEFI partition (1) (
set 1 esp on)
Using the usual tools:
mkfs -t fat -F 32 -n EFI /dev/nbdp1 mkswap -L swap /dev/nbd0p2 mkfs -t ext4 -L root /dev/nbd0p3
Now we have a filesystem, we can mount it and continue with the Debian installation instructions for installing a new Debian system with debootstrap.
Using the UUIDs to mount will help grub configure itself correctly (or it will add
root=/dev/.. to the kernel options, and Linux will not be able to find its root filesystem as the device names will be different inside the VM. Finding the UUIDs can be easily scripted (remember I used generic but specific labels for the filesystems?), since I know which NBD mount I used:
swap_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="swap" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )" root_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="root" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )" efi_uuid="$(blkid | grep '^/dev/nbd0' | grep ' LABEL="EFI" ' | grep -o ' UUID="[^"]\+"' | sed -e 's/^ //' )"
mount $root_uuid /mnt
Installing the OS
I am now in familiar territory for building Debian chroots.
Bootstrapping the basic system
Begin with bootstrapping the install. Note I am installing my configuration management tool of choice (SaltStack) immediately:
debootstrap --arch amd64 --include=salt-minion buster /mnt http://ftp.uk.debian.org/debian
Mount proc and dev
Deviating slightly from the Debian manual, to do it the way I used to in my Gentoo days:
mount -o bind,ro /dev /mnt/dev mount -t proc none /mnt/proc mount -t sysfs none /mnt/sys # Grub will need this to install
The bind mount is read-only to address this note in the Debian manual:
There are different ways to go about this and which method you should use depends on the host system you are using for the installation, on whether you intend to use a modular kernel or not, and on whether you intend to use dynamic (e.g. using udev) or static device files for the new system.
A few of the available options are: … bind mount /dev from your host system on top of /dev in the target system; note that the postinst scripts of some packages may try to create device files, so this option should only be used with care
chroot into the new system
LANG=C.UTF-8 chroot /mnt /bin/bash
Once in the chroot, set the terminal to something sensible:
Despite what the manual says,
/proc does not need to be specified in fstab (and is not on the host system) these days.
The claimed statement is generally true:
It is not necessary to list /proc and /sys in the fstab unless some special options are needed. The boot system will always mount them. (Debian Wiki)
- The only reason Debian (and by extension, Ubuntu) includes it is as a forgotten remnant of the bad old days when it used to be necessary.
- This behavior is a bug in the installer package partman-target and was recently fixed in Debian Testing (wheezy) on June 15, 2012.
So we only need to list the disks we have created. I have used the UUID method, as is current good practice (header taken from host system’s fstab, substituting in the UUIDs looked up from earlier):
# /etc/fstab: static file system information. # # Use 'blkid' to print the universally unique identifier for a # device; this may be used with UUID= as a more robust way to name devices # that works even if disks are added and removed. See fstab(5). # # <file system> <mount point> <type> <options> <dump> <pass> $swap_uuid none swap sw 0 0 $root_uuid / ext4 errors=remount-ro 0 1 $efi_uuid /boot/efi vfat defaults 0 1
We now need to mount the extra filesystems:
# Make sure the efi mount point exists [[ -d /boot/efi ]] || mkdir /boot/efi mount -a
Configuring the timezone
This can be done interactively (
dpkg-reconfigure tzdata) or by preseeding the answers to debconf then non-interactively letting dpkg reconfigure it. The latter is more scriptable (which it should be obvious is where this blog post is heading by now):
debconf-set-selections <<EOF tzdata tzdata/Areas select Europe tzdata tzdata/Zones/Europe select London EOF # This is necessary as tzdata will assume these are manually set and override the debconf values with their settings rm -f /etc/localtime /etc/timezone DEBCONF_NONINTERACTIVE_SEEN=true dpkg-reconfigure -f noninteractive tzdata
When the VM is configured and boots we will want networking to come up automatically. With Debian’s splitting of
/etc/interfaces preparing the system for that is easy by creating
allow-hotplug enp1s0 iface enp1s0 inet dhcp
The debootstrapped system also has no loopback (
lo) interface configured, so I added it to
auto lo iface lo inet loopback
The debootstrap install manual said this about loopback, but I am yet to come across a real system without it configured so I have put it in anyway:
The loopback interface isn’t really required any longer, but can be used if needed
Since the system will be configured by DHCP, there is not point editing
/etc/resolv.conf as it will be rebuilt when the VM reconfigures its network.
Put the correct hostname into
echo "debian-buster" > /etc/hostname
127.0.0.1 localhost 127.0.1.1 debian-buster.my.domain.tld debian-buster # The following lines are desirable for IPv6 capable hosts ::1 localhost ip6-localhost ip6-loopback ff02::1 ip6-allnodes ff02::2 ip6-allrouters
debootstrap will have created a very minimal
/etc/apt/sources.list but we also want to get source and security packages:
deb http://ftp.uk.debian.org/debian/ buster main deb-src http://ftp.uk.debian.org/debian/ buster main deb http://security.debian.org/debian-security buster/updates main deb-src http://security.debian.org/debian-security buster/updates main # buster-updates, previously known as 'volatile' deb http://ftp.uk.debian.org/debian/ buster-updates main deb-src http://ftp.uk.debian.org/debian/ buster-updates main
Don’t forget to refresh the cache:
Locales and keyboard layout
Preseed the answers before installing the packages:
debconf-set-selections <<EOF locales locales/locales_to_be_generated multiselect en_GB.UTF-8 UTF-8 locales locales/default_environment_locale select en_GB.UTF-8 keyboard-configuration keyboard-configuration/layoutcode string gb keyboard-configuration keyboard-configuration/variant select English (UK) keyboard-configuration keyboard-configuration/model select Generic 105-key PC (intl.) EOF # Stop anything overriding debconf's settings rm -f /etc/default/locale /etc/locale.gen /etc/default/keyboard apt-get install locales console-setup
As this will be a bootable VM image, we need a kernel installed:
apt-get install linux-image-amd64
I just used the usual grub install commands:
apt-get install grub-efi-amd64 # Add console=ttyS0 so we get early boot messages on the serial console. sed -i -e 's/^\\(GRUB_CMDLINE_LINUX="[^"]*\\)"$/\\1 console=ttyS0"/' /etc/default/grub # Tell GRUB to use the serial console cat - >>/etc/default/grub <<EOF GRUB_TERMINAL="serial" GRUB_SERIAL_COMMAND="serial --unit=0 --speed=9600 --stop=1" EOF grub-install --target=x86_64-efi update-grub
In the chroot, grub is using the disk path (e.g.
/dev/nbd0s3) rather than the UUID - presumably because this is how it was mounted.
I also copied the fallback bootloader to the default bootloader location, as this is the only OS and when we first boot our VM we will need this to initialise the boot options in the nvram:
mkdir /boot/efi/EFI/BOOT cp /boot/efi/EFI/debian/fbx64.efi /boot/efi/EFI/BOOT/bootx64.efi
Enable the serial console:
systemctl enable serial-getty@ttyS0.service
Set a password for root (this can be automated by passing a ‘username:encrypted_password’ to
chpassed -e on stdin):
Delete the cached base packages:
Exit the chroot:
Unmount the partitions:
umount /mnt/dev /mnt/proc /mnt/sys /mnt/boot/efi /mnt
Disconnect the image from the NBD:
qemu-nbd -d /dev/nbd0
Moving the image
When building a VM around an existing image
virt-install does not move it, so ideally we need to put it in the same place a new images would be created and with the same permissions:
mv debian-buster.qcow2 /var/lib/libvirt/ chown libvirt-qemu:libvirt-qemu /var/lib/libvirt/debian-buster.qcow2 chmod 600 /var/lib/libvirt/debian-buster.qcow2
Scripting the whole thing
The build-debian-image script, which automates all of these steps, is available in my GitHub repository. At the time of writing, it supported these options:
Usage: build-debian-image [-hSDM] [-s suite] [-f file] [-z size] [-r passwd] [name] -h: Display this message and exit -S: Skip debootstrap initialising a blank image and go stright to mounting. Requires an already setup image file as target. -D: Do debootstrap only and exit without chrooting and doing the stage 2 process. As the debootstrap is the most time-consuming element, this can be helpful combined with -S during debugging and development. The stage 2 script will still be written out. -M: Leave the NBD attached and chroot filesystems mounted at the end. -s suite: the Debian suite to build, defaults to buster (taken from first line matching /^deb / in /etc/apt/sources.list). -f file: filename for the image, defaults to <name>.qcow2 -z size: Size of the image, in a format understood by the qemu-img command. Defaults to 5G, first 1GiB is entirely consumed by boot and swap partitions. -w size: Size of the image's swap partition in MiB (set to 0 to disable), defaults to 753 -r var: Use passwd in var environment variable as the encrypted root password in the built image (avoids exposing the password via the command-line, which any user on the system can see). -d domain: Use domain as the new hosts domain, defaults to my.domain.tld ( taken from this host's 'hostname -d') name: hostname to configure the image to, defaults to debian-<suite>
Building a virtual machine around the image
The key to this is to specify
virt-install. Note that I am now using EFI for the VM’s boot, not the default BIOS emulation.
virt-install --virt-type kvm --name debian-buster \ --vcpus 2 \ --memory 2000 \ --os-variant debian10 \ --disk /var/lib/libvirt/debian-buster.qcow2 \ --import \ --network bridge=br0 \ --boot uefi \ --graphics none \ --noautoconsole \ --console pty,target_type=serial
Performance comparison (Mythbusters’ style)
As I was going through this, and discovering that both methods (vanilla install with preseed vs build disk image directly via debootstrap) have their pain points I did wonder which method was quickest to deploy a new VM. Clearly neither will be as fast as taking a “golden image”, changing the host-specific features (pretty much just hostname in a disk-image-only, although it might be advantageous to regenerate the virtual disk, partition and filesystem UUID to be unique) and booting a new VM around it.
In true Mythbusters’ style I timed each method once, so one sample datapoint.
debootstrap / direct disk image creation
$ time ( sudo --preserve-env=my_root_pass build-debian-image -z 10G -r my_root_pass && virt-install --connect qemu:///system --virt-type kvm --name debian-buster \ --vcpus 2 \ --memory 2000 \ --os-variant debian10 \ --disk ./debian-buster.qcow2 \ --import \ --network bridge=br0 \ --boot uefi \ --graphics none \ --noautoconsole \ --console pty,target_type=serial ) ... Domain creation completed. real 13m35.923s user 1m59.798s sys 0m26.988s
Full install / preseed
Note that the preseed file has been updated for an efi install - mainly the partitioning and I also added
console=ttyS0 to the kernel options (
d-i debian-installer/add-kernel-opts string console=ttyS0).
$ time virt-install --connect qemu:///system --virt-type kvm --name debian-buster \ --vcpus 2 \ --memory 2000 \ --location http://deb.debian.org/debian/dists/buster/main/installer-amd64/ \ --os-variant debian10 \ --extra-args "auto-install/enable=true netcfg/get_hostname=debian-buster netcfg/get_domain=$( hostname -d ) preseed/url=http://debian-preseed.$( hostname -d )/d-i/buster/./preseed-vm.cfg console=ttyS0" \ --disk size=10 \ --network bridge=br0 \ --boot uefi \ --graphics none \ --console pty,target_type=serial ... real 10m22.966s user 0m3.136s sys 0m1.782s
While this was a fun excursion that reminded me of installing Gentoo, in the end it was as painful as figuring out the preseed configuration file (albeit in different ways) and scripting it is reinventing a wheel that is already provided by virt-install and preseed. The preseeded install is also almost 3 minutes faster (approx. 22% faster).
The only remaining deficiency in that method is I have still not managed to get the scripting hooks work in a way that allows me to dynamically configure the partitioning based on kernel options (e.g.
auto-install/classes). With RedHat’s kickstart this is easy with a pre script to generate a partition scheme into a temporary file, and then including that file back in the kickstart. This StackExchange answer looks like it might help with the Debian version of this.
It is important to note that in order to remove a UEFI VM, I have to give the
--nvram option (presumably to confirm I know I am deleting the VM’s persistent nvram storage):
virsh --connect qemu:///system undefine --nvram debian-buster