Following on from tuesday’s post on setting up my first microserver, I am starting to set-up KVM and automate building virtual-machines in order to start migrating services off my router.

This was all done through SaltStack however I have described how I did it as though it were done manually.

Network bonding

The first step is to set-up the network. As the virtual machines will require direct access to the network, as they will be running key services for my infrastructure, they should be connected to a bridge. The microservers also have 2 Ethernet connections to the network, which I want to combine using IEEE 802.3ad (& LCAP) aggregation, just as I did for my NAS 3 weeks ago.

Install pre-requisite package

Debian needs the ifenslave package installed to enable bonding and we need [bridge-utils] for bridging.

apt-get install ifenslave bridge-utils

Configure the bond

The bond interface can now be configured in /etc/interfaces.d. I considered switching to a more modern network configuration tool, such as systemd-networkd or NetworkManager but SaltStack’s network state module still only supports this way, and if it is good enough for them I guess it is good enough for me.

auto bond0

iface bond0 inet dhcp
  bond-miimon 100
  bond-mode 802.3ad
  bond-slaves eno1 eno2
  bond-xmit-hash-policy layer3+4

Do not forget to remove any old configuration for the slaves - it is worth setting them to manual mode:

iface eno1 inet manual
iface eno2 inet manual

Check this is working before continuing.

Configure the bridge

Now that the bond is set-up, I can create a bridge which will allow the VMs to present to the rest of the network as though they are directly connected (the host effectively becomes a software switch, connected to the upstream physical switch).

This is just a case of adding another network configuration for our bridge:

auto br0
iface br0 inet dhcp
  bridge_fd 0
  bridge_ports bond0
  bridge_stp False
  bridge_waitport 0

And, as before, do not forget to remove the ip address (in my case just the dhcp mode) from the slave interface (bond0) and set it to manual. Unlike the (physical) slave interfaces, it will not come up for br0 if it is not explicitly set to auto so keep that.

Installing KVM

Required packages

Strictly speaking only the qemu-kvm is required but libvirt gives me virsh and other helpful management tools. virtinst provides virt-install and virt-clone, however it recommends (as opposed to the weaker ‘suggests’ link) graphical tools (virt-viewer) so they will also be installed by default on my headless, minimal, host. Which I do not want. To avoid this, I have to install with apt-get install --no-install-recommends (or the equivalent install_recommends salt option to pkg.installed).

apt-get install qemu-kvm libvirt-clients libvirt-daemon-system
apt-get install --no-install-recommends virtinst

For my user to be allowed to interact directly with virsh, it is worth adding it to the libvirt group.

Creating a VM

By default virt-install will create VMs with graphical consoles. For headless servers, which my VMs will all be (and I am more used to managing), they can instead be set-up with serial consoles like this (from the Debian Wiki KVM page):

virt-install --virt-type kvm --name buster-amd64 \
  --location http://deb.debian.org/debian/dists/buster/main/installer-amd64/ \
  --os-variant debian10 \
  --disk size=10 --memory 1000 \
  --graphics none \
  --console pty,target_type=serial \
  --extra-args "console=ttyS0"

To connect to the console, use (--connect qemu:///system is only required if not running as root):

virsh --connect qemu:///system console buster-amd64

Fully automated install (FAI)

Automating the install was painful, not because of kvm or virt-install but getting Debian’s preseed to completely automate the partitioning and grub installation. In the end, I managed to get it working with this command:

virt-install --name new_vm \
  --vcpus 2 \
  --memory 2000 \
  --virt-type kvm \
  --location http://deb.debian.org/debian/dists/buster/main/installer-amd64/ \
  --os-variant debian10 \
  --console pty,target_type=serial \
  --extra-args "auto-install/enable=true netcfg/get_hostname=new-vm netcfg/get_domain=$( hostname -d ) preseed/url=http://debian-preseed.$( hostname -d )/d-i/buster/./preseed-vm.cfg console=ttyS0" \
  --disk size=10 \
  --network bridge=br0 \
  --autostart

And this preseed file:


# Localisation
d-i debian-installer/locale string en_GB.UTF-8
d-i keyboard-configuration/xkb-keymap select gb

# Time
d-i time/zone select Europe/London
# HW clock set to UTC?
d-i clock-setup/utc boolean true
# Use NTP during install
d-i clock-setup/ntp boolean true

# Network
# Networking will need to be set-up to download the preseed file.  Although
# there are hacks (see: https://help.ubuntu.com/lts/installation-guide/s390x/apbs04.html#preseed-network)
# to work around this, it is easier just to set them on the kernel command
# line.

# Mirror
d-i mirror/country string GB
d-i mirror/http/mirror select ftp.uk.debian.org
d-i mirror/http/proxy string

# Users
# Skip creation of a normal user account (Salt will do this later for us)
d-i passwd/make-user boolean false
# Standard root password
d-i passwd/root-password-crypted password [crypted password]

# Partitioning
# VMs: auto partition whole disk, no LVM
d-i partman-auto/method string regular
# Do not complain there is no swap
d-i partman-basicfilesystems/no_swap boolean false
# Expert partitioning, 1000MB minimum, -1 maximum (all available space)
d-i partman-auto/expert_recipe string singleroot :: 1000 50 -1 ext4 \
   $primary{ } $bootable{ } method{ format } \
   format{ } use_filesystem{ } filesystem{ ext4 } \
   mountpoint{ / } \
   .
d-i partman-auto/choose_recipe select singleroot
# This makes partman automatically partition without confirmation, provided
# that you told it what to do using one of the methods above.
d-i partman-partitioning/confirm_write_new_label boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true

# Base system install
d-i base-installer/kernel/image select linux-image-amd64

# Package selection
# No 'task' selections
tasksel tasksel/first multiselect
# Auto-install salt-minion, so the system comes up ready to be managed
d-i pkgsel/include string salt-minion
# Partake in the popularity contest
popularity-contest popularity-contest/participate boolean false

# Bootloader
# This is fairly safe to set, it makes grub install automatically to the MBR
# if no other operating system is detected on the machine.
d-i grub-installer/only_debian boolean true
# Due notably to potential USB sticks, the location of the MBR can not be
# determined safely in general, so this needs to be specified:
d-i grub-installer/bootdev string /dev/vda

# Avoid that last message about the install being complete.
d-i finish-install/reboot_in_progress note

Post-install

With the (finally working!) automated install, I just need to accept the new host’s Salt minion’s key on the master and salt can manage it.

Further work

With hindsight (and nearly 2 days of beating my head against the Debian installer and pre-seeding the partitioning and grub install), it would almost certainly have been easier to create and mount a blank image, use debootstrap to do the install and virt-install’s --import option to “skip the OS installation process and build a guest around an existing disk image”. I tried and failed to get it to automatically partition, or not, based on auto-install/classes kernel option and scripts with either preseed/run, preseed/early_command or partman/early_command as documented in the Debian installation manual but the first two did not attempt to fetch the scripts from the webserver (even using the provided preseed_fetch command directly with early_command failed) and the latter ran the script but did not seem to feed the values for partman, set with debconf-set per the manual, to the installer.

For future sanity, this is probably worth exploring properly.