After restoring the router, the next step I needed to complete was to restore my configuration management server in order to start orchestrating the rest. On the live network this resides in a virtual machine on my HP Microserver. This means I need to restore the host and the VM to get this up and running.
Since this system is being restored in a working network, now the router has been restored, I began by configuring the network port onto main network:
lab-switch(config)#interface gigabitEthernet 1/0/17 lab-switch(config-if)#description starbase2 lab-switch(config-if)#switchport pvid 20 lab-switch(config-if)#switchport general allowed vlan 20 untagged lab-switch(config-if)#no switchport general allowed vlan 1 lab-switch(config-if)#exit
Preparing the system
As with the router, I updated and configured the BIOS on the system before I started.
Partitioning the disk
Next I configured the disk using the same process as for the router - in this case, the live system has both (software) RAID 1 and LUKS encryption on its disk which I am currently not mirroring into the lab environment. In the case of encryption, I set this up so I would not have to worry as much about disposing of disks when they fail - a concern that equally applies to the lab kit however the ILO make remotely entering the passphrase to unlock the encryption key very convenient and the same is not true of the very cheap lab kit. I will need to ponder this decision some more, as I do want to turn off the lab when not in use but not have to go down each machine entering a passphrase locally every time I power them on. The RAID is to provide redundancy as this system hosts VMs for both the backup and configuration systems, both of which are amongst the most critical in my home network - the lab environment as a whole is non-critical so this concern does not apply.
The live system also has a BIOS with no EFI support - I reconfigured this particular lab machine to boot ‘legacy only’ (instead of EFI only) and partition accordingly. I made var_lib slightly smaller (190G vs 200G) than the live system due to the lab machine having a smaller disk and it not fitting otherwise:
parted /dev/sda mklabel msdos parted /dev/sda mkpart primary ext4 1001 5000 parted /dev/sda toggle 1 boot parted /dev/sda mkpart primary 5000 100% parted /dev/sda toggle 2 lvm pvcreate /dev/sda2 vgcreate starbase2 /dev/sda2 lvcreate -L 190G -n var_lib starbase2 lvcreate -L 30G -n swap starbase2 lvcreate -L 20G -n var starbase2 lvcreate -L 20G -n home starbase2 lvcreate -L 10G -n srv starbase2 lvcreate -L 10G -n usr starbase2 lvcreate -L 5G -n root starbase2 lvcreate -L 5G -n tmp starbase2
Then formatting the partitions:
# Create filesystems mkfs.ext4 -L starbase2-boot /dev/sda1 mkfs.ext4 -L starbase2-home /dev/mapper/starbase2-home mkfs.ext4 -L starbase2-srv /dev/mapper/starbase2-srv mkfs.ext4 -L starbase2-var /dev/mapper/starbase2-var mkfs.ext4 -L starbase2-var_lib /dev/mapper/starbase2-var_lib mkfs.ext4 -L starbase2-usr /dev/mapper/starbase2-usr mkfs.ext4 -L starbase2-root /dev/mapper/starbase2-root mkfs.ext4 -L starbase2-tmp /dev/mapper/starbase2-tmp mkswap -L starbase2-swap /dev/mapper/starbase2-swap
Mount the formatted devices:
# Mount new devices swapon /dev/mapper/starbase2-swap mount -t ext4 /dev/mapper/starbase2-root /mnt for volume in /dev/mapper/starbase2-* do short_vol=$(basename $volume | sed 's/^[^-]\+-//') if [[ $short_vol != 'swap' ]] && [[ $short_vol != 'root' ]] && [[ $short_vol != 'var_lib' ]] then mkdir /mnt/$short_vol mount -t ext4 $volume /mnt/$short_vol fi done mkdir /mnt/boot mount /dev/sda1 /mnt/boot mkdir /mnt/var/lib mount /dev/mapper/starbase2-var_lib /mnt/var/lib
Restoring the backup
In order to transfer the restored backup file, as I have a working network since restoring the router, I used netcat after (again) downloaded the backup as a tar archive on the system with the DR backup mounted. In the longer term I think I need to investigate creating a custom Debian install image based on the minimal (net install) image that adds (at least) an ssh client, ssh server, rsync and telnet client but remains small enough to fit on my usual Debian install USB drive (which is 1GB in capacity) which will enable the possibility of direct restores and managing the switch.
To start the receiving end:
nc -l -p 9000 | tar -C /mnt
To send (where
192.168.20.154 is the IP address of the receiving host):
cat /path/to/restore_starbase2.home.entek.org.uk_*.tar | busybox nc 192.168.20.154 9000
Post restore configuration
The UUIDs for disks in
need updating - lsblk -o name,uuid,partuuid
or ls -l /dev/disk/by-uuid
can help find the new ones. I also blanked /mnt/etc/crypttab` as not (currently) using encrypted filesystem on this device.
The files in
/mnt/etc/network/interfaces.d were edited and renamed to match the new system’s interface names (in my case
eno2 were deleted,
enp2s0 created with configuration
iface enp2s0 inet manual and
br0 updated with
bridge_ports enp2s0 instead of
bridge_ports bond0 as the new system isn’t dual-homed (and hence not using bonded interfaces).
I then created the missing mount points:
mkdir /mnt/dev /mnt/proc /mnt/sys /mnt/run
Next, to try and pre-empt the boot problem I had with the router, I chrooted into the environment and ran
mount -t proc none /mnt/proc mount -o bind /dev /mnt/dev mount -o bind /sys /mnt/sys mount -o bind /run /mnt/run chroot /mnt /bin/bash grub-install /dev/sda exit
Finally, I unmounted everything in the new system image and shutdown the machine:
umount /mnt/var/lib umount /mnt/* umount /mnt swapoff /dev/mapper/starbase2-swap poweroff
After removing the USB boot drive, I powered it back on and tried to boot into the restored system - this time it failed because the encrypted volume that the initial ram disk was trying to unlock could not be found (so the
grub-install had worked fine). I booted off the install media again, into rescue mode and ran
update-initramfs from within a chroot to the restored system. After this it booted fine.