For many years I have run an OpenVPN server on my home router, which provides access to my home network remotely when I am away from home and a private tunnel to my cloud server through which most monitoring and all command-and-control (via SaltStack at the time of writing) take place.

For complicated reasons, we are moving temporarily (probably - unless it turns out to be fantastically reliable) to getting our home internet over the 4G network rather than phone-line based broadband. While 4G is substantially faster than the speeds we get over the wire, I have not used it “in anger” for a sustained continuous period (weeks or months). That aside, the biggest change is that we will no longer have a public IP address at our router - instead we get a 10.x.x.x address from the carrier which is then NATed out to the real internet. This precludes running any endpoints I can remotely connect to, such as OpenVPN (or SSH), on our router. Instead, I am moving the OpenVPN service to my cloud server and will be pushing traffic back to my router, which will become an OpenVPN client instead of the server.

There are already some significant changes from my previous notes on setting up OpenVPN (they are 6 years old!), most notably:

  • I moved to embedding the certificates into the configuration files even for the server
  • using ‘tun’ device (routed) instead of ‘tap’ (bridged)
  • no longer persisting ips for clients (removed ifconfig-pool-persist ipp.txt)
  • added client-specific configuration for the cloud server to get a specific IP address
  • pushing more routes since the segregation of my home network

In the course of updating the configuration I will try to fully document here the current state of affairs. The short version of what needs to happen is:

  1. New certificates generated for the server and 1 client
  2. that the server needs iroute stanza in the client-specific configuration for the client we will route the traffic to
  3. (if not already setup) additional pushed route stanzas, matching the iroutes for the client acting as gateway, so other clients know to access these networks via the VPN
  4. local network configuration to route that traffic to the VPN client (if we want to be able to access these networks from the local box - I believe OpenVPN may route internally between clients but have not verified this)

Certificates

Regarding the certificates, generating these is necessary because the server and client that will be swapping places (1 machine that is currently a client is becoming the server and the current server is becoming a client). While is may initially seem possible to continue to use the same certificates, this is not the case as the allowed purposes of the server (serverAuth) and client (clientAuth) are different, so validation will fail if used the other way around. I could copy the existing server certificate to the new server and the existing client to the new client, however the common names are the hostnames of those machines and swapping them seems like a recipe for future confusion - so I concluded it is best to revoke the existing certificates and issue new ones.

I also considered moving to using OpenVPN’s EasyRSA scripts, instead of manually generating using OpenSSL. Since migrating most things to Let’s Encrypt, including most internal services, my own certificate authority is almost only used for OpenVPN. However, using EasyRSA requires either installing OpenVPN or extracting that part of the OpenVPN distribution on the (non-internet-connected) machine housing my CA or having the CA secrets on an internet-connected machine which presents an increased security risk. In relation to SSL certificates, I am erring on the side of paranoia and so continuing to manually manage the certificates from my CA.

So, I followed my own instructions and generated a new CSR on each host and signed them with my OpenVPN intermediary CA (remembering to select the _server variant in the OpenSSL configuration to sign the server certificate) as well as revoking the existing certificates and updating the CRL.

Server configuration

In addition to the certificates, the server configuration (/etc/openvpn/server.conf) looks as follows:

Firstly using UDP, routed (‘tun’ device) and server mode:

proto udp
dev tun
mode server
tls-server

Next the client configuration directory - this allows us to provide client-specific configuration files in a directory with this name, alongside the server configuration file:

client-config-dir ccd

IP configuration for the VPN - this is largely self-explanatory, the subnet topology functions like a traditional IP subnet. The routes to go via a client require that client’s IP address or OpenVPN will not be able to resolve the gateway (and not add to the system route table) - I am not sure if this is a bug and it should resolve them this from the client iroute:

topology subnet
ifconfig 10.10.10.1 255.255.255.0
route 10.10.10.0 255.255.255.0
# Extra routes that go via OpenVPN from the server
route 192.168.10.0 255.255.255.0 10.10.10.11 # Switch management subnet
route 192.168.20.0 255.255.255.0 10.10.10.11 # "Live" subnet
route 192.168.30.0 255.255.255.0 10.10.10.11 # IoT subnet
route 192.168.31.0 255.255.255.0 10.10.10.11 # CCTV subnet
# Dynamic IP address pool
ifconfig-pool 10.10.10.100 10.10.10.150
# Uncomment to assign the same dynamic IP to each client - see ccd for host-specific static IPs
#ifconfig-pool-persist ipp.txt

Allow clients to see all other clients:

client-to-client

N.B. OpenVPN provides no mechanism for creating rules between clients, so there is no way to firewall traffic between clients this way. It should possible to achieve this by removing the client-to-client stanza, which causes traffic to go trough the tun interface instead, and use netfilter with ip forwarding enabled to provide rules - I may want to explore this in the future but for now I am going to allow VPN clients to co-exist as though attached to a virtual dumb switch.

Configuration to push to the clients - the key features are to match the server topology, the routes for traffic that should go via the VPN and (if we want internal name resolution to work) DNS server:

push "topology subnet"
# Will want to use the VPN server if using kernel ip filtering/forwarding
#push "route-gateway 10.10.10.1"
# For efficiency, send directly to the home router with client-to-client enabled
push "route-gateway 10.10.10.11"
push "route 192.168.10.0 255.255.255.0" # Switch management subnet
push "route 192.168.20.0 255.255.255.0" # "Live" subnet
push "route 192.168.30.0 255.255.255.0" # IoT subnet
push "route 192.168.31.0 255.255.255.0" # CCTV subnet
push "dhcp-option DNS 192.168.20.250" # Use our router for DNS

Connection keepalive setting (keepalive interval timeout - on the server, the timeout will automatically be 2 x timeout to ensure clients detect timeouts before the server drops the connection):

keepalive 10 30

Who to run the daemon as:

user nobody
group openvpn

Allow keys and the device to persist across daemon restarts (e.g. SIGUSR1 or timeouts). This is required if dropping root privileges (which we do in the lines above), otherwise the daemon will not have permission to read protected files:

persist-key
persist-tun

Set the output verbosity and limit the number of consecutive messages of the same category:

verb 3
mute 20

Client-specific configuration

With the client-config-dir ccd option in the server configuration, we can create client specific configurations by adding a file named after the common-name (in the certificate) of the client in a directory called ‘ccd’ alongside the server configuration file. For example, if the server configuration is /etc/openvpn/server.conf, the ccd directory should be /etc/openvpn/ccd and the configuration file for a client with common name ‘client1.my.domain.tld’ would be /etc/openvpn/ccd/client1.my.domain.tld.

Within that client specific configuration file, we can give it a fixed address like this (n.b. like with DHCP, you want to use addresses outside the server’s pool range to avoid accidental clashes):

ifconfig-push 10.10.10.11 255.255.255.0

And we can tell the server to route extra networks down to the client with iroute (internal route) stanzas like this:

iroute 192.168.10.0 255.255.255.0 # Switch management subnet
iroute 192.168.20.0 255.255.255.0 # "Live" subnet
iroute 192.168.30.0 255.255.255.0 # IoT subnet
iroute 192.168.31.0 255.255.255.0 # CCTV subnet

From the OpenVPN manual:

Remember that you must also add the route to the system routing table as well (such as by using the –route directive). The reason why two routes are needed is that the –route directive routes the packet from the kernel to OpenVPN. Once in OpenVPN, the –iroute directive routes to the specific client. The –iroute directive also has an important interaction with –push “route …”. –iroute essentially defines a subnet which is owned by a particular client (we will call this client A). If you would like other clients to be able to reach A’s subnet, you can use –push “route …” together with –client-to-client to effect this. In order for all clients to see A’s subnet, OpenVPN must push this route to all clients EXCEPT for A, since the subnet is already owned by A. OpenVPN accomplishes this by not not pushing a route to a client if it matches one of the client’s iroutes.

Client configuration

The client configuration (again, ignoring the certificates) broadly mirrors settings in the server configuration, so I will not discuss those. nobind tells OpenVPN not to listen on a local address, resolv-retry infinite tells it to retry indefinitely if the remote hostname cannot be resolved.

client
dev tun
proto udp
nobind
remote vpn-server.my-domain.tld 1194
resolv-retry infinite
persist-key
persist-tun
verb 3