This post documents my attempt to send traffic from specific local hosts (Sky TV boxes) via my existing VPN connection, whilst retaining direct access to my broadband connection for all other hosts as a workaround to the annoying problems Sky services have when the client is behind a carrier-level network address translation (carrier-grade NAT or CGNAT).

Background

Since switching our broadband to mobile broadband, our Sky box has been unable to download on-demand shows. It appears this is a well known issue, Three (who we are not with but our provider uses their network) even have a help page on the issue.

There are many complaints in the forums, the predominant suggestion seems to be Sky’s servers from the same box appearing to come from different public IP addresses - a common situation when you are behind CGNAT. It appears to affect a number of traditional-broadband ISPs, as well as BT’s fibre-to-the-premise services and almost all mobile broadband services. One workaround is to use a VPN for Sky traffic. The suggestion on the forum is that it works because all traffic from the Sky box then comes from the same IP, however I am not convinced that it is as simple as that.

VPNs tend to be highly tolerant to underlying network “glitches”, giving symptoms that I would refer to as “lumpy” - where the connection appears to function at full speed then drop-off or hang for a bit before recovering back to full speed. Again, this is problem that can be prevalent in mobile broadband connections - and VPN tunnels usually handle lumpy networks more gracefully for connection orientated protocols like TCP. If I assume the Sky boxes are using TCP for the download, it might be plausible that the VPN protects the connection from a momentary broadband connection issue that, going directly, closes the connection resulting in a failed download.

As a slight aside (engaging rant mode), I am very frustrated that 9 times out of 10 when the download was failing the Sky box did not keep the, sometimes large (60-70%) portion of the program it claimed to have downloaded before the connection failed. 1 in 10 times, it would say the download failed but still retain the portion it had downloaded and was able to resume from there. Why does it behaves differently in different network-failure scenarios?

Configuring the VPN server

Currently, the VPN server does not route any traffic out to the internet - it is a route into my network only (achieved via a set of pushed routes that create a split tunnel on the client) so the first thing to do is to setup completely classic NAT rules on the VPN server so that it will route if we send non-VPN-bound traffic at it:

# Specity external and vpn interfaces
EXT=ens3
VPN=tun0
# Set forward policy to drop, to only allow fowarding that we explicitly permit
iptables -P FORWARD DROP
# Established connections (even from outside to inside) can be forwarded
iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow from VPN interface
iptables -A FORWARD -i $VPN -o $EXT -j ACCEPT
iptables -t nat -A POSTROUTING -o $EXT -j MASQUERADE
# Enable IPv4 routing
echo 1 > /proc/sys/net/ipv4/ip_forward

Configuring the home router

The next part, which is the new part for me, is to tag the traffic from the Sky boxes. I could allocate them ‘sticky’ IP addresses by adding their mac addresses with a specific IP assignment to the DHCP server but since iptables will let me match on media access control(MAC) address, why not just use that directly and leave them with dynamically assigned IP addresses? I did make a disturbing (well, I found it disturbing) discovery - the MAC address (according to the DHCP server logs) is NOT the MAC address displayed in the Sky box’s “settings” interface for the physical port - the real Ethernet MAC is what is displayed with 2 added to the final octet (e.g. mine ended 30 and d8, so the real MACs were 32 and da respectively). So, on the local network gateway (I suspect only the main box is necessary but I did our mini box too, to be sure):

iptables -t mangle -A PREROUTING -m mac --mac-source D0:58:FC:66:A9:32 -j MARK --set-mark 1
iptables -t mangle -A PREROUTING -m mac --mac-source D0:58:FC:B8:E9:DA -j MARK --set-mark 1

Tip: I initially tested this by setting the mac-source to be my laptop’s MAC and using a “what is my IP” service to check where the traffic was appearing.

Then setup a parallel routing table for the marked packets, which sends the default route down the VPN:

ip route flush table 100
ip route add table 100 default via 10.10.10.1 dev tun0
ip route add table 100 10.10.10.0/24 dev tun0 src 10.10.10.11
ip route add table 100 192.168.10.0/24 dev enp3s0.10 src 192.168.10.250
ip route add table 100 192.168.20.0/24 dev enp3s0.20 src 192.168.20.250
ip route add table 100 192.168.30.0/24 dev enp3s0.30 src 192.168.30.250
ip route add table 100 192.168.31.0/24 dev enp3s0.31 src 192.168.31.250
ip route add table 100 192.168.40.0/24 dev enp3s0.40 src 192.168.40.250

Finally, add a rule to use this routing table for the marked packets:

ip rule add fwmark 0x1 table 100

Tip: When I added this to my firewall script, I protected it with [ -z "$( ip rule list fwmark 0x1 )"] && ip rule add fwmark 0x1 table 100 to test for the rule being missing, to avoid adding it multiple times.

The all important question: did it work?

Short answer: yes.

Long answer: we have still had 3 download failures, which puts the failure rate so far at somewhere between 25-34% of attempts at downloading an episode from the on-demand service however that is from a position of 100% failure (not successfully downloaded a single one of these episodes), so a vast improvement.

The mobile broadband is a temporary solution, we are waiting for Black Friday which, thanks pretty much solely to Amazon, has become a thing in the UK (this is not a good thing, in my opinion). Last year the major TV and Broadband providers did a raft of promotions for the event, and as our existing Sky contract ends about the same time we are planning to look at the deals Virgin Media have to offer. We found Virgin’s TV box’s series-link record feature worked much better (rather ironically given Sky’s “Believe in better” tag-line) however Virgin mandate using their broadband router (and hence broadband service) with their current generation of TV boxes, so we do not want to be tied into another contract as our current broadband one ended recently. It is a surprisingly small set of options if you want short-term broadband for 6 months.

I do like the Welsh translation (source: Welsh Government Twitter, but via Wikipedia so treat with a pinch of salt):

Did you know [Black Friday] in Welsh is “Dydd Gwener y Gwario Gwirion” – this literally translates to “Silly Spending Friday”