For a very long time I have been running my own Linux-based routers as gateways to the internet. The configuration was setup a very long time ago and it has not been revisited, other than to update logins etc., since. I discovered today that there is now a kernel-mode PPPoE module (since 2.4, from what I can gather) that I have missed the arrival of and a more performant driver for it. As I was monitoring the route due to line-speed problems, I noted that I could see the process pppoe appear in top hovering around the 12% CPU mark and wondered if that was expected or not. Googling this lead to these changes.

The Debian wiki has terse instructions for setting up pppoe, if you want to hand over control to pppoeconf, and I found this blog very helpful too - of particular interest was the note about MSS clamping that I did not know before:

Ethernet packages have a fixed size and they will define the amount of payload the can carry via MTU (maximum transmission unit). By default, Ethernet packages have a MTU size of 1500 bytes, which is fine. However, PPPoE packages have to define an additional header of 6 bytes, and a PPP ID of 2 bytes. This means in PPPoE packages, we “lose” 8 bytes and so the MTU size is reduced to 1492. This isn’t a problem in the first place, but it will become one when a client (e.g. browser / surfing the internet) connects to a server (e.g. webserver). They negotiate a MTU size of 1500 bytes, and the server starts sending data with 1500 bytes chunks, while the PPPoE package can only hold 1492 bytes. There’s a technique called MSS clamping which will fix this issue. Cisco has a really nice white paper which describes exactly that.

I have set the IP tables rule as indicated, however this box routes between 4 different networks so I suspect I should be jailing it to packets traversing the ppp0 interface - more research required there.

The changes required to move from pppoe to Roaring Penguin’s PPPoE plugin are documented below.

  1. Change from ifconfig to ip in /etc/network/interfaces.d/dsl-provider:

    auto dsl-provider
   
iface dsl-provider inet ppp
  post-down /sbin/ip link set dev enp4s0 down
  pre-up /sbin/ip link set dev enp4s0 up
  provider dsl-provider

  2. Remove pppoe package and install ppp.

  3. Remove the password and pty '/usr/bin/pppoe... from /etc/ppp/peers/dsl-provider, adding plugin rp-pppoe.so:

    plugin rp-pppoe.so enp4s0
user "dsllogin@your-provider.tld"
noauth
hide-password
   
# Keep trying to connect indefinitely
persist
maxfail 0
holdoff 5
   
# IP settings
noipdefault
defaultroute
#debug
#dryrun
   
# LCP settings
lcp-echo-interval 20
lcp-echo-failure 3
   
# PPPoE compliant settings
noaccomp
default-asyncmap
mtu 1492
  4. Put the username and password into /etc/ppp/chap-secrets.

  5. Add the rule MSS clamping rule to the IP tables firewall script:

    iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu

We are still getting <1Mbps down and 2-10x that speed up, however the load on the router has dropped to zero so at least that is definitely ruled out as a contributor now.