Benchmarking PPPoE connections with OpenWRT and OPNSense

Maximizing your PPPoE Internet connection

Where we are based (Kuala Lumpur, Malaysia), PPPoE connections are still used by ISPs to serve the Internet. At the time of writing (Jan 2021), all three main ISPs (Telekom Malaysia, Time Broadband, and Maxis) are using PPPoE in their routers for their Internet connections.

We realize that in other parts of the world, ISPs have mostly moved away from PPPoE to a direct LAN connection. I guess we would still have to wait until this happens…

Many customers opt to use a custom Firewall to replace their stock routers. Reasons include:

• For Maxis and Telekom Malaysia, they are supplying a pretty (in my opinion) low-performing TP-Link WiFi router. If you use Port Forwarding, you may realize that you cannot access the Public IP interface from within your home/office LAN. This is because most TP-Link routers do not support NAT hairpinning. This is a big minus
• You would need to host a VPN server for connections from outside your home/office LAN, or to create a Site-to-Site connection to another location

There is a big range in VPN Firewall prices, ranging from less than ~250 USD(MYR 1000) on the affordable side to an average of a thousand USD (MYR 4000) or above.

Hardware based on Intel Atom (N) or Celeron (J) processors are quite popular and affordable.

For the OS, there are many choices for the consumer. My favorites are OPNSense and PFSense, both based on HardenedBSD (FreebSD but more secure).

However, we found out that these HardenedBSD iterations suffer from poor performance if you have a PPPOE Internet connection more than 300 Mbps. For example, if you have a 500 Mbps or a 800 Mbps connection (the latter being the highest speed offered at the time of writing), you may find that the Firewall might not take advantage of the full speed offered by your ISP.

These are test results taken from a Firewall running Intel Atom N2600 processor (using a 1 Gbps PPPOE connection):

OPNSense gives a weak connection over PPPoE

For this reason, since we have many customers using Internet subscriptions over 300 Mbps, we had to use OpenWRT, which is a Linux-based router OS.

The test results taken from the same hardware, but this time running OpenWRT:

What a big difference. OpenWRT is able to run very close to the theoretical maximum in a 1 Gbps PPPOE link.

TL;DR

If you are using an Atom-based processor and your PPPoE Internet subscription is more than 300 Mbps, you will need to use OpenWRT (Linux) rather than OPNSense/PFSense (HardenedBSD).