Corporate Internet Connectivity fixes — Nigerian Episode 1
I have had a repeated experience with Internet Service Providers in Nigeria and I’ve been able to work around it. Implementing same might be useful to you. If the advice doesn’t make sense to you, you can pay me some money and I will assist with actual implementation.
Background:
Much of the Internet access in the country is provided by mobile network operators using a mishmash of cable and microwave links. The links to customers are mostly over MPLS and end users rarely have visibility into what is going on with the underlying infrastructure.
With many of the network operators, corporate connectivity is billed by maximum connection speed. If I’m paying for 20Megabits per second of Internet, it means I would be allowed to move packets up to a speed of 20Mbps; Once my utilization attempts to exceed 20Mbps, the network operators throttling mechanism will kick in.
The problem:
With network operators that are under-subscribed (for whatever reasons), the tendency is to allow generous burst periods or queue buckets whose end result is a smoother Internet experience (even when you are over-using your link).
Network operators that have larger subscriber bases tend not to be so generous however. On such large networks, you would find that your Internet experience is almost perfect as long as you do not reach the ceiling of the allocation that has been set for you.
Because such operators mostly have very aggressive/punitive control measures, once you hit the defined limit (20Mbps in my previous example), everything falls apart and the network speed is reduced to a crawl for most intents and purposes.
When you complain to your ISP, they will very likely show you a graph that shows you are “exceeding” your bandwidth. The solution as far as they are concerned is for you to fork out more money to them for more Internet bandwidth.
The part the graphs do not tell you however is that much of the “bandwidth utilization” you see after you hit the maximum is retransmission due to dropped packets.
Solution:
By now, many of you would have guessed the obvious solution for those who do not have the millions to fork out for upgrades :-)
A fix that has worked for me is to (attempt to) bring the bottleneck to my own infrastructure and avoid the harsh penalties of the provider’s rate-limiters. Depending on the hardware in place, I simply implement a QoS limit (Simple Queue or interface speed definition) that slows down my WAN link to 1Mbps less than the subscribed capacity.
In the 20Mbps example, I would set the WAN link of my edge router to a maximum speed of 19Mbps. Some would argue that this can only control upload traffic since downloads are originating outside your sphere of control but I have found that putting such limits in place do help to improve quality of service for my clients. I believe this is possibly because a control of upload requests reduces the amount of incoming traffic when the link is already congested and dropping packets.
In any case, a summary of what happens is that requests exceeding the set limit are queued up inside your router and they go out in an orderly fashion as opposed to crossing out of your router only to be dropped by the harsh network operator and then re-sent by the source application.
