To learn about edge delivery networks, start by building your own

Building a simple edge network isn’t as hard as it sounds — especially with the infrastructure options available today — and it’s a great way to learn about the challenges of global application delivery, traffic management, infrastructure automation, and more.

A couple weeks back I wrote about the increasing trend I’m seeing among many of our most sophisticated customers toward building out custom edge delivery networks. Often, these networks are being deployed to service application specific needs that can’t be addressed by typical content delivery networks or through other services.

Since that article, I’ve talked with a whole pile of NS1’s customers about that trend — there’s more activity around custom edge delivery networks than I’d realized! I’ve heard from operators solving ridiculous SSL termination challenges across millions of domains, from developers seeking to ingest and process IoT data from bajillions of devices close to the endpoints themselves, from service providers iterating on their own network tech to enable more effective deployment of application specific logic to their edges, and more. There’s a genuine trend here. Cool.

One of the other groups I’ve heard from is the relatively uninitiated — folks wondering why on earth you’d ever need or want to build your own edge network, and more importantly, wondering how you’d even get started down that path.

It’s not that hard to build an edge delivery network! (Well, it’s not that hard to get started building one. You’ll never be finished.)

A great exercise to start getting familiar with the technologies at play is to build your own simple CDN. Building a first-pass caching CDN for HTTP delivery is simpler than you might expect — and you can get as fancy (and expensive) as suits your interest.

Here are a few ideas to get started:

• Deploy some edge nodes. Cloud service providers like DigitalOcean, AWS, Vultr, Packet, and others let you point and click to provision servers around the world. Spin up a few boxes in markets you’d like to deliver to. Getting more serious? You’ll need to look toward colocation providers like Equinix or DRT, take down connectivity from transit vendors like NTT, GTT, Telia, etc, and develop a deep understanding of BGP, network gear, server architectures, DDoS mitigation, and the like. Fun times.
• Drop a web server on your edge nodes. Nginx is a good bet. You can do this by hand (apt-get install nginx, or whatever makes sense for your OS), or you might consider exploring configuration management tools like Ansible or Salt for interacting with distributed environments. At scale, managing infrastructure by hand is a recipe for disaster. If you want to get even fancier, think about the kinds of traffic you’ll serve — small requests? large objects? slow client connections? fast ones? — and tune your system for your use case. Again, a great opportunity to bring configuration management tools into the mix to automate system tuning across your network. If you were building something application specific, here’s where you’d be figuring out how to deploy your software and its supporting stack, manage its configuration and keep it in sync, and so on.
• Set up caching. You can configure your web servers as caching reverse proxies pointing at an origin server, e.g., the current host for your website. If you were building a real CDN, you’d start thinking here about some seriously tough topics, like disk layouts for your cache, ways to stripe your cache across infrastructure in your facility, and how to enable purging of content across all the caches in your network. Alternatively, if you were deploying an application specific edge network, you’d probably be thinking about how to manage and replicate the dataset for your application across your global infrastructure.
• Get traffic to your edge network. The most straightforward approach here is to use a Managed DNS platform like NS1 with deep traffic management capabilities. Set up a trial account, create a domain and a DNS A record (like cdn.example.com), and add answers for each of your nodes to the record, then set up geotargeting, health checking, and the like. Here’s an article on how to set up traffic management with NS1: Building a Simple CDN With NS1. If you’re up for some real craziness, you can try building your own edge DNS delivery network. (I’ve done that a few times — it’s kind of my thing!) And if you’re thinking about servicing at-scale production applications with your network, then make sure you think about DNS redundancy, and look into high availability options like anycasting for your IP space. One of the most common modern edge architectures we see in practice involves regional anycasting (sharing IP space among facilities in a regional vicinity) and DNS based selection of regional anycast groups using geography, ASN or network prefix, load telemetry, or other information. You may also need to look into local traffic distribution mechanisms like layer 4–7 load balancing, equal cost multipath (ECMP — my personal favorite), or similar.
• Don’t forget about operational visibility. You can’t operate a distributed network service without monitoring its availability and performance. You can monitor your individual edge nodes for service failures, but I’d recommend focusing on the overall availability of your service with a tool like Catchpoint, ThousandEyes, or — great for getting started — Grafana’s worldPing. These tools leverage highly distributed vantage points to help you understand global performance and availability of your network. If you’re building something more serious, you’ll want to augment outside-in visibility with deep real-time telemetry from your systems, anomaly detection on that telemetry, tooling for rapidly shifting around your traffic by controlling BGP or pushing changes to NS1’s API, and automation around scaling up and down in your edge facilities.
• Play around, break things, and then… use a real CDN. Building a simple caching CDN is a great way to learn all kinds of key concepts in application delivery, network and application architecture, service operations, traffic management, and other areas. But if you’re just delivering cat pictures or Javascript blobs, don’t put your own CDN in production: go use one of the many awesome CDNs out there in the market and benefit from their hard-won operational expertise, their incredible network depth and breadth, and their advanced features and functionality. Focus on your own application and do what you do best. That said: if you’ve got an application — like those of many of our customers at NS1 — that can benefit from a customized edge delivery network, and conventional CDNs don’t meet your needs, then keep going!

There are some other great resources out there on how to get started building simple CDNs or edge delivery networks. Three of my favorites are: Chris Ueland’s article from a few years back on how to Build a 3 Continent CDN for $25 in 1 Hour; Samir Jafferali’s more recent article on how to Build Your Own Anycast Network in 9 Steps; and NS1’s own Jon Sullivan in a lightning talk showing how to tackle the traffic management challenge and Build a CDN in 5 Minutes With Intelligent DNS.