How To Be Popular

Uptime engineers such as ourselves have long referred to something known as the ‘Slashdot Effect’. The name comes from popular news aggregation website, Slashdot, and the violent upturn in visitors that occurs if you are (un?)fortunate enough to find yourself on the front page or mentioned in their daily emails. Indeed, this is precisely why Stephen Fry used to warn charities and organisations before he mentioned them on Twitter. Being too popular can be the death of any site — even ones with reasonably meaty infrastructure can be floored by the volume of requests, if engineering mistakes or oversights mean they can’t cope with the demand.

So what do you do if you’re a small charity, probably operating your website on a budget, and you’re going to get a special mention on the 10 o’clock news? Sure, on one level you’re celebrating, but you’re also probably panicking. One chance where millions of people might come to your website to sign-up or donate… And you haven’t a clue if it’ll cope.

First, if it’s a dynamic content management system (CMS) like WordPress or Drupal, it won’t cope. Simply, your website will die. I’ve seen it dozens of times and helped people recover. The only thing you can do is make sure people don’t hit your website.

Fortunately, there are a number of ways to allow people to access your content without hitting your CMS, and it doesn’t have to be expensive. In fact, none of the techniques I’m proposing will take more than an hour or two of someone’s time and a sub-£100 budget. Take caution, if you’re not someone who likes to dabble in websites; you may need some help.

1. Static HTML

If you’ve made your website in a system like WordPress, Drupal, Umbraco (choose a dynamic CMS, they all have the same problem) then whenever a page is loaded the website has to ask a database what content it should display. In more complex set-ups there are layers in front of the actual website which ‘cache’ much of this data (keep a copy more readily available that can be served more rapidly). But there’s always an overhead whenever you have to go and build a page before you can send it to the reader.

When we talk about ‘static’ content, we mean a simple page — a single document that can just be served by the web server (the computer hosting your website). This is the easiest thing to serve for a web server to handle, it’s a simple transaction. Someone asks for the page, the server hands it back, job done. If all they are doing is serving static HTML, even relatively poorly resourced web servers can handle literally hundreds of requests a second.

You’re probably wondering how you get a static HTML website if you have a CMS. Well, if you look in your favourite search engine for page scrapers, software which downloads all the parts of a web page to static files on your computer for you, you’ll find lots of options, from standalone applications to browser extensions. All you need do is install one of these and ‘scrape’ your site. Then, assuming you have access to your web hosting, you can move your CMS files out of the way and replace them with your static files. Just temporarily, until the storm passes. Then you can put your CMS back for ‘business as usual’.

Couple of caveats, anything dynamic, such as “Hey Joe” personalisation, search facilities, and so on won’t work while the site is static, and clearly editing things will be harder, especially if you don’t know your way around HTML. You’ll probably still want a friend with know-how.

Note, if you don’t want to take your whole CMS down, you could also simply use your browser to ‘Save As’ your home page and make a static copy of that, but this is a little more complex, as you will have to tweak the web server’s configuration a little to serve a static front page but still have available your dynamic website in the background.

2. Relocation, Relocation, Relocation

In most cases, the ‘Static HTML’ step is all you have to do. If you’ve got a static front page or site copy, your web server will probably cope. But what if your web server is really cheap, and you’re not sure it’ll survive? Maybe your terms of service won’t allow above a certain level of traffic? Maybe you gave your hosting company the heads up and they freaked out? Well, don’t worry — as it happens, it’s pretty cheap and easy to move out for the weekend and come back on Monday.

Assuming you control your domain name (specifically, you can alter your DNS records) — and you certainly should — then you can move your website whenever and wherever you like, though do remember DNS changes may take up to 48 hours to complete, so it’s not something you want to be doing at the last minute. All you need to do is login to your Internet registrar (where you bought the domain name) and find their DNS control panel. Once there, you should be able to alter the records that tell other Internet users where they can find your website, and signpost them elsewhere.

Obviously before you start you’ll want to make a note of what’s there currently. And you’ll also want your static site already in place somewhere else. But once you’re ready, change the necessary records, wait patiently, see you’ve moved — and reverse the process when you want to go back to normal.

But where is ‘somewhere else’? There are a few options:

• GitHub Pages — this is free! There are limits, but they’re decent and there’s a built in content distribution network (CDN) that will be serving your pages, so for a few days you ought to be able to benefit from this service without them noticing or caring. It’s always polite to drop them a support ticket and let them know, of course, they’ll appreciate the warning. Their help section is full of guidance on getting set up.
• Amazon Web Services S3 — an AWS S3 ‘bucket’, as they call it, is not free, but it’s very cheap and it’s metered usage — you’ll only pay for what you actually use. You can open an AWS account, provide a credit card, create an S3 bucket, put your site in it and change the DNS to point to your bucket’s address, which AWS will provide to you. Again, there are limits, but 5,500 requests per second is good enough for most people!

There are plenty of other free/cheap temporary storage options, these two sprang to mind first. The point being once you have your static site from step 1, you can place it pretty much anywhere, if you’re worried about your current arrangements.

3. Content distribution networks

CDNs are handy; they deliver your content over a network of ‘edges’ all around the World. An ‘edge’ is a kind of jumping off point for your content, CDN providers do clever things to make sure that the content is served to you by the nearest servers. A typical CDN will likely have an ‘edge’ or two in Europe, several in Asia, several in North America, and so on. They are specifically designed to distribute traffic load and ensure connections are geographically as close to the source request as possible.

That all sounds expensive, right? You’d be surprised. Actually, if you don’t want to do steps 1 and 2, a CDN might save you the hassle. For example, Sucuri do a Web Application Firewall (WAF) that costs $20 a month and includes a CDN with no stated limits I can find. Pay your $20, change your DNS to point your website to their service (they’ll help you) and you’re done. You don’t need to scrape your website if you set their caching service and CDN to ‘aggressive’ mode.

A little more complex to set up, but equally good, is AWS CloudFront, though you’ll certainly need some more technical assistance — I won’t say more on this other than it’s also very cost-effective and someone else has written an entire blog post about how to set it all up for a static site. Hurrah!

4. Other considerations

Any services you depend upon need to be able to cope with the traffic you’re going to receive as well. It’s no good relying on a service that presents your nice bubbly text, for example, that’s limited to 100 requests an hour and then stops working. Your site will be up, but will look awful. Also no good taking people’s email addresses for a newsletter via a service that cannot handle 200 new newsletter requests a second when shit gets real (that’s a technical term). So check your partners, think about everything your site connects with, assure yourself any services you’re reliant on can also handle the traffic.

Also, do make sure you let everyone know in advance, the further in advance, the better. Most major services will shrug and say “whatever”, but you never know. Better let people know than find yourself getting angry emails from the support department of your payment gateway because you caused them major performance hassles with your media-driven donation spike!

That’s it, hopefully that helps those of you with the happy problem of success and popularity get through it without too much pain.

If you need some uptime engineering help, don’t hesitate to reach out to us at sales@codeenigma.com — we’re always willing to see if we can help you.