Horizontally Scalable WordPress in the cloud

Note: This article has been cross-posted from my WordPress blog and is written from the perspective of being viewed on there.

This post you’re reading was served to you by Amazon Web Services (AWS). To be more specific; when you clicked on the link to this page, a few things happened:

1. Your computer performed a DNS lookup on the domain danielparker.com.au which was handled by Amazon Route 53.
2. Route 53 contained an aliased ‘A’ record to the public DNS of an Elastic Load Balancer instance.
3. Depending on a variety of conditions, the load balancer responded with one or more ‘A’ records pointing to the Amazon EC2 instances that it wants web traffic to be routed to.
4. Your browser then requested this page from the IP address of the first ‘A’ record in the response it received.

Note: Source code for the cloud-ready site is here

Under the hood

Step 1 above is standard for any web request (replacing Route 53 with whatever DNS hosting was used), but after that everything starts to get a bit strange.

Google deep dreaming

When designing this infrastructure I defined some perhaps non-standard requirements for a WordPress site or any popular PHP content management system. These were:

• The site should be capable of scaling horizontally.
• The infrastructure should allow for Continuous Delivery to be configured.

Anyone with experience managing the hosting of a WordPress site knows that the first point caused me a world of pain. WordPress out of the box serves media (images) from the filesystem, gets configuration values from the wp-config.php file in the web root. Furthermore, the process of updating WordPress involves backing up all the files on the filesystem because the update itself will modify the WordPress files. Reading the official WordPress update documentation highlights how ridiculously manual this process this. Some may say that the ‘One-click’ update feature reduces the need for this but I’ve performed one-click updates that have broken my site.

Scalability and Continuous Delivery

To achieve the horizontal scaling requirement I needed to leverage Amazon Elastic Beanstalk. Elastic Beanstalk is a deployment service which manages the provisioning and deployment of a web application to multiple EC2 servers based on whatever scaling thresholds you require. All you need to provide is an application package containing your web application. Elastic Beanstalk will also provision an Elastic Load Balancer for you to point Route 53 at. Like most other AWS features, a deployment can be kicked off via the AWS API which satisfies the continuous delivery requirement as well. I used Travis CI to automatically package and deploy the site from GitHub.

Unfortunately using Elastic Beanstalk to deploy a WordPress site required me to address the system coupling that the CMS currently has. Fortunately, I’m not the first to do such a thing and I was able to borrow some existing ideas from others.

WordPress-Skeleton

A dev named Mark Jaquith maintains a repo on GitHub called WordPress-Skeleton which separates the user content part of WordPress away from the core of WordPress. It just so happens that Mark is also a lead developer of WordPress, which made me all warm and fuzzy about relying on the Skeleton repository code. WordPress-Skeleton is intended to be forked each time you use it for a site of your own so that you can make modifications based on your needs, which I did. Here is a list of the changes I made to the skeleton to get it working for my setup:

• I changed the wp-config.php to source it’s database credentials and site URL from environment variables so that I wouldn’t be compromising the database security by committing that information to my public GitHub repository.
• Instead of using a symlink for the uploads directory I just commit that directory to source (reason coming soon).
• I updated the WordPress git submodule to 4.3.

The reason I chose to commit my uploads directory is because I installed a WordPress plugin called WP Offload S3 which uploads the sites media files such as images to Amazon S3, changes the link in the database to point to the S3 file and then deletes the local media file so that the site doesn’t rely on it being available on the local web server. Without this key change, I wouldn’t have been able to achieve multi-instance WordPress due to the requirement to synchronise the files across all the sites. Furthermore, if Elastic Beanstalk terminated all the running WordPress instances for some reason, I would lose all those media files which were stored on the instances.

WP Offload S3 has a free and paid version. If you are modifying an existing site like I have, and you don’t want to manually re-upload all the media files then you will need to purchase a Bronze level subscription to the plugin which allows you to upload all existing media files to S3. I would definitely recommend this approach as the plugin works flawlessly.

RDS (the last piece of the puzzle)

Amazon Relational Database Service or RDS is where my existing MySQL database will now be hosted. This was perhaps the easiest component to configure because it boiled down to:

1. Select a database engine (MySQL) and a bunch of other parameters (eg. SSD or magnetic storage).
2. Logon using phpmyadmin and restore the database backup.
3. Add a user with access to that database.
4. Add the database credentials as Elastic Beanstalk environment variables for the app to use.

The only extra step was that I had to ensure that I had configured the correct inbound and outbound rules for both the EC2 and RDS security groups on the Virtual Private Cloud they both run in.

But how do I update WordPress and plugins now?

Great question!

I will assume you have your RDS database doing automatic snapshots, if not then you should do a manual snapshot in case something breaks.

WordPress:

1. Checkout the WordPress version you are updating to in the Git submodule of your repository.
2. Do a deployment and wait until all the instances have been updated.
3. Put your site in maintenance mode. (Yeah I know, so annoying)
4. Do the database upgrade through the admin section.
5. Fin.

Plugins (haven’t quite mastered this one yet):

1. Update your plugins locally but with WordPress pointing to the production database.
2. Commit and deploy the plugin source changes to Beanstalk.

What did we learn?

It’s possible to do this. AWS is really powerful and reasonably easy to learn. Use other people’s stuff to make yours better. Thanks for reading.