Scaling Wordpress Beyond a Single Server in Google Cloud

One big adventure of scaling a software company is found in how you manage uncertainty.

How do you stay online? How do you handle influx of traffic? These questions are engineering questions of little to no exact answers. If you ask one person what scale means, you’ll talk to someone else who has a completely different answer. I’ll be sharing a little bit of my philosophy going forward here which will hopefully provide a good mental construct as to how we ended up in the server structure that we did.

I’m writing this to help other CTO’s, CIO’s, hackers, engineers, software developers, mom and pop’s, or ANYone that’s interested in making their Wordpress run better at scale. I am the CTO over at SocialCurator where we provide social media consulting via regular monthly releases of stories, images, captions, and plans for how to run a successful social media account for your business.

Whether you’re running a server rendered website, a single page application, a multifaceted distributed system, or an event driven masterpiece, software in the cloud has 3 fundamental layers.

1. What the user sees
2. The math that’s required to present them information
3. A place to store the information

You can store information in #1 and even #2 you could do all the data transformation right in the browser. Or you can create HTML from step #3 from SQL Queries (maybe). So these are not hardest rules, but I’m generalizing and for the sake of clarity, we’re not going to create our website using only a database, but that does sound like a fun side project.

No, the three layers need to be separate, but when it comes to building servers it is so much easier to all of these things on one server. That means the Database (and related data), the code that needs to run (PHP in the case of Wordpress), and the delivery of dynamic and static assets like HTML/CSS and Javascript ALL fall under the umbrella for one machine.

The Single Point of Failure

Having your DB, application code, and local files/content all on one node means that if that bad boy goes down, you’ve lost everything. The first benefit of splitting your app into more parts will be decreasing the risk of failure. At each point that we break our app into, we have the opportunity to inject more strength. At the upfront cost of taking more time to get set up, if you’re planning to be around for a while and plan to continue to hit your growth metrics, you’re going to want something like this. Otherwise, you’re going to have to pay top dollar for a similar service that may or may not work as advertised.

At a high level what we’re doing is putting our database on its own server (with the option to add more than one or even fail overs is painless inside GCP), we’re going to run the PHP application on 2 separate Linux servers in an instance group from within a managed instance inside GCP’s Compute Engine. We’ll also go through how to enable this application to autoscale this PHP layer so that as load increases to the site, these servers will grow. But at night or non-launch periods, it will shrink back down saving your CPU cost that you don’t want to spend if you don’t need to!

In front of those instances we’ll use GCP provided load balancers to receive HTTPS traffic and distribute the load to our group as HTTP to reduce the overhead that Apache needs to handle. I don’t know that there’s too much of a concern to terminate like this. We’re talking about machines within the same local network so it’s pretty unlikely someone malicious is literally sitting inside the server room sniffing traffic. Though, if you are going for HIPPA compliance or other higher level government compliance, these could be something you avoid.

A smaller part of this project was to get the `wp-content/uploads` folder moved outside of the application files. We’re going to use a combination of a Wordpress plugin called WP-Stateless to accomplish that along with a little creative NGINX proxying so that we can point our existing Wordpress file links to the bucket files instead of the local versions. Our goal in moving that folder out was to reduce the file size of the core application. Wordpress itself is about 12mb zipped. The version that we’re running at SocialCurator is about 250mb withOUT the uploads folder. Our upload folder is >10GB and growing. Deploying all of those images every time we have a deployment was not feasible.

Lastly, we’re going to link up Cloudflare so that we can take advantage of their edge caching. We’ll be using a 100% free account which takes only moments to setup.

Step 1: Creating your GCP account

If you don’t already have a Gsuite or Gmail account, you’ll need to make one. For the sake of this article I’m signing up for a new trial under my personal account. Just sign your rights away and click continue!

$300 in free credits is pretty amazing. I was able to get our full site built and running (with a few scaling mistakes in between) with $180 of credits left while it was in production

As it mentions, there’s not autocharge after the trial credits are used up. On the next page you’ll see credit card details are required for the trial. You will not be charged until you hit a big fat noticeable “Activate” button within your account. There will not be any surprises. Actually, when it comes to GCP, most of the time they will only surprise you in good ways like if you have over allocated a server, they’ll outline how much extra your spending and provide a simple one click button to downgrade sizes based on your actual usage.

I’ll be honest, the biggest part of their dashboard is just how many options there are to go through.You might just be thinking, “I’m just here for a simple website!?” Well, even if that is the case, once you find what you need, you can just pin what you need and forget the rest.

This is the landing page after creating the trial. As soon as we have some machines the new landing page will have heads up views and graphs as to how your servers are doing and how active they are.

Step 2: Create MySQL Instance

Within the sidebar, scroll down to the storage sub-section. We’re going to find anything related to storing files, folders, images, children (ok no, maybe they don’t do daycare yet), or anything data related. I’d suggest pinning “SQL” to your sidebar at this point since we’ll be accessing it a lot. Clicking on SQL will bring us to a Create Instance page.

There’s a migration option here, I attempted to use this but it wasn’t successful for me. The code we were bringing in had something that didn’t gel with a new version of MySQL which I had to edit manually. So I just created the instance and imported the data via the command line. In this case, we’re going to build a new Wordpress from scratch so there’s not import step needed. The next step takes us to a fork in the road:

MySQl — 5.6,5.7 these are open source versions of MariaDB

PostgreSQL — I’ve never touched it. Hipsters only!

SQL Server — Pretty sure this is the windows licensed version. Not my cup of tea.

Alright, so we’ve chosen our DB type. I’m pretty sure MySQL will be the best out of the box solution for Wordpress but feel free to experience and experiment the others. $300 in credits will go a long ways as long as you clean up your tests when you’re done.

Copy that password into a secret place. I stored mine inside LastPass and updated it with the IP which I grabbed after the instance was created. I stuck with the default selected option for the version. As well as ‘us-central’ as a default is just fine for me. I hardly notice a difference in hosting locally in LA rather than central. It would just cause me to forget where I picked and have to constantly keep looking it up so going with the default at least saves me that extra lookup. Leaving zone as “any” will do just fine.

Below are the settings shown under “Show Configuration Options”:

This is the lowest option available suitable for a production instance. It will run about $50/month. There are two lower options that run on shared CPU’s. I attempted to use shared CPU’s inside GCP and it was not worth the headache. I had issues with my CPU just blowing to all heck and things crapped out left and right.

They take a nightly snapshot of the database. The point in time recovery is amazing for the times where things do break and you need an exact moment of time to be brought back into existence. I think both of these options will incur a little bit more in storage cost for the instance based on GB of the data itself, but it’s worth it.

For this project I’m just keeping it single zone since this is mostly for fun. High availability means the master node can take a complete crap with your data, and it will be taken offline and the backup will automatically take its place. Unlink in my experience with managed DB’s inside (now Atlas) the fail over will incur some downtime and should only be used because of actual emergencies.

This is one of the things that freaks me about about using this. You mean my database is going to come offline because of routine maintenance??? I have no further comment here. I’ll be riding this out and watching closely. I can only assume they say this to cover their butts.

Now we see a loading wheel indicating that are our MySQL instance is being created. These operations vary in time. This should only be a few minutes. EDIT: it’s been about 10 minutes and it’s still running. EDIT: okay, 15 minutes isn’t THAT bad. You can navigate to other things while this is happening which is convenient. From the last step you might have seen that “private IP/network” was not an option due to a list of permission issues. This is because we haven’t been inside the compute engine so your account hasn’t initialized any networking related things. This would be a good time to pin the “Compute Engine” to your sidebar. This is something we’ll use heavily. Go to that Compute Engine page and will say, “initializing”.

We’re not going to make anything new yet. Now that it’s been initialized, I want to get the private network open on the MySQL instance. Because we’re only going to be connecting to this database from machines inside GCP, we can leverage their private network for security, speed, but also avoid data egress from MySQL which I believe even accessing the public IP from a local machine would incur. But lastly, the public IP address requires a manually update list of allowed IP addresses which we’d have to keep track of our server IPs. Which is especially not going to work when autoscaling groups give each new node a new IP address.

Click on SQL from the sidebar. We should see our new instance there. In my case it’s called “sc-blog” but it will be called whatever you named it earlier. Clicking into it we’ll see the overview for this server. We can see it’s CPU over time. We’re interested in the EDIT section of this page, so let’s click EDIT. It’s towards the top.

Under configuration options, click the dropdown for Connectivity. Check the option for Private IP. That should prompt an alert asking you to turn on the API for the Compute Engine.

Clicking Enable API should take just a couple seconds. Once the popup is closed, click Allocate and connect within the Connectivity section we were already in.

If this article is too wordy, or there’s too much detail, I’m sorry about that. I just found myself coming across the same articles for GCP related tasks that were personality-less and boring. #sorryNotSorry

One it’s been allocated and connected, scroll to the bottom of the page and save the page.

Step 3: Creating Instances

In this area, a few things are key, Instances, Instance Templates, Instance Groups. Templates are used to create instances based on a set of parameters and most importantly, startup scripts. Instance groups use the instance templates to automatically create instances.

…to be continued




Co-founder (with my wife!) of Dubsado. Currently the CTO of SocialCurator. We have 2 lovely boys, a pet cat, and I’m currently shopping for a small SoCal Ranch.

Love podcasts or audiobooks? Learn on the go with our new app.

Guide to Using Regular Expressions in Golang

Micromouse from scratch| Microcontroller- STM32| PICCOLA

clone script | HWINFOTECH

WandX blog update — Bringing trust to Defi

How to Use BLoC to Build a Reactive App with Flutter

Trillian log sequencing: demystified?

I want to learn web development, how do I Start

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Jake Berg

Jake Berg

Co-founder (with my wife!) of Dubsado. Currently the CTO of SocialCurator. We have 2 lovely boys, a pet cat, and I’m currently shopping for a small SoCal Ranch.

More from Medium

Industrial Exposure: Building FinTech with INDMoney

Technical Design and Development of a Self-Sovereign Identity Management Platform for…

8 Inspiring Examples Of Modern UI Design — Hashlin