Aardy Engineering
Published in

Aardy Engineering

How we migrated a Monolith LAMP app to the JAM Stack and AWS

3am wakeup

It’s 03:00 on a Sunday, the familiar Slack ‘knock knock’ notification echos from my phone. I already know why... The AARDY site is down for the fifth time this week. The server has decided to s**t the bed again. A memory leak caused by a rogue Wordpress plugin is causing Apache to run out of memory. I quickly jump onto my laptop, SSH into the server and restart Apache.

Problem fixed 🥳 … well not really, but for now… 🙌

Of course I’m waaaaaaayyy too busy to to fix the above issue ‘properly’ so I write a cron job to restart Apache at a set time every day, yeehaaaaa! 🤠

Time to wakeup

Things were getting serious. Our travel insurance business was growing, our traffic was doubling every month and the server problems were happening multiple times daily.

  • We scaled our EC2 instances horizontally, but this just amplified the problem.
  • We scaled our EC2 instances vertically, but this was expensive.
  • We removed plugins from Wordpress to be told by the Marketing team that they ‘really needed’ that plugin.

It was a mess, our team were not experienced enough in the LAMP stack to truly resolve the issue in the long term.

We needed to find a solution and fast!

We needed some JAM (Stack)

Around this time, a CRAZY new idea of creating static sites burst on to the scene. Which, if you didn’t get the irony, turns out is not actually a new idea at all and is simply how you create a website. However, this coupled with React made this idea a very powerful proposition indeed and Gatsby was the hottest in town.

So we jumped head first into Gatsby.

We replaced our entire view layer, previously server rendered PHP / TWIG with React, switched the CMS from Wordpress to Contentful and pulled the page data at build time via GraphQL. We now had a static site that was blazing fast and great for SEO. Excellent!

But what do we do with our PHP / MySQL Quote and Purchase engine??

We re-wrote it of course! 🙃

We played to our strengths and ported the app to NodeJS / Mongo. The engine was split into two, dare I say… ‘micro services’ using Typescript. Re-writing key functionality and adding new features as we went. We containerised these services using Docker and deployed them to AWS Elastic Beanstalk.

While most of the team are experienced enough to configure and manage server infrastructure, we decided to use Elastic Beanstalk and focus on developing the product.

What was the end result?

We now have totally clean separation of concerns; with an extremely secure, blazing fast frontend hosted on Netlify and resilient NodeJS services running on AWS Elastic Beanstalk.

No more 3am wake ups! If a service goes down now, it can usually recover on its own, either at the container level (if the app crashes) or at the infrastructure level if the instance becomes unhealthy and is replaced in the auto scaling group.

Because of this separation we can now focus more on the functionality of the product. For example we are now using AWS SNS for communication between micro services and other systems. We have various Lambda functions that we use for data ingestion and scheduling customer communications. We use SQS in our data pipeline, Sendgrid and Salesforce integration.

We are no longer scared to make a change to production. We know that any changes made are isolated to that specific service so we can deploy 20+ times per day to production on any one service.

Oh yeah and… The blip was my fault 🤦‍♂️

It’s not all rainbows 🦄

Whilst this project has been a huge success for us, there are some negatives! The main issue we now face is exponentially slow build times with Gatsby, it’s just the nature of the beast. The more content we add, the slower it will become.

We have patched up the problem by introducing various caching and incremental build solutions but it wont hold forever.

On the AWS side of things, we now have more infrastructure than ever before to manage. But everything is much more resilient, it’s a balancing act.

In conclusion, Gatsby came just at the right time for us, it solved a lot of our problems and allows us to focus more on our product and less on the monolith.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store