Intro to Re-engineering the Yak

The goal of this blog series is to share some of the lessons learned as we evolved Yik Yak from early-stage startup code running in Amazon Web Services to an eventual incremental rewrite, re-architecture, and live-migration to Google Cloud Platform. By sharing our journey with the engineering community, we hope to help other startups benefit from our findings, avoid running into some of the same issues we encountered and find useful implementation tips for your own processes.

Before we start, it is important to understand that both Amazon Web Services and Google Cloud Platform have strengths and weakness, and both can be used well or poorly. Our experiences should not be viewed as an indictment of one or an endorsement of the other — we will cover the good and the bad of both. Rather, this series is meant to trace out the ramifications of technical decisions made early in a startup’s life and how how we, specifically, chose to address those as we scaled the platform.

Our original backends were a mixture of PHP and NodeJS, they used Elasticache (REDIS) as a geostore/datastore and DynamoDB as a “backup”, everything was driven off of SQS queues, they did not do any autoscaling, they did not log, they did not expose much in the way of operational metrics, deployments were manual and the system was a big monolith. Most of these things were probably reasonable at one point or another in the startup’s life, but at scale these things added up to a system that was very, very expensive to run and very difficult to operate.

The final system was very different. It was entirely written in Go, used a combination of Google’s S2 Library and Google Bigtable to create a low-cost and performant geostore, had standardized, analyzable logging, standard metrics collection/dashboarding, used Google Container Engine (Kubernetes) for efficiency, used gRPC for communication within the “datacenter” and the transition was executed live with zero downtime. We were able to achieve the same performance as the old stack without optimizations but the cost to operate was 1/10th that of the original stack.

We will cover a lot of ground. Up first: DynamoDB — What you should know…