Rebuilding a Web Platform for 30M Users

We’re redesigning our playground’s tech stack, and we’re sharing our experiences as we go.

Sebastiaan Moeys
Feb 1, 2017 · 5 min read
Lots of fun, a pain to maintain — via RollerCoaster Tycoon/Hasbro

Since I founded Poki, we’ve grown a lot. Every month, tens of millions of players around the world use our platform for easy access to free games. More recently, we’ve launched a publishing service to help game developers succeed on web and mobile.

What once started as a collection of web games is now growing into a platform for play: Poki brings game developers and users together in a sustainable way and gives developers a fair share of gameplay revenue. We’ve only just begun, but we’ve been quick to realize that building a platform means reimagining every aspect of our business.

One such aspect is our codebase: the current version of Poki.com — currently serving 30 million unique users every month — is still powered by an outdated PHP back-end and legacy front-end, making it increasingly difficult to build the playground we envision.

Our codebase has reached its limits: years of product evolution, market changes and technical debt are taking their toll. Unrelated subsystems are entangled, unit tests have abandoned ship, and hacks upon hacks have taken over. Our monolith took us to where we are now: a thriving web game platform where millions of kids of all ages come together to play. But we have the ambition to grow further, and to do so, we have to think fresh.

We just started and have decided to document this journey so that our decisions and learnings can be shared with others. We’ll share the good, the bad and the ugly, so if you don’t agree with our ideas or think we totally missed the mark, don’t hesitate to give us a shout-out. Feedback is appreciated.

Our Monolith

Poki started as a personal collection of web games, a website created in 2005. It started as a project for fun, and it taught me to code along the way. The result was a website consisting of plain PHP files, linked to a MySQL backend and chained together with “php-includes”.

The early days

Not an architectural miracle, but it got the job done and allowed users to start playing. The site grew organically, and in the years afterwards I localized the site to 20 languages and and made it available for a global audience.

By 2014, I’d found a great co-founder and, and together we evolved my sites and games into Poki. By 2016 we were a team of 25, and incremental product improvements and partnerships with popular game developers had grown our user base to tens of millions of monthly active users.

Our business leveled up, but the underlying codebase was still built upon a dated foundation. Despite periodic rewrites and a move to AWS, there had never been a properly designed architecture.

The system had become a complex mix of AWS products (Elastic Beanstalk, RDS, Elasticache), legacy PHP code, custom scripts and hacky work-arounds. More and more sub-systems were added, and the system slowly became loaded with dependencies and coupled packages.

We were outgrowing our stack.

At the time, there were valid reasons for not going all-in on a rebuild or refactoring. But looking back we already saw these red flags surfacing:

  • Single responsibility principle: most features were built around the same database model, which meant they would end up in the same classes, slowly turning them into god objects. As a result everything became tightly coupled.
  • Lack of conventions and standards: there were no team agreements on how code/tests should be written. With little pair-programming and rare code reviews, every developer could change the application any way they deemed fit. The ‘don’t repeat yourself’ principle slowly vanished, since developers didn’t know the state/features of the codebase.
  • Testing: god objects (and especially entangled code) is difficult to test, so at some point the testing pipeline broke and was never fixed.

The end result was a tightly-coupled monolithic codebase filled with peculiarities and specific gotchas. Needless to say this was undesirable: hard to get into for new hires and a pain to maintain.

That said — we don’t have any hard feelings about our monolith. It’s our sacrificial architecture: we used it to iterate, learn and grow, and it allowed us to get where we are today. We want to throw it away and that’s fine: it means that we grew, both as a company and as an engineering team.

Designing from Scratch

In the summer of 2016 we decided that our systems were no longer capable of keeping up with our platform ambitions, and we declared the codebase “end-of-life”. We decided to fully redesign of our architecture, rebuild only the core features of our current product, and gradually develop our new platform from there. We’re looking ahead, and we’re keeping quality in mind from the start.

Our architecture objectives:

  1. Maintainability. Our architecture should be modular. We’re aiming for a highly aligned, loosely coupled system that is testable, expandable, and easily accessible for new team members.
  2. Scalability. We’re designing our system for 100x its current load. We want linear scalability, meaning that twice the load will cost us twice the resources (as opposed a situation in which additional load leads to exponential cost increases). We also want scalability on a per-module basis, so that we can scale modules that are experiencing heavy load without having to scale our entire platform.
  3. Speed. We’ll be promising our users a product that is blazing fast. Part of this challenge will be tackled by designing for cost-effective scalability. However, we also want our architecture to give us flexibility: if speed is an objective, we need to be able to pick the right tools for the job, on a per-module basis.
  4. Vendor agnostic. Our system should be able to run on every cloud platform, or bare metal servers if that’s what we want.

We’ll be designing our architecture by implementing and testing as early possible. Once we have a functioning infrastructure, we’ll be releasing an MVP of our web platform in a test market with only very basic features. We’ll build and iterate on our new platform from there, testing our assumptions along the way.

The engineering team will be sharing their progress over the upcoming weeks in a series of posts covering:

→ Enjoyed this story? Follow Poki to stay up to date about future posts!

Let me know what you think by clicking the 👏 below or leaving a comment.

Poki

Poki's blog on culture and engineering.