Web engineering at Outfit7: A decade of evolution
Outfit7 is a world-famous digital entertainment company. We’re known and loved for our mobile games and the goofy, adventurous characters that populate them, along with our animated series and shorts. One of the smallest parts of our huge interactive landscape is our websites, which attract a combined audience of 20 million unique visitors every month, generating approximately 50 million page views. In short: we’ve got a pretty big fanbase.
But this is only the most visible part of our work. Web technologies also power in-game integrations, such as our compliance library and in-game user support, as well as an ever-expanding suite of tools that empower our teams to deliver the world-class mobile games we’re known for.
Today, we’re taking a look at the engineering journey that got us to where we are today. But to start this tale, we have to take a deep dive into the past.
My name is Primož and I’m a Senior Lead Software Engineer with Outfit7 Group. When I joined the company back in 2012 there was no web engineering department. Web development was the responsibility of a colleague of mine, who was already knee-deep into going full-time on UI design. Back then, my only responsibility was the Outfit7 website and our web-based cross-promotional interstitial ads, which were manually prepared each week and shown in our apps. More than a decade later, the team now takes care of an ever-growing portfolio of 52 websites and web apps, spanning from stunning landing pages to complex dashboards and hybrid native integrations. But how did we get here?
Let’s turn back time…
Back then it was a simpler time. We used Wordpress, wrote templates in PHP and ran a standard Apache server with two collocations and a load balancer. We had a beta instance of the website, so changes to both code and content could be verified before going live.
Our setup was clunky. Deployments were manual. Changes made to content on beta had to be manually recreated on production, which introduced the possibility for errors. Reverts were pretty common and they took a couple of minutes to be deployed. If the issue was a bug affecting the deployment procedure, they couldn’t be performed, as the revert was essentially a new deployment of an older tag.
Going all-in on mobile
Mobile-first as a concept was all the rage in 2013. In its essence, it advocated for responsive web design with mobile screen sizes as the standard, using media queries to adapt the layout for bigger screens. Conceptually, it transcended the way we thought about layouts and CSS. It was also very much about latency, load times and response sizes.
At a time when 83% of worldwide internet traffic was still on desktop, our websites were already mostly visited by mobile users. This meant we needed to put our websites on CDN, get the content closer to our users and use caching to shorten load times. Yikes! Our stack was PHP and we had dynamically-generated websites. 😭
We started tackling the issue gradually. First came a Wordpress filter function, which performed a URL rewrite of uploaded assets, allowing us to serve uploaded content directly from our CDN through an origin configuration. This change already substantially reduced the load and cost of serving websites and, for a time, everything was fine.
// enable CDN upload folder path - images from upload folder are served from CDN
function cdn_upload_url() {
$protocol = isset($_SERVER["HTTPS"]) ? 'https' : 'http';
return $protocol.'://cdn-o7.outfit7.com/wp-content/uploads';
}
add_filter( 'pre_option_upload_url_path', 'cdn_upload_url' );During this phase, a large chunk of our time was dedicated to preparing cross-promotion campaigns — also called “news”. These were our biggest success drivers, since we could cross-promote our games to existing users and grow fast.
There was a lot of innovation during this time. Android KitKat (4.4) was released in 2013, along with iOS 7, both of which brought wider adoption of the CSS3 standard in their WebView implementations. This enabled us to experiment with animations and transitions and (most importantly) to leverage hardware acceleration, through which we could launch more engaging campaigns across a wider range of devices.
Our cross-promotion engine went through several major iterations between late 2012 and 2016, when the system was finally retired in favor of a native integration in our apps. We started with manual weekly collages of new & top-performing news items from the previous week. This was quickly followed with localisation support for our major markets and a standardized set of creative types or templates that allowed us to dynamically generate the list of creatives — moving from having to deploy an updated web app every week to deploying only the configuration.
Over time we expanded support for more complex creative types, moved the configuration to the backend, which first introduced dynamic campaigns by geographical location. Later, with the help of machine learning, we completely automated and optimized campaigns based on our games’ expected LTV in a certain region.
Over the following years, with the successful releases of several new games, as well as a bigger focus on marketing campaigns, our team’s responsibilities grew. There was only so far we could go with the same processes before needing to dramatically expand the team.
At the same time, we were also being faced with a steady increase in our number of visitors, the majority of them coming from mobile devices. We’d also realized that a large chunk of our visitors try to access our websites from low end-mobile devices, and from places with poor internet connectivity.
First major tech stack change
We knew we needed to drastically change the tech stack and process of how we developed our websites. In 2018, in order to be able to focus more on user experience and faster turnaround times, we started to decouple our monolithic full-stack websites into separate backend and frontend elements. This brought a slew of goodies that made our lives much easier.
Separation of concerns
This was perhaps the change that was felt the most by us developers. Frontend development became purely a matter of Javascript, HTML and CSS. No more nasty PHP! We still kept Wordpress, which was now in headless mode. Our editorial team didn’t even notice any change, since their favorite CMS remained exactly the same. On the other hand, we were now pulling content from Wordpress through the API each time the frontend was built. Win, win!
Statically built websites, served directly from CDN
In 2019, we decided to go ahead with Gridsome, a Vue.js based framework designed for statically built websites. Since updates were periodic, a cron job that rebuilt the website a couple of times per day was more than sufficient. We also had several daily deploys for regular updates and a hook in Wordpress to trigger a new deploy with any updated content.
By using a CDN, we no longer served users from two locations, but instead suddenly had local points of presence for most of the world. We saw load times and Document Interactive Times reduced by 67% on average.
Introduction of CI/CD and atomic deploys
A deployment pipeline connected to our GIT process meant that we didn’t have to worry about deploys. Each push was a new deploy. We established a clear separation with pushes to master branch triggering production deploys. Other branches were used for deploy previews of ongoing features, making it easy to share with teams and decision-makers to speed up approvals.
Now, our deploys were atomic — changes went live all at once, thanks to deploys being published as a separate instance, with the final stage being the domain pointing to the latest one.
Reverts also became easy and almost instantaneous, since a revert was simply a change to which deploy the domain was actually pointing towards. With the adoption of a deployment pipeline came the ability to extend it, allowing us to run unit and e2e tests, as well as additional pre or post-deploy tasks (i.e. deploying auto generated documentation).
Modern frontend framework adoption
We adopted Vue as the framework for most of our projects, which enabled us to shorten development cycles and prototype faster. This allowed us to shift our focus to projects with greater complexity, while still being able to maintain existing websites and run regular marketing campaigns.
This ushered an era in which web technologies were used to develop in-game user support, the News feature in our Mythic Legends game, the compliance library and web apps that helped solve internal problems either at the team level (e.g. user testing and QA dashboards) or the company level (e.g. the cafeteria app and Confluence updater).
A couple of years earlier, in 2016, our backend team had started using Google Spreadsheets in ever-more complex ways, as sources for configurations and data for services, game releases, live ops, testing scenarios, localisations and more.
These spreadsheets were regularly parsed and served as a sort of user interface through which teams could configure the crucial services they were responsible for. Of course, this system was prone to human error, not to mention being extremely limiting.
In some scenarios, the need for a custom tool was identified, such as facilitating QA by enabling access to a test device’s API request flow, behavior overrides and logs. Or in another instance, enabling customer support teams to more easily respond to support tickets, by giving them the ability to make adjustments to individual user’s game accounts. This is how Dashboards were born, with AngularJS (and later Angular) as the framework of choice. With Typescript support and a fully fledged full-stack-oriented framework, it was the perfect candidate for our then-backend team to branch out into web development.
The tech stack unification
Over time, as the scope of the team grew, we reached a point where it made sense for the teams working on user-facing and internal websites to merge into one, which we did in 2022. With a single team, a unified tech stack was the natural next step, which allowed us to adapt faster, share knowledge and expertise among projects and more seamlessly transition between projects, with project specifics being the only outstanding delta.
Where are we now?
We now have several processes in place that allow us to be efficient, have unified workflows across projects and keep things running smoothly. For example, we have common coding guidelines, code reviews, “starter” repositories for new projects, automated builds with CircleCI and custom common libraries — most notably a UI component library, which allows us to build dashboards quickly, while also unifying the look and feel across the board with the help of a dedicated UXer.
Most of our projects now use either Vue.js 3, or Nuxt.js 3 (as a replacement for Gridsome) if there is a need for statically-built websites. The rest are in the process of being migrated. Vue 3 with Typescript and Composition API is the perfect mix for developing projects ranging from marketing campaigns with immersive UX to complex, logic-heavy dashboards and in-game integrations.
We also expanded the use of unit and e2e tests from a few dashboard projects to the majority of workflows. For larger, critical web apps we monitor application errors, server error logs and perform uptime checks to detect failures and major outages.
What holds the future?
History is repeating itself yet again, as we go back to full-stack. This time, we’re going with NestJS, a progressive Node.js framework for building efficient, reliable and scalable server-side applications; and Prisma, the next-generation Typescript ORM. This will allow us to fully own the web applications we create, iterate faster and support our end users better. Things are about to get exciting! Become part of the story, and help us shape the next chapter of Web Engineering at Outfit7.
