Introducing Backstage: deploy individual branches with ease!

Illustration by Marta Pucci

You know what I miss from my PHP days? (…said no one, ever.)

No, but really: I used to host my Drupal projects on Pantheon¹, and they had a feature with which you could easily spin up a Drupal instance for a single Git branch.

Who needs that?!

It occurred to me recently how useful that setup would be here at Clue. I’m working on a team of two developers that’s responsible for the all-new React-based, Contentful-powered helloclue.com. Not long ago, I was working on the “welcome mat” at the top of each page that introduces first-time visitors to the site, while my teammate Omosola was working on a feature that enhances the excerpts of articles that you see on the home page.

We both needed our QA team to review the changes we’d made, but we only have one staging server, and it’s reserved for auto-deployments from the master branch. So to make our work available for testing, we both had to merge our changes to master and then send the staging link to our QA team for review.

But what if the welcome mat had a bug, or needed its text tweaked before it could be deployed to production? We’d have to delay deploying both features to production, since master would be in a bad state.

One way around this is to have a dedicated qa branch. We could both merge our work into qa, then deploy it to a dedicated QA server. But this entails a bit of overhead as well: the qa branch will often have fragments of old work that were merged in, which can cause merge conflicts. In order to avoid such conflicts, I’d need to reset the branch to what was on master. Before I could do that, though, I’d first need to check with Omosola to see whether she had unmerged features in qa that shouldn’t be removed.

These workflow problems can become endlessly frustrating, especially as a project and team grow in size. So I was prompted to reflect back on my PHP days, and the workflow I used at Pantheon.

One solution: per-branch servers

Using Pantheon, I was able to deploy a single branch, and it would create a server instance whose lifetime was limited to that of the branch. My QA team could then use that instance to test the changes I’d made, after which I’d merge my branch into master and remove the server instance. master would then be deployed to the app-wide “dev” instance, from where it could eventually be pushed to staging, and then production.

Having a separate server instance for each feature branch solved all of the aforementioned workflow complications: there were never merge conflicts, and I never stepped on teammates’ toes by accidentally blowing away their deployed feature on a shared server.

Since moving on to other software frameworks, though, I haven’t found a service with features comparable to Pantheon’s. Of course, deploying a new instance to Heroku is relatively easy, but there’s more overhead involved than simply git pushing my current branch to Heroku, since I need to set up an entirely new “dyno” to serve my new branch. Plus, free tier dynos go to sleep after thirty minutes; so if I send a link to someone on my QA team to view, and they only get to it a few hours later, chances are they’ll get pretty frustrated that they have to wait for slow load times to QA my code.

The closest solution I’ve found for front-end apps is Now by ZEIT. But while Now does allow for easy one-off deploys, the URL changes each time, so I couldn’t provide my QA team with a dedicated URL for a branch. Furthermore, URLs are publicly accessible, and I have little control over the server until I start paying monthly fees.

In short, what I want doesn’t seem to exist yet: an open-source server that I can configure to my liking, which serves per-branch deploys of my front-end app via an easy-to-use command-line tool.

Enter Backstage

This seemed like an easy enough problem to solve with code, and I know how to code! 🤓 So I started working on a simple solution, inspired by tooling that Dean Marano created for Groupon back when we were colleagues there. The goal was to have two complementary tools:

1. A CLI program for deploying individual branches of my front-end app to a server.
2. A server that receives these deployments, and then serves them on request.

To these ends, I created Backstage. The backstage-server library is dead simple: it returns an Express server instance with three routes:

• POST /__backstage/deploy for uploading a .tar.gz package of the app
• GET /__backstage/go/:appName/:branchName for setting necessary cookies and then redirecting to the deployed package
• GET /* for serving all deployed assets

To use backstage-server, you simply import it into your main app file, configure it with a storage backend, and call .listen(<port>) on it:

// your-app.js
const path = require('path')
const { backstage } = require('backstage-server')
const { fileSystem } = require('backstage-server/dist/storage-backends/file-system')
const fileDir = path.join(__dirname, '../files')
const storageBackend = fileSystem(fileDir)
backstage({ storageBackend }).listen(3000)

(Note: you can clone backstage-example for a fully functional example Backstage instance.)

I won’t bore you with all the details of how I implemented these features (if you’re interested, you can read the source code for both the server and the client!). But the end result is that our workflow for testing new branches at Clue is much easier now!

clue-website$ git checkout welcome-mat
clue-website$ backstage deploy
Your deploy can now be viewed at https://clue-backstage-instance/__backstage/go/clue-website/welcome-mat

I can then paste that link into Slack for my colleagues to review before I merge my branch into master.

If Omosola is working on a separate branch that also needs QAing, she can do the same thing:

clue-website$ git checkout article-excerpts
clue-website$ backstage deploy
Your deploy can now be viewed at https://clue-backstage-instance/__backstage/go/clue-website/article-excerpts

Our feature branches can now be tested separately and merged independently of each other, thereby ensuring that our master branch can always be deployed.

Coming up: MOAR FEATURES 🔨🗜🔧

At the moment, Backstage is in alpha. It stores deployed packages locally on the machine running the server instance, and uses cookies to remember which app and branch to serve you.

There’s a lot more I’d like to do with it. For starters, I’d like to have the option of using subdomains rather than cookies to determine which app/branch to serve. For example, for a new-feature branch of the clue-website app, I’d like to be able to preview the branch at https://new-feature.clue-website.clue-backstage-instance/. This way, QA folks can have one tab open for each branch they want to test, which isn’t possible when cookies are used to determine which branch to serve.

Another feature I’ll be working on soon is basic HTTP authentication. At the moment, we just restrict access to our Clue Backstage instance via a whitelist of IP addresses. I’d like to be able to give our QA team a username and password they can use to access the server, so they can easily access it from any location, such as when they’re working from home.

These and other features and ideas are tracked via GitHub issues in the respective repos. If you have additional suggestions, bug reports, or just general feedback, open up a GitHub issue or shoot me a tweet at @jessepinho!

¹ No affiliation, but 5/5 would recommend.