Image for post
Image for post

Dragoon: an automated UI testing tool

Introduction

Andrew Pillsbury
Jul 23 · 4 min read

The Big Idea

Historically, my team’s automated tests have mostly consisted of function-level unit and integration tests. These give us great code coverage, but have a few drawbacks. The biggest is that each test has limited scope and requires using mock data. In addition, creating these tests is time-consuming as each needs to be written by hand.

  1. automatically cover most of our code with a single generic function, and automatically test new code without having to update the tool
  2. test using a variety of accounts (which each have different permissions and feature flags)
  3. record how long the code takes to execute

The Solution

Enter Dragoon. Historically, dragoons were mounted soldiers who sometimes dismounted and walked. We named our tool after them because it “walks” our pages and “mounts” each one. Dragoon is a lot like Jeffbot, but instead of calling the APIs directly, it loads pages in our UI. Like Jeffbot, it uses several different accounts, and for each one iterates through our site’s pages using that account’s permissions and feature flags. The data for the pages comes from APIs running on a test environment, and is very similar to our production data. Dragoon records how long each page takes to load and any errors or warnings it encounters.

First Implementation (Enzyme)

Our initial implementation of Dragoon used much of the same tooling we already had in place for our frontend unit tests. Karma connected the tests to an environment, ran them, and collected output. Enzyme rendered the frontend code locally in headless chrome and connected it to the test environment APIs. Jasmine was used for the test cases and assertions.

Second Implementation (Cypress)

Fortunately, around the second time that Dragoon broke we discovered a new technology called Cypress. Crucially, Cypress can bypass CORS, so we decided to scrap our initial implementation and rewrite Dragoon to run on Cypress instead.

  1. Dragoon needs to know the list of routes to iterate through, but that list comes from a helper function in our code that Cypress doesn’t have access to. We fixed this by bundling our code in with Cypress using webpack.
  2. We want console errors to count as test failures, but, by default, Cypress ignores them. We resolved this by having Cypress intercept calls to ‘console.error’ and fail the test if there are any.

Conclusion

Despite some challenges along the way, Dragoon is now consistent and stable. It automatically provides broad coverage of our UI, acting as a smoke test that gives us increased confidence in the code we ship. It also provides consistent timing metrics so that we can track the performance of our pages, either across all users or just for a specific account. We’re excited to take advantage of these features and explore how we can use Cypress to continue improving Dragoon.

Xandr-Tech

Our latest thoughts, challenges, triumphs, try-again’s…

Thanks to Joe Roepcke

Andrew Pillsbury

Written by

Software Engineer at Xandr (formerly of clypd)

Xandr-Tech

Our latest thoughts, challenges, triumphs, try-again’s, most snarky and profound commit messages. Our proudest achievements, deepest darkest technical debt regrets (just kidding, maybe). All the humbling yet informative things you learn when you try to do things with computers.

Andrew Pillsbury

Written by

Software Engineer at Xandr (formerly of clypd)

Xandr-Tech

Our latest thoughts, challenges, triumphs, try-again’s, most snarky and profound commit messages. Our proudest achievements, deepest darkest technical debt regrets (just kidding, maybe). All the humbling yet informative things you learn when you try to do things with computers.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch

Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore

Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store