Maintainable User Interface Testing with Ember
Maintainability is a common goal in software development. We seek to code maintainably so that we can return to it in the future and understand what it’s doing. We seek to architect maintainably so that we have well-defined places to build new features and add new files. We seek to style maintainably so that we can tweak the background color of a “primary” button without the paint spilling anywhere else.
But what does it mean to test maintainably?
Unit testing — testing the inputs and outputs of isolated functions — can be pretty straightforward. Maintenance often means accounting for the 1:1 relationship between the subject’s arguments its return value. And unambiguous metrics like code coverage are often reliable gauges of our unit test suite’s overall health.
User interface testing, however, is an entirely different beast. When we write UI tests (also known as acceptance tests in Ember), we’re running the app in full: We’re initializing state and rendering the DOM; we’re sending out HTTP requests and loading data; we’re clicking on buttons, filling out forms, placing books in carts, and adding songs to our new mixtape. In most cases, we’re also repeating this process over and over for different types of users and different routes within the app. An easy analogy would be test driving a car, but really, it’s more like programming a plane to fly through different obstacle courses and occasionally inspecting the black box during the flight.
Being able to sustain this process as we build, refactor, and build even more is perhaps testing’s greatest challenge. So without further ado, let’s examine some of the techniques that can be used to overcome that challenge. We’ll walk through things like searching in the DOM, conquering asynchrony, managing state across tests, and designing reusable page objects. We’ll also see how Ember is perfectly equipped to help us at every step of the way 🐹.
Searching the DOM: Decoupling Complexion from Dissection
A large part of UI testing resolves around finding things. DOM elements, to be exact.
Let’s take the example of writing an acceptance test for a user registration page containing a form that requires typing the same password twice in order to submit.
A common instinct — even among those of us who try to follow Test-Driven Development — is to implement the form first, generate an acceptance test by running ember g acceptance-test registration , look at the markup in our registration.hbs template, and then write something like this:
While a test like this might pass initially, it’s very susceptible to breaking later for reasons having nothing to do with password confirmation validation:
- A designer decides to change the button text from ‘Submit’ to ‘Register’.
- A CSS-savvy developer decides to remove the
.username,.password, and.confirm-passwordclasses because they’re all just.text-inputs. - The development team decides to adopt BEM for their CSS, and
.form-field-errorsbecomes.form-field--has-errors. - The company decides to require all user passwords to have at least one special character, making the number of errors after validation equal to 2.
With these possibilities in mind, to make our selector queries more maintainable, we need an approach that combines a surgeon’s depth with a radiologist’s breadth.
We need Ember Test Selectors.
Embracing a convention commonly used on QA-heavy teams, Ember Test Selectors allows us to leverage the power of HTML attributes by attaching any attribute prefaced with data-test- to an element, and then finding it later by passing that attribute’s suffix to testSelector helper:
Most crucially, this allows us to keep our selector queries data-driven — data-test-submit-button will always be an attribute of our submit button, whether its text is “Submit” or “Register”. At the same time, we can assign values to our data-test- attributes and pass them later as a second argument to the testSelector helper. From the snippet above…
find(testSelector('form-field-error-message', 'password-confirmation'));…allows us to precisely target an element with a data-test-form-field-error-message attribute equaling password-confirmation — without needing to know how many other errors are present or what the exact message text contains.
If you’re still on the fence about using Ember Test Selectors, consider also the fact that it offers built-in binding for data-test- attributes on components, and the ability to strip data-test- attributes from production build.
For example:
{{comments-list data-test-comments-for=post.id}} gives us something like:
<div id="ember123" data-test-comments-for="42">
<!-- comments -->
</div>which ships to the end user as:
<div id="ember123">
<!-- comments -->
</div>Achieving this level of utility with a hand-rolled implementation would be a heavy maintenance burden to say the least. But with Ember Test Selectors, it’s never a concern.
If you’re interested in learning more about using Ember Test Selectors, I’d highly recommend this walk-through from EmberMap as a good compliment to the project’s own excellent documentation.
DOM API: The Web’s OG
Another coupling in the first example above that’s a bit more subtle, but still important to point out: a reliance on jQuery:
assert.equal(find('.error-text').text().trim(), 'Passwords must match');I’ll spare my views as to whether or not jQuery itself is a bad thing, but using its selector syntax and its matched-element methods without explicitly importing it — without explicitly saying that you need it and know it will be there — is a recipe for trouble in the future. How, so? As it happens the “future”, pertaining to Ember testing, is coming sooner than you might imagine: The Grand Testing Unification RFC for Ember has gained widespread support (and traction) among the core team, and one of its many proposals is the removal of jQuery from Ember’s built-in test helpers.
Fortunately, that’s where the outstanding Ember Native DOM Helpers project steps in to help. Using its helpers (which currently map to the same acceptance test helpers we’re already used to), we can read elements using standard Element.querySelector syntax — and we can guarantee that the return values are references to real, raw, in-the-flesh elements (you know, these).
Using the native DOM API will help standardize the syntax we use for finding and operating on elements, provide a more standard set of expectations about the behavior of fired events, and reduce the amount of cognitive overhead for developers who may not be familiar with jQuery.
One caveat, however, is that unlike Ember’s built-in acceptance test helpers, the replacements offered through ember-native-dom-helpers don’t automatically wait for the completion of any asynchronous behavior going on under the hood.
With the next pattern at our disposal, however, they won’t have to.
Patiently Async-Awaiting
Most of the interfaces we make are driven by data and pliable to interaction. Their shape, while ultimately deterministic, is rarely complete before events can be handled, before Promises can be resolved, before generators can yield, before AJAX requests can return, before… well… you get the idea: Analyzing our “images” of the DOM, to extend our radiology metaphor, often requires time for them to develop.
But it doesn’t need to be our time 😛.
While JavaScript provides many patterns for concurrent programming, there’s one unifying syntax we can use to control this flow on the surface in a way that appears synchronous: async / await .
For the purposes of improving the readability, maintainability, and structure of our code, few features promise to have as profound an impact on the JavaScript language as async /await (which, if you can’t tell, I highly recommend getting acquainted with) — and although it’s still a Stage 3 proposal for ECMAScript 2017, enabling it inside of Ember requires just a single piece of configuration within ember-cli-build.js :
const app = new EmberApp({
'ember-cli-babel': {
includePolyfill: true
}
});And with that, there’s little reason not to begin utilizing asnyc / await in Ember acceptance tests.
Because awaitis essentially concurrency sugar, using it with Ember tests means that we no longer need to worry about where and when to nest code inside of an andThen block, and it also means that the framework itself can begin freeing up its built-in helpers from the intricacies of asynchronous timing.
Here’s yet another improvement to our validation test. Notice how we simply preface the function inside of the test block with the async keyword:
Data State: Mopping It Up and Mocking It Out
Localized data isn’t just an important concept for clean code — it’s also vital for clean tests. When we visit a page, the number of comments under a post or the score being displayed next to the user’s avatar are a direct result of the app’s data state, and it’s important to make sure that A) our tests are focused on only initializing the data they need and B) our test suite is able to prevent side-effects to this state with subsequent tests.
Ember CLI Mirage is the perfect tool for this. I’d go as far as to call it a must-have for most projects. With factories and fixtures, it gives us fine-grained control over where, when, and what we generate for our app’s in-memory database (which it also provides 🙂). With route handlers, it wraps Pretender to enable easy mocking of API responses. And with models, it allows us to define the relationships according to the way we expect them to be defined in our app.
Consider a set of acceptance tests for a playlist details page. With Ember CLI Mirage, Ember Native DOM Helpers, and Ember Test Selectors, it might start out looking like this:
Right away, we can see where Mirage offers a world of utility in just a few lines:
const playlist = server.create('playlist');server.createList('song', STARTING_SONG_COUNT, { playlist });
This creates a clean playlist record and a related set of song records just for our test. And it does so without us having to fire up our own server somewhere, hit its proper endpoints, and then manage that state across the rests of our tests (not to mention on the server itself 😀).
And let’s keep going with that point. What if we want to add a few more tests for this page as things get a bit more CRUD-heavy? Without proper database setup and teardown logic, one could easily imagine having to write tests that are tightly coupled to each other — and thus break if we decide to… say… add a test for our new “multi-select and delete” feature for song items.
Once again, though, Mirage alleviates these concerns. By hooking in to Ember’s destroyApp helper (which is used during each test by the moduleForAcceptance helper) it ensures behind the scenes that each test starts with its own clean data state — allowing it to be focused, fine-grained, and blissfully unaware of what’s going on in the world beyond:
Ember CLI Mirage has been a mainstay of the Ember community for several years now, and I highly recommend checking out its own comprehensive documentation for more guidance.
DRYing Things up with Custom Test Helpers
In any significantly sized project, we’ll likely find ourselves making similar assertions about the DOM across very different pages and in the wake of very different behaviors. Our playlist example above demonstrates this: Testing for a certain count of items on a page is almost certainly not exclusive to the songs in a playlist. And the more this repetition increases, the nicer it would be nice to have a well-defined, conventional pattern to DRY this assertion up with a helper that can be shared across test files.
Enter custom test helpers. Just by running ember g test-helper <helper-name> , Ember will generate a file inside of the of the tests/helpers/ directory that stubs out a function which can be called from anywhere in our acceptance tests. By default, Ember will stub out a function viaEmber.Test.registerAsyncHelper , but we can also use Ember.Test.registerHelper if we aren’t running any asynchronous code and if our helper isn’t dependent upon any asynchronous helpers that will run before it or after it.
Here’s one potential implementation of a helper for our element-counting needs, given we run ember g test-helper shouldHaveElementCount and substitute Ember.Test.registerAsyncHelper with Ember.Test.registerHelper:
For more on custom test helpers, I’d recommend looking in a great slide presentation created by Jeffrey Biles, which demonstrates even more ways they can be used — including mixing them with other test helpers and encapsulating large chunks of application functionality.
I’d also recommend using — or at least referencing — popular community test helpers. Ember CLI Acceptance Test Helpers is a well-maintained addon by 201 Created offering goodies such as expectComponent, expectElement, andclickComponent , and Ember Power Select is a great example of a component addon project that ships custom test helpers to abstract its own internal behavior.
The Page Object Pattern
A discussion about maintainable UI testing would be incomplete without touching on one of its most resilient patterns: the page object pattern.
Like many of the topics covered here, this can, and has, inspired much longer write-ups of its own, but in short, the page object pattern involves, well, treating pages as objects. The elements of a page and the selectors used for finding them are defined as part of the object’s “class”, and so external users get an API where they only need to know about the former.
A page object wraps an HTML page, or fragment, with an application-specific API, allowing you to manipulate page elements without digging around in the HTML.
~ Martin Folwer describing the page object
And while we could build out this functionality ourselves, we can once again leverage community conventions with Ember CLI Page Object.
This addon provides a generator that we can use to create reusable “pages” for our tests. If we wanted to refactor acceptance for user registration around page objects, we could run ember g page-object registration and then build something like this:
As you can see, selector logic is managed exclusively — and once — in our page file, and our tests only need to know about the properties exposed by the registrationPage object.
Potential Page Object-ions
Unlike the patterns mentioned above, embracing the page object pattern is a major architectural investment. There’s no way around it. Ember CLI Page Object is highly configurable and does a good job of being unobtrusive, but it may be one more level of abstraction that you or your team are comfortable with (or need). You may also run in to trouble when attempting to combine it with tools such as Ember Native DOM Helpers due to its usage of jQuery behind the scenes. (Fortunately, though, at the time of this writing, there’s an effort underway to change that.)
That said, overall, Ember CLI Page Object is yet another great example of how we can leverage Ember’s ecosystem and conventions to tackle complex architectural challenges.
Conclusion (and Honorable Mentions)
Testing, to paraphrase a few million developers who came before me, is a powerful tool for writing quality code. But knowing how to write tests that are maintainable will take this to the next level. Testing maintainably is testing the right things — and continuing to test the right things. It’s quickly shipping new features and still keep coverage in check. It’s “move fast and don’t break things”. And quite frankly, it’s pretty darn fun.
Furthermore — similar to accessibility — when you start to get familiar with a lot of the principles maintainable testing demands, it tends to make all parts of your application better. And that includes the people making it.
We’re fortunate that Ember has cultivated an amazing ecosystem for putting the aforementioned principles of maintainable UI testing into practice — so much so, that this article still feels a bit like scratching the tip of the iceberg. Here are some quick recommendations for going even further:
- Running tests with past, present and future versions of Ember using Ember Try.
- Cross-browser, cross-platform, and cross-device testing with SauceLabs using Ember CLI Sauce.
- Visual regression testing with Percy using Ember Percy.
