Writing end-to-end tests in 2021

Do you know where to start?

There’s no way around it — testing software is tedious. For the longest time, testing has been an unfortunate mix of brittle unit tests, stubbed integration tests, dubious smoke tests and manual verification regression tests that takes a day or two for a team of QA engineers to complete. And even with all of those tests passing, there is a real risk your app crashes anyway. So what is a developer to do?

Enter “end to end” testing. With a few articles like this one on board, you’ll write much less tests and have greater confidence in your code, all while making refactoring easier. Sounds too good to be true, right?

We will discuss what end to end tests are, what tests you should write, what tools to use and how to write your first few tests. In addition, we will make sure to summarize what not to do with end-to-end tests.

What are end to end tests?

End to end tests (or E2E tests for short) are tests that test your entire application flow, to make sure your entire application works as expected. Think of E2E tests as automated versions of real user interactions, where you tell the computer the same you would have told a test user to do.

These kinds of tests are extremely powerful, in that you can test large swaths of your application with only a few lines of code. Your test framework will spin up a regular or headless browser, load your application into memory and start interacting with it through click and keyboard events. This will give you confidence that if these tests pass, it will work for your users as well.

What tests should you write?

Even if this all sounds amazing, you should limit the amount of E2E tests you write. Since these tests spin up a real browser and interact with it, running them will be more resource intensive and slower than unit and integration tests. Therefore, you should focus your E2E testing on the main usage scenarios of your application.

Let’s look at an example restaurant review application. It requires authentication, and you can browse restaurants and review them. Here are the end to end tests I would have created for it:

• When unauthenticated, show login form, enter valid credentials, verify logged in
• When unauthenticated, show login form, enter erroneous credentials, show error, verify not logged in
• When authenticated, show list of restaurants, click one, verify reviews show up
• When authenticated, go to restaurant, click add review, enter review, submit review, verify review shows up

With these tests in place, and passing, I am pretty certain the core functionality of my application will work for my users. There might still be bugs, and edge cases I haven’t accounted for, but I can write integration tests or unit tests to make sure that code works as intended. The key is — I won’t have to write lots of those to have a high degree of trust!

Setting up a modern E2E test rig

End to end testing has been around for decades, but most of the tools available were made for enterprises and old windows machines. Names like Selenium and TestCafé comes to mind, and I have terrible experiences of flaky, slow and hard-to-maintain tests in both of them.

Luckily for us, there’s a new player in town. Cypress is a great tool that provides us with a modern solution for creating, running and maintaining code. Combined with Cypress Testing Library and the accessibility audit tool aXe, you’ll have all the tools you need to gain confidence in your application, and to never deal with manual verifications in prod ever again.

To get started, we need to install some dependencies:

yarn add --dev cypress @testing-library/cypress cypress-axe

This command will — in addition to installing your dependencies- also create a cypress folder with a bit of boilerplate, and a cypress.json file you can use to specify some global settings. There isn't anything there by default, but we can add a base URL to avoid having to start every test with navigating to the same URL. Add the following to your cypress.json file:

Next, we want to include the helper methods from Cypress Testing Library and cypress-axe, so that we can use them in our tests. Navigate to the ./cypress/support/commands.js file, and add the following lines:

Finally, we’ll add two new scripts to our package.json to trigger test runs — one running a real browser and one running a headless one in your CI pipeline:

And with that, we’re ready to start writing tests!

Writing tests

Writing end to end tests are pretty similar to writing regular tests. We start off by creating a new file — ./cypress/integration/main-customer-flows.js , making a describe block, and injecting the accessibility plugin before each test.

We’re going to be using the global cy object to interact with Cypress.

This looks like pretty known territory for anyone who has ever written a unit test or two. Let’s write our first test — a test that checks the login functionality of our app.

We start the test by navigating to the login page, and ensuring there are no major accessibility errors in that view Any missing labels, inadequate color contrasts or other WCAG violations will be caught here — a great safety net in a single line of code.

We then find the input labeled by the text “Username”, and call the type method to type text into it - just like a user would. We do the same with the password field, and hit "enter", logging the user in.

To ensure the login worked as expected, we make sure that the URL now includes “/profile” — the URL we redirect to after login. Finally, we ensure the profile view is accessible as well.

Let’s write another test for when the user types in the wrong credentials:

Here, we do the exact same test, but entering a different password. We assert that we’re still on the login page, and that we see an error indicating that the username or password was incorrect. We also ensure the page is accessible in this state as well.

This practice of verifying that the page is in an accessible state at every point in the user journey is one of my favorite things about end-to-end tests. It’s a thing that’s extremely time consuming to do manually, and incredibly valuable for both end users and your compliance department. You’ll still have to do some manual accessibility testing to be sure things work by the way.

I love how readable and straight forward these tests are. There are no scary test IDs to remember, no brittle element selectors, and it’s easy to understand what’s happening in a test by just looking at it.

Bonus — Adding your own custom commands

As you write more of these tests, you will probably write some pieces of logic several times over. Logging the user in to your app is one of those. Luckily, Cypress lets us specify our own custom commands to make our tests even more readable!

We define our custom command in our cypress/support/commands.js file:

This will make the cy.login(username, password) function available. Now we can refactor our tests a bit:

As your test suite grows in size and complexity, you might even want to avoid interacting with the UI at all for logging the user in. Instead, you can fire off HTTP requests to the server with the cy.request method. You can see an example of how this is implemented in the documentation.

Pitfalls

End-to-end tests are great for a number of reasons, but they should not be the only way to verify your application.

As we mentioned at the outset, end-to-end tests are slow and resource intensive, which make them great to run before deploying your code, but not while developing or committing change. Instead, you can get a lot of the same benefits with regular integration tests. These don’t necessarily hit your backend system, and can be run in Node instead of in the browser. The end result is a much faster feedback loop, where you can test much more specialized scenarios than would be practically possible with end-to-end tests. Unit tests should also be a part of your testing suit, but should focus more on complex business logic than how your components render.

Therefore, I suggest you only create end-to-end tests for the main customer actions and business objectives of your app. This will make your tests run fast enough for you to actually run them, and they will function as the safety net they’re supposed to be.

Frontend Application Monitoring

A broken experience is not always due to an error or a crash, but maybe the consequence of a bug or a slowdown that went unnoticed. Did my last release introduce that? Is it coming from the backend or the frontend? Asayer helps answer those questions and figure out what part of your code requires fixing or optimization. Because inconsistent performance simply drives customers away, causing retention and revenues to drop.

As we embrace agility, we push code more frequently than ever, and despite our best testing efforts, our code may end up breaking for a variety of reasons. Besides, the frontend is different. It runs on different browsers, relies on complex JS frameworks, involves multiple CDN layers, gets affected by 3rd-party APIs, faulty internet connections, not-so-powerful devices, and slow backends. In fact, the frontend requires better visibility, and Asayer provides just that.

Your frontend is your business. It’s what people touch and feel. Let’s make it fast, reliable, and delightful!

Start monitoring your web app for free.

Get to testing!

End-to-end testing is an incredibly powerful and effective way to test your applications. You can verify that your app works as expected, and that there aren’t any major accessibility errors, all without validating a single thing by hand.

With tools like Cypress, Cypress Testing Library and cypress-axe, you’re ready to start writing end-to-end tests like a pro. Just remember not to go overboard with it!

Originally published at https://blog.asayer.io on March 30, 2021.