Why End-to-End Testing May Save Your Mobile App
The wonderful world of testing your app, protecting your users from bugs & documenting changes on the fly
Today’s world is ruled by the mobile phone.
Whether someone needs to look something up, contact a friend, or they even just want something to stare at to avoid an awkward encounter, the mobile phone has firmly solidified its role in the hands of billions of users across the world.
Even in 2009, Apple were keenly aware that what they were building was going to become the main way that people accessed the Internet. After all, if you can think of it, there’s an app for that.
It’s no wonder that almost everyone — from the tiniest startup to the largest mega-corporate conglomerate — has an app of some kind. It’s a product that’s appealing for so many reasons; it can go with you wherever you want, it can connect to the Internet wherever there’s signal, it can let you know immediately by pinging you if something happens or changes.
But with mobile apps still in their infancy, and as with any technology — fledgling or otherwise — bugs and errors in the code can become a big problem. When it comes to mobile apps in particular, users have much less patience for bugs. It’s on my shiny new iPhone and I downloaded it via the App Store, it should just work, right?
This isn’t a new phenomenon either, this has been the case for mobile apps for a long time. A TechCrunch article from 2013 reported that:
79 percent report that they would only retry an app once or twice if it failed to work the first time.
Only 16 percent said they would give it more than two attempts.
That’s a low enough percentage to having mobile-first companies shaking in their boots pre-launch.
Clearly, bugs are simply not acceptable to the average user. Your mobile app, no matter what it is, needs to be robust. So the age-old question of a sleepless product manager comes up: How can we, to the best of our ability, prevent bugs from reaching our users?
Enter end-to-end tests.
Why end-to-end testing?
Ultimately, we can’t avoid making any bugs — even a codebase with world-class quality assurance will have bugs. The ultimate goal is to make sure any bugs that we do make get caught before getting into the hands of our users.
We can run through some basic parts of the app ourselves and make sure nothing crashes; absolutely. We can write some intricate unit tests to make sure our individual pieces of code work; definitely. We can examine these pieces of code in excruciating detail; of course we can.
But can we test these flows consistently, every time we release a new version of the app? Can we make sure we never forget to test a feature before shipping it? Will we ever forget to test it, or decide that we don’t have time to test it, and that it’s “probably fine”? Perhaps we played with it on Android, so it probably works on iOS too, right?
Coming up with the edge-cases is one thing; testing them all is a whole different ball game.
We could do all of these things, manually, and more. But the truth is that such rigorous testing can be extremely time-consuming, and the odds are that we won’t do it consistently, every time, all the time. We’re only human.
Your End-to-End Friend
Think of end-to-end tests (E2Es) as a more reliable version of your own manual testing. It’s an automated friend that, once you’ve told it exactly what to look for, it’ll go off and make sure that your app is working as you expect, whenever you want it to.
One incredibly useful aspect of writing these E2E tests is that we can use those test instructions we give it as living documentation.
At Birdie, we use Detox to test our mobile application, which is a grey-box testing framework designed specifically for testing apps and was designed with the aim to “tackle flakiness head on” when it comes to E2E tests.
By combining Detox with a tool called Cucumber, we can write user stories as our test instructions, giving us a useful new set of documented user stories that are directly tested.
As a basic example of this, let’s say we wanted to test that the user can log-in/log-out. We might write a user story using a straightforward Given/When/Then structure, something like this:
Feature: Logging in & out
Scenario: Logging out
Given I am logged in
When I logout
Then I should be logged outWith a user story like this, we can write code that breaks down these individual statements and executes them, allowing us to convert this story into a test. Here’s an example of something we might write:
This is exceptionally powerful. It opens up the testing of our app to the entire team, and any user story can be validated and tested against. Even a simple logging-in scenario like this can be extremely useful by being repurposed for much more complex scenarios.
With enough code to cover every individual statement, Detox & Cucumber essentially allows anyone, technical or otherwise, to start devising tests for themselves that not only help keep the product well-tested, but also add to the documented knowledge and understanding of how the product is supposed to function.
Ultimately, these tests are a time investment of sorts. It’s a short-term investment for long-term pay off. They will take a fair chunk of development time to get going, a large amount of user story creation to keep going, and they’ll likely need maintenance or iteration on them from developers to keep them working as expected.
But the time that E2E tests can save you in the long-run when debugging and testing your app manually is immeasurable. More importantly, the value translated to the user is unparalleled, because you’re giving them a strong, well-tested app that they know they can rely on and that you know has been documented exceptionally well.
And in a world where mobile app quality is the difference between life and death of your product, end-to-end tests may just save your life.
We’re hiring for a variety of roles within engineering. Come check them out here:
