Why we ditched Unit tests for Integration & End-to-End tests

Arsene Toumani
Feb 7, 2019 · 6 min read

With the rise of agile development practices like continuous integration, development teams have struggled to find the right balance between unit, integration and end-to-end tests. Most developers settle for the 70/20/10 rule as promoted in this Google Testing article which splits testing to 70% unit tests, 20% integration tests, and 10% end-to-end tests.

Testing pyramid

For some spending 70% of their testing efforts on unit tests is not optimal; this was the case for our team at TelosTouch, a SaaS platform that allows financial advisors to guide client behavior at scale. We tried it for a few months and it didn’t work, so we ditched unit tests in favor of integration and end-to-end tests.

The problem

As a team, we wanted to be able to answer questions like is our application working the way we want it to, or more specifically:

  • Can users log in?

The problem with the 70/20/10 strategy which focuses on unit tests is that it doesn’t answer these questions and many other important high level questions.

Unit tests don’t test how well components work together

Unit tests are too isolated to test real user stories. They are fast and isolate failures really well, but don’t provide the same confidence integration and end-to-end tests do. Especially with a frontend heavy application like ours, where the user’s behaviour is complex and unpredictable.

Furthermore, most issues reported in our JIRA were a result of miscommunication between multiple UI components and backend services. When incorrect information was presented to our users it was usually because a service didn’t access the required information from the right location, or because different frontend components didn’t work together properly.

Our solution

Since unit tests didn’t have much impact on the reliability of our application, we needed a better testing strategy to help us answer high level questions like, can a user still send a message after all these code changes?

After close inspection, we noticed the small amount of end-to-end tests we had did a better job of spotting issues we hadn’t seen before; they proved to be valuable in our quest for reliability. This prompted us to to flip the pyramid on its head and focus on both end-to-end and integration tests.

Inverted testing pyramid

Our new strategy was to split our automated tests into 20% unit tests, 30% integration tests and 50% end-to-end tests. With the help of testing frameworks like Jasmine, Protractor, Sinon and Karma, this new strategy helped us catch 95% of issues before they made it to production.

20% unit tests

We still believe unit tests provide many advantages to our team; they run fast, provide great isolation for failures, and are much more reliable than integration or end-to-end rests.

Because we now only use 20% of our testing efforts on unit testing, we’re picky about which functions to test. We only focus on functions that have a high probability of breaking when the logic is incorrect or when it receives wrong inputs.

Unit test example at TelosTouch

This is an example of a typical function we unit test; the getTaxedIncome function receives an income amount as input, runs a tax algorithm and returns an output. It’s a good case for unit testing, instead of Getter and Setter functions that usually never break.

30% integration tests

Integration tests are tricky to implement especially on the frontend because larger functions tend to interact with many different services and APIs. We use Sinon.js to spy, stub and mock external services since they’ve already been tested. And just like our unit tests, we only focus on high reward functions.

Integration test example at TelosTouch

This is an example of an integration test we wrote for a function that makes sure a user’s request to a client is denied if they are authenticated but not authorized. It’s a high reward function because it deals with an important part of our application security; only authorized users can access client information.

50% end-to-end tests

We focus most of our efforts on end-to-end tests; they test the whole application from start to finish. They ensure that all the integrated pieces of our application function and work together as expected. They simulate real user scenarios, essentially testing how a real user would use the application.

End-to-end test example at TelosTouch

This is one of our end-to-end tests which executes the following user actions:

  1. Click on the nudge button to open the nudge view (chat conversations)

We use Protractor to control the browser and simulate a client nudging and starting a conversation with their advisor regarding an update about a life change. Running multiple tests like these to test every single functionality within our application helps us ship code confidently.

There are however hurdles we had to overcome in order to make this possible within our continuous integration environment where code is released every week. Some of them include:

  • End-to-end tests take a long time to run. This can be a problem when you’re trying to deliver code fast (continuous delivery FTW!).

We took several steps to solve these issues, and I’ll be sharing them with you in my next blog post so stay tuned!

Conclusion

Finding the right balance to how much unit, integration and end-to-end test to write is not easy. Every team and application is different and requires a different ratio. A good way to identify the best strategy is to analyze and reflect on questions like which tests within your codebase have made a big impact? which ones saved you from having your application crash on production? For us, it was clear that end-to-end tests made the most impact and found the most bugs, with integration tests coming in second.

This is one of the many creative ways my team and I are solving engineering problems. If you like this, make sure to follow, check out our careers page and share it with your friends.

TelosTouch

TelosTouch’s collaboration platform enables financial…

TelosTouch

TelosTouch’s collaboration platform enables financial institutions to reveal client needs in real-time & deliver personalized and tangible advice at scale. Our technology unlocks a rich behavioral data set that drives client engagement, advisor capacity & business opportunities.

Arsene Toumani

Written by

I help build products, teams and processes for industries I’m most passionate about. Co-Founder & CTO @systelos

TelosTouch

TelosTouch’s collaboration platform enables financial institutions to reveal client needs in real-time & deliver personalized and tangible advice at scale. Our technology unlocks a rich behavioral data set that drives client engagement, advisor capacity & business opportunities.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store