Why e2e testing matters

I’ve recently started a series of articles about end to end testing, because I think such tests are currently underrated. Lot of people think that having 80/90% of code covered by Unit Tests are sufficient.

I’ve often heard/read that end to end testing is expensive because they take time to write, maintain, and check — whether they really failed or returned a false positive.

Long story short :if you write your e2e tests for your main user scenarios correctly, they are not expensive at all!

End to end test prevents you to deploy broken applications.

Before going deeper, what is end-to-end testing?

The concept behind End to End testing is pretty simple: Simulate the behavior of your end user. In the case of a webapp, end to end testing opens a browser and follow the scenario you wrote. Exactly like your users will.

A real scenario can be :

  • Visit my webapp at https://foo.bar
  • You should see a login form
  • Click on the first input and write this email address “test@foo.bar”
  • Click on the second input and write this password : “foobar”
  • Click on the button with the label “Login”
  • URL should be https://foo.bar/profile
One of our scenario

Since you’re convinced you need some end-to-end tests, you need to :

  • Define your critical scenarios
  • Choose the right tools to write and run them

How to define your critical scenarios?

Unfortunately, there is no magical recipe to decide which scenarios you need to write. So let’s try answering these questions:

  • What are the most critical steps in my application?
  • What are the steps I always do when I test my application manually?
  • What are the steps that require a lot of time when testing manually?

At InTeach, I’ve identified several critical scenarios:

  • Login with email / password
  • “Read” the content of a module
  • Try the “timed evaluation” (because waiting 2 minutes when testing manually is too much)
  • Logout
  • Check if customized versions for our clients show right icons / images (because I’m too lazy to do these manual check and we have more and more customized versions to deal with.)

And that’s all. I think that we don’t need more end-to-end tests. They cover our main scenarios and for me it’s enough. Writing too much tests can be time consuming. Not only for writing these tests, but also maintaining it. The more tests you write, the more they will be dependent on your UI or your functionalities. And since both will change at some point… Be careful not to couple things too tightly.

Try to write generic tests, depending only on your main functionalities.

How to choose the right tools?

Once again, it’s up to you. But if I can give you a tip, you can take a look at my previous articles, in which I cover some use cases with cypress.io. The important part is not the tools themselves, but the global workflow.

If you write end to end tests, you should integrate them into your Continuous Integration system, running them locally does not make sense. They generally take some time to run… (Our current tests take 4 minutes to run)

You can read more on how we integrated Cypress.io and CircleCI here. Every time a PR is created on our github repository, our end to end tests are launched.

Conclusion

End to end testing doesn’t exist to simplify the developers’ life, that’s the purpose of unit tests. E2E tests are meant to:

  • give you confidence when you deploy a new version
  • prevent regression on your critical scenarios

But don’t write too much end to end tests ;).