Define Testing Strategy using the Testing Pyramid

Colin But
Colin But
Aug 4, 2018 · 6 min read
Image for post
Image for post

For a software project to succeed you need a successful strategy to testing. One way of doing this is to use the Testing Pyramid. Whether you are starting out a new software development project or working on an existing project, it is important to have a test strategy in place.

We all know it’s bad to not have any test at all but it is also bad to have useless tests. Partial tests, intermittent tests that fail for no reason, hard to maintain tests are just some examples of bad dev testing. We all know there’s different tests to put in place. For example, unit tests, integration tests, end-to-end tests, UI tests, API tests, component tests, service tests, system tests, and the list goes on.

So you start to put in tests for your code. But where do you start? How do you tackle it?

This is where a Test Strategy comes in.

In this strategy it is best to utilise the Testing Pyramid to help you define and categorise the different types of tests.

Testing Pyramid

Image for post
Image for post

Very simple. 3 levels.

  1. At the bottom, is Unit Tests. These should count for the majority of tests you have for your codebase. They tests the smallest unit of code possible. Normally, only test a single method each.
  2. Middle level consists of Integration Tests which are tests designed to verify the integration of different parts of separate components of a software system together. This can be integration with a Database, with a Framework, with third party external software systems, or even with inter-software components between different layers (if you’re implementing a multi-layered software architecture software)
  3. At the top is the end-to-end tests which are tests that verifies the end to end workflows of your codebase. They tests the system from the user-action entry point right to the end of the system down to the database level. However, these are typically black-box tests.

A more detailed Testing Pyramid may look like this:

Image for post
Image for post

Component Tests

At the same level as Integration Tests you may have what is called Component Tests along side it. These are tests which tests the different components of a software system. Of course, you need to define within your system what constitutes your ‘Components’. This assumes your software is a modular distributed software system with separate ‘Components’, SOA, or Microservices.

API Tests

Sitting in its own level and in-between the top 2 levels is API tests which are by the name of it tests the API endpoints of your system. Assuming you have APIs of any sort, be it RESTful or non-RESTful. These tests are common because a lot of web apps these days have so-called backend web services which expose its functionality (CRUD like operations) via endpoints. So it is important to test the invoking of these APIs as well. Some people argue these are just Integration Tests. And they’re probably right, they’re merely just a more specific classification of Integration Tests.

UI Tests

Perhaps not so well known until recent years are these tests which tests the ‘UI’ aspect of your system (if your system has an UI). If you’re system is just a backend web service exposing APIs then obviously you don’t really need these and they’re likely not valid. i.e. You don’t have an UI!

Manual Tests

At the very top of the pyramid now sits a cloud or bubble of ‘Manual Tests’. These are tests which is performed manually. They test the whole system in a black box way. They are important as they test the system as a whole which each of the individual area tests cannot do.

Who should do what?

In a team, you have developers, QAs, DITs, and maybe even SDETs. For a team to function properly you probably need clear responsibilities assigned.

Image for post
Image for post
  1. QAs are obviously responsible for manual tests. Sometimes, SDETs do these tests too. They perform them alongside QAs.
  2. DITs would tackle the top level of UI & E2E tests. This is what normally people classified them as test automation. Automating tests which automate the testing of the whole system. One way to see them is to think of them as the automatic equivalent of manual tests but one must not think they can replace manual tests completely.
  3. Developer Tests are basically tests that is deep in the code itself. From the bottom level of Unit Tests right up to API tests. Sometimes, SDETs work alongside Developers to share this responsibility.
  4. a final word on SDET is that although they normally perform the tests of a QA & DIT together (because they basically are QA + DIT), sometimes more often than not, they work closely with the Developer to share his responsibility too. It is not uncommon to see SDET write tests for the whole Pyramid…. Because, at the end of the day, a SDET is by the name suggests — a Software Developer doing Testing.

Avoid the Ice-Cream Cone Anti-Pattern

Image for post
Image for post

This is bad. This basically is the inverse of the Testing Pyramid. It is the opposite of what the Testing Pyramid suppose to achieve.

You won’t achieve much with this approach. You could think of this as a bad strategy. Having a big portion of manual testing compared to that of Unit Tests cries for disaster. Development time increases. As it takes much longer to do things manually than having an automated test suite at hand which you can easily run within a small time frame (if you have fast tests that is).

One of my previous company i worked for, we had this anti-pattern in place. I didn’t notice it at first until i left that company only later on to come across a blog post which mentions that. Only then i realise… I’d wish i pointed that anti-pattern out to my former employers earlier so we could have saved a lot of time in development which would have resulted in quicker time to market for our business features.

We had little to none Unit Tests let alone Integration Tests. We did however had a large number of E2E tests with a bit of API tests. But mostly our testing was done by hand manually.

If only…

So the bottomline is… avoid this as much as possible.


Takeaways

  1. Write Integration Tests — integrate with the Database, Framework, 3rd Party external systems etc. Most people treat these as optional and i can’t blame them but i suggest put them in place. They will eventually save a lot of pain and you’ll thank them later.
  2. If you have APIs, write API tests.
  3. If you have an UI, write UI tests, otherwise no need.
  4. Automate as much of the testing you can via the different tests of the Pyramid.
  5. Don’t think automation can replace Manual Tests entirely. You’ll best to still keep doing these.

Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store