On Testing — Part 2
Types of Integration Testing Approaches
There are quite a few different approaches to how to go about testing your application and the decision solely depends on the team and the product. Below are the most commonly mentioned approaches:
Big Bang
The big bang approach is effective as its name makes you feel. Shortly, it tests the whole system in one go. This also means, the tests are written after the development is completed.
- This approach can be mostly useful when the system is not too big, because the bigger the system gets the more unmanageable the tests will become. Big bang takes all the units and tests them all in one go, which takes planning out of the equation. This may make you feel like you will save time somehow, but because the idea is to test everything together, it actually could consume the most time.
- The biggest problem that could be faced with big bang apart from maintenance of the tests is finding the problems that caused the failure of the tests. Because everything is tested in one go, it may be hard to pinpoint what’s wrong and where it is wrong.
Bottom-up
Bottom-up approach can be a good way to go if the integration testing started together with the development of the application. As the name suggests, in a tree of modules, the testing begins at the very bottom, which consists of units that are disconnected from one another and moves up from there. Since the modules at the very bottom usually don’t work on their own and need some motivation to work, they would need drivers that would run them within the tests.
- The best part about the bottom-up approach is, if a small unit stops working, it will immediately be obvious where the problem is.
- The disadvantage of starting the development of an application with this approach is that you won’t realise if you actually have problems with bigger modules until you are done with everything else, because the testing started at the very bottom.
- Another issue due to the same thing will be the time consumption of the tests. Considering all the smaller units are already tested, you will have so much integration tests right off the bat on your hands, which will take a lot of time and will be hard to maintain.
Top-down
Again, as the name suggests, this is the exact opposite of bottom-up. The top-most modules are initially tested during development and at this point the bottom modules would not be available.
Just like the drivers that were mocking the parent modules, top-down uses stubs. The child modules and necessary functionalities are replaced as stubs and this allows the modules in development to be tested freely.
Unfortunately this approach can get really tricky as it is hard to consider everything about a smaller module and what it is supposed to do. For example, a smaller module might be handling some http request or a database connection and it is hard to even determine how to use a stub for this and test it with the parent module.
- Just as the opposite of bottom-up, it becomes easier to find problems with the system and harder for smaller units.
- The testing time is actually shorter as the initial approach is to test bigger modules, which are much fewer in count.
Hybrid/Sandwich
Here we would see both bottom-up and top-down approaches combined, which creates the hybrid style. The parent modules would be tested separately and once the implementation is completed the lower level modules would also be tested. This approach would require the usage of both stubs and drivers.
First you need to select a middle layer to start the testing process from. This middle layer needs to be the ideal layer that would allow you to use minimal amount of stubs and drivers. After this, starting from this middle layer, you would start your bottom-up and top-down approaches. For the bottom-up approach you would start from the middle layer and go up and for the top-down approach you would start from the middle layer and go down.
- Big projects can really benefit from this approach as they usually have too many modules to mock on both top and bottom parts of the application.
- At the same time, as expected, it would require more effort from the team and takes more time. Especially if your modules depend on each other too much, it will become even harder to go with this approach, you will start getting so many drivers and stubs that the whole testing process might start getting complicated.
What about my application with a GUI?
Now, after reading about integration test types, you might be wondering how you could possibly implement any of the test types for testing your application with a GUI and why you would use drivers or stubs, especially if you are writing E2E tests for an application with a GUI like a mobile or a web application. My simple answer to that is, you don’t. Yes, for the business logic and such, this is still possible, you can simply test the units and integrations of these units, but that’s unfortunately not enough testing most of the time. For such applications you would want to test things out from the user’s real perspective and use some E2E tests.
E2E testing is supposed to test the user’s behavior as close to reality as possible. Filling your tests with drivers and stubs would simply move you away from this reality and put you in an impossible situation. So, while these testing types may be easily considerable for other systems, E2E testing is a beast of its own.
How and what should be tested with E2E tests for such systems is a different subject. This is not something that a developer can simply say “I have implemented this feature, I will test literally everything about this.” The best way to manage this is together with your team and your PM. The important thing here is that E2E testing can take much, much longer time on your CI builds. You have to carefully go over your product’s needs and what you can provide for your tests and implement them accordingly.
A Good Approach to E2E UI Testing
First of all, the vital parts of the application should be determined, you have to make a plan for testing these parts and create user paths for them. These user paths can best be taken from a funnel from an analytics tool, because it is really important to know what your users are mostly doing and to test this. This doesn’t however mean that you shouldn’t test any edge cases.
Because of the long build times on CI builds, you have to make some hard decisions. You cannot test every edge case, this wouldn’t make sense, because there are always endless edge cases for every such application, especially if it includes dynamic data with something like filters.
Assuming you are done with the coverage of the vital parts of your application, it’s time to cover the rest of the application with such tests. At this point you have to decide how far you are willing to go, because if you want to cover your whole application with tests, this means that your tests will run for a very long time. You can split these tests into parallel tests, but this also means more machine power and higher costs. It is important to test everything, but you should also understand that the more E2E tests you write, the less important they will get, unless they are for a new important feature. With this and the cost and the time consumption in mind, you have to find a middle-ground on what should be tested and what should be disregarded.
