A Complete Guide to Testing in Flutter. Part 1: Introduction to The Types of Testing Methods

Testing Methods

Credit: Nguyễn Thành Minh (Mobile Developer)

Introduction

This is a comprehensive series on Testing in Flutter, covering Unit Testing, Widget Testing, Golden Testing, and Integration Testing. This series will guide you from knowing nothing to having a thorough understanding of different types of testing in Flutter. In each article, besides explaining the concepts, I will provide many examples along with common errors encountered when writing tests. Additionally, I will demonstrate how to write code to make testing easier and specifically introduce how to use AI tools like ChatGPT or GitHub Copilot to enhance test writing speed. Now, let’s start with the first part: Testing Methods in Flutter.

Why Do We Need to Write Tests?

Writing tests brings a lot of benefits to a project. In my opinion, there are three main reasons why writing tests is essential:

1. Ensuring Quality: Writing tests helps us detect bugs early and ensures the app operates correctly in various scenarios across different operating systems, versions, and even different devices. Moreover, some test cases are very challenging to test manually, such as calling more than 1000 APIs simultaneously, but we can easily write tests for such scenarios.
2. Supporting Code Refactoring: Code refactoring is extremely risky because it is easy to inadvertently introduce bugs. However, by running the tests after refactoring, we can detect these bugs if they exist. This makes us very confident when refactoring code.
3. Saving Time and Costs: Although writing tests adds initial costs to the project, in the long run, it saves time and costs. Clearly, detecting and fixing bugs early in the coding process or before building the app is less costly than after the app is built for tester testing or after the app is published. Furthermore, we only need to write one test case that can run on various operating systems, versions, and devices. In contrast, manual testing requires a significant cost to achieve this. Additionally, minimizing risks during code refactoring also helps us save time and costs significantly.

Testing Methods

We have four common testing methods in Flutter: Unit Testing, Widget Testing, Golden Testing, and Integration Testing. They differ in terms of purpose, scope, and test execution time.

1. Unit Tests

Unit tests are used to test functions like static functions, top-level functions, or methods individually. The goal of unit tests is to verify the correctness of a function or method under various conditions. For example, suppose we have three functions: saveToken, getToken, and login:

bool saveToken(String token) {
  return sharedPreferences.saveToken(token);
}

String get token => secureStorage.token; // cố ý code sai

bool login(String email, String password) {
  final token = apiClient.login(email, password);
  return saveToken(token);
}

So, with unit testing, we need to write tests for each function independently. For example, for the saveToken function, when a token is passed, the function's output should be either true or false depending on the test scenario. Similarly, when calling the token getter, it should return the token stored in SecureStorage. For the login function, depending on the input email and password, the function should return true or false. Besides testing the output based on the input, we can also test whether the apiClient.login function is called and how many times it is called. If it is not called or is called more than once, our code might still have a bug.

However, writing unit tests is only a necessary condition to ensure our app works correctly. In the example above, I intentionally made a mistake where I saved the token in SharedPreferences but tried to get the token from SecureStorage, which would definitely fail to retrieve the correct token. Why can’t Unit Tests detect this error? Because they only focus on testing each function independently. The saveToken function used to save the token in SharedPreferences, and it doesn't care where the token will be retrieved from. Similarly, the token getter used to get the token from SecureStorage, and it doesn't care where the token was saved. Of course, when each function performs its task correctly, we pass the unit tests. However, when running the app, these two functions working together cause a bug. This is where Integration Tests come in.

2. Integration Tests

Integration Tests are used to test how individual classes and functions work together or to test the performance of an application running on a real device.

Source: Reddit

For example, we need to test the login feature. When the user enters the correct email and password, we will navigate to the Home screen.

Source: Youtube

When running Integration Tests, it will start the app on a real device or emulator and automatically operate as if a tester is testing the app. Therefore, it ensures that the app works more accurately than relying only on Unit Tests. However, its drawback is that the execution time is much longer compared to Unit Tests. Additionally, when it detects a bug, it is very difficult to pinpoint exactly which function has the bug.

Unit tests and integration tests are commonly used to test the app’s logic. If we want to test the UI, such as whether the button’s color matches the design, whether the button is enabled or disabled, and whether the button is visible, we need to use Widget Tests and Golden Tests.

3. Widget Tests

The goal of Widget Tests is to verify that the UI of a widget matches the design and that its interactions work as expected.

Similar to unit tests, widget tests do not require running the app on a real device or emulator, so we do not spend much time executing widget tests.

4. Golden Tests

Golden Tests are essentially Widget Tests, but Golden Tests can verify whether the position of the widget on the screen is correct. For example, besides verifying that the button’s color matches the design, whether the button is enabled or disabled, and whether the button is visible, it also verifies that the button’s position on the screen matches the design. It does this by generating an expected UI image of the widget, known as Golden Images, and comparing it to the current UI image of the widget. If both images match, the test will pass. It can generate golden images on multiple devices with different sizes, such as phones and tablets.

For example, here are golden images, showing the expected UI of the initial state and the state after clicking the Floating Action Button once on two different devices: a phone and a tablet in landscape mode.

Expected UI (golden images)

If we accidentally change the color of the Floating Action Button to red and move the position of the two Text widgets to the top like this:

Actual UI

Golden Tests will detect these differences and notify us of the errors through comparison images as shown below:

isolatedDiff

maskedDiff

Golden Tests will save us a lot of time and project costs because we can verify the UI correctness by comparing images while normal Widget Tests can not. Therefore, I prefer Golden Testing to Widget Testing.

Conclusion

In this article, I have shared an overview of the four common testing methods in Flutter. I will introduce the details of Unit Testing i.