Project Miniclient — Testing
That’s the part of the Project Miniclient tutorial. If you missed the beginning, you may want to check the introduction first.
Testing is often misunderstood as a process of finding bugs. However, in reality, its primary goal is to validate that the software under test works according to the specification. Thus, having clear specifications is the key to meaningful test suites. In software development, specifications typically come in two forms:
- Engineering specification: Defines how the code functions under the hood.
- User specification: Describes the application’s expected behavior from the user’s perspective.
For client app testing, user specification is much more important. It all starts with the description of what is useful for a user, and never from the hierarchy of classes or what classes, functions, or methods should do. Engineering specification appears as a result of development and refactoring. With these specification types requiring different testing approaches, we recommend starting by covering the app’s behavior. However, since engineering requirements typically are easier to cover with tests, let’s start with them in this tutorial.
Solitary Unit Tests
Unit tests cover individual classes in isolation. This kind of testing creates a one-to-one mapping between code and test and is usually used when you need to cover reusable code components with defined engineering requirements.
For example:
- The
Resultclass from thepreludepackage (if you’ve completed all bonus exercises, you’ve already tested it). - The
MarvelApiAuthInterceptorfrom theapi_clientpackage. - The
Monitoringclass from themonitoringpackage, which we’ll cover next.
Setting Up Dependencies
Let’s start by updating monitoring’s pubspec.yaml file with two new dependencies:
...
dev_dependencies:
mocktail: ^1.0.4
test: ^1.16.5
...test package provides a standard way of writing and running tests in Dart. mocktail is a helper library that simplifies work with external dependencies.
Creating Mocks
TheMonitoring class depends onFirebaseCrashlytics, which is an external class for the package. There are multiple techniques that help to deal with external dependencies in unit tests. In this tutorial, we will use “mocks”. A mock simulates the behavior of a real object. Mock follows the interface of a class, so mock objects can be used in place of a real one
Create a new firebase_mock.dart file under test/util:
import 'package:firebase_crashlytics/firebase_crashlytics.dart';
import 'package:mocktail/mocktail.dart';
class MockFirebaseCrashlytics extends Mock implements FirebaseCrashlytics {}
FirebaseCrashlytics createMockFirebaseCrashlytics() {
final mock = MockFirebaseCrashlytics();
when(() => mock.log(any())).thenAnswer((_) async {});
return mock;
}MockFirebaseCrashlutcs extends Mock and implements FirebaseCrashlytics. That means it has the same functions and properties as FirebaseCrashlytics but will return null for every function invocation. That’s not always what we want. Hence, we create a helper function createMockFirebaseCrashlytics, which instantiates the MockFirebaseCrashlytics and configures it in a way that when the log method is called with any argument, the mock answers with an empty asynchronous function. With this approach, by default, the mock will always do nothing without failing a test. We will follow the same strategy for all mocks.
Writing a Unit Test
It’s time to write a test. Create the test/monitoring_test.dart file with the following content:
import 'package:firebase_crashlytics/firebase_crashlytics.dart';
import 'package:mocktail/mocktail.dart';
import 'package:monitoring/src/monitoring.dart';
import 'package:test/test.dart';
import 'util/firebase_mock.dart';
void main() {
group('Monitoring tests', () {
late Monitoring sut;
late FirebaseCrashlytics crashlyticsMock;
setUp(() {
crashlyticsMock = createMockFirebaseCrashlytics();
sut = Monitoring(crashlyticsMock);
});
test('Tracking event should call Firebase log', () async {
// Arrange
const event = 'test event';
// Act
await _sut.trackEvent(event);
// Assert
verify(
() => crashlyticsMock('log: $event'),
).called(1);
});
});
}There is a naming convention to call the “system under test” class instance variable sut. All sut‘s dependencies should be instantiated and pre-configured in the setUp function.
Notice we used the Arrange-Act-Assert pattern in the test. That’s a common pattern that divides the test into three strict areas:
- during the arrangement, you are preparing the test environment for invocation,
- during the acting, you are running the scenario,
- and during the assertion step, you validate whether the testing environment is in the desired state, i.e. checking the expected outcome.
It is important for a test to have a single purpose, so if you have multiple “act”s and multiple “assert”s in a single test, you are mixing scenarios under the same umbrella. That makes tests harder to maintain.
While // Arrange, // Act, // Assertcomments are optional, I recommend having them in the codebase till it becomes natural for you and the team to follow the rule of having a single purpose for the test.
Bonus exercises
1. Reflect on what is needed to be covered with tests. Are there any tests missing?
2. Reflect on why it is important to follow the Arrange-Act-Assertpattern. Are there any scenarios when this pattern is not working?
3. Read the mocktail’s source code. Make sure you understand how it works (in general, not necessarily becoming an expert).
4. Write unit tests for all other utility packages.
Sociable Feature Tests
Feature tests validate the interaction of multiple components, simulating real-world usage. To streamline testing, we’ll create a utility package for common mocks, network stubbing, and initialization.
Creating a Test Utilities Package
Create a package test_utils under utility with the following pubpspec.yaml file:
name: test_utils
description: Miniclient test utilities
version: 0.0.1
publish_to: none
environment:
sdk: ^3.2.0
dependencies:
dio: ^5.4.0
flutter:
sdk: flutter
flutter_test:
sdk: flutter
mocktail: ^1.0.4
monitoring:
path: ../monitoring
tide_di:
path: ../tide_di
dev_dependencies:
tide_analysis:
path: ../tide_analysisMocking Networking Requests
To avoid real network calls in tests, we’ll create an interceptor for mocking responses. If you’re unfamiliar with this approach, watch the last part of the “When Your Backend Is Not Ready” video.
Create a new file lib/src/test_test_interceptor.dart with the following content:
import 'dart:convert';
import 'package:dio/dio.dart';
class FeatureTestInterceptor extends Interceptor {
final _mockedRequests = <String, String>{};
final _mockedFailures = <String, String>{};
void mockRequest(String path, String json) {
_mockedRequests[path] = json;
}
void mockFailure(String path, String json) {
_mockedFailures[path] = json;
}
@override
void onRequest(
RequestOptions options,
RequestInterceptorHandler handler,
) {
if (_mockedRequests[options.path] case String successJson) {
handler.resolve(
Response(requestOptions: options, data: jsonDecode(successJson)),
);
} else if (_mockedFailures[options.path] case String failureJson) {
handler.reject(
DioException(requestOptions: options, error: jsonDecode(failureJson)),
);
} else {
throw Exception('No mocked request found for "${options.path}"');
}
}
}Mocking Monitoring
In the previous section, we covered Monitoring with unit tests, so we are sure that this class behaves correctly. As it is an external dependency for feature tests, we will create a mock for Monitoring so that its implementation not affects feature tests.
Create lib/src/mock.dart file:
import 'package:mocktail/mocktail.dart';
import 'package:monitoring/monitoring.dart';
class MockMonitoring extends Mock implements Monitoring {}
Monitoring createMockMonitoring() {
final mock = MockMonitoring();
when(() => mock.trackEvent(any())).thenAnswer((_) => Future.value());
return mock;
}Test DI Initializer
We use TideDIInitializer to register factories and objects in DI and prepare a feature or a module. We’ll do the same for tests.
Create the lib/src/test_container.dart file with the following content:
import 'dart:async';
import 'package:dio/dio.dart';
import 'package:get_it/get_it.dart';
import 'package:test_utils/src/feature_test_interceptor.dart';
import 'package:test_utils/src/mock.dart';
import 'package:tide_di/tide_di.dart';
class TestContainer extends TideDIInitializer {
const TestContainer() : super(_init);
}
FutureOr<GetIt> _init(GetIt getIt, String? environment) {
final featureTestInterceptor = FeatureTestInterceptor();
getIt.registerSingleton(featureTestInterceptor);
getIt.registerSingleton(_createTestDio(featureTestInterceptor));
getIt.registerSingleton(createMockMonitoring());
return getIt;
}
Dio _createTestDio(FeatureTestInterceptor interceptor) {
final dio = Dio();
dio.interceptors.add(interceptor);
return dio;
}Flow Initializer
We are nearly done with preparations. Let’s create a helper function that will prepare the DI, wrap the flow we are going to test in a dummy Material App, and pump the flow’s widget in widget tests. Create lib/src/init_flow.dart:
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:test_utils/src/test_container.dart';
import 'package:tide_di/tide_di.dart';
Future<void> initFlow(
WidgetTester tester,
Widget Function() flowBuilder,
) async {
await initializeDIContainer([TestContainer()]);
final app = MaterialApp(home: flowBuilder());
await tester.pumpWidget(app);
}There is a problem with this implementation, the DI state will preserve in between test invocations. We may either fix this in the initFlow or modify the initializeDIContainer function. Let’s reset the container during the DI initialization and change the function from the tide_di package:
...
Future<DIContainer> initializeDIContainer(
List<TideDIInitializer> initializers,
) async {
await GetIt.instance.reset();
...Barrel File
We will interact with the networking interceptor and initialize the flow from feature tests. To make this code visible from outside of the package, create the following barrel file lib/test_utils.dart:
export 'package:test_utils/src/feature_test_interceptor.dart';
export 'package:test_utils/src/init_flow.dart';Feature Testing
It’s time to start testing the feature. Update feature/marvel_characters/pubspec.yaml:
dev_dependencies:
bdd_widget_test: ^1.7.4
flutter_test:
sdk: flutter
mocktail: ^1.0.4
test_utils:
path: ../../utility/test_utils
...Testing Data
First, we need to prepare testing data. Go to https://developer.marvel.com/docs, expand the /v1/public/characters, and click theTry it out! button. It is important to get the sample data from outside of the implementation (API specification, Swagger, samples, etc.), otherwise your tests will test what you have written and not what you were supposed to write.
Create a new file test/util/data.dart with the following content:
final error = '{}';
final listOfCharacters = r'''
{
"code": 200,
"status": "Ok",
...
''';Put the output from the documentation in the listOfCharacters variable. Notice r in the r''' declaration. Without r the output will not be parsed correctly.
Feature File
Create a file with tests test/feature/marvel_characters.feature with the following content:
Feature: Marvel Characters
Background:
Given I am on the Marvel characters page
Scenario: The Marvel characters flow should display a list of Marvel characters
When backend responds with {'list of characters'}
Then I see a list of Marvel characters
Scenario: The Marvel characters flow should display an error message when the backend fails
When backend responds with {'error'}
Then I see an error messageNotice that it is written in Gherkin. Usually, it is positioned as a language that can be readable by both business and engineering. While it is indeed awesome that your PMs, POs, designers, QAs, and other team members can understand the testing specification and check whether we are testing the right things, we found that it is beneficial to use Gherkin even when developers are the only “users” of the testing platform. Gherkin aligns the team and provides answers to questions you may get from less senior colleagues, like: “how to structure tests properly?”, “when to extract code?”, “where to extract code?”, and many others. The usual answer to most such questions is — “if you were writing tests in plain English, how would you structure the specification?”. With Gherking, we ask engineers to actually write tests in plain English, which simplifies readability and increases maintainability.
Run code generation with the dart run build_runner build --delete-conflicting-outputs. Several files were generated for you. Let’s change their content.
Updating Step Files
First of all, we need to tell our testing suite how to initialize the flow. Update the test/feature/step/i_am_on_the_marvel_characters_page.dart file like that:
import 'package:flutter_test/flutter_test.dart';
import 'package:marvel_characters/src/flow/marvel_characters_flow.dart';
import 'package:test_utils/test_utils.dart';
/// Usage: I am on the Marvel characters page
Future<void> iAmOnTheMarvelCharactersPage(WidgetTester tester) async {
await initFlow(tester, () => MarvelCharactersFlow());
}Now we need to mock the BE. Update the test/feature/step/backend_responds.dart file:
import 'package:flutter_test/flutter_test.dart';
import 'package:test_utils/test_utils.dart';
import 'package:tide_di/tide_di.dart';
import '../../util/data.dart';
/// Usage: backend responds with {value}
Future<void> backendRespondsWith(
WidgetTester tester,
String scenarioName,
) async {
const path = 'v1/public/characters';
final interceptor = diContainer<FeatureTestInterceptor>();
switch (scenarioName) {
case 'list of characters':
interceptor.mockRequest(path, listOfCharacters);
case 'error':
interceptor.mockFailure(path, error);
default:
throw ArgumentError('Invalid scenario name: $scenarioName');
}
await tester.pumpAndSettle();
}Notice the pumpAndSettle at the end of the function implementation. That’s not always what we need, however, in this simplified implementation, we want the test to start running after we configure the BE. Think of it as a convention we agreed on.
It’s time to work on the assertions. Update the test/feature/step/i_see_a_list_of_marvel_characters.dart file:
import 'package:bdd_widget_test/step/i_see_text.dart';
import 'package:flutter_test/flutter_test.dart';
/// Usage: I see a list of Marvel characters
Future<void> iSeeAListOfMarvelCharacters(WidgetTester tester) async {
await iSeeText(tester, '3-D Man');
}And the test/feature/step/a_see_an_error_message.dart file:
import 'package:bdd_widget_test/step/i_see_text.dart';
import 'package:flutter_test/flutter_test.dart';
/// Usage: I see an error message
Future<void> iSeeAnErrorMessage(WidgetTester tester) async {
await iSeeText(tester, 'DioException [unknown]: null\nError: {}');
}Problem With Images
Tests will fail if you run them now asImage.network will not be able to download images. There are multiple ways to solve the problem. For example, use mocktail_image_network or implement a custom HttpClient with HttpOverrides. Alternatively, you may use cached_network_image, which plays with tests nicely. As a bonus, Marvel characters avatars will be cached, improving the user experience. Add the following line to the pubspec.yaml file:
...
dependencies:
cached_network_image: ^3.4.1
...And replace the Image widget with CachedNetworkImage in marvel_characters_page.dart like that:
...
child: CachedNetworkImage(imageUrl: character.thumbnail.url),
...Run tests with the command:
flutter test --coverageTests should pass now.
Inspect what code is covered with tests. Follow the How to Generate and Analyze a Flutter Test Coverage Report in VSCode tutorial for details. Notice that most of the code in the package is covered now. With the sociable approach, we covered bloc, API, Repository, DI, and every other unimportant detail of the implementation. If you decide to replace bloc with signals, introduce use cases, get rid of retrofit, or make any other refactoring, your tests would check whether refactoring succeeded and the app works according to requirements. Tests are not driven by implementation but by requirements. That makes them useful.
Bonus exercises:
1. Inspect the feature file and the generated Dart file with tests. Make sure you understand how each line of the feature file is translated into Dart. Check the bdd_widget_test documentation for details.
2. Tests helped us identify that the error message could be more user-friendly. Change the test to the new expected behavior, make sure you have a failing test now, and then change the implementation to make the test pass.
3. In the i_see_a_list_of_marvel_characters.dart we check for the text on the screen. What are the pros and cons of this implementation? What other solution we could use to check that the list is loaded properly?
4. Add missing tests to the feature. Remember, you shouldn’t mock any internal classes, only external dependencies (via mocktail) and API responses (via interceptor).
5. Read about sociable and solitary tests. Why do you think we cover utilities with unit tests and features with sociable tests?
5. In features tests, we used theGiven-When-Then approach. Are there any differences between it and Arrange-Act-Assert from vanilla unit tests?
Testing Best Practices
- Test only classes that are exported from the package. Control the coverage for the unexported classes. If there is a class or a function that can not be tested via exported classes, delete it, it’s unused.
- Aim for 100% of coverage in the package. Set the 90% threshold on CI to fail the PR if the coverage is lower.
- Run tests as often as possible. Learn how to shorten the test feedback cycle.
- Keep tests fast. If the test suite takes more than a few seconds to run, the package has too much functionality, so split it.
About Tide
Founded in 2015 and launched in 2017, Tide is the leading business financial platform in the UK. Tide helps SMEs save time (and money) in the running of their businesses by not only offering business accounts and related banking services, but also a comprehensive set of highly usable and connected administrative solutions from invoicing to accounting. Tide has 600,000 SME members in the UK (more than 10% market share) and more than 275,000 SMEs in India. Tide has also been recognised with the Great Place to Work certification.
Tide has been funded by Anthemis, Apax Partners, Augmentum Fintech, Creandum, Salica Investments, Jigsaw, Latitude, LocalGlobe, SBI Group and Speedinvest, amongst others. It employs around 1,800 Tideans worldwide. Tide’s long-term ambition is to be the leading business financial platform globally.
