An Introduction to Flutter Application Testing

Developers remember about Flutter when they need to quickly make a beautiful and responsive application for several platforms, but how can you guarantee the quality of the so-called “fast” code?

You’ll be surprised, but Flutter has the tools to ensure not only the quality of the code, but also the performance of the visual interface.

In this article we will take a look at tests in Flutter and analyze widget testing as well as integration testing of a complete app.

I started studying Flutter over a year ago before its official release. Finding information on app development wasn’t a problem before I decided to try out TDD. It turned out that there was hardly anything on the topic, not to mention any sources in Russian. I had to learn about testing on my own using Flutter sources and scarce articles in English. I wrote down everything I learned about visual testing to help out those who are just delving into this subject.

Widget Testing

Overview

A widget test tests a single widget. It is also referred to as component test. The goal of a widget test is to verify that the widget’s UI looks and interacts as expected. Testing a widget requires a test environment that provides the appropriate widget lifecycle context.

For instance, the test widget is able to receive and respond to user actions and events, perform layout, and instantiate child widgets. Therefore, widget tests are more complex than unit tests. However, like a unit test, a widget test’s environment is just a simulation, much simpler than a full-blown UI system.

Widget testing allows you to isolate a single visual element and test its behavior and, what’s remarkable, to carry out all verifications in the console — perfect for tests started as a CI/CD process stage.

Files containing tests are usually located in the test subdirectory of the project.

Tests can be run either from the IDE or the console using the command:

$ flutter test

In this case, all tests with the mask *_test.dart will be executed from the test subdirectory. You can run a separate test by specifying a file name:

$ flutter test test/phone_screen_test.dart

The test is created by the testWidgets function, which acts as a tester parameter and therefore gets a tool that allows the test code to interact with the widget under test:

testWidgets(’Test name’, (WidgetTester tester) async {
    // Test code
  });

In order to combine tests into logical blocks, test functions can be combined into groups within the group function:

group(’Test group name’, (){
    testWidgets(’Test name’, (WidgetTester tester) async {
        // Test code
      });
    testWidgets(’Test name’, (WidgetTester tester) async {
        // Test code
      });
  });

The setUp and tearDown functions allow you to run some code “before” and “after” each test. Accordingly, the setUpAll and tearDownAll functions allow you to execute code “before” and “after” all tests, and if these functions are called within a group, they will be called “before” and “after” all tests of the group:

setUp(() {
    // Test initialization code
  });
tearDown(() {
    // Test completion code
  });

Finding widgets

In order to perform some actions on a nested widget, you need to find it in the widget tree. To do this, there is a global object find, which allows you to find widgets:

• in the tree by text — find.text, find.widgetWithText;
• based on the key — find.byKey;
• by icon — find.bylcon, find.widgetWithIcon;
• by type — find.byType;
• by position in the tree — find.descendant и find.ancestor;
• using the function that analyzes widgets on the list — find.byWidgetPredicate.

Interaction with the test widget

The WidgetTester provides the functionality to create a widget to test, to wait for its change of state and to perform some actions on those widgets.

Any change to the widget causes its state to change. But the test environment doesn’t rebuild the widget at the same time. You need to indicate to the test environment that it is required to rebuild the widget by calling the functions pump or pumpAndSettle.

• pumpWidget — create a widget to test;
• pump — start the process of changing a widget’s state and wait for its completion with the given duration (100 ms by default);
• pumpAndSettle — repeatedly call pump in a state change cycle with the given duration (100 ms by default) and wait for all animations to complete;
• tap — send a tap to the widget;
• longPress — send a long press;
• fling — a fling/swipe;
• drag — dragging;
• enterText — enter text into a TextField.

Tests can implement either positive scenarios by checking scheduled opportunities, or negative ones to make sure they don’t lead to fatal consequences, e.g. when the user taps in a wrong place and enters wrong data:

await tester.enterText(find.byKey(Key(’phoneField’)), ’bla-bla-bla’);

After performing any actions with widgets you need to call tester.pumpAndSettle() to change states.

Mocks

Many developers are familiar with the Mockito library. This library from the world of Java was so successful that there are now implementations of it in lots of programming languages, including Dart.

To use mockito you need to add a dependency to the project. Add the following lines to the pubspec.yaml:

dependencies:
  mockito: any

And connect it in the file with tests:

import ’package:mockito/mockito.dart’;

This library allows you to create mock classes on which the test widget depends, so that the test is simpler and only covers the code we are currently testing.

For example, if we are testing the PhoneInputScreen widget which makes a request to the back-end authInteractor.checkAccess() using the AuthInteractor service, we can place a mock instead of the service to check whether we can access this service at all.

Dependency mocks are created as inheritors of the Mock class and implement the dependency interface:

class AuthInteractorMock extends Mock implements AuthInteractor {}

A class in Dart is also an interface, so there is no need to declare the interface separately, as in some other programming languages.

We use the when function to determine the functionality of a mock which allows you to determine the mock’s response to a call of any given function:

when(
  authInteractor.checkAccess(any),
).thenAnswer((_) => Future.value(true));

Mocks can return errors or invalid data:

when(
  authInteractor.checkAccess(any),
).thenAnswer((_) => Future.error(UnknownHttpStatusCode(null)));

Verifications

As you run the test, you can verify the presence of widgets on the screen. This allows you to make sure that the new state of the screen is correct in terms of visibility of the desired widgets:

expect(find.text(’Phone number’), findsOneWidget);
expect(find.text(’SMS verification code’), findsNothing);

After running the test, you can also verify which methods of the mock class had been called during the test, and how many times. This is crucial, for example, to understand whether certain data is requested too often, or whether there are some unnecessary changes in the application state:

verify(appComponent.authInteractor).called(1);
verify(authInteractor.checkAccess(any)).called(1);
verifyNever(appComponent.profilelnteractor);

Debugging

Tests run in the console without any graphics. You can run tests in debug mode and set breakpoints in the widget code.

To get an idea of what’s going on in the widget tree, you can use the debugDumpApp() function, which, when called in test code, provides a textual representation of the entire widget tree hierarchy to the console at a given time.

Use the logInvocations() function to gain an understanding of how the widget uses mocks. It takes a list of mocks as a parameter and outputs a sequence of method calls to the console that were performed in the test.

Here’s an example of such output. Calls that were verified in the test using the verify function have the VERIFIED mark:

AppComponentMock.sessionChangedInteractor
[VERIFIED] AppComponentMock.authInteractor
[VERIFIED] AuthInteractorMock.checkAccess(71111111111)

Initiation

All dependencies should be supplied to the test widget in the form of a mock.

class SomeComponentMock extends Mock implements SomeComponent {}
class AuthInteractorMock extends Mock implements AuthInteractor {}

Adding dependencies to a test component has to be carried out in a way accepted by your application. For the sake of simplicity let’s consider an example where dependencies are added through a constructor.

In this example PhoneInputScreen is the widget to test, it is based on StatefulWidget and wrapped in Scaffold. It is created in the test environment with the pumpWidget() function.

await tester.pumpWidget(PhoneInputScreen(mock));

However, an actual widget can use alignment for nested widgets which requires MediaQuery in the widget tree and maybe Navigator.of(context) as well for navigation, therefore it is more practical to wrap the test widget in MaterialApp or CupertinoApp:

await tester.pumpWidget(
  MaterialApp(
    home: PhoneInputScreen(mock),
  ),
);

After creating a test widget and after any actions with it, you need to call tester.pumpAndSettle() to ensure that the test environment has processed all of the widget’s states.

Integration tests

Overview

Unlike widget tests, an integration test verifies the entire application or a large part of it. The purpose of the integration test is to ensure that all widgets and services work together as expected. The integration test can be observed in the simulator or on the device screen. This method is a great substitute for manual testing. You can also use integration tests to test the performance of your application.

An integration test is usually performed on a real device or emulator, such as iOS Simulator or Android Emulator.

As a rule, files containing integration tests are located in the test_driver subdirectory of the project.

The application is isolated from the test driver code and runs after it. The test driver allows you to control the application during the test. It looks like this:

import ‘package:flutter_driver/driver_extension.dart’;
import ‘package:app_package_name/main.dart’ as app;
void main() {
  enableFlutterDriverExtension();
  app.main();
}

You can run the tests from the command prompt. If the launch of the target application is described in the app.dart file and the test script is called app_test.dart, the following command will be sufficient:

$ flutter drive target=test_driver/app.dart

If the test script has a different name, you need to specify it explicitly:

$ flutter drive --target=test_driver/app.dart --driver=test_driver/home_test.dart

The test is created using the test function and grouped by the group function.

group(’park-flutter app’, () {
    // driver used to connect to the device 
    FlutterDriver driver;
    // create a connection to the driver 
    setUpAll(() async {
      driver = await FlutterDriver.connect();
    });
    // close the connection to the driver
    tearDownAll(() async {
      if (driver != null) {
        driver.close();
      }
    });
    test(’Test name’, () async {
      // test code 
    });
    test(’Other text’, () async {
      // test code
    });
});

This example shows the code for creating a test driver which allows tests to interact with the application under test.

Interaction with the app under test

The FlutterDriver tool interacts with the application under test via the following methods:

• tap — send a tap to the widget;
• waitFor — wait for the widget to appear on the screen;
• waitForAbsent — wait for the widget to disappear;
• scroll and scrollIntoView, scrollUntilVisible — scroll the screen by the given offset or to the required widget;
• enterText, getText — enter text or use the widget’s text;
• screenshot — get a screenshot of the screen;
• requestData — a more complex interaction via a function call within the application.

You might want to influence the global state of the application from the test code. For example, to simplify the integration test by replacing part of the services within the application with mocks. You can also specify a request handler that can be accessed via driver.requestData(‘some param’) in the test code:

void main() {
  Future<String> dataHandler(String msg) async {
    if (msg == “some param”) {
      // some call handling in the application
      return ’some result’;
    }
  }
  enableFlutterDriverExtension(handler: dataHandler);
  app.main();
}

Finding widgets

Finding widgets in integration testing using the global object find differs from similar functionality in widget testing in the composition of methods. However, the overall meaning remains virtually the same:

• in the tree by text — find.text, find.widgetWithText;
• based on the key — find.byValueKey;
• by type — find.byType;
• by tip — find.byTooltip;
• by semantics label — find.bySemanticsLabel;
• by position in the tree — find.descendant and find.ancestor.

Summary

We’ve looked at the ways to test an application interface written on Flutter. We can implement tests to verify the compliance of the code with the requirements described in the technical task as well as make tests this very task. One of the drawbacks of integration testing I’ve noticed is that you can’t interact with the dialogue systems of the platform. But at the same time you can, say, avoid permission requests by giving permissions from the command prompt during the installation step, as described in this ticket.

This article is a starting point for exploring the topic of testing. It gives the reader an idea about how UI testing works. It won’t spare you the trouble of reading documentation, though. But from it you can easily learn about how a certain class or method functions. After all, while exploring a new topic you only need a general understanding of the processes without going into too many details.

Translation Alisa Zdorovik