Hot Reload For Flutter Integration Tests

A How-to

Published in

Flutter Community

7 min readDec 10, 2018

Edit: updated to use --disable-service-auth-code (for local testing only, not for use in automated scenarios due to possible security risk).

You’ve developed an amazing app, curated the user experience, and now want to unleash it onto the world. It’s important to make sure the user experience is the same on all devices as you continue to make improvements to your app. This, among other places, is where integration testing comes into play.

Developing integration and end-to-end tests with Flutter may appear to have minimal tooling support. For example, running an integration test often requires running a build and install on each test iteration. As a result, each test-iteration can take several minutes.

For most flutter developers, who are already accustomed to very rapid development iterations, this can be something of a slow experience:

$ flutter driver
Using device Android SDK built for x86.
Starting application: lib/main.dart
Initializing gradle...                                       0.8s
Resolving dependencies...                                    0.9s
Installing build/app/outputs/apk/app.apk...                  2.1s
Gradle task 'assembleDebug'...
Gradle task 'assembleDebug'... Done                          7.9s
Built build/app/outputs/apk/debug/app-debug.apk.
Installing build/app/outputs/apk/app.apk...                  1.8s
I/flutter (16870): Observatory listening on http://127.0.0.1:33407/
00:00 +0: app test:  (setUpAll)
[info ] FlutterDriver: Connecting to Flutter application at http://127.0.0.1:50244/
[trace] FlutterDriver: Isolate found with number: 980848669
[trace] FlutterDriver: Isolate is paused at start.
[trace] FlutterDriver: Attempting to resume isolate
[trace] FlutterDriver: Waiting for service extension
[info ] FlutterDriver: Connected to Flutter application.
00:01 +0: end-to-end test tap on the floating action button; verify counter
00:03 +1: end-to-end test (tearDownAll)
00:03 +1: All tests passed!
Stopping application instance.
$

What if it were possible to avoid having to rebuild and re-install the flutter app each time you ran an integration test? Wouldn’t it be nice if you could do a hot-reload on the flutter app and also use the debugger on both the app and the integration test?

There is a way to do all that! What’s more, it dramatically improves the time required to develop integration tests…. and it works from both the command line and an IDE!

Integration testing with flutter driver

Before showing you how to configure tooling for integration test development, let me first explain how flutter driver works.

flutter driver is a standalone test harness for integration testing. It makes calls to the existing tooling to process your source code. It does a set-up of the test harness, builds and installs your app, and runs your integration test on your app.

Running an integration test using flutter driver automatically starts two processes from your current source code.

The flutter app is started in integration-test-mode in it’s own process on the emulator or device.
The integration test that connects to the app (a pure dart program) is started in it’s own process on your computer.
Both processes discover each other by negotiating a shared port.
The integration test runs and completes, and both processes are stopped.

The app and/or test are then typically modified, and flutter driver is called again, with a new build and a new install. Multiple calls to flutter driver are repeated in this way until test development is complete. As mentioned, this can be a time consuming process.

Integration testing with `tooling`

However, the main purpose that flutter driver serves during integration test development is simply to connect the test to the app using a negotiated port. You can therefore by-pass flutter driver altogether during test development and use the existing tooling directly. All that is required is to pick an available port and configure the app and the test to use that port.

By using a pre-configured port in this way, you get access to all the tooling already available to Flutter developers. This allows you to hot-restart your flutter app, and rerun your integration test as many times as needed without having to rebuild and re-install the app. Plus you now have access to the debugger for both the app and the test.

During integration test development, direct access to flutter tooling can increase your productivity by as much as 10x (by my estimate) compared to using flutter driver.

Configuring for hot-reload

Picking-up this productivity gain is straightforward. I will show you how to configure from the command line and then give you a more detailed example of how to configure in an IDE.

The sample code I used in this how-to can be auto-generated from the command line using:

flutter create --with-driver-test flutter_app

Note: this command adds the flutter driver extension to lib/main.dart. In practice, the extension should be in a separate file, usually test_driver/main.dart, that calls the lib/main.dart. (See https://github.com/mmcc007/flutter_app for example.)

Configuring from the command line:

Pick an available port. In this case we will use port 8888

In one window run:

flutter run --observatory-port 8888 --disable-service-auth-codes lib/main.dart

This produces the following output if you have an android emulator running:

$ flutter run --observatory-port 8888 --disable-service-auth-codes lib/main.dart
Using hardware rendering with device Android SDK built for x86. If you get graphics artifacts, consider enabling software rendering with "--enable-software-rendering".
Launching lib/main.dart on Android SDK built for x86 in debug mode...
Initializing gradle...                                       0.7s
Resolving dependencies...                                    0.9s
Gradle task 'assembleDebug'...
Gradle task 'assembleDebug'... Done                          3.4s
Built build/app/outputs/apk/debug/app-debug.apk.
Installing build/app/outputs/apk/app.apk...                  1.9s
Syncing files to device Android SDK built for x86...
D/        (16530): HostConnection::get() New Host Connection established 0x92a7f040, tid 16552
D/EGL_emulation(16530): eglMakeCurrent: 0xaa088a00: ver 3 0 (tinfo 0x96e46f10)
 2.2s🔥  To hot reload changes while running, press "r". To hot restart (and rebuild state), press "R".
An Observatory debugger and profiler on Android SDK built for x86 is available at: http://127.0.0.1:8888/
For a more detailed help message, press "h". To detach, press "d"; to quit, press "q".

2. In another window run:

$ export VM_SERVICE_URL=http://127.0.0.1:8888/
$ dart test_driver/main_test.dart

This produces the following output:

$ export VM_SERVICE_URL=http://127.0.0.1:8888/
$ dart test_driver/main_test.dart
00:00 +0: app test:  (setUpAll)
[info ] FlutterDriver: Connecting to Flutter application at http://127.0.0.1:8888/
[trace] FlutterDriver: Isolate found with number: 448955887
[trace] FlutterDriver: Isolate is not paused. Assuming application is ready.
[info ] FlutterDriver: Connected to Flutter application.
00:01 +0: end-to-end test tap on the floating action button; verify counter
00:03 +1: end-to-end test (tearDownAll)
00:03 +1: All tests passed!
Stopping application instance.
$

You can now modify the app (followed by a hot-reload/hot-restart) and/or modify the test and re-run the test as often as is necessary.

Configuring from an IDE (in this case Android Studio):

The following is the IDE equivalent to the command line configuration with the added bonus of easy access to the debugger for both the app and the test.

Configure the app to listen on a shared port:

Add ‘ — observatory-port 8888 **— disable-service-auth-codes**’ to ‘Additional Arguments’ (note: ‘— ’ should be 2 dashes with no spaces)

2. Configure the integration test to connect on the same shared port:

Add ‘VM_SERVICE_URL=http://127.0.0.1:8888/’ to ‘Environment Variables’

3. Start the app in run or debug-mode (only required once, with hot-reload when needed).

In this case we are using an iOS simulator. We could also run the app on an android emulator, or an android or iOS device:

Start in run or debug mode, modify app, and hot-reload/hot-restart as many times as you want

4. Start the integration test in run or debug mode:

Start in run or debug mode, modify integration test, and re-run as many times as you want

You can now hot-reload or hot-restart the app and re-run the integration test as necessary and avoid having to rebuild and re-install the app on each test-run. You can change code in both the app and test and re-run as often as you want. Plus you can also use the debugger in both the app and the integration test simultaneously!

Let’s see it in action!

App is in debug mode and can be modified and then hot-reloaded or hot-restarted.

Integration test is in debug mode and can be modified and rerun as many times as necessary.

Feel free to set breakpoints in the app and/or in the test to examine the respective source code and variables.

Conclusion

Developing integration and end-to-end tests using flutter tooling can result in very fast test development iterations. This opens-up a whole new strategy of incorporating integration testing into the early stages of your development and delivery process.

For using integration tests to monitor changes to your UI and for uploading screenshots to Google and Apple stores, see Automated Screenshots for Flutter.

Finally…

Let me know if you have any questions or comments about this method of integration test development below or on GitHub or Twitter.

If you think this article is useful, clap, like, subscribe and share with your friends for more Flutter developer articles.