Key learnings from Flutter Driver
I’ve been exploring Flutter’s test framework Flutter driver since a while and wrote about it here and here on how we can use it to test various UI elements we see on the screen that helps us to write end to end integration tests.
I recently wrote integration tests using Flutter driver for app I am working on and although most of the scenarios were straight-forward and fairly easy to implement to validate UI, I learned a lot of new things along the way and that will be the focus of today’s article.
Following sections are purely based on my findings and experiences using flutter driver.
What worked well
A) The setup to start using flutter driver is very easy and minimal. You just need to perform following to get going:
- Add dependency in
pubspec.yaml - Import required dart files
- Create
test_driverfolder along with required test files inside it (ex :app.dartandapp_test.dart).
B) If you have used Google Espresso test framework for writing UI tests for Android apps, you might know a very first pre-requisite before starting test execution, which is to turn-off animations as shown below to avoid flakiness of tests on Android test devices.
Since flutter driver tests run in isolation and are not linked with the main app, we don’t need to turn-off animations before running tests. This saves a lot of logistical and manual work before we could start test execution.
Case in point: Imagine we have 20 test devices on a cloud farm (Sauce labs or AWS) on which we run our tests, although a one-time setup, we would have to manually turn-off animations on each device. We don’t have to worry about this when we are using flutter driver.
C) Test scenarios can be easily split into multiple files. This helps to structure our tests based on screens we are testing and also helps to improve the readability of tests. New team members or members of other teams can utilize this scenario to help them better understand the scope and navigate through the scenarios as they like.
Case in point: app.dart which enables flutter driver extension and calls main.dart is nothing but the target app. The test file which has actual test scenarios app_test.dart is the driver of the app which kicks off the target app. So, based on this structure, we can have as many drivers as we want that ultimately points to single app.dart .
For a login screen, we can have test scenarios in app_test.dart . For home screen, we can have test scenarios in home_test.dart and then we can tell flutter driver to execute scenarios from these two files using below command:
flutter drive — target=test_driver/app.dart — driver=test_driver/app_test.dart
flutter drive — target=test_driver/app.dart — driver=test_driver/home_test.dart
Although the terminal will take and execute these commands one after other and not at once, a better way would be to find a mechanism to automate these commands maybe using a test configuration file or using a shell script to avoid manually running these commands.
D) Almost all UI widgets seen on screen can be tested using 5 types of identifiers provided by flutter driver. Along with the widgets I mentioned in my previous articles, we can also test functioning / behavior of following in integration tests:
- BottomNavigatonBar
- TabBar
- Toggle Button (Switch)
- Calendar / Date picker
- Radio Button
- DropdownButton
- CheckBox
Apart from out-of-box identifiers, we can write custom identifiers to validate some specific scenarios such as:
- checking if a particular widget is present or visible on the screen, before performing an action on it.
- clearing out textfield to input new text. There’s no
clearText()the method, but we can input an empty string usingenterText()and achieve this behavior.
Tips & Tricks
TextFormFieldwith hintText,IconButtonsare some of the widgets that need to havekeyproperty to be able to be identified for tests usingbyValueKeyidentifier. Thetextvalue these widgets comes with were not enough to be uniquely identify them.
2. For those widgets, whose primary property is text such as RaisedButton and Text, it’s a good practice to identify these widgets using corresponding text value.
3. Whenever app interacts with an api (ex: login button tap for authentication), our tests need to wait till api returns with a response. For that, we use waitFor whose default timeout is 5 seconds, but depending on internet speed or api behavior, the tests might have to wait for more than 5 seconds and as a result, tests may fail to result in stopping in execution.
In order to negate this case, we can leverage timeout parameter that waitFor provides which allows us to provide a custom timeout period, so that will allow api to come back with a response and our tests can continue execution with next steps.
4. If a screen has a formField and displays inline error messages upon incomplete input, we can validate these inline error messages directly using the text value. This way we can minimize the usage of key property for each widget and can simply make use of text the property for validations.
What could be improved
More Identifiers !!
Currently, flutter driver provides 5 identifiers using which we need to write test scenarios. This limits the way widgets on screens are identified and hence scope of writing exhaustive and effective tests becomes a bit limited.
What we would like to see is Flutter driver providing various other type of identifiers based on UI properties and objects to match widgets that will greatly make writing integration tests more effective and will broaden scope as well as test coverage to the maximum.
Below are some of the matchers Espresso test framework provides. If similar types are introduced in flutter driver framework, it will only help in writing more concise tests.
isDisplayed(), isVisible(), isClickable(), isEnabled(), isChecked(), isNotChecked(), withHint(), startsWith(string), endsWith(string)
What we can’t test with Flutter driver (as of today) :
- Google or social media sign-in interactions. Source.
2. Native iOS widgets such as dialog after google sign in / Facebook sign in and runtime permission dialogs (ex: while accessing the camera). Same source as above.
3. There is no support to find RichText a widget. Source.
Issues found during this period
I found this issue wherein once flutter driver completes test execution and stops application instance, the textformfield widget input in the target app doesn’t work.
More areas to explore
- Testing animations is one area I haven’t yet got a chance to explore. I am not sure if testing this would fall under integration tests or would be more of a WidgetTest. If anybody has done any research or have any leads, I would be glad to learn more about it.
2. Localization is another important part of app development as well as testing since many apps are available in local languages. It’ll be interesting to get to know the approach and scope of using flutter driver to test this aspect of the app.
These were the key takeaways/learnings I experienced using flutter driver. If anybody would like to share their experience using this framework, I would be happy to know about it.
Also, feel free to comment below your thoughts/comments/questions you might have and I’ll be glad to answer per my ability. If you would like to get in touch with me, I am available on Twitter, LinkedIn, and Github.
Thanks for reading.
My other articles on Flutter are:
