Flutter Demo by KBTG: Flutter Integration Test for Dummy

Flutter integration test is an end-to-end test for mobile, which is a part of the software development process. We usually divide the tests into three types:

1. Unit Test developers usually handle this by themselves
2. Integration Tests or API testing, is mostly handled by test automation teams
3. UI Tests is a real testing on real devices to see the final result if everything’s working correctly

Credit: https://medium.com/android-testing-daily/the-3-tiers-of-the-android-test-pyramid-c1211b359acd

The traditional way to group software tests is in the shape of a pyramid. However, lately I have read some articles that shows the test trend starting to shift to a diamond shape. I think it’s pretty interesting, though it‘s still being tested and not all software development parties use it. The idea is to focus more on the business side aka the integration than technical aka the unit test. Why do we need this? What’s the issue with the traditional way?

Credit: http://testerstories.com/2020/09/test-shapes/

The problem how costly the unit test is to maintain. The initial idea is that we write a unit tests to catch bugs before it reaches the testers, and since our app is getting bigger, it becomes harder to maintain. It doesn’t mean that we are going to write less unit tests though. We will still write it but not with 100% test coverage. Google suggests about 70: 20 : 10, which is the ratio for unit: API: UI

Just Say No to More End-to-End Tests

before we go off topic going… let’s get back to Flutter

Having done UI tests in Native iOS, I find it pretty cool and exciting. You can write the test by coding, add it to the Storyboard, or even use the record mode to save what you’re going to test without writing any code. It can do a lot of things that are not related to your app like opening Safari or sending SMS.

When I started writing Flutter, I tried to apply a lot of my past experiences from Swift to Flutter.

UI test in Flutter is... horrible

That was my sentiment when I compare it to Native iOS. Flutter has a lot of limitations for UI testing. Here’s a list of what I found…

1. Can’t interact with native components such as the permission pop-up. That means you can’t really test it end-to-end. When a pop-up asks for a notification, you can’t use Flutter UI test to close it. Instead, you have to hack around and make your app accept the notification in advance. For this, I give it a cry 😢
2. Can’t interact outside of Flutter. Unlike Native iOS, you can’t browse Safari or open a SMS.
3. Can’t do record mode, everything must work from coding. For this, I give it two cries 😢. why do I miss this? Well, in my iOS days, I can just teach the testers to do the UI test with record mode for me, then I just clean up the generated code from the record mode. 70–80% of the UI test is already handled by the testers, whereas in Flutter developers have to do it all by themselves.

What do you gain from losing all of these? Cross-platform UI testing, of course! 👍 It’s not about the development only, you can write code once and test across all platforms at once as well. The Flutter team put a nice tutorial here.

Integration testing

There are two types of Flutter UI tests, but don’t use Flutter driver. It’s old and deprecated. I went down that road and only found out that they have a newer version after implementing it 😿

An introduction to integration testing

After following their tutorial, I have one trick in my sleeves. It started off when my spinning animation during the loading process froze my app and couldn’t find the widget I wanted.

integration_test package pauses isolates and calling pumpAndSettle() never finishes. · Issue #73355…

Here’s the function that pumps and skips the animation until you find the widget. Please don’t use it too often, but use it only when you want to skip the loading animation because it’s lagging a bit as it keeps pumping until that widget’s found.

After I have spent a couple of weeks working on the UI test. Here’s my working demo. I also have a shell script to test the UI and write the report into the report.txt

Here’s the summary of what I got from the UI testing:

1. Add a key into the widget and browse by the key. I’m not quite sure if this is the right approach since some articles suggest that the key is not for testing, but it’s designed to keep the state of the widget. That said, I will follow the Flutter team as they show it in their official tutorial.
2. Using pumpAndSettle and pump after each time you do an action like tap, drag, or swipe. pumpAndSettle is used for non-animation, while pump is for animation.
3. PumpUntilFound is needed when you want to pump for a long period of time because animation is in progress all the time.
4. Keep checking your UI with expect as a checkpoint so that we can guarantee that our app properly transitions to another page, or some information is shown correctly. After you’ve done UI testing, I’m sure that you don’t want to keep monitoring it, right? That’s why we’re using expect to monitor it for us.

For the test, all you have to do is run bash ui-test.sh to make things work. You can check this demo and play with it, at least you’ll have a sample to study for.

theamorn/flutter-ui-test

Here’s my video for the test in a simulator.

Personally, it’s so much fun to write UI tests. I’m planning to implement the Flutter UI test with Robot Framework in a couple of months, so I’m hoping to come back to this article and update it. 👯‍♂

Want to read more stories like this? Or catch up with the latest trends in the technology world? Be sure to check out our website for more at www.kbtg.tech