Adding unit tests to open-source Flutter projects for fun with AI.

Fun has many definitions, but for engineers, writing unit tests was definitely never one. Well, until now.

Hey there, if we haven’t met, I’m Samyak, the founder of Welltested AI, a testing autopilot designed for Flutter developers.

With Welltested, developers are adding tests to their existing codebases in just minutes. In this blog post, we are going to cover this journey by contributing unit tests to an open-source project while chit-chatting about benefits, challenges and tips about testing as we proceed.

Let’s get started by first cloning the FlutterEbookApp which is “A simple Flutter app to Read and Download eBooks” with a 2.2K starred repository by Flutter GDE Festus Olusegun.

From FlutterEbookApp Github Repository

The project is nicely designed to cover the UI for an EbookApp and is using provider as state management with this directory structure:

├─ lib
│  ├─ components
│  │  ├─ body_builder.dart
│  │  ├─ book.dart
│  │  ├─ book_card.dart
│  │  ├─ book_list_item.dart
│  │  ├─ custom_alert.dart
│  │  ├─ description_text.dart
│  │  ├─ download_alert.dart
│  │  ├─ error_widget.dart
│  │  └─ loading_widget.dart
│  ├─ database
│  │  ├─ download_helper.dart
│  │  ├─ favorite_helper.dart
│  │  └─ locator_helper.dart
│  ├─ main.dart
│  ├─ models
│  │  └─ category.dart
│  ├─ theme
│  │  └─ theme_config.dart
│  ├─ util
│  │  ├─ api.dart
│  │  ├─ consts.dart
│  │  ├─ dialogs.dart
│  │  ├─ enum
│  │  │  └─ api_request_status.dart
│  │  ├─ functions.dart
│  │  └─ router.dart
│  ├─ view_models
│  │  ├─ app_provider.dart
│  │  ├─ details_provider.dart
│  │  ├─ favorites_provider.dart
│  │  ├─ genre_provider.dart
│  │  └─ home_provider.dart
│  └─ views
│     ├─ details
│     │  └─ details.dart
│     ├─ downloads
│     │  └─ downloads.dart
│     ├─ explore
│     │  └─ explore.dart
│     ├─ favorites
│     │  └─ favorites.dart
│     ├─ genre
│     │  └─ genre.dart
│     ├─ home
│     │  └─ home.dart
│     ├─ main_screen.dart
│     ├─ settings
│     │  └─ settings.dart
│     └─ splash
│        └─ splash.dart

Since we are focused on unit testing, we can exclude all widget-related code and only focus on business logic which for us are view models, utils and database files. great!

First, let’s quickly set up the welltested package to get started:

1: Add welltested and other supporting dependencies:

flutter pub add welltested

flutter pub add mockito
flutter pub add --dev build_runner

2. Get our free Welltested API key from here: Signup Here and it to the .env file in the project root.

Specify the .env file as an asset in our pubspec.yaml

assets:
  - .env

That’s all! we’re ready to add unit tests.

The Welltested Annotation

To specify any class in the code to be tested, we just need to add the @Welltested() annotation to it. Let’s start with HomeProvider in view models.

Welltested Annotation at your service.

That’s it. That was all that was required.

Generate with welltested:ai build

Now to generate the unit tests for HomeProvider , we can go to our command line and run flutter pub run welltested:ai build and see the welltested magic happen.

HomeProvider has 7 functions and in about 7 mins we see the tests generated in the /tests folder, not bad. You’ll notice that tests for each method in HomeProvider have been added as a separate file. We believe this helps keep the tests more organized and easier to navigate.

We can see some syntax errors in the code, let’s look into them 🕵🏼

Fixing Generated Tests

Welltested takes a few shots to learn and get an understanding of your codebase. So we might need to do some fixes in the initals test outputs. Not to worry, they’re mostly simple syntax corrections. Let’s dive into some examples:

For instance, in thesetApiRequest function:

  void setApiRequestStatus(APIRequestStatus value) {
    apiRequestStatus = value;
    notifyListeners();
  }

we have two issues:

1. Welltested tried to create a mock of API class even though its not being used in the function.
2. APIRequestStatus enum doesn’t have a success value but a loaded value instead.

Let’s quickly make the fixes:

Once the syntax problems are resolved, we can run the tests by simply tapping the run option just above main() or using flutter test. And voila! All tests ran and passed 🎉

Context Understanding: We’re proud 😌

One thing that we focused the most on while building welltested was context understanding(knowledge of the complete codebase). As you can see, method checkError in HomeProvider calls another static method checkConnectionError which is defined in another file but is critical to writing funcitoning tests.

Welltested understood this and used checkConnectionError as the context while generating the tests. Hence you can see SocketException being used and mapped against a APIRequestStatus.connectionError .

One thing that also captures the essence of why we write tests in the third test “checkError with no error”.

It suggest that in the case with error as null, user should not be shown an API error and show the status as loading or loaded which isn’t rightly handled in the code and is an indication for us to improve to handle this scenario.

Great! moving on, we similarly fixed the other tests as well and have them all running here: +15 passed and 1 failed. Good Job 👏🏼

Self-Learning: Save the tests

Welltested is self-learning, which means that if the generates tests have any mistakes or doesn’t match your code style, you can manually make the fixes and save the tests with:

flutter pub run welltested:save build

This instructs the system to learn from the changes and not repeat the same mistake again.

Adding remaining coverage

Now that we have one good example representing our code style and testing pattern, let’s go ahead and generate tests for pending providers.

Add @Welltested annotation to AppProvider , GenreProvider , FavoritesProvider and DetailsProviders and run the flutter pub run welltested:ai build .

We have the test generated now but there are general issues appearing due to the code not following testability guidelines. Let’s look into how we fix them:

1. app_provider.dart method: checkTheme

Here SharedPreferences is instantiated within the function itself and hence cannot be mocked externally. Writing unit test for this will throw the exception

MissingPluginException(No implementation found for method getAll on channel plugins.flutter.io/shared_preferences)

  Future<ThemeData> checkTheme() async {
    SharedPreferences prefs = await SharedPreferences.getInstance();
    ThemeData t;
    String r = prefs.getString('theme') ?? 'light';

    if (r == 'light') {
      t = ThemeConfig.lightTheme;
      setTheme(ThemeConfig.lightTheme, 'light');
    } else {
      t = ThemeConfig.darkTheme;
      setTheme(ThemeConfig.darkTheme, 'dark');
    }

    return t;
  }

This makes sense because our unit test environment doesn’t support platform channels.

Fix: Get an instance of SharedPreferences from the constructor of the class which allows us to easily pass the mocked instance.

2. genre_provider.dart method: showToast

Uses the FlutterToast package which exposes a static method showToast. The ideal way to mock this for testing is to create a FlutterToast wrapper and override it’s value:

There were some other issues along the same lines and we’ve resolved then all except for one:

Testing methods in the DetailsProvider class is difficult because its methods are tightly coupled and often call each other. This makes it challenging to isolate and test a single method, as we also need to account for the behavior of the other methods being called, leading to complex and less maintainable test cases.

For this reason, we are excluding tests for it.

The Verdict

With tests all the other providers in place, let’s run them with flutter test 💪

00:16 +57 -6: Some tests failed.

Total Executtion time was 00:16 seconds with 57 passing and 6 failing tests. Almost 100% coverage for all the providers we’ve covered.

Haha, didn’t know testing can be so satisfactory. We’ve a PR live for the unit tests for everybody to review. Cheers!

Test Coverage: Providers by samyakkkk · Pull Request #94 · JideGuru/FlutterEbookApp

Welltested.ai - AI Autopilot for Testing