Golden testing using CachedNetworkImage
cached_network_image is a popular package to handle network images in a Flutter application, it provides:
- Cache on iOS, Android, and recently macOS,
- Placeholder widget and nice fade-out transition to the image,
- Progress download indicator for large images,
- Offline support (cached images can be displayed without an active connection).
Using it on my Flutter projects has greatly improved the user experience. On the first launch, the app displays the placeholders sized as close as possible to the final images to smoothen the transition when the images are loaded. Then, the subsequent launches are blazing-fast, thanks to the cache (no network requests).
However, such improvement doesn’t come for free, and this package introduces a slight technical overhead: the cached_network_image package relies on the popular flutter_cache_manager package (by the same author), which, in turn, relies on the yet-another-popular package (and Flutter Favorite) sqflite.
This is usually not a problem since this machinery is abstracted by the straight-forward API of the CachedNetworkImage widget… except when it comes to testing:
The MissingPluginException is the classic exception thrown when accessing to a native library (in our case, sqflite) in a unit test or widget test that doesn’t build any native counterparts.
Let’s see how to configure our test environment to correctly mock the cache mechanism used by the CachedNetworkImage widget and, as a bonus, configure some stubbing to return a test asset in place of the expected image, so we can generate a golden image of a widget that uses CachedNetworkImage.
Golden testing (a.k.a. visual regression testing) consists of rendering a widget into an image (the golden) and compared this image to the existing one for each subsequent test execution. Check this other article from Flutter Community for more info about golden testing:
Sandbox application
This application is a mono-page application listing some services such as https://placekitten.com that deliver images in any resolution declared in the URL itself:
The source code is available here.
The main widget is InsetCard: it is responsible for displaying the characteristics of the service as well as an example image. Moreover, its implementation is tricky and requires a double build cycle:
- the first one to get the intrinsic height of the
Insetwidget, - the second one to center this widget at the bottom edge of the
Backdropimage.
Finally, this widget has to be flexible and renders correctly with data of different lengths (short or long name/description, various amount of tags, …). Therefore it’s the perfect candidate for a widget test.
InsetCard widgetThe code of this widget is no interest here but can be found on the app repository. The only thing we need to be aware of is the usage of CachedNetworkImage within the Backdrop widget.
As we already know, the most basic golden test as below is failing because of the sqflite plugin.
Improve testability
Some native packages provide a stubbing API for usage in unit tests or widget tests (like shared_preferences) but this is not the case for sqflite. The cached_network_image package doesn’t provide anything either, however, it does allow injecting a custom BaseCacheManager via the cacheManager property! This class is responsible for the caching mechanism and the default implementation is the one who relies on sqflite, see DefaultCacheManager.
The first step then is to use a custom BaseCacheManager when running tests while the app still normally uses the DefaultCacheManager. This can be accomplished with get_it. This simple package is basically a singleton registry. It may don’t seem useful at first sight (after all, we can create plain Dart singletons) but it quickly becomes indispensable if you’d like to bring the test coverage to a decent level in your project.
We use get_it to register an instance of DefaultCacheManager in the project main.dart:
Then we can inject this singleton in the CachedNetworkImage widget:
By doing this (see the complete diff), the app works exactly the same, we haven’t changed how the CachedNetworkImage works, however, we now have the possibility to declare our own implementation of BaseCacheManager for the widget test, in its main function:
Writing the TestCacheManager
We follow the plan by creating a class that extends BaseCacheManager. This class needs a concrete implementation for the method getFilePath as well as a mandatory positional argument in the constructor. By using the IDE quick fixes, we can write the following without too much hassle:
This, however, leads to another error when executing the test. If not provided, the BaseCacheManager creates a CacheStore object used to wrap a CacheInfoRepository:
This abstract class is the interface between the BaseCacheManager and the storage. The default implementation is using sqflite and is responsible for the MissingPluginException. It means that we can finally get rid of this dependency by providing a dummy implementation of CacheInfoRepository.
Good news, this implementation already exists! Its name is NonStoringObjectProvider. It’s the one used to ensure flutter_cache_manager compatibility for Flutter Web. This library on the Web is basically a no-op (image caching on the Web is mostly handled by the browser) but most importantly: it doesn’t crash.
We can enhance the TestCacheManager to provide an instance of CacheStore which is, in turn, initialized with an instance of NonStoringObjectProvider :
Important note: During this step, we have imported files that are not exposed by the flutter_cache_manager package. By doing that, we introduced a coupling with the internals of the package, and therefore expose ourselves to breaking changes in future versions (changes that could not be considered as “breaking” in the version changelog or documentation since they don’t concern public API).
The last barrier that separates us from the green test result is an error due to an undisposed Timer:
A Timer is still pending even after the widget tree was disposed.
This is caused by the CacheStore that performs an asynchronous cache clean at some point in the test execution. By default, the WidgetTester asserts that all asynchronous resources must be properly closed at the end of the test. Because we don’t have control over this Timer, we can’t close it manually. We have to wrap the pumpWidget method with a runAsync method:
We finally have a successful test and a generated golden image!
Improve the golden image
There are indeed some aspects that require an inevitable amount of boilerplate when running golden tests. Let’s tackle them one by one:
Canvas size
The default canvas used to render the widget is 800x600. It is, by sheer coincidence, the same aspect ratio as the Backdrop widget. Therefore, the CachedNetworkImage placeholder takes the whole canvas.
We can configure a more real realistic value by running the following code on a device: print(MediaQuery.of(context).size);. In the case of the iPhone 11 simulator, it gives a resolution of 414x896.
Then we can write a utility method to configure the WidgetTester to use this resolution:
Font files
The most noticeable difference is the text lines. By default in widget tests, all the fonts are replaced by a test font named Ahem which renders all characters as a plain square. In the case of visual regression testing, it doesn’t suit our interest.
When using the MaterialApp widget, the default font is Roboto. We have to load it manually, as well as the Material icon font (the icon in the top right corner of the Inset):
Inset layout
Remember how the Inset widget is supposed to be centered around the bottom edge of the Backdrop? The golden image reveals it’s just aligned at the bottom.
This is because of an implementation detail of the InsetCard widget: it requires 2 layout passes to be correctly positioned: the first one to get the height of the Inset, and the second to add a margin equal to half the height on the Backdrop, to obtain the expected layout. Check the repository for the full implementation.
When calling pumpWidget, only the first pass is performed. For the second one, we have to call pump to trigger a second frame. To be less dependant of implementation details like this one (and more frequently used: animations), we can use pumpAndSettle, which actually means “trigger new frames until the app is idle”.
Let’s run the test command for another try:
This is the corresponding updated test, using the 2 methods previously implemented:
The golden image starts to look like the real widget. In addition to the missing backdrop image, we can notice another difference: the grey border below the Inset.
This is because of the Inset shadow: shadows are not guaranteed to be pixel-perfect every time they are rendered. So they are disabled by default in golden tests and replaced by solid borders, otherwise, it could lead to false positives when running the golden tests.
Note: it’s technically possible to force the activation of the shadows (for documentation purpose for instance) by doing:
Mock image asset
This leads us to the final problem we have to take care of: the backdrop image. In widget tests, we don’t want to be dependent on any remote server, that’s why the low-level Dart HttpClient is configured by default to return an error 400 on every request (the reason why our image is not loaded). It’s possible to mock this client and stub a custom response for a given request however it is highly verbose and cumbersome to set up.
By chance, we are already using a package, cached_network_image, whose sole purpose is to spare a network request when the image is found in the cache. In addition to that, it already implements the logic to fetch an image from the filesystem (sqflite is used to store the cached URLs and associated metadata, but the image file itself is stored in the device’s temporary folder).
It would be neat then if we could trick the BaseCacheManager by faking a cache hit and configuring it to return an image file from the test/assets directory!
Let’s start by saving an image in the test/assets folder:
By looking at the internal of the CachedNetworkImage component, we can see a call to the method getFileStream of the BaseCacheManager class.
If the returned Stream emits a valid FileResponse object, then the widget builds the Image and starts the fade-out transition from the placeholder.
In BaseCacheManager, this method contains the cache logic (look into the cache, then download the file if not found). We can bypass that logic by overriding the method in our TestCacheManager:
That’s it! The Stream emits an event and the CachedImageWidget interprets as a ready-to-use image and build the Image widget.
If you re-run the test now, unfortunately, the image is still missing. That’s because the Image widget loads the image in an asynchronous task and the WidgetTester can’t know when the loading is complete.
But thanks to Chun-Heng Tai from the Flutter team, a workaround exists, based on precacheImage method:
And voilà!
Wrap up
We now have a test to guarantee the UI will never change unintentionally. In addition to that, once the moderated amount of boilerplate has been written, we can multiply the test cases endlessly for different screen sizes, text factors, landscape mode (it doesn’t look good…), different lengths of data, etc.
If you’d like to know more about golden testing in production, eBay has used it extensively for their app eBay Motors and wrote interesting feedback in this blog post:
Feel free to share any suggestions or feedback in the comments and… happy testing!
Thank you to Rene Floor @FloorRene, main contributor of cached_network_image, for his review of this article.
