Snapshot Testing. Testing the UI and Beyond (Part 3)
This is the final part of a posts series that aims to provide a complete overview of snapshot testing and its use cases, helping the developers understand what problems it can solve effectively and how to make it part of their software verification tools.
With part 1 focusing on describing how one should approach screenshot testing in mobile UI development, this final part illustrates how this can be done in practice. An iOS hands-on guide that I hope you will find useful.
Therefore, for a better understanding of the context, it is strongly recommended to read part 1 first before continuing.
P.S. Part 2 goes beyond UI testing and explores other interesting non-UI related applications. Backend people might find this interesting as well.
Before applying snapshot testing in our project, we need to define a device configuration matrix that lays out our strategy for testing against multiple device configurations (language, device size, theme, dynamic font size).
The device configuration matrix groups the configurations that one must test into two final configurations. A large one (common device size, light theme, and default font size) and a small one (small device size, dark theme, and XXXLarge font size). Let’s take a look at how this translates into code.
Device configuration matrix in practice
We will be using the core-testing¹ library, an iOS snapshot testing utility library that we created. It is built upon the excellent SnapshotTesting library provided by the PointFree team and extends it by providing structured snapshot configuration generation.
Testing screens
For testing how a UIViewContoller looks like when it covers the full screen, the library defines the following three public snapshot configuration generator functions.
SnapshotTestConfig.VC.large { config in ... }
SnapshotTestConfig.VC.small { config in ... }
SnapshotTestConfig.VC.all { config in ... }where “all” generates both large and small configurations (the closure is going to be called twice). These functions are defined here.
The assertion happens inside the closure using the configuration that it is passed, as can be seen in the example below.
You can check the DocumentHistoryViewController snapshot tests example in core-testing found here.
Testing views
Also for testing a UIView in isolation, the library defines the aforementioned three public snapshot configuration generator functions.
SnapshotTestConfig.View.large { config in ... }
SnapshotTestConfig.View.small { config in ... }
SnapshotTestConfig.View.all { config in ... }These generators are used for testing (full width, dynamic height) views (i.e. views that span the full width of the screen and their height is dynamically defined by its content), like a UITableViewCell. You can check the DocumentHistoryCell snapshot tests example in core-testing found here.
The library also provides the following two extra configuration generator functions for dealing with view specific scenarios that are not covered by the (full-width, dynamic height) configurations.
SnapshotTestConfig.View.fixed(fixedSize) { config in ... }
SnapshotTestConfig.View.free { config in ... }The fixed configuration generator is used for testing (fixed width, dynamic height), (dynamic width, fixed height), or (fixed width, fixed height) views. Similar to the “all” generator, two configurations as defined in the configuration matrix (the light theme — default font size, and the dark theme — XXXLarge font size) are generated. The only difference is that the size is fixed in one or both dimensions.
The six subviews of the following example should be tested using a (fixed width, dynamic height) configuration generator. We need to fix the width to a specific size (1/3 of the screen width and not the full width) and allow it to grow in height. The XXXLarge font size configuration will reveal if the content fits inside the view nicely.
The free configuration generator is used for (dynamic width, dynamic height) views, that take as much space as needed to accommodate for their content, like a custom UIButton. Similar to the “all” generator, two configurations as defined in the configuration matrix (the light theme — default font size, and the dark theme — XXXLarge font size) are generated. The only difference is that the size always adjusts to the growing content. You can check the SupportButton snapshot tests example in core-testing found here.
Testing how the view reacts to changes in the environment
While writing the snapshot test, if you instantiate the view outside of the configuration generator closure, the same instance of the view is tested in both configurations. Thus, as a nice side effect, when the configuration is changed, you also get to test how your view adapts to interface style (dark/light theme) and dynamic text size changes in the runtime.
Applying screenshot testing in practice
Let’s assume that we would like to use screenshot testing to develop and test the account info screen in our Forex trading app. The account info screen is the screen where the user can see his balance, profit and loss, and other account-related information.
This is how it should look like (for a demo account) according to the designs.
Let’s see how we apply step by step everything we learned in part 1 using the snapshot configurations generated by the core-testing library¹.
Testing all device configurations
We must always start by testing how the view looks like in all device configurations. We will be using the core-testing library to help us set up the screenshot tests. Here is how the test case looks like.
- Create a view model object (which is the dependency of the view that we would like to test)
- Configure the view with the view model (in this case it is a view controller since we want to test the whole screen)
- Generate the snapshot configurations (in this case we want to generate all device configurations for a view controller as defined in our configuration matrix)
- Snapshot the view rendered with the provided configuration
If it is the first time the test is run (or the test is run explicitly in record mode), it will save the snapshot as a reference on the disk. Otherwise, the test will assert that the reference snapshot and the new one match.
The following two reference screenshots are generated.
In this way, we can check how the view looks like in both large and small configuration and as you can see, there is a problem with small width devices (like iPhone SE) when users have selected larger dynamic font size.
Important: To avoid flakiness, please remember to always run all snapshot tests of your project against the same iPhone simulator (e.g. iPhone 11).
Testing domain cases
After fixing how the view looks like in the small configuration, we now should check how it looks like in all other domain cases. That means we should at least write tests for
- a demo user
- a real user that has been validated
- a real user that has not been validated
And since device configurations have been already checked, we can perhaps now test the domain cases against only the large configuration.
And this is how the view looks like for a real user (validated or not).
Discovering missing domain cases
If you want to make sure that you have covered all possible domain cases, then you can check which parts of the view UI code are not covered by the code coverage 😀. Needless to say that having clean views that only deal with the UI and have no business logic really helps here.
Take a look at the example below. Using code coverage report integration with Xcode, we can quickly spot that we are probably missing a case where an extra label is presented as a warning if the margin drops below 50%.
This seems quite important and as you can imagine it should be covered under a separate screenshot test.
Having covered all device configurations and domain UI cases, the view has been fully tested and more importantly future-proofed against UI regression errors.
This hands-on guide concludes the 3 part posts series and everything you need to know about snapshot testing. You can follow me on Twitter and LinkedIn for more.
References — Further reading
[1] Core testing library. This is a fork of the original we use in XM, as I wanted to adjust it to the content of this posts series. Feel free to play with both and open a PR for feedback.
