Inline Snapshot Testing
… will automatically write your tests for you, with better code coverage and help your maintainability over time, with just one neat trick!
- Example Code: bitbucket.org/rjchatfield/InlineSnapshotTesting
- Video: vimeo.com/326120139
- Slides: slideshare.net/rjchatfield/inline-snapshot-testing-rob-chatfield-sydney-cocoaheads-march-2019
Kudos to the guys at pointfree.co for making the amazing SnapshotTesting open source library: github.com/pointfreeco/swift-snapshot-testing
Snapshot testing has been around for a while. Facebook popularised Visual Regressions a few years ago with github.com/uber/ios-snapshot-test-case (now maintained by Uber). Instead of testing every ivar of every object, we could test the whole view all at once. Brilliant! However, filling up your git repo with binary files that don’t diff just doesn’t scale.
Next was “snapshotting text-based descriptions”. Game changing! Why be limited to views, when you test literally anything.
Today, we’re going walk through a first principles look at snapshot testing, then introduce Pointfree’s library, then smoosh the two ideas together until we get inline snapshot testing. I’ll also share some tidbits about what we’ve learnt along the way.
1. First Principles: Snapshot Testing 101
“Do we really need a library?”
Let’s start with a simple text-based snapshot test case using first principles.
// Arrange
let object = Object()// Act
object.update(foo: "foo string")// Assert
let testableDescription = customDescription(of: object)
XTCAssert(testableDescription, """
Object
- foo: “foo string”
""")
Here we have the classic Arrange/Act/Assert in action, and we’re checking that the object’s description matches our expectation. Since we’ll use this technique a lot, let’s make a cute little helper function that does this assertion for us:
assertSnapshot(of: object, as: customDescription, is: """
Object
- foo: "foo string"
""")🤓 Dev Happiness Checklist
✅ Simple one-liner
✅ Git diff is descriptive
✅ Expectation is co-located with test
✅ Doesn’t rely on File Path
😖 Error message sucks
😖 No “record” feature
My two issues with this “first principles” approach is that the error messages suck, and you can’t just “record” the text string. So if this is any longer than the two lines above, you’ll have a hard time keeping these maintained (or even using it in the first place). Let’s see what we can do about that…
2. PointFree: Object -> Text
“How about a library?”
The boys at pointfree.co have open sourced a SnapshotTesting library that will help us.
If you want to learn more, check out their free episode: https://www.pointfree.co/episodes/ep41-a-tour-of-snapshot-testing
let view = MyView()
view.update(foo: "foo string")assertSnapshot(of: view, as: .recursiveDescription)
assertSnapshot(of: view, as: .image)
In 4 lines, we’ve generated a text-based snapshot and a PNG-based snapshot. Super awesome!
🤓 Dev Happiness Checklist
✅ Simple one-liner
✅ Git diff is descriptive
✅ Error is descriptive
✅ Re-run and re-record is easy
😖 Expectation is not co-located with source code
😖 Relies on FilePath
It’s great that we can now record and re-record our snapshots, but now we have a slightly weirder issue: we can’t see the assertion! While this technique is totally sufficient for catching regressions, we have found that it is a little annoying bouncing back and forth between source code and external files.
Another problem specific for us was around using a FilePath. When the app is compiled, it uses #file and #function to generate the path to the snapshot files. However, during CI we compile our tests on one Build Agent, and then farm out the compiled binary to 8 other Test Agents to run a subset of tests. So unfortunately we can’t depend on our Build Agent’s FilePaths in our Test Agent’s, and thus, our snapshot tests were failing. While it isn’t ideal, we now add each snapshot file to the Test’s ResourceBundle.
3. Recordable Inline Snapshot
“Can it write the test for me?”
Now it’s time to got back to our “first principles” example but now leverage PointFree’s .dump, and inherit all the goodness that it brings.
let object = Object()
object.update(foo: "foo string")
assertInlineSnapshot(of: object, as: .dump, matches: """
Object
- foo: "foo string"
""")What we’ve also managed to ship is an “auto-recording” feature. By providing an empty string literal, or setting record = true, our test framework will render the String data to your source code file right where it needs to be.
🤓 Dev Happiness Checklist
✅ Simple one-liner
✅ Git diff is descriptive
✅ Error is descriptive
✅ Re-run and re-record is easy
✅ Doesn’t rely on File Path
What else can I test?
The simple answer is “Anything you can print!” Things we’ve experimented with:
- Views with Subviews
- TableView and CollectionView hierarchies
- Logs
- Analytics
- Network Requests
- View State (aka. ViewModels)
- JSON Decodable objects
- Streams of values over time (as an array)
- Hybrid components that might have HTML or React, they are perfect for Snapshot testing
One of the hidden benefits of this technique is that new devs can onboard very easily. No matter which layer of your architecture you’re testing, this technique is the first thing you’ll reach for.
6. Bonus Round: SnapshotDescription
“Not everything has a meaningful description.”
In the presentation I demonstrate that NSObject doesn’t give a great description. And sometimes you don’t want to fully dump all of the ivars of a structure.
Internally I wrote an intermediate data structure called SnapshotDescription. Initially it helped pretty-print a tree hierarchy for our TableViews; It would walk over our DataSource and print out all the rows for all the sections, and ultimately turn that into a big long String that we can test.
This SnapshotDescription turned into a pretty helpful way to print many things. Essentially, if you could define a function (T) -> SnapshotDescription, I could take that and print it in ASCII art.
Thank you for reading. If you enjoyed this, please like, share, read my other rants, and follow me on Twitter.
