How to run Flutter Golden (Snapshot) Tests with Codemagic CI/CD
Testing is a very important aspect of development, it helps to make sure that your app works and looks properly for the users. It is great that there are CI/CD tools that automate some of the work for you in checking the app quality, notifying the team if something goes wrong, and releasing the app to the end-user.
In this article, I’m going to explain how to run Golden tests with Codemagic CI/CD.
Step 1: Create the Golden (Snapshot) Test
In one of my previous articles, I’ve covered how to create Flutter Snapshot tests, therefore I won’t focus on this topic here. Instead, I’ll just copy over some code snippets so that you don’t have to go back and forth between articles to follow.
Create golden_widget_test.dart in the test folder of the counter app with the next code:
void main() { testWidgets(‘Golden test’, (WidgetTester tester) async { await tester.pumpWidget(MyApp()); await expectLater(find.byType(MyApp), matchesGoldenFile(‘goldens/main.png’)); });
}
Generate snapshots:
flutter test — update-goldensI’ve created a counter app with a Golden test and pushed it to GitHub. I was able to see my new app in Codemagic Applications.
Step 2: Run Golden (Snapshot) Test with Codemagic
When you generate Golden (Snapshot) Tests, the different OS platforms generate different files. There is an issue filed on GitHub, but it’s closed because this is working as intended, even if not very convenient. It means if you generate your Golden files on Linux and, afterward, run your tests on MacOS, your Golden Tests will most likely fail. However,
All tests that you run with Codemagic are executed on MacOS.
Thus, people who use MacOS have an advantage. All passing Golden Tests will succeed on Codemagic CI/CD without any extra work.
If you are a MacOS user you need to press “Start new build” and you are all set, just do not forget to locally generate (see above) and commit golden files before.
If everyone in your team uses MacOS, you are done 🥳. No need to continue reading.
Step 3: Run Golden Tests if you are not a MacOS user
Codemagic supports free remote connection to MacOS infrastructure and also has FCI_EXPORT_DIR variable to upload any files from builder machines when a build has finished.
You don’t need to generate Golden files on your machine, instead, you can do it using Codemagic. I’ve pushed my code without generated snapshots to GitHub.
After that, you need to create a new workflow just for the generation of snapshots, since you don’t want to lose the build minutes and run the release workflow with tests that will fail anyway.
In the Test section, you need to enable Flutter test and set Flutter test arguments to:
test --update-goldens
The command above will generate Snapshots for you. Then you should copy generated Snapshots to $FCI_EXPORT_DIR with the following command in Post-test script section:
cp -r test/goldens $FCI_EXPORT_DIR
Don't forget to disable Build and Publish section as well!
Save settings and start a new build. After finishing the build you should see the generated golden files in the Artifacts section on the left.
What you need now is to just download this zip and extract generated golden files into your project.
Tip: If you have Golden tests in your project you should add the skip parameter if the platform is not MacOS. Then if you run flutter test locally, the tests won’t fail because of golden differences with Mac.
testWidgets('Golden test', (WidgetTester tester) async { await tester.pumpWidget(MyApp()); await expectLater( find.byType(MyApp), matchesGoldenFile('goldens/main.png'), );}, skip: !Platform.isMacOS);
Useful links:
If you like the article, you can support me 👏 . And follow me on Twitter!
