Running Flutter tests in Bamboo on AWS

Here in Pittsburgh, we are building mobile apps using the Flutter framework. Like all good engineering teams, we write unit tests, and want to run them in our CI system. We happen to use Bamboo and here’s what we did to get our Flutter unit tests to run on each git commit.
TL;DR
The key lessons learned from this experience are:
- Needed to install
mesa-libGLU.x86_64 - Needed to workaround
libstdc++.so.6being missing - The
--machineflag causesflutter testto output result in JSON - The
junitreportpackage is used to convert JSON output to JUnit.xmlfiles (via thetojunitcommand), a format Bamboo understands
Installing Flutter on Amazon Linux
Our Bamboo installation runs on an Amazon Linux AMI (which is for all intents and purposes, a variant of CentOS). We have a very basic installation right now, running only a single agent on one box, so keep that in mind as you view what we did below. If you are running multiple agents, you’ll need to make sure each agent has Flutter installed as well as the junitreport tool.
It’s also worth noting that we have an EFS (elastic file system) mounted at /custom. We download files into /custom/downloads and install software into /custom/opt. You, obviously, can pick your own download and install locations. Finally, we run our Bamboo instance using a bamboo user.
The installation steps we used to install flutter are outlined below.
Prerequisite
As we were getting this all to work, we ran into a problem running out unit tests where we got an error saying that ligGLU.so.1 was not found. With 20/20 hindsight, we recommend installing this library up-front. Amazon Linux, being CentOS-like, usesyum:
sudo yum install mesa-libGLU.x86_64Install Flutter
Next we download the version of flutter we are currently using (0.6.0 when we went through this exercise — feel free to use the latest and greatest).
# Login to bamboo box as ec2-user
cd /custom/downloads
mkdir flutter
cd flutterwget https://storage.googleapis.com/flutter_infra/releases/beta/linux/flutter_linux_v0.6.0-beta.tar.xzcd /custom/opt
sudo mkdir flutter
sudo chown bamboo:bamboo flutter
After downloading the install tar and creating a place to install it, we do everything else as the bamboo user.
First, we un-tar the file, which has a top-level directory called flutter.
sudo su - bamboo
cd /custom/opt/flutter
tar xvf /custom/downloads/flutter/flutter_linux_v0.6.0-beta.tar.xzThen, anticipating having multiple versions of flutter in the future, we rename the extracted flutter directory to v0.6.0 and create a latest symbolic link to this version.
mv flutter v0.6.0
ln -s v0.6.0 latestWe ran into an issue about libstdc++.so.6 being missing, so followed this recommendation to fix the problem:
cd latest/bin/cache/artifacts/engine/cp android-arm64-profile/linux-x64/gen_snapshot android-arm-profile/linux-x64/gen_snapshotcp android-arm64-release/linux-x64/gen_snapshot android-arm-release/linux-x64/gen_snapshot
Then we set some paths so we could run Flutter and Dart commands (you may want to add this to the ~/.bashrc):
export FLUTTER_HOME=/custom/opt/flutter/latest
export PATH=${FLUTTER_HOME}/bin:${PATH}
export PATH=${FLUTTER_HOME}/bin/cache/dart-sdk/bin:${PATH}Finally, we install the junitreport tool (which translates Dart JSON test output to JUnit format) and add the install location to the PATH so we can verify it installed.
pub global activate junitreport
export PATH=~/.pub-cache/bin:${PATH}
tojunit --help If tojunit prints out a usage message, you are good to go. If not, then retrace your steps and engage those debugging skills.
It’s important to note that pub global activate installs executables in the current user’s home directory. The reason we do all this as the bamboo user is so this executable can easily be found when running a Bamboo task (obviously there are other ways to solve for this, but we took the easy path).
Configuring Bamboo
We will assume you know a bit about configuring Bamboo and how to setup plans. For our Flutter repository, we setup a simple plan with 3 tasks.

Task 1
Task 1 is the default repository source code checkout from git.
Task 2
Task 2 is where the magic happens. We chose an inline shell script. Note that this script assumes it is running from the root of the Flutter project (e.g., the directory that contains the pubspec.yaml, lib and test directory).
Here is the script:
#!/usr/bin/env bash# exit on any error
set -e
# setup paths
export FLUTTER_HOME=/custom/opt/flutter/latest
export PATH=${FLUTTER_HOME}/bin:${PATH}
export PATH=${FLUTTER_HOME}/bin/cache/dart-sdk/bin:${PATH}
export PATH=~/.pub-cache/bin:${PATH}# Make sure output dir is there
RESULTS=build/test-results/flutter-tests.json
mkdir -p build/test-results
# Get all packages
flutter packages get
# Run unit tests
flutter test --no-pub --machine test/unit | tee ${RESULTS}
# Parse JSON to JUnit XML
echo "Parsing test results from ${RESULTS}"
cat ${RESULTS} | tojunit -o build/test-results/flutter-tests.xml
A couple of notes on the above script:
- Results are output into the
build/test-resultsdirectory - The
--machineflag is a hidden option offlutter testthat causes test results to be output in JSON format. - The JSON output is what
tojunitknows how to parse — it turn it into standard JUnit.xmloutput - We separate our unit tests and integration tests into
unitandintegrationdirectories respectively under thetestdirectory. This is why you seetest/unitin theflutter testcommand. If you do not have a similar separation, you can eliminate thetest/unitparameter (which will run all tests undertest.
Task 3
The 3rd task, set as a “final” task (so it runs even if things fail), is a JUnit Parser task which looks for test files using this pattern: **/test-results/*.xml.
What this does is allow Bamboo to interpret the Flutter test results as it does Java JUnit results. It will track what fails/succeeds on a build-by-build basis.
End Result
Our Bamboo plan is triggered with each checkin to our repository. It successfully runs our Flutter unit tests and emails us when the build breaks (which I can assure you happens infrequently :-).
While we used Bamboo for our builds, this should be easily adaptable to other CI systems like Jenkins.

