How we improved our development workflow using Unity Cloud Build

Published in

Lonely Vertex Development

8 min readJul 2, 2019

An article about connecting Unity Cloud Build with Bitbucket to simplify development and testing of new features.

We are working in a team of five people on an iOS game. Since we are not doing this as our full-time job, we are trying to automate as much as possible during the development. This article explains how we utilised Unity Cloud Build to automate testing and distributing builds among the team while working on new features.

Struggling with the development workflow

We are using Git, specifically Bitbucket, as a source control management tool. The typical workflow is that developers create a new branch from the master branch for a feature or a bug fix. When they finishes the work, they create a pull request, and once it is accepted in the review by other developers, it is merged to the master branch.

We wanted to have a working, playable build available all the time. Therefore, we needed to test the build before the feature branch was merged to the master branch. The problem with iOS games is that they are not as simple to distribute because you need to set up the certificates for the devices or build it directly from Xcode while having the device connected to the computer.

That’s a lot of work doing this for every pull request. What we end up with was something like: “Did you test it on the device?” — “Yes, it worked.” — “Ok, let’s merge it.” And from time to time, master branch got broken.

The other problem was to even distribute the latest build from the master branch among the team. Building it every time is tedious. Our graphic designer didn’t even know how to build the game for the device, so he had to meet a developer physically to get the latest build.

Once we started using automated tests, it got even worse, because it was so easy to forget to run them before committing the changes. Sometimes we found out that the tests (thus maybe something in the game) were broken after we merged several other branches.

Cloud Build to the rescue

Then, we discovered Unity Cloud Build. You can connect your source control management tool to the Cloud Build and automatically get the project built when there are new changes pushed to the given branch. After the build is finished, it can send a link to email or Slack where you can simply download it to the device directly. It can even run tests and fail the build if the tests fail. That is a huge step forward to automate most of the tedious tasks. The whole workflow we are using is shown in the following activity diagram.

There was still one problem, though. Cloud Build can be set up from the specific branch, and every time it is changed, a new build is created, which is nice to get the latest build from the master branch. The thing is, the build was available after the feature branch was merged to the master branch. That didn’t help with preventing it from breaking. For the feature branches, we would have to set up a new target in the Cloud Build manually. Another tedious task which can be easily skipped due to laziness. We needed to automate creating a new target in the Cloud Build whenever somebody opens a new pull request in Bitbucket. Luckily, Cloud Build has an API, and also Bitbucket has an API that can be used for that.

Implementing Cloudbuildify

We needed an intermediate service for the communication between Bitbucket and Cloud Build. We called it Cloudbuildify. The requirements were:

create a new build target in Cloud Build whenever a new pull request is created in Bitbucket
update build status in Bitbucket whenever a build status in Cloud Build is changed
delete the build target in Cloud when the pull request in Bitbucket is merged or closed

The communication is shown in the following sequence diagram:

Cloudbuildify communication with Bitbucket and Unity Cloud Build

For the implementation part, we chose Python language and Flask framework for its simplicity. The goal was not to create a battle-tested solution that would cover all the possible edge cases, instead to build a minimum viable solution that would work in usual scenarios and fix possible bugs later on if they appear. Our primary goal is to develop our game, not some development tools.

Link to the full source is at the end of the article. The following sections will describe some essential parts.

Bitbucket webhooks

There is a Webhooks Settings in the Bitbucket repository. When creating a new webhook, we can choose from a list of triggers we are interested in. For our use case, we need only triggers connected to Pull Requests. Then, we can find in the documentation what the payload for pull request events is.

Here is the Flask route for handling Bitbucket webhooks. Notice that there is also a secret included in the URL from the configuration to make the URL more secure. The final URL is then https://{cloudbuildify_host}/bitbucket/{secret}. The secret is important for security because nobody can trigger the webhooks without the knowledge of the secret.

There is a function get_pull_request_args that extracts some data we are interested in from the payload:

name of the branch — so we can use it to name the build target
commit hash — the build status in Bitbucket is connected to a specific commit, we need to save this for later
name of the user who created the pull request — we also use this in the build target name

The function itself is nothing complicated, it just tries to get the data if possible:

We also convert the name to ASCII because otherwise it could not be used for a Cloud Build target name.

Creating a Cloud Build target

A new cloud build target can be created via the API according to the documentation. However, it requires a lot of fields to be filled in. We want to have everything the same as in the master branch. The only things, we need to change are the name of the build target and git branch that it should build from. So instead of setting everything, we can simply get the master build target as a template and only change it a little bit. How to get a build target is also described in the documentation. Here is a simple Python function to get the template build target:

Notice that we need to remove links and buildtargetid to be able to use the data for creating a new build target.

Then, there is another function that uses the data from the template, new branch name and user name of the user who created the pull request. It calls Cloud Build API to create the build target.

There is a 64 characters limit on a build target name length, and the name cannot use any special characters. Therefore, there is some juggling with the name in the function. Then the name is replaced in the template build target data and also the branch.

The data about the build target and pull request (git branch name, commit hash, build target ID and build target name) are stored into the SQLite database to be used later. E.g., when a pull request is merged, we need to know which build target to delete.

The template build target has an auto-build option set to on. However, it only builds when the branch changes. Since we created it after the pull request was created, there are no changes now. We need to start the first build manually using Cloud Build API again.

Setting the build status back in Bitbucket

We can set up webhooks in Cloud Build as well and then receive the information about the build status. Bitbucket has an API to update the build status for commits.

So first, we need another Flask route for the Cloud Build webhook. The specific event is provided via a header called X-UnityCloudBuild-Event. We are interested only in events connected to build status and can safely ignore the rest.

Then, we load the data from the database based on the build target received and update the build status for the commit with a simple API call.

By setting the build status back to Bitbucket, we get nice icons and build details by the pull requests.

Deleting the build target

To keep the things clean, we want to delete the build target once the pull request is either merged or declined. It is just a simple call to the Cloud Build API once we receive the webhook call from Bitbucket.

Updating the latest commit

There’s one more thing, though. Since we set the build status in Bitbucket for the commit, not branch or pull request, we need to keep track of what the latest commit in the branch is. There is a webhook event from Bitbucket for updating the pull request. We can then save the latest commit when handling that event.

We don’t need to update anything about the build target, because it has the auto-build option on, so it will build anytime the branch is updated. And when the Cloud Build webhook is called with the updated build status, we already have new commit saved in the database and can update the status for the latest commit.

There is some potential for race conditions since we can push other commits when the build is running, and then the status would be set for an incorrect commit. However, it is not a big concern for us because the branch will be eventually built again for the latest commit.

Conclusion

This was a brief description about how we connected Bitbucket and Unity Cloud Build to automatically create builds for each pull request so we can test it before merging it to the master branch.

The complete source code for the project is available on GitHub. You can have a look at it to find out all the details not described in the article, or reuse it if you find it useful.

We are Lonely Vertex, a small indie game studio located in Prague, Czech Republic. Currently getting ready to release our first game, Sine. You can subscribe for newsletter, read our development blog posts or follow our progress on Twitter or Facebook.