Improving your Workflow with Bitbucket Pipelines for Continuous Delivery of Android Apps
There is a common challenge of improving Feedback loops in Agile environments
How can you automate your tests, review code, and circumvent the process of having to manually generate the .apk of that awesome Villain Decepticon App you are tugging at?
We can save a lot of time when we automate repetitive tasks. This also applies greatly to testing and releasing Android Apps.
Some options may involve using a dedicated CI server like Jenkins with some Continuous Delivery tool like Fastlane.
In this article, however, we are going to be discussing releasing multiple iterations of your application on Slack through Bitbucket Pipelines.
Article Overview
1) Why use Bitbucket Pipelines?
2) Setup Your Repository ( This includes creating a Slack Legacy token and Steps on Creating your own Slack App)
3) Configuring your Project.
4) Conclusion
* Why use Bitbucket Pipelines?
Bitbucket pipelines are a cloud-native CI/CD tool and thus, the concern of managing physical infrastructure or provisioning a server is eliminated.
It also supports simple integration with services that you would otherwise use as stand-alone tools. This is done through custom pipes such as Slack-notify that lets you and your team know important stuff like when your build fails or passes. This ease makes it a good option for automating tests and frequent releases.
Bitbucket Pipelines are also awesome because you don’t have to pay separately for it. Each Bitbucket Plan comes with some build minutes per month.
If you are new to Bitbucket, it would help to know that the Standard pricing per month is $3 per user and there is no limit to the users you can add to your team. At this price, you get 2500 minutes per month build time as shown in the image above. On the other hand, the premium plan costs $5 per user and you get 1000 more build minutes than what you get on the Standard plan.
For teams with ≤ 5 developers and fast builds, you may be able to use the Free plan i.e 50 build minutes for a month.
How to Setup Bitbucket Pipelines and integrate with Slack and /or Deliver to Bitbucket Downloads of your repository.
* Setup your Repository
Note: Steps 1 to 3 are only required if you want your .apk file to also be uploaded in the Downloads section of your repository on bitbucket.
Build artifacts are torn down after a while so you can also use these steps to keep your .apks on Bitbucket instead of on local machines.
You can, however, skip these steps if you need just the Slack uploads.
- Click on your profile picture at the bottom left corner of Bitbucket and click App Passwords under Access Management.
2. Grant write Permissions and then click the create button.
3. Copy out the 20-character string app password and keep it somewhere safe as you will not be able to access it after closing the “pop-up”.
4. Create a personal Slack Legacy Token or build a simple Slack App for the workspace that you are trying to deliver your application to
Note: Although creating a personal slack legacy token in your workspace is the shorter approach, the little issue with this is once the app is delivered to the specified channels on the workspace, it will be a message from the account whose Slack Legacy Token is used to authenticate.
Pointing to individual tokens on the repository can solve this issue but creating a Slack app is a cleaner approach to this.
The next few steps will highlight creating the slack app and adding it to your workspace.
Steps 5 to 11 highlight steps to create a slack app. If you would rather use Legacy tokens copy that out instead and skip to 12.
5. Visit https://api.slack.com/apps. If you are not signed in, sign in and then click create App.
6. Choose a nice name for your app and select your workspace from the dropdown. Confirm the App creation after this.
7. On the Basic Information page enable Information Webhooks and you get Bots and Permissions checked.
8. Click on Permissions and Under the Bot Token Scopes, Click Add an OAuth scope and search for the files:write permission then add it.
9. Under Basic Information, you can update the icon of your app and you can add a brief description. Click Install App to Workspace.
10. Confirm the Installation and select a default channel to invite the app.
11. Copy out the token assigned under OAuth Tokens. It should start with xoxb-…
12. On your Bitbucket Settings page of the particular team and /or repository where your code lives and go to Repository Variables under Pipelines on the sidebar.
13. If you followed steps 1–3, you will need to add the token string you got, let us call it BB_AUTH_STRING for releases to your Bitbucket Downloads and either your Slack Legacy Token from step 4 or your Slack App Token from steps 5 to 11 as SLACK_TOKEN.
This can be done in any order and for collaborative repositories, be sure to check the secured box while creating the variables so they are masked.
You can pick your own repository names but be sure to replace the variables you are pointing to in the .yml file we will create in the Configure Your Project Section.
After this step, your repository should be ready for Bitbucket Pipelines.
* Configure Your Project
For this step, we are going to need our Android project to be able to be compiled and assembled in the pipelines. But wait how? Bitbucket pipelines are technology agnostic. How are we expected to build that .apk file and deliver it then?
Remember apart from having clear and manageable methods of doing this, we are trying to avoid having all these dependencies on a local machine or cloud Infrastructure. The SDK itself is resource-intensive. Disk usage for the Android-SDK itself is about 1.9G and using such space on your precious CI server can be costly and inefficient.
There are two ways one can achieve this.
One approach can be to pull your dependencies like the Android SDK in your configuration file with wget e.g
- wget — quiet — output-document=android-sdk.zip https://dl.google.com/android/repository/sdk-tools-linux-3859397.zip — unzip -o -qq android-sdk.zip -d android-sdkWith this approach, we will have to set the Path to our Android SDK in our configuration file just like you would on a local machine. This will look like
- export ANDROID_HOME=“/opt/atlassian/pipelines/agent/build/androidsdk”
-export PATH=“$ANDROID_HOME/tools:$ANDROID_HOME/tools/bin:$ANDROID_HOME/platform-tools:$PATH”Finally, we would have to fetch the packages we will need via sdkmanager.
- yes | sdkmanager "platform-tools"
- yes | sdkmanager "platforms;android-27"
- yes | sdkmanager "build-tools;27.0.3"
- yes | sdkmanager "extras;android;m2repository"
- yes | sdkmanager "extras;google;m2repository"
- yes | sdkmanager "extras;google;instantapps"
- yes | sdkmanager --licensesWith this, we will then be able to build our artifact with Gradle. For this article, we will not be discussing this approach in detail. This is because we can achieve the same goal with our second approach which ensures shorter lines in our configuration file and is in my opinion, a cleaner approach.
Recommended Approach
Well, we can use a Docker image that has all our dependencies and abstracts all that is being done in the first approach.
There are a couple of notable Docker Images for Building Android like Uber’s Android Build Environment and XmartLabs Android Docker Image. For this tutorial, we would be using Mingchen’s Android Build Box. It is quite ubiquitous and ships with Android SDKs from 16 to the more recent 28. It also contains many versions of Android build tools and Fastlane. Other stuff it supports out of the box can be found on its Github page.
Steps:
We will be creating in our project’s root directory, a bitbucket-pipelines.yml file or editing the existing file.
The selected docker image variable is declared at the top of the file like so
image: mingc/android-build-box:latestThis ensures that all the Android build dependencies are available in the Pipeline environment.
We then configure the default and other branches. The default gets run after every push to any branch i.e even like before creating a pull request. In this step, you can run your tests and ensure that your tests run.
pipelines:
default:
- step:
caches:
- gradle
script:
- chmod +x gradlew
- echo 'inside default branch. tests should run now.'
- ./gradlew testWhen this is done and it is confirmed your tests are run and perhaps your code changes have been reviewed and everything is set to be merged to say master or develop, the following block handles releases
branches:
branchname:
- step:
caches:
- gradle
name: Build App
script:
- ./gradlew assembleFlavourVariant
- . ./latestbuild.sh#- curl -s -u "${BB_AUTH_STRING}" -X POST "https://api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads" -F files="@$LATEST_APK"- curl -F file="@changelog" -F "initial_comment=What is New" -F channels=channel-one,channel-two -F token=${SLACK_TOKEN} https://slack.com/api/files.upload- curl -F file="@$LATEST_APK" -F channels=channel-one,channel-two -F token=${SLACK_TOKEN} https://slack.com/api/files.uploadartifacts:
- app/build/outputs/apk/flavour/variant/*.apk
What the above block does from top to bottom is highlighted below:
a) use gradle build tools to assemble the particular variant eg StagingDebug, ProductionRelease. If you don’t already manage Build variants in your gradle scripts, you can replace the assembleFlavourVariant with assembleDebug instead.
b) Finds the latest build that has just been released. Please note that Latestbuild.sh should also live in your project’s directory and should look like below.
latestbuild.sh
LATEST_APK=$(ls -lrt app/build/outputs/apk/staging/debug/*.apk | tail -1 | awk -F" " '{ print $9 }')
FILE_NAME=$(basename $LATEST_APK .apk)".apk"c) This commented out line (with the Pound symbol in front of it) handles uploading to the Downloads section of your repository. You can delete the line if you did not follow steps 1 to 3 in the Repository setup section.
d) Over curl, it uploads your changelog/release notes. i.e what is new in your application. You can create a text file in the root directory of your project and itemize what has changed and always edit this for your new Pull requests.
e) Over curl, it also uploads the artifact(.apk) to the specified channel(s).
Just like in the step above, you can specify more than one channel your .apk and release notes will be sent to by simply separating them with commas.
Note: It is best to pin your mingc buildbox version to the version where your target-sdk version was added. e.g 1.3.0 for SDK version 28. This to prevent build breaks and compatibility issues. For instance Android’s Databinding breaks on mingc/android-build-box:1.12.0 and throws the JAXB Exception because the class has been removed from versions of Java greater than 8.
In the end, your bitbucket-pipelines.yml file should be similar to this.
image: mingc/android-build-box:1.3.0pipelines:
default:
- step:
caches:
- gradle
script:
- chmod +x gradlew
- echo 'inside default branch. tests should run now.'
- ./gradlew test
branches:
master:
- step:
caches:
- gradle
name: Staging Debug Build
script:
- ./gradlew assembleStagingDebug
- . ./latestbuild.sh#- curl -s -u "${BB_AUTH_STRING}" -X POST "https://api.bitbucket.org/2.0/repositories/${BITBUCKET_REPO_OWNER}/${BITBUCKET_REPO_SLUG}/downloads" -F files="@$LATEST_APK"- curl -F file="@changelog" -F "initial_comment=What is New" -F channels=channel_one,channel_two -F token=${SLACK_TOKEN} https://slack.com/api/files.upload- curl -F file="@$LATEST_APK" -F channels=channel_one,channel_two -F token=${SLACK_TOKEN} https://slack.com/api/files.upload -vartifacts:
- app/build/outputs/apk/staging/debug/*.apk
Kindly note that the indentation of your .yml files matter and you can use Bitbucket’s YML Validator tool to check that your file is nicely laid out.
With the above files, when your remote branch gets merged to your origin branch, your build will be released to the channel(s) specified.
You can find all the files needed here. Kindly star this repository and cheer for this article if it is of any help.
* What else? / Conclusion
There are a lot of other things Pipelines can be used for like app signing and releasing to Playstore or Moving the artifacts to the Downloads tag on your repository.
This is just an example and a lot of cool things can be done with this like having parallel steps to reduce build time and optimizing builds in your project.
References and Links
https://api.slack.com/custom-integrations/legacy-tokens
https://gist.github.com/albka1986/f1c6bdca1823e8d7a894f1c47bc3819a
https://hub.docker.com/r/uber/android-build-environment/https://hub.docker.com/r/mingc/android-build-box
https://github.com/mingchen/docker-android-build-box
