The Anatomy of Android Continuous Delivery
How Android developers can harness the power of Continuous Delivery and stay secure
An evolution of Continuous Integration
Continuous Integration (CI) is ubiquitous. Virtually all software industries today use the practice to automate their integration processes, helping to ensure that software development has the required quality standards through the various processes, using a Version Control System (VCS) and Automated Tests. Typically, CI uses a build server to implement the continuous processes of applying general quality control and of reducing the manual Quality Assurance (QA) processes.
Continuous Delivery is the natural evolution of CI, since it provides the processes to ensure that any of the developments in the CI cycle are in an easily deployable state. In the context of mobile development, Continuous Delivery is the ability to produce valid builds that are ready to be published in an app store. Continuous Deployment is the final status of the process, which ensures that any of the changes applied to the code is automatically deployed to the production environment.
In this post, I’m going to break down the advantages and challenges posed by Continuous Delivery, and demonstrate how you can look to implement it for Android mobile development.
Is Continuous Delivery right for you?
The common CI scenario
Forming the backbone of CI is Version Control, built on rock-solid Unit Tests that reduce error-prone code and raise the quality of the final result. Version Controls are used by everyone from small dev teams all the way up to enterprise organisations. While traditionally, CI uses a locally-built and maintained server, many companies have now begun migrating their CI process to a cloud-based solution, primarily because of the flexibility and control that cloud computing can give you over costs.
Breaking down the challenges
Despite the proliferation of CI, not all companies will be able to implement Continuous Delivery — let alone Continuous Deployment — because of the challenges involved. There are several reasons why mobile developers might struggle to automate deployments. One example is a review step in the iOS deployment process, while some of the Android QA processes can require tests on the alpha or beta channel before proceeding with the final deployment. But for those who can overcome these hurdles, replacing a CI process with Continuous Delivery can bring huge improvements to mobile development.
Meeting company policy
It’s important to consider a company’s security policies when looking to implement Continuous Delivery. Locally-built CI servers can give you greater control over the information being stored and processes being run during development, and some companies have very tight rules on what can be run outside the organisation’s local network. This means that for some, the sharing of release keys and certificates will not be an attractive option.
Key signing… and a possible solution for mobile developers
The Android release deployment process requires a key to sign off the production build before it moves to the Google Play store. Keeping keys and passwords on VCS is bad practice, because VCS is available to any member with access to the project, which includes any temporary 3rd parties. It’s common practice therefore to create release builds on a local computer, but this erodes one of the great advantages to Continuous Delivery: having a scalable online platform and the ability to easily deploy any stages of the CI at any time.
Because of the need to secure the signing keys of the app, building Continuous Delivery for mobile development can be tricky. For Android app development, you could improve security further by using the new Google Play App Signing, which applies a second key when it is uploaded to the Google Play Store. But even after employing this method, there will still be at least a key and a couple of passwords required to build a release (signing key, key and upload password). The CI system needs these to build an APK ready for release, but as mentioned it’s not a good idea to put them on the Version Control. However, using Encrypted Cloud Storage services is one solution to this problem; they can keep the keys safe and are only limited by the integration capabilities of your chosen CI system.
A speed and cost boost
While a local CI server does not require a powerful computer to run build processes, the execution of tests can be time-consuming. If a regression test takes dozens of hours to complete, it becomes potentially expensive in terms of time lost to testing. Furthermore, a locally-based server will have associated costs for maintenance and eventual replacement.
By comparison, a cloud-based solution gives you the opportunity to use only what you need and when you need it. This means you can scale up or down at will, making the process cost-effective and without the need for ongoing maintenance. It offers a highly-efficient configuration in terms of compute power, reliability and flexibility.
Implementing Continuous Delivery
Picking the right tools for the job
Assuming your Version Control System is based on GIT or Mercurial (either is a valid option), you can choose from a number of CI systems, including:
- TeamCity
- Travis CI
- Jenkins
- Bamboo
- GitLab CI
- CircleCI
This post uses CircleCI, GitHub and AWS as the preferred available solutions, but any of the other services can be valid considerations, as long as they provide the minimum required specifications.
Making a start with GitHub & CircleCI 2.0
Using CircleCI (otherwise known as CI system) begins with the integration of the Version Control System, an easy feat to accomplish with GitHub and CircleCI. You start the process by setting the GitHub Key in CircleCI. You can choose to set a GitHub Deploy Key that can allow access only to a specific repository, or a User Key that allows CircleCI to get access to all the user repos. Once you have the Deploy or User Key, the process to set it in CircleCI is to go to Project Settings > Checkout SSH keys and add the SSH Key.
The process is explained quite well in the GitHub security and SSH keys documentation.
CircleCI 2.0 — Anatomy for Android
CircleCI 2.0 comes with huge improvements for Android processes, but the benefits can be summarised with two main points:
- Dockerized Android Images
CircleCI 2.0 provides all the necessary SDK and Build Tools in a Docker image to build an Android app (which means hassle-free SDK downloads and automated licence acceptances!) - Workflows
The ability to build specific workflows based on dependencies running in parallel
Furthermore, CircleCI 2.0 drastically reduces the CI process time. We saw a 60% reduction in processing time compared with CircleCI 1.0, so I would strongly urge you to move directly to 2.0 for the new projects or migrate the old 1.0 to the 2.0.
Here a CircleCI 2.0 config.yml sample
version: 2references:# ----------------------------
# Cache Configuration
# ---------------------------- cache_key: &cache_key
key: cache-{{ checksum "gradle/wrapper/gradle-wrapper.properties" }}-{{ checksum "build.gradle" }}-{{ checksum "app/build.gradle" }}
restore_cache: &restore_cache
restore_cache:
<<: *cache_key
save_cache: &save_cache
save_cache:
<<: *cache_key
paths:
- ~/.gradle
- ~/.m2# ----------------------------
# Docker Configuration
# ----------------------------android_config: &android_config
working_directory: ~/your-project-root
docker:
- image: circleci/android:api-27-alpha
environment:
TERM: dumb
ANDROID_SDK_VERSION: 27
ANDROID_BUILD_TOOLS_VERSION: 27.0.2
GRADLE_OPTS: '-Dorg.gradle.jvmargs="-Xmx2560M -XX:+HeapDumpOnOutOfMemoryError"'# ----------------------------
# Job Section
# ----------------------------jobs:# Job Debug
build_debug:
...# Job Release
build_release:
...# Unit Tests
unit_test:
...# ----------------------------
# Workflows
# ----------------------------workflows:
version: 2
debug_build_deploy:
jobs:
- build_debug: release_build_deploy:
jobs:
- build_release:
In the reference area, the Cache and Docker Configuration ensures that the environment for the Android build is installed and working correctly, handling the cache strategy and loading the prepared Docker image for Android. Before CircleCI 2.0, these steps were not Dockerised, which meant that they were much more time consuming. This step in 2.0 is what makes up the majority of your efficiency savings for the build time, when compared with 1.0.
The Job sections points to the single available atomic tasks grouped under a custom name (i.e. build_debug, build_release, etc…).
Last but not least, the Workflow area defines the execution of the job strategy above.
Fabric, AWS and the custom helper
To support the delivery process, the Fabric Beta service is an excellent choice to internally distribute the build to the proper departments (QA, Marketing, Advertisement, etc…) while AWS stores, in an encrypted S3 bucket, the app signing key and certificates.
Since the Fabric Beta deployment can easily be configured in the app project and executed through a gradle task, the AWS integration needs a specific customisation for the CircleCI 2.0 config.yml.
It’s good practice to create a specific IAM user within AWS for the CircleCI configuration and to assign it a specific KMS key to be used for the encrypted S3 bucket.
Configure CircleCI to get AWS access
To grant CircleCI access to AWS, within CircleCI, go to Project Settings > AWS permission. Supply CircleCI with the Access Key ID and Secret Access Key for the new AWS IAM user, which are generated when you create the user in AWS.
Retrieve keys and certificates from AWS S3
To complete the Android release build, the build_release job needs to get the signing key, store and upload password archived on AWS S3, using AWS CLI.
The following build_release job in the CircleCI script contains the following steps:
- A call for the android_config to be set up for the Docker and Android environment
- To checkout the repo from GitHub
- To restore a previous cache (restore_cache)
- To run the Android dependencies needed for the project
- To save the cache (save_cache)
- To run the download and installation of the AWS CLI
Job Release
build_release:
<<: *android_config
steps:
- checkout
- *restore_cache - run:
name: Download dependencies
command: ./gradlew androidDependencies - *save_cache - run:
name: Install AWS CLI
command:
sudo apt-get -y -qq install awscli
The next part of the process is to get keys to complete the build_release job. To do this, we need to run the following steps to download the signing key from AWS S3:
- Download the signing App Key (keyfile.jks)
- Download the store and upload password
- Download the config.sh file, which contains the env $STORE_PWD, $ALIAS_NAME, $UPLOAD_PWD
This is an example of a script to do this:
- run:
name: AWS S3
command: |
mkdir temp
aws configure set s3.signature_version s3v4
aws s3 cp s3://remote_encrypted_s3/keyfile.jks tmp --region us-west-2
aws s3 cp s3://remote_encrypted_s3/config.sh tmp --region us-west-2
Finally, we add the build release step with the following script:
- run:
name: Build Release
command: |
. tmp/config.sh
./gradlew clean assembleRelease \
-Pandroid.injected.signing.store.file=tmp/keyfile.jks \
-Pandroid.injected.signing.store.password="${STORE_PWD}" \
-Pandroid.injected.signing.key.alias="${ALIAS_NAME}" \
-Pandroid.injected.signing.key.password="${UPLOAD_PWD}" - store_artifacts:
path: app/build/outputs/apk/
- store_artifacts:
path: app/build/outputs/mapping/
Deliver the APK on the Google Play alpha or beta channel
The final step is to deliver the APK on Google Play once the release build is completed and has passed all the automated tests. The only method of automating this process is to develop a custom component (i.e. CustomApp.jar) using the Google OAuth Client Library.
To build the custom app to facilitate the upload on Google Play, the OAuth Client Library requires a Service Account with the permission to manage the APK over the Store (See Creating a Service Account for more information).
However, it’s good practice to store the Google Service Account credentials in a similar way as the App signing key (in an encrypted AWS S3 bucket). For instance, in a JSON file (i.e. config_oauth.json):
- run:
name: Deploy to Google Play with CustomApp
command: java -jar tmp/CustomApp.jar tmp/config_oauth.json
Final Result
When completed, your Continuous Delivery setup for Android apps should look like this. Congratulations!
References
Google Play App Signing (documentation)
Google OAuth Client Library
AWS S3 & AWS CLI
CircleCI 2.0 GitHub security and SSH keys
Fabric
