CI / CD at SwiftKey (Part 2)

Praveen Pendyala
Microsoft Mobile Engineering
5 min readApr 28, 2021


This is the second part of our multi-part series. You can find the previous part below. In this article, we will cover the building blocks of Azure Pipelines and how we set up agents within the SwiftKey Android team.

Let us start with some key concepts of Azure Pipelines.

Key concepts

If you are familiar with how Azure Pipelines work, feel free to jump ahead to the SwiftKey Agents setup section below.

An overview of key concepts of Azure Pipelines
  • A trigger tells a pipeline to run.
  • A pipeline is made up of one or more stages. A pipeline can deploy to one or more environments.
  • A stage is a way of organising jobs in a pipeline and each stage can have one or more jobs.
  • Each job runs on one agent. A job can also be agentless.
  • An agent runs a job that has one or more steps.
  • A step can be a task or script and is the smallest building block of a pipeline.
  • A task is a pre-packaged script that performs an action, such as invoking a REST API or publishing a build artifact.
  • An artifact is a collection of files published by a run.

Azure Pipelines come with certain built-in tasks to enable basic build and deployment scenarios. The full catalogue of built-in tasks can be found here. Visual Studio Marketplace has a plethora of additional custom extensions from Microsoft or 3rd Parties. Installing a custom extension, expands the task catalogue in your own Azure DevOps (ADO) subscription and it requires a security approval from the subscription admin.

Azure Pipeline Execution

This is a high-level overview of the series of events that take place in executing a pipeline.

Series of events that happen between starting a pipeline run to reporting the results back.
  1. Start a pipeline: Either a developer or one of the trigger events starts the pipeline on Azure DevOps (ADO).
  2. Agent lookup: ADO looks at the pipeline configuration to find all jobs that can run in parallel and finds a suitable agent for each job based on the demands specified in the job configuration.
  3. Submit job: Once an agent is allocated, ADO submits that job to the agent. Every job is executed on an agent except agentless jobs which run on the ADO server.
  4. Job execution: Agent executes each step defined in the job sequentially and report the results back to ADO.
  5. Logs and artifacts: ADO gathers all the execution logs from each step and artifacts in every job and makes them accessible through ADO.

Agents on Azure Pipelines

An agent on Azure Pipelines is the entity on which jobs are executed. An agent can be a physical machine, virtual machine or a docker container. Agents are grouped into agent pools. Broadly there are two types of agent pools: Microsoft-hosted agents and Self-hosted agents.

There are two types of agent pools on Azure Pipelines. Both of them connect to the ADO server.

Microsoft-hosted agents

These are managed entirely by Microsoft, so software updates and maintenance are taken care of. These agents are re-imaged between each job so we can expect clean and predictable state for each job. They come pre-installed with commonly used tools and packages. At the time of writing, these agents have the following limitations:

  • Custom packages need to be installed at runtime*
  • Run on Azure DSv2-series VMs
  • 10 GB of storage
  • 360 mins execution time limit

*Microsoft-hosted agents based on Windows 2019 and Ubuntu support running jobs in docker containers. This could speed up jobs that depend on installing custom packages.

Self-hosted agents

ADO supports deploying an agent pool where the agents are entirely self-managed. These agents can be based on Windows, Linux, or Mac operating systems. Users have full control over how these agents are built and configured so, it is possible to:

  • Pre-install any custom packages
  • Use more-powerful machines
  • Add agents to internal networks (useful if certain resources are available only over internal network)
  • Host agents on-premises to attach physical devices such as Android phones

SwiftKey Agents setup

At SwiftKey we need agents with different capabilities as we discussed in the part 1 of this series. Our agents are broadly divided into three categories:

Overview of different types of agent pools used at SwiftKey and their ownership divided between Android and Infrastructure team.

Microsoft-hosted agents

These agents are managed by ADO. We use them for build jobs which use only the pre-installed packages. Jobs that run on these agents include builds for open-source repositories, and builds that are not time critical.

Self-hosted agents in Cloud

These agents are fully managed by us and contain custom packages that are required for certain jobs. These agents are Docker containers deployed on Azure Cloud Infrastructure. We use these agents for jobs that have at least one of the following requirements:

  • Need an emulator
  • Need custom packages
  • Need more memory or CPU

Jobs that typically run on these agents include Android nightly tests, and APK builds.

Self-hosted agents on-premises

These agents are also fully managed by us, containerised using the same image as the Self-hosted agents in Cloud. These agents are used for jobs that have at least one of the following requirements:

  • Need a physical device
  • Need to access resources available only on the internal network

Jobs that run on these agents are performance tests, smoke tests etc.

Config ownership

As shown in the figure above, the Android team now owns bigger part of our CI/CD infrastructure. This allows us to rapidly iterate on changes to the agents and their configurations. Our Cloud Infrastructure (CLI) team owns the infrastructure which deploys agents following the configuration defined by the Android team.

Next Up

In the next article in this series, we will discuss different Azure Pipeline features and how we leveraged them in our CI / CD processes.



Praveen Pendyala
Microsoft Mobile Engineering

Passionate about many things tech. Driven by logic and compassion. I care about equal opportunities, unconscious bias, decentralization, and blockchains.