Making the Most of Self-Hosted Build Agents in Azure DevOps
As a software engineer on Microsoft’s Commercial Software Engineering (CSE) team, I’ve been working to familiarize myself with Azure products— it’s critical for my own success and for the success of my team. A mentor recommended to me that, since I’m still relatively new to the world of technology, I should pick only one domain and try to master it.
“Interesting,” I thought. “That’s a great approach. I think I’ll try my hand at Machine Learning.”
“Interesting,” I was told. “That is a great topic. I think you’ll try your hand at DevOps.”
So I embraced the idea of becoming a DevOps engineer, and picked up a skill that I will now be sharing in this article — using a virtual machine as a build agent, and utilizing it efficiently.
Why you Should Listen to Me
Azure DevOps build agents are what allow continuous integration and delivery to occur within Azure Pipelines. These virtual machines are responsible for building and testing code for solutions, then delivering the product to end users. By default, Microsoft provides the user with build agents, which they maintain and update when necessary. This is usually the simplest way to run builds.
While the default Microsoft-Hosted build agents definitely have their benefits, using virtual machines as build agents has benefits as well. They include:
- Builds are typically faster: All of these factors below (and probably more) contribute to the fact that builds that utilize self-hosted agents generally take less time to complete, because they more run more efficiently.
- Control of the machine: Because the machine is controlled by the user(s), they are given the power to configure the size, processing power, memory, file structure, etc. of the machine, which is not the case with the default build agents. Scripts that are run in the build can target specific directories, machines can be resized to the user’s liking — anything the user wants to do to help make builds more efficient, they are allowed to.
- Dependencies only need to be installed once: Installing software on machines always takes some amount of time, depending on the size and complexity of the download. With default machines, any software that is not already installed on the machine (you can check to see if the software is already installed here), must be installed in the pipeline on each and every run. On self-hosted agents, however, the installation process only needs to occur one time, and the software is kept on the machine. While dependency updates should be routine, dependencies that are required by builds typically only need to be installed one time.
- Less constraints on parallel jobs: Jobs in a pipeline that are not dependent on one another can be usually be run in parallel, to save development time. Using the default agents, there are limits on how many jobs can be run in parallel, and for how long, especially if the project is a private one (see here). With self-hosted agents, there are less constraints on private projects using a self-hosted agent, instead of a default build agent, and no limits on how many parallel jobs are run if the project is public (see here).
- Cheaper parallel jobs: Also, if the user wanted to increase the amount of parallel jobs, it’s significantly cheaper to do so, if using a self-hosted agent over a default agent. At the time of writing, with the default build agents, the cost of having one parallel job run for unlimited minutes, costs $40/month, and it costs to $40/additional parallel job. With the self-hosted agents, however, the user is allowed one parallel job with unlimited minutes for free, and it only costs $15/month for each additional job. These prices are subject to change, however. See here for updated prices. The user should, however, consider the cost of the machine itself, to see if this approach is for them.
- Multiple build agents: Using a self-hosted machine allows the user to install as many build agents on their machine as they’d like (keeping machine memory and space constraints in mind). This is a relatively simple process (with a script below, for reference), which is what allows the parallelism to occur. But also, were one build agent to be nonfunctional, for one reason or another, a different build agent would be able to take on tasks while the nonfunctional one is being fixed or removed.
- Utilization of cache: When using the default agents, after each build is completed, the build agent is discarded. Using a VM that is controlled by the user allows cache utilization (which comes in handy when the solution involves containers, for example). This can speed up builds drastically if the solution utilizes cache to prevent redundancy in builds.
Now that you’re completely convinced that self-hosted agents are the way to go, and you have no reservations whatsoever, let’s find out how to set one up.
Setting Up Build Agent(s)
Note: All commands/steps are accurate as of February 2020. Please reach out if you see any corrections that should be made!
1. Create a Linux VM in Azure ¹
- This is the VM that will be running your builds.
- Please note, a Windows machine can also be used, but these steps are specifically for Linux.
2. Create, Prepare, and Mount a Data Disk for Use (Optional, but Strongly Recommended) ²
- This is an optional step that can be run to keep build artifacts and files in a separate directory from files on your main drive.
3. In Azure DevOps, Prepare Permissions Needed to Create Build Pool ³
- In order to prepare your pool, there are permissions that you will need to set, and a personal access token (PAT) that you will need to store.
- For the scope select Agent Pools (read, manage) and make sure all the other boxes are cleared.
- Be sure to keep track of this token, as you will need it to set up your agent, and it will be shown to you only once after generation.
4. Create the Build Agent Pool ⁴
- This is the location within Azure DevOps where the agent(s) that will run your builds will be stored.
- To do this: Go to Azure DevOps → Select the current project → Click “Project Settings” on the bottom- right → Click “Agent Pools” → Click “Add Pool” → Add the Name of the Pool → Click “Create”
5. Specify the Build Pool in Pipelines YAML File ⁵
- In the pipelines YAML file, you must specify the agent pool that you want your job to run in as the one that contains your build VM.
- If your YAML file is set correctly, it should have a line similar to the one below:
pool: name: ${NAME_OF_YOUR_BUILD_POOL_HERE}
6. On the VM, Install the Agent(s) That Will be Running Your Builds ⁶
- NOTE: You can run the script that is referenced here on your build machine, which should do everything covered in step 6. Be sure the file has LF line endings when saved, as it will not run otherwise.
- To download the agent, you will need to download the file located here.
- You will then need to extract the file via the command:
tar zxvf ${directory_file_is_located}/vsts-agent-linux-x64–2.160.1.tar.gz- Run the commands below, as these commands are needed for the agent(s) to be run in the background.
./config.sh # Configures the agentsudo ./svc.sh install # Installs the agentsudo ./svc.sh start # Initiates the agent
- Alternatively, you can run ./config.sh, then ./run.sh, to run the agent interactively.
- You can install as many agents as you’d like on your machine, but please be aware that there may be memory constraints, which depend on the size and processing power of the machine. Also, you’d need to have purchased the ability to run N-1 parallel jobs, in order to utilize all agents in your builds.
7. On the machine, install the dependencies required for the solution
- Identify and install all dependencies on the virtual machine for building your application.
8. Run Your First Build Using a Self-Hosted Agent
- After setup is complete, create your pipeline, and run your branch, ensuring it’s pointed at the pool that contains your newly created agent. If needed, add system diagnostics to your build, to help you diagnose any potential issues you might run into.
Like that, you should be all set with using self-hosted build agents! If any issues come up, please feel free to share them with the community via a site like StackOverflow, Quora, or Microsoft’s Support Center— if it’s a problem for you, it might be a problem for many.
Also, please see my colleague Mahsa Hanifi’s post How to Save Time Building WebRTC, which also explains the benefits of using virtual machines as build agents.
