Self-hosted Azure DevOps agent on Azure virtual machine

Azure DevOps offers a nice opportunity to create so called self-hosted agents in addition to (or instead of) predefined ones hosted by Microsoft. You may consider this option because of following advantages it gives:

• 2 agents by the cost of 1. When you start using Azure DevOps you get a free MS-hosted agent which has a limit of 1800 minutes/month. It could be enough for a small project which doesn’t run parallel jobs but bigger projects might suffer from these limitations. A logical move would be to start paying but for $40/mo you’ll only get rid from the minutes limit; if you want to run several jobs simultaneously you’d have to buy another one. A better option could be to set up an Azure VM with a self-hosted agent which would cost somewhat $45–50, but you’d still get your 1800 minutes. Thus, you’d be able to split some jobs between those 2 agents (true parallelization is unavailable ATM).
• More control. Obviously you’d be able to install any software you like on your VM.
• It keeps state. A VM-based agent could save you some build time minutes because it keeps all your files and caches (npm, NuGet etc.). An MS-hosted agent is completely fresh every time you start a build and you always start from scratch. This approach is super-duper true, but most of the time it makes no harm to pull just few latest commits instead of a full repo or to reuse npm packages from the cache. It really saves the time.

Ok, so what’s the plan?

The easiest way of getting a build agent with OS and software installed is to clone an existing one from Microsoft. Happily they post their VM templates on a GitHub repo “Virtual Environments”.

Now, if you have a look at their build agents you’ll notice that they bring lots of stuff, i.e. they are prepared for different development platforms like .NET, Java, Python, C++ etc. I believe it makes sense to edit the template and simply not include the stuff you won’t need. It’ll help to reduce the VM image generation time and required disk space, since we want to have a VM for a comparable price of around $40/mo.

Speaking of which. We’ll build our agent using the B-series Azure VM, namely the B2s, which at the moment of writing was offered by $36.

This approach has one downside worth mentioning. VMs of Azure’s B-serie are this cheap because they are not fully performative all the time. They “typically run at a low to moderate baseline CPU performance, but sometimes need to burst to significantly higher CPU performance when the demand rises”. Such a VM has a certain amount of credits, virtual coins which are spent during high CPU load and collected while it’s idled.

But wait, a build agent is usually like this: we run a job, at that time we need higher CPU, then we simply wait for another one to come, restore our credits and this whole thing sounds like a perfect match!

To give you some overview: it took almost a day of some hard work to drain our VM credits

And another day to recollect them

I’m persuaded, show me the steps

Preparation

• Install packer. Packer is a tool which can create a VM using a configuration file. It can do that on a whole bunch of target platforms and Azure is in that number.
• Create a new service principal in Azure (steps 1–4 of https://medium.com/@stukalin/adding-a-new-azure-service-principal-connection-to-azure-devops-ad2da535bf8f).
• Create new resource group.
• Generate a new GitHub personal access token with the read:packages permission.
• Clone the Virtual Environments repo.
• Now identify your source template. That most likely will be the one you choose among available on Azure DevOps. Template files are stored under /images/<platform> folder. For this tutorial I’ll use the Windows2019-Azure.json. You might want to copy that into a new file, let’s call it agent.json.
• The agent.json is our packer configuration file. It contains a number of variables which we need to set and a sequence of actions we might wanna edit. That’s the next big task.

Edit the packer config file

In the variables section set client_id, client_secret, subscription_id, tenant_id, github_feed_token and resource_group variables with values from steps above. Define and set the location variable (Azure region most suitable for you where the B-series VMs are available). Set the install_password just in case.

In the builders section we wanna set the following:

• name — as you wish
• os_disk_size_gb — perhaps somewhat smaller (128 or even 64)
• rename resource_group_name into managed_image_resource_group_name
• remove storage_account, capture_container_name and capture_name_prefix
• create new managed_image_name key and as its value put the name of the image which will be created, e.g. “ci-agent”

Now go over the provisioners section of the file and remove those elements which do not make sense to your environment. For instance we didn’t need the Docker, Perl, PHP and whole lot of other stuff, so these sections were simply thrown out.

Create the build

Run packer build .\agent.json. For troubleshooting it might be better to use the on-error flag (like -on-error=ask or -on-error=abort). Without it the temporary VM will be deleted on any error.

Create the VM

The most difficult step is over and you will find a new resourse of type image in your Azure portal. Go there and click the Create VM button. Go throught the VM creation process. Perhaps the most important settings are the VM size (here the recommendation is the B2S as the closest to the desired price) and the disk type which could be set as simple standard HDD. The rest is up to you. Make sure you’ve memorized your login and password and enabled the RDP.

Note, if you don’t need the agent to run in the interactive mode you might want to set up the agent software as a VM extension, which you can choose on the Advanced tab. If you need the interactive mode skip this step and set it up over the RDP.

Manually install the agent software and run it in the interactive mode

RDP to the VM using the credentials from the creation step. Follow this instructions to install the agent. After you restart the VM the agent should automatically start and connect to your DevOps server.

And that’s it. Hope that helps anyone and see you next time!