AKS as Azure DevOps agents- Intro
You might have heard of AKS for deploying apps, but did you know it can also improve your DevOps process? In this article, we’ll dive into how we can use AKS as agents specifically in azure devops, to make the DevOps life easier.
AKS isn’t just for deploying apps. It’s like a magic box of containers and these containers are perfect for your DevOps agents. Why? Because they’re like mini VMs that can run your tasks without messing up anything else. The isolation and consistency in image configuration suits well for running a pipeline. Following are the other advantages of using kubernetes as devops agents. This article is an intro for resolving the aks docker build task limitations.
Advantages
- Scalability on demand and improved Autoscaling : Utilizing Kubernetes Event-driven Autoscaling (KEDA), AKS can dynamically scale agent pods based on Azure DevOps job events, optimizing resource allocation and ensuring efficient workload handling.
- Cost Savings: AKS allows multiple agents to run on a single virtual machine (VM), reducing infrastructure costs compared to the traditional approach of one VM per agent.
- Auto Garbage Collection and Auto Healing: AKS takes advantage of Kubernetes’ auto garbage collection and auto healing features, decreasing the risk of memory failures and enhancing system reliability.
- Consistent Configuration: With AKS, all agent pods share the same configuration due to containerization. This minimizes the potential for configuration errors, resulting in predictable and consistent agent behavior.
- Enhanced Isolation and Security: The containerized nature of AKS ensures strong isolation between different agent tasks. This isolation minimizes the risk of conflicts and enhances security, as issues in one agent won’t affect others.
- Simplified Maintenance: AKS handles updates and patches for underlying infrastructure, reducing the operational burden of managing and maintaining the agent environment.
Incorporating these advantages enhances your CI/CD pipeline’s efficiency, reliability, and cost-effectiveness through the utilization of AKS’s containerized environment and Kubernetes orchestration capabilities.
Limitation
Using Docker tasks on AKS agents faces hurdles due to specific restrictions:
- Security Risks with Docker Socket: Employing Docker tasks from within a container necessitates binding the Docker socket. Unfortunately, this action poses serious security threats. It permits the code inside the container to operate as the root user on the Docker host.
- Incompatibility with AKS 1.19+: Docker-based tasks won’t function on AKS versions 1.19 and beyond. This is due to the shift from Docker to containerd in Kubernetes 1.19, rendering Docker-in-Docker functionality unavailable.
Steps for configuring AKS
- Identify the following values: (Can be added as env values)
+----------------+------------------------------------------------------------------------------------------------------------------------------------------+
| Variable | Description |
+----------------+------------------------------------------------------------------------------------------------------------------------------------------+
| AZP_URL | The URL of the Azure DevOps or Azure DevOps Server instance. |
| AZP_TOKEN | Personal Access Token (PAT) with Agent Pools (read, manage) scope, created by a user who has permission to configure agents, at AZP_URL. |
| AZP_AGENT_NAME | Agent name (default value: the container hostname). |
| AZP_POOL | Agent pool name (default value: Default). |
| AZP_WORK | Work directory (default value: _work). |
+----------------+------------------------------------------------------------------------------------------------------------------------------------------+Note: Obtain AZP_TOKEN as mentioned in the doc, Token should have the access: “Agent Pools (read, manage)”, other scopes can be deselected. https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops#authenticate-with-a-personal-access-token-pat
- Create and configure agentpool in devops portal
https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/linux-agent?view=azure-devops#download-and-configure-the-agent - Configure agentpool in aks following the docker article.
Run a self-hosted agent in Docker — Azure Pipelines | Microsoft Learn
Elaborating on the AKS agent setup isn’t necessary as referring to the official documentation should suffice. If you have any concerns, please feel free to mention them in the comments section.
Solving docker build task limitation.
- Running buildkit daemon in separate pod (Ideal approach)
Buildkit — AKS as Azure DevOps agents | Medium - Kaniko for build tasks instead of docker (Test approach)
Kaniko for build tasks — AKS as Azure DevOps agents | Medium - Using acr tasks for building images (Easiest to enable)
Ref: https://learn.microsoft.com/en-us/cli/azure/acr?view=azure-cli-latest#az-acr-build
Builds the image in azure environment and pushes the image. Requires further configuration for custom agentpool when building (Preview option) https://learn.microsoft.com/en-us/azure/container-registry/tasks-agent-pools
Disadvantage: Requires additional VM just for processing the build commands if using custom agentpool Self hosted option is in preview
Dig deeper…
- About azure devops agents: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/agents?view=azure-devops&tabs=browser
