Exploring Azure Kubernetes Service

8 min readOct 27, 2022

Salut.

Kubernetes being the industry standard for container orchestration, it is vital to learn how to set it up in the cloud. This article describes the steps for setting up an Azure Kubernetes Service(AKS) Cluster. The article highlights AKS insights from Richard Hooper in his discussion with hosts Kevin Evans and Robin Smorenburg on StreamingClouds.

Hop on.

To get started, you need to have an Azure account. Azure offers a free trial tier account with 200$ credits for one month for you to explore their platform. Go to portal.azure.com.

Basic Settings

Once you have created your account, in the Search bar, look up Kubernetes Services and click on it. Create a new cluster.

Project Details

Subscription — In Azure, you have subscriptions which could be your logical boundaries between your development and production environments or different applications.

Choose your subscription, then create a new Resource Group

Resource Group — Create a new Resouce Group

Cluster Details

Cluster preset configuration — Click on the drop-down. Standard, Dev/Test, Cost-Optimized, Batch Processing, and Hardened Access are listed options. For this demonstration, we explore the features offered under the Standard Tier.

The Standard Tier is described as the best for getting started to deploy production-ready applications out of the box quickly. It works well with most applications.

Kubernetes cluster name — Name your service cluster.

Availability zones(AZ) — are different data centers inside an Azure region featuring separate power sources, cooling facilities, and buildings. If one zone goes down, your deployed applications can still run from another zone. Azure automatically populates the availability zones depending on the region you are deploying your cluster services.

When you deploy your workloads in AKS, use the availability zones. Otherwise, the workloads could all end up in the same AZ, posing a risk to your application.

Kubernetes version — AKS will always support the last three minor versions and the last two patch versions of each minor version. Also available is the latest version.

API Server availability — 99.95% is efficient enough.

Primary Node Pool

Node Size — The Standard tier offers the DS2 v2. You could also look into the v5 Node size for a faster CPU with a little cost difference.

You do not get a User Node Pool despite this being production ready. Production-ready requires a worker node pool because the system node pool should not be running any of your workloads.

Scale Method — Cluster autoscaling is enabled by default which is good practice.

A private cluster option is not available. A private cluster is recommended if you want a highly secure enterprise-grade set-up with restricted access within your vNet. However, for most deployments, you can configure IP Whitelisting with the Standard Tier configuration unless you are bound by governance within your organization.

Node count range — specifies the minimum and the maximum number of nodes for your service cluster.

Node Pools

The agent node pool is the default system node pool. It always runs Linux and has 110 pods per node by default.

Virtual Nodes — You could enable virtual nodes, allowing burstable scaling in container instances.

Node Pool OS disk encryption — The default setting uses a platform-managed key. However, you can bring your key to a secure environment.

Access

Under access, we configure how to connect to the AKS service cluster.

Resource identity — By default, Azure uses a System assigned managed identity. To use a service principal, use the cli.

Authentication and Authorisation

Local accounts with Kubernetes RBAC — when selected, your AKS cluster will have RBAC enabled inside it so you can use roles and role bindings.

Azure creates a local account inside your AKS cluster, e.g., a cluster admin. Anyone who can access Azure and has the RBAC authentication in Azure to the AKS, e.g., owners and contributors, can then go in as a cluster admin in the AKS cluster.

Azure AD authentication with Kubernetes RBAC — Only members of the selected Azure AD group can become cluster admins. You can also do privileged identity management for the group, so no one is assigned as a global administrator in the tenant. If a user wants to access the cluster with cluster-admin rights, they have to pin into that group for a time with approved reasons.

Azure AD authentication with Azure RBAC — enables you to control your K8s RBAC via the Azure Identity and Access Management (IAM) page in the portal.

Thus instead of creating the Azure roles and role bindings, you can do it from the Azure portal — you create custom roles in the Azure portal, which synchronizes down. This is especially useful if you are coming from an enterprise where you synchronize your enterprise AD up to the cloud, that is, your user groups and entities. You also synchronize office 365 with Azure, and those Azure groups are then synchronized to Kubernetes.

If someone leaves the organization, you only remove them from your local AD and filter them up, denying them access to the Kubernetes Cluster. If you don’t use this option and give someone access inside your Kubernetes Cluster, you would have to log in to Kubernetes, find their role, and delete the role. Otherwise, they would still have access.

Networking

Azure offers two types of network configurations: Kubenet and Azure Container Network Interfaces(CNI).

Azure Container Network Interface — If you want to use windows containers in your AKS cluster, you must pick Azure CNI. Azure CNI allocates any IP addresses you could potentially use in your AKS cluster. We had 110 pods per node and had it scale up to 5 nodes. So that’s (110x5), plus 1 for the node.

We want to ensure that if we use Azure CNI, our IP address range can accommodate that number and allow future growth. You could, for example, use a /18 network. However, enterprise setups limit this because of overlapping IP addresses. They also want to peer. That’s where Kubenet comes in.

Kubenet — is software-defined networking(SDN) inside Kubernetes that ensures that your on-prem IP ranges do not overlap with your Azure IP ranges. It is all NATted inside Kubernetes.

Also, Azure CNI has a new version that is SDN based and dynamically allocates the IP addresses. A Bring your own CNI is also available through Azure CLI, i.e., through Cilium and Calico. Support, however, is 3rd party.

DNS Name Prefix — Can be changed to anything you want, but the default works because it picks the cluster name DNS

Traffic Routing

Load balancing — the standard load balancing feature offers the required services.

Enable HTTP application routing — this is a previous version of the Web Routing Ingress Controller and is designed for dev. It creates a private DNS with a random name to get you up and running quickly in your dev environment. Please do not use it for production environments. Also, it is currently being replaced by Web Application Routing.

Security (everyone’s favorite topic in the world)

Enable private cluster — Everything is vNet integrated and secure in the vNet. Private clusters are necessary when there is a requirement to have a highly secure customer. However, it still offers a secure connection through authorized IP ranges.

Set authorized IP ranges — enables you to lock down IPs that should access the API, i.e., from your VPN, the customer’s VPN, or your hosted Azure DevOps ranges.

Network Policies — With the kubenet network configuration, you get the option of None or Calico. With the Azure CLI network configuration, you can also enable Azure Policies.

Network Policies control communication between namespaces and pods (just like Network Security Groups). They are formatted in YAML manifest files. Calico also works with windows, and you get the benefit of windows container support. If you are going to work with windows at one point, you do not have to relearn a new manifest file system to secure your workloads.

Always enable it at the beginning because once you cannot enable it after building your AKS cluster, you will have to redeploy your whole cluster.

Integrations

Azure Container Registry

Container Registry — container registries are where you store your customer images, like dockerhub. When you create an AKS, Azure creates a managed identity that is given access to pull the container images from the container registry. Alternatively, you can create a new container registry.

Azure Monitor

Container Monitoring — is enabled on the standard tier. When not properly configured, monitoring costs can be expensive; hence keep an eye on it.

Log analytics workspace is always recommended to create one because the default workspace name is not user-friendly.

User-managed identity — When enabled, it uses your managed identity to talk to the log analytics workspace instead of using service principles.

Azure Policy

Azure Policy — should always be enabled to ease management.

Advanced

Enable secret store CSI driver — enables you to inject secrets into your AKS cluster. It creates daemon sets inside the workloads in your AKS cluster.

Review and Create

This step validates all the settings. Click create to provision your cluster.

Following these steps, we created a more secure cluster than we would have if we had just followed the baseline steps. The AKS cluster is not production ready because we did not provision the extra workloads, e.t.c; it is that simple to get started with AKS. We now have a cluster that is dev ready — a few tweaks, and you can get it ready for production. These steps help you to wrap your head around the basic concepts before you get onto automated deployments using Azure Bicep and Terraform.

Post Setup

Your AKS cluster is created, and you get a dashboard view to manage it. You can create a starter application to test your cluster through the Create wizard. Azure automatically deploys the application to your cluster and makes it available through a link.