Mastering AKS Resource Cost: A Deep Dive into Cost Optimization with AKS Cost Analysis (preview)

An Azure Kubernetes Service (AKS) cluster is reliant on Azure resources like virtual machines, virtual disks, load-balancers and public IP addresses.
Resource consumption patterns of those applications are often nonuniform, and thus their contribution towards the total cluster resource cost is often nonuniform. Some applications can also have footprints across multiple clusters.
This can pose a challenge when performing cost attribution and cost management.
Third-party solutions, like Kubecost or OpenCost, can be used to gather and analyze resource consumption and costs by Kubernetes-specific levels of granularity, such as by namespace or pod.

AKS has integrated with MCM to build the The AKS cost analysis addon which offer detailed cost drill down scoped to Kubernetes constructs, such as cluster and namespace, in addition to Azure Compute, Network, and Storage categories.

In this article in order to test this new AKS preview features , we will create an azure aks kubernetes service,deploy some applications on differntes namespaces , enabled the AKS cost analysis addon and finally check the cost analysis dhasboard.

Prerequisites

• The aks cluster must be either Standard or Premium tier, not the Free tier.
• You must have one of the following roles on the subscription hosting the cluster: Owner, Contributor, Reader, Cost management contributor, or Cost management reader.
• Your cluster must be deployed with a Microsoft Entra Workload ID configured.
• If using the Azure CLI, you must have version 2.44.0 or later installed
• You need to have the aks-preview Azure CLI extension version 0.5.155 or later installed.
• The ClusterCostAnalysis feature flag must be registered on your subscription.
• Kubernetes cost views are available only for Enterprise Agreement or Microsoft Customer Agreement Microsoft Azure Offer types.

Prepare our environment

A- Deploy the infrastructure

To test this new AKS preview features ,We need first to create an aks cluster and deploy some applications in this cluster .
To build the aks cluster we can use automate provisionnig tools such as Terraform ,for this demostration I will use the azure cli .

1- Create the aks cluster

 az group create - name rg-aks-test -l "France Central"

2- Create a virtual network and a subnet

az network vnet create --name vnet-aks --resource-group rg-aks-test --address-prefix 10.0.0.0/16  --subnet-name subnet-aks --subnet-prefixes 10.0.0.0/18

3- Create an aks cluster, with managed identity enabled

aks create -g rg-aks-test -n my-aks-test --kubernetes-version 1.27.7 --enable-managed-identity  --enable-azure-rbac --vnet-subnet-id "/subscriptions/xxxxxxxxxxxxxx/resourceGroups/rg-aks-test/providers/Microsoft.Network/virtualNetworks/vnet-aks/subnets/subnet-aks"

4- Connect to the cluster and check the aks deployment :
Get access credentials for a managed Kubernetes cluster. By default, the credentials are merged into the .kube/config.Run the kubectl get command to chek connectivity to the cluster.

$az aks get-credentials -n my-aks-test -g rg-aks-test 
$kubectl get nodes
NAME                                STATUS   ROLES   AGE     VERSION
aks-nodepool1-18395102-vmss000000   Ready    agent   2m6s    v1.27.7
aks-nodepool1-18395102-vmss000001   Ready    agent   115s    v1.27.7
aks-nodepool1-18395102-vmss000002   Ready    agent   2m30s   v1.27.7

B- Deploy some application into the aks cluster

For this demo , . We will use an application called “ Emojivoto” , it uses a mix of gRPC and HTTP calls to allow the user to vote on their favorite emojis.
You can clone the application git repos from here : https://github.com/linkerd/website/blob/main/run.linkerd.io/public/emojivoto.yml.
I have made some changements in the emojivoto.yml Yaml file to deploy each applicaitons in a deffirents namesapce :emoji , voto and web instead od one namespace.

$kubectl apply -f emoji-voting-app.yaml
 
$kubectl get deploy -A
NAMESPACE     NAME                 READY   UP-TO-DATE   AVAILABLE   AGE
emoji         emoji                1/1     1            1           2m24s
kube-system   coredns              2/2     2            2           6m13s
kube-system   coredns-autoscaler   1/1     1            1           6m13s
kube-system   konnectivity-agent   2/2     2            2           6m13s
kube-system   metrics-server       2/2     2            2           6m12s
voto          vote-bot             1/1     1            1           2m24s
voto          voting               1/1     1            1           2m23s
web           web                  1/1     1            1           2m23s

Note : to visit the Emojivoto application , you can use :

$ kubectl port-forward service/web-svc -n web 8080:80
Open a web browser and point to localhost:8080

Test the AKS cost analysis features

To test this features, we need to follow this steps :
1- First of all we need Install the aks-preview Azure CLI extension.

$az extension add --name aks-preview 

$az extension update --name aks-preview 

2- Register the ‘ClusterCostAnalysis’ feature flag

$az feature register --namespace "Microsoft.ContainerService" --name "ClusterCostAnalysis" 

$az feature show --namespace "Microsoft.ContainerService" --name "ClusterCostAnalysis"
{
  "id": "/subscriptions/xxxxxxxxxxxxxxxx/providers/Microsoft.Features/providers/Microsoft.ContainerService/features/ClusterCostAnalysis",
  "name": "Microsoft.ContainerService/ClusterCostAnalysis",
  "properties": {
    "state": "Registering"
  },
  "type": "Microsoft.Features/providers/features"
}
$az provider register --namespace Microsoft.ContainerService

3- Enable cost analysis on your AKS cluster
Like we have deployed the cluster with the default sku ; the my-aks-test cluster is right now in the “free” tier , to be aiable to use the aks cost analysis you need to have a cluster with a “standard” or “Premium” tier.

Note : An aks cluster with a “standard” tier adds an additional cost of $73 per month

Bellow I am using the az aks update command to change the aks tier and to enable the cost analysis addon :

$az aks update --name my-aks-test --resource-group rg-aks-test --tier standard  --enable-cost-analysis
Argument '--enable-cost-analysis' is in preview and under development. Reference and support levels: https://aka.ms/CLI_refstatus
The behavior of this command has been altered by the following extension: aks-preview
 / Running ..

Note : It might take up to one day for data to finalize

View cost information

You can view cost allocation data in the Azure portal , We need to navigate to Subscriptions and search the one hosting our aks cluster
In the View list, We have select the list drop-down item and then select Kubernetes clusters.We will have three views : the kubernetes cluster view , the namespace view and Kubernetes assets view .

1- The Kubernetes clusters view

The Kubernetes clusters view shows the costs of all clusters in a subscription. For our example we have only one clustehosting in this subscription ,With this view, you can drill down into namespaces or assets for a cluster

2- Kubernetes assets view

The Kubernetes assets view shows the costs of assets in a cluster categorized under one of the service categories: Compute, Networking, and Storage. The uptime SLA charges are under the Service category.

3- The namespace view :

The Kubernetes namespaces view shows the costs of namespaces for the cluster along with Idle and System charges. Service charges, which represent the charges for Uptime SLA, are also shown.

Based on this informations, we will be able to manage the costs at the namespace level (application level) and make some decisions in order to optimize costs.

that’s all folks 👏 thanks for reading 🙏