Setup Azure Kubernetes Service and Container Registry via Private End Point

A private endpoint is a network interface that uses a private IP address from your virtual network. This network interface connects you privately and securely to a service that’s powered by Azure Private Link. By enabling a private endpoint, you’re bringing the service into your virtual network.

Lets provision a secure AKS environment using Private Endpoint.

The only allowed connections will be through AAD Enabled Test VM. AKS and ACR both will be private and accessible only from the private VNET network using Private Endpoint.

High level plan :-

1. Create a private AKS cluster within its own VNET — Step 1
2. Create a Test VM within the same VNET as AKS — Step 2 and 3
3. Test connection to AKS from Test VM — Step 4,5 and 6
4. Configure Private End point for ACR — Step 7
5. Test connection to ACR from Test VM — Step 8 and 9
6. Test connection between AKS and ACR — Step 10and 11

Lets begin the fun…..!!

Step 1 : From the Azure portal, create a new AKS cluster and make sure to enable Private cluster.

In “Integrations” section, add and attach an ACR. Even if AKS is private here, ACR remains public, but we’ll make it private later. Click “Review and Create” to create the resources.

If you want to create from Azure CLI then refer —

Create a private Azure Kubernetes Service cluster - Azure Kubernetes Service

Observation:

Resources created in “privateendpoint” RSG post AKS and ACR Creation

Resources created in “mc_privateendpoint_privateakstest_centralindia” RSG

Especially note Private Endpoint, Network Interface and Private DNS zone created inside the AKS node Resource Group.

This means any resource in the PrivateEndPoint-vnet VNET will be able to resolve the private IP of the Private Endpoint for communication to the API server. If we have a VM in this VNET, it will be able to connect to AKS with no additional steps.

Step 2 : Create a VM in the same VNET as AKS Cluster

Refer below link —

Quickstart - Create a Linux VM in the Azure portal - Azure Virtual Machines

Observation:

Resources created in “privateendpoint” RSG post VM Creation

Step 3 : Login to TestVM

 az ssh vm -n TestVM --resource-group PrivateEndPoint

Step 4 : Install azure cli, docker and kubectl on TestVM

Refer below link —

Install the Azure CLI on Linux

Install Docker

Install Docker Engine on Ubuntu

Verify Docker is installed

Install kubectl

Install and Set Up kubectl on Linux

Now Az login will work

Step 5 : Since a VM is created in the same VNET as AKS. We can directly connect to AKS Cluster. Let’s try it….!!

az account set --subscription 62086a1d-1261-4c50-8a2d-734ed260fd8d
az aks get-credentials --resource-group PrivateEndPoint --name PrivateAKSTest
kubectl cluster-info
kubectl get nodes
kubectl run nginx --image=nginx:1.21.4
kubectl get pods

Step 6 : Confirming a private connection made between VM and AKS via Private End Point/Private DNS.

Api server private IP — 10.224.0.4 defined in Private end point/DNS via which VM connects to AKS.

Step 7 : ACR is Public lets make it Private

Refer —

Set up private endpoint with private link - Azure Container Registry

ACR is Public

Mandatory step — Update SKU to Premium

Set Public network access to Disabled to disable public access to the ACR through the internet. Then create Private end point — Using the private endpoint will allow only resources from our network to access ACR.

Then select Private access and create private endpoint connection —

Observation:

Resources created in “privateendpoint” RSG post VM Creation

Step 8 : Confirm ACR is accessed via private IP.

Private IP confirmed from ACR Private End Point/DNS

Step 9 : Import an image from Microsoft Container Registry (MCR) to ACR

az acr import -n PrivateACRToday - source mcr.microsoft.com/oss/nginx/nginx:1.21.4 - image nginx:1.21.4

Confirmed the image got imported to nginx registry

Lets try to pull an image from MCR, tag it, and push it to ACR.

sudo docker pull mcr.microsoft.com/hello-world

Was able to pull and tag the image but could not push — I did not have access to push image

sudo docker push "PrivateACRToday.azurecr.io/hello-world:v1.0"

Give Push to ACR IAM Access to logged in user

Then get TOKEN for docker push

TOKEN=$(az acr login --name PrivateACRToday --expose-token --output tsv --query accessToken)

Finally login to ACR with the TOKEN and Push image.

sudo docker login PrivateACRToday.azurecr.io --username 00000000-0000-0000-0000-000000000000 --password $TOKEN

CONFIRMATION — Image got pushed to ACR

az acr repository list --name PrivateACRToday --output table
az acr repository show-tags --name PrivateACRToday --repository nginx --output table
az acr repository show-tags --name PrivateACRToday --repository hello-world --output table

Step 10 : Confirm AKS is able to connect to Private ACR

ACR’s Private Endpoint is in the same AKS VNET, hence with no effort AKS should have access to ACR Private Endpoint

kubectl get pods
kubectl run privateacrnginx --image="PrivateACRToday.azurecr.io/nginx:1.21.4"

Step 11 : Last but not the least — Disable Public FQDN of API Server on Private Cluster

az aks update -n PrivateAKSTest -g PrivateEndPoint --disable-public-fqdn

That’s about setting up Azure Kubernetes Service and Container Registry via Private End Point……Lets keep learning together .. Lets sail together.