How a Pod assumes an AWS identity?
In my previous post, I talked about how an AWS identity can be delegated the permission to manage a Kubernetes namespace. This time, I am going to do the reverse, implement a role based access for a pod in AWS EKS to assume an IAM role for consuming AWS services.
How it works
This is achieved by associating an IAM role with a Kubernetes service account. By authenticating with an OpenID Connect (OIDC) that supports federated identities, a valid OIDC JSON web token (JWT) is returned and passed to AWS STS AssumeRoleWithWebIdentity
API operation, where an IAM temporary role credential is generated in return.
After that, the K8s mutating admission controller in EKS (via a webhook) will inject the temporary role credentials in the form of environment variables AWS_ROLE_ARN
and AWS_WEB_IDENTITY_TOKEN_FILE
into pods based on the annotation on the Service Account used by the pod. Thus the pods are given the credentials for making subsequent AWS API calls.
Example setup
For illustration, I will make use of official AWS CLI docker image ( amazon/aws-cli) to demonstrate the IAM permission. I will grant the pod the needed IAM permission to make AWS CLI commands for EC2.
I will focus on the permission setup and will not go into the details of how to run the image in a pod container. If you are new to Kubernetes, I strongly suggest you visiting https://kubernetes.io/docs/concepts/workloads/pods/
Some commands are making use of eksctl which you can find at https://eksctl.io/
Before the permission setup, let’s see whether our docker container can make any AWS EC2 calls.
kubectl run --image amazon/aws-cli awscli -n <serviceAccountNamespace> ec2 describe-instances
The aws-cli container will try to describe ec2 instances, but it failed:
Next we proceed our setup.
- Create an OIDC provider for your cluster.
eksctl utils associate-iam-oidc-provider --cluster <clusterName> --approve
Note the OIDC provider:
OIDC_PROVIDER=$(aws eks describe-cluster --name <clusterName> --query "cluster.identity.oidc.issuer" --output text | sed -e "s/^https:\/\///")
2. Create a new namespace for running docker pod. (Optional)
kubectl create ns <serviceAccountNamespace>
3. Setup a Kubernetes service account for the namespace.
Every namespace has a default service account resource called default
. When a pod is created, if a service account is not specified, it is automatically assigned to run under the default
service account in the same namespace. You can use the default service account for learning purpose. But it is advisable to create a separate service account for each of your application for better management.
Create a new service account:
kubectl create sa <serviceAccountName> -n <serviceAccountNamespace>
Example yaml file for specifying the service account to use for running a pod:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
serviceAccountName: serviceAccountName
...
4. Create an IAM role and attach IAM policy for designated AWS services access. (In our case, we will use the AWS managed policy named “arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess”)
Copy the following code block to a file named trust.json
Replace with your ACCOUNT_ID, OIDC_PROVIDER, serviceAccountNamespace and service AccountName with the above values. Note the StringEquals statement will confine the access to match a particular service account in that namespace only.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::<ACCOUNT_ID>:oidc-provider/${OIDC_PROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"<OIDC_PROVIDER>:sub": "system:serviceaccount:<serviceAccountNamespace>:<serviceAccountName>"
}
}
}
]
}aws iam create-role --role-name <IAM_ROLE_NAME> --assume-role-policy-document file://trust.json --description "<IAM_ROLE_DESCRIPTION>"
Next assign the “AmazonEC2ReadOnlyAccess” policy to this new IAM role.
aws iam attach-role-policy --role-name <IAM_ROLE_NAME> --policy-arn arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess
5. Associate the IAM role with the service account by adding the following annotation to the service account.
kubectl annotate sa -n <serviceAccountNamespace> <serviceAccountName> eks.amazonaws.com/role-arn=$IAM_ROLE_ARN
When we describe the service account again, we see that the service account include the ARN of the IAM role as an annotation.
5. The Amazon EKS Pod Identity Webhook on the cluster will apply the aforementioned environment variables AWS_ROLE_ARN
and AWS_WEB_IDENTITY_TOKEN_FILE
to the new pods that are running under this service account with this annotation.
You can examine the AWS credential environment variables set in the pod.
kubectl exec -n <serviceAccountNamespace> <podName> env | grep AWS
6. Relaunch your aws-cli pod and verify the pod’s IAM permission by issuing AWS CLI command to verify EC2 permission.
kubectl run --image amazon/aws-cli awscli -n <serviceAccountNamespace> ec2 describe-instances
You should be able to see a list of EC2 instances if any.
This conclude the EKS identity mapping to AWS IAM role.
Be sure to visit us at https://airwalkconsulting.com/ if you want to know more about digital transformation.