Google Cloud Anthos Series: Anthos Config Management

Google Cloud Anthos Series: Part-5

Welcome to Part-5 of the ‘Google Cloud Anthos series’. You can find the complete series Here.

Set the PROJECT_ID environment variable and ensure the Google Kubernetes Engine and Cloud Operations APIs are enabled.

PROJECT_ID="<your-project-id>"
gcloud services enable container.googleapis.com --project ${PROJECT_ID}
gcloud services enable monitoring.googleapis.com \
    cloudtrace.googleapis.com \
    clouddebugger.googleapis.com \
    cloudprofiler.googleapis.com \
    --project ${PROJECT_ID}

Clone the Online Boutique repository.

git clone https://github.com/GoogleCloudPlatform/microservices-demo.git
cd microservices-demo

Create GKE clusters in three different regions. Make sure you have a VPC which spans across at-least 3 regions where you would want your clusters to reside.

ZONE1=us-central1-b
ZONE2=europe-west1-b
ZONE3=asia-south1-b
gcloud container clusters create us-gke-cluster \
    --project=${PROJECT_ID} --zone=${ZONE1} \
    --machine-type=e2-standard-2 --num-nodes=4 \
    --scopes=cloud-platform \
    --workload-pool=${PROJECT_ID}.svc.id.goog
gcloud container clusters create eu-gke-cluster \
    --project=${PROJECT_ID} --zone=${ZONE2} \
    --machine-type=e2-standard-2 --num-nodes=4 \
    --scopes=cloud-platform \
    --workload-pool=${PROJECT_ID}.svc.id.goog
gcloud container clusters create asia-gke-cluster \
    --project=${PROJECT_ID} --zone=${ZONE3} \
    --machine-type=e2-standard-2 --num-nodes=4 \
    --scopes=cloud-platform \
    --workload-pool=${PROJECT_ID}.svc.id.goog

Enable Anthos Config management API

gcloud beta container hub config-management enable

Create a Cloud Source repository

gcloud source repos create <repo name>

Make sure the default service account PROJECT_NUMBER-compute@developer.gserviceaccount.com for the cluster has the source.reader access to the repository.

PROJECT_NUMBER=$(gcloud projects describe $PROJECT_ID --format="value(projectNumber)")
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member serviceAccount:${PROJECT_NUMBER}-compute@developer.gserviceaccount.com --role roles/source.reader

Push the Online Boutique application to the newly created cloud source repository.

Deploy the application to all the clusters.

kubectx gke_${PROJECT_ID}_${ZONE1}_us-gke-cluster
kubectl apply -f ./release/kubernetes-manifests.yaml
kubectx gke_${PROJECT_ID}_${ZONE2}_eu-gke-cluster
kubectl apply -f ./release/kubernetes-manifests.yaml
kubectx gke_${PROJECT_ID}_${ZONE3}_asia-gke-cluster
kubectl apply -f ./release/kubernetes-manifests.yaml

We need to create constraint and push the file to the cloud source repository.

Here is a sample constraint which will deny creation of any Privileged containers.

Folder structure: /allpolicies/policies/policy.yaml (screenshot)

#policy.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: K8sPSPPrivilegedContainer
metadata:
  name: psp-privileged-container
spec:
  match:
    excludedNamespaces:
    - kube-system
    kinds:
    - apiGroups:
      - ""
      kinds:
      - Pod

Push the policy to the source code repository.

Install Anthos Config Management and configure the policy controller through the GKE console.

Get the endpoint of the repository using the below command. We will need it in next step.

gcloud source repos list

Paste the URL of your Cloud source repository as below and click on Show Advanced options.

You can verify that Config Sync is syncing this constraint to your GKE clusters.

kubectx gke_${PROJECT_ID}_${ZONE1}_us-gke-cluster
kubectl get constraint
kubectx gke_${PROJECT_ID}_${ZONE2}_eu-gke-cluster
kubectl get constraint
kubectx gke_${PROJECT_ID}_${ZONE3}_asia-gke-cluster
kubectl get constraint

You should be able to see the output as below.

Now, let’s create a manifest (privileged.yaml) for pod which has privileged container.

#privileged.yaml
apiVersion: v1
kind: Pod
metadata:
 labels:
   app: nginx-privileged
 name: nginx-privileged-disallowed
spec:
 containers:
 - image: nginx
   name: nginx
   securityContext:
     privileged: true

Try to deploy the manifest which violates the policy to any of the cluster.

kubectx gke_${PROJECT_ID}_${ZONE1}_us-gke-cluster
kubectl apply -f privileged.yaml

It should fail with the below error.

Error from server (Forbidden): error when creating "privileged.yaml": admission webhook "validation.gatekeeper.sh" denied the request: [psp-privileged-container] Privileged container is not allowed: nginx, securityContext: {"privileged": true}

If we run the constraint in dryrun mode, we can view the violations with the below command in status field.

kubectl get K8sPSPPrivilegedContainer psp-privileged-container -o yaml

Coming up..

In this blog we discussed Anthos Config Management. In upcoming blogs we will continue the Samajik’s journey with other Anthos features.

Contributors: Shijimol A K, Pushkar Kothavade, Dhandus