I recently broke our AWX installation, which meant provisioning workflows ground to a halt! We were hit by a bug running on Docker (https://github.com/ansible/awx/issues/3705) that meant all secret keys had been deleted, rendering the database useless. I could have restored from a VM backup, recovered the files and fixed the compose file — but what better time than now to deploy the latest version using the Ansible Operator on a Kubernetes stack!
I wanted to jot this down as I spent a day with the AWX documentation and struggled to get a working install
In this tutorial, we’ll walk through:
- Installing MicroK8s on Ubuntu 20.04
- Enabling required MicroK8s Addons
- Deploying the AWX Operator
- Deploying AWX
Installing MicroK8s on Ubuntu 20.04
What you’ll need:
- Ubuntu 20.04 LTS or 18.04 LTS environment
- 15 minutes
- 1 coffee
Let us dive right in!
- Install MicroK8s
sudo snap install microk8s --classic --channel=1.21
- Add your user to the microk8s group, and grant permission to the ~/.kube caching directory
sudo usermod -a -G microk8s $USER && sudo chown -f -R $USER ~/.kube
- Verify the installation was successful
microk8s status — wait-ready
Continue when you seemicrok8s is running
- MicroK8s uses a namespaced
kubectl
command so let us add an alias to make future commands easyecho “alias kubectl=’microk8s.kubectl’” >> ~/.bash_aliases && source ~/.bash_aliases
- Let’s verify we can run kubectl commands and ensure node status is Ready
kubectl get nodes
Enabling required MicroK8s Addons
Now that we have a working Kubernetes installation we can proceed with enabling the required add-on services for ingress routing, container storage, and DNS.
I have decided to use Traefik as we should be able to automate SSL certificates easily!
MicroK8s makes this all super simple!
- Enable Addons — This will take a few minutes,
microk8s enable dns storage traefik
- Verify all pods are running, we will need the
--all-namespaces
flag here as these services don't run in the default namespacekubectl get --all-namespaces pods
- Verify Storage is ready for use
kubectl get storageclass
Did I say how simple this was? It’s also what caught me out for a day trying to get the stack working! (It’s *always* DNS)
Deploying the AWX Operator
The AWX Operator creates a CRD (Custom Resource Definition) that extends the Kubernetes API with an object that describes an AWX installation. The operator also deploys a pod that watches for these objects and performs the actual installation of AWX.
Deploying the operator
This will take a few seconds to complete
kubectl apply -f https://raw.githubusercontent.com/ansible/awx-operator/0.8.0/deploy/awx-operator.yaml
Verifying the installation
After a few seconds, the installation should be ready for use. You'll need the name of the operator pod for the next few steps, so let us get that handy!
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
awx-operator-f768499d-65bt7 1/1 Running 0 17h
Once the pod is running, we can move on to deploying AWX
Deploying AWX
In a second terminal, let us start watching the AWX operator logs to monitor the deployment.
kubectl logs -f awx-operator-f768499d-65bt7
remember to use the name from your pod!
We now need a YAML file that describes our AWX installation. You will need to pick a domain name for the AWX web interface and ensure DNS points to your server IP.
$ cat << EOF > awx.yaml
---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
name: awx
spec:
tower_ingress_type: Ingress
tower_hostname: awx.mycompany.com
EOF
We can now deploy AWX!
$ kubectl apply -f awx.yml
awx.awx.ansible.com/awx created
The AWX database migrations will take a few minutes to complete. After a few minutes, the pods should be downloaded, running and ready for use.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
awx-operator-f768499d-65bt7 1/1 Running 0 17h
awx-postgres-0 1/1 Running 0 17h
awx-b5f6cf4d4-7jftg 4/4 Running 0 17h
Once all 6 pods are running, we should now be able to access AWX. We’ll need to get the admin password first
$ echo Username: Admin$'\n'Password: `kubectl get secret awx-admin-password -o jsonpath='{.data.password}' | base64 --decode`
Username: Admin
Admin Password: jhnjhjhGThytyuijhgr56789oiuyf
Traefik listens on port 8080 by default, but you should now be able to access AWX using the credentials from above at your URL https://awx.mycompany.com:8080/
Thanks for reading!
Now it is on to the next problem — repairing my playbooks to work on the newer version of AWX!