Over the past few weeks I’ve been working extensively with all three technologies. Here are some of my notes:
- Terraform’s kubernetes provider is great for initializing the k8s cluster (cloud agnostically), but that’s about as far as you can reasonably go with it. In other words, provision with Terraform, orchestrate app with K8s via kubectl and yaml files.
- An auto-generating/renewing certbot flow is painful. No great solutions AFIK. +1 for AWS handling certs for users out-of-the-box — would love to see GCP offer something similar for it’s L7 load balancer.
- The non-immutable parts: Configuring stateful application volumes and secrets within k8s and terraform is not trivial. Especially, GCP’s awesome managed Postgres instance. Due to this, using a standard postgres/deployment/service configuration is much more straight forward but fully managed would be preferable for critical web apps.
- GCP’s load balancer used with the Ingress object is powerful. Specifically, the recommended GCE Balancer rather than the k8s nginx Load Balancer service.
- Kubernetes deployment rolling updates are lovely.
- Terraform does not solve for cross-cloud abstraction, but it does a great job at cloud abstraction in general.
- Terraform’s ability to make DNS management immutable gives great flexibility for creative deployments.
- Both K8s and GCP are incredibly fast moving technologies. For a team to adopt, they must plan on keeping up.
- Working in GCP is a pleasure compared to AWS. There is a level of thoughtfulness and engineering that is just not available with any of their competitors IMO.
- Light weight docker images (w/ alpine) make for much faster deployments. Always use if applicable.
- GitLab’s private registry fits very well into this tooling stack.
