Managing multi-cloud Kubernetes with Anthos Attached Clusters
Managing multiple Kubernetes clusters across clouds is hard… but for Google Cloud users, Anthos attached clusters can make life that little bit easier.
So… what’s the problem?
Fancy operating and governing multiple Kubernetes distributions across different cloud platforms? Making sure they are all up to date with consistent configurations and security posture? Nope, me neither — but increasingly it’s an issue facing organisations across many industry sectors. A few years ago, your poor old CISO and Ops teams just had to look after all the tin in their data centre(s). Now they find themselves staring at a sprawl of k8s clusters (if they can find them), running who-knows-what workloads, with different setups, distros, registries, CICD, configurations, RBAC, etc. And as we all know, k8s is a mighty powerful, flexible, and configurable beast — but having visibility and getting that config and security right is your responsibility.
Given the above — you’d have to say anyone who chooses to make their life more difficult by operating across clouds and their various k8s flavours must either have taken a wrong turn or otherwise must a) be mighty good at what they do and b) have some very good reasons for doing it. Staying “cloud-neutral” is sometimes talked about (and hey, k8s is that kinda tech), but you generally end up having to compromise a lot in terms of tooling/services in order to maintain that neutrality. In general, if you’re all in on a cloud — if it does what you need at a decent price with the features that you need, and you’re invested — the general consensus is that you may as well stick with it and leverage all that that platform has to offer.
So how does it happen?
Well, generally it’s a mix of the unavoidable and the accidental, rather than a chosen strategy. Sometimes its regulatory (e.g. some financial services organisations and government bodies are mandated to operate across clouds to mitigate ‘risk’); sometimes it may be for access to geographies where your primary cloud does not have availability; perhaps your workloads have to be next to a specific SaaS service that’s only offered in one cloud? All very valid reasons — however, accidental multi-cloud — through company acquisition, shadow IT, or even just team autonomy and stakeholder preferences — is something we are starting to see more and more of. Add to that the ever-increasing popularity of k8s, and… shazam! You’re a single organisation faced with cluster sprawl across multiple clouds. And you, dear reader, have to find a way to look after them all…
I see — Anthos clusters to the rescue then?
Well, not quite. Yes, with Google Anthos you can deploy GKE clusters on Azure, AWS, VMWare, or good ol’ bare metal — but ripping up the road and moving to a brand new k8s distro is going to be a hard sell to those teams who may be already operating reasonably happily on AKS, EKS, OpenShift, k3d clusters etc etc on their various clouds. I mean, there’s probably not going to be much in it for them; from their perspective, your quest for normalisation and control will exist in a S.E.P. field.
But you said Anthos?
Yes — but here we’re talking Anthos attached clusters. This rather smart technology allows you to attach any compliant cluster to the Anthos control plane, just like ‘standard’ Anthos (i.e. GKE distro) clusters. This gives you access to a whole heap of unique Anthos capabilities around managing the configuration of your clusters, just like you can with GKE clusters in your Google Cloud project or elsewhere.
So how does it work exactly?
An agent (‘Connect Agent’) is deployed as a pod to each cluster to be attached. This acts as the “phone home” element, securely transmitting encrypted information about the cluster state back to the Anthos control plane, as well as enabling access to the Anthos cluster and workload management features. With this in place, the cluster can be put under much of the same control as ‘standard’ Anthos clusters — meaning you get access to all of the rather spiffing Google toys around observability, configuration management — you can deploy a service mesh, sync configs, apply policies, even deploy apps that can then use Google Cloud services in the same Google Cloud project, should you wish.
Got it — now give me some examples of why I need it
Well ok, as you’re asking…
Observability — so first up, you’re going to want to be able to see what’s going on in your clusters in near real-time. Attaching your clusters to the Anthos control plane gives you a similar amount of insight into running workloads, resources, and hardware that you do for your standard GKE clusters.
The development of this would be to employ a service mesh for enhanced telemetry, plus traffic shaping and security— and yes, workloads on attached clusters can form part of a managed service mesh (i.e. Anthos Service Mesh)
Configuration and policy control — as referred to earlier, managing cluster configurations at scale can be an awkward and labour-intensive business. The good news is your attached clusters can be first-class citizens when it comes to being compliant fleet members via Anthos Config Management — you can define your configurations as YAML in git repos and know these are being continuously synced in your clusters, providing a declarative security posture that your SecOps folks can review to their heart's content. And as for workloads making their way onto these clusters — you can use Policy Controller to act as an admission controller, preventing anything that's inconsistent with your declared security policies (again in Git repos) is getting deployed. By the way, those policies can cover your service mesh behaviour too.
Leveraging Google services — once your cluster is attached to your Anthos control plane, it’s really operating as an extension of your Google Cloud project — you can use the same Workload Identity constructs (via Workload Identity Federation) as you use on your GKE clusters, aligning with Google best practice for securely managing service access for cluster workloads. And if you’re savvy enough to be already operating on Google, you can leverage services like Cloud Build and Artifact Registry for building your containers, and then use the rather excellent Google Cloud Deploy to deploy your apps to clusters wherever they might be located. You can even hit other Google APIs that are enabled on your project should you wish, like Cloud Vision (admittedly, whether this is practical for you across clouds is another matter, but I suspect with the advent of features like dedicated cross-cloud connections with Cross-Cloud Interconnect, it’s going to become more of a conceivable use case).
So what do I need — and what will it cost me?
Well, essentially all you need is a Google Cloud project with the Anthos API and associated multi-cloud APIs enabled, appropriate IAM permissions for the multi-cloud API functions needed, and connectivity to the cluster you wish to attach to.
You first register the cluster with an Anthos fleet; fleets are logical collections of clusters that can be considered to have a degree of ‘sameness’, such as environment type, teams, or locale. The fleet construct is what allows you to apply consistent management across multiple clusters via the Anthos control plane — you can define common configurations and policies for your clusters in code and have them continuously synced to all the clusters in the fleet.
You then need to configure the installation of the ‘phone home’ connect agent, add some logging configuration IAM to enable log forwarding, and get the control plane to install logging and monitoring agents on the clusters as you wish — and that's pretty much it. You’re free to start using all of the Anthos goodies, including Anthos Config Management, Policy Controller, and Anthos Service Mesh — as well as integrating with any other Google services you may be using already.
As regards the cost — well, on the Google side, for the privilege of having a cluster belong to an Anthos fleet on PAYG is a princely $8 per month vCPU. Considering all the features you could take advantage of, that seems pretty reasonable for the value add. (Of course, whatever infra your clusters consume on other clouds is still on your dollar — but you’re paying that already).
So what does all this mean?
Well, whatever you want it to, if nothing else, it demonstrates Google’s commitment to supporting a multi-cluster, multi-cloud reality that more and more orgs are going to face. (If you want to see some really interesting thinking around this using Anthos, attached clusters, and Crossplane to completely manage clusters across clouds, check this out).
But regardless — as I’ve mentioned in previous blogs — if you are operating multiple k8s clusters, you really need to be exploring some kind of scalable, automated solutions for configuration management, observability, telemetry, and policy control, or else you're going to have problems. Add in the dimension of multiple clouds, and those problems are going to multiply geometrically. If one of those clouds is Google, you may have a compelling and powerful solution to explore right here.
About CTS
CTS is the largest dedicated Google Cloud practice in Europe and one of the world’s leading Google Cloud experts, winning 2020 Google Partner of the Year Awards for both Workspace and GCP.
We offer a unique full stack Google Cloud solution for businesses, encompassing cloud migration and infrastructure modernisation. Our data practice focuses on analysis and visualisation, providing industry specific solutions for; Retail, Financial Services, Media and Entertainment.
We’re building talented teams ready to change the world using Google technologies. So if you’re passionate, curious and keen to get stuck in — take a look at our Careers Page and join us for the ride!
