Registering a cluster to a GKE Enterprise Fleet — will my cluster be OK?
GKE Enterprise (GKEE) is a premium Google Kubernetes Engine (GKE) edition for managing containerized apps at scale across hybrid and multi-cloud environments. And while there are many articles outlining its architecture and features, a conversation I had the other day showed me that there is some confusion around GKEE and Fleets. Specifically around connecting clusters to GKEE and understanding the actions on that cluster, if any, when that connection is completed.
This article will discuss the different scenarios of connecting a cluster to GKEE, and the gotchas you need to be aware of with respect to the impact to the cluster for each. It is hoped that this will help identify the areas where there is friction to adding your clusters to GKEE, and where there is none.
But first — a quick recap
Managed clusters in GKEE can be GKE based or non-GKE based clusters (with support for AKS, EKS and any CNCF compliant Kubernetes cluster).
Clusters in GKEE can be grouped into Fleets, a logical grouping resource. Fleets can then be configured with features. Fleet features include advanced security assessments, configuration and policy management tools, service mesh and networking capabilities, advanced audit tools, scalable permission management, smart metrics, efficient multi-cluster management, and multi-tenancy support.
Reduce Operational load with Fleets
Using Fleets, Platform administrators can reduce the need to perform repeated operations in a multi-cluster environment. Instead, configurations can be implemented once at Fleet level and applied across clusters within the Fleet.
You can see a complete list of available features for each supported Kubernetes cluster type (GKE or non-GKE) in GKE Enterprise deployment options.
So clearly, adding a cluster to a Fleet is good in the way of simplifying a multi-cluster environment.
Ready to go!
So let’s go ahead and connect my cluster to GKE and register to a Fleet.
Wait, not so fast…
Configurations on fleet features are intended to have some effect on the clusters. That’s the point. Whether its policy configuration or security assessment, there will need to be some sort of thought around the configuration of these features. For example — take Policy Controller. It is pre-configured with a set of policy bundles. If this is enabled for the Fleet, some thought needs to go into what is required from this bundle and what is not — e.g. is PCI-DSS compliance required? NIST? What version? Is this configuration required for a subset of clusters, or for all?
And there’s more — Fleet sameness
There is also the concept in GKEE of Fleet sameness which will also require some thought. Sameness means that some Kubernetes objects with the same name in different clusters are treated as the same thing — specifically Namespaces, and subsequently Services and Identity within those namespaces. This facilitates more granular managment of Fleets.
So, for example, services with the same namespace and service name are considered to be the same service enabling features like service mesh and multi-cluster ingress to work smoothly across clusters in the Fleet.
Features that leverage sameness, listed here include cloud service mesh, and multi cluster ingress.
The key thing to be aware of here is that namespace naming is critical. Careful planning is needed to ensure that namespaces with the same name across clusters truly represent the same logical unit, and that there is no clash between namespaces in incoming clusters to GKEE and existing Fleet namespaces.
So maybe multi cluster management is hard.. yes, but easy is dangerous!
I think of it as having layers — managing multiple clusters for deployments, security, networking and governance will require planning and configuration. The more complex the capabilities required over multiple clusters, the more planning has to be done — and rightly so. Configuring something that could end up impacting lots of clusters should require thought, and in my opinion, pose some friction — like planning around what is implemented in GKEE in terms of Fleets, features and sameness. Examples on setting up and planning for Fleets is documented here.
One more thing about fleet registration……
Remember, there are 2 distinct times a cluster can be registered to a fleet.
- After cluster creation. So, a cluster that already exists and is subsequently registered to a Fleet. This cluster is added to the Fleet
- At cluster creation time. So, a cluster has fleet membership as part of its create configuration. This cluster is born in the Fleet
These concepts are important.
So, after all that, will my cluster be OK?
Back to the original question that started this article. Having recapped Fleets and features and sameness, we can see the great benefits that cluster registration to a Fleet can bring — from management, governance, security across a multi-cluster environment.
But — clusters do not have to be registered to a Fleet — at least not right away. Many features can be viewed for your cluster simply through cluster connection to GKEE — including cost estimations, security assessments, capacity utilization and more. Have a look at the components list here, and their requirements.
So the simplest option to get up and running with connecting clusters to GKEE is to just connect the cluster and don’t add it to a Fleet.
The cluster will be just fine!
But what if I do want to register the cluster to a Fleet? What Then?
Of course you do. Fleets provide extra control, management and features as described above. But here is where some friction may come in. Remember the born in Fleet and added to Fleets concepts? Well, Features configured for a Fleet are inherited by default, but ONLY for born in fleet clusters.
Clusters added to a Fleet must be specifically selected in order for a feature to be applied. You have to opt-in. Good to know right? Registering a cluster to an existing Fleet with configurations set will not be automatically applied to a new cluster — it must be selected.
The friction here, or the extra planning needed is to check if there are any namespaces configured with the Fleets and if there are any conflicts with namespaces already configured in the cluster about to be registered to the Fleet.
One more final angle to consider is this.
What happens to clusters born in the Fleet if a default feature is changed?
It’s a default feature, so it gets applied to all clusters on the next sync right?
Wrong.
Again, clusters have to be opted-in specifically to get that update — see information panel within GKEE console below, outlining what happens to new and existing clusters on customizing Fleet settings.
Understand the risk, for all the gains
So simply adding a cluster to GKEE poses no risk. Registering it to a Fleet requires some thought. It will obviously get more complex as you want to add more Fleet features to added clusters, in addition to Fleet identity, services and namespace sameness.
For more information on GKE Enterprise, have a look at the Google documentation here.
