In my years working with Google Cloud a problem that has come up time and time again is how to handle service account keys. I’ve seen and heard all kinds of horror stories where service account keys were mishandled and exposed because despite companies spending $$$ on obtaining and implementing secrets management tools, humans are going to human. New people may not have gotten trained yet. Or someone might be focused on getting something done fast vs. doing something right. It’s a human problem as much as it is a tool problem. Google recognized this as an important issue to solve permanently and has released a number of features specifically to deprecate any need for service account keys. If you are a Google Cloud customer and you are using service account keys… why?
Accessing Google Cloud resources within Google Cloud
It’s perhaps unsurprising that operating within Google Cloud gives the most seamless experience here. You have an app running in App Engine that needs to be able to talk to a Cloud Storage bucket as well as BigQuery. To make your life easier you include Google’s libraries in your app for those services and you come to discover it just works. You don’t do anything special to authenticate and yet every action your app makes is performed as the service account you assigned to your App Engine app. What you’ve been using, maybe without realizing it, are Application Default Credentials (ADCs). ADCs are a baseline functionality that exists in Google’s compute related products, which Google extended to Google Kubernetes Engine (GKE) with a feature called Workload Identity.
Serverless and Compute Engine
When running in Google Cloud’s serverless products (e.g. App Engine, Cloud Functions) or on a Compute Engine VM you can leverage Google libraries which will automatically obtain tokens for the service account assigned to resource running your app. It really is that easy. ADCs have greatly reduced the barrier for strong identity security consumable by your app.
“But Derek!” you say. “All my workloads run in GKE! ADCs are no help!” Well, that’s sorta true. In 2019 Google launched Workload Identity. Workload Identity is a way to allow Kubernetes service accounts in your GKE cluster to impersonate Google service accounts by obtaining tokens for them. There are a few configuration steps involved because both the Kubernetes side and the Google IAM side need to have changes made but once you’ve completed that your apps can function just the same as using ADCs on non-Kubernetes compute. Just use the Google libraries and let them handle the authentication.
Impersonation, advanced token usage
Of course you may have some more advanced identity needs with your application. In that case, Google makes their OAuth client library, commonly referred to as “Google Auth Library,” available across a wide variety of languages. Using this library you can perform service account impersonation or just take a more hands on approach to token management.
Bonus: Cloud SQL, Identity Aware Proxy
Really leaning on Google service accounts for your applications can help you in other areas where you would traditionally need to store secrets. If you are using MySQL or PostgreSQL in Cloud SQL you can use those same Google service accounts to not only access your instance but authenticate to your database. This makes for very easy use of your database with no stored secrets needed. Similarly, if you use Google’s Identity Aware Proxy you can also authenticate your app to your API with, yup, no stored secrets.
Accessing Google Cloud resources outside of Google Cloud
Thanks to the launch of Workload Identity Federation this year, Google has you covered there too. Since all this work is built on top of OAuth, Google has extended it to also function with identity federation so that you can securely access Google Cloud resources without the need of a service account key. You are able to use an identity from a different identity provider than Google, even AWS or Azure, and access your Google Cloud resources much like if you were running within Google Cloud. This new feature is critical for cutting out the last major use case of service account keys.
Using ADCs, Workload Identity, and Workload Identity Federation you can eliminate all, or nearly all, of your service account key usage. You may also see it as an incentive to transition some non-Google Cloud workloads into Google Cloud simply to take advantage of the more integrated approach, which I’m sure Google would emphatically support. If you’ve read all this and are still thinking “Well I use [insert secret manager] so it doesn’t really matter for me” then you have missed the point. This isn’t about securing a secret, it’s about moving most of the burden of securing a secret from you to Google. Take the opportunity to reduce your operations workload and improve security! After all, nobody can accidentally commit a service account key to a git repo if it doesn’t exist.