GCP: Secret Manager
So a couple of months ago, Google shared with us a Service called Secret Manager. It had been in the works for a while and the streets were talking already.
Secrets management (usernames, passwords, keys e.t.c) on GCP was frankly speaking, a nightmare. Solutions around this involved layers of encryption and decryption with one or two 3rd-party-tools here and there to make it work. As you can imagine, the challenges here were many. For starters, layers of complexity and bundling of tools just called for management time and effort that no one really has. This problem was further compounded, when you tried to be portable with your application. Poor visibility and transparency of who is accessing what and when was also a thing. As you can already tell, moving around keys is just not funny.
Ever so loyal to the ecosystem, developers tried to throw Cloud KMS at this problem.
To be fair, Cloud KMS really tried to be innovative here, but, always felt like a square plug in a round hole. It was clunky, complicated to use and always taking you on an unnecessary adventure. You may be reading this and thinking, this is one man’s opinion, but actually, its not. Here, by Google’s own admission:
A key management system, such as Cloud KMS, allows you to manage cryptographic keys and to use them to encrypt or decrypt data. However, you cannot view, extract, or export the key material itself.
Similarly, you can use a key management system to encrypt sensitive data before transmitting it or storing it. You can then decrypt the sensitive data before using it. Using a key management system to protect a secret in this way is more complex and less efficient….
This is not to say that Cloud KMS is a useless tool. No! Its just that, while it is a great tool, not all use-cases lend themselves to KMS. Sometimes you just need to access your secrets and you don’t want all that encryption decryption business.
If you’ve used Hashicorp Vault, you knew exactly what you were missing.
Enter Secret Manager!
Secret Manager allows you to store, manage, and access secrets as binary blobs or text strings. With the appropriate permissions, you can view the contents of the secret.
There’s a whole lot of benefit here. Secrets can now be anchored in a centralized source of truth, using a fully managed service. On top of this, we can now tap into Cloud IAM to manage authorization. This is a huge win for the “portability” cause.
Let’s also talk a bit about version control. CI/CD (continuous integration and continuous delivery) pipelines have made versioning a bit of a staple. The average build-pipeline today will have multiple versions that swing from dev to prod and everything in between. As such databases will be different, keys will be unique, usernames will be shared, and passwords will differ. In the interest of centralized management, a key-value pair can have a single key with multiple values (which then by definition seizes to be a pair…. no?). Anyway, is this feature even necessary or are we all just being a bit extra? Absolutely necessary! There are dozens of times when you need different credentials for the same key. For instance, as a matter of best practice, there's a quote that say, “Don’t go to prod with dev credentials!”. Or vice-versa! So in a CI/CD pipeline, you might want to keep everything as isolated but keep your static variables.
Having said that, if you’d like to see Secret Manager in action, here’s a running example. I use spring cloud and Secret Manager, with Spring-Cloud-GCP-Secret-Manager, on a relatively portable solution. The necessity here for Secret Manager is that weneed to attach database secrets to CI/CD build pipeline that is hosted on a public repo on Github. Tested and works like a charm!
