Where should you store your application secrets, you wonder?
A comparison between source control, CI/CD tools and cloud providers for storing application secrets.
I’d really be surprised if you’d tell me that your web application is truly independent. Web applications cannot be imagined without integrations and every application needs to connect to peer microservices, external services, databases, file stores and what not. Which brings us to maintaining the secrets (passwords/API keys/auth tokens/certificates/..) required to connect to these services, which are in most cases different for your different application environments.
It is absolutely a no-brainer that the sensitive data used by any application must be externalised and not be directly used in the code. Developers, more often that not, tend to put the secrets either directly in the code or push their .env or .properties or config files to source control. It is a very bad practice and must be avoided at all costs. The secrets should be procured to the code either during build or deploy step or when it is running.
There are a number of ways to manage your secrets. You can store secrets in your source control (GitHub/Bitbucket/GitLab/..), CI/CD tool (GitHub Actions/CircleCI/Jenkins/..) or cloud (AWS Secret Manager/Azure Key Vault/GCP Secret Manager/..). You can even opt for third party key vaults like HashiCorp Vault but I am keeping them out of this discussion. Let’s analyse the pros and cons of storing secrets in the mentioned options in different scenarios.
Note: You could be using CI/CD feature of your source control itself like GitHub Actions/Bitbucket Pipelines/GitLab Pipelines/.. in which case, your source control and CI/CD tool would be same.
Runtime secrets
Some secrets can be volatile in nature and might keep getting updated, like some access token. These secrets cannot be read just once during the build or deploy and need to be accessed at runtime. This scenario gives cloud providers an edge over the source control & CI/CD tools. Services can update their cloud secrets at runtime which can be used by other services.
Accessibility
Storing secrets in cloud definitely takes the cake in accessibility too. Be it accessing secrets at runtime or during build, secrets in cloud are relatively more accessible. Services should be restartable directly from the cloud, without needing to rely on the CI/CD or source control for secrets.
Build once run anywhere
If you want your builds to be self sufficient and can be run anywhere, keeping your secrets in the source control or CI/CD tool would be the way to go. This way, you can create your builds with all the secrets supplied and deploy them anywhere. This could also be ideal for multi-cloud infrastructure.
Devops
Storing and managing secrets in repositories and CI/CD pipelines can be cumbersome. The repositories and the pipelines keep increasing and managing their specific secrets would not be an easy task. It would be rather simpler and easier to maintain them in cloud. You can create environment specific secret managers each having their own copy of secrets for all the applications.
Storing cloud credentials
Let’s be honest, what purpose would storing your cloud credentials in cloud serve (:D). To deploy your builds to cloud you would need to store your cloud credentials either in the source control or CI/CD tool. CI/CD would again be the preferred choice of the two because it would be a good practice to isolate deployment environment details from source control and keeping them closer to the CI/CD pipelines.
Environment specific secrets
Most of your application secrets would be different for different environments and it is better to keep them in cloud. You can have environment specific secret managers in cloud and your CI/CD tool does not even need to handle different environments differently. All it needs to do is specify the environment and load the corresponding cloud credentials.
Configuring these secrets in source control or CI/CD would require you to handle them separately in build-deploy pipelines. Spinning up new environments would also be not as easy.
As you can see, the choice of secret manager depends on your use case. But, in most cases, it would be a combination of all the options. Say, for example, you could store universal constants in source control, cloud credentials in CI/CD tool and runtime/environment specific secrets in cloud.
