The Problem Statement
During the software development life cycle, we need to deal with credentials, passwords, and secrets all the time.
Earlier, when we were doing application development, database password management was the only thing we had to consider. But nowadays, as the use of cloud is increasing, we need to deal with various keys, service accounts, service principals, etc. as well.
Often, I have seen that, if the developer is inexperienced or not mature enough to understand security aspects, then they might end up committing passwords, service accounts, or any other secrets in source code repositories.
Which can be disastrous if you are using public or publicly hosted repos, or even private repos which are accessible across your organization.
Do remember, no matter how strong the technology policies are that you have in place, your system is only as secure as its weakest link.
Existing Solutions
Currently, there are solutions like SonarQube and SonarLint static code analysis tools, which detect if there are any passwords or pass phrases that are checked into the code.
So, these are useful to detect if such things are already checked into the source code repositories. But most of the time, these analyses are run at the server-side when the code is already checked in.
In SonarLint’s case, developers can even run the analysis locally before checking in the code. But SonarLint fails to detect if there are any service accounts or keys from AWS, GCP, etc.
Also, SonarLint’s local validation is mostly a manual process and does not stop someone from committing credentials, even locally.
The Solution
To overcome this challenge, I tried to look around for some open-source projects and found a project from AWS Labs, called git-secrets.
The project works perfectly fine to prevent you from committing secrets and credentials into Git repositories that are specific to AWS.
I was looking for a solution that can be extended to Google Cloud Platform (GCP) credentials as well. Hence, I extended this project to add support for GCP.
How Do I Use It?
To start, you need to clone the git-secrets repo to your local machine.
git clone https://github.com/deshpandetanmay/git-secrets.git
cd git-secrets/If you are on a Unix machine, then run the following command to install this utility:
sudo make installYou can look at the instructions to install this on Windows or macOS here.
Once installed, you need to go to the Git repo where you need to use this utility. For the demo, I am cloning another repo from my GitHub.
git clone https://github.com/deshpandetanmay/cdr-data-generator.gitcd cdr-data-generator/
Now, to install git secrets for this repo, you can run the following command:
git secrets --install
✓ Installed commit-msg hook to .git/hooks/commit-msg
✓ Installed pre-commit hook to .git/hooks/pre-commit
✓ Installed prepare-commit-msg hook to .git/hooks/prepare-commit-msgThis will install the executables for secrets to scan and it will also install three Hooks for this repo.
To install AWS and GCP specific checks:
$ git secrets --register-aws
$ git secrets --register-gcpNow you are all set. To test if everything is working as expected, I created a service JSON account from GCP and copied it into my repo.
Now, when I run the git commit command, I see:
git commit -m "Updated config"
test.json:4: "private_key_id": "d30b0f8858589d3d1294aee5b",
test.json:5: "private_key": "-----BEGIN PRIVATE KEY-----<Actual Private Key>a\n-----END PRIVATE KEY-----\n",[ERROR] Matched one or more prohibited patternsPossible mitigations:
- Mark false positives as allowed using: git config --add secrets.allowed ...
- Mark false positives as allowed by adding regular expressions to .gitallowed at repository's root directory
- List your configured patterns: git config --get-all secrets.patterns
- List your configured allowed patterns: git config --get-all secrets.allowed
- List your configured allowed patterns in .gitallowed at repository's root directory
- Use --no-verify if this is a one-time false positive
It shows me that there is a file called test.json which contains a prohibited pattern and it stops me from committing the GCP service account.
In the case that git secrets detects something as a false positive, you run the following command to ignore the checks and proceed with commits.
git commit -m "Updated config" --no-verifyThis not only detects secrets in source-code files but also in commit messages and stops developers from committing secret information in commit messages.
The current implementation so far only supports AWS and GCP and I am planning to extend this to Azure as well.
Conclusion
To conclude, you can use the above-mentioned technique to prevent developers from committing credentials into source-code repositories and help keep your application safe.
