The Problem Statement
During the software development life cycle, we need to deal with credentials, passwords, and secrets all the time.
Earlier, when we were doing application development, database password management was the only thing we had to consider. But nowadays, as the use of cloud is increasing, we need to deal with various keys, service accounts, service principals, etc. as well.
Often, I have seen that, if the developer is inexperienced or not mature enough to understand security aspects, then they might end up committing passwords, service accounts, or any other secrets in source code repositories.
Which can be disastrous if you are using public or publicly hosted repos, or even private repos which are accessible across your organization.
Do remember, no matter how strong the technology policies are that you have in place, your system is only as secure as its weakest link.
So, these are useful to detect if such things are already checked into the source code repositories. But most of the time, these analyses are run at the server-side when the code is already checked in.
Also, SonarLint’s local validation is mostly a manual process and does not stop someone from committing credentials, even locally.
To overcome this challenge, I tried to look around for some open-source projects and found a project from AWS Labs, called git-secrets.
The project works perfectly fine to prevent you from committing secrets and credentials into Git repositories that are specific to AWS.
I was looking for a solution that can be extended to Google Cloud Platform (GCP) credentials as well. Hence, I extended this project to add support for GCP.
How Do I Use It?
To start, you need to clone the git-secrets repo to your local machine.
git clone https://github.com/deshpandetanmay/git-secrets.git
If you are on a Unix machine, then run the following command to install this utility:
sudo make install
You can look at the instructions to install this on Windows or macOS here.
Once installed, you need to go to the Git repo where you need to use this utility. For the demo, I am cloning another repo from my GitHub.
git clone https://github.com/deshpandetanmay/cdr-data-generator.gitcd cdr-data-generator/
Now, to install
git secrets for this repo, you can run the following command:
git secrets --install
✓ Installed commit-msg hook to .git/hooks/commit-msg
✓ Installed pre-commit hook to .git/hooks/pre-commit
✓ Installed prepare-commit-msg hook to .git/hooks/prepare-commit-msg
This will install the executables for secrets to scan and it will also install three Hooks for this repo.
To install AWS and GCP specific checks:
$ git secrets --register-aws
$ git secrets --register-gcp
Now you are all set. To test if everything is working as expected, I created a service JSON account from GCP and copied it into my repo.
Now, when I run the
git commit command, I see:
git commit -m "Updated config"
test.json:4: "private_key_id": "d30b0f8858589d3d1294aee5b",
test.json:5: "private_key": "-----BEGIN PRIVATE KEY-----<Actual Private Key>a\n-----END PRIVATE KEY-----\n",[ERROR] Matched one or more prohibited patternsPossible mitigations:
- Mark false positives as allowed using: git config --add secrets.allowed ...
- Mark false positives as allowed by adding regular expressions to .gitallowed at repository's root directory
- List your configured patterns: git config --get-all secrets.patterns
- List your configured allowed patterns: git config --get-all secrets.allowed
- List your configured allowed patterns in .gitallowed at repository's root directory
- Use --no-verify if this is a one-time false positive
It shows me that there is a file called
test.json which contains a prohibited pattern and it stops me from committing the GCP service account.
In the case that
git secrets detects something as a false positive, you run the following command to ignore the checks and proceed with commits.
git commit -m "Updated config" --no-verify
This not only detects secrets in source-code files but also in commit messages and stops developers from committing secret information in commit messages.
The current implementation so far only supports AWS and GCP and I am planning to extend this to Azure as well.
To conclude, you can use the above-mentioned technique to prevent developers from committing credentials into source-code repositories and help keep your application safe.