Two Ways to Keep Secret Keys Out of Your Git History
Not long ago I was writing a feature relying on Twilio — an API service for text messaging. I was testing it locally and then accidentally committed and pushed the Auth Token and the CID to github — the secret identifiers used for account authentication.
While the misstep wasn’t critical — our repo is private and we were able to swap the credentials without downtime immediately after, it still left me wondering if there are some ways I can prevent that from happening in the future, given it wasn’t the first time I’ve pushed a secret key by accident.
Why the Buzz?
When building a product, there are many secrets — account passwords, API keys, private tokens and SSH keys — that we safeguard. The problem with secrets, is that once they’re revealed, they can never be un-shared.
Git is very powerful and it doesn’t destroy your overwritten or removed commits most of the time, even when there is no direct reference to them from any of your branches.
Once pushed to a remote, your files are part of that git history and completely removing them might prove tricky or even impossible.
Here is a great article on the why’s of keeping sensitive data out of Github (even for a private repo).
Part I: Using Git Hooks
We’re going to look at an example of pre-commit git hook that displays a confirmation prompt when it detects changes to .env file.
N.B. These examples assume that you’re using dotenv or a similar pattern to manage secret keys. This is a common method, but by far not the only one.
#!/bin/sh# This warns you and confirms that you want to commit changes in .env files# You can change this line to grep any files
CHANGED=`git diff --staged --name-only | grep '\.env$'`if [ "$CHANGED" ]; then
echo
echo "Changed files:"
echo $CHANGED # Allows us to read user input below, assigns stdin to keyboard
exec < /dev/tty while true; do
read -p "Are you sure you want to commit? (Y/n)" yn
case $yn in
[Yy] ) exit 0; break;;
[Nn] ) echo Did not commit; exit 1;;
* ) echo "Please answer y or n for yes or no.";;
esac
done
fi
Put this in .git/hooks/pre-commit.
Now let’s change some of your project files including the .env file:
➜ myrepo git:(master) ✗ git status
On branch master
Changes not staged for commit:modified: .env
modified: app/controllers/static_pages_controller.rb
modified: config/routes.rb➜ myrepo git:(master) ✗ git add .
➜ myrepo git:(master) ✗ git status
On branch master
Changes to be committed:modified: .env
modified: app/controllers/static_pages_controller.rb
modified: config/routes.rb
➜ myrepo git:(master) ✗ git commit -m 'display home page'Changed files:
.env
Are you sure you want to commit? (Y/n)
The pre-commit hook is being run before commit is created. Entering “n” aborts the commit and returns you to the previous state.
Pro tip: git hooks are local and do not go into your remote repo, so it’s wise to back them up in case your local folder is replaced. Mine is here on medium. :)
Part II: Configure shell to export environment variables into your program
Okay, stuff is about to get really cool.
Export your app environment variables in your shell configuration file. I use zsh shell (alternative to bash), so the below example is using ~/.zshrc file.
# ~/.zshrcexport TWILIO_AUTH_TOKEN=bar
export TWILIO_SID=foo
Reopen your terminal window and you should have access to these variables:
➜ ~ echo $TWILIO_AUTH_TOKEN
bar
➜ ~ echo $TWILIO_SID
fooPrepending key-value pair with export makes these variables available to the child processes (e.g. your app server) via the environment.
Pro tip: Variables exported via the shell take precedence over those defined in your repo’s .env file. I find it useful to commit default values for secret keys: it serves as documentation for all existing keys.
Good resources to learn:
Git Maintenance and Data Recovery
If you don’t want to deal with bash scripts, check out this repo, which allows you to write git hooks using ruby.
