Where do I put my database password?

Photo by Kristina Flour on Unsplash

I was recently reminded how I struggled with handling secrets in my first hobby projects. It all started when I began using third party web APIs to get data, later it was when I needed my first database. I wanted to keep my source in a public git repository but now I had this “thing” that could not be shared. At the time I lacked the vocabulary to describe what I needed and it doesn’t help that secret management is actually a very hard problem that at the highest level is a complex and expensive distributed service. But this is the lowest level and all I really need to know is; where do I put my database password?

The Short Answer with Link to a Better Article

Set your secret as the value of an environment variable, then in your program have a section that reads that data and assigns it to a constant that can be accessed wherever it is needed. A good article that goes into language specific details is “The Rubyist’s Guide to Environment Variables.” No one can argue this is the safest solution but at least it keeps sensitive data out of the source code and passes the buck to OS for security. For me this is acceptable for hobby or demonstration projects.

A Slightly More Detailed Answer Regarding Docker

I use environment variables for configuration in all my side projects. Because I find myself exploring a lot of different technologies I have invested in docker as a way to build immutable images of my development environments on a per project basis. A lot of them are special snowflakes but recently I open sourced python.mk which is my solution for python scripting. This is how I manage secrets in that framework.

I maintain a file called env.list, which as the name implies is a list of all the environment variables. Critically this file is also referenced in the .gitignore and should never be tracked by version control.

#env.list
MAGIC_NUMER=42
SECRET_API_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

For others who might clone or fork this project I also have a file called env.list.example which contains all the same keys but is stripped of secret values and is tracked by git.

#env.list.example
MAGIC_NUMER=42
SECRET_API_KEY=

This way a contributor can

cp env.list.example env.list 

and source the missing secrets for themselves. These values are usually specific for your local development, but the great thing about storing your secrets like this is that they transfer well. I think Heroku popularized this method but CI tools like Travis also have first class support for sourcing data from the environment.

The trick now is to get this file of important data into the docker image that tests or runs the code.

#python.mk
IMAGE_TAG=pythonmk:latest
UGID=$(shell id -u):$(shell id -g)
run: build_quiet
        @docker run \
                --rm \
                --user $(UGID) \
                --volume $(CURDIR):/script \
                --env-file $(CURDIR)/env.list \
                $(IMAGE_TAG) \
                python3 /script/$(APP_NAME).py

This is accomplished with a phony target in make called run, which synthesises a docker command to start the image. The key part being the env-file argument which will take all your variables and write them to the OS of the docker environment for the user you are assuming.

In the source I will usually have a file like config.py. The Python os library has the getenv() method with the signature:

def getenv(key: str, default: Optional[Any] = None) -> str:

I use this to set up constants for all the variables in the env.list. Since os.getenv() accepts defaults and returns a string, this is also a good place to manage any type casting that might need to happen.

#config.py
import os
SECRET_API_KEY: str = os.getenv("SECRET_API_KEY", "")
MAGIC_NUMBER: int = int(os.getenv("MAGIC_NUMBER", 42))

Other files can now import these constants and use them however they like.

#magic.py
from config import MAGIC_NUMBER
print("The answer to the ultimate question of life, "
      + f"the universe and everything is {MAGIC_NUMBER}")

Concluding Advice

The first company I worked at just hard coded all the passwords in to the source code and relied on the fact that their github repositories were private. It took me a while to realize that that company was bad at security, my first hint should have been that all their production database passwords were some variation of 2Secr3t4U. Please don’t do this.

Using environment variables to store sensitive information is not a professional solution. Remember that storing other people’s data is a responsibility and a liability. Using this method is appropriate for a hobby but the best way to keep sensitive data secret is to not persist it at all.