Docker in version 1.13 introduced the concept of secrets. For any application managing configuration variables, ssh keys, passwords, API keys etc. is a crucial part, This data is not be disclosed to anyone outside the authority.
Initially this data was set manually by setting environment variables on the node (which is a part of the swarm), But setting this data on all the nodes turned out to be quite a tedious task, I mean one can also copy a config file but again how many times will you copy the same file on different nodes, But why would anyone copy when you could put all these variables either in your YML stack files or Dockerfiles (NOTE: Anyone can have access to your data when you push your image to docker hub), YML stack files is not a good option as pushing sensitive data on websites like Github (A must to work in a team) is not what is preferred and YML stack files are to be shared for the application to work.
So what we are left with is manual copying, a correction what we were left with! Thanks to this new feature of SECRETS!!!!!!!!!
What are Secrets?
As the name suggests, Secrets are sensitive pieces of data visible having restricted access. Secrets acts as a envelope which contains your data but security lies in only the one who has that envelope will be able to open it.
Secrets are encrypted pieces of sensitive data, Also, Secrets offer restricted access to itself, ie. Only some people will have access to the envelope (Analogy).
How does Secrets Work?
Before dive deep, Keep this in mind that secrets are available only in docker swarms ie. one cannot use secrets by using a single container and docker compose.
Docker swarms work on a consensus algorithm known as Raft, Raft consensus maintains a log of all the events and data, This where secrets are stored encrypted in the raft logs, Because these raft logs are replicated on all the managers, Our little secret is also replicated and there is no need of manual copying. All you need is to define a secret (We’ll get our hands on this soon, Keep reading :) ) But as we read, Secrets do not allow full access it itself, Only certain services which are given access can have access to the secrets otherwise our secret is kept encrypted in the raft logs.
Before we dive into the practical part, There is one gotcha I want you to think.
Maybe you got it, When secrets are added to the raft logs and replicated on other managers, It is still present on the docker host but we want the data decrypted into the container running our application, We can also have multiple applications grouped into different services in need of the same secret,
Do not worry docker does all this for you!
All you need to do is write a command to provide access of a particular secret to a service and manager docker daemon will take care of transferring the secret to the service over a secure channel and storing it decrypted in the container at the path:
C:\ProgramData\Docker\secretsSo, All you need to do is read this sensitive piece of data from this path and use it in your application.
Creating our First Docker Secret Using CLI
To create secrets make sure you have to have a Docker daemon and Docker CLI installed.
There are two ways you can create a docker secret using Docker CLI:
- Taking input from STDIN —
echo "A SECRET" | docker secret create SECRET_NAME -
- Using a File to Create a Secret —
docker secret create SECRET_NAME FILE_NAME
Next, Step after creating a secret a is to give it’s access to a service:
Allowing a Service to Access a Secret Using CLI
docker service update --secret-add SECRET_NAME SERVICE_NAME
Revoking a Services Access to a Secret Using CLI
docker service update --secret-rm [... SECRET_NAME] SERVICE_NAME
If you noticed in the last 3 headings I have mentioned using CLI, So, What will be another way to handle secrets?
You got it right! Compose or Stack files!
Creating a Secret within Compose/Stack files
In the above example you can see under secrets > SECRET_1 there is an external key, This is used when you have already set a secret using docker CLI and you want to use it while deploying your stack, Another case as seen under SECRET_2 we have used file key, This key is used when you have a sensitive data file but secret is not created yet, In this case, while deploying docker will create a secret itself.
If you have noticed we have already given access to both of these secrets to our application service by defining a secrets array in the service definition.
Note: There is also a more verbose way of giving access to a secret within the compose/stack file, Visit https://docs.docker.com/compose/compose-file/#long-syntax-2 for details.
Getting our hands on, What we have read so far:
Note: Watch the video to get an overall view of how to use secrets with scalable swarm services.
I have used https://labs.play-with-docker.com for creating swarms multiple nodes.
Also, Visit https://github.com/akshitgrover/docker-secret_demo to check the node application I have been using in the video to play with secrets.
If you have watched the video you might have noticed that I just deployed the stack and a secret was created, As soon as I scaled the service and curled the localhost on node 2 it returned the correct secret data, Thus docker saves us from manual copying and secure storage and transfer of sensitive data with the help of secrets.
Secrets are used to store sensitive data related to the application, securely and transfer these to services which require them, over a secure and trusted channel between the master and the service containers.
- Node Application (Github): https://github.com/akshitgrover/docker-secret_demo
- Play with Docker (To configure multiple node swarms): https://labs.play-with-docker.com/
- Official Docker Docs on Secrets: https://docs.docker.com/engine/swarm/secrets/
- Stack File Format for Defining Secrets: https://docs.docker.com/compose/compose-file/#secrets
A big thanks to Luc Juggery for these articles to help me understand the concept better and use it in my application :)