Secrets Management in Kubernetes
Kubernetes Secrets are secure objects which store sensitive data, such as passwords, OAuth tokens, and SSH keys, etc. with encryption in your clusters.
Using Secrets gives you more flexibility in a Pod Life cycle definition and control over how sensitive data is used. It reduces the risk of exposing the data to unauthorized users.
- Secrets are namespaced objects.
- Secrets can be mounted as data volumes or environment variables to be used by a container in a pod.
- Secret data is stored in tmpfs in nodes
- API server stores secrets as plain text in etcd
- A per-secret size limit of 1MB
Creating a secret:
Create username.txt and password.txt files.
echo -n 'root' > ./username.txt
echo -n 'Mq2D#(8gf09' > ./password.txt
And
kubectl create secret generic db-cerds \
--from-file=./username.txt \
--from-file=./password.txt
secret "db-cerds" created
List secret:
kubectl get secret/db-cerdsNAME TYPE DATA AGE
db-cerds Opaque 2 26s
View secret:
kubectl describe secret/db-cerds
Name: db-cerds
Namespace: default
Labels:
Annotations:Type: OpaqueData
====
password.txt: 11 bytes
username.txt: 4 bytes
Using YAML file:
The Secret contains two maps: data and string data. The data field is used to store arbitrary data, encoded using base64.
echo -n 'root' | base64
cm9vdA==echo -n 'Mq2D#(8gf09' | base64
TXEyRCMoOGdmMDk=
Write a Secret yaml file
---
apiVersion: v1
data:
password: TXEyRCMoOGdmMDk=
username: cm9vdA==
kind: Secret
metadata:
name: database-creds
type: Opaque
Create the secret using kubectl create
kubectl create -f creds.yaml
secret "database-creds" createdkubectl get secret/database-creds
NAME TYPE DATA AGE
database-creds Opaque 2 1m
View secret:
kubectl get secret/database-creds -o yaml
---
apiVersion: v1
data:
password: TXEyRCMoOGdmMDk=
username: cm9vdA==
kind: Secret
metadata:
creationTimestamp: 2019-02-25 06:22:37 +00:00
name: database-creds
namespace: default
resourceVersion: "2657"
selfLink: /api/v1/namespaces/default/secrets/database-creds
uid: bf0cef90-38c5-11e9-8c95-42010a800068
type: Opaque
Decoding secret values:
echo -n "cm9vdA==" | base64 --decode
rootecho -n "TXEyRCMoOGdmMDk=" | base64 --decode
Mq2D#(8gf09
Usage of Secrets
A Secret can be used with your workloads in two ways:
- specify environment variables that reference the Secret’s values
- mount a volume containing the Secret.
Environment variables:
---
apiVersion: v1
kind: Pod
metadata:
name: php-mysql-app
spec:
containers:
-
env:
-
name: MYSQL_USER
valueFrom:
secretKeyRef:
key: username
name: database-creds
-
name: MYSQL_PASSWORD
valueFrom:
secretKeyRef:
key: password
name: database-creds
image: "php:latest"
name: php-app
Secret as Volume:
---
apiVersion: v1
kind: Pod
metadata:
name: redis-pod
spec:
containers:
-
image: redis
name: redis-pod
volumeMounts:
-
mountPath: /etc/dbcreds
name: dbcreds
readOnly: true
volumes:
-
name: dbcreds
secret:
secretName: database-creds
Additional Info :
Secret creation syntax
kubectl create secret [TYPE] [NAME] [DATA]
TYPE can be one of the following:
generic: Create a Secret from a local file, directory, or literal value.
docker-registry: Creates a dockercfg Secret for use with a Docker registry. Used to authenticate against Docker registries.
tls: Create a TLS secret from the given public/private key pair. The public/private key pair must exist beforehand. The public key certificate must be.PEM encoded and match the given private key.
DATA can be one of the following:
— from-file
kubectl create secret generic credentials \
--from-file=username=./username.txt \
--from-file=password=./password.txt
--from-env-file
— from-env-file
cat credentials.txt
username=admin
password=Ex67Hn*9#(jwkubectl create secret generic credentials \
--from-env-file ./credentials.txt
— from-literal flags
kubectl create secret generic literal-token \
--from-literal user=admin \
--from-literal password="Ex67Hn*9#(jw"
Thanks
Questions /Answers
How to make the Kubernetes pods unable to decrypt the Kubernetes secrets without a key?
I am trying to making this as step by step solution.
Encrypt your data using your-key (your encryption-logic, probably, in a script).
./encrypt.sh --key your-key --data your-data
Create a secret of this encrypted data
kubectl create secret generic your-secret-name --from-literal=secretdata=your-encrypted-data
You could add decryption logic like this in your pod ( either as a sidecar or init container)
# decrypt.sh will decode base64 then your decryption logic using your-key
./decrypt.sh --key your-key --data /var/my-secrets
Also, you need to mount this secret as volume to your container.
spec:
containers:
- image: "image"
name: app
...
volumeMounts:
- mountPath: "/var/my-secrets"
name: my-secret
volumes:
- name: my-secret
secret:
secretName: your-secret-name
Refreshing PODs automatically when mounted secrets get updated, but it looks not happening
Kubernetes does not support this feature at the moment and there is a feature in the works (https://github.com/kubernetes/kubernetes/issues/22368).
You can use custom solution available to achieve the same and one of the popular ones include Reloader
.
The doc you linked describes that the secret values inside the mounted volume will get updated when you update the Kubernetes Secret
object.
The application running within the pod can get to re-read the key once it’s updated tho’ and a brand new pod isn’t created to change the key itself.
Also, note that there can be some delay between the actual update of the secret and getting those values reflected in the volume.
As a result, the overall delay from the instant, once the key is updated to the instant once new keys are projected to the Pod, will be as long as the kubelet sync period + cache propagation delay, where the cache propagation delay depends on the chosen cache type (it equals to watch propagation delay, TTL of cache, or zero correspondingly)
Kubernetes deployment mounts secret as a folder instead of a file
Secrets
let you store and manage sensitive information (e.g. passwords, private keys) and ConfigMaps
are used for non-sensitive configuration data. As you’ll see within the Secrets associated ConfigMaps documentation:
A Secret is an object that contains a low quantity of sensitive information like a countersign, a token, or a key
A ConfigMap allows you to decouple environment-specific configuration from your container images, so that your applications are easily portable.
Mounting Secret as a file
It is possible to create Secret
and pass it as a file or multiple files to Pods
.
I've created a simple example for you to illustrate how it works. Below you can see a sample Secret
manifest file and Deployment
that uses this Secret:
NOTE: I used subPath
with Secrets
and it works as expected.
---
apiVersion: v1
kind: Secret
metadata:
name: my-secret
data:
secret.file1: |
c2VjcmV0RmlsZTEK
secret.file2: |
c2VjcmV0RmlsZTIK
---
apiVersion: apps/v1
kind: Deployment
metadata:
...
spec:
containers:
- image: nginx
name: nginx
volumeMounts:
- name: secrets-files
mountPath: "/mnt/secret.file1" # "secret.file1" file will be created in "/mnt" directory
subPath: secret.file1
- name: secrets-files
mountPath: "/mnt/secret.file2" # "secret.file2" file will be created in "/mnt" directory
subPath: secret.file2
volumes:
- name: secrets-files
secret:
secretName: my-secret # name of the Secret
Note: Secret
should be created before Deployment
.
After creating Secret
and Deployment
, we can see how it works:
$ kubectl get secret,deploy,pod
NAME TYPE DATA AGE
secret/my-secret Opaque 2 76sNAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/nginx 1/1 1 1 76sNAME READY STATUS RESTARTS AGE
pod/nginx-7c67965687-ph7b8 1/1 Running 0 76s$ kubectl exec nginx-7c67965687-ph7b8 -- ls /mnt
secret.file1
secret.file2
$ kubectl exec nginx-7c67965687-ph7b8 -- cat /mnt/secret.file1
secretFile1
$ kubectl exec nginx-7c67965687-ph7b8 -- cat /mnt/secret.file2
secretFile2
Projected Volume
I think a better way to achieve your goal is to use projected volume.
A projected volume maps many existing volume sources into an equivalent directory.
Within the Projected Volume documentation you’ll be able to notice elaborated explanations however in addition, I created associate degree example that may assist you to perceive however it works. Using projected volume I mounted secret.file1, secret.file2 from Secret and config.file1 from ConfigMap as files into the Pod
---
apiVersion: v1
kind: Secret
metadata:
name: my-secret
data:
secret.file1: |
c2VjcmV0RmlsZTEK
secret.file2: |
c2VjcmV0RmlsZTIK
---
apiVersion: v1
kind: ConfigMap
metadata:
name: my-config
data:
config.file1: |
configFile1
---
apiVersion: v1
kind: Pod
metadata:
name: nginx
spec:
containers:
- name: nginx
image: nginx
volumeMounts:
- name: all-in-one
mountPath: "/config-volume"
readOnly: true
volumes:
- name: all-in-one
projected:
sources:
- secret:
name: my-secret
items:
- key: secret.file1
path: secret-dir1/secret.file1
- key: secret.file2
path: secret-dir2/secret.file2
- configMap:
name: my-config
items:
- key: config.file1
path: config-dir1/config.file1
We can check how it works:
$ kubectl exec nginx -- ls /config-volume
config-dir1
secret-dir1
secret-dir2
$ kubectl exec nginx -- cat /config-volume/config-dir1/config.file1
configFile1
$ kubectl exec nginx -- cat /config-volume/secret-dir1/secret.file1
secretFile1
$ kubectl exec nginx -- cat /config-volume/secret-dir2/secret.file2
secretFile2
If this response doesn’t answer your question, please provide more details about your Secret
and what exactly you want to achieve.
For More, Recommendations
https://kubernetes.io/docs/concepts/configuration/secret/
https://platform9.com/blog/kubernetes-secrets-management/
https://blogs.oracle.com/developers/5-best-practices-for-kubernetes-security
https://dev.to/martinpham/secure-your-kubernetes-application-with-https-ng7
👋 Join us today !!
If this post was helpful, please click the clap 👏 button below a few times to show your support! ⬇