ECR with on premises Kubernetes

Glen Tomkowiak
Sep 11 · 2 min read

Deploying Kubernetes on premises can be challenging enough.

You’ll quickly find out things you’d expect to work, simply don’t. That’s because cloud providers do lots of things for you (which is good).

I am currently storing docker images on ECR (AWS elastic container registry). It’s easy to use some simple commands to authenticate with your configured credentials then push / pull images when developing.

But this is something you are going to want to just do with Kubernetes. It is also essential for Kubernetes to be able to pull images from the cloud when it needs to. AWS ECR tokens expire every 12 hours so you will need a something that takes care refreshing tokens for you. Fortunately the registry-cred project on Github does just this.

Prerequisite: Create an AWS service account with programmatic access that can only interact with ECR. Be sure to save the access key id and secret access key.

Step 1: Log the host that you run the kubectl command from or you can copy / paste to the dashboard.

Step 2: git clone https://github.com/upmc-enterprises/registry-creds

Step 3: Enter the k8s sub directory within the cloned repo

Step 4: Edit the secrets.yaml file.

Replace the text YOUR_BASE64_HERE with a base64 encoded value for each field. You can accomplish this in bash by running echo -n “SECRET” | base64.

Then copy the output into each field as shown below.

apiVersion: v1
kind: Secret
metadata:
name: registry-creds-ecr
namespace: kube-system
labels:
app: registry-creds
kubernetes.io/minikube-addons: registry-creds
cloud: ecr
data:
AWS_ACCESS_KEY_ID: YOUR_BASE64_HERE
AWS_SECRET_ACCESS_KEY: YOUR_BASE64_HERE
aws-account: YOUR_BASE64_HERE
aws-region: YOUR_BASE64_HERE
type: Opaque

Run kubectl create -f secret.yaml to create the new secret.

Step 5: Edit replicationController.yaml and remove what you are not going to use.

apiVersion: v1
kind: ReplicationController
metadata:
name: registry-creds
namespace: kube-system
labels:
version: v1.6
spec:
replicas: 1
selector:
name: registry-creds
version: v1.9
template:
metadata:
labels:
name: registry-creds
version: v1.9
spec:
containers:
- image: upmcenterprises/registry-creds:1.9
name: registry-creds
imagePullPolicy: Always
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: registry-creds-ecr
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: registry-creds-ecr
key: AWS_SECRET_ACCESS_KEY
- name: awsaccount
valueFrom:
secretKeyRef:
name: registry-creds-ecr
key: aws-account
- name: awsregion
valueFrom:
secretKeyRef:
name: registry-creds-ecr
key: aws-region

Now you will see secrets magically appearing in all namespaces. They should be named: awsecr-cred

    Glen Tomkowiak

    Written by

    Things that interest me: cloud computing, cyber security, DevOps, and mobile / web development. Posts are my own.

    Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
    Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
    Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade