Docker and Artifactory as a Registry Mirror

Life for developers in corporate networks is a living hell. You bump into network security limitations 20 times every single day and productivity goes south. Corporate proxies have evolved into the worst possible enemy for those who dare to pursue future technologies and better solutions for long lasting problems.

Some of us are lucky enough to escape from that developing in the cloud and eventually deploying locally, but even that doesn’t go without some frustration.

Most corporations fail to notice that they have long ago crossed a line where their ancient approach to manage operational risk has become the greatest risk of all. Some sort of middle ground must be reached in order to bring a minimal amount of sanity, otherwise old-school IT will soon become the cause of talent evasion and anecdote.

This is 2017 — you can't possibly disguise corporate inertia as process maturity. We have learnt that lesson already, thanks.

But how can we concede access to outside resources in a flexible — but manageable — way to developers?

Artifactory can make its way as a mirror/cache for repositories of many types of artifacts (from external sources or from your own builds). This means that it can act as a local mirror for outside yum, apt, apk, npm, PyPi, etc. repositories, including any Docker registry (such as Docker Hub).

Such use can guarantee that outside resources will flow in through a single point — simplifying network and proxy configuration and allowing better auditing and whatever makes server zealots feel safer about it.

A proxy for Docker Hub is a very simple setting in Artifactory:

Docker Hub as a remote repository

Docker Hub (or any other Registry) can be cached by Artifactory with a rather simple configuration (seen above) but in order to use it "as-is" you will have to pull/push from a new registry URL. Yes, Artifactory responds as a Registry (V2) on a new URL of your choosing, but you also have to set up a web proxy (like nginx) in order to respond in these URLs and forward requests properly to Artifactory. I am not going to dwell into that, but the reader may relax — Artifactory itself generates a nginx config for you. You need a web proxy for docker registries, because Artifactory's dynamic URLs must be translated to Registry API compatible ones.

Tip of the day: create a local "docker-local" Docker repository to publish your own images, a remote "docker-remote" repository mirroring Docker Hub and a virtual "docker" repository aggregating both. This repository will behave as a private docker registry and will be available at "docker.somedomain.blabla". Oh, yes, you'll certainly have to create (or ask for) such a DNS name for your friendly Ops.

As mentioned before, a new registry URL must be used if we keep things "as-is". So, for example, where you used to pull the “centos” public image:

docker pull centos

…you will now have to do this to pull the same image through Artifactory's proxy URL you created:

docker pull docker.mycompany.com/centos

The great advantage is that Artifactory caches this image so that any other pulls for this image (or for any image layered above it) will benefit from this cache.

Registry Mirror Engine Option

Problem with the approach above is that it changes how the image is named. You can always tag ther image manually to is original name after pull:

docker tag docker.mycompany.com/centos centos

But this is cumbersome, to say the least. Changing the image namespace is a bad idea, generally speaking.

Luckly there is a feature on Docker Engine that goes mostly unnoticed: the “--registry-mirror” daemon option. Engine options are configured somewhat differently on each Linux distro, but in CentOS/RHEL you can do it editing the “/etc/sysconfig/docker” file and restarting Docker:

OPTIONS=”--selinux-enabled --registry-mirror=http://docker.mycompany.com --insecure-registry=docker.mycompany.com"

Tip of the day: make sure this works without HTTPS first. After that you can fight your way into making HTTPS work on the nginx web proxy and remove the "insecure-registry" setting. Pick your battles one at a time.

In Docker for Mac the same settings can be provided on the GUI. Here is a example:

Registry mirror on Docker for Mac

With this setting any “docker pull” will go through your mirror (Artifactory) without the need to force a Registry URL other that the original one:

docker pull centos

Works like a charm.

Other Linux distros will have specific ways of doing the same thing, but a more generic approach is to create a text file named "daemon.json" (located at "/etc/docker" in most cases) with the correct set of daemon options (JSON syntax).

An example of a"daemon.json" file containing the mirror options mentioned before:

{
 “registry-mirrors” : [
   “http://docker.mycompany.com"
 ],
 "insecure-registries" : [
    "docker.mycompany.com"
  ]
}

Do I need Artifactory?

No, you can do the mirroring with the plain regular and public registry image:

https://docs.docker.com/registry/recipes/mirror/

You can also setup local yum, apt, apk etc. mirrors just syncing remote folders with quite simple scripts and serving them on HTTP. The problem is that you'll soon face yourself managing a ton of local mirrors and that will quickly become another problem on its own. Artifactory is a one-stop solution for pretty much all mirrors and local repos you'll ever need.