Deep Learning Docker Images for Production

When a deep learning network has been trained the work of bringing it to production has just begun. Here at MoonVision different projects have very different production environments. Some projects lead to cloud hosted, some are run in the data centers of our customers and others are run as appliances on embedded industrial computers.

What all these environments have in common? They run Docker!

But what about the differences? In some environments we have NVIDIA GPUs for CUDA hardware acceleration, in some environments we don’t.

Why we re-engineered our container build pipeline

As you might know, to get the most out of CUDA requires CUDA-specific versions of various tools and libraries. In the past, we solved that by maintaining two versions of most of our Dockerfiles, one with CUDA acceleration and one for CPU only execution.

Another issue is managing custom builds of the software tools and libraries we use. If you want to use the newest features of libraries like PyTorch before they are released in new versions, you have to build them yourself. It was a mess to cover this problem, but with the new pipeline this issue was solved.

What we changed

Solid lines are child image/base image relationships. Dotted lines are other dependencies. Boxes off the same color are built from the same Dockerfile.

Instead of writing one Dockerfile per image we switched to separating by concerns and injecting dependencies as build arguments. This reduces code duplication in Dockerfiles, improving maintainability but also allows us to extend and customize Docker images without having to modify existing Dockerfiles.

By storing build artifacts like Debian .deb packages or Python Wheel .whl packages in Docker images we can use those artifacts in future builds without the additional cost and complexity that comes from having to maintain package repositories. Instead, we just push our build artifacts to Docker Hub. To keep the image sizes of our custom-builds images small we utilize Docker multi-stage builds, copying just the artifacts into an otherwise empty image.

Easily run GenICam based machine vision apps

For example, to run machine vision applications with modern deep learning frameworks, we extend the python-base image, add the PyTorch and FFmpeg artifacts that were built in the custom-builds images and install the open source GenICam Harvesters library. The result is a Moonbox image (moonvision/moonbox:basler-latest) equipped with powerful capabilities to acquire, process and save image data streams from vendors like Basler.

So if you are using a Basler industrial camera you are ready to go.

If you want to use a GenICam compliant camera from a different vendor, you by extending the moonvision/moonbox:genicam-latest Docker image yourself.

To see if the library was installed correctly and the connected camera is recognized you can query the cameras available to Harvesters.

Change the vendor-tag and the path of the CTI file as appropriate.

Try it out yourself

If you want to use Moonbox docker images you can pull them here: https://hub.docker.com/r/moonvision/moonbox

If you want to find out how they are made or built them yourself you can checkout the source here: https://github.com/moonvision/moonbox-docker