Hidden Secrets of Dockerfile

Rachit Shrivastava
Dec 8, 2020 · 3 min read

Have you ever faced an issue in which while building the docker image, even though you have mentioned “apt-get -y update” the package you are trying to install is not found in the repository?

Here, in this blog, we are going to discuss two common scenarios that I faced along with the root cause of this problem. Let’s take a look.

First, it’s important to remember that valid intermediate images are not built again. They are loaded from cache. Dockerfile cache is considered valid if the build command gives a 0 return code.

1. Consider the following Dockerfile.

From ubuntu RUN apt-get -y update RUN apt-get install -y python-pip

Now when we build it:

docker build -t test:image1 .

Suppose while building the above Dockerfile we created an intermediary image with the directive “RUN apt-get update”.

This intermediate image will be used for all the subsequent docker builds.

Let’s say a couple of months down the line, there is a new package-manager release of python-pip which is required by the application. Now, even though in Dockerfile we have mentioned “RUN apt-get update”, the apt repo will not get updated because that image will be taken from the cache. Hence the latest version of python-pip won’t be installed.

2. Our story is not over yet, consider the following.

Before building the below Dockerfile, turn your internet connection off.

From ubuntu
RUN apt-get -y update ; echo "status is: $?" RUN apt-get install -y python-pip

Now when we build it:

Docker build -t test:image2 .

Output:

Err http://security.debian.org stable/updates Release.gpg Could not resolve 'security.debian.org' Hit http://192.168.1.100 stable Release.gpg Hit http://192.168.1.100 stable Release Hit http://192.168.1.100 stable/main i386 Packages Hit http://192.168.1.100 stable/contrib i386 Packages Hit http://192.168.1.100 stable/non-free i386 Packages Ign http://192.168.1.100 stable/contrib Translation-en Ign http://192.168.1.100 stable/main Translation-en Ign http://192.168.1.100 stable/non-free Translation-en Reading package lists... Done W: Failed to fetch http://security.debian.org/dists/stable/updates/Release.gpg W: Some index files failed to download. They have been ignored, or old ones used instead. status is: 0

Aside from the ADD and COPY commands, cache checking does not look at any other file in the container to determine a cache match. For example, when processing a RUN apt-get -y update command, the files updated in the container are not examined to determine if a cache hit exists. In such a case, just the command string itself is used to find a match.

As we can see above, “apt-get update” throws a “WARN” and not an “ERROR” and the return code is 0, which is a successful response. Since the response code is zero hence this image will get cached.

The problem comes when we re-build the Dockerfile with our internet connection on, Docker will still take the “apt-get -y update” image from the cache and subsequent “RUN apt-get install -y python-pip” will fail because “apt-get -y update” didn’t trigger successfully in the last run.

Solution:

Try building the image again with the “no-cache” option. This will force the rebuilding of layers already available.

Docker build --no-cache -t test:image3 .

Conclusion :
Once an image is successfully built, all the intermediate images will be kept and reused. A valid docker intermediate image will not be built again unless we pass the “no-cache” option or delete the existing image.

Image Resource

Opstree is an End to End DevOps solution provider

Originally published at http://blog.opstree.com on December 8, 2020.

Opstree

Opstree is an end-to-end DevOps consultant company

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store