Google Container Builder is an interesting tool that allows you to use the power of the Google Cloud Platform to enable a fast and reliable build for your container images.
The name is a bit confusing to understand in terms of the things that it can do but I am going to talk about a process scenario that I have been using and which I believe applies to most of us and see how Google Container Builder makes it more efficient.
Our use case
If you have been using Docker, one of the key things is to build a Docker image and make it available for others to pull the image to use in their local environment. The process goes something like this:
- You should have a local Docker environment setup with all the tools.
- Write a Docker file that builds out your Docker image.
- Execute the
docker buildcommand to create the image and tag it with a image name and version
- Use the
docker pushcommand to push the resultant image to the Docker Hub
The process is no doubt simple but there are a few things that you would have observed if you have been using it for a while. These points are based on my experience, doing both local development and while teaching Docker to a wide range to developers:
docker buildrequires that you have a got a good network in place to pull down the dependencies. Ideally you would eventually look at moving this step to part of an overall build system.
- It takes a long time to execute the
docker pushcommand, that moves the image to the Docker Hub. The problem gets compounded when your Docker image files are big.
- Integrating other Container Registries like Google Container Registry is not simple. It takes some settings to make it work with the standard Docker tools.
- In many scenarios, the network is such a limiting factor that I often advise that when learning Docker or using Docker, the Cloud is your best friend. I suggest folks to have a VM in the cloud and then get amazed at the speed at which the
docker build/pushcommands run. It seriously boosts your efficiency while working with the Docker tool set.
What we are going to see next is how Google Container Builder addresses the above by giving you an environment in the Cloud that will speed up your Container builds, automatically push the images to Google Container Registry. We are only scratching the surface of Google Container Builder that has the capabilities for you to integrate with your build process, trigger events to notify external users, custom build scripts and more.
Using Google Container Builder to build Docker images
We are going to assume here that you already have your application and the associated Dockerfile to help build your Docker image. If not, you can pull a simple project from here (instructions on doing that are given later in this post):
We are going to be using the Google Cloud SDK that provides us command line utilities (commands) to work with Container Images. It contains commands to build, list and perform other operations with Container images.
I shall assume that you are familiar with the Google Cloud Platform (GCP) to the extent that you have a billing account, can navigate the GCP Console, have a Google Cloud Platform project and most importantly have downloaded and initialized the Google Cloud SDK.
If the above is true, we need to ensure a few things as given below:
- Go to the GCP Console for your project and ensure that the Google Container Builder API is enabled. From the console, visit API Manager → Library as shown below:
Type in Container builder and it should bring up the Google Container Builder API. Click that and make sure that you enable it.
2. From the console or terminal, we will assume that you have authenticated your Google Cloud SDK and set the default project with a set of commands as shown below:
$ gcloud auth login and
$ gcloud config set project [YOUR_PROJECT_ID]
The first thing that we are going to do is have a simple application with its associated Dockerfile. I have provided a project that is just about enough to see this process in action.
Simple pull in the project from here:
Google-Container-Builder-Demo - Sample Project to build Docker image
Assuming that you have
git installed on your local machine, do the following in a specific directory of your choice:
Great! All we need to do next is to submit our image build to Google Container Builder. That’s it.
gcloudtool command to submit a build to the Container Builder environment in the Google Cloud is as follows:
$ gcloud container builds submit --tag gcr.io/[PROJECT-ID]/[IMAGENAME]:[VERSION] .
The command is easy to understand:
- It submits a build to the Google Container Builder.
tagoption is what you know from Docker. In this case, the image that will be finally built will be pushed to the Google Container Registry, which is a fast and reliable Container Image hosting service. The prefix for that is gcr.io. The next two parts are your project name in Google Cloud Platform and then the Docker Image and Version.
- The last parameter is the
.which is the directory in which the
Dockerfilecan be located.
In my case, my GCP project is named mindstormclouddemo and I wish to name my Docker image as myapp and the version is 1.0. The command for that is as follows:
$ gcloud container builds submit --tag gcr.io/mindstormclouddemo/myapp:1.0 .
This will trigger the process as shown below and it makes for an interesting series of steps to see what is going on (I have annotated and formatted some parts of it with my comments):
$ gcloud container builds submit --tag gcr.io/mindstormclouddemo/myapp:1.0 .Creating temporary tarball archive of 37 file(s) totalling 34.1 KiB before compression. <-- This zips the folder and moves to Google Cloud Storage
Uploading tarball of [.] to [gs://mindstormclouddemo_cloudbuild/source/1497163737.39.tgz]
Logs are available at [https://console.cloud.google.com/gcr/builds/a329e313-5ff4-4d51-9a7b-78c1f11e13b0?project=mindstormclouddemo].
------------------------------------------------- REMOTE BUILD OUTPUT --------------------------------------------------
starting build "a329e313-5ff4-4d51-9a7b-78c1f11e13b0"--> NOW START THE BUILD. PICK UP THE ZIP FILE THAT WAS PUT IN CLOUD STORAGE
Fetching storage object: gs://mindstormclouddemo_cloudbuild/source/1497163737.39.tgz#1497163745785739
/ [1 files][ 18.1 KiB/ 18.1 KiB]
Operation completed over 1 objects/18.1 KiB.
Already have image (with digest): gcr.io/cloud-builders/docker
Sending build context to Docker daemon 79.36kB--> THE STEPS BELOW SHOULD BE FAMILIAR TO EVERYONE WHO HAS DONE A DOCKER BUILD. IT IS EXECUTING THE COMMANDS THAT IT HAS FOUND IN DOCKERFILE
Step 1/8 : FROM python:2.7
2.7: Pulling from library/python
ef0380f84d05: Already exists
24c170465c65: Pulling fs layer
4f38f9d5c3c0: Pulling fs layer
4125326b53d8: Pulling fs layer
35de80d77198: Pulling fs layer
ed5d9d659d3d: Pulling fs layer
cdb5322408aa: Pulling fs layer
9c9c3d6f5264: Pulling fs layer
24c170465c65: Verifying Checksum
24c170465c65: Download complete
24c170465c65: Pull complete
4f38f9d5c3c0: Verifying Checksum
4f38f9d5c3c0: Download complete
35de80d77198: Verifying Checksum
35de80d77198: Download complete
4f38f9d5c3c0: Pull complete
cdb5322408aa: Verifying Checksum
cdb5322408aa: Download complete
ed5d9d659d3d: Verifying Checksum
ed5d9d659d3d: Download complete
4125326b53d8: Verifying Checksum
4125326b53d8: Download complete
9c9c3d6f5264: Verifying Checksum
9c9c3d6f5264: Download complete
4125326b53d8: Pull complete
35de80d77198: Pull complete
ed5d9d659d3d: Pull complete
cdb5322408aa: Pull complete
9c9c3d6f5264: Pull complete
Status: Downloaded newer image for python:2.7
Step 2/8 : MAINTAINER Romin Irani "firstname.lastname@example.org"
---> Running in 10673c027bb1
Removing intermediate container 10673c027bb1
Step 3/8 : COPY . /app
Removing intermediate container 2b7ef1e2af6d
Step 4/8 : WORKDIR /app
Removing intermediate container bafd9afe6536
Step 5/8 : RUN pip install -r requirements.txt
---> Running in 73260323b338
Collecting flask (from -r requirements.txt (line 1))
Downloading Flask-0.12.2-py2.py3-none-any.whl (83kB)
Collecting itsdangerous>=0.21 (from flask->-r requirements.txt (line 1))
Downloading itsdangerous-0.24.tar.gz (46kB)
Collecting Jinja2>=2.4 (from flask->-r requirements.txt (line 1))
Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB)
Collecting Werkzeug>=0.7 (from flask->-r requirements.txt (line 1))
Downloading Werkzeug-0.12.2-py2.py3-none-any.whl (312kB)
Collecting click>=2.0 (from flask->-r requirements.txt (line 1))
Downloading click-6.7-py2.py3-none-any.whl (71kB)
Collecting MarkupSafe>=0.23 (from Jinja2>=2.4->flask->-r requirements.txt (line 1))
Building wheels for collected packages: itsdangerous, MarkupSafe
Running setup.py bdist_wheel for itsdangerous: started
Running setup.py bdist_wheel for itsdangerous: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/fc/a8/66/24d655233c757e178d45dea2de22a04c6d92766abfb741129a
Running setup.py bdist_wheel for MarkupSafe: started
Running setup.py bdist_wheel for MarkupSafe: finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/88/a7/30/e39a54a87bcbe25308fa3ca64e8ddc75d9b3e5afa21ee32d57
Successfully built itsdangerous MarkupSafe
Installing collected packages: itsdangerous, MarkupSafe, Jinja2, Werkzeug, click, flask
Successfully installed Jinja2-2.9.6 MarkupSafe-1.0 Werkzeug-0.12.2 click-6.7 flask-0.12.2 itsdangerous-0.24
Removing intermediate container 73260323b338
Step 6/8 : ENTRYPOINT python
---> Running in 9b562217e6f0
Removing intermediate container 9b562217e6f0
Step 7/8 : CMD app.py
---> Running in c6406b0f22b4
Removing intermediate container c6406b0f22b4
Step 8/8 : EXPOSE 8080
---> Running in 2600ada9fd32
Removing intermediate container 2600ada9fd32--> THE IMAGE IS BUILD SUCCESSFULLY NOW
Successfully built c75fbbe837f9
Successfully tagged gcr.io/mindstormclouddemo/myapp:1.0--> PUSH IT TO GOOGLE CONTAINER REGISTRY NOW WITH THE TAGGED NAME and VERSION
The push refers to a repository [gcr.io/mindstormclouddemo/myapp]
ecf5c2e2468e: Layer already exists
201187968504: Layer already exists
bb07d0c1008d: Layer already exists
4902b007e6a7: Layer already exists
007ab444b234: Layer already exists
1.0: digest: sha256:3bc4dacb99b2e4234bfda8d8e617dbf96320d7d79b1a81f9d80ce1b7ef7888fe size: 2431
------------------------------------------------------------------------------------------------------------------------ID CREATE_TIME DURATION SOURCE IMAGES STATUS
a329e313-5ff4-4d51-9a7b-78c1f11e13b0 2017-06-11T06:49:09+00:00 49S gs://mindstormclouddemo_cloudbuild/source/1497163737.39.tgz gcr.io/mindstormclouddemo/myapp:1.0 SUCCESS
As you can see from the build process above, every step of it is provided to you from the process that Google Container Builder has launched in the cloud. I have annotated the output above with some comments to help you understand what is going on and if you have dealt with a Docker build before, this process should be familiar.
Do note that each build process thus fired from the
gloud container buildcommand is unique. You can get a status/logs on the build too via the
gcloud container buildscommand via the
log sub command.
One important thing to note from here is the time that it took to complete the build. In my case, it was 49 seconds as seen in the final statement that it printed in the build output. This is important because there is a free tier that Google Container Builder provides i.e. First 120 build-minutes per day per billing account are free.
Viewing Container Images list
You can view the list of Container images via the gcloud command as shown below:
$ gcloud container images listA sample output (from my system) is shown below:NAME
Do explore the
gcloud container command for multiple other operations that you can do with Container images like tagging, untagging, deleting and more.
Container Registry in Google Cloud Console
If you use the Google Cloud Console more than the command line (I do) — you can take a look at Google Container Registry option in the console.
This brings up the main screen of Container Registry, where you can see your image that just got built:
What is interesting here is that you can also view the Build History for the builds that you submitted. You can view the build process output in detail over here:
Click on any of the Build instances and you should see the build process output.
You could also use Stackdriver Logging to check out your build logs. Container Builder is a first class citizen with StackDriver too:
You can view the build output logs here too:
You might be wondering if this whole process is free for you as a user. It sounds too good to be true since there is a compute environment that is being provisioned for you to execute the build, your files are zipped and stored in Google Cloud Storage and there could be network egress for your Container images.
I suggest that you should look at the pricing page for full details, but here are some points at a glance.
- Google Container Builder comes with a free tier. You are provided 120 build minutes per day for free. After that you are charged at $0.0034 / build-minute. Again the free tier minutes are a promotion and subject to change.
- Now let’s come to Container Registry where our Container images are getting stored. Check out the pricing for that too. To quote the official documentation on pricing, “ Container Registry only charges for the Cloud Storage storage and network egress used by your Docker images.”
Hope you enjoyed this tutorial. What we have seen is one area that Container Builder can streamline for you. It has a lot more capabilities like custom builds, triggers, notifying users in external apps like Slack when certain Build events happen, tying into Google Source Repositories to kick in builds when a code is pushed, etc. Do check those details out too.