Installing Docker on Gcloud VMs

Ever since I started programming, I have been slightly overwhelmed with all of the amazing options of where and how to best run all of the various programs I am working on. First I started using AWS Sagemaker largely due to its ease of opening Jupyter notebooks. Next I started using Gcloud which I have started to like more. However, one of the biggest issues I have has always been starting a new project or running another persons code from github initially is the packages that I have to change each time. After speaking with my friend who works at Red Hat he pretty much was like ya you need to use docker. So this is going to be a walkthrough on how I setup docker on my gcloud VMs in order to best be able to run programs. All of the codes below are the scripts used to get docker up and running without the need to go through the large amount of docs from docker and google.

Containers and virtual machines Overview:

A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.

By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtualaccess to host resources through a hypervisor. In general, VMs provide an environment with more resources than most applications need.

Local Install and run Docker Desktop for Mac Info :

Double-click Docker.dmg to open the installer, then drag Moby the whale to the Applications folder.

https://docs.docker.com/docker-for-mac/install/

Docker Toolbox Info:

https://docs.docker.com/docker-for-mac/docker-toolbox/

Docker Toolbox installs docker, docker-compose, and docker-machine in/usr/local/bin on your Mac. It also installs VirtualBox. At installation time, Toolbox uses docker-machine to provision a VirtualBox VM called default, running the boot2docker Linux distribution, with Docker Engine with certificates located on your Mac at $HOME/.docker/machine/machines/default.

INSTALL DOCKER CE

1. Update the apt package index.

$ sudo apt-get update

2. Install the latest version of Docker CE and container, or go to the next step to install a specific version:

$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Docker and GCLOUD Integration before you begin…

1. Installed the most recent version of the Cloud SDK, which includes the gcloud command-line tool
2. Installed Docker
3. Have access to the registries which you will be pushing to and pulling from
4. Configured Docker to use gcloud as a credential helper, or are using another authentication method. To use gcloud as the crediential helper, run the command:

gcloud auth configure-docker

When creating a VM or an instance template, you can provide a Docker image name and launch configuration. Compute Engine will take care of the rest including supplying an up-to-date Container-Optimized OS image with Docker installed and launching your container when the VM starts up

Uninstall old versions

Older versions of Docker were called docker, docker.io , or docker-engine. If these are installed, uninstall them:

$ sudo apt-get remove docker docker-engine docker.io containerd runc

Create your first repository

To create a repo:

1. Sign in to Docker Hub
2. Click on Create Repository on the Docker Hub welcome page:

Build and push a container image to Docker Hub from your computer

Start by creating a Dockerfile to specify your application as shown below. (More on Dockerfiles here)

cat > Dockerfile <<EOF
FROM busybox
CMD echo "Hello world! This is my first Docker image."
EOF

1. Run docker build -t <your_username>/my-first-repo . to build your Docker image
2. Test your docker image locally by running docker run <your_username>/my-first-repo
3. Run docker push <your_username>/my-first-repo to push your Docker image to Docker Hub

You should see output similar to:

Preparing a container for deployment

Choose one of the methods below to make your container image accessible to Compute Engine:

• Upload your Docker image to Google Container Registry.
• Use any publicly available container images from Docker Hub or other registries.

Deploying a container on a new VM instance

You can deploy a container on a new VM instance using the Google Cloud Platform Console or the gcloud command line tool.

gcloud compute instances create-with-container [INSTANCE_NAME] \
     --container-image [DOCKER_IMAGE]

Updating a container on a VM instance:

When you update a VM running a container, Compute Engine performs two steps:

• Updates container declaration on the instance. Compute Engine stores the updated container declaration in instance metadata under the gce-container-declaration metadata key.
• Stops and restarts the instance to actuate the updated configuration, if the instance is running. If the instance is stopped, updates the container declaration and keeps the instance stopped. The VM instance will download the new image and launch the container on VM start.

gcloud compute instances update-container nginx-vm \
    --container-image gcr.io/cloud-marketplace/google/nginx1:1.13

Deploying Containers on VMs and Managed Instance Groups

You can deploy a container to a new managed instance group using Google Cloud Platform Consoleor the gcloud command line tool by following these steps:

1. Create an instance template, based on a Docker image.
2. Create a managed instance group from the new instance template.

gcloud compute instance-templates create-with-container [TEMPLATE_NAME] \
 --container-image [DOCKER_IMAGE]

• Compute Engine
• Documentation

This page describes how to deploy Docker images on Google Compute Engine virtual machine instances and managed instance groups.

To deploy and launch your container on a Compute Engine VM or a managed instance group, you provide a Docker image name and configure how your container should run when creating a VM or an instance template. Compute Engine will take care of the rest including supplying an up-to-date Container-Optimized OS (COS) image with Docker installed and launching your container when the VM starts up. For more information on the advantages of deploying containers on VMs, readChoosing to deploy containers on VMs and instance groups below.

Before you begin

• If you want to use the command-line examples in this guide:

Install or update to the latest version of the gcloud command-line tool.

Set a default region and zone.

• If you aren’t familiar with Containers, read What are Containers and their benefits.
• If you aren’t familiar with Docker, read the Docker documentation.
• Read about Container-Optimized OS.
• Read about Managed Instance Groups.

Choosing to deploy containers on VMs and instance groups

By deploying containers on Compute Engine, you can simplify application deployment while controlling your VM infrastructure.

• Manage VMs that are running containers in the same way you would treat any other VM when configuring and managing your Compute Engine infrastructure.
• Create scalable services using managed instance groups running containers, which offer features like autoscaling, autohealing, rolling updates, multi-zone deployments and load balancing.
• Use familiar processes and tools such as the gcloud command-line tool or the Compute Engine API to manage your VMs with containers.

Alternatively, you might consider deploying to Kubernetes Engine to:

• Run a large number of microservices
• Have faster container startup time
• Take advantage of Kubernetes automated orchestration, including auto upgrades, node auto repair, and autoscaling

Running each microservice on a separate VM on Compute Engine could make the operating system overhead a significant part of your cost. Kubernetes Engine allows you to deploy multiple containers and groups of containers for each VM instance, which can lead to more efficient host VM utilization for microservices with a smaller footprint.

How deploying containers on Compute Engine works

The common methods of deploying software onto a Compute Engine VM instance include:

• Deploying software on VM boot using a startup script or cloud-init.
• Creating a custom boot disk image with software pre-installed.

Both of the above methods combine the tasks of configuring the application and setting up the host operating system environment. As the developer, you must carefully track and resolve any runtime dependencies. For example, if two applications running on a VM use different versions of the same library, you must install both versions and point to them through system variables.

You can also deploy software in a container directly onto a VM instance or to a managed instance group. Each container carries both application software and the required libraries and is isolated from the host OS applications and libraries. Containers can be easily moved between deployment environments without dealing with conflicting library versions in a container and its host OS.

The following is the process for deploying a container on Compute Engine:

1. You bundle your application and required libraries into a Docker image and publish the image to Container Registry (or publish publicly on Docker Hub or other registry).
2. You specify a Docker image name and the docker run configuration when creating a VM instance or an instance template for a managed instance group.

Compute Engine executes the following tasks after you make the request to create a VM instance or instance template:

1. Compute Engine creates a VM instance or an instance template using a Google-provided Container-Optimized OS image. This image includes Docker runtime and additional software, responsible for starting your container.
2. Compute Engine stores your container settings in instance metadata under the gce-container-declaration metadata key.
3. The Container-Optimized OS image pulls the container image from the repository and starts the container when the VM starts, using the docker run command configuration stored in the instance’s metadata.

Limitations

• You can only deploy one container for each VM instance. Consider Kubernetes Engine if you need to deploy multiple containers per VM instance.
• You can only deploy containers from a public repository or from a private repository at Google Container Registry. Other private repositories are currently not supported.
• You cannot map a VM instance’s ports to the container’s ports (Docker’s -p option).
• You can only use Container-Optimized OS images with this deployment method.
• You can only use this feature through the Google Cloud Platform Console or the gcloudcommand-line tool.

Preparing a container for deployment

Choose one of the methods below to make your container image accessible to Compute Engine:

• Upload your Docker image to Google Container Registry.
• Use any publicly available container images from Docker Hub or other registries.

Note: Consider running your container image builds on Cloud Build, with none of the overhead of managing your own build servers.

Deploying a container on a new VM instance

You can deploy a container on a new VM instance using the Google Cloud Platform Console or the gcloud command line tool.

Use the gcloud compute instances create-with-container command:

gcloud compute instances create-with-container [INSTANCE_NAME] \
     --container-image [DOCKER_IMAGE]

For example, the following command creates a new VM instance named nginx-vm which will launch and run Docker image gcr.io/cloud-marketplace/google/nginx1:1.12.

gcloud compute instances create-with-container nginx-vm \
     --container-image gcr.io/cloud-marketplace/google/nginx1:1.12

Learn more about gcloud compute instances create-with-container command.

You must always specify a full Docker image name when using a public image from Docker Hub. For example, specify the following image name to deploy an Apache container image:

docker.io/httpd:2.4

Updating a container on a VM instance

You can update a Docker image and configuration options to run the container on a VM instance using Google Cloud Platform Console or gcloud command line tool.

When you update a VM running a container, Compute Engine performs two steps:

• Updates container declaration on the instance. Compute Engine stores the updated container declaration in instance metadata under the gce-container-declaration metadata key.
• Stops and restarts the instance to actuate the updated configuration, if the instance is running. If the instance is stopped, updates the container declaration and keeps the instance stopped. The VM instance will download the new image and launch the container on VM start.

Note: If you specify a Docker image labeled with latest, the VM instance downloads the latest image and launches a container from the new image each time the VM starts. Do not use the latest label in production if you depend on a specific image version.

Update container declaration using the gcloud compute instances update-container command. For example:

gcloud compute instances update-container nginx-vm \
    --container-image gcr.io/cloud-marketplace/google/nginx1:1.13

This command sets the container image to gcr.io/cloud-marketplace/google/nginx1:1.13 and restarts the instance to actuate the changes. You can also update any of the properties described in Configuring Options to Run Your Container by adding corresponding flags.

Once the instance restarts, it will download the new container image and start the container with the new configuration.

Deploying a container on a managed instance group

You can deploy a container to a new managed instance group using Google Cloud Platform Consoleor the gcloud command line tool by following these steps:

1. Create an instance template, based on a Docker image.
2. Create a managed instance group from the new instance template.

Create an instance template for running Docker images using the gcloud compute instance-templates create-with-container command:

gcloud compute instance-templates create-with-container [TEMPLATE_NAME] \
 --container-image [DOCKER_IMAGE]

You can also configure options to run your container if desired.

For example, the following command creates a new instance template with name nginx-template which includes information about the Docker image. A VM instance created from this template will launch and run the Docker image gcr.io/cloud-marketplace/google/nginx1:1.12 when the VM starts.

gcloud compute instance-templates create-with-container nginx-template \
     --container-image gcr.io/cloud-marketplace/google/nginx1:1.12

Next, create a managed instance group using the new instance template.

Now that you have an instance template, you can create a managed instance group using the instance template. For example, to create a managed instance group using the gcloud tool with the nginx-template that you just created, run the following command:

gcloud compute instance-groups managed create example-group \
    --base-instance-name nginx-vm \
    --size 3 \
    --template nginx-template

Updating a managed instance group running a container

This is a Beta release of Instance Group Updater. This feature is not covered by any SLA or deprecation policy and might be subject to backward-incompatible changes.

You can update a managed instance group to deploy a new version of a Docker image or a new version of the Container-Optimized OS image.

Updating a managed instance group to a new version of a container image

You can deploy a new version of a Docker image to a managed instance group using the Managed Instance Group Updater, in three steps:

1. Prepare a new Docker image for deployment.
2. Create an instance template based on the new Docker image, the same way you would create a container-based template.
3. Update a managed instance group with the new instance template using the Managed Instance Group Updater.

Updating a managed instance group to a new version of Container-Optimized OS image

Google updates Container-Optimized OS images regularly, and you might want to apply those updates to your containerized managed instance groups without changing your Docker image. You can update a managed instance group to a new version of Container-Optimized OS image using Google Cloud Platform Console or the gcloud command line tool in two steps:

1. Create an instance template based on the current version of your Docker image, the same way you would create a container-based template for a new managed instance group. The latest supported version of a Container-Optimized OS image will be used by default.
2. Update a managed instance group with the new instance template using Managed Instance Group Updater.

Connecting to a container using SSH

You can connect to a container on a VM using SSH. Using the gcloud tool, run the gcloud compute ssh command with the --container flag:

gcloud compute ssh [INSTANCE_NAME] \
   --container [CONTAINER_NAME]

where:

• [INSTANCE_NAME] is the name of the VM instance.
• [CONTAINER_NAME] is the name of the container.

Learn more about gcloud compute ssh command and its arguments

Quickstart for Container Registry: Configure docker to use the gcloud command-line tool as a credential helper

Before you can push or pull images, you must configure Docker to use the gcloud command-line tool to authenticate requests to Container Registry. To do so, run the following command (you are only required to do this once):

gcloud auth configure-docker

Tag the image with a registry name

Before you push the Docker image to Container Registry, you need to tag it with its registry name. Tagging the Docker image with a registry name configures the docker push command to push the image to a specific location. For this quickstart, the host location is gcr.io.

To tag the Docker image, run the following command:

docker tag quickstart-image gcr.io/[PROJECT-ID]/quickstart-image:tag1

where:

[PROJECT-ID] is your Google Cloud Platform Console project ID, which you need to add to your command

gcr.io is the hostname

quickstart-image is the name of the Docker image

tag1 is a tag you're adding to the Docker image. If you didn't specify a tag, Docker will apply the default tag latest.

You are now ready to push the image to Container Registry.

Push the image to Container Registry

Once docker has been configured to use gcloud as a credential helper, and the local image is tagged with the registry name, you can push it to Container Registry.

To push the Docker image, run the following command:

docker push gcr.io/[PROJECT-ID]/quickstart-image:tag1