Playing with Caffe and Docker to build Deep Learning models.

Ever tried installing Caffe in your system? I failed. There are hundreds of blog posts written on it and every approach is different. So I looked up at Docker and found it very helpful. This tutorial will help those people who want to build deep learning models independent of platform without much worrying about installation setups.

Caffe — Docker

Docker & its Eco-system:

For a long time, I had an aversion for Docker. I thought they were just to make things complex and was some fancy ‘DevOps’ thing that I could never wrap my head around. But this statement isn’t entirely correct as I found out later, It also helps Data scientists while building ML/DL models in the following ways.

signup for my AI newsletter

1. Install a relevant docker image and you no need to worry about installation set-ups. Pull relevant docker image and you are done.
2. Work platform independent. No matter whatever system you are working on, your code runs.
3. Share code with ease and never ever you will say “Okay! But that ran on my system”.

A lot of people have written about Docker and its advantage. My goal for this post will be to give you an hands on experience on building deep learning models using Docker. This is somewhat I didn’t find it on Internet and with the help of my colleague Prathamesh Sarang, I have gained enough knowledge on Docker and thought of sharing my experience.

Why Caffe?

Caffe is one of the most popular Deep learning framework developed by Berkeley Vision and Learning Center(BVLC).

Its primary advantage is command line interface and Model Zoo. All the popular deep learning models have been developed on caffe and made available in Model Zoo. RCNN, Fast RCNN and Faster RCNN ( Object detection algorithms), which I need to implement and experiment around for my project were developed using Caffe.

At least for image classification purpose, I loved Caffe. It took me just 6 hours (minus the time I spent on Installation setup) to learn Caffe and run my first deep learning model.

Though this part of the tutorial primary focus is on “Using Caffe with Docker”. All Deep Learning frameworks have their own Docker images made available. All you need to do is just pull them from Docker hub and run in your local systems.

Docker resources:

My colleague Prathamesh Sarang has written about Docker and that was my starting point. I will share along a few other links which helped me in the process:

• Docker for Data Science
• Docker official tutorial
• Docker Video lectures

There are 4 steps in building deep learning models

• Install Docker (Once in a system)
• Create a Docker file (Understanding it is of primary importance)
• Run the Docker file.
• Experiment with Deep learning models.

Install Docker:

Its platform dependent and you can check the official Docker documentation for more details . Below are the instructions to download Docker in Ubuntu.

> sudo apt-get update 
> sudo apt-get install -y docker.io # this will install docker in your system 
> docker -v # should give you the docker version you are using.
> docker run hello-world # Should successfully download the docker and print the following msg.
"Hello from Docker!
This message shows that your installation appears to be working correctly."

Files Structure

folder
├── data
│   └── readme.md
├── Dockerfile
├── notebooks
│   └── readme.md
├── README.md
└── requirements.txt

Make the Dockerfile

Below is a simple Dockerfile.

I will be explaining each line below. To run the above file, you need to go to the particular folder and execute the following command

> docker build -t name . #name: Image name of your choice. ex: model1
> docker run -p 8887:8888 -v ~/folder/notebooks:/model1/notebooks -v ~/folder/data:/model1/data -d name #

FROM bvlc/caffe:cpu : This is the Docker image you are pulling from Docker hub. Officially caffe has released two Docker images — one for CPU and the other for GPU. We are downloading the CPU version here. All the required software is installed inside the Docker image and you just need to run it for building the models. Running this for the first time, will take some time for downloading the image. below is a simple Dockerfile to check the version of caffe we are using.

FROM bvlc/caffe:cpu
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
ADD . /app
CMD ["python", "-c", "import caffe ; print(caffe.__version__)"]

install caffe using docker

We can see that it consists of caffe 1.o.0.

Add . /model1 : Add all the files in the folder into the Docker container that we will create later on.

WORKDIR /model1 : This will convert the model1 into a working directory from where we will be operating primarily.

VOLUME [“/model1/data”, “/model1/notebooks”]

Volumes are very important. Following are the advantages:

• It is sometimes not advisable to keep all your files inside your Docker images, Say copying your 100 GB of image data into Docker image and sharing it with someone. It is not feasible.
• Docker run by converting an image into a container. These containers just images running as instances and perish/can be deleted once their purpose is done (eg. to do model training).

So creating volumes will help. volumes are the way Docker communicates with your local systems. Here I kept all my data in data folder and all the code (jupyter notebooks) are saved inside my notebooks folder. Make sure that you have a data folder and notebooks folder before running the Docker file.

EXPOSE 8888: This will expose the Docker port 8888. you can connect it to your local port.

CMD jupyter notebook — no-browser — ip=0.0.0.0 — allow-root — NotebookApp.token=’demo’: This will help you run your jupyter notebook.

Run the Dockerfile

Now your Dockerfile is completed. You need to run it. In the below image you can see the commands and their outputs.

> docker build -t model2 . # This will build a docker file and create an image. 
> docker images # will show the images in your local system 

building a docker image

> docker run -p 8887:8888 -v ~/Desktop/caffee/model2/notebooks:/model1/notebooks -v ~/Desktop/caffe/model2/data:/model1/data -d model2

The above command starts a browser with jupyter notebook. The command is self-explanatory, you are just telling Docker to which folders should communicate with which volumes.

Now go to the web browser and run localhost:8887. you should see the below output. Enter the token “demo”.

jupyter notebook initial screen

After entering token, you will get the below screen

jupyter notebook

Now your local data folder and the data folder inside the Docker will communicate. the jupyter notebook in notebooks folder will be available outside Docker and you can just share the Docker file and notebooks folder.

Experiment with Deep learning models

Create a jupyter notebook inside the “notebooks” folder. Once, the above image shows up in your browser, open the notebooks folder and create your jupyter notebook.

Here is the Jupyter notebook I created for this work.

Prakashvanapalli/caffe_blogpost

Here is the repo which contains all code.

Prakashvanapalli/caffe_blogpost

Extras:

• How to generate requirements.txt file?

# create an environment of your choice 
> conda create -n caffe python=2.7
# Install the required softwares 
> pip install jupyter 
# generate requirements.txt file
> pip freeze >requirements.txt

• Important Docker commands

> docker images # Will show all the images in the local system 
> docker ps # will show the present running containers 
> docker ps -a # will show all the containers in the local system
> docker ps -l # display only 1st container 
> docker ps -n 2 # display first 2 containers 
# running a docker file 
> docker build -t name . 
> docker run -it name # will run the docker file from command line 
> docker run -d name # will run the docker in the backend 
> docker logs container_id # will show the logs of the container running in the backend 
> docker commit container_id name:tag # will save the container as a new docker image with specific tag. 
# deleting docker images 
> docker rmi -f name:tag # if tag is latest, just use name 
> docker rmi -f $(docker images | grep "<none>") # using regex to delete several docker images at a time . here i am deleting all the dockers which have name <none>.
# running docker interactively like a remote server 
> docker run -it name:tag bash 
# How to push containers to cloud ?
> docker login #In case if you don't have an account create one. 
> docker push username/repository:tag 

Ending Notes:

I hope you have liked this blog post. like and Share if you do. Also please drop in comments if you have any.

Would like to thank Prathamesh Sarang for his help in making me understand Docker. With someone who can explain the nitty-gritties of Docker, working with Docker took me a day’s time.

P.S: There’s always a better way of organizing code.