Everything you need to know about it

Image for post
Image for post
Photo by Snappy Shutters on Unsplash

As we all know that necessity is the mother of invention.
Let’s see some needs and issues that are solved by Dockerising Applications.

To develop a dynamic application we need the following:
1: Web Server (like node+ express or Django etc)
2: Database (MongoDb or PostgreSQL etc)
3: Messaging Service (RabbitMQ or SQS etc)
4: Orchestration tool (Ansible etc)

Issues one faces while developing, deploying, and setting up applications are mentioned below:

1: We need to ensure the compatibility with the underline OS version (Windows, Linux, etc) of the packages required for the above services
2: We need to ensure the compatibility with the libraries installed on the OS, one service might require some specific version of some specific library and which might not be the case for all services
3: It’s usually very difficult to onboard a new developer as you need to set up a new environment (sometimes multiple for dev/test/prod) and then install dependencies and follow other tailor-made instructions to get everything up and running
4: Since we do the development work in a dev environment and finally runs the code on the prod environment, we can not guarantee that the code will run exactly the same as we might be developing in windows and running on Linux in prod.

Docker to the rescue

1: Now with docker, we can run each component/service (database, web server, etc) in their separate container having its own dependencies and own libraries and thus no compatibility issues.

2: We have to configure docker only once and all the developers can have their’s system up and running with simple docker run command irrespective of what underlying OS they are using but need to have docker installed in their’s systems.

3: Since code actually runs as a container on the Docker host for all environments and hence it will run exactly the same in development and prod irrespective of the underlying OS.

What is Docker or Dockerization

Docker is a containerization system with many other useful tools,
It basically allows you to run the application anywhere (where docker can be installed) independent of how and where the application was created.

Note: Application/Services runs as a container in docker

Image for post
Image for post
Containers running on docker which is installed on the operating system

Containers

Isolated environments that can have their own processors, network interfaces, etc but share the same OS kernel.

Note: Docker utilizes LXC containers.

How docker works

See, every operating system consists of 2 things:
1: OS kernel
2: Softwares on the top of it

Note: Docker actually utilizes the underlying kernel and not the operating system.

So even if the machine on which docker installed is of ubuntu, docker can still install containers based on fedora, etc as the underlying kernel is same ie -linux

and hence you’ll not be able to run Linux based container on a docker host based on windows (unless using Linux VM under the hood)
To solve this we can use docker for windows/mac which I’ll explain later

Difference between Containers and VM

Simply stating:
Every Virtual Machine’s Application has its own Operating System but this is not the case for Docker containers/Applications.

Image for post
Image for post
Applications having their dedicated operating system managed by hypervisors
Image for post
Image for post
Docker- Every container sharing the same operating system

Hence a Virtual Machine will have
1: Higher utilization of resources as it runs multiple operating systems.
2: Higher disk space, in GBs as compared to MBs for docker
3: Boot up slower, in minutes as compared to seconds for docker

But that doesn’t mean you can always choose Docker over Virtual Machines, both have their own use cases like if you have to develop a software for a specific operating system then you have to use VM or case when you have to clone an OS into another machine, these things are not feasible with docker.

Rather, we can use both technologies to take advantage of their specialties and develop an integrated system like:

Image for post
Image for post

Getting started with docker

So thankfully, there already are containerized versions of popular applications available made by other developers on docker hub (public docker registry)

You can find images of the common OS, Services, Databases and use them using simple commands like:
docker run redis
docker run mongodb
docker run nodejs

I’ll explain the commands in details later

Docker Image

An application is packed as an image that can run on any host that has Docker installed on it.

The application can be any tech like:
webserver, DB, or even your custom application or software product.

Image is a package/template used to create one or more containers.
Rather containers that we discussed above are actually running instances of images only.

You can also create your own image and push on a docker up repository or any private registry provided by your cloud providers.

You’ll have more clarity on this as we proceed further, meanwhile bear with me and these jargons.

Installing docker

Follow these guidelines

So I hope the concepts are cleared theoretically, now it’s time to actually implement the concepts.

Docker Host

Where docker is installed like Ubuntu OS etc

Everything in docker like building images or running containers based on them are done using commands.

Let’s understand the popular and most useful docker commands in detail.

Docker commands

This command is use to start a docker container using a docker image.
eg: docker run hello-world

This command will create a running containerize version of the imageName given.

Note: If the image is not already installed in the docker engine then it pulls out the image from the docker registry platform: Docker Hub

Note: Unless you specify some other registry, docker by default uses hub.docker.com as a public registry to pull images.

Image for post
Image for post
Output of the command mentioned above

eg: docker run nginx
It runs an instance of nginx image as a container

Note: Image is generally stored as accountName/Imagename on docker hub
if you only specify nginx that will mean that both account and image name is nginx

docker ps

This command will list all the running containers with some info like:
container id
name of the image we used to run containers
current status
name of the container (automatically generated if not given), etc

Image for post
Image for post

docker stop containerName

This command will stop the running instance of the image
eg: docker stop sad_wilson

Image for post
Image for post

docker ps -a

It will list all the containers (running + previously exited/stopped)

Like here, a container of image “nginx” was exited (check status column )but you still can list it using the above command

Image for post
Image for post

docker rm containerName

This command comes handy when you don’t want the exited containers lying around as they are still using RAM and other resources.

So below you can see that even if I’ve exited nginx container, it’s not completely removed

Image for post
Image for post

after using docker rm sad_wilson only docker-tutorial is up

Image for post
Image for post

docker images

To list the images downloaded

Image for post
Image for post

See even if I’ve exited the nginx and after that removed the container, the image for nginx is still installed so that when I containerize it the next time the docker host can use this image to create a running instance rather than pulling again from the docker registry

docker rmi imageName

This command will remove the downloaded image

Image for post
Image for post

docker pull imageName

This command comes handy when you only have to pull/download the image so that it can be used later when we use docker run command to containerize the image (At that time we don’t have to wait for downloading the image as it’s available locally)

Image for post
Image for post

Exiting a container

A container can exit in 2 ways:
1: Stoping the container manually
2: Container automatically exits when the process/application/service running in the container stops or crashes

For eg:
docker run ubuntu exits as soon as the container is created because there’s no actual process running inside ubuntu container rather it is used as a base image for other containers to develop applications.

Appending a command

You can instruct docker to run a command using docker run command

For eg:
docker run ubuntu sleep 5

here an instance of ubuntu image is launched as a container, a command is executed, and then the container exits after the process inside it stops which is to sleep for 5 seconds.

So, this time ubuntu’s container doesn’t exit immediately like mentioned above.

exec — Executing a command on a running container

In the above example we were able to run a command just after container is created, but what if we want to execute a command inside the already running container?

then exec comes into action.

for eg:
docker exec containerName cat /etc/hosts

where cat /etc/hosts is the command to list the hosts

Image for post
Image for post

Run- attach or detach

There are 2 modes to run containers

Mode 1:
when you run a docker run command as:
docker run nginx, It runs in the foreground or attached mode (able to see the output on the console you are using to run docker) but won’t be able to do anything on the same console until the container stops.

Mode 2:
we can also run our containers in the detach mode (background) and can perform actions on it using the same console by just adding -d before the imageName.

docker run -d nginx

now, nginx will run in the background/detach mode

obviously you can attach the container to the console later using:

docker attach containername/ID command

Run — tag

Image for post
Image for post

As you can see in the above screenshot, redis is attached with a tag
latest
latest denotes the highest version of the redis image available on the docker hub.

But you can also specify the tag/version for the image you want to download after confirming that it exists in the docker registry like:
docker run redis:4.0

Image for post
Image for post
See the tag here associate with the image is 4.0 rather than the latest

Run — stdin

Consider a small python app which prints out “hello {name}”
where name is variable which you enter using the console.

By default, the docker doesn’t listen to the input command as it runs in non-interactive mode.
Running the container in interactive mode means that you have to map standard input of the docker host to the container using “i” parameter like:
docker run -i imageName

Similarly, to display a message on prompt we need to map the standard output using “-t” parameter,

for both input and output use something like:
docker run -it imageName

Run — PORT mapping

Docker host or Docker engine:
The underlying host on which docker is installed

So does containers run on Docker engine?
No, containers run on docker which is installed on the host.

Note: Every container has its own IP address (internal IP) which is only accessed within the docker host, and users outside of the docker host can not access it.

hence we need to map the ports of the docker host and docker containers if we want to access the container from outside.

So consider you run a web application as a container on port 5000 and you want the end-users to access the application on port 80 then you’ll use command like:

docker run -p 80:5000 imageName

Here the application is running on port 5000 of the container is mapped with port 80 of the host and you can connect the application as:
192.168.1.5:80

where 192.168.1.5 is the Host’s IP address.

Now, it’s very easy to run even multiple instances of the application as:
1: docker run -p 80:5000 imageName
2: docker run -p 8000:5000 imageName
3: docker run -p 8001:5000 imageName

Run — Volume mapping

Let’s now understand how data is persisted in a docker container and docker host.

So every docker container has its own file system where the data is stored.

If I run a MySQL container using:
docker run mysql
The data will be stored in var/lib/mysql inside the container.

This leads to an issue that whenever the container is removed we’ll lose the data stored in the container service.

ie:
docker stop ContainerName
docker rm ContainerName

will remove the data as well

So if you want to persist or store the data even if the container is removed then we need to map the storage of the container to the docker host using the command:

docker run -v optional/datadir:var/lib/mysql mysql

This is called the mounting of the external directory to a folder inside the docker container.

Here the data will be stored in optional/datadir of the docker host which you’ll provide in the command.

Inspecting a container

docker ps we studied before was only able to display few details about all the containers but if we want to fetch more details about a specific container, use docker inspect command as:

docker inspect containerName
it will provide with more details like state, mounts, etc

I’ll be using this command for a few specific cases below so that you’ll understand it better.

Container Logs

As discussed previously if you run a container in detached mode using -d you’ll not be able to see the logs on the console.

Hence we need to use:
docker logs containerName/ID
to fetch the logs of the container

Docker environment variables

Exporting

export ENV_VARIABLE=value;

and then inside the code (python), you can use as:
variable = os.environ.get(‘ENV_VARIABLE’)

runtime variable

You can also always pass
env_variable inside run command as well like:

docker run -e ENV_VARIABLE=value imageName

Note: To inspect already exported environment variables you can use the inspect command listed above, it will list the env variables under the config section.

Docker images (here we go again :)

There are 2 main reasons for creating your own image:
1: You can not find the base image you need to use on the docker hub and create your own.
2: You decide to dockerize your own application for ease of shipping and deployment (in this case we usually upload our image in private registry)

Steps for creating image for your application:

1: Create a Dockerfile
Dockerfile is a file which instructs how to build a container from the image.

It consists of multiple Instructions followed by its arguments.

let say we want to manually deploy a web application, these are the steps that we’ll follow in general.

1: choosing os — ubuntu
2: update source repository using apt command
3: install dependencies using apt command
4: install python dependencies using pip command
5: copy the code/root folder of my code onto ubuntu server
6: Run the application

Now, Dockerfile which automates the above steps is written as:

FROM ubuntu

RUN apt-get update
RUN apt-get install python

RUN pip install flask

COPY . /opt/source-code

ENTRYPOINT FLASK_APP=opt/source-code/app.py flask run

Note:
1: all docker files start with FROM command
2: The string in the left in all caps is the instruction and latter is the argument

2: Run docker build command with docker file and tag name for the image

docker build Dockerfile -t my-web-app

This will create an image locally on your system

3: To make it publicly available on docker hub you can use the docker push command

→ docker push imageName

Now to avoid name clashes we generally choose name as accountName/imageName

Note: you can push image on private registry as well if you don’t want your app to be publically available.

Layered architecture

Whenever an image is built, it’s built-in a layered architecture.

Each instruction in the file creates layers and store it as cache with just the changes from the previous layer so that if it fails at any step can be continued where it left rather than starting over.

It is very helpful, as whenever we need to deploy new changes only the source code changed, and hence the above layers (like OS, installing deps, etc ) do not need to start from scratch rather it can be rebuilt using cache only.

Docker cmd VS entrypoint

If you see the Dockerfile of nginx you can see “CMD nginx” at the bottom of the file.

Note: you can override cmd command as we done in the ubuntu case above by running it as docker run ubuntu sleep 5

ie: Everything after ImageName is considered as CMD and original CMD command mentioned in the dockerFile is overriden.

But, ENTRYPOINT command and parameters can not be overridden rather all the arguments that are passed in the run command are appended after entry point as you can see below:

ie: if in ubuntu image, you mention ENTRYPOINT as [“sleep”]

then you just need to run the container as:

docker run modifiedUbuntuImageName 5 and it will sleep for 5 seconds and then exits.

Docker Networking

The bridge is the default network a container get attached to, but if you want your container to be attached to some other network you can specify that in run command like:
docker run ubuntu — network= None
or
docker run ubuntu — network=host

If multiple containers are hosted on the default (bridge network) then they can communicate with each other using the Internal IP assigned to each container or the name of the container

Note: we should not connect using assigned IP because there is no guarantee that the IP will be same after the system reboots and hence containerName is preferred (Docker built-in DNS helps to resolve the container using container names)

If you need external communication there are 2 ways to do it:
1: Map the ports as explained above.
2: Associate the container to the host network

Image for post
Image for post
by @kodekloud

User defined networks

By default, Docker creates a single internal bridge network to which all the containers are connected but sometimes we want the containers to be connected with different internal bridges then we can create our own internal bridge as:

docker network create\
— driver bridge
— subnet 182.18.0.0/16
networkName

Image for post
Image for post
by @kodekcloud

Note: if you need to inspect network config for the container

docker inspect containerName comes into action again and details are under network section which is inside network settings

Docker Storage

Now, If you remember the layered architecture when the image is build, all the layers are read-only, only when you run a container the final container layer is read-write.

Moreover, anything you write on the container layer is volatile, ie: it will be destroyed when container reboots or crashes.

To build a persistent storage:
1: Volume mounting: We can add a persistent volume to the container
This can be done by creating the volume:
docker volume create volumneName

and I can mount this volume as
docker run -v volumneName: /var/lib/mysql mysql

2: Bind mounting: we can directly mount with a folder as explained previously

Image for post
Image for post

Docker Compose

So we write multiple commands as:
docker run x
docker run y
docker run z

The better way of doing/manage this is by using docker-compose.

Using docker-compose we create a configuration file in YAML format (docker-compose.yml) and group together the services along with the options to run them.

After creating the compose file, we can just use a docker-compose up command to get the application running.

This is how a docker-compose/ file looks like:

Image for post
Image for post

These were the important docker concepts from my end :)

If you want to learn more about docker’s advance concepts I highly recommend you to read some articles I am mentioning below:

1: Read more about docker-compose at:
docker-compose
**super important

2: Read about Kubernetes (orchestration of the containers)

Thanks!

Written by

Aut viam inveniam aut faciam

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store