Intro into the Clouds — Part 3 — CaaS Compute
If you have got to this page without reading Part 1, I recommend to start with Part 1 first to know more about what is it about, why I decided to write it etc. (Or Part 2).
If you ready, let’s continue…
—
It is not possible to talk about Cloud Computing, without mentioning Containers.
This was definitely the hardest lecture to prepare. On the one hand Containers and Container Orchestration Systems are so important that there is no way to skip these topics in lectures about Cloud Computing. On the other hand, those topics are so immense that it is very easy to dedicate a whole course just to specifically this topic (and in fact many did, and you can find many great courses on Docker Containers and Container Orchestration Systems yourselves). So, the challenge for me here is to find a good balance and to write enough to explain the general purpose of those technologies without going in too much detail. If you found this lecture trying to learn those technologies in every nuance I think the only useful part you can find here is the links to Docker/K8S courses provided at the end of the lecture.
But, as we agreed, we won’t start by describing what it is, but instead we would rather start by formulating a problem and then we’ll see how each particular technology, Containers in this case, can solve it.
In this particular case (and that is probably one of the things that made this technology so widely adopted), there are actually several problems.
Reproducibility. Have you ever had such a situation: your program perfectly runs on your local PC, but when uploaded to the server — it stops working? Or this situation: your program is working weirdly on the server and has some bugs, but when you run it locally to reproduce (and hopefully) fix the problem — everything works fine?
Those situations are what is called the Reproducibility problem. The issue is, your program is an not isolated and self-sufficient technology, and not even an isolated and self-sufficient piece of software. Its work relies and depends on the OS version, the version of specific libraries, services, and tools installed on your PC, etc. So, if some of those libraries, services or tools are missing on your server, or just different from your local PC — your program may run a little bit differently…
Was there a solution to that? Well, potentially yes. Instead of using your PC directly, you could have a VM put on it, and then try to develop your program on this VM. Then, you can try and make a VM identical to your server and run your program there. However, to write a program (or at least to be efficient in doing that) you need special tools, which have to be installed on that VM (and that would bring libraries and services with it), and your “production” server won’t have the same tools. And Developers like to use many tools… (and install some of them, just in case…).
Not to mention that such an approach is not easy but resource consuming. And just imagine that you work on 2 versions of your program. Do you need 2 VMs?
Isolation vs OS Overhead. Earlier, we talked about the Isolation problem and discussed that running each program on a separate VM can be a potential solution to that problem. However, it is quite an expensive solution, because OS’s are designed to be able to do almost anything. Would you run a .NET program or Java? Or would you use Notepad? As a result, there are a lot of tools and services pre-installed on a VM. All those start with a VM start At the same time, for example, with Windows you won’t get a JVM pre-installed, but you get IIS…and Notepad.
While those pre-installed tools and services give you lots of flexibility, this flexibility has its cost:
- At minimum, all those tools and services take some space on your VM disk (and on your host disk as a result).
- All those tools and services, when running, consume CPU and RAM. And not all of them can be disabled.
- All those tools and services may bring instability to the environment caused by their interaction. What if your program tries to handle the biggest load while Antivirus decides to scan the disk at the same time?.
- And last but not least, those tools/services bring potential security risks. The more things are installed and running — the more potential attack vectors may exist.
So, while a separate VM (and a separate OS) can potentially solve the Isolation problem, this solution brings big overheads and, as a result, it is usually used only by big companies AND separation is done only into reasonably big components.
Note: The idea of service-oriented architecture (SOA) was created long before containers. However, only creating containers (and container orchestration systems, which we will discuss next) reduced overheads to really allow this idea to be transformed to the microservice architecture.
Note: Another problem of “full-scale” OSs is the start time. Since the OS needs to load and start all those services, it can take a significant amount of time. That can result in significant downtime (in case a VM needs to be restarted) and limits speed of scaling (adding new VMs) for such systems.
So, what if you can have a very small OS dedicated to only 1 purpose (e.g. running your Python code), that you run on your local machine to develop your program, and then run the same OS with the same program as a VM on a server within your production environment?
That became possible with Docker containers.
Note: Here it is important to mention that Docker containers have not emerged from nowhere. It was first based on LinuX Containers (LXC), which was created in 2008, and which itself was based on Process Containers created by Google engineers in 2006, which, in turn, was based on …. earlier ideas and technologies (some people will even trace them back to 1979…). However, I promised that this won’t be a history lesson, so while it is important to recognize those technologies and people who created those, let’s move further…
So, what is a Docker container? Let’s check docker.com:
A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.
Containers isolate software from its environment and ensure that it works uniformly despite differences for instance between development and staging.
So, Docker containers solve the Reproducibility and Isolation problems we have discussed above!
Note: There is another important characteristic of Docker containers — they are immutable. Opposite to traditional VM/OS, there is no ability to “login” into a Docker container and change it (e.g. by installing new software). Docker containers are “packaged” during the build process and can’t be changed after that. This allows better control over what is running in production and removes a lot of security threats.
You may want to take a pause and complete this lab: Introduction to Docker.
So, let’s get back to our problem-solution story…
While Docker containers solved the Reproducibility and Isolation problems, rapidly growing demand for that technology, driven by a new wave of giant web-based systems, like Google, Netflix etc. created more problems.
To successfully build, run, and improve giant web-based systems, developed by multiple teams, like Netflix and Google, it makes sense to press on isolations. Isolation into smaller parts allows for minimizing the impact of any change, having better traceability of any change, moving faster with new features, and even have different teams building different parts in different languages (as long as they all communicate via documented APIs). Since Docker containers allowed for a good level of isolation, it made sense to separate applications into smaller parts.
However, what if instead of 1, 2 or 5 parts, your system now has 100 or 1000? How would you manage all that fleet of containers?
And here is another issue — scalability. Everyone’s always dreamt about perfect scalability, that is to be able to quickly add more resources when required, and then, once no longer needed, to remove those. In fact, scalability is so important that it is one of the main characteristics of cloud computing. And since Docker containers are immutable (which also means — stateless), it becomes very easy to scale them: just add more containers of the same type.
However, how can you make it easy to add/remove those containers? Can the system automatically detect spikes in load and dispatch additional containers when needed (and remove them when the load goes down again).
The answer to those problems was Container Orchestration Systems:
Container orchestration is the automation of much of the operational effort required to run containerized workloads and services. This includes a wide range of things: software teams need to manage a container’s lifecycle, including provisioning, deployment, scaling (up and down), networking, load balancing and more.
Since industry felt the need for such a system pretty quickly, there were many Container Orchestration Systems created, including Docker Swarm, Mesos, and, finally, Kubernetes created by Google. When created, it was not obvious which one to use (and, for example, while Google was obviously pushing Kubernetes, Amazon had its own Elastic Containers Service, which was quite different).
This resulted in what is known as “Container Orchestration War”, which was finally won by Kubernetes on November 29, 2017 when Amazon announced the support of Kubernetes in AWS (with similar preceding announcements from Mesosphere, Pivotal, and Docker). But let’s not get too deep into history…
At present, if you want to learn about Cloud Computing, there is no way you can skip Kubernetes (K8S). In fact, Docker Containers and K8S have transformed the industry a lot and triggered creation of such things as cloud-native applications, microservice architectures etc.
Docker and K8S are not only important technologies right now, but also quite extensive. You can find complete courses on these subjects alone. Coursera even have a a specialization (4 courses included) on K8S: Architecting with Google Kubernetes Engine. Those of you who want to get an even deeper understanding of those technologies can go directly to CNCF (and even get certified).
The purpose of this course is definitely not to replace 9 CNCF courses in 1 lecture, but to give students the first introduction to the Cloud Computing and familiarity with all the services available there. That is why I will stop talking about Containers and Kubernetes, and let everyone choose their own path in learning more about those technologies.
Suggested labs:
- Build and Deploy a Docket Image to a Kubernetes Cluster
- Kubernetes in Google Cloud
- Deploy ASP.NET Core App to Kubernetes Engine
- Hello Node Kubernetes (If you prefer Node.js instead of ASP.NET)
Do you want to have another sign of how important K8S is? Do you remember the App Engine that we discussed before? Google has basically migrated it to K8S as well, calling that App Engine Flexible.
