Joyent CoPilot: Bringing Application Awareness to Cloud Infrastructure. Part I.
Intro
In the early fall of 2016 the joined forces of Make Us Proud and YLD have begun a long-term collaboration with Joyent. The goal of the project is to design and build an experimental application management platform that would incorporate Joyent’s cloud infrastructure service Triton and application orchestration software ContainerPilot.
About Joyent
Joyent is a San Francisco-based company, well-known for technological innovations in the field of cloud computing and application virtualisation. It pioneered public cloud and hybrid computing, contributed to development of Node.js, was among the early adopters of software containerisation technology and, prior to acquisition by Samsung in 2016, was the biggest privately owned cloud provider in the world.
Already too much jargon, explain!
Before we begin delving into the project, it might be useful to cover some of the concepts and technologies that are involved in it. If you are a software engineer, you might want to skip this and read the next part, otherwise — brace yourself, we’re going for a ride. And to keep your curiosity peaked, here’s a little teaser of what we’ve been building:
The project’s goal is to create an application management dashboard: with application being any kind of software designed to perform specific tasks. For example, a blog website is an application. Your first question might sound something like “What does managing an application” mean? Hold your horses, partner. First of all, we need to look at how applications are designed.
Service-oriented vs monolithic architecture
Broadly speaking, there are two approaches to designing an application.
You can design an application that has all functionally discernible features tightly interwoven. This approach — known as monolithic architecture — might be considered undesirable. Such applications can be too big and complex to fully comprehend, its size can slow down performance, any kind of error can bring the entire application down, maintaining and changing it requires working on an entire application.
The alternative to this is service-oriented approach. Services (or microservices) are independent components that constitute a whole larger application. This approach makes it easier to understand and develop an application as well as test it. Maintenance and changes can be performed more easily, as a service can be replaced with a newer version without bringing the whole application to a stop. This approach enables continuous development and deployment of an application.
So when we talk about application management, we are talking about an application constituted of services. Going back to the previous example, a blog could be constituted of 5 separate services — WordPress, MySQL, Memcached, Nginx and NFS (at the moment, it is not important what they do or how they relate, what is important is to know that they are discrete and provide different functionality that contribute to the overall application).
Application virtualisation
Now that you know about application services, we can talk about different ways of running an application. To run an application you need — spoilers! — an application itself, an operating system (an underlying software that manages computer hardware, software resources and enables other applications to run) and a hardware unit (in other words, a computer).
If you are designing and planning to run an application, you might want to use a more powerful hardware (think pimped-out data centers with multiple hardware units). You may also want to have more configuration flexibility or the ability to increase the number of hardware units doing the work. Why? Because with the number of users who want to access your application increasing, you require more computing power to handle that traffic. In another scenario, due to security issues, you may have a need to isolate an application from other applications using the same resource and belonging to other users. All of these and many other needs can be met with virtualisation.
In a nutshell, virtualisation means reproducing actions and functions of something (in this context, software or hardware) without either of those actually existing. There are two methods of virtualisation — hardware and operating-system-level.
Hardware virtualisation can be described as “running a computer inside a computer”. It enables you to run multiple operating systems (virtual machines — VMs — with applications running in them) on a single piece of hardware.
Operating-system-level virtualisation allows you to have multiple isolated environments to run applications in, without emulating hardware and using only one operating system. Multiple applications can use one operating system without affecting each other’s performance. In theory these environments (usually referred to as containers) can encapsulate an entire application (but it defeats the purpose of having discrete components — remember the chart of a blog website from before?) or individual services. Utilising only one operating system (or more specifically, its central component — the kernel), shared by all the containers on top of it, allows containers to be super light-weight and get started faster than virtual machines.
The exact model of the OS-level virtualisation may differ from case to case, however the basic principle remains the same. In case of Joyent, company has developed its own operating system — SmartOS — optimised for container delivery. Without going into too much details, Joyent has reduced the number of components that constitute their virtualisation model, thus enabling applications to achieve highest possible performance, without wasting resources on hardware virtualisation.
Lets return to the blog website example. The blog is constituted of 6 services — Nginx, WordPress, Memcached, NFS, MySQL and Consul. Let’s say that this application was designed and your laptop can act as the hardware (physical server). In theory you can make it available to other users just as it is — but it sounds a bit old-fashioned to keep your laptop forever switched on and online. You want to move your application to a computer in some far away data center. First thing that you need to do, is to make sure your app can run on that computer and be isolated from any other stuff on it. That can be achieved by containerising its services.
Containers, images and Docker
Now that you aware of containers as a method of virtualisation, let’s talk about how a container can be created. Containerisation technology isn’t something completely new — there are multiple open-source technologies that are conceptually very similar — but the most likely name to run into in this field is Docker.
Docker is a software container platform used to build, ship and run applications. It enables applications to be quickly assembled from components.
A container is created based on an image — a self-contained, unchangeable package of a piece of software. In other words, it’s a system of files and folders. An image cannot be run — it’s a read-only template with some instructions for creating a runnable container. When building an application, you can use images created by other people (for example Nginx), create your own custom image from scratch or take an existing image and customise it (say you have an image of a WordPress service and you want to install some custom software components into it). What tools does Docker provide to do that?
The brain behind the magic is Docker Daemon — software responsible for building, running and distributing Docker containers. You can’t directly interact with Daemon — for that you need a command line tool called Docker Client. With the help of Docker Client, Daemon receives a Dockerfile — a simple text file written by a user and containing instructions how an image needs to be created — what files need to be copied, what additional software components need to be added into an image to make its execution successful. And finally Docker Hub — a registry of Docker images, responsible for hosting and distributing Docker images. It’s not necessary to use Docker Hub — there are other registries and you can also create your own one.
Let’s say you are creating a custom WordPress image for a blog website. In your Dockerfile you provide several instructions to Docker Daemon. The first instruction starts out by copying the public WordPress image. Second instruction installs additional software components — for example this could be a custom graphical user interface. Third instruction converts the source code into an executable software and fourth specifies what commands to run within a container. Boom — your image is ready.
What’s next? Next you need to upload your custom image to a registry (or in other words, a directory where your service image would be available when running an application). This can be done by providing Docker Client with a relevant command.
“Ok, ok, I get it — we have created a service image, pushed it to a Docker image registry — but how the hell are we going to run a service container in the cloud?” My friend, this question is so much larger — how do you define a multi-container (or multi-service) application? At first, lets see how you can do that with Docker and then we can look at Joyent’s variation of this method.
Creating and running multi-container applications
Our blog website is a multi-container application — it has multiple services and they all need to run together. You want services to behave as a single holistic entity.
To do so, Docker provides a tool called Compose. Compose uses a file called docker-compose.yml, describing and configuring application’s services. A description of a service in docker-compose.yml file has to have at least a name of a service and a path to it’s image, other configuration parameters are optional. You can then run an entire application with just a single command that starts all of the application’s services.
Joyent’s Triton
Ok, so let’s recap quickly: services are discrete parts of an application and containers encapsulate services to run on top of a single operating system, without the need to waste resources on hardware emulation. Containers are created from images (image = blueprint, container = living breathing creature, ‘aight?). Containerised services that constitute an application need to run as a cohesive unit. That can be achieved by creating a docker-compose.yml file, that defines application’s services, their configuration parameters and enables you to run an application with a single command. Now let’s see where Joyent comes in.
As it was mentioned before, Joyent specialises in cloud computing (what a twist, huh?). Joyent provides clients with the ability to run virtual machines and containers on top of physical hardware in data centres around the world. Along with those VMs and containers is an ability to customise networking, security, analytics, backup, etc The sum of this is called Triton. Triton supports both VMs and Docker-compatible containers, but containers are their primary focus.
ContainerPilot and application orchestration
To run properly, a container needs to know when and in what sequence certain tasks (such as provisioning, starting, stopping and deleting) need to be performed. Automation of these operations — known as scheduling or container orchestration — is managed by a piece of software called scheduler.
On top of that, multi-service applications also require operational tasks, such as configuring containers as they start, managing dependencies, re-configuring, performing health-checks, handling errors and more. Automation of these tasks is called ‘application orchestration’.
For a second, let’s go back to the blog website example to understand how scheduling and application orchestration share duties. One of the app’s services — WordPress — needs to start up. This task is managed by the scheduler. Following that, the container needs to be configured, learn where the MySQL database is and connect with it — all of which represents application orchestration.
Traditionally, both scheduling (container orchestration) and application orchestration are closely interconnected and are tied to the cloud provider. This means that application orchestration software is dependant on the scheduler and vice versa — which is not ideal, as you want to ensure scheduler portability, i.e. have the freedom to use any kind of scheduling software or even move to a different cloud provider. Some schedulers attempt to manage even more aspects of an application such as service discovery, configuration and more, making them highly integrated with the application. Either way, you end up having a lot of interdependent pieces — which is never a good thing. And this is where Joyent’s ContainerPilot comes in.
ContainerPilot is application orchestration software that lives inside a service container and supervises processes within a container, runs health checks, registers the service with the service registry (about this — in a moment), observes service registry for changes and executes other user-defined tasks that ensure that the container runs correctly. The power of ContainerPilot lies with its independence from the scheduler. You can use it with any scheduler or cloud provider.
The approach to moving application orchestration into the container itself is called Autopilot Pattern. With Autopilot Pattern in place, containers start up (thanks to the scheduler) and then automatically configure (thanks, ContainerPilot!). ContainerPilot uses a simple configuration file, that (among other things) facilitates service discovery — a process of services finding and connecting with each other. Let’s look at our blog website example. Nginx service starts up and (as part of its configuration) is instructed to reach out to and register with Consul (a service registry), learn from it where Nginx is and connect with it. Pretty straightforward, isn’t it?
Managing an application
You’re not going to believe it, but we have finally arrived to the matter of application management. Let’s imagine that an application utilising Autopilot Pattern is now running on Joyent Cloud. One of the most important things is to keep its performance optimal. In the context of this project we will be talking about the performance based on utilisation of resource allocated to an application. Without going into too much technical details, every instance of a service has a certain amount of RAM, CPU and disk memory allocated to them. It’s important to keep the utilisation of these resource optimal. On one hand, If the resources’ limits are exceeded, the application might become slow, unresponsive or crash. On the other hand, if the resources are underused you will be spending money for nothing.
Let us return to our blog example. When you deploy your application for the first time to the cloud, you might have 1 container of each service (or in other terms, one instance of each service). As the traffic to the blog grows and more users try to access its content, the usage of resources grows too. To keep the performance of the application optimal, number of instances of each service can be changed — in this case, increased. Increasing or decreasing number of instances/containers is called scaling. After a scaling action is performed, ContainerPilot makes sure that instances of a service perform equally, by distributing the load between them.
Effective application management involves many other operations such as monitoring and alerting, application/service versioning, networking and security and many more, however, for now let’s stick with scaling as one of the most crucial operations that can be performed to keep an app healthy.
How are you feeling?I will tell you how you are feeling — you are awesome, because you’ve just learned all the fundamentals that took the design team approximately 5 months to properly comprehend. To quote the legend:
- “I know kung fu”. — Neo, The Matrix.
On this high poetic note, let’s begin the actual story of us trying to figure out the potential future of application management on Joyent in Part II.
