So what is a Datacenter Operating System (DCOS)? As per the definition of Mesosphere, the commercial shepherd of Apache Mesos:
… a new kind of operating system that spans all of the servers in a physical or cloud-based datacenter, and runs on top of any Linux distribution.
OK, that was easy. But how does that compare to a — for the lack of terminology I just called it local—operating system such as GNU/Linux, MacOS or the Windows family?
How does a DCOS relate to a local OS?
My attempt of comparing local OS characteristics to those of a DCOS is as follows:
So for most if not all subsystems present in a local OS you’ll find corresponding subsystems in the DCOS including but not limited to: Mesos (the DCOS kernel), Marathon (your distributed init.d) and frameworks that enable domain-specific things such as Myriad for Big Data workloads based on YARN.
What about the architecture?
Above we’ve seen how the DCOS is really just a logical evolution of the concept of an OS with the scope of an entire datacenter. If you want to learn more about the underlying concepts of warehouse-scale computing I strongly suggest to read up on it from the horse’s mouth so to speak: The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines.
In below architecture diagram I’ve tried to organise my thinking of the DCOS layout.
The elegance and usefulness of the DCOS concept based on the Mesos stack is due to a number of things:
- It doesn’t matter if you operate an on-premise datacenter with bare metal, using VMs and/or Docker or are cloud-native, say in a Google Compute environment. You’ll benefit from the dynamic partitioning capabilities in either way and that means more bang for your bucks.
- Through Myriad, the Mesos framework integrating YARN-based solutions — such Kafka-on-YARN, MapReduce2, etc. — you can run both Big Data applications and your usual suspects (Nginx, Node.js, JBoss, etc.) together in one cluster, increasing utilisation while meeting SLAs.
- From a developer’s perspective: you can consider the Mesos stack as a SDK for building distributed applications and that in a fashion allowing you to focus on the business logical rather than on infrastructure code, such as networking, co-ordination of tasks, and making the app fault- tolerant.
There are several more things coming up in the DCOS space and I’m very excited about following & contributing to the efforts as well as helping our prospects and customers to benefit from the DCOS in their deployments.