ML Ops: Kubernetes as a Service
Kubernetes as a cornerstone of the Data Science/ML Ops team
Kubernetes is the gold standard for managing tons of containerized applications, whether they are in the cloud or on your own hardware. Whether it is pipeline building, model building or ML application building Kubernetes enable containerization which is a safe way to build and scale any of these scenarios.
Kubernetes is a powerful open-source tool developed by Google for managing containerized applications, making configuration and automation easier. The tool aims to provide better ways to manage related components and services, distributed over several infra-structure. It’s a platform designed to completely manage the applications and services lifecycle using methods that provide predictability, scalability, and high-availability.
Operating and scaling Kubernetes and integrating it into an existing environments is challenging and requires the right tooling, automation and processes, as well as hard-to-find expertise. Instead of building Kubernetes containers where each team utilizes them in their own ways, Dev ops and ML ops teams should work together to come up with a standardized version of utilizing Kubernetes as a Service which could be leveraged across different teams.
Implementation Options
There are different implementation options to enable Kubernetes as a service.
Building In-house
To implement Kubernetes as a service, the team should know the different building blocks of the service.
It begins with identifying a Kubernetes controller. This requires developers to define a set of managed pods and set a corresponding label.
A label is a value that is attached to any Kubernetes resource. Labels can be attached to resources, like pods, when they are created, or added and modified at any time. Each resource can have multiple labels.
Once resources are labeled, developers must then manage a Kubernetes controller. This will help organize a set of pods to ensure that the desired cluster is in a specified state.
Kubernetes-as-a-service pods, need to be maintained by a replication controller. These pods are automatically replaced if they fail, get deleted, or are terminated. There are several controller types, such as replication controllers or deployment controllers.
A replication controller can be used for running number of pod copies — or replicas — across your data science or software team’s clusters. Software teams with similar micro-service requirements or different environments with the same service can utilize the replication controller. Data science teams working on the same problem can also use it for experiments.
A deployment controller defines a desired state for a group of pods. It creates new resources or replaces the existing resources when needed. Data Science teams when it comes to deploying models or coupled data science applications can use special deployment controllers.
Usually, larger engineering teams which have dedicated dev ops resources manage large networks of pods and easily label bigger clusters of pods while automating services to fit their needs. Smaller data science teams, on the other hand, can focus on just a few pods at a time and set different labels to corresponding clusters.
Choosing one of the major cloud vendors
Azure Kubernetes Service (AKS), Amazon Elastic Kubernetes Service (EKS) or Google Kubernetes Engine (GKE). If you have huge investments in any one of these providers, you could go with their service.
Go Cloud Neutral
Going neutral has its own advantages. It provides flexibility for future changes on one or more micro-services enabling them to be multicloud in the future. Platform8 and Rancher are a couple of options in that area. Additionally, data science teams could leverage Kubernetes based services for specific parts like data pipelines. Pachyderm is a service in that area.
Recommendations
Big technology companies always prefer building in house. They aim to get flexibility, save costs and get economies of scale. In addition, they can attract talent to help them build these systems. Smaller teams tend to focus to choose cloud vendors or go neutral. Smaller teams who are looking to diversify their micro-services across a multi-cloud strategy, will leverage neutral providers. Companies in between usually look at speed to market and access to talent as two of the important factors. It is advisable to start the Trojan horse approach to start building small microservices utilizing Kubernetes to start transitioning within a company. Newer tech companies are usually Kubernetes default.
Subscribe to our Acing AI newsletter, I promise not to spam and its FREE!
Thanks for reading! 😊 If you enjoyed it, test how many times can you hit 👏 in 5 seconds. It’s great cardio for your fingers AND will help other people see the story.
References:
Pachyderm: https://www.pachyderm.com/blog/kubernetes-as-a-service/
Platform8: https://platform9.com/managed-kubernetes/
Rancher: https://rancher.com/
