Why we’re writing machine learning infrastructure in Go, not Python

Production machine learning is about more than just algorithms

Source: Free Gopher Pack
Source: Cortex GitHub
  • Autoscaling, so that traffic fluctuations don’t break your API
  • API management, to handle simultaneous API deployments
  • Rolling updates, so that you can update models while still serving users

1. Concurrency is crucial for machine learning infrastructure

  • Kubernetes APIs, which Cortex calls to deploy models on the cluster.
  • Various AWS APIs—EC2 Auto Scaling, S3, CloudWatch, and others—which Cortex calls to manage deployments on AWS.

2. Building a cross-platform CLI is easier in Go

Source: Cortex GitHub

3. The Go ecosystem is great for infrastructure projects

  • kubectl: Kubernetes’ CLI
  • minikube: A tool for running Kubernetes locally
  • helm: A Kubernetes package manager
  • kops: A tool for managing production Kubernetes
  • eksctl: The official CLI for Amazon EKS

4. Go is just a pleasure to work with

Python for machine learning, Go for infrastructure



ML infrastructure (https://github.com/cortexlabs/cortex) Formerly at AngelList. Originally a Cadillac.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store