Polyaxon: A Deep Learning stack on Kubernetes

This post tries to answer a couple of practical questions about the project and the intent behind it’s creation.

Mission

Polyaxon’s mission is to make deep learning simple and accessible to enterprises.

Polyaxon is an open source platform for building, training, and monitoring large scale deep learning applications.

Polyaxon is built on Kubenernetes to make machine learning, reproducible, scalable, and portable. It uses the container technology to abstract any dependencies required for the process of the experimentation. And it has a built in versioning mechanism for code, data, and experiments.

The goal is to provide a stack that is easy to use and at the same time extensible, to enable data scientists to iterate faster on their algorithms.

Architechture

Polyaxon runs both in the cloud and on premise, and provides access via:

  • Polyaxon command line interface
  • Polyaxon dashboard
  • Polyaxon sdk targeting the Polyaxon api

These interfaces hides the powerful abstraction provided by the Polyaxon architecture. When a machine learning engineer or a data scientist deploys a model, Polyaxon relies on Kubernetes for:

  • Managing the resources of your cluster (Memory, CPU, and GPU)
  • Creating an easy, repeatable, portable deployments
  • Scaling up and down as needed

Polyaxon does the heavy lifting of:

  • Scheduling the jobs
  • Versioning the code
  • Creating docker images
  • Monitoring the statuses and resources
  • Reporting back the results to the user

should I use Polyaxon

You should consider using Polyaxon in one of the following cases:

  • Enabling collaboration within your team/organization.
  • Having full control over your data and your cluster.
  • Being independent of the provider and the environment (Polyaxon can be used locally or on a cluster).
  • Simplifying resources (CPU/GPU) management.

Get involved!

Polyaxon is open source, we think it’s really effective and we think everyone could benefit from it. We’re also hoping that the community will get engaged and help us make Polyaxon an environment for building deep learning applications.