Polyaxon: A Deep Learning stack on Kubernetes
This post tries to answer a couple of practical questions about the project and the intent behind it’s creation.
Polyaxon’s mission is to make deep learning simple and accessible to enterprises.
Polyaxon is built on Kubenernetes to make machine learning, reproducible, scalable, and portable. It uses the container technology to abstract any dependencies required for the process of the experimentation. And it has a built in versioning mechanism for code, data, and experiments.
The goal is to provide a stack that is easy to use and at the same time extensible, to enable data scientists to iterate faster on their algorithms.
Polyaxon runs both in the cloud and on premise, and provides access via:
- Polyaxon command line interface
- Polyaxon dashboard
- Polyaxon sdk targeting the Polyaxon api
These interfaces hides the powerful abstraction provided by the Polyaxon architecture. When a machine learning engineer or a data scientist deploys a model, Polyaxon relies on Kubernetes for:
- Managing the resources of your cluster (Memory, CPU, and GPU)
- Creating an easy, repeatable, portable deployments
- Scaling up and down as needed
Polyaxon does the heavy lifting of:
- Scheduling the jobs
- Versioning the code
- Creating docker images
- Monitoring the statuses and resources
- Reporting back the results to the user
should I use Polyaxon
You should consider using Polyaxon in one of the following cases:
- Enabling collaboration within your team/organization.
- Having full control over your data and your cluster.
- Being independent of the provider and the environment (Polyaxon can be used locally or on a cluster).
- Simplifying resources (CPU/GPU) management.
Polyaxon is open source, we think it’s really effective and we think everyone could benefit from it. We’re also hoping that the community will get engaged and help us make Polyaxon an environment for building deep learning applications.