Okteto as a Sagemaker alternative
Our data science team has been working on a new feature, one of the team members led this effort. His workflow on a small dataset was great and the results were promising, but when he wanted to check if he can achieve the same results with production size datasets — he had to make a new decision: work locally with his favorite IDE and at some point reach the local capacity or turn To a cloud based solution and abandon his IDE.
Where we started:
Sagemaker is the de facto framework to enable data scientists access to data and heavy computation when working within the AWS ecosystem — and it served us well for a long time. But although it is easy to use, it also has some drawbacks that take their toll when using the framework for custom production-oriented code.
- It has limited IDE productivity capabilities
- It is “notebook based” and not code-oriented — and so makes it harder to manage code using git
- It is more expensive than a simple EC2 instance
Finding the middle ground
We searched for a solution that will keep most of the capabilities Sagemaker granted and enhance them:
- Shared code between data scientists
- Run large-scale processes and computations
- Quick and Easy setup
- Strong IDE capabilities
And also, we searched for a solution that would play to our company strengths: we manage a k8s cluster, we have a strong ops team to support the effort, and our DS team is tech-savvy.
The search:
Our first solution was obvious — a simple server containing the code. In this scenario, using git and SSH, the DS connects the IDE to the server and works directly on it, using a remote interpreter.
The problem:
- instead of leveraging the power of our system we just create a server that needs maintenance — a liability
The second solution led us to contemplate a custom k8s resource consisting of a project image and resource.
we were aligned with our criteria but:
- The development cost was very high.
- The DS team wanted a simpler solution.
In the third iteration, we found Okteto — our choice
What is Okteto:
When defining Okteto, the authors described it as
“Okteto provides a local development experience for Kubernetes applications. You code locally in your favorite IDE and Okteto synchronizes it automatically to your cluster”
Okteto’s Features:
- Work locally and run on a remote configurable machine
- works directly with k8s.
- Quick and easy setup using a single yaml file
Okteto checked all our boxes. it kept all of the features we had with Sagemaker while giving us the enhanced capabilities we wanted.
Working with Okteto:
Since implementing Okteto, our team members are happy with the solution, but we still wanted to mention:
- There is a learning curve — Okteto is still a bit more technical to work with than a simple notebook. It Requires a DevOps team or a ML software engineer in order to help with setting it up and fixing casual errors
- Stability — the Resources provided by Okteto are susceptible to all k8s ailments, such as removed or restarted pods depending on the cluster setup
Summary:
Leaving Sagemaker was not easy and we are still working on performance and ease of use on the new solution, but we managed to achieve our goals and enhance the daily routine of the DS team when writing code — big win!
link to our setup manual
