BentoML Quick Start and Initial Impressions

As a Machine Learning Engineer, I’m always on the lookout for tools and processes that would improve the quality of Data Scientists.

One area that I’ve found especially lacking is that most Data Scientists are not familiar with the details surrounding deploying their models. Model development shouldn’t just be simply be handing over model.pkland calling it a job done.

Self-service model deployment has a few advantages. Data Scientists :

• Gain confidence upfront that their model is deployable in the first place and therefore can be tested. Potential issues are quickly surfaced and they can be fixed.
• Can experiment with better hardware on K8s vs on their own laptops.
• Don’t have to wait for MLEs to deploy their model and therefore are not bottle-necked.

The last point doesn’t mean that Data Scientists should perform production-level deployments — it’s still up to the MLEs to handle the final deployment. Data Scientists instead can deploy their models to non-production environments (Sandbox, Staging).

For us at DKatalis, model deployment is still very much a manual process, yet this is something that is screaming for automation. For example, most model deployments that we’ve seen have the following similarities: