Automated Model Deployment with BentoML and Kubeflow

How to automate model deployment and even model retraining in your own setup.

One of the things that I’ve been dissatisfied with so far with our current workflow is that model deployment is not as automated as it could be. This is partly because in model deployment there are several things to consider:

1. How the model was built, which affects how the model would be served
2. How the served model would be consumed
3. How to scale workloads
4. How to monitor the service and implement logging
5. How the model can be retrained and automatically deployed if it performs better

I’ve been experimenting with BentoML and Yatai for a few weeks now, intending to come up with a proof-of-concept that would be able to address the above points. I’ve already covered my initial impressions of BentoML/Yatai in a previous post.

While the proof-of-concept that I will be presenting is by no means complete, I think it proves that BentoML is indeed a very compelling solution to address most of the pain points when it comes to model deployment.

Note that your tech stack most likely would look different, it doesn’t matter because you should…