Bumpy Rides in Taking ML Systems Into Production
What has been your experience in taking ML systems into production and sustaining them? This daunting task has not been adequately paid attention to. However with efforts like ML-Ops are step in the right direction.
Does the life of a Data Scientist/Engineer listed below sound familiar?
- Know your business requirements (ensure customers and management are happy)
- What the data engineers choose to do impacts our work (make sure to work with them closely)
- Do Model selection, build it right, rightly build, experiment.. experiment.
- Give go-ahead for deployment without intruding into the production environment
- Deploy @ scale, A/B testing, watch for training-serving Skews, Concept drift… and keep on improving current model in parallel
- By the way, did we deploy the right version? OMG, for certain scenarios the performance (well functionality?) seems to be pathetic!
- Let us Go… Repeat this cycle… Avoid mistakes.
Wait, we haven’t still talked about hidden technical debt in ML systems! See the picture below. It is vast and complex. How do we assure quality? How do we minimize risks? How do we run businesses built using ML technology smoothly?
I will be posting a series of articles on the topic of taking ML systems to production in the upcoming days.
What are YOUR top 3 best practices in taking “ML Solution” to production? Please add it in comments
We would love to meet up with you and chat at the “Software Testing & QA for ML Applications” Tutorial track @ CoDS-COMAD 2020 : The ACM India Joint International Conference on Data Sciences and Management of Data