How to Prevent Machine Learning Models from Failing in Practice?

Have you seen machine learning solutions fall flat in practice?

Well, I have. Several times. I get occasional panic calls from teams about their 98% accurate models generating questionable predictions once released to actual users.

Did they build a bad model? Maybe.