How Do You Know if Your Classification Model Is Any Good?

Guidance for determining whether your model is successful in the context of your goals

If you are new to machine learning and you developed a classification model, then congrats! You might be thinking, “now what?”

That’s a great question.

With auto ML technology, model creation is more accessible than ever. The difficulty lies in determining if that model is any good. In this article, I will explore how to determine if your model is satisfactory for your business use case (spoiler: it’s not black and white).

What Kind of Classification Are We Talking About?

Before I jump into how to evaluate your classification model, I want to clarify that while the examples I give in this article are all binary classification, there are also multi-class classification problems. The difference is that in binary classification, the target variable has only two values, and in multi-class, it has more than two values.

Many of the metric calculations I talk about later in the article will change slightly for a multi-class model, so make sure to look up the correct formula if that is the kind of model you are evaluating.

Comparing Your Model to the Baseline

“Machine learning model performance is relative and ideas of what score a good model can…