Deep dive analysis of Binary Classification, Multi-class classification, and Multi-label classification

Satyam Kumar

Follow

Jan 21 · 4 min read

Classification in machine learning refers to a supervised approach of learning target class function that maps each attribute set to one of the predefined class labels. In other words, classification refers to predictive modeling where a target class is predicted given a set of input data.

There are various types of Classification problems, such as:

1. Binary Classification
2. Multi-class Classification
3. Multi-label Classification

In the further article, you can read about a deep-dive understanding of the above-mentioned classification types along with their evaluation metrics and examples.

1. Binary Classification:

Binary Classification is a type of supervised classification problem where the target class label has two classes and the task is to predict one of the classes. Typically, the task involves one class in a normal state and another class in an abnormal state.

Example Problems:

• Spam Detection: The task of the spam detection problem is to detect whether the input mail/message belongs to spam or not. In this problem, the ‘not spam’ is a normal state and ‘spam’ is an abnormal state.
• Cancer Detection: The task of the cancer detection problem is to detect whether the prospect is a cancer patient or not. In this problem, the ‘cancer’ is a normal state and ‘no cancer’ is an abnormal state.

ML algorithms:

Some of the popular machine learning algorithms that can serve as a classification algorithm are:

• Logistic Regression
• k-Nearest Neighbour
• Naive Bayes
• Decision Tree

and many more.

Metrics:

Evaluation metrics depend on the problem statement. The metrics are also dependent on the distribution of the target class label. Different metrics are involved in a balanced and imbalanced binary classification problem. Some of the popular binary classification metrics are:

• Accuracy
• Log-loss
• F1-score
• AUC-ROC score

and many more.

2. Multi-class Classification:

Multi-class classification also referred to as Multinomial Classification refers to the classification task that has more than two class labels. Unlike binary classification, this does not have any concept of normal and abnormal states. The classifier predicts as belonging to only one among a range of known classes.

There are two proposed approaches for a multi-class classification problem:

• One-vs-Rest: N classifier models are fitted for N number for classes. The class with the highest prediction probability will be predicted as the final output.
• One-vs-One: N*(N-1) classifier models are fitted for every pair of classes.

Example Problems:

• Face Recognition
• Animals Classification
• Optical Character Recognition

and many more. The number of classes in the target column varies for each problem. For MNIST digit recognition, the number of target classes is restricted to 10, whereas for Face Recognition the number of target class labels can be in hundreds or thousands.

ML algorithms:

Many algorithms used for binary classification can also be used for multi-class problems. Some of the popular machine learning algorithms used for multi-class classification problems are:

• k-Nearest Neighbour
• Decision Tree
• Naive Bayes
• Random Forest

and many more.

Metrics:

• Micro averaged F1, Precision, Recall score
• Macro averaged F1, Precision, Recall score
• Multi-class log loss

3. Multi-label Classification:

Multi-label Classification refers to a classification task where the number of target class labels are more than two, and more than one class can be predicted as output. Unlikely binary classification or multi-class classification problems where only one class is predicted, for multi-class classification multiple classes can be predicted.

Example Problems:

• Stackoverflow Tags Prediction
• Movie Genre Prediction

ML Algorithms:

The classification algorithms used for binary and multi-label classification problems cannot be directly employed with multi-label classification problems. Multi-labeled versions for classification algorithms can be used for it, some of these algorithms are:

• Multi-labeled Decision Tree
• Multi-labeled Random Forest
• Multi-labeled Gradient Boosting

Another approach is to use a one-vs-rest method, that employs n-classification models (n is the number of target class labels), each of the n-models predicting labels of each of the classes.

Metrics:

• Micro averaged F1, Precision, Recall score
• Macro averaged F1, Precision, Recall score
• Hamming Loss
• Log-loss

Conclusion:

In this article, we have covered different types of classification predictive modeling in machine learning along with some famous examples and most commonly used metrics.

Read this article to know more about the types of Regression techniques in machine learning.

Thank You for Reading