Ensemble Learning / Voting Classifier OR How to aggregate results from different classifiers
Today I am covering a really really important and cool technique in machine learning, it involves using a group of classifiers together and then aggregating results for these classifiers into one result.
This technique of using multiple classifiers together and then aggregating the results is called Ensemble technique and we’re going to use the Moons Dataset to highlight the same.
Ensemble technique is
- A way to use different classifiers or regressors on same training dataset using different algorithms.
- Or a way to use same training algorithm on different predictors using subsets of training algorithms (Random instances of training data).
The classifier’s we’ll use are common and popular — Logistic Regression, SVM and RandomForest Classifiers.
First let’s initialise all classifiers together -
log_clf = LogisticRegression()
rnd_clf = RandomForestClassifier()
svm_clf = SVC()Once we have loaded the dataset and made training and test sets, we will initialize the ensemble algo also known as Voting Classifier.
from sklearn.ensemble import VotingClassifier
voting_clf = VotingClassifier(
estimators = [('lr',log_clf),('rf',rnd_clf),('svm',svm_clf)],
voting='hard')
voting_clf.fit(X_train,y_train)Here this is it, this is how simple initializing and fitting data is in Ensemble Learning algorithm, let’s now see how Classifiers have worked individually on the dataset and what’s there accuracy score, also let’s find the score for our voting classifier.
for clf in (log_clf, rnd_clf, svm_clf, voting_clf):
clf.fit(X_train, y_train)
y_pred = clf.predict(X_test)
print(clf.__class__.__name__, accuracy_score(y_test, y_pred))LogisticRegression 0.8895
RandomForestClassifier 0.965
SVC 0.9655
VotingClassifier 0.975
The last value/Voting Classifier aggregates the results and gives us an accuracy score which is better than all 3 classifiers.
Fin.
If you would like to try out the code for yourself, checkout the SC here >
