Wine Review PT9 — Evaluate models — ML

Photo by Breno Assis on Unsplash

Introduction

In the part8, we have a convenient function that can preprocess data and output a dictionary that contain information and preprocessed data.

Now we need to find which model is fitted for our problem. To do this, we can simply select different models and compare their performance and this will give us a performance for each baseline models, the one that has potential will be use in further tuning.

Here is what I am going to do.

1. Select a set of model we are going to evaluate
2. Cross-validate each model
3. Compare the results

Dataset

Kaggle

We will use winemag-data-130k-v2.csv dataset for machine learning.

Source Code

Code in google colab

Task

• Find the baseline performance for each models
• Compare models’ performance

Baseline performance

I select 3 different models LinearSVM, Logstic Regression, Perceptron with SGDClassifier as optimizer. SGD(Stochastic Gradient Descent).

A dictionary of selected models.

Then we can do Cross-validation on each model and store the final results which will be used to do comparison. The metrics for the performance here I picked accuracy but you can use different metrics. If accuracy is not metric you are looking for then you can change scoring parameter to others in cross_validate function and here is the list.

For example.

I also store the result of average and best training time.

We define a function to handle cross validation on each models.

Here we are going to do cross validation.

After a while we can see the result in DataFrame.

It looks like LinearSVM is doing a bit better than others.

Compare models visually

Accuracy

Average training time

Conclusion

Through cross validation, we can find out the performance of baseline model. As above gave us a clue that LinearSVM is the model I am going to use for tuning.

If model we selected is not working as expected or under performing in the future, we can always come back to this stage and selected different models then repeat the steps.

There are other metrics for cross validation such as f1, precision and recall, so it depend on what you are going to measure.