Hyperparameter Tunning

about Hyperparameter Tunning

In machine learning, we have two types of parameters: model parameters and hyperparameters.

Model parameter:-

model that is determined by training with training data. it's considered an internal parameter. Weight and bias are examples of Model parameters.

y = wx+b (w and b are model parameters here)

Hyperparameter:-

Parameter values control the learning process. These are adjustable parameters for obtaining an optimal model. They are considered external parameters because we can change the value of the Hyperparameter.

like learning rate and number of iterations.

Hyperparameter Tuning:-

Referring to the process of selecting the optimal set of hyperparameters for a machine learning model, this process is also known as hyperparameter optimization.

we have two types of Hyperparameter tuning:-

Grid Search CV & Random Search CV

Grid Search CV:

In this process, we have different types of values. We will check each value and determine the accuracy of each one.

Random Search CV:

Instead of calculating the performance of the model for different combinations of hyperparameters in the randomized search CV, we will randomly select only a few parameters and then try to find the best value.

implementation of Grid Search CV & Random Search CV:

#import library's
import pandas as pd
import numpy as np
import sklearn.datasets
from sklearn.svm import SVC
from sklearn.model_selection import GridSearchCV
from sklearn.model_selection import RandomizedSearchCV

#loading data set
breast_cancer = sklearn.datasets.load_breast_cancer()
print(breast_cancer)

output

df = pd.DataFrame(breast_cancer.data,columns=breast_cancer.feature_names)
df.head()

#adding the target columns to the data frame
df['lable']= breast_cancer.target
df.head()

#find missing value
df.isnull().sum()

#checking the distribution of target variables
df['lable'].value_counts()

#separating the features and target
x = df.drop(columns='lable',axis=1)
y= df['lable']
print(x.head())

#converting in numpy array
x = np.asanyarray(x)
y = np.asanyarray(y)

#loading the SVC model
model = SVC()

#hyperparameters

parameters = {
    'kernel': ['linear', 'poly', 'rbf', 'sigmoid'],
    'C': [1, 5, 10, 20]
}

#grid search**
classifier = GridSearchCV(model,parameters,cv=5)
#fitting the data to our model
classifier.fit(x,y)

classifier.cv_results_

#best parameter
best_para = classifier.best_params_
print(best_para)

best parameter or best values for our model

#highest accuracuy
high_acc= classifier.best_score_
print(high_acc)

this is our highest accuracy

Highest accuracy = 95.2%
best parametres= {‘c’:10, ‘kernel’:’linear’}

This is the highest accuracy and best parameter of our model through Grid Search CV. It checks every value in the database so many times, creating problems for our computing power and consuming a lot of time.

now we will perform Random Search CV

All codes are the same as grid search CV, we just need to import randomized search CV instead of grid search CV.

#random  search**
classifier = RandomizedSearchCV(model,parameters,cv=5)

also we have same output

Highest accuracy = 95.2%
                 best parametres= {'c':10, 'kernel':'linear'}

but its will not happened every time.

Due to its random nature, it may not always find the best hyperparameters, but it often quickly finds good ones. It doesn’t guarantee that all possible combinations will be explored, unlike grid search.

Python’s Gurus🚀

Thank you for being a part of the Python’s Gurus community!

Before you go:

• Be sure to clap x50 time and follow the writer ️👏️️
• Follow us: Newsletter
• Do you aspire to become a Guru too? Submit your best article or draft to reach our audience.