(Hyper) Parameters in Machine Learning
Hi there! if you are a beginner or even an experienced professional in Machine Learning (ML)/Deep Learning (DL), you would have come across the terms parameters and hyperparameters. Ever wondered what those are? let's find out!
Hyperparameters
So before diving into the technical stuff, let's take an example.
When we play video games with a controller, we use the joysticks to control the motion of the characters in the video game. we make them jump, run, sprint, crouch, etc. Hyperparameters do the same thing with algorithms. To learn more about ML algorithms, check out my other articles.
Hyperparameters control the learning process
Choosing the right hyperparameter is key to optimizing the learning process of an algorithm. You can either make or break an algorithm’s efficiency using a hyperparameter. Hence it's key to choose the right one. One interesting thing about hyperparameters is that their values are set even before the model begins learning and is constant throughout the learning process. Hence we call them external to the model.
So let us look at some examples of hyperparameters
- Learning rate in the Linear Regression algorithm (alpha): This controls the pace of learning and determines the values of theta0 and theta1 (in case of univariate linear regression)
- The number of layers in a neural network: Setting this at the start of the learning process helps in yielding better results and a faster learning process.
- Determining the number of clusters (k) in the kNN algorithm: This value is external and set before the clustering begins.
Parameters
Imagine that you have bought a new phone. You might face battery retention issues at the start. But have you paid attention to how your battery backup improves over time? this is because your phone learns your usage passage over time and distributes power accordingly. This is one such case where the ML model learns the parameters by changing their values and bringing the error margin close to zero.
Parameters are values learnt by the model to reduce the error quotient of the algorithm
These parameter values are internal to the model and change throughout the learning process. In most cases, these parameter values are randomized before the learning process and optimized during the learning process by optimization algorithms like Gradient Descent. By optimization, it means that these parameters are tweaked to certain values where the error between the learned and the actual values is very minimal.
Some examples of parameters are:
- Theta values in Linear Regression
- Weights in neural networks
Conclusion
To conclude, hyperparameters determine and control the values of the parameters. These are two of the main concepts to learn and understand for the efficient execution of any ML algorithm. Comment below for any specific topic to be covered in my subsequent articles.
Happy Learning!
