Parametric and Non-Parametric algorithms in ML
“Any device whose actions are influenced by past experience is a learning machine.” — Nils John Nilsson
This is exactly what Artificial Intelligence (AI) does to a device, it trains the device to mimic the cognitive functioning of the humans, in order to make it more intelligent.
Machine Learning (ML) being a branch of the same, takes care of the algorithms, the statistical parts of AI that are responsible for teaching the computer to self program and learn from examples, to use them further.
These algorithms can be distinguished in various ways and parameters needed to solve a problem, directly influence the time and output of the training process of an algorithm.
Since, one of the major roles of ML is learning a function to be able to map input variables/parameters to the output parameters, hence the algorithms in machine learning can be classified in the following two types :
PARAMETRIC ALGORITHMS
These are the ones that are able to simplify a function of an algorithm to a known form. Parametric algorithms require a fixed amount of parameters to solve any problem, irrespective of the size of the data.
A model running on parametric algorithms summarizes the data on the basis of a few fixed parameters independent of the size of training model in two steps :
- Selecting the form of function
Parametric algorithms ASSUME the functional form. The form of the function is unknown and different algorithms make different assumptions about about it. Hence, evaluation of various machine learning algorithms is required to determine the form appropriate for approximating the underlying functions.
2. Learning the coefficients of the function
After determining the form of an equation, if we are able to guess the coefficients to the equation correctly, then the job is done. We get the final predictive model. e.g.
ax+ by + c = 0
Here, we are able to determine that the form of the equation required to solve the problem is of the above type. Then, if we are also able to make correct guesses on the values of ‘a’ , ‘b’ and ‘c’ then we will successfully be able to find the answers as well.
Some major examples of parametric machine learning algorithms are:
(Models) Poisson distribution , Normal Distribution , Exponential Distribution (Algorithms) Logistic Regression, Linear Regression, Linear Discriminant Analysis (LDA), Naive Bayes , Perceptron and Simple Neural Networks.
NON-PARAMETRIC ALGORITHMS
The algorithms NOT simplifying the function of an algorithm to a known form. Non-parametric algorithms do not assume and seek to BEST FIT the training data while constructing the mapping function.
They also step in when there are a large number of parameters. As choosing the parameters gets risky, expensive and time consuming in such cases.
They are able to generalize data & learn any functional form from the training data (i.e. to be able to fit a large number of functional forms). It is possible because these algorithms do not make assumptions.
The common types of non-parametric machine learning algorithms are: Support Vector Machines (SVM), K Nearest Neighbors (KNN) , Decision Trees etc.
Machine Learning is a topic, so vast that its algorithms can never be just classified on one basis. The above basis is just one of the infinite.
