Overview of the statistical models

Approaches in Machine Learning

Parametric Model

The parametric model algorithm is a machine learning algorithm that learns using a predefined mapped function. It has a set of parameters of fixed size, independent of the number of training data. These algorithms simplify the function to a known form. For example, the equation of the straight line. Assumptions are made about the nature of the data distribution. These assumptions are known as inductive bias and are embedded in the form of parametric models. During the training process, it learns the coefficients for the function, and then it is able to predict unknown data in the future.

Some of the popular parametric models are

• Logistic Regression
• Naive Bayes
• Perceptron
• Neural networks

The pros of using a parametric model are that they are easier to understand and are very fast in training from the given data due to the fact that they make strong assumptions and they do well with less data requirement.

On the other hand, there are also cons in using the parametric models. They are highly constrained to specific assumptions. They work well on simpler problems and poor fit in practice. And have limited complexity in the produced model.

Non-parametric Model

The algorithm that does not make strong assumptions on the mapping function is known as a non-parametric machine learning algorithm. The number of parameters increases with the amount of training data. These models are applicable only to large sets of data. They are free to learn any functional form of mapping function from the training data.

Some of the non-parametric machine learning algorithms are

• K-nearest neighbors
• Support vector machines
• Tree: Random-Forest and Decision Tree

Non-parametric models are flexible because it is capable of fitting a large number of functional forms. It has high performance while predicting the answers in the future.

On the other hand, they require a lot of data to estimate the mapping function and are comparably slower and overfit during the training process.