ML-E2: Outline of 12 machine learning algos (with python code demos)

Here we go (note in many cases I give Classifier examples but corresponding Regression modules are often also available):

ML series menu: E1 E2 E3 E4 E5 E6 E7 E8 E9

1. K-Nearest Neighbors (KNN)

• Year introduced: 1970
• Python Library: Scikit-Learn (sklearn / KneighborsClassifier) )
• How it works: Classifies a data point based on how its neighbors are classified.

K-Nearest Neighbors is a type of instance-based learning, or lazy learning, where the function is only approximated locally and all computation is deferred until function evaluation.

KNN works by finding a predetermined number of training samples closest in distance to the new point, and predicting the label from these. The number of samples can be a user-defined constant (k-nearest neighbor learning), or vary based on the local density of points (radius-based neighbor learning). The distance can, in general, be any metric measure: standard Euclidean distance is the most common choice.

The advantages of KNN are that it is simple to implement, and it doesn’t make any assumptions about the data — this is useful in cases where the data doesn’t meet the assumptions of other algorithms. The main disadvantage is that it can become time-consuming when analyzing large datasets because it searches for similar instances through the entire dataset.