What is weighted KNN and how does it work

Weighted KNN is a modified version of the KNN algorithm. If you don't know about KNN algorithm, then first you should understand that before learning weighted KNN.

So, One of the many issues that affect the performance of the KNN algorithm is the choice of the hyperparameter k. If k is too small, the algorithm would be more sensitive to data points which are outliers. If the value of k is too large, then the neighbourhood may include too many points from other classes.

Consider the following training set

The red labels indicate the class “Red” points and the green labels indicate class “Green” points.

Now, in the below image Consider the white point as the query point( the point whose class label has to be predicted)

If we give the above dataset to a kNN based classifier, then the classifier would declare the query point to belong to the class “Red”. But in the plot, it is clear that the point is more closer to the class “Green” points compared to the class “Red” data points.

To overcome this disadvantage, weighted kNN is used.

In weighted kNN, the nearest k points are assigned a weight. The intuition behind weighted KNN is to give more weight to the points which are nearby and less weight to the points which are farther away...

The simple function which is used is the inverse distance function which implies that as the distance increases weight decreases and as the distance decreases, weight increases