ML algorithm K-Nearest Neighbors, simple explanation
In a previous publication — Classify with Machine Learning — we discovered that one of the most common algorithms to tackle classification problems with Machine Learning is K-Nearest Neighbor, in this article, we will know a little more about this concept and how it works.
K-Nearest Neighbors (KNN) is a supervised machine learning algorithm, which can be used to predict or identify the category of new value in the data. Essentially, KNN allows us to classify the values by searching for data points similar to the new entry, and by comparing the sample with the data points of the “nearest neighbors”.
This is an algorithm that is characterized by being easy to implement and is instance-based, which means that it does not require explicit learning of a model. These types of algorithms are called lazy learning methods, in which the generalization of the training data is delayed until a query is made to the system.
How does it work?
In the K-Nearest Neighbors example graph, the red dot represents the new entry that we want to know if it belongs to Class A (yellow) or Class B (purple). The k represents the number of neighbors that will be taken into account to assign the category of the new value, this decision is made by a majority vote.
Thus, our example with k=3 would have a result of 1 instance class A and 2 instances class B. Then, by majority, the result of the prediction will be that the red dot, for k=3, belongs to class B.
But for k=6 the situation is reversed, within the 6 nearest neighbors, 4 are yellow and only 2 are purple, so the algorithm will decide that the red dot belongs to class A.
In other words, the choice of the value of k is a sensitive process that must be the product of a calculation and not of intuition. For this, there is a technique called model performance measure, which allows knowing the accuracy of an algorithm based on its results, I will talk more in-depth about this technique in the next article.
