K-nearest neighbours algorithm​ is one of the simplest machine learning algorithm and an

great point to start is you are new to machine learning. As not only KNN (K-nearest neighbours)
algorithm is simple to understand and implement but also its very easy to visualise how to the
classification is performed in the algorithm.

In this post you will come to know about KNN algorithm and it's implementation in JavaScript
and for those impatient readers who directly wants to view the source code here is link to source
code of ​KNN-Visualization​ , program uses P5.js library to preform graphical operation and run
the html file in local host.

Now, lets get started with KNN algorithm is instance based learning or lazy learning type
algorithm. KNN predicts the label of point by comparing distance of that point from K data points
in training set. Time complexity for this algorithm is O(n) where n is size of training set.
Distance between to points is calculated using ​euclidean distance formula​.

for 2 dimensional feature space d = sqrt ( pow(x2 - x1 , 2) + pow(y2 - y1 , 2) ) .

After calculating distance of unknown point from each point in training set K points with lowest
distance are selected and major class label is assigned to unknown point and that's it, this is
KNN algorithm.
Advantages :

● Faster to implement
 ● Can be easily implemented and visualize

Disadvantages :

● Accuracy will be easily affected if the number sample point of one class are very large in
number than other
● The choice of value of K is also very important

The problem with unbalanced number of sample point of classes can be avoided by multiplying
distance with suitable weights proportional to number of sample points and optimal value of K
can be obtained using ​hyperparameter optimization​.
KNN can perform very well for dataset of higher dimension if dimensional reduction is performed
using algorithms such as PCA component analysis.

Now, talking about program that I have implemented to classify iris flower species using KNN
is and web app which shows how the classification is performed based on k-nearest sample
points which are compared to predict its class. If you move cursor over the graph you will be
able to see the lines between new sample point and its k nearest sample points which decides
its label , you can also change the value of K in order to understand how K value affect the
prediction.
The purpose of selecting lines to help visualize the decision instead of region like as follows

is that the region don't exactly gives idea how the prediction was made. So that was all about
KNN algorithm try to implement the algorithm on your own. Happy coding.