Comparison of classification performance of kNN and WKNN algorithms

Fatih Tarakci; Ilker Ali Ozkan

Comparison of classification performance of kNN and WKNN algorithms

Fatih Tarakci, Ilker Ali Ozkan

Abstract

In this study, K nearest neighbor (kNN) algorithm which is the most popular and widely used among the machine learning classification algorithms and the weighted kNN (WKNN) algorithm which takes the weight of the feature index into consideration, are used. As the weighting method, a weighting is made by taking the inverse of the distance squared (w = 1 / d²). The confusion matrix of the data sets was created by applying the algorithms to five data sets via MATLAB program and the classification success was compared by conducting performance measurements of algorithms. It was observed that in two of the five data sets used in the study kNN algorithm turned out to make a more successful classification than WKNN while in three data sets the WKNN algorithm performed a more successful classification than the kNN.

Keywords

kNN, WKNN, UCI, Machine Learning, Classification

Full Text:

PDF

References

H. Yigit, "A weighting approach for KNN classifier," in 2013 International Conference on Electronics, Computer and Computation (ICECCO), 7-9 Nov. 2013 2013, pp. 228-231, doi: https://doi.org/10.1109/ICECCO.2013.6718270.

H. Zhang, K. Hou, and Z. Zhou, "A Weighted KNN Algorithm Based on Entropy Method," in Intelligent Computing and Internet of Things, Pt Ii, vol. 924, (Communications in Computer and Information Science. Berlin: Springer-Verlag Berlin, 2018, pp. 443-451, doi: https://doi.org/10.1007/978-981-13-2384-3_41.

N. Biswas, S. Chakraborty, S. S. Mullick, and S. Das, "A parameter independent fuzzy weighted k-Nearest neighbor classifier," Pattern Recognition Letters, vol. 101, pp. 80-87, 2018/01/01 2018, doi: https://doi.org/10.1016/j.patrec.2017.11.003.

T. Huy Quang and H. Cheolkeun, "High Precision Weighted Optimum K-Nearest Neighbors Algorithm for Indoor Visible Light Positioning Applications," IEEE Access, vol. 8, pp. 114597-114607, 2020, doi: https://doi.org/10.1109/ACCESS.2020.3003977.

B. Karabulut, G. Arslan, and H. M. Ünver, "A Weighted Similarity Measure for k-Nearest Neighbors Algorithm," Celal Bayar University Journal of Science, vol. 15, pp. 393 - 400, 2019, doi: https://doi.org/10.18466/cbayarfbe.618964.

G.-F. Fan, Y.-H. Guo, J.-M. Zheng, and W.-C. Hong, "Application of the Weighted K-Nearest Neighbor Algorithm for Short-Term Load Forecasting," Energies, vol. 12, no. 5, p. 916, 2019, doi: https://doi.org/10.3390/en12050916.

L. Zhang, C. Zhang, Q. Xu, and C. Liu, "Weigted-KNN and its application on UCI," in 2015 IEEE International Conference on Information and Automation, 8-10 Aug. 2015 2015, pp. 1748-1750, doi: https://doi.org/10.1109/ICInfA.2015.7279570.

I. Cinar and M. Koklu, "Classification of Rice Varieties Using Artificial Intelligence Methods," International Journal of Intelligent Systems and Applications in Engineering, vol. 7(3), pp. 188-194, 2019, doi: https://doi.org/10.18201/ijisae.2019355381.

I. Cinar, M. Koklu, and S. Tasemir, "Classification of Raisin Grains Using Machine Vision and Artificial Intelligence Methods," Gazi Mühendislik Bilimleri Dergisi, vol. 6(3), pp. 200-209, 2020, doi: https://doi.org/10.30855/gmbd.2020.03.03.

T. Ahmad, A. Munir, S. H. Bhatti, M. Aftab, and M. A. Raza, "Survival analysis of heart failure patients: A case study," PLOS ONE, vol. 12, no. 7, p. e0181001, 2017, doi: https://doi.org/10.1371/journal.pone.0181001.

R. M. Sobar and W. Adi, "Behavior determinant based cervical cancer early detection with machine learning algorithm," Advanced Science Letters, vol. 22, pp. 3120-3123, 10/01 2016, doi: https://doi.org/10.1166/asl.2016.7980.

S. Jain, S. Shukla, and R. Wadhvani, "Dynamic selection of normalization techniques using data complexity measures," Expert Systems with Applications, vol. 106, pp. 252-262, 2018, doi: https://doi.org/10.1016/j.eswa.2018.04.008.

O. Altay, M. Ulaş, and K. E. Alyamaç, "Prediction of the Fresh Performance of Steel Fiber Reinforced Self-Compacting Concrete Using Quadratic SVM and Weighted KNN Models," IEEE Access, Article vol. 8, pp. 92647-92658, 2020, doi: https://doi.org/10.1109/access.2020.2994562.

M. Hossin and N. Sulaiman, "A Review on Evaluation Metrics for Data Classification Evaluations," International Journal of Data Mining & Knowledge Management Process, vol. 5, pp. 01-11, 03/31 2015, doi: https://doi.org/10.5121/ijdkp.2015.5201.

C. Bergmeir and J. M. Benitez, "Forecaster performance evaluation with cross-validation and variants," in 2011 11th International Conference on Intelligent Systems Design and Applications, 22-24 Nov. 2011 2011, pp. 849-854, doi: https://doi.org/10.1109/ISDA.2011.6121763.

A. Soofi and A. Awan, "Classification Techniques in Machine Learning: Applications and Issues," Journal of Basic & Applied Sciences, vol. 13, pp. 459-465, 08/29 2017, doi: https://doi.org/10.6000/1927-5129.2017.13.76.

A. E. Taşçı and A. Onan, "K En Yakın Komşu Algoritması Parametrelerinin Sınıflandırma Performansı Üzerine Etkisinin İncelenmesi," presented at the Akademik Bilişim, Aydın,Türkiye, 2016.

G.-f. Fan, Y.-H. Guo, J.-M. Zheng, and W.-C. Hong, "Application of the Weighted K-Nearest Neighbor Algorithm for Short-Term Load Forecasting," Energies, vol. 12, p. 916, 03/09 2019, doi: https://doi.org/10.3390/en12050916.

Y. Gao and F. Gao, "Edited AdaBoost by weighted kNN," Neurocomputing, vol. 73, no. 16, pp. 3079-3088, 2010/10/01/ 2010, doi: https://doi.org/10.1016/j.neucom.2010.06.024.

W. H. Jung and S. G. Lee, "An Arrhythmia Classification Method in Utilizing the Weighted KNN and the Fitness Rule," Irbm, Article vol. 38, no. 3, pp. 138-148, Jun 2017, doi: https://doi.org/10.1016/j.irbm.2017.04.002.