Propose a Proper Algorithm for Incremental Learning Based on Fuzzy Least Square Twin Support Vector Machines
Subject Areas : electrical and computer engineeringJavad Salimi Sartakhti 1 * , Salman Goli 2
1 -
2 -
Keywords: Incremental learning, SVM, fuzzy classification, FLSTSVM,
Abstract :
Support Vector machine is one of the most popular and efficient algorithms in machine learning. There are several versions of this algorithm, the latest of which is the fuzzy least squares twin support vector machines. On the other hand, in many machine learning applications input data is continuously generated, which has made many traditional algorithms inefficient to deal with them. In this paper, for the first time, an incremental version of the fuzzy least squares twin support vector algorithm is presented. The proposed algorithmis represented in both online and quasi-online modes. To evaluate the accuracy and precision of the proposed algorithmfirst we run our algorithm on 6 datasets of the UCI repository. Results showthe proposed algorithm is more efficient than other algorithms (even non-incremental versions). In the second phase in the experiments, we consider an application of Internet of Things, and in particular in data related to daily activities which inherently are incremental. According to experimental results, the proposed algorithm has the best performance compared to other incremental algorithms.
[1] D. Bhattacharya and M. Mitra, Analytics on Big Fast Data Using Real Time Stream Data Processing Prchitecture, EMC Corporation, 2013.
[2] B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. of the 5th Annual Workshop on Computational Learning Theory, pp. 144-152, Pittsburgh, PA, USA, 27-29 Jul. 1992.
[3] R. T. Rockafellar, "Lagrange multipliers and optimality," SIAM Review, vol. 35, no. 2, pp. 183-238, 1993.
[4] J. A. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Processing Letters, vol. 9, no. 3, pp. 293-300, Jun. 1999.
[5] J. A. K. Suykens, T. Van. Gestel, J. De Brabanter, B. De Moor, and J, Vandewalle, Least Squares Support Vector Machines, World Scientific, 2002.
[6] R. Khemchandani and S. Chandra, "Twin support vector machines for pattern classification," IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 905-910, May 2007.
[7] M. A. Kumar and M. Gopal, "Least squares twin support vector machines for pattern classification," Expert Systems with Applications, vol. 36, no. 4, pp. 7535-7543, May 2009.
[8] J. S. Sartakhti, H. Afrabandpey, and N. Ghadiri, "Fuzzy least squares twin support vector machines," Engineering Applications of Artificial Intelligence, vol. 85, pp. 402-409, Oct. 2019.
[9] V. Losing, B. Hammer, and H. Wersing, "Incremental on-line learning: a review and comparison of state of the art algorithms," Neurocomputing, vol. 275, pp. 1261-1274, Jan. 2018.
[10] R. KhemchandaniJayadeva, and S. Chandra, "Incremental twin support vector machines," in Modeling, Computation and Optimization, in S. K. Neogy, A. K. Das, and R. B. Bapat (eds.), pp. 263-272, World Scientific, 2009.
[11] P. Mitra, C. A. Murthy and S. K. Pal, "Data condensation in large databases by incremental learning with support vector machines," in Proc.of 15th Int. Conf. on Pattern Recognition. ICPR'00, vol.2, pp. 708-711, , Barcelona, Spain, 3-7 Sept. 2000,
[12] Y. Hao and H. Zhang, "A fast incremental learning algorithm based on twin support vector machine," in Proc. IEEE 7th Int. Symp. on Computational Intelligence and Design, , vol. 2, pp. 92-95, Hangzhou, China, 13-14 Dec. 2014.
[13] F. Alamdar, S. Ghane, and A. Amiri, "On-line twin independent support vector machines," Neurocomputing, vol. 186, no. C, pp. 8-21, Apr. 2016.
[14] G. Fung and O. L. Mangasarian, "Incremental support vector machine classification," in Proc. of the SIAM Int. Conf. on Data Mining, pp. 247-260, Arlimgton, VA, USA, 11-13 Apr. 2002.
[15] A. Tveit, M. L. Hetland, and H. Engum, "Incremental and decremental proximal support vector classification using decay coefficients," in Proc. of the Int. Conf. on Data Warehousing and Knowledge Discovery, pp. 422-429, Prague, Czech Republic, 3-5 Sept. 2003.
[16] A. R. Mello, M. R. Stemmer, and A. L. Koerich, "Incremental and decremental fuzzy bounded twin support vector machine," Information Sciences, vol. 526, pp. 20-38, Jul. 2020.
[17] J. Xu, C. Xu, B. Zou, Y. Y. Tang, J. Peng, and X. You, "New incremental learning algorithm with support vector machines," IEEE Trans. on Systems, Man, and Cybernetics, Systems, vol. 49, no. 11, pp. 2230-2241, Nov. 2018.
[18] I. A. Lawal, "Incremental SVM learning," Studies in Big Data Learning from Data Streams in Evolving Environments, pp. 279-296, 2019.
[19] W. Xie, S. Uhlmann, S. Kiranyaz, and M. Gabbouj, "Incremental learning with support vector data description," in Proc. IEEE 22nd Int. Conf. on Pattern Recognition, pp. 3904-3909, Stockholm, Sweden, 24-28 Aug. 2014.
[20] Z. Zhu, X. Zhu, Y. F. Guo, and X. Xue, "Transfer incremental learning for pattern classification," in Proc. of the 19th ACM International Conf. on Information and Knowledge Management, pp. 1709-1712, Toronto, Canada, 26-30 Oct. 2010.
[21] G. Cauwenberghs and T. Poggio, "Incremental and decremental support vector machine learning," in Proc. of the 13th Int Conf. on Neural Information Processing Systems, pp. 388-394, Denver, CO, USA, 1-1 Jan. 2000.
[22] H. Duan, X. Shao, W. Hou, G. He, and Q. Zeng, "An incremental learning algorithm for Lagrangian support vector machines," Pattern Recognition Letters, vol. 30, no. 15, pp. 1384-1391, 1 Nov. 2009.
[23] H. Galmeanu, L. M. Sasu, and R. Andonie, "Incremental and decremental SVM for regression," International J. of Computers Communications & Control, vol. 11, no. 6, pp. 755-775, Dec. 2016.
[24] H. Galmeanu and R. Andonie, "A multi-class incremental and decremental SVM approach using adaptive directed acyclic graphs," in Proc. IEEE Int Conf. on Adaptive and Intelligent Systems, pp. 114-119, Klagenfurt, Austria, 24-26 Sept. 2009.
[25] J. Wang, D. Yang, W. Jiang, and J. Zhou, "Semisupervised incremental support vector machine learning based on neighborhood kernel estimation," IEEE Trans. on Systems, Man, and Cybernetics: Systems, vol. 47, no. 10, pp. 2677-2687, Oct. 2017.
[26] M. S. Chen, T. Y. Ho, and D. Y. Huang, "Online transductive support vector machines for classification," in Proc. IEEE Int. Conf. on Information Security and Intelligent Control, pp. 258-261, Yunlin, Taiwan, 14-16 Aug. 2012.
[27] P. Rai, H. Daume, and S. Venkatasubramanian, "Streamed learning: one-pass SVMs," in Proc. 21st Int. Joint Conf. on Artificial Intelligence, pp. 1211-1216, Pasadena, CA, USA, 11-17 Jul. 2009.
[28] N. A. Syed, S. Huan, L. Kah, and K. Sung, "Incremental learning with support vector machines," in ¬ Proc. IEEE Int. Conf. on Data Mining, San Jose, CA, USA, 29 Nov.-2 Dec. 2001.
[29] F. Orabona, C. Castellini, B. Caputo, L. Jie, and G. Sandini, "On-line independent support vector machines," Pattern Recognition, vol. 43, no. 4, pp. 1402-1412, Apr. 2010.
[30] L. Ralaivola and F. d'Alche-Buc, "Incremental support vector machine learning: a local approach," in Proc. Int. Conf. on Artificial Neural Networks, pp. 322-330, Vienna, Austria, 21-25 Aug. 2001.
[31] M. N. Kapp, R. Sabourin, and P. Maupin, "Adaptive incremental learning with an ensemble of support vector machines," in Proc. IEEE 20th Int. Conf. on Pattern Recognition, pp. 4048-4051, Istanbul, Turkey, 23-26 Aug. 2010.
[32] C. Domeniconi and D. Gunopulos, "Incremental support vector machine construction," in Proc. IEEE Int. Conf. on Data Mining, pp. 589-592, San Jose, CA, USA, 29 Nov.-2 Dec. 2001.
[33] A. R. de Mello, M. R. Stemmer, and A. L. Koerich, Incremental and Decremental Fuzzy Bounded Twin Support Vector Machine, arXiv preprint arXiv:1907.09613, 2019.
[34] "Activities of Daily Living (ADLs) Recognition Using Binary Sensors Data Set," ed, 2013.
[35] F. Ordonez, P. De Toledo, and A. Sanchis, "Activity recognition using hybrid generative/discriminative models on home environments using binary sensors," Sensors, vol. 13, no. 5, pp. 5460-5477, 2013.