A Transfer Learning Algorithm to Improve the Convergence Rate and Accuracy in Cellular Learning Automata
Subject Areas : electrical and computer engineeringSeyyed Amir Hadi Minoofam 1 , Azam Bastanfard 2 * , M. R. Keyvanpour 3
1 - Karaj Islamic Azad University
2 -
3 -
Keywords: Cellular learning automata, convergence rate, transfer learning, knowledge transfer,
Abstract :
Cellular learning automaton is an intelligent model as a composition of cellular automaton and learning automaton. In this study, an extended algorithm of cellular learning automata is proposed based on transfer learning as the TL-CLA algorithm. In this algorithm, transfer learning is used as an approach for computation deduction and minimizing the learning cycle. The proposed algorithm is an extended model based on merit function and attitude vector for transfer learning. In the TL-CLA algorithm, the value of the merit function is computed based on the local environment, and the value of the attitude vector is calculated based on the global environment. When these two measures get the threshold values, the transfer of action probabilities causes the transfer learning from the source CLA to the destination CLA. The experimental results show that the proposed TL-CLA model leads to increment the convergence accuracy as 2.7% and 2.2% in two actions and multi-action standard environments, respectively. The improvements in convergence rate are also 8% and 2% in these two environments. The TL-CLA could be applied in knowledge transfer from learning one task to learning another similar task
[1] S. A. H. Minoofam and A. Bastanfard, "Square kufic pattern formation by asynchronous cellular automata," in Proc. Int Conf. on Cellular Automata, pp. 79-82, Ascoli Piceno, Italy, 21-24 Sept. 2010.
[2] S. A. H. Minoofam and A. Bastanfard, "A novel algorithm for generating Muhammad pattern based on cellular automata," in Proc. WSEAS 13th Int. Conf. on Applied, pp. 339-344, Puerto De La Cruz, Spain, 15-17 Dec. 2008.
[3] E. Fredkin, "An informational process based on reversible universal cellular automata," Phys. D Nonlinear Phenom., vol. 45, no. 1-3, pp. 254-270, Sept. 1990.
[4] S. A. H. Minoofam, M. M. Dehshibi, A. Bastanfard, and P. Eftekhari, "Ad-hoc ma'qeli script generation using block cellular automata," J. Cell. Autom., vol. 7, pp. 321-334, 2012.
[5] S. Wolfram, Cellular Automata and Complexity: Collected Papers, CRC Press, 2018.
[6] J. L. Schiff, Cellular Automata: A Discrete View of the World, vol. 45, John Wiley & Sons, 2011.
[7] S. A. H. Minoofam, M. M. Dehshibi, A. Bastanfard, and J. Shanbehzadeh, "Pattern formation using cellular automata and L-systems: a case study in producing Islamic patterns," in Cellular Automata in Image Processing and Geometry, Ch. 12, pp. 233-252, Switzerland: Springer, 2014.
[8] K. S. Narendra and M. A. L. Thathachar, "Learning automata-a survey," IEEE Trans. Syst. Man. Cybern., vol. 4, no. 4, pp. 323-334, Jul. 1974.
[9] F. Esmaeily and M. R. Keyvanpour, "WMat algorithm based on Q-Learning algorithm in taxi-v2 game," in Proc. 4th Int. Conf. on Smart City, Internet of Things and Applications, SCIOT’20, pp. 112-118, Mashhad, Iran, 16-17 Sept. 2020.
[10] A. Rezvanian, A. M. Saghiri, S. M. Vahidipour, M. Esnaashari, and M. R. Meybodi, Recent Advances in Learning Automata. Springer, 2018.
[11] Z. Movahedi and A. Bastanfard, "Toward competitive multi-agents in polo game based on reinforcement learning," Multimed. Tools Appl., vol. 80, no. 17, pp. 26773–26793, May 2021.
[12] M. Savargiv, B. Masoumi, and M. R. Keyvanpour, "A new random forest algorithm based on learning automata," Comput. Intell. Neurosci., vol. 2021, Article No.: 5572781, 19 pp., 2021.
[13] S. A. H. Minoofam and A. Bastanfard, "Learning automata: a comparative analysis of estimator algorithms," in Proc. 25th Iran. Conf. Electr. Eng., pp. 1772-1778, Tehran, Iran, 2-4 May 2017.
[14] M. A. L. Thathachar and P. S. Sastry, Networks of Learning Automata: Techniques for Online Stochastic Optimization, Springer Science & Business Media, 2011.
[15] M. Savargiv, B. Masoumi, and M. R. Keyvanpour, "A new ensemble learning method based on learning automata," J. Ambient Intell. Humaniz. Comput., vol. 11, no. 4, pp. 1-16, Apr. 2020.
[16] M. R. Meybodi, H. Beigy, and M. Taherkhani, "Cellular learning automata and its applications," Sharif J. Sci. Technol., vol. 19, no. 25, pp. 54-77, Autumn 2003.
[17] H. Beigy and M. R. Meybodi, "A mathematical framework for cellular learning automata," Adv. Complex Syst., vol. 7, no. 03n04, pp. 295-319, 2004.
[18] M. N. Qureshi, M. I. Tiwana, and M. Haddad, "Distributed self optimization techniques for heterogeneous network environments using active antenna tilt systems," Telecommun. Syst., vol. 70, no. 3, pp. 379-389, Jul. 2019.
[19] M. Torshizi and M. J. Sheikhzadeh, "Optimum k-coverage in wireless sensor network with no redundant node by cellular learning automata," Wirel. Pers. Commun., vol. 110, no. 2, pp. 1-18, Oct. 2019.
[20] M. J. Moghaddam, A. Esmaeilzadeh, M. Ghavipour, and A. K. Zadeh, "Minimizing virtual machine migration probability in cloud computing environments," Cluster Comput., vol. 23, no. 4, pp. 3029-3038, Feb. 2020.
[21] M. K. Sohrabi and R. Roshani, "Frequent itemset mining using cellular learning automata," Comput. Human Behav., vol. 68, no. C, pp. 244-253, Mar. 2017.
[22] ا. حضرتی بی‌شک، ک. فائز، ح. برقی جند و س. قطعی "انتخاب زیرمجموعه بهینه از ویژگی‌های استخراج‌شده توسط عملگر بهینه‌شده LBP بر مبنای CLA - EC در سیستم بازشناسی چهره،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، ب- مهندسی کامپیوتر، سال 12، شماره 2، صص. 74-67، زمستان 1393.
[23] C. Di, B. Zhang, Q. Liang, S. Li, and Y. Guo, "Learning automata-based access class barring scheme for massive random access in machine-to-machine communications," IEEE Internet Things J., vol. 6, no. 4, pp. 6007-6017, Aug. 2018.
[24] م. رضاپور میرصالح و م. ر. میبدی "ارائه يک مدل جديد ممتيکي مبتني بر اتوماتاي يادگير ساختار ثابت،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، ب- مهندسی کامپیوتر، سال 16، شماره 3، صص. 195-183، پاییز 1397.
[25] م. ر. ملاخلیلی میبدی و م. ر. میبدی، "یک معیار مبتنی بر واریانس برای ارزیابی یادگیری آتاماتای یادگیر در حل مسایل بهینه‌سازی گراف تصادفی،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، ب- مهندسی کامپیوتر، سال 15، شماره 1، صص. 13-1، بهار 1396.
[26] م. ر. ملاخلیلی میبدی و م. ر. میبدی، "یک چارچوب مبتنی بر آتاماتای یادگیر توزیع شده توسعه یافته برای حل مسأله یافتن زیرگراف بهینه تصادفی،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، ب- مهندسی کامپیوتر، سال 12، شماره 2، صص. 95-85، زمستان 1393.
[27] م. طهماسبی، ف. فتوحی و م. اسماعیلی، "پیشنهاد‌دهنده تطبیقی منابع آموزشی بر اساس سبک یادگیری، بازخورد کاربر و الگوریتم اتوماتای یادگیر،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، ب- مهندسی کامپیوتر، سال 18، شماره 1، صص. 71-60، بهار 1398.
[28] م. ر. خجسته و م. ر. میبدی، "همکاری در سیستم¬های چند عامله با استفاده از اتوماتاهای یادگیر،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، سال 1، شماره 2، صص. 11-81، پاییز و زمستان 1382.
[29] ب. معصومی و م. ر. میبدی، "مدلی مبتنی بر آنتروپی و اتوماتاهاي یادگیر برای حل بازی‌های تصادفی،" نشریه مهندسی برق و مهندسی كامپیوتر ایران، سال 8، شماره 2، صص. 106-97، تابستان 1389.
[30] A. Rezvanian, B. Moradabadi, M. Ghavipour, M. M. D. Khomami, and M. R. Meybodi, Learning Automata Approach for Social Networks, vol. 820, Springer 2019.
[31] س. روح¬الهی، ع. خطیبی بردسیری و ف. کی¬نیا،" نمونه¬گیری از شبکه¬های اجتماعی به کمک بهره گیری از واحد ارزیاب در ماشین های یادگیر با ساختار ثابت،" مجله هوش مصنوعی و داده¬کاوی، سال 8، شماره 1، صص. 148-127، زمستان 1399.
[32] T. Tommasi, F. Orabona, and B. Caputo, "Learning categories from few examples with multi model knowledge transfer," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 5, pp. 928-941, May 2014.
[33] L. Fei-Fei, R. Fergus, and P. Perona, "One-shot learning of object categories," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 4, pp. 594-611, Feb. 2006.
[34] S. Rahman, S. Khan, and F. Porikli, "A unified approach for conventional zero-shot, generalized zero-shot, and few-shot learning," IEEE Trans. Image Process., vol. 27, no. 11, pp. 5652-5667, Jul. 2018.
[35] X. Dong, L. Zheng, F. Ma, Y. Yang, and D. Meng, "Few-example object detection with model communication," IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 7, pp. 1641-1654, Jun. 2018.
[36] A. Rezvanian and M. R. Meybodi, "LACAIS: learning automata based cooperative artificial immune system for function optimization," in Proc. Int. Conf. on Contemporary Computing, vol. 2, pp. 64-75, Noida, India, August 9-11 Aug. 2010.
[37] M. Mozafari, M. E. Shiri, and H. Beigy, "A cooperative learning method based on cellular learning automata and its application in optimization problems," J. Comput. Sci., vol. 11, no. 1, pp. 279-288, Nov. 2015.
[38] M. R. Khojasteh and M. R. Meybodi, "Evaluating learning automata as a model for cooperation in complex multi-agent domains," Robot Soccer World Cup, vol. 4434, pp. 410-417, 19-20 Jun. 2006.
[39] F. Zhuang, et al., A Comprehensive Survey on Transfer Learning, arXiv Prepr. arXiv1911.02685, 2019.
[40] Z. Wan, R. Yang, M. Huang, N. Zeng, and X. Liu, "A review on transfer learning in EEG signal analysis," Neurocomputing, vol. 421, pp. 1-14, 15 Jan. 2021.
[41] S. J. Pan and Q. Yang, "A survey on transfer learning," IEEE Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345-1359, Oct. 2010.
[42] R. Vafashoar, H. Morshedlou, A. Rezvanian, and M. R. Meybodi, "Learning from multiple reinforcements in cellular learning automata," in Cellular Learning Automata: Theory and Applications, Springer, vol. 307, pp. 111-156, 2021.
[43] M. M. D. Khomami, A. Rezvanian, A. M. Saghiri, and M. R. Meybodi, "Overlapping community detection in social networks using cellular learning automata," in Proc. 28th Iranian Conf. on Electrical Engineering, 6 pp., Tabriz, Iran, 4-6 Aug. 2020.
[44] H. Beigy and M. R. Meybodi, "Open synchronous cellular learning automata," Adv. Complex Syst., vol. 10, no. 04, pp. 527-556, Dec. 2007.
[45] H. Beigy and M. R. Meybodi, "Asynchronous cellular learning automata," Automatica, vol. 44, no. 5, pp. 1350-1357, May 2008.
[46] M. Esnaashari and M. R. Meybodi, "Irregular cellular learning automata," IEEE Trans. Cybern., vol. 45, no. 8, pp. 1622-1632, Oct. 2014.
[47] M. Ahangaran, N. Taghizadeh, and H. Beigy, "Associative cellular learning automata and its applications," Appl. Soft Comput., vol. 53, pp. 1-18, Apr. 2017.
[48] A. M. Saghiri and M. R. Meybodi, "An adaptive super-peer selection algorithm considering peers capacity utilizing asynchronous dynamic cellular learning automata," Appl. Intell., vol. 48, no. 2, pp. 271-299, Jul. 2018.
[49] R. Vafashoar, H. Morshedlou, A. Rezvanian, and M. R. Meybodi, "Cellular learning automata for collaborative loss sharing," in Cellular Learning Automata: Theory and Applications, vol. 307, pp. 255-284, Springer, 2021.
[50] M. M. D. Khomami, A. Rezvanian, A. M. Saghiri, and M. R. Meybodi, "Utilizing cellular learning automata for finding communities in weighted networks," in Proc. 6th In. Conf. on Web Research, ICWR’20, pp. 325-329, Tehran, Iran, 22-23 Apr. 2020.
[51] M. M. D. Khomami, A. Rezvanian, A. M. Saghiri, and M. R. Meybodi, "SIG-CLA: a significant community detection based on cellular learning automata," in Proc. 8th Iranian Joint Congress on Fuzzy and Intelligent Systems, CFIS’20, pp. 39-44, Mashhad, Iran, 2-4 Sept. 2020.
[52] L. Zuo, M. Jing, J. Li, L. Zhu, K. Lu, and Y. Yang, "Challenging tough samples in unsupervised domain adaptation," Pattern Recognit., vol. 110, Article No.: 107540, Feb. 2021.
[53] S. Niu, Y. Liu, J. Wang, and H. Song, "A decade survey of transfer learning (2010-2020)," IEEE Trans. Artif. Intell., vol. 1, no. 2, pp. 151-166, Feb. 2020.
[54] D. Sarkar, R. Bali, and T. Ghosh, Hands-On Transfer Learning with Python: Implement Advanced Deep Learning and Neural Network Models Using TensorFlow and Keras, Packt Publishing Ltd, 2018.
[55] M. Arora, P. Mangipudi, and M. K. Dutta, "Deep learning neural networks for acrylamide identification in potato chips using transfer learning approach," J. Ambient Intell. Humaniz. Comput., vol. 2, no. 12, pp. 10601-10614, Jan. 2021.
[56] Q. Yang, Y. Zhang, W. Dai, and S. J. Pan, Transfer Learning, Cambridge University Press, 2020.
[57] Q. Sun, Y. Liu, Z. Chen, T. S. Chua, and B. Schiele, "Meta-transfer learning through hard tasks," IEEE Trans. Pattern Anal. Mach. Intell., Early access, Aug. 2020.
[58] B. Al-Helali, Q. Chen, B. Xue, and M. Zhang, "Multi-tree genetic programming with new operators for transfer learning in symbolic regression with incomplete data," IEEE Trans. Evol. Comput., Early access,, May 2021.
[59] Q. Gu, Q. Dai, H. Yu, and R. Ye, "Integrating multi-source transfer learning, active learning and metric learning paradigms for time series prediction," Appl. Soft Comput., vol. 109, Article No.: 107583, Sept. 2021
. [60] R. Vafashoar, H. Morshedlou, A. Rezvanian, and M. R. Meybodi, Cellular Learning Automata: Theory and Applications, Springer 2019.
[61] M. He, J. Zhang, and J. Zhang, "Restoring latent factors against negative transfer using partial-adaptation nonnegative matrix factorization," CCF Trans. Pervasive Comput. Interact., vol. 2, no. 1, pp. 42-50, Oct. 2020.
[62] C. Di, S. Li, F. Li, and K. Qi, "A novel framework for learning automata: a statistical hypothesis testing approach," IEEE Access, vol. 7, pp. 27911-27922, Feb. 2019.
[63] A. M. Saghiri and M. R. Meybodi, "On expediency of closed asynchronous dynamic cellular learning automata," J. Comput. Sci., vol. 24, no. 1, pp. 371-378, Jan. 2018.
[64] H. Ge, H. Huang, Y. Li, S. Li, and J. Li, "Two approaches on accelerating bayesian two action learning automata," in Proc. Int.Conf. on Intelligent Computing, pp. 239-247, Lanzhou, China, 2-5 Aug. 2016.
[65] H. Ge, Y. Yan, J. Li, Y. Guo, and S. Li, "A parameter-free gradient bayesian two-action learning automaton scheme," in Proc. of the Int Conf. on Communications, Signal Processing, and Systems, pp. 963-970, Chengdu, China, 23-24 Oct. 2016.
[66] Y. Guo, H. Ge, and S. Li, "A loss function based parameterless learning automaton scheme," Neurocomputing, vol. 260pp. 331-340, Oct. 2017.
[67] Y. Guo and S. Li, "A non-monte-carlo parameter-free learning automata scheme based on two categories of statistics," IEEE Trans. Cybern., vol. 49, no. 12, pp. 4153-4166, Aug. 2018.
[68] H. Ge, J. Li, S. Li, W. Jiang, and Y. Wang, "A novel parallel framework for pursuit learning schemes," Neurocomputing, vol. 228, pp. 198-204, Mar. 2017.
[69] D. Neider, R. Smetsers, F. Vaandrager, and H. Kuppens, "Benchmarks for automata learning and conformance testing," in Models, Mindsets, Meta: the What, the How, and the Why Not?, pp. 390-416, Springer, 2019.
[70] C. Carson, S. Belongie, H. Greenspan, and J. Malik, "Blobworld: image segmentation using expectation-maximization and its application to image querying," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 8, pp. 1026-1038, Nov. 2002.