A New Statistical Characteristics Based Method for Adaptive Learning Rate Adjustment in Learning Automata
Subject Areas : electrical and computer engineeringM. R. Mollakhalili Meybodi 1 * , M. R. Meybodi 2
1 -
2 -
Keywords: Learning automata dynamic learning rate learning rate adjustment Chebyshev’s inequality,
Abstract :
The value of learning rate and its change mechanisms is one of the issues in designing learning systems such as learning automata. In most cases a time-based reduction function is used to adjust the learning rate aim at reaching stability in training system. So the learning rate is a parameter that determines to what extent a learning system is based on past experiences, and the impact of current events on it. This method is efficient but does not properly function in dynamic and non-stationary environments. In this paper, a new method for adaptive learning rate adjustment in learning automata is proposed. In this method, in addition to the length of time to learn, some statistical characteristics of actions probability vector of Learning Automata are used to determine the increase or decrease of learning rate. Furthermore, unlike existing methods, during the process of learning, both increase and decrease of the learning rate is done and Learning Automata responds effectively to changes in the dynamic random environment. Empirical studies show that the proposed method has more flexibility in compatibility to the non-stationary dynamic environments and get out of local maximum points and the learned values are closer to the true values.
[1] H. Beigy, M. R. Meybodi, and M. B. Menhaj, "Adaptation of learning rate in back propagation algorithm using fixed structure learning automata," in Proc. 6th Iranian Conf. on Electrical Engineering, pp. 117-123, Tehran, Iran, 1998.
[2] H. Shah-Hosseini and R. Safabaksh, "Automatic adjustment of learning rates of the self-organizing feature map," Scientia Iranica, vol. 8, no. 4, pp. 277-286, Oct. 2001.
[3] C. Chinrungrueng and C. H. Sequin, "Optimal adaptive k - means algorithm with dynamic adjustment of learnig rate," IEEE Trans. on Neural Networks, vol. 6, no. 1, pp. 157-169, Jan. 1995.
[4] J. Akbari Torkestani and M. R. Meybodi, "Learning automata - based algorithms for solving stochastic minimum spanning tree problem," Applied Soft Computing, vol. 11, no. 6, pp. 4064-4077, Sep. 2011.
[5] J. Akbari Torkestani and M. R. Meybodi, "Learning automata - based algorithms for finding minimum weakly connected dominating set in stochastic graphs," International J. of Uncertainty, Fuzziness, and Knowledge - Based Systems, vol. 18, no. 6, pp. 721-758, Dec. 2010.
[6] H. Beigy and M. R. Meybodi, "Utilizing distributed learning automata to solve stochastic shortest path problems," International J. of Uncertainty, vol. 14, no. 5, pp. 591-615, Oct. 2006.
[7] M. R. Mollakhalili Meybodi and M. R. Meybodi, "A new distributed learning automata based algorithm for solving stochastic shortest path," in Proc. 6th Conf. on Intelligent Systems, Kerman, Iran, 26-27 Nov. 2004.
[8] M. R. Meybodi and H. Beigy, "Solving stochastic shortest path problem using distributed learning automata," in Proc. 6th Annual CSI Computer Conf., CSICC 2001, pp. 70-86, Isfehan, Iran, Feb. 2001.
[9] H. Beigy Intelligent Channel Assignment in Cellular Networks: A Learning Automata Approach, Ph.D. Thesis, Amirkabir University of Technology, 2004.
[10] M. L. Thathachar and P. S. Sastry, "Varieties of learning automata: an overview," IEEE Trans. on Systems, Man, and Cybernetics, Part B, Cybernetics, vol. 32, no. 6, pp. 711-22, Jan. 2002.
[11] K. S. Narenda and M. A. L. Thatacher, Learning Automata, Prentice - Hall, 1989.
[12] M. L. Tsetlin, "On the behaviour of finite automata in random media," Automata., Telemekh., vol. 22, pp. 1345-1354, Oct. 1961.
[13] B. J. Oomen and H. Masum, "Switching models for nonstationary random environments," IEEE Trans. on Systems, Man, and Cybernetics., vol. 25, no. 9, pp. 1334-1347, Sep. 1995.
[14] K. S. Narendra and M. A. L. Thathachar, "On the behavior of a learning automaton in a changing environment with application to telephone traffic routing," Systems, Man, and Cybernetics, IEEE Trans. on, vol. 10, no. 5, pp. 262-269, May 1980.
[15] S. M. Ross, Introduction to Probability and Statistics dor Engineers and Scientists, 3rd. Elsevier Academic Press, 2004.
[16] A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd Ed. New York, USA: McGrawHill, 1991.