Transfer Learning Through Policy Abstraction Using Learning Vector Quantization

Authors

  • Ahmad Afif Mohd Faudzi Faculty of Electric and Electronics Engineering, Universiti Malaysia Pahang, Pekan, Pahang, Malaysia.
  • Hirotaka Takano Department of Electrical and Electronics Engineering, Faculty of Engineering, University of Fukui, Fukui, Japan.
  • Junichi Murata Department of Electrical Engineering, Kyushu University, Fukuoka, Japan.

Keywords:

Abstraction, Learning Vector Quantization, Reinforcement Learning, Transfer Learning,

Abstract

Reinforcement learning (RL) enables an agent to find a solution to a problem by interacting with the environment. However, the learning process always starts from scratch and possibly takes a long time. Here, knowledge transfer between tasks is considered. In this paper, we argue that an abstraction can improve the transfer learning. Modified learning vector quantization (LVQ) that can manipulate its network weights is proposed to perform an abstraction, an adaptation and a precaution. At first, the abstraction is performed by extracting an abstract policy out of a learned policy which is acquired through conventional RL method, Q-learning. The abstract policy then is used in a new task as prior information. Here, the adaptation or policy learning as well as new task's abstract policy generating are performed using only a single operation. Simulation results show that the representation of acquired abstract policy is interpretable, that the modified LVQ successfully performs policy learning as well as generates abstract policy and that the application of generalized common abstract policy produces better results by more effectively guiding the agent when learning a new task.

Downloads

Published

2018-01-22

How to Cite

Mohd Faudzi, A. A., Takano, H., & Murata, J. (2018). Transfer Learning Through Policy Abstraction Using Learning Vector Quantization. Journal of Telecommunication, Electronic and Computer Engineering (JTEC), 10(1-3), 163–168. Retrieved from https://jtec.utem.edu.my/jtec/article/view/3505

Issue

Vol. 10 No. 1-3: Moving Towards Industry 4.0 II

Section

Articles

License

TRANSFER OF COPYRIGHT AGREEMENT

The manuscript is herewith submitted for publication in the Journal of Telecommunication, Electronic and Computer Engineering (JTEC). It has not been published before, and it is not under consideration for publication in any other journals. It contains no material that is scandalous, obscene, libelous or otherwise contrary to law. When the manuscript is accepted for publication, I, as the author, hereby agree to transfer to JTEC, all rights including those pertaining to electronic forms and transmissions, under existing copyright laws, except for the following, which the author(s) specifically retain(s):

  1. All proprietary right other than copyright, such as patent rights
  2. The right to make further copies of all or part of the published article for my use in classroom teaching
  3. The right to reuse all or part of this manuscript in a compilation of my own works or in a textbook of which I am the author; and
  4. The right to make copies of the published work for internal distribution within the institution that employs me

I agree that copies made under these circumstances will continue to carry the copyright notice that appears in the original published work. I agree to inform my co-authors, if any, of the above terms. I certify that I have obtained written permission for the use of text, tables, and/or illustrations from any copyrighted source(s), and I agree to supply such written permission(s) to JTEC upon request.