Multiple model-based reinforcement learning

DMCA

Multiple model-based reinforcement learning (2002)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Kenji Doya , Kazuyuki Samejima

Venue:	Neural Computation
Citations:	85 - 5 self

BibTeX

@ARTICLE{Doya02multiplemodel-based,
    author = {Kenji Doya and Kazuyuki Samejima},
    title = {Multiple model-based reinforcement learning},
    journal = {Neural Computation},
    year = {2002},
    volume = {14},
    pages = {1347--1369}
}

OpenURL

Abstract

We propose a modular reinforcement learning architecture for non-linear, non-stationary control tasks, which we call multiple model-based reinforcement learn-ing (MMRL). The basic idea is to decompose a complex task into multiple domains in space and time based on the predictability of the environmental dynamics. The 1 system is composed of multiple modules, each of which consists of a state predic-tion model and a reinforcement learning controller. The “responsibility signal,” which is given by the softmax function of the prediction errors, is used to weight the outputs of multiple modules as well as to gate the learning of the predic-tion models and the reinforcement learning controllers. We formulate MMRL for both discrete-time, finite state case and continuous-time, continuous state case. The performance of MMRL was demonstrated for discrete case in a non-stationary hunting task in a grid world and for continuous case in a non-linear, non-stationary control task of swinging up a pendulum with variable physical parameters. 1

Keyphrases

multiple model-based reinforcement learning multiple module non-stationary control task continuous case continuous state case non-stationary hunting task grid world multiple domain responsibility signal state predic-tion model predic-tion model modular reinforcement learning architecture softmax function finite state case basic idea complex task environmental dynamic discrete case multiple model-based reinforcement learn-ing variable physical parameter prediction error

DMCA