Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning

DMCA

Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning (1999)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Richard S. Sutton , Doina Precup , Satinder Singh

Citations:

568 - 38 self

BibTeX

@MISC{Sutton99betweenmdps,
    author = {Richard S. Sutton and Doina Precup and Satinder Singh},
    title = {Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning},
    year = {1999}
}

OpenURL

Abstract

Learning, planning, and representing knowledge at multiple levels of temporal abstraction are key, longstanding challenges for AI. In this paper we consider how these challenges can be addressed within the mathematical framework of reinforcement learning and Markov decision processes (MDPs). We extend the usual notion of action in this framework to include options|closed-loop policies for taking action over a period of time. Examples of options include picking up an object, going to lunch, and traveling to a distant city, as well as primitive actions such as muscle twitches and joint knowledge and action to be included in the reinforcement learning framework in a natural and general way. In particular, we show that options may be used interchangeably with primitive actions in planning methods such as dynamic programming and in learning methods such as Q-learning. Formally, a set of options defined

Keyphrases

reinforcement learning temporal abstraction distant city dynamic programming general way multiple level primitive action longstanding challenge primitive action asmuscle twitch option closed-loop policy mathematical framework joint knowledge usual notion markov decision process

DMCA