Multi-time Models for Temporally Abstract Planning (1997)
| Venue: | In Advances in Neural Information Processing Systems 10 |
| Citations: | 72 - 8 self |
BibTeX
@INPROCEEDINGS{Precup97multi-timemodels,
author = {Doina Precup and R. Sutton},
title = {Multi-time Models for Temporally Abstract Planning},
booktitle = {In Advances in Neural Information Processing Systems 10},
year = {1997},
pages = {1050--1056},
publisher = {MIT Press}
}
Bookmark
 |
 |
 |
 |
 |
 |
OpenURL
Abstract
Planning Doina Precup, Richard S. Sutton University of Massachusetts Amherst, MA 01003 fdprecupjrichg@cs.umass.edu Abstract Planning and learning at multiple levels of temporal abstraction is a key problem for artificial intelligence. In this paper we summarize an approach to this problem based on the mathematical framework of Markov decision processes and reinforcement learning. Current model-based reinforcement learning is based on one-step models that cannot represent common-sense higher-level actions, such as going to lunch, grasping an object, or flying to Denver. This paper generalizes prior work on temporally abstract models [Sutton, 1995] and extends it from the prediction setting to include actions, control, and planning. We introduce a more general form of temporally abstract model, the multi-time model, and establish its suitability for planning and learning by virtue of its relationship to the Bellman equations. This paper summarizes the theoretical framewo...
