Anytime point-based approximations for large pomdps

DMCA

Anytime point-based approximations for large pomdps (2006)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Joelle Pineau , Geoffrey Gordon , Sebastian Thrun

Venue:	Journal of Artificial Intelligence Research
Citations:	104 - 7 self

BibTeX

@ARTICLE{Pineau06anytimepoint-based,
    author = {Joelle Pineau and Geoffrey Gordon and Sebastian Thrun},
    title = {Anytime point-based approximations for large pomdps},
    journal = {Journal of Artificial Intelligence Research},
    year = {2006},
    pages = {27--335}
}

OpenURL

Abstract

The Partially Observable Markov Decision Process has long been recognized as a rich framework for real-world planning and control problems, especially in robotics. However exact solutions in this framework are typically computationally intractable for all but the smallest problems. A well-known technique for speeding up POMDP solving involves performing value backups at specific belief points, rather than over the entire belief simplex. The efficiency of this approach, however, depends greatly on the selection of points. This paper presents a set of novel techniques for selecting informative belief points which work well in practice. The point selection procedure is combined with point-based value backups to form an effective anytime POMDP algorithm called Point-Based Value Iteration (PBVI). The first aim of this paper is to introduce this algorithm and present a theoretical analysis justifying the choice of belief selection technique. The second aim of this paper is to provide a thorough empirical comparison between PBVI and other state-of-the-art POMDP methods, in particular the Perseus algorithm, in an effort to highlight their similarities and differences. Evaluation is performed using both standard POMDP domains and realistic robotic tasks.

Keyphrases

large pomdps point-based approximation point selection procedure effective anytime pomdp algorithm partially observable markov decision process novel technique exact solution theoretical analysis standard pomdp domain control problem perseus algorithm rich framework value backup well-known technique second aim specific belief point informative belief point entire belief simplex point-based value iteration belief selection technique thorough empirical comparison real-world planning realistic robotic task point-based value backup first aim state-of-the-art pomdp method

DMCA