A method for speeding up value iteration in partially observable markov decision processes (1999) [12 citations — 3 self]
Download:
|
|
by Nevin L. Zhang, Stephen S. Lee, Weihong Zhang
In Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence
http://www.cs.ust.hk/~lzhang/paper/pspdf/uai99.ps.gz
Add To MetaCart
Abstract:
We present a technique for speeding up the convergence of value iteration for partially observable Markov decisions processes (POMDPs). The underlying idea is similar to that behind modified policy iteration for fully observable Markov decision processes (MDPs). The technique can be easily incorporated into any existing POMDP value iteration algorithms. Experiments have been conducted on several test problems with one POMDP value iteration algorithm called incremental pruning. We find that the technique can make incremental pruning run several orders of magnitude faster. 1
Citations
| 361 | Markov Decision Processes – Puterman - 1994 |
| 221 | The optimal control of Partially Observable Markov Processe – Sondik - 1971 |
| 132 | A Survey of Partially Observable Markov Decision Processes: Theory, Models, and Algorithms – Monahan - 1982 |
| 112 | Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes. UAI-97 – Cassandra, Littman, et al. - 1997 |
| 79 | Exact and Approximate Algorithms for Partially Observable Markov Decision Processes – Cassandra - 1998 |
| 29 | Finite-Memory Control of Partially Observable Systems – Hansen - 1998 |
| 28 | Partially observed Markov decision processes: A survey – White - 1991 |
| 12 | A survey of POMDP applications – Cassandra - 1998 |
| 3 | Action elimination procedures for modified policy iteration algorithms – Puterman, Shin - 1982 |
| 1 | A set of successive approximation methods for discounted Markov decision problems – Nunen - 1976 |
