Results 11  20
of
106
Undecidable Problems for Probabilistic Automata of Fixed Dimension
 Theory of Computing Systems
, 2001
"... We prove that several problems associated to probabilistic finite automata are undecidable for automata whose number of input letters and number of states are fixed. As a corollary of one of our results we prove that the problem of determining if the set of all products of two 47 × 47 matr ..."
Abstract

Cited by 44 (4 self)
 Add to MetaCart
(Show Context)
We prove that several problems associated to probabilistic finite automata are undecidable for automata whose number of input letters and number of states are fixed. As a corollary of one of our results we prove that the problem of determining if the set of all products of two 47 &times; 47 matrices with nonnegative rational entries is bounded is undecidable.
A pointbased POMDP algorithm for robot planning
, 2004
"... We present an approximate POMDP solution method for robot planning in partially observable environments. Our algorithm belongs to the family of pointbased value iteration solution techniques for POMDPs, in which planning is performed only on a sampled set of reachable belief points. We describe a s ..."
Abstract

Cited by 41 (2 self)
 Add to MetaCart
We present an approximate POMDP solution method for robot planning in partially observable environments. Our algorithm belongs to the family of pointbased value iteration solution techniques for POMDPs, in which planning is performed only on a sampled set of reachable belief points. We describe a simple, randomized procedure that performs value update steps that strictly improve the value of all belief points in each step. We demonstrate our algorithm on a robotic delivery task in an office environment and on several benchmark problems, for which we compute solutions that are very competitive to those of stateofthe art methods in terms of speed and solution quality.
Nonapproximability Results for Partially Observable Markov Decision Processes
, 2000
"... We show that for several variations of partially observable Markov decision processes, polynomialtime algorithms for nding control policies are unlikely to or simply don't have guarantees of nding policies within a constant factor or a constant summand of optimal. Here "unlikely" ..."
Abstract

Cited by 40 (0 self)
 Add to MetaCart
We show that for several variations of partially observable Markov decision processes, polynomialtime algorithms for nding control policies are unlikely to or simply don't have guarantees of nding policies within a constant factor or a constant summand of optimal. Here "unlikely" means \unless some complexity classes collapse," where the collapses considered are P = NP, P = PSPACE, or P = EXP. Until or unless these collapses are shown to hold, any controlpolicy designer must choose between such performance guarantees and ecient computation.
Solving Factored POMDPs with Linear Value Functions
 In IJCAI01 workshop on Planning under Uncertainty and Incomplete Information
, 2001
"... Partially Observable Markov Decision Processes (POMDPs) provide a coherent mathematical framework for planning under uncertainty when the state of the system cannot be fully observed. However, the problem of finding an exact POMDP solution is intractable. ..."
Abstract

Cited by 32 (2 self)
 Add to MetaCart
Partially Observable Markov Decision Processes (POMDPs) provide a coherent mathematical framework for planning under uncertainty when the state of the system cannot be fully observed. However, the problem of finding an exact POMDP solution is intractable.
AEMS: an anytime online search algorithm for approximate policy refinement in large POMDPs
 In: Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI
, 2007
"... Solving large Partially Observable Markov Decision Processes (POMDPs) is a complex task which is often intractable. A lot of effort has been made to develop approximate offline algorithms to solve ever larger POMDPs. However, even stateoftheart approaches fail to solve large POMDPs in reasonable t ..."
Abstract

Cited by 28 (9 self)
 Add to MetaCart
Solving large Partially Observable Markov Decision Processes (POMDPs) is a complex task which is often intractable. A lot of effort has been made to develop approximate offline algorithms to solve ever larger POMDPs. However, even stateoftheart approaches fail to solve large POMDPs in reasonable time. Recent developments in online POMDP search suggest that combining offline computations with online computations is often more efficient and can also considerably reduce the error made by approximate policies computed offline. In the same vein, we propose a new anytime online search algorithm which seeks to minimize, as efficiently as possible, the error made by an approximate value function computed offline. In addition, we show how previous online computations can be reused in following time steps in order to prevent redundant computations. Our preliminary results indicate that our approach is able to tackle large state space and observation space efficiently and under realtime constraints. 1
Sequential monte carlo in probabilistic planning reachability heuristics
 Artificial Intelligence
, 2008
"... The current best conformant probabilistic planners encode the problem as a bounded length CSP or SAT problem. While these approaches can find optimal solutions for given plan lengths, they often do not scale for large problems or plan lengths. As has been shown in classical planning, heuristic searc ..."
Abstract

Cited by 27 (15 self)
 Add to MetaCart
(Show Context)
The current best conformant probabilistic planners encode the problem as a bounded length CSP or SAT problem. While these approaches can find optimal solutions for given plan lengths, they often do not scale for large problems or plan lengths. As has been shown in classical planning, heuristic search outperforms CSP/SAT techniques (especially when a plan length is not given a priori). The problem with applying heuristic search in probabilistic planning is that effective heuristics are as yet lacking. In this work, we apply heuristic search to conformant probabilistic planning by adapting planning graph heuristics developed for nondeterministic planning. We evaluate a straightforward application of these planning graph techniques, which amounts to exactly computing the distribution over reachable relaxed planning graph layers. Computing these distributions is costly, so we apply Sequential Monte Carlo to approximate them. We demonstrate on several domains how our approach enables our planner to far outscale existing (optimal) probabilistic planners and still find reasonable quality solutions.
Robot planning in partially observable continuous domains
 In Robotics: Science and Systems I
, 2005
"... Abstract — We present a value iteration algorithm for learning to act in Partially Observable Markov Decision Processes (POMDPs) with continuous state spaces. Mainstream POMDP research focuses on the discrete case and this complicates its application to, e.g., robotic problems that are naturally mod ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
(Show Context)
Abstract — We present a value iteration algorithm for learning to act in Partially Observable Markov Decision Processes (POMDPs) with continuous state spaces. Mainstream POMDP research focuses on the discrete case and this complicates its application to, e.g., robotic problems that are naturally modeled using continuous state spaces. The main difficulty in defining a (beliefbased) POMDP in a continuous state space is that expected values over states must be defined using integrals that, in general, cannot be computed in closed from. In this paper, we first show that the optimal finitehorizon value function over the continuous infinitedimensional POMDP belief space is piecewise linear and convex, and is defined by a finite set of supporting αfunctions that are analogous to the αvectors (hyperplanes) defining the value function of a discretestate POMDP. Second, we show that, for a fairly general class of POMDP models in which all functions of interest are modeled by Gaussian mixtures, all belief updates and value iteration backups can be carried out analytically and exact. A crucial difference with respect to the αvectors of the discrete case is that, in the continuous case, the αfunctions will typically grow in complexity (e.g., in the number of components) in each value iteration. Finally, we demonstrate PERSEUS, our previously proposed randomized pointbased value iteration algorithm, in a simple robot planning problem with a continuous domain, where encouraging results are observed. I.
What makes some POMDP problems easy to approximate
 In NIPS
, 2007
"... Pointbased algorithms have been surprisingly successful in computing approximately optimal solutions for partially observable Markov decision processes (POMDPs) in high dimensional belief spaces. In this work, we seek to understand the beliefspace properties that allow some POMDP problems to be ap ..."
Abstract

Cited by 20 (8 self)
 Add to MetaCart
(Show Context)
Pointbased algorithms have been surprisingly successful in computing approximately optimal solutions for partially observable Markov decision processes (POMDPs) in high dimensional belief spaces. In this work, we seek to understand the beliefspace properties that allow some POMDP problems to be approximated efficiently and thus help to explain the pointbased algorithms ’ success often observed in the experiments. We show that an approximately optimal POMDP solution can be computed in time polynomial in the covering number of a reachable belief space, which is the subset of the belief space reachable from a given belief point. We also show that under the weaker condition of having a small covering number for an optimal reachable space, which is the subset of the belief space reachable under an optimal policy, computing an approximately optimal solution is NPhard. However, given a suitable set of points that “cover ” an optimal reachable space well, an approximate solution can be computed in polynomial time. The covering number highlights several interesting properties that reduce the complexity of POMDP planning in practice, e.g., fully observed state variables, beliefs with sparse support, smooth beliefs, and circulant statetransition matrices. 1