M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of policy existence problem for partially-observable nite-horizon Markov decision processes. In Mathematical Foundations of Computer Science, pages 129-38, 1997.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....time steps, and in case of UMDPs we will consider both nite and in nite sequence of actions. This contrasts with nite horizon objective, where the decision maker has a speci ed time limit in executing actions. The complexity of nite horizon POMDPs has been extensively studied; see for example [98, 76, 92, 93]. 2.3.3 Measures of Value of Action Sequences In solving UMDPs, we need a measure of value of an action sequence to formulate optimization objectives and computational problems. We will sometimes refer to a choice of value measure as an optimality criterion. Once a value measure is de ned, an ....

M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of policy existence problem for partially-observable nite-horizon Markov decision processes. In Mathematical Foundations of Computer Science, pages 129-38, 1997.

....and in case of UMDPs we will consider both finite and infinite sequences of actions. This contrasts with finite horizon objectives, where the decision maker executes a fixed and known number of actions. The complexity of finite horizon MDPs and POMDPs has been extensively studied; see for example [25,17,22,23]. 2.2.3 Measures of Value of Action Sequences In solving planning and MDPs problems, we need a measure of value of an action sequence to formulate optimization objectives and computational problems. We will sometimes refer to a choice of value measure as an optimality criterion. Once a value ....

.... either nonzero probability or probability one, which reduce to reachability computations and are decidable, had been studied by Alur et al. 1] and Littman [17] It is now well established that optimal planning without full observability is prohibitively difficult both in theory and practice [25,17,22]. These results suggest that it may be more promising to explore alternative problem formulations, including restrictions on the system dynamics and the agent s sensing and effecting powers that are useful for realistic problem domains yet are more amenable to exact or approximate solution ....

M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of policy existence problem for partially-observable finite-horizon Markov decision processes. In Mathematical Foundations of Computer Science, pages 129--38, 1997.

.... of this work were performed while the fourth author was at the Institute of Mathematical Sciences, Chennai (Madras) India, and at the Wilhelm Schickard Institut fur Informatik, Universitat Tubingen (supported by DFG grant TU 7 117 1) Preliminary versions of some of this work appeared as Mundhenk, Goldsmith, and Allender [1997] and Goldsmith and Mundhenk [1998] Name: Martin Mundhenk Address: Universitat Trier, FB IV Informatik, D 54286 Trier, Germany, mundhenk ti.uni trier.de Affiliation: Universitat Trier Name: Judy Goldsmith Address: Dept. of Computer Science, University of Kentucky, Lexington KY 40506 0046, ....

Mundhenk, M., Goldsmith, J., and Allender, E. 1997. The complexity of the policy existence problem for partially-observable finite-horizon Markov decision processes. In Proc. 25th Mathematical Foundations of Computer Sciences (1997), pp. 129--138. Lecture Notes in Computer Science #1295: Springer-Verlag.

.... of this work were performed while the fourth author was at the Institute of Mathematical Sciences, Chennai (Madras) India, and at the Wilhelm Schickard Institut f ur Informatik, Universit at T ubingen (supported by DFG grant TU 7 117 1) Preliminary versions of some of this work appeared as Mundhenk, Goldsmith, and Allender [1997] and Goldsmith and Mundhenk [1998] Name: Martin Mundhenk Address: Universit at Trier, FB IV Informatik, D 54286 Trier, Germany, mundhenk ti.uni trier.de Aliation: Universit at Trier Name: Judy Goldsmith Address: Dept. of Computer Science, University of Kentucky, Lexington KY 40506 0046, ....

Mundhenk, M., Goldsmith, J., and Allender, E. 1997. The complexity of the policy existence problem for partially-observable nite-horizon Markov decision processes. In Proc. 25th Mathematical Foundations of Computer Sciences (1997), pp. 129-138. Lecture Notes in Computer Science #1295: Springer-Verlag.

....315 PPP g u ab 1 For POMDPs with nonpositive rewards, Papadimitriou and Tsitsiklis [PT87, Corollary 1] showed that optimal policies are not polynomially representable unless PSPACE equals p 2 , the second level of the Polynomial Time Hierarchy. Later on, Mundhenk, Goldsmith, and Allender [MGA97] showed the same NP and PSPACE completeness results of optimal policy computation for POMDPs with both negative and positive rewards. This gives rise to the assumption that optimal policy computation for POMDPs with nonpositive rewards has the same complexity as for POMDPs with both negative and ....

....for any polynomially representable policies (see also [PT87, Corollary 1] for a de nition) Moreover, it is not hard to see that the hardness results also hold for in nite horizon policy existence problems. NC 1 representable policies are a slight generalization of time dependent policies. In [MGA97], the policy existence problem is proven NP complete for time dependent policies. This shows that already a slight change in the problem parameters has a big in uence on the problem complexity. The optimal performance of a POMDP under any small policy of a given size can be calculated using a ....

M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of the policy existence problem for partially-observable nite-horizon Markov decision processes. In Proc. 25th Mathematical Foundations of Computer Sciences, pages 129-138. Lecture Notes in Computer Science #1295, Springer-Verlag, 1997.

....than or equal to k. For example, the question of whether there exists a history dependent policy for a POMDP with nonnegative rewards is NL complete [MGLA97] as compared to PSPACE completeness in [PT87, Theorem 6] Beauquier et al. BBS95] also considered various optimality criteria for MDPs. In [MGA97, MGLA97], a systematical investigation of the complexity of Markov decision process decision problems under various parameters continued the work initiated in [PT87] Methods from [MGA97] were applied in [LGM98a] to show complexity results for planners, as described in the AI literature. The AI planning ....

....in [PT87, Theorem 6] Beauquier et al. BBS95] also considered various optimality criteria for MDPs. In [MGA97, MGLA97] a systematical investigation of the complexity of Markov decision process decision problems under various parameters continued the work initiated in [PT87] Methods from [MGA97] were applied in [LGM98a] to show complexity results for planners, as described in the AI literature. The AI planning community has analyzed the complexity of several variations of MDPs and a wide variety of other related models. Most of these AI papers, especially [Bac95, Byl94, Cha87, EHN96, ....

[Article contains additional citation context not shown here]

M. Mundhenk, J. Goldsmith, and E. Allender. The complexity of the policy existence problem for partially-observable finite-horizon Markov decision processes. In Proc. 25th Mathematical Foundations of Computer Sciences, pages 129-- 138. Lecture Notes in Computer Science #1295, Springer-Verlag, 1997.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC