C. Boutilier, R. Brafman, and C. Geib. Structured reachability analysis for Markov decision processes. In Proceedings of UAI'98, 1998.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....structured value function representations (in fact, most of the work described in Section 2.6) are in some ways orthogonal to the ideas we present and could potentially be combined with SPI. For reachability analysis, one example of this kind of composite approach has already been described in [14]. Combining SPI with the explanation based reinforcement learning ideas of [38] is another fruitful area of research. The feature based representation used by SPI increases the effectiveness of the funnel actions used by Dietterich and Flann by allowing generalization of a funnel action over ....

Craig Boutilier, Ronen I. Brafman, and Christopher Geib. Structured reacha- bility analysis for Markov decision processes. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 24 32, Madison, WI, 1998.

....POMDPs are conducted over the entire belief space. Our work focuses on accelerating value iterations for such POMDP class by restricting them over a subset of belief space. The approach we use to exclude belief states from being considered works much like that in reachability analysis (e.g. see [6, 3]) In fully observable MDP, this technique is used to restrict value iteration over a small subset of state space. Even although value iteration is restricted into a subspace for informative POMDPs, we show that value function of good quality over entire belief space can be obtained from value ....

Boutilier, C., Brafman, R. I. and Geib, C. (1998). Structured reachability analysis for Markov decision processes. In Proceedings of UAI-98.

....of the underlying state space. Algorithms have been developed that can efficiently solve decisiontheoretic planning problems of reasonable size : in particular, 4] reviews a number of approaches developed to prune as many branches as possible in the search graph. More specifically, the work in [3] is remarkable as proposing to reuse ideas from Graphplan [1] for reachability analysis in solving MDPs. Yet, it may not always be easy to draw an adapted MDPlike discretization of the state space from the initial problem definition : in [7] for example, the workspaces of the pursuer and the ....

....tools, which seems a good idea to generalize whenever equivalent powerful tools exist. This again, leads to the idea that classical planning methods could be somewhat adapted so as to take uncertainty measures into account at planning time, or alternatively, as already proposed by C. Boutilier [3], in order to build an appropriate search graph and allow a subsequent decision process to deal with it properly. For instance, PGraphplan in [2] takes probabilistic actions into account and Sensory Graphplan [21] handles uncertainty about the initial state. 10] presents another attempt to extend ....

C. Boutilier, R.I. Brafman, and C. Geib, `Structured reachability analysis for markov decision processes',inUAI'98, pp. 24--32, Madison, (jul. 1998).

....to produce partially ordered action sets from them using UCPOP. The flexibility in the partial order makes it easier to merge the component policies into a policy for the original reward 18 function, again much as an SNLP based planner such as UCPOP might merge subplans for individual goals. In (Boutilier, Brafman, Geib 1998), a reachability analysis inspired by Graphplan is used to restrict the states considered for policy creation given an initial state. Givan and Dean (Givan Dean 1997) show that STRIPS style goal regression computes an approximate minimized form of the finite state automaton corresponding to the ....

Boutilier, C.; Brafman, R.; and Geib, C. 1998. Structured reachability analysis for markov decision processes. In Proc. Fourtheenth Conference on Uncertainty in Artificial Intelligence. Madison, Wisconsin: Morgan Kaufmann.

....onto a stochastic satisfiability problem, solving the problem in that representation. Maxplan is a blind planner, while Zander produces contingent plans. There has also been work on probabilistic planning explicitly using representations motivated by Graphplan. In particular, Boutilier et al. [BBG98] generalize Graphplan s pairwise mutual exclusion constraints to k wise constraints, and examine the reduction in the size of the MDP that is implicitly represented by the layer at which the planning graph levels off. The work of Dean et al. DKKN95] has a close connection to the goals of ....

C. Boutilier, R. I. Brafman, and C. Geib. Structured reachability analysis for Markov Decision Processes. In Proceedings of the Fourteenth Annual Conference on Uncertainty in Artificial Intelligence (UAI--98), pages 24-- 32, 1998.

....influence on the ability to propagate reachability and relevance information via unit propagation and, more generally, k clause resolution. We do so by comparing the pruning ability of these techniques to that of variants of existing algorithms that operate on the original problem formulation [Boutilier et al. 1998] . Our work is motivated by unit resolution s central role in the Davis Putnam algorithm [Davis et al. 1962] and many of its offsprings (e.g. Freeman, 1995; Crawford and Auton, 1993; Li and Anbulagan, 1997; Gomes et al. 1998] and its use as a preprocessing step when stochastic methods are ....

....1996; Nebel et al. 1997] or both [Kambhampati et al. 1997] The importance of reachability and relevance analysis has been noted in the context of decision theoretic planning as well. For example, Boutilier and Dearden, 1994] employ relevance analysis to reduce the state space, and [Boutilier et al. 1998] describe a general method for reachability analysis for MDPs. Below, we discuss this method in a simplified form suitable for classical planning problems described using the STRIPS representation language [Fikes and Nilsson, 1971] We shall also present a counterpart of this method for ....

[Article contains additional citation context not shown here]

C. Boutilier, R. I. Brafman, and C. Geib. Structured reachability analysis for markov decision processes. In Proc. UAI'98, 1998.

....the pruning ability of these techniques to that of a class of algorithms for reachability and relevance analysis that operate on the original problem formulation: Reachable k and Relevant k. Reachable k is a simpli ed variant of a similar algorithm for state pruning in Markov decision processes (Boutilier, Brafman, Geib, 1998), while Relevant k is a natural counterpart used for relevance analysis. Both algorithms provide a coherent framework for discussing di erent grades of reachability and relevance based pruning methods that appear in the literature. Our work is motivated by the growing role that forward and ....

.... Koehler, 1997) or both (Kambhampati, Parker, Lambrecht, 1997) The importance of reachability and relevance analysis has been noted in the context of decisiontheoretic planning as well. For example, Boutilier and Dearden (1994) employ relevance analysis to reduce the state space, and Boutilier, Brafman, and Geib (1998) describe a general method for reachability analysis for MDPs. Below, we discuss this method in a simpli ed form suitable for classical planning problems described using the Strips representation language (Fikes Nilsson, 1971) In Section 4, we present a counterpart of this method for performing ....

[Article contains additional citation context not shown here]

Boutilier, C., Brafman, R. I., & Geib, C. (1998). Structured reachability analysis for markov decision processes. In Proc. of 14th Conference on Uncertainty in AI, pp. 24-32.

....that estimates a value for these states. 52 As time permits, the set of neighboring states can be expanded, hopefully increasing solution quality by more accurately evaluating the quality of alternative actions. Some of the ideas underlying Graphplan have been applied to more general MDPs in [19], where the construction of a planning graph is generalized to deal with the stochastic, conditional action representation offered by 2TBNs. The reachability constraints discovered by this process are then used to simplify the action and reward representation of an MDP so that it refers only to ....

....to be solved much more dramatically than either does in isolation. Just as reachability constraints can be used to prune regression paths in deterministic domains, they can be used to prune value function and policy estimates generated by decision theoretic regression and abstraction algorithms [19]. 52 The approximate abstraction techniques described in Section 5.1.3 might be used to generate such heuristic information. 72 5.2.2 Serial Problem Decomposition and Communicating Structure The communicating or reachability structure of an MDP provides a way to formalize different types of ....

Craig Boutilier, Ronen I. Brafman, and Christopher Geib. Structured reachability analysis for Markov decision processes. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 24--32, Madison, WI, 1998.

....the pruning ability of these techniques to that of a class of algorithms for reachability and relevance analysis that operate on the original problem formulation: Reachable k and Relevant k. Reachable k is a simpli ed variant of a similar algorithm for state pruning in Markov decision processes (Boutilier, Brafman, Geib, 1998), while Relevant k is a natural counterpart used for relevance analysis. Both algorithms provide a coherent framework for discussing di erent grades of reachability and relevance based pruning methods that appear in the literature. Our work is motivated by the growing role that forward and ....

....such actions ahead of time, we reduce the space that needs to be searched to nd a valid plan. In principle, full edged reachability analysis requires forward search in the space of possible states. This is a very expensive operation, and instead, we can opt for sound, but incomplete methods (see Boutilier, Brafman, Geib, 1998). Such methods do not discover all the actions that can be ruled out. However, any action that they rule out is infeasible and need not be considered when searching for a plan. The Graphplan planner provides a good example of the utility of approximate reachability analysis. Graphplan has two ....

[Article contains additional citation context not shown here]

Boutilier, C., Brafman, R. I., & Geib, C. (1998). Structured reachability analysis for markov decision processes. In Proc. of 14th Conference on Uncertainty in AI, pp. 24-32.

....31, 37, 47, 56, 72, 71] This should prove possible because the structure assumed by SPI can be exploited in a way that is orthogonal to the types of structure assumed by many other solution methods. One example of this is the integration of abstraction methods like SPI with reachability analysis [10]. 58 SPI and other decision theoretic regression methods need to be tested empirically on more realistic domains. Further testing will give an idea as to the types of problem structure that exist in naturally occurring MDPs. This will also suggest the types of representations (and associated ....

Craig Boutilier, Ronen I. Brafman, and Christopher Geib. Structured reachability analysis for Markov decision processes. In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pages 24--32, Madison, WI, 1998. 59

No context found.

C. Boutilier, R. Brafman, and C. Geib. Structured reachability analysis for Markov decision processes. In Proceedings of UAI'98, 1998.

No context found.

Boutilier, C.; Brafman, R.; and Geib, C. 1998. Structured reachability analysis for Markov Decision Processes. In Proceedings of UAI'98.

No context found.

C. Boutilier, R. I. Brafman, and C. Geib. Structured reachability analysis for markov decision processes. In Proc. of the 14th Conf. on Uncertainty in Artificial Intelligence (UAI98) , 1998.

No context found.

Boutilier, C., Brafman, R., and Geib, C. (1998). Structured reachability analysis for Markov decision processes. In Proc. of UAI, pages 24--32.

No context found.

Boutilier, C., Brafman, R. I., and Geib, C. W. (1998). Structured reachability analysis for markov decision processes. In Proc. of UAI, pages 24--32.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC