Ann E. Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In AAAI Spring Symposium on Decision Theoretic Planning, pages 190 196, Stanford, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....Networks do in order to greatly reduce the size of the description of an MDP. We will describe the details of this in Section 2.5.1. There has been some previous work in using structured world representa tions with MDPs for planning. As we shall see in Section 2. 6, both Nicholson and Kaelbling [80] and Dearden and Boutilier [31, 33] suggest using Bayesian Networks or similar representations for MDPs. Both these approaches improve performance by producing approximately optimal policies through the use of abstract MDPs. In Section 3.1, we will describe a method that uses a structured ....

....mechanism. Ideally we would like an abstraction that is non uniform and adaptive. This allows the algorithm to concentrate its computation in parts of the state space that need it the most, and allows it to discover those parts as it runs. Nicholson and Kaelbling Nicholson and Kaelbling [80] represent MDPs using the 2TBN representation we described in Section 2.5. They leave out features of this MDP to produce a sequence of abstract MDPs that range from quite similar to the original MDP to being a very coarse grained approximation of it. The idea is that a rough policy for the ....

Ann E. Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In AAAI Spring Symposium on Decision Theoretic Planning, pages 190 196, Stanford, 1994.

.... MDPs [56] is there some way of extending this theory to general MDPs Or is it possible to show that the problem is somehow inherently intractable There are representations for rewards and transitions that make it possible to specify compact models for MDPs with exponential size state spaces [87, 21, 24, 113]. What are the complexity issues It is probably computationally intractable 48 to find e optimal policies using compact representations, but are there useful subclasses of MDPs that can be solved efficiently This question is explored by Boutilier, Dean, and Hanks [23] The dual ....

Ann Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, Stanford, California, 1994. 257

....in the manner of multi attribute utility theory [28] and to sum the individual rewards of each proposition satisfied by s to determine R(s) This would provide a very direct encoding of the reward function we adopt in this example. A related action representation uses two stage Bayes nets [16, 38, 10], in which each action is modeled with a Bayesian network with two slices or sets of variables. The first slice represents the values of (possibly multi valued) variables before the action is performed while the second slice represents the value after the action. Arcs in the diagram represent ....

Ann E. Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In AAAI Spring Symposium on Decision Theoretic Planning, pages 190--196, Stanford, 1994.

.... : ym i) Pr(y 1 ja; x 1 ; Delta Delta Delta ; x n ) Y i 1 Pr(y i ja; x 1 ; Delta Delta Delta ; x n ; y 1 ; Delta Delta Delta ; y i Gamma1 ) Even with a compact representation of the dynamics of the entire world, we will rarely want or need to work with the whole model [5, 22]. Given different goals, different time constraints, or different current world states, we might want to take very different views of the world. One way to construct different world views is by specifying only a subset of the possible attributes in the complete world model. In some cases, these ....

Nicholson, Ann and Kaelbling, Leslie Pack, Toward Approximate Planning in Very Large Stochastic Domains, Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, Stanford, California, 1994.

....complex fuzzy controllers from user provided behaviors schemas. We intend to go one step further, by automatically generating fuzzy policies. For long [Sacerdoti, 1974] it has been suggested that abstraction is a useful concept for reducing the complexity of real world problems. Recently, [Nicholson et al. 1994] have proposed an interesting abstraction based framework for coping with very large stochastic domains. Abstract world views (derivations from an MDP which are MDPs themselves, and which consider only a subset of the possible attributes in the complete world model) are constructed by projecting ....

A. E. Nicholson and L. P. Kaelbling. Towards Approximate Planning in Very Large Stochastic Domains, In Proceedings AAAI Spring Symposium on Decision Theoretic Planning (1994).

....be bounded. Discounting can be incorporated in such a model to further reduce the number of relevant atoms; essentially, effects from a distance can be given less weight. A crucial feature of this extension is the fact that abstractions are generated reasonably quickly. Nicholson and Kaelbling [10] have proposed abstracting state spaces in a similar fashion using sensitivity analysis to determine relevant variables; however, such a method has high computational cost. A key problem is the adaptation of our method to different action and utility representations (e.g. using causal networks, ....

....techniques. However, there are certain technical difficulties associated with nonuniform clusters. We hope to investigate the features of both the envelope and abstraction methods and determine to which types of domains each is best suited and how the intuitions of both might be combined (see [10]) Features that will ensure the success of our technique include: a propositional domain representation; approximately additive utilities over features; a wide range of utilities; goals with possible minor improvements, and so on. The extent to which real domains possess these qualities is ....

Ann E. Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In AAAI Spring Symposium on Decision Theoretic Planning, pages 190--196, Stanford, 1994.

....[1991] and Lin and Dean [1994] for planning and temporal reasoning in deterministic environments. There is a long history of using abstractions in both artificial intelligence and disciplines such as adaptive control that deal with Markov decision processes. Davidson and Fehling [1994] and Nicholson and Kaelbling [1994] provide examples of approaches that analyze factored state transition models to generate abstract representations that serve to expedite planning and decision making. Haddawy and Suwandi [1994] construct abstract models by organizing actions in a hierarchy of abstractions. For a survey of ....

Nicholson, Ann and Kaelbling, Leslie Pack, 1994, Toward approximate planning in very large stochastic domains, In Proceedings of the AAAI Spring Symposium on Decision-Theoretic Planning.

....the sizes of the underlying state spaces usually grow exponentially in the sizes of the problem instances, which makes the naive approach infeasible. To cope with this computational barrier, researchers have explored different alternatives such as heuristic algorithms without quality guarantee [NK94] DKKN93b] DKKN93a] DKKN95] approximation algorithms applicable under certain circumstances [BD94] TVR96] and exact algorithms aggregating states on the fly in improving computational efficiency [BDG95] BCn89] In this thesis, we present exact decomposition techniques for planning and ....

Ann Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, 1994.

....specific form, taking advantage of utility independence and probabilistic independence in action effects. Much recent research has focussed on using representations for MDPs that make some of this structure explicit and automatically discovering appropriate problem abstractions and decompositions [9, 6, 14, 11, 5]. The extent to which effective Markov task decompositions can be automatically extracted from suitable problem representations remains an interesting open question. ....

A. E. Nicholson and L. P. Kaelbling. Toward approximate planning in very large stochastic domains. AAAI Spring Symp. on Decision Theoretic Planning, pp.190-- 196, Stanford, 1994.

....the state space has structure, however, the planner ought to be able to do better by taking advantage of this structure to focus the computation and improve the approximations. One possibility is to make the reduced state space (or envelope) E the cartesian product of a subset of these dimensions [8, 14]. Excluding a dimension from E in this way is called uniform abstraction or simply abstraction . It is also possible to exclude different dimensions from different parts E this is called non uniform abstraction , and is the focus of the research presented in this paper. We assume an agent ....

A. E. Nicholson and L. P. Kaelbling. Toward approximate planning in very large stochastic domains. In Proc. of Spring Symposium on Decision Theoretic Planning, 1994.

....may be a dozen doors, each of which may be at least open or closed, increasing the size of the domain several thousand fold compared to an openplan layout without doors. Planning in S quickly becomes intractable. One possibility is to make E the cartesian product of a subset of these dimensions [8, 15]. Excluding a dimension from the envelope E in this way is called uniform abstraction or simply abstraction , and the more dimensions are excluded, the more abstract E is. It is also possible to exclude different 1 See [17] for an example of such a specific sequence. dimensions from ....

Ann Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, 1994.

....states. But to handle the thousands of states needed to address realistic problems, other techniques will be needed. Other approaches to scaling up, including various kinds of factoring and decomposition of the transitions and belief states (e.g. the sort of approach Boutilier et al. 1995) and Nicholson and Kaelbling (1994) used in fully observable domains) may be able to be used in concert with techniques described in this paper to yield practical results in moderately large pomdp problems. A Detailed Problem Descriptions For each of the pomdp problems presented in this report, we give a short description, its ....

Nicholson, A. and Kaelbling, L. P. (1994). Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, Stanford, California.

....Even with a compact representation of the dynamics of the entire world, we will rarely want or need to work with the whole model. Given different goals, different time constraints, or different current world states, we might want to take very different views of the world. Nicholson and Kaelbling [ Nicholson and Kaelbling, 1994 ] investigate the construction of different world views by specifying only a subset of the possible variables in the complete world model. In some cases, these abstract views capture all of the world dynamics relevant to the problem at hand. In other cases, they will serve as tractable ....

Nicholson, Ann and Kaelbling, Leslie Pack 1994. Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, Stanford, California.

....states. But to handle the thousands of states needed to address realistic problems, other techniques will be needed. Other approaches to scaling up, including various kinds of factoring and decomposition of the transitions and belief states (e.g. the sort of approach Boutilier et al. 1995) and Nicholson and Kaelbling (1994) used in fully observable domains) may be able to be used in concert with techniques described in this paper to yield practical results in moderately large pomdp problems. ....

Nicholson, A. and Kaelbling, L. P. (1994). Toward approximate planning in very large stochastic domains.

No context found.

Ann Nicholson and Leslie Pack Kaelbling. Toward approximate planning in very large stochastic domains. In Proceedings of the AAAI Spring Symposium on Decision Theoretic Planning, Stanford, California, 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC