Boutilier, Craig and Dearden, Richard 1994. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the Eleventh National Conference on Artificial Intelligence. AAAI. 1016-1022.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....explosion. Abstraction algorithms developed by Boutilier and colleagues can be modeled as converging to constrained forms of structured morphisms assuming various representations of the conditional probability tables when the space of morphisms is defined by Boolean formulae of the features [14], when it is defined by decision trees on the features [15] and when it is defined by first order logic formulae [16] 4 Abstraction in Hierarchical Systems SMDP homomorphisms can readily be employed to model various abstraction schemes in flat MDPs and SMDPs. SMDP homomorphisms are a ....

C. Boutilier and R. Dearden. Using abstractions for decision theoretic planning with time constraints. In Proceedings of the AAAI-94, pages 1016--1022. AAAI, 1994.

....results in a combinatorial explosion. Abstraction algorithms developed by Boutilier and colleagues can be modeled as converging to constrained forms of structured morphisms assuming various representations of the CPTs when the space of morphisms is defined by boolean formulae of the features [Boutilier and Dearden, 1994] , when it is defined by decision trees on the features [Boutilier et al. 1995] and when it is defined by first order logic formulae [Boutilier et al. 2001] 6 Abstraction in Hierarchical Systems In the previous section we showed that SMDP homomorphisms can model various abstraction schemes ....

C. Boutilier and R. Dearden. Using abstractions for decision theoretic planning with time constraints. In Proceedings of the AAAI-94, pages 1016--1022. AAAI, 1994.

....encodes the set of points obtained by applying the cc operator from the leaves up toward the root. Cc trees whose leaves are annotated with intervals are called icc trees. Figure 1 depicts an example of an icc tree. We now discuss the evaluation of the icc operator sum i=1 i w i , where i [0 1] and w i are closed intervals. Note that the set of convex sets of real numbers is exactly the set of intervals. It then follows directly from Theorem 2.2 (cc operator and convex hull operator commute) that i=1 i w i is also an interval. We show that it is a closed interval by showing how ....

....using a greedy approach as follows. We first sort the values u i in descending order, breaking ties arbitrarily. Then proceeding in this order, we try to put into the knapsack as much material of the current category as possible, subjected to two constraints: 7 [1.12 4] 2.4 3] 2 1 6 [1.6 3. 5] [1 3] [2 4] 0 5] 2 .3] 4 .5] 2 .4] 3 .5] 5 .7] 2 .3] 2 4] 4 .5] Figure 1. Example of an icc tree. 1) the weight must be in the constraint interval i and 2) The sum of the already assigned weights must be at most one minus the sum of the lower bounds of the remaining ....

[Article contains additional citation context not shown here]

C. Boutilier and R. Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 1016--1022, Seattle, July 1994.

....SSP partitions. It should be trivially possible to extend those methods to our extended de nitions. 17 It is also possible to extend their results on structured state spaces. We are working on this presently. Dean and Givan [5] show that model reduction algorithms such as state space abstraction [3] and structured policy iteration [4] are special cases of model minimization. These results also hold for our extended de nition. In fact it is possible to show that a larger class of algorithms t into our general framework. We outline one such example next. Zinkevich and Balch [24] de ne ....

Boutilier, C. and Dearden, R. 1994. Using abstractions for decision theoretic planning with time constraints. In Proceedings of the AAAI-94, pp. 1016-1022. AAAI

....We conclude with a discussion on related work and some future directions of research. 2 Notation A Markov Decision Process is a tuple ( A, k, P, R) where is a finite set of states, A is a finite set of actions, kr C x A is the set of admissible state action pairs, P: kr x . 4 [0, 1] is the transition probability function with P(s, a, s ) being the probability of transition from state s to state s under action a, and R: kr .4 is the expected reward function, with R(s, a) being the expected reward for performing action a in state s. We assume that the rewards are bounded. Let ....

....R(s, a) being the expected reward for performing action a in state s. We assume that the rewards are bounded. Let As (a[ s, a) k C A denote the set of actions admissible in state s. We assume that for all s , As is non empty. A stochastic policy r is a mapping from kr to the real interval [0, 1] with EaEA, 7r(S, a) 1 for all s 6 . For any (s, a) 6 k, r(s, a) gives the probability of picking action a in state s. The solution of an MDP is an optimal policy r that uniformly dominates all other possible policies for that MDP. Let B be a partition of a set X. For any x 6 X, x]B denotes ....

[Article contains additional citation context not shown here]

C. Boutilier and R. Dearden. Using abstractions for decision theoretic planning with time constraints. In Proceedings of the AAAI-9J, pages 1016-1022. AAAI, 1994.

....reachability analysis (e.g. Bonet et al. 1997] relevance analysis (e.g. McDermoot, 1996; Nebel et al. 1997] or both [Kambhampati et al. 1997] The importance of reachability and relevance analysis has been noted in the context of decision theoretic planning as well. For example, [Boutilier and Dearden, 1994] employ relevance analysis to reduce the state space, and [Boutilier et al. 1998] describe a general method for reachability analysis for MDPs. Below, we discuss this method in a simplified form suitable for classical planning problems described using the STRIPS representation language [Fikes and ....

C. Boutilier and R. Dearden. Using abstractions for decision theoretic planning with time constraints. In Proc. of AAAI'94, 1994.

....there is an accompanying need for reasoning methods that can operate within limited time bounds. Our work is directed towards incrementally building therapy plans under uncertainty. In particular, we adopt Markov models, which have received increasing attention in the AI planning literature [6,3,2]. Various approximation techniques have been developed for incrementally estimating probabilities of possible outcomes within a belief net [10,16,11] more recent work with Markov models has begun to explore incremental methods for producing policies [6] Markov models naturally represent many of ....

....theoretical elegance and practical applicability. In deterministic approaches to AI planning, incremental algorithms have become widely accepted as the way to achieve reasonable performance under time constraints [7,14,19] Similar ideas have begun to be used for Markov model planning as well [6,3,1], although only for FOMDPs. The approach of Boutilier and Dearden [3] looks for abstractions that can be made automatically, but provides no guarantee of a result in limited time cases. The approaches of Dean et al. [6] and Barto et al. [1] both suffer from a blindness to points in the space outside ....

[Article contains additional citation context not shown here]

C. Boutilier and R. Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of AAAI-94, 1994.

....on the sense value when it is sensed, and having different branches depending on whether the door was open or not. 7 Comparison with Other Representations One of the popular action representations for stochastic actions is probabilistic STRIPS [Kushmerick et al. 1995; Draper et al. 1994; Boutilier and Dearden, 1994; Haddawy et al. 1995] In this section we show that the proposed representation is more concise in the sense that the ICL SC representation will not be (more than a constant factor) larger than then corresponding probabilistic STRIPS representation plus a rule for each predicate, but that ....

R. Boutilier and R. Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proc. 12th National Conference on Artificial Intelligence, pages 1016--1022, Seattle, WA, 1994.

....but can expect to eventually face events and challenges that may lead to real time computational bottlenecks. Research on continual computation comes in the spirit of a body of work, centering on the value of optimizing the performance of reasoning and decision making under limited resources [7,11,22,26,27,29,43,46,48,51,59,63,68]. The work also shares motivations and goals with a variety of related efforts on compilation, precomputation, and prefetching in Computer Science. Policies for guiding the precomputation and caching of complete or partial solutions of potential future problems are targeted at enhancing the ....

C. Boutilier, R. Dearden, Using abstractions for decision-theoretic planning with time constraints, in: Proc. AAAI-94, Seattle, WA, American Association for Artificial Intelligence, AAAI Press, Menlo Park, CA, 1994, pp. 1016--1022.

....unacceptable (as in realtime planning and execution) Thus, there has recently been intense interest in methods for improving the scalability of MDPs. Researchers have proposed techniques based on factoring the transition model of the MDP [Koller and Parr, 2000] exploiting irrelevant variables [Boutilier and Dearden, 1994; Dietterich, 2000] or approximating the model s value function [Bertsekas and Tsitsiklis, 1996] We draw on two other traditions, both based on developing macros, or local plans for accomplishing limited sub goals over restricted, semi autonomous regions of the state space. The first follows ....

.... states; reusing macros between structurally similar regions; approximating the value function over some regions with regression functions, as in neuro dynamic programming [Bertsekas and Tsitsiklis, 1996] or searching only over relevant subsets of the state variables, as do Dietterich [2000] and Boutilier and Dearden [1994] . 7 Conclusions and Future Directions We have demonstrated a method for decomposing large state spaces in two ways: spatially, in which the state space is broken up into semi autonomous regions, and teleologically, in which macros are independently developed to solve each subgoal of the MDP. ....

C. Boutilier and R. Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the Twelfth National Conference on Artificial Intelligence, pages 1016--1022. AAAI Press, 1994.

....plan may be a random guess in the worst case. As time passes, the plan is refined to become more accurate, until the planner is interrupted by the anytime monitoring process for the best result it has computed given planning time available so far. Abstraction planning algorithms such as that in [10] and [31] illustrate the utility of anytime planning. For a variety of problems, the combined anytime abstraction approach is sufficiently fast to produce high quality approximate plans. Additionally, different 29 anytime algorithms can be merged to produce an optimal result given deliberation ....

C. Boutilier and R. Dearden, Using Abstractions for Decision-Theoretic Planning with Time Constraints, in: Proceedings of the Twelfth National Conference on Artificial Intelligence, Seattle, Washington, (1994) 1016-1022.

....functions are frequently used in approximate algorithms for solving MDPs; Lovejoy [12] describes their use in solving partially observable MDPs. Puterman [13] provides an excellent introduction to Markov decision processes and techniques involving bounding value functions. Boutilier and Dearden [5] and Boutilier et al. 7] describe methods for solving implicitly described MDPs and Dean and Givan [8] reinterpret this work in terms of computing explicitly described MDPs with aggregate states. Bounded parameter MDPs allow us to represent uncertainty about or variation in the parameters of a ....

Boutilier, C. and Dearden, R., "Using Abstractions for Decision Theoretic Planning with Time Constraints," Proceedings of AAAI-94, Seattle, Washington, 1994, pp. 1016-1022.

....Loerincs, Ge ner, 1997) relevance analysis (e.g. McDermoot, 1996; Nebel, Dimopoulos, Koehler, 1997) or both (Kambhampati, Parker, Lambrecht, 1997) The importance of reachability and relevance analysis has been noted in the context of decisiontheoretic planning as well. For example, Boutilier and Dearden (1994) employ relevance analysis to reduce the state space, and Boutilier, Brafman, and Geib (1998) describe a general method for reachability analysis for MDPs. Below, we discuss this method in a simpli ed form suitable for classical planning problems described using the Strips representation language ....

Boutilier, C., & Dearden, R. (1994). Using abstractions for decision theoretic planning with time constraints. In Proc. of AAAI'94, pp. 1016-1022.

....formula of LT a given plan will either falsify or satisfy it. 7 The fact that evaluating database queries in relational databases is essentially the same as evaluating logical formulas against finite models is a central theme in database theory [31] 8 Work on reactive plans [6] and policies [18, 54, 15] has concerned itself with on going interactions between the agent and its environment. However, there are still many applications where we only want the agent to accomplish a task that has a finite horizon, in which case plans that are finite sequences of actions can generally suffice. 13 ....

Craig Boutilier and Richard Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the AAAI National Conference, pages 1016--1022, 1994.

....a good approximation of the optimal policy. This can be done using heuristics to compute the policy on the whole environment, or computing only a partial policy, only considering a subset of the state space. Computing an approximation of a complete policy can be done using state aggregation [Boutilier and Dearden1994, Laroche et al..1999b] or environment decomposition [Dean and Lin1995, Precup and Sutton1997, Parr1998] techniques. Computing near optimal policies using these two techniques can considerably reduce computing times. But in many realistic applications, not only the goal state is known, but also the ....

Craig Boutilier and Richard Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the National Conference on Artificial Intelligence, pages 1016-- 1022, 1994.

....new to the planning community, but few architectures guarantee high quality, hard real time response. Planning time limits have been enforced via techniques ranging from anytime (Dean et al. 1993) to design to time (Garvey and Lesser, 1993) A variety of abstraction algorithms such as that from (Boutilier and Dearden, 1994) allow fast, approximate planning so that a restrictive time limit will enable creation of the best plan possible within the available time. However, as domain complexity increases and available planning time decreases, the quality of the planning solution may suffer to the extent that the ....

Boutilier, C. and R. Dearden: 1994, `Using Abstractions for Decision-Theoretic Planning with Time Constraints'. In: Proceedings of the Twelfth National Conference on Artificial Intelligence. pp. 1016--1022.

....hierarchical approaches While the envelope extension method ignores portions of the state space, other techniques have considered abstractions of the state space that try to group together sets of states that behave similarly under the chosen actions of the optimal policy. Boutilier and Dearden (Boutilier Dearden 1994) assume a representation for actions that is similar to that of Buridan (Kushmerick, Hanks, Weld 1994) described earlier and a state utility function that is described in terms of domain literals. They then pick a subset of the literals that accounts for the greatest variation in the state ....

Boutilier, C., and Dearden, R. 1994. Using abstractions for decision-theoretic planning with time constraints. In Proc. Twelfth National Conference on Artificial Intelligence, 1016--1022. AAAI Press.

....bounding value functions are frequently used in approximate algorithms for solving MDPs; Lovejoy [ 1991 ] describes their use in solving partially observable MDPs. Puterman [ 1994 ] provides an excellent introduction to Markov decision processes and techniques involving bounding value functions. Boutilier and Dearden [ 1994 ] and Boutilier et al. 1995b ] describe methods for solving implicitly described MDPs and Dean and Givan [ 1997 ] reinterpret this work in terms of computing explicitly described MDPs with aggregate states. Bounded parameter MDPs allow us to represent uncertainty about or variation in the ....

Boutilier, Craig and Dearden, Richard 1994. Using abstractions for decision theoretic planning with time constraints. In Proceedings AAAI-94. AAAI. 1016--1022.

....when states are grouped together even if they cannot be shown a priori to have the same value. State aggregation can be approached from many different angles, depending upon the notion of similarity that is used to group states together. Boutilier and Dearden use another approach to aggregation (Boutilier Dearden, 1994) in which states are grouped together based upon their relevance to the agent s expected performance in the environment. If, for example, a robot receives no benefit from playing or winning chess, then a chess board in the environment, and the vast number of states it induces, would all be ....

Boutilier, C., & Dearden, R. (1994). Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI-94) Seattle, Washington. AAAI Press.

....scheduling of plans to allow execution timing guarantees for critical planned actions. For temporal aspect 1) I have no intentions of inventing a revolutionary algorithm to reason about deliberation time, particularly since many others are concentrating their research efforts in this area [3], 7] 11] 12] 15] 33] and [35] Instead, I plan to use a combination of design to time [9] and anytime [7] strategies, modifying the planner such that it can dynamically alter planner parameters to control expanded state space size and halt search if time expires. Limiting online planner ....

C. Boutilier and R. Dearden, "Using Abstractions for Decision-Theoretic Planning with Time Constraints," Proceedings of the Twelfth National Conference on Artificial Intelligence, pp. 1016-1022, 1994.

....Loerincs, Ge ner, 1997) relevance analysis (e.g. McDermoot, 1996; Nebel, Dimopoulos, Koehler, 1997) or both (Kambhampati, Parker, Lambrecht, 1997) The importance of reachability and relevance analysis has been noted in the context of decisiontheoretic planning as well. For example, (Boutilier Dearden, 1994) employ relevance analysis to reduce the state space, and (Boutilier et al. 1998) describe a general method for reachability analysis for MDPs. Below, we discuss this method in a simpli ed form suitable for classical planning problems described using the Strips representation language (Fikes ....

Boutilier, C., & Dearden, R. (1994). Using abstractions for decision theoretic planning with time constraints. In Proc. of AAAI'94, pp. 1016-1022.

....are frequently used in approximate algorithms for solving MDPs; Lovejoy [1991] describes their use in solving partially observable MDPs. Puterman [Puterman, 1994] provides an excellent introduction to Markov decision processes and estimation techniques that involve bounding value functions. Boutilier and Dearden [1994] and Boutilier et al. 1995b] describe methods for solving implicitly described MDPs and Dean and Givan [1997] reinterpret this work in terms of computing explicitly described MDPs with aggregate states. Littman [1996] provides a generalization of MDPs that may encompass some aspects of the theory ....

Boutilier, Craig and Dearden, Richard 1994. Using abstractions for decision theoretic planning with time constraints. In Proceedings AAAI-94. AAAI. 1016--1022.

....more apparent. At a conceptual level, most sequential decision problems can be viewed as instances of Markov decision processes (MDPs) and we will use the MDP framework to make the connections explicit. Much recent research on DTP has explicitly adopted the MDP framework as an underlying model [8, 20, 22, 38, 122, 130], allowing the adaptation of existing results and algorithms for solving MDPs (e.g. from the field of operations research) to be applied to planning problems. In doing so, however, this work has departed from the traditional definition of the planning problem in the AI planning community one ....

....than a simple 2TBN representation. Fortunately, if certain variables are affected deterministically, these do not cause the PSO representation to blow up. Furthermore, PSO representations can also be modified to exploit the independence of an action s effects on different state variables [20, 44], thus escaping this combinatorial difficulty. For instance, we might represent the DelC action shown in Figure 17 in the more factored form illustrated in Figure 20 (for simplicity, we only show the effect of the action and the exogenous event ArrM) Much like a 2TBN, we can determine an ....

[Article contains additional citation context not shown here]

Craig Boutilier and Richard Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedingsof the Twelfth National Conferenceon Artificial Intelligence, pages 1016--1022, Seattle, WA, 1994.

No context found.

Boutilier, Craig and Dearden, Richard 1994. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the Eleventh National Conference on Artificial Intelligence. AAAI. 1016-1022.

No context found.

Craig Boutilier and Richard Dearden. Using abstractions for decision-theoretic planning with time constraints. In Proceedings of the AAAI National Conference, pages 1016--1022, 1994.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC