Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219--283, 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....descriptions generated. Furthermore, it assumes that the model is given. In LOMDPs, these assumptions are not made, the algorithms are simpler and traditional model free learning methods apply. The idea of solving large MDP by a reduction to an equivalent, smaller MDP is also discussed e.g. in [Dearden and Boutilier, 1997; Givan et al. 2003; Ravindran and Barto, 2002] However there, only finite MDPs and no relational or first order representations have been inverstigated. Furthermore, there has been great interest in abstraction on other levels than state spaces. Abstraction over time [Sutton et al. 1999] or ....

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

....and the utility function is defined as a realvalued function on the set of final states. In this approach, only what happens at the end of plan execution counts. In the second approach, the utility function is defined on infinite horizon plan execution paths via the use of a timediscounted factor [4] . We choose to define the utility function on execution paths (as opposed to final states) because a) CIRCADIA plans are reactive (as opposed to sequential) and b) only an execution path contains necessary information to compute MG directed and RAG directed subutilities. We choose to restrict ....

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 1:219-- 283, 1997.

....applications one will probably need heuristics and approximation methods to arrive at feasible solutions. Decision theoretic settings such as ours have been tackled in the past using MDP solving methods. Several general exact and approximate computational methods for MDPs [ Howard, 1960; Daerden and Boutilier, 1997; Boutilier et al. 2000; Wiering, 1999 ] have been proposed. In fact MDPs and MDP solving methods are very popular due to their generality. Our work can be seen as closely related to that on computing approximate solutions for MDPs. However unlike most work with MDPs, we consider a speci c ....

....MDPs. However unlike most work with MDPs, we consider a speci c problem: that of robot surveillance in an of ce environment. Standard MDP methods would fail to solve our problem due to its sheer size [ Massios, 2000a ] Further MDP methods that exploit structure in the state transitions like [ Daerden and Boutilier, 1997; Boutilier et al. 2000 ] to reduce the problem complexity, would fail because not enough structure is available in the state transitions of our problem. However, we believe that if the geometrical structure of the environment is exploited and if we are prepared to approximate then good results ....

R. Daerden and C. Boutilier. Abstraction and approximate decisiontheoretic planning. Articial Intelligence, 89(1):219{ 283, 1997.

....we address in our work. There has been extensive research in AI in recent years on solving stochastic planning problems with large action and state spaces and variety of techniques for reducing the complexity (typically exponential in the number of components) of these problems have been proposed [3, 7, 6, 5, 9, 12, 2, 11]. However, all these works assume a fixed structure and a fixed parameterization of the planning problem. The unique aspect of our planning problem is that the underlying topology characterizing the problem can vary and it is itself subject to random changes and fluctuations (due to failures) The ....

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219--283, 1997.

....generates an approximate optimal trading policy with volume and storage constrains. There has been extensive research in AI in recent years on solving MDPs with large state spaces exploiting specific problem structures, in particular through factoring and decompositions [Boutilier et al. 1995; Dearden and Boutilier, 1997; Dean and Lin, 1995; Meuleau et al. 1998] However, all these work assume finite or at least discrete state space. Our solution relies on structure analysis of the problem combined with a Monte Carlo approximation technique. We show that when prices follow a mean reverting process, the optimal ....

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219--283, 1997.

....function using a linear factored approximation. This ability, in turn, depends on the choice of the basis functions and on the properties of the domain. The algorithm currently requires the designer to specify the factored basis functions. This is a limitation compared to other algorithms (e.g. [Dearden and Boutilier, 1997] ) which are fully automated. However, our experiments indicate that a few simple rules are often quite successful in designing a basis. First, we ensure that the reward function is representable by our basis. A simple basis that, in addition, contains a separate set of indicators for each ....

....to extend our algorithms to utilize other types of structure in the representation. One interesting direction involves factored action models, where multiple actions are taken simultaneously. Another involves the use of asymmetries (context specificity) in the value function, as in the work of Dearden and Boutilier [1997] , providing a complementary source of structure to the factorization used in our work. Acknowledgments: We are very grateful to Dirk Ormoneit and Uri Lerner for many useful discussions. This work was supported by the ONR under the MURI program Decision Making Under Uncertainty and by the Sloan ....

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

....explicitly in the controller. Our controllers are unclocked (although they meet timing constraints) making them less powerful but easier to synthesize; representing time explicitly in the controller can lead to a state space explosion. Recent research based on Markov Decision Processes (e.g. (Dearden Boutilier 1997)) differs from ours in considering uncertainty to be more important than timing. For the control problems that interest us, a simple representation of uncertainty is adequate, while correct handling of timing is critical. Conclusions and Future Work We argue that, for many autonomous hybrid ....

Dearden, R., and Boutilier, C. 1997. Abstraction and approximate decision-theoretic planning. Artificial Intelligence 89(1--2):219--283.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artif. Intell., 89:219--283, 1997.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Art. Intel., 89:219--283, 1997.

....reduce the size of the description of an MDP. We will describe the details of this in Section 2.5.1. There has been some previous work in using structured world representa tions with MDPs for planning. As we shall see in Section 2. 6, both Nicholson and Kaelbling [80] and Dearden and Boutilier [31, 33] suggest using Bayesian Networks or similar representations for MDPs. Both these approaches improve performance by producing approximately optimal policies through the use of abstract MDPs. In Section 3.1, we will describe a method that uses a structured representation to pro duce an optimal ....

....away. The algorithm is intended to work hand in hand with the Plexus algorithm described above, building an envelope to reduce the size of the state space, and then swiftly finding a policy for the envelope by abstracting away some of the detail. Dearden and Boutilier Dearden and Boutilier [33, 31] take a somewhat different approach to finding good abstractions, and again produce an approximately optimal policy. Although they represent the MDP using probabilistic STRIPS rules [42, 66] rather than 2TBNs, the representational power is similar, and we will explain their approach here using ....

[Article contains additional citation context not shown here]

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219 283, 1997.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artif. Intel., 89:219--283, 1997.

.... methods do this by aggregating a set of states and treating the states within any aggregate state as if they were identical [3] Within AI, abstraction techniques have been widely studied as a form of aggregation, where states are (implicitly) grouped by ignoring certain problem variables [14, 7, 12]. These methods automatically generate abstract MDPs by exploiting structured representations, such as probabilistic STRIPS rules [16] or dynamic Bayesian network (DBN) representations of actions [13, 7] In this paper, we describe a dynamic abstraction method for solving MDPs using algebraic ....

....for pre action variables g : since the process is fully observable, we need only use the DBN to predict state transitions. We require one DBN for each action J . In order to illustrate our representation and algorithm, we introduce a simple adaptation of a process planning problem taken from [14]. The example involves a factory agent which has the task of connecting two objects and . Figure 2(a) illustrates our representation for the action bolt, where the two parts are bolted together. We see that whether the parts are successfully connected, depends on a number of factors, ....

[Article contains additional citation context not shown here]

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artif. Intel., 89:219--283, 1997.

....(MDPs) Structured algorithms allow problems to be solved without explicit state space enumeration by aggregating states of identical value. Structured approaches using decision trees have been applied to classical dynamic programming (DP) algorithms such as value iteration and policy iteration [7, 3]. Recently, Hoey et.al. 8] have shown that significant computational advantages can be obtained by using an Algebraic Decision Diagram (ADD) representation [1, 4, 5] Notwithstanding such advances, large MDPs must often be solved approximately. This can be accomplished by reducing the level of ....

....provides support for manipulation of ADDs. Experimental results from this section were all obtained using one processor on a dualprocessor Pentium II PC running at 400Mhz with 0.5Gb of RAM. Our approximation methods were tested on various adaptations of a process planning problem taken from [7, 8]. 4.1 Approximation All experiments in this section were performed on problem domains where the variable ordering was the one selected implicitly by the constructors of the domains. Value time iter nodes leaves yU Function ( s) int) Optimal 0 270.91 44 22170 527 0.0 1 562.35 44 ....

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219--283, 1997.

No context found.

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89:219--283, 1997.

No context found.

Dearden, R., and C. Boutilier. 1997. Abstraction and approximate decision theoretic planning. Arti cial Intelligence 89:219 283.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

No context found.

Dearden, R. and C. Boutilier (1997). Abstraction and approximate decision theoretic planning. Artificial Intelligence 89 (1), 219--283.

No context found.

R. Dearden and C. Boutilier, Abstraction and approximate decision theoretic planning. Unpublished manuscript.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

No context found.

R. Dearden and C. Boutilier, "Abstraction and approximate decision theoretic planning," Artificial Intelligence, vol. 89, no. 1, pp. 219--283, 1997.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 89(1):219--283, 1997.

No context found.

Richard Dearden and Craig Boutilier. Abstraction and approximate decision theoretic planning. Arti cial Intelligence, 1997.

No context found.

R. Dearden and C. Boutilier. Abstraction and approximate decision theoretic planning. Artificial Intelligence, 1:219-- 283, 1997.

No context found.

R. Dearden and C. Boutilier, "Abstraction and Approximate Decision Theoretic Planning," Artificial Intelligence, vol. 89, 1997, pp. 219--283.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC