We provide a method, based on the theory of Markov decision processes, for efficient planning in stochastic domains. Goals are encoded as reward functions, expressing the desirability of each world state; the planner must find a policy (mapping from states to actions) that maximizes future rewards. Standard goals of achievement, as well as goals of maintenance and prioritized combinations of goals, can be specified in this way. An optimal policy can be found using existing methods, but these methods require time at best polynomial in the number of states in the domain, where the number of states is exponential in the number of propositions (or state variables). By using information about the starting state, the reward function, and the transition probabilities of the domain, we restrict the planner's attention to a set of world states that are likely to be encountered in satisfying the goal. Using this restricted set of states, the planner can generate more or less complete ...

: For many years, research in AI plan generation was governed by a number of strong, simplifying assumptions: that the planning agent is omniscient, that its actions are deterministic and instantaneous, that its goals are fixed and categorical, and that its environment is static. More recently, researchers have developed expanded planning algorithms that are not predicated on such assumptions. But changing the way in which plans are formed is only part of what is required when the classical assumptions are abandoned. The demands of dynamic, uncertain environments mean that in addition to being able to form plans---even probabilistic, uncertain plans---agents must be able to effectively manage their plans. In this paper, which is based on a talk given at the 1998 AAAI Fall Symposium on Distributed, Continual Planning, we first identify reasoning tasks that are involved in plan management, including commitment management, environment monitoring, alternative assessment, plan elaboration, ...

. We describe the interface between a real-time resource allocation system with an AI planner in order to create fault-tolerant plans that are guaranteed to execute in hard real-time. The planner specifies the task set and all execution deadlines required to ensure system safety, then the resource allocator schedules these plans off-line to analyze execution platform resource utilization. A new interface module combines information from planning and resource allocation to enforce development of plans feasible for execution during a varietyofinternal system faults. Plans that over-utilize any system resource trigger feedback to the planner, which then searches for an alternate plan. A valid plan for each specified fault, including the nominal no-fault situation, is stored in a plan cache for subsequent real-time execution. We situate this work in the context of CIRCA, the Cooperative Intelligent Real-time Control Architecture, whichfocusesondeveloping and scheduling plans that make hard real-time safety guarantees, and provide an example of an autonomous aircraft agent to illustrate how our planner-resource allocation interface improves CIRCA performance. Keywords: AI architectures, planning, real-time scheduling, fault-tolerance 1.

Many current and potential AI applications are intended to operate in dynamic environments, including those with multiple agents. As a result, standard AI plan-generation technology must be augmented with mechanisms for managing changing information, for focusing attention when multiple events occur, and for coordinating with other planning processes. The DIPART testbed (Distributed, Interactive Planner's Assistant for Real-time Transportation planning) was developed to serve as an experimental platform for analyzing a variety of such mechanisms. In this paper, we present an overview both of the DIPART system and of some of the methods for planning in dynamic environments that we have been investigating using DIPART. Many of these methods derive from theoretical work in real-time AI and in related fields, such as real-time operating systems. Introduction Many current and potential AI applications are intended to operate in dynamic environments, including those with multiple agents. An...

Until recently, techniques for AI plan generation relied on highly restrictive assumptions that were almost always violated in real-world environments; consequently, robot designers adopted reactive architectures and avoided AI planning techniques. Some recent research efforts have focused on obviating such assumptions by developing techniques that enable the generation and execution of plans in dynamic, uncertain environments. In this paper, we discuss one such technique, rationale-based monitoring, originally introduced by Veloso, Pollack, and Cox [9], and describe our use of it in a simple mobile robot environment. We review the original approach, describe how itcanbe adapted for a causal-link planner, and provide experimental results demonstrating that it can lead to improved plans without consuming excessive overhead. We also describe our use of rationale-based monitoring in a mobile robot o ce-assistant project currently in progress.

Abstract not found

In this work we extend the work of Dean, Kaelbling, Kirman and Nicholson on planning under time constraints in stochastic domains to handle more complicated scheduling problems. In scheduling problems the sources of complexity stem not only from large state spaces but from large action spaces as well. In these problems it is no longer tractable to compute optimal policies for restricted state spaces via policy iteration. We, instead, borrow from Operations Research in applying bottleneck-centered scheduling heuristics to improve initial policies and make use of Monte Carlo simulation for selectively constructing partial policies in large state spaces. Additionally, we employ a variant of Drummond's situated control rules to constrain the space of possible actions. 1 Introduction In this work we are interested in solving scheduling problems with time constraints in stochastic domains. Stochastic domains imply that there are events outside the system's control. Scheduling in this contex...

my first and most influential teachers. For their encouragement, understanding, and love. ii Acknowledgements I would very much like to thank my thesis advisor Craig Knoblock for the many enjoyable years of mentorship, support, and friendship. Craig has always given me the freedom to explore my own paths towards solving a problem, encouraged me to take chances, while at the same time challenging me to back up my claims and to sometimes consider alternative approaches. Through him, I learned how to read research papers as well as how to write them. His thoughts and advice greatly influenced and improved this thesis. I am extremely grateful for his guidance and I know that it will continue to inspire me as I work with and mentor others.

of " Exploiting Structure for Planning and Control " by Shieu-Hong Lin, Ph.D., Brown University, May 1997. Thesis advisor Thomas L. Dean. Discrete dynamical systems in the form of finite automata or Markov decision processes have been used as a representational and computational foundation for planning under uncertainty. Many AI planning problems can be conveniently viewed as control problems over the underlying discrete dynamical systems. Using AI-style representation, the features of application domains are represented as state variables, and planning problem instances compactly encode very large discrete dynamical systems. The standard algorithms to solve the corresponding control problems require explicit enumeration of the underlying state spaces. This is impractical since the sizes of the state spaces are exponential in the number of state variables. In this thesis, we develop decomposition techniques to exploit structure for planning problems in different application domains. Gi...

We present a flexible procedure for a resource-bounded agent to allocate limited computational resources to on-line problem solving. Our APPROXIMATE AND COMPENSATE methodology extends a well-known greedy time-slicing approach to conditions in which performance profiles may be non-concave and there is uncertainty in the environment and/or problem-solving procedures of an agent. With this method, the agent first approximates problem-solving performance and problem parameters with standard parameterized models. Second, the agent computes a risk-management factor that compensates for the risk inherent in the approximation.