Partially observable Markov decision processes (POMDPs) provide a natural and principled framework to model a wide range of sequential decision making problems under uncertainty. To date, the use of POMDPs in real-world problems has been limited by the poor scalability of existing solution algorithms, which can only solve problems with up to ten thousand states. In fact, the complexity of finding an optimal policy for a finite-horizon discrete POMDP is PSPACE-complete. In practice, two important sources of intractability plague most solution algorithms: large policy spaces and large state spaces. On the other hand,

The ability to reason about action and change has long been considered a necessary component for any intelligent system. Many proposals have been offered in the past to deal with this problem. In this paper, we offer a new approach to belief change associated with performing actions that addresses some of the shortcomings of these approaches. In particular, our approach is based on a well-developed theory of action in the situation calculus extended to deal with belief. Moreover, our account handles nested belief, belief introspection, mistaken belief, and handles belief revision and belief update together with iterated belief change. 1

In (Petrick & Bacchus 2002), a “knowledge-level ” approach to planning under incomplete knowledge and sensing was presented. In comparision with alternate approaches based on representing sets of possible worlds, this higher level representation is richer, but the inferences it supports are weaker. Nevertheless, because of its richer representation, it is able to solve problems that cannot be solved by alternate approaches. In this paper we examine a collection of new techniques for increasing both the representational and inferential power of the knowledge-level approach. These techniques have been fully implemented in the PKS (Planning with Knowledge and Sensing) planning system. Taken together they allow us to solve a range of new types of planning problems under incomplete knowledge and sensing.

Abstract not found

In this article we propose a Probabilistic Situation Calculus logical language to represent and reason with knowledge about dynamical worlds in which actions have uncertain effects. Two essential tasks are addressed when reasoning about change in worlds: Probabilistic Temporal Projection and Probabilistic Belief Update. Uncertain effects are modeled by dividing an action into two subparts: a deterministic input (agent produced) and a probabilistic reaction (nature produced). The probability distributions of the reactions are assumed to be known. Our logical language is an extension to Situation Calculae in the style proposed by Raymond Reiter. There are three aspects to this work. First, we extend the language to accommodate terms dealing with belief and probability. Second, we provide a operational semantics based on Randomly Timed Automata. Finally, we develop Monte-Carlo algorithms to efficiently interpret the probability and belief terms. With the framework proposed we discuss how to develop a reasoning system in Mathematica capable of performing temporal projection and belief update in the Probabilistic Situation Calculus. Finally, we present a sound basis to set rewards and observation planning. (1) Center for Logic and Computation, Departamento de Matematica, IST, Av. Rovisco Pais, 1049-001 Lisboa, Portugal. email: pmat@math.ist.utl.pt. Supported by FCT SFRH/BPD/5625/2001 and the FibLog initiative. (2) Applied Mathematics Center, Departamento de Matematica, IST, Av. Rovisco Pais, 1049-001 Lisboa, Portugal. email: apacheco@math.ist.utl.pt (3) Unfortunately J. Pinto passed away in an accident while this paper was being prepared. Formerly, he was at Bell Labs, Database Systems Research Dept., 600 Mountain Ave., New Jersey 07974, U.S.A. OBSERVATIONS AND THE P...

. The actions of a robot like lifting an object are often best thought of as low-level processes with uncertain outcome. A highlevel robot plan can be seen as a description of a task which combines these processes in an appropriate way and which may involve nondeterminism in order to increase a plan's generality. In a given situation, a robot needs to turn a given plan into an executable program for which it can establish, through some form of projection, that it satisfies a given goal with some probability. In this paper we will show how this can be achieved in a logical framework. In particular, low-level processes are modelled as programs in pGOLOG, a probabilistic variant of the action language GOLOG. High-level plans are like ordinary GOLOG programs except that during projection the names of low-level processes are replaced by their pGOLOG- definitions. 1 Introduction The actions of a robot like lifting an object are often best thought of as low-level processes with uncertain ou...

The Spatial Semantic Hierarchy (SSH) is a set of distinct representations for large scale space, each with its own ontology and each abstracted from the levels below it. At the control level, the agent and its environment are modeled as continuous dynamical systems whose equilibrium points are abstracted to a discrete set of distinctive states. The control laws whose execution defines trajectories linking these states are abstracted to actions, giving a discrete causal graph representation for the state space. The causal graph of states and actions is in turn abstracted to a topological network of places and paths (i.e. the topological map). Local metrical models of places and paths can be built within the framework of the control, causal and topological levels while avoiding problems of global consistency. ...

We present the language PC+ for probabilistic reasoning about actions, which is a generalization of the action language C+ that allows to deal with probabilistic as well as nondeterministic effects of actions. We define a formal semantics of PC+ in terms of probabilistic transitions between sets of states. Using a concept of a history and its belief state, we then show how several important problems in reasoning about actions can be concisely formulated in our formalism.

This paper is directed towards combining Pearl’s structural-model approach to causal reasoning with high-level formalisms for reasoning about actions. More precisely, we present a combination of Pearl’s structural-model approach with Poole’s independent choice logic. We show how probabilistic theories in the independent choice logic can be mapped to probabilistic causal models. This mapping provides the independent choice logic with appealing concepts of causality and explanation from the structural-model approach. We illustrate this along Halpern and Pearl’s sophisticated notions of actual cause, explanation, and partial explanation. Furthermore, this mapping also adds first-order modeling capabilities and explicit actions to the structural-model approach.

An effective method is presented for deriving state knowledge in the presence of sensing actions. It is shown how conditional plans can be inferred with the help of a generalized concept of plan skeletons as search heuristics, which allow the planner to introduce conditional branching points by need.