Abstract Partially observable Markov decision processes (POMDPs) are a principled mathematical framework for planning under uncertainty, a crucial capability for reliable operation of autonomous robots. By using probabilistic sampling, pointbased POMDP solvers have drastically improved the speed of POMDP planning, enabling POMDPs to handle moderately complex robotic tasks. However, robot motion planning tasks with long time horizons remain a severe obstacle for even the fastest point-based POMDP solvers today. This paper proposes Milestone Guided Sampling (MiGS), a new point-based POMDP solver, which exploits state space information to reduce the effective planning horizon. MiGS samples a set of points, called milestones, from a robot’s state space, uses them to construct a compact, sampled representation of the state space, and then uses this representation of the state space to guide sampling in the belief space. This strategy reduces the effective planning horizon, while still capturing the essential features of the belief space with a small number of sampled points. Preliminary results are very promising. We tested MiGS in simulation on several difficult POMDPs modeling distinct robotic tasks with long time horizons; they are impossible with the fastest point-based POMDP solvers today. MiGS solved them in a few minutes. 1

Partially observable Markov decision processes (POMDPs) have been successfully applied to various robot motion planning tasks under uncertainty. However, most existing POMDP algorithms assume a discrete state space, while the natural state space of a robot is often continuous. This paper presents Monte Carlo Value Iteration (MCVI) for continuous-state POMDPs. MCVI samples both a robot’s state space and the corresponding belief space, and avoids inefficient a priori discretization of the state space as a grid. Both theoretical results and preliminary experimental results indicate that MCVI is a promising new approach for robot motion planning under uncertainty.

Partially observable Markov decision processes (POMDPs) provide a principled, general framework for robot motion planning in uncertain and dynamic environments. They have been applied to various robotic tasks. However, solving POMDPs exactly is computationally intractable. A major challenge is to scale up POMDP algorithms for complex robotic tasks. Robotic systems often have mixed observability: even when a robot’s state is not fully observable, some components of the state may still be so. We use a factored model to represent separately the fully and partially observable components of a robot’s state and derive a compact lower-dimensional representation of its belief space. This factored representation can be combined with any point-based algorithm to compute approximate POMDP solutions. Experimental results show that on standard test problems, our approach improves the performance of a leading point-based POMDP algorithm by many times. 1

Before unmanned aircraft can fly safely in civil airspace, robust airborne collision avoidance systems must be developed. Instead of hand-crafting a collision avoidance algorithm for every combination of sensor and aircraft configuration, we investigate the automatic generation of collision avoidance algorithms given models of aircraft dynamics, sensor performance, and intruder behavior. By formulating the problem of collision avoidance as a Markov Decision Process (MDP) for sensors that provide precise localization of the intruder aircraft, or a Partially Observable Markov Decision Process (POMDP) for sensors that have positional uncertainty or limited field-of-view constraints, generic MDP/POMDP solvers can be used to generate avoidance strategies that optimize a cost function that balances flight-plan deviation with collision. Experimental results demonstrate the suitability of such an approach using four different sensor modalities and a parametric aircraft performance model. I.

In the last two decades, there has been an enormous effort to deploy a network of autonomous mobile platforms in various scenarios related to military as well as civilian applications. Interesting research problems related to security range from the development of secure communication protocols for a network of autonomous mobile agents to the development of novel deployment algorithms for a group of mobile agents trying to secure a network or an area from malicious intruders. In this thesis, we investigate the interaction between the mobile agents and an intruder in the environment or the communication network. In contradistinction to the previous research in this area, we model the intrusion as a pursuit-evasion game in continuous time and space. We model the intruder as an antagonistic agent and apply tools from differential game theory in order to obtain the optimal motion strategies for the agents to track the intruder as well as evade intrusion.

In the last two decades, there has been an enormous effort to deploy a network of autonomous mobile platforms in various scenarios related to military as well as civilian applications. Interesting research problems related to security range from the development of secure communication protocols for a network of autonomous mobile agents to the development of novel deployment algorithms for a group of mobile agents trying to secure a network or an area from malicious intruders. In this thesis, we investigate the interaction between the mobile agents and an intruder in the environment or the communication network. In contradistinction to the previous research in this area, we model the intrusion as a pursuit-evasion game in continuous time and space. We model the intruder as an antagonistic agent and apply tools from differential game theory in order to obtain the optimal motion strategies for the agents to track the intruder as well as evade intrusion.

Abstract — In this paper, we present a game theoretic analysis of a visibility based pursuit-evasion game in an environment containing obstacles. The pursuer and the evader are holonomic having bounded speeds. Both players have a complete map of the environment. Both players have omnidirectional vision and have knowledge about each other’s current position as long as they are visible to each other. Under this information structure, the pursuer wants to maintain visibility of the evader for maximum possible time and the evader wants to escape the pursuer’s sight as soon as possible. We present strategies for the players that are in Nash Equilibrium. The strategies are a function of the value of the game. Using these strategies, we construct a value function by integrating the retrogressive path equations backward in time from the termination situations provided by the corners in the environment. From these value functions we recompute the control strategies for them to obtain optimal trajectories for the players near the termination situation. I.