S. Koenig and R. Simmons, "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models," in Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press, 1998, pp. 91 -- 122.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....as a POMDP [85, 63] The interest in MDPs has motivated considerable research in this area with more emphasis on the computational aspects (algorithms and computational complexity) 18, 49, 76] than previously. Recent applications of POMDPs in AI include stochastic control and navigation problems [71, 72], medical diagnosis and therapy [53, 120] and control of attention in a machine vision system [28] 1.2 The Scope of the Thesis We investigate the computational aspects of in nite horizon fully observable and partially observable MDP problems using (computational) complexity theory. Complexity ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable Markov decision process model. In Arti cial intelligence based Mobile robotics: case studies of successful robot systems. MIT Press, Cambridge MA, 1997.

....of its problem domain, and use this model for localization, state tracking, and planning. As of recently, the use of probabilistic models and decision theoretic planning and state tracking methods is becoming increasingly popular, and has resulted in several successful mobile robot architectures [5, 2]. Consequently, our choice of representation has also been a probabilistic model. The practical downside to providing a full and detailed world model to the robot is the significant expense of time and effort on the part of a human designer. Machine learning algorithms for acquiring autonomously ....

....of most sensors used on practical robotic systems. In spite of the high uncertainty present in the operation of mobile robots, a number of algorithms exist, which allow successful localization, state tracking, and planning based on probabilistic models of the robot and its environment [2, 5]. These algorithms maintain a belief distribution ###### over all possible states and update it by means of the transition and emission matrices of a probabilistic model in a well known prediction estimation loop. During prediction, the agent estimates how the belief state at time ### will change ....

[Article contains additional citation context not shown here]

Sven Koenig and Reid Simmons. Xavier: a robot navigation architecture based on partially observable Markov decision process models. In D. Kortenkamp, R.P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots, pages 91--122. MIT Press, Cambridge, 1998.

....D s in equation 8 due to the large upper body of the robot) Preliminary results suggest that the same performance is displayed as in the implementation described here. There are other successful indoor navigation systems using a topological map described in the literature: for example Xavier [17] and Dervish [18] to name just two. These approaches use assumptions on probabilities of detecting features and progressing to the next node (state) In our approach the robot only roughly knows its position, and also the detection of features is not entirely reliable. However, the dynamic ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91--122. MIT Press, Cambridge, MA, 1998.

....suggest a real time procedure for the robot to navigate through the environment in order to move to the goal. This method is inspired by how hippocampus of a rat operates [ 13 ] A probabilistic approach can be also incorporated into a topological map framework. For example, Koenig and Simmons [5] implemented a POMDP based navigation architecture for a mobile robot. Given a predefmed topological map, the system can estimate the current state while the robot navigates in an indoor environment using this probabilistic method, and it can suggest the most rewarding action to take in order to ....

Koenig, S. and R.G. Simmons, R.G. "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models." Artcial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, MIT Press. 1998.

....of conducting long trials through a largescale real world oce environment. It can also be seen that the competition among the behaviours is able to deal with more complex situations like halfblocked doors. There are other approaches to navigate in large scale environments. Many of them (Xavier [7], for example) need more detailed models of the environment and sophisticated algorithms (e.g. Markov decision process models) to determine an appropriate control action for the robot at all times. An other approach [14] uses, as we do, the superposition of di erent local behaviours (motor ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. MIT Press, Cambridge, MA, 1998.

....it is in than exactly where it is in the corridor) A number of heuristics for mapping belief states to actions provide good performance in robot navigation (e. g, the most likely state (MLS) heuristic assumes the agent is in the state corresponding to the peak of the belief state distribution) [35, 63, 47]. Such heuristics work much better in H POMDPs because they can be applied at multiple levels, and belief states over abstract states usually have lower entropy (Figure 8) For a detailed study of the H POMDP model, as well as its application to robot navigation, see [77] Jonsson and Barto [30] ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....The robot s position is updated based on the actions it executed as well as observations gathered during navigation. The method takes into account various sources of uncertainty, including approximate knowledge of the environment, and actuator and sensor uncertainty [Simmons and Koening, 1995, Koening and Simmons, 1998, Nourbakhsh, 1998] A partially observable Markov decision process (POMDP) model is constructed from topological information about the connectivity of the environment, approximate distance information, plus sensor and actuator characteristics. See section A.1.2, page 203. 190 Conclusions ....

....most probable state. Whenever this is not the case, the agent could change its goal from going to a given destination to gathering relevant information that allows it to localize better. Once the agent is better localized, navigation continues to the destination goal ( Simmons and Koening, 1995, Koening and Simmons, 1998] When the agent s location is represented by a set of possible locations (section A. 1.1) the next action to execute is found by selecting I from the set of possible locations, and generating a deterministic plan from I to the destination. The first action of this plan is executed, ....

S. Koening and R.G. Simmons. XAVIER: A robot navi- gation architecture based on partially observable markov decision process models. In Artificial Intelligence and Mobile Robots, pages 91-122. MIT press, 1998.

....have been expressed in a uni ed framework. This framework comprises a mathematically sound basis, where behaviours are gradually turned on and o on di erent time scales. There are other successful indoor navigation systems using a topological map described in the literature: for example Xavier [7] and Dervish [9] to name just two. These approaches use assumptions on probabilities of detecting features and progressing to the next node (state) In our approach the robot only roughly knows its position, and also the detection of features is not entirely reliable. However, the dynamic ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. MIT Press, Cambridge, MA, 1998.

....assumption is usually referred to as the Markov property. Markov processes have become the mathematical foundation for much current work in reinforcement learning [33] decision theoretic planning [1] information retrieval [7] speech recognition [10] active vision [20] and robot navigation [13]. In this paper, I focus on the abstraction of sequential Markov processes, and present two main strategies for removing irrelevant detail : state aggregation decomposition and temporal abstraction. State decomposition methods typically represent states as collections of factored variables [1] ....

.... policy, which can be found by solving a nonlinear set of equations, one for each state (such as by a successive approximation method called value iteration) s) max a2A(s) r(s; a) X js; a)V (s (1) MDPs have been applied to many real world domains, ranging from robotics [13, 16] to engineering optimization [2, 17, 17] and game playing [36] In many such domains, the model parameters (rewards, transition probabilities) are unknown, and need to be estimated from samples generated by the agent exploring the environment. Q learning was a major advance in direct policy ....

[Article contains additional citation context not shown here]

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....The robot s position is updated based on the actions it executed as well as observations gathered during navigation. The method takes into account various sources of uncertainty, including approximate knowledge of the environment, and actuator and sensor uncertainty [Simmons and Koening, 1995, Koening and Simmons, 1998, Nourbakhsh, 1998] A partially observable Markov decision process (POMDP) model is constructed from topological information about the connectivity of the environment, approximate distance information, plus sensor and actuator characteristics. See section A.1.2, page 203. 190 Conclusions ....

....most probable state. Whenever this is not the case, the agent could change its goal from going to a given destination to gathering relevant information that allows it to localize better. Once the agent is better localized, navigation continues to the destination goal ( Simmons and Koening, 1995, Koening and Simmons, 1998] When the agent s location is represented by a set of possible locations (section A.1.1) the next action to execute is found by selecting l from the set of possible locations, and generating a deterministic plan from l to the destination. The first action of this plan is executed, ....

S. Koening and R.G. Simmons. XAVIER: A robot navigation architecture based on partially observable markov decision process models. In Artificial Intelligence and Mobile Robots, pages 91--122. MIT press, 1998.

....of conducting long trials through a large scale real world oce environment. It can also be seen that the competition among the behaviours is able to deal with more complex situations like half blocked doors. There are other approaches to navigate in large scale environments. Many of them (Xavier [5], for example) need more detailed models of the environment and sophisticated algorithms (e.g. Markov decision process models) to determine an appropriate control action for the robot at all times. An other approach [9] uses, as we do, the superposition of di erent local behaviours (motor ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. AAAI, Menlo Park, CA, 1998.

....navigation approach is sucient and thus more complex models are unnecessary. It should be noticed that even with the adapted solution we developed, the assumptions made about the environment are less stringent than approaches which rely heavily on walls and junctions being perpendicular ( 11] [4]) Also since navigation is not based on the replication of exact paths the robot is not sensitive to signi cant changes in the topology, forcing longer or shorter return paths. Future work The question of the generality of the approach still remains. To answer it, a larger environment will be ....

Sven Koenig and Reid Simmons. Xavier: a robot navigation architecture based on partially observable markov decision process models. In Kortenkamp et al. [5], pages 91-122.

....and actuator errors (e.g. sonar is prone to numerous specular errors, and odometry is also unreliable due to wheel slippage, uneven floors, etc. We will be using a navigation system based on a probabilistic framework, formally called partially observable Markov decision processes (POMDP s) 4] [13], 21] This framework uses an explicit probabilistic model of actuator and sensor uncertainty, which allows a robot to maintain belief estimates of its location in its environment. The POMDP approach uses a state estimation procedure that takes into account both sensor and actuator uncertainty to ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....in environments with high levels of perceptual aliasing. A variety of methods have been proposed for resolving perceptual ambiguity by incorporating previous location information into the recognition of locations. The most popular is the probabilistic approach known as Markov localisation (e.g. [3, 10, 13, 17], etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13, 17] Here, the robot maintains a probability ....

....known as Markov localisation (e.g. 3, 10, 13, 17] etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13, 17]. Here, the robot maintains a probability distribution over a set of discrete locations. Similarly, possible landmarks and actions are typically de ned according to a set of human de ned categories, e.g. possible landmarks might be doors , junctions , etc. and possible actions might be Go ....

Sven Koenig and Reid Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In 19 D. Kortenkamp, R. Bonnasso, and R. Murphy, editors, Articial Intelligence Based Mobile Robots: Case Studies of Successful Robot Systems. MIT Press, 1998.

.... planning, the state spaces are often large and finding optimal or close tooptimal POMDPs becomes extremely time consuming (Papadimitriou Tsitsiklis 1987) Thus, existing robot systems have so far only been able to use greedy POMDP planning methods that produce extremely suboptimal plans (Koenig Simmons 1998). Our robot architecture, on the other hand, is able to find close to optimal plans. In the following, we first give an example of sensor planning and then give overviews of behavior based robotics and POMDPs using this example. Next, we describe how our robot architecture combines these ideas by ....

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable Markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press. 91--122.

....in environments with high levels of perceptual aliasing. A variety of methods have been proposed for resolving perceptual ambiguity by incorporating previous location information into the recognition of locations. The most popular is the probabilistic approach known as Markov localisation (e.g. [3,10,13,17], etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13,17] Here, the robot maintains a probability ....

....approach known as Markov localisation (e.g. 3,10,13,17] etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13,17]. Here, the robot maintains a probability distribution over a set of discrete locations. Similarly, possible landmarks and actions are typically defined according to a set of human defined categories, e.g. possible landmarks might be doors , junctions , etc. and possible actions might be Go ....

S. Koenig, R. Simmons, Xavier: A robot navigation architecture based on partially observable Markov decision process models, in: D. Kortenkamp, R. Bonnasso, R. Murphy (Eds.), Artificial Intelligence Based Mobile Robots: Case Studies of Successful Robot Systems, MIT Press, Cambridge, MA, 1998.

.... systems surveyed are: RHINO, by Thrun and his colleagues at CMU and Bonn [93] Spatial Semantic Hierarchy DRAFT: February 18, 2000 47 CARMEL, by Kortenkamp and his colleagues at Michigan [38] DERVISH, by Nourbakhsh and colleagues at Stanford [76] and XAVIER, by Koenig and Simmons at CMU [37]. The SSH and its precursors have both influenced, and been influenced by, these systems. An occupancy grid with a single frame of reference is the primary representation for the spatial structure of the environment for RHINO, CARMEL, and XAVIER. DERVISH uses a purely topological map, using ....

S. Koenig and R. G. Simmons. Xavier: a robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots: Case Studies of Successful Robot Systems, pages 91--122. AAAI Press/The MIT Press, Menlo Park, CA, 1998.

No context found.

S. Koenig and R. Simmons, "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models," in Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press, 1998, pp. 91 -- 122.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. Bonasso, and R. Murphy, editors, Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, pages 91--122. MIT Press, 1998.

No context found.

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MITPress. 91--122.

No context found.

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MITPress. 91--122.

No context found.

S. Koenig and R. G. Simmons, "Xavier: a robot navigation architecture based on partially observable Markov decision process models," Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, Kortenkamp, D., Bonasso, R., and Murphy, R.(editor), pp. 91--122, 1998.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

No context found.

Koenig, S., Simmons, R.G.: Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models. In: Kortenkamp, D., Bonasso, R.P., Murphy, R.: Artificial Intelligence and Mobile Robots. AAAI Press/The MIT Press (1998) 91-122

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC