S. Koenig and R. Simmons, "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models," in Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press, 1998, pp. 91 -- 122.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....as a POMDP [85, 63] The interest in MDPs has motivated considerable research in this area with more emphasis on the computational aspects (algorithms and computational complexity) 18, 49, 76] than previously. Recent applications of POMDPs in AI include stochastic control and navigation problems [71, 72], medical diagnosis and therapy [53, 120] and control of attention in a machine vision system [28] 1.2 The Scope of the Thesis We investigate the computational aspects of in nite horizon fully observable and partially observable MDP problems using (computational) complexity theory. Complexity ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable Markov decision process model. In Arti cial intelligence based Mobile robotics: case studies of successful robot systems. MIT Press, Cambridge MA, 1997.

....of its problem domain, and use this model for localization, state tracking, and planning. As of recently, the use of probabilistic models and decision theoretic planning and state tracking methods is becoming increasingly popular, and has resulted in several successful mobile robot architectures [5, 2]. Consequently, our choice of representation has also been a probabilistic model. The practical downside to providing a full and detailed world model to the robot is the significant expense of time and effort on the part of a human designer. Machine learning algorithms for acquiring autonomously ....

....of most sensors used on practical robotic systems. In spite of the high uncertainty present in the operation of mobile robots, a number of algorithms exist, which allow successful localization, state tracking, and planning based on probabilistic models of the robot and its environment [2, 5]. These algorithms maintain a belief distribution ###### over all possible states and update it by means of the transition and emission matrices of a probabilistic model in a well known prediction estimation loop. During prediction, the agent estimates how the belief state at time ### will change ....

[Article contains additional citation context not shown here]

Sven Koenig and Reid Simmons. Xavier: a robot navigation architecture based on partially observable Markov decision process models. In D. Kortenkamp, R.P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots, pages 91--122. MIT Press, Cambridge, 1998.

....D s in equation 8 due to the large upper body of the robot) Preliminary results suggest that the same performance is displayed as in the implementation described here. There are other successful indoor navigation systems using a topological map described in the literature: for example Xavier [17] and Dervish [18] to name just two. These approaches use assumptions on probabilities of detecting features and progressing to the next node (state) In our approach the robot only roughly knows its position, and also the detection of features is not entirely reliable. However, the dynamic ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91--122. MIT Press, Cambridge, MA, 1998.

....suggest a real time procedure for the robot to navigate through the environment in order to move to the goal. This method is inspired by how hippocampus of a rat operates [ 13 ] A probabilistic approach can be also incorporated into a topological map framework. For example, Koenig and Simmons [5] implemented a POMDP based navigation architecture for a mobile robot. Given a predefmed topological map, the system can estimate the current state while the robot navigates in an indoor environment using this probabilistic method, and it can suggest the most rewarding action to take in order to ....

Koenig, S. and R.G. Simmons, R.G. "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models." Artcial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, MIT Press. 1998.

....of conducting long trials through a largescale real world oce environment. It can also be seen that the competition among the behaviours is able to deal with more complex situations like halfblocked doors. There are other approaches to navigate in large scale environments. Many of them (Xavier [7], for example) need more detailed models of the environment and sophisticated algorithms (e.g. Markov decision process models) to determine an appropriate control action for the robot at all times. An other approach [14] uses, as we do, the superposition of di erent local behaviours (motor ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. MIT Press, Cambridge, MA, 1998.

....it is in than exactly where it is in the corridor) A number of heuristics for mapping belief states to actions provide good performance in robot navigation (e. g, the most likely state (MLS) heuristic assumes the agent is in the state corresponding to the peak of the belief state distribution) [35, 63, 47]. Such heuristics work much better in H POMDPs because they can be applied at multiple levels, and belief states over abstract states usually have lower entropy (Figure 8) For a detailed study of the H POMDP model, as well as its application to robot navigation, see [77] Jonsson and Barto [30] ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....The robot s position is updated based on the actions it executed as well as observations gathered during navigation. The method takes into account various sources of uncertainty, including approximate knowledge of the environment, and actuator and sensor uncertainty [Simmons and Koening, 1995, Koening and Simmons, 1998, Nourbakhsh, 1998] A partially observable Markov decision process (POMDP) model is constructed from topological information about the connectivity of the environment, approximate distance information, plus sensor and actuator characteristics. See section A.1.2, page 203. 190 Conclusions ....

....most probable state. Whenever this is not the case, the agent could change its goal from going to a given destination to gathering relevant information that allows it to localize better. Once the agent is better localized, navigation continues to the destination goal ( Simmons and Koening, 1995, Koening and Simmons, 1998] When the agent s location is represented by a set of possible locations (section A. 1.1) the next action to execute is found by selecting I from the set of possible locations, and generating a deterministic plan from I to the destination. The first action of this plan is executed, ....

S. Koening and R.G. Simmons. XAVIER: A robot navi- gation architecture based on partially observable markov decision process models. In Artificial Intelligence and Mobile Robots, pages 91-122. MIT press, 1998.

....have been expressed in a uni ed framework. This framework comprises a mathematically sound basis, where behaviours are gradually turned on and o on di erent time scales. There are other successful indoor navigation systems using a topological map described in the literature: for example Xavier [7] and Dervish [9] to name just two. These approaches use assumptions on probabilities of detecting features and progressing to the next node (state) In our approach the robot only roughly knows its position, and also the detection of features is not entirely reliable. However, the dynamic ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. MIT Press, Cambridge, MA, 1998.

....assumption is usually referred to as the Markov property. Markov processes have become the mathematical foundation for much current work in reinforcement learning [33] decision theoretic planning [1] information retrieval [7] speech recognition [10] active vision [20] and robot navigation [13]. In this paper, I focus on the abstraction of sequential Markov processes, and present two main strategies for removing irrelevant detail : state aggregation decomposition and temporal abstraction. State decomposition methods typically represent states as collections of factored variables [1] ....

.... policy, which can be found by solving a nonlinear set of equations, one for each state (such as by a successive approximation method called value iteration) s) max a2A(s) r(s; a) X js; a)V (s (1) MDPs have been applied to many real world domains, ranging from robotics [13, 16] to engineering optimization [2, 17, 17] and game playing [36] In many such domains, the model parameters (rewards, transition probabilities) are unknown, and need to be estimated from samples generated by the agent exploring the environment. Q learning was a major advance in direct policy ....

[Article contains additional citation context not shown here]

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....The robot s position is updated based on the actions it executed as well as observations gathered during navigation. The method takes into account various sources of uncertainty, including approximate knowledge of the environment, and actuator and sensor uncertainty [Simmons and Koening, 1995, Koening and Simmons, 1998, Nourbakhsh, 1998] A partially observable Markov decision process (POMDP) model is constructed from topological information about the connectivity of the environment, approximate distance information, plus sensor and actuator characteristics. See section A.1.2, page 203. 190 Conclusions ....

....most probable state. Whenever this is not the case, the agent could change its goal from going to a given destination to gathering relevant information that allows it to localize better. Once the agent is better localized, navigation continues to the destination goal ( Simmons and Koening, 1995, Koening and Simmons, 1998] When the agent s location is represented by a set of possible locations (section A.1.1) the next action to execute is found by selecting l from the set of possible locations, and generating a deterministic plan from l to the destination. The first action of this plan is executed, ....

S. Koening and R.G. Simmons. XAVIER: A robot navigation architecture based on partially observable markov decision process models. In Artificial Intelligence and Mobile Robots, pages 91--122. MIT press, 1998.

....of conducting long trials through a large scale real world oce environment. It can also be seen that the competition among the behaviours is able to deal with more complex situations like half blocked doors. There are other approaches to navigate in large scale environments. Many of them (Xavier [5], for example) need more detailed models of the environment and sophisticated algorithms (e.g. Markov decision process models) to determine an appropriate control action for the robot at all times. An other approach [9] uses, as we do, the superposition of di erent local behaviours (motor ....

S. Koenig and R. G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Arti cial Intelligence and Mobile Robots: Case studies of successful robot systems, chapter 4, pages 91-122. AAAI, Menlo Park, CA, 1998.

....navigation approach is sucient and thus more complex models are unnecessary. It should be noticed that even with the adapted solution we developed, the assumptions made about the environment are less stringent than approaches which rely heavily on walls and junctions being perpendicular ( 11] [4]) Also since navigation is not based on the replication of exact paths the robot is not sensitive to signi cant changes in the topology, forcing longer or shorter return paths. Future work The question of the generality of the approach still remains. To answer it, a larger environment will be ....

Sven Koenig and Reid Simmons. Xavier: a robot navigation architecture based on partially observable markov decision process models. In Kortenkamp et al. [5], pages 91-122.

....and actuator errors (e.g. sonar is prone to numerous specular errors, and odometry is also unreliable due to wheel slippage, uneven floors, etc. We will be using a navigation system based on a probabilistic framework, formally called partially observable Markov decision processes (POMDP s) 4] [13], 21] This framework uses an explicit probabilistic model of actuator and sensor uncertainty, which allows a robot to maintain belief estimates of its location in its environment. The POMDP approach uses a state estimation procedure that takes into account both sensor and actuator uncertainty to ....

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

....in environments with high levels of perceptual aliasing. A variety of methods have been proposed for resolving perceptual ambiguity by incorporating previous location information into the recognition of locations. The most popular is the probabilistic approach known as Markov localisation (e.g. [3, 10, 13, 17], etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13, 17] Here, the robot maintains a probability ....

....known as Markov localisation (e.g. 3, 10, 13, 17] etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13, 17]. Here, the robot maintains a probability distribution over a set of discrete locations. Similarly, possible landmarks and actions are typically de ned according to a set of human de ned categories, e.g. possible landmarks might be doors , junctions , etc. and possible actions might be Go ....

Sven Koenig and Reid Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In 19 D. Kortenkamp, R. Bonnasso, and R. Murphy, editors, Articial Intelligence Based Mobile Robots: Case Studies of Successful Robot Systems. MIT Press, 1998.

.... planning, the state spaces are often large and finding optimal or close tooptimal POMDPs becomes extremely time consuming (Papadimitriou Tsitsiklis 1987) Thus, existing robot systems have so far only been able to use greedy POMDP planning methods that produce extremely suboptimal plans (Koenig Simmons 1998). Our robot architecture, on the other hand, is able to find close to optimal plans. In the following, we first give an example of sensor planning and then give overviews of behavior based robotics and POMDPs using this example. Next, we describe how our robot architecture combines these ideas by ....

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable Markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press. 91--122.

....in environments with high levels of perceptual aliasing. A variety of methods have been proposed for resolving perceptual ambiguity by incorporating previous location information into the recognition of locations. The most popular is the probabilistic approach known as Markov localisation (e.g. [3,10,13,17], etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13,17] Here, the robot maintains a probability ....

....approach known as Markov localisation (e.g. 3,10,13,17] etc. which can be applied to either topological or metric maps. The main paradigm for probabilistic navigation using topological maps is that of Hidden Markov Models, and their extension to Partially Observable Markov Decision Processes [13,17]. Here, the robot maintains a probability distribution over a set of discrete locations. Similarly, possible landmarks and actions are typically defined according to a set of human defined categories, e.g. possible landmarks might be doors , junctions , etc. and possible actions might be Go ....

S. Koenig, R. Simmons, Xavier: A robot navigation architecture based on partially observable Markov decision process models, in: D. Kortenkamp, R. Bonnasso, R. Murphy (Eds.), Artificial Intelligence Based Mobile Robots: Case Studies of Successful Robot Systems, MIT Press, Cambridge, MA, 1998.

.... systems surveyed are: RHINO, by Thrun and his colleagues at CMU and Bonn [93] Spatial Semantic Hierarchy DRAFT: February 18, 2000 47 CARMEL, by Kortenkamp and his colleagues at Michigan [38] DERVISH, by Nourbakhsh and colleagues at Stanford [76] and XAVIER, by Koenig and Simmons at CMU [37]. The SSH and its precursors have both influenced, and been influenced by, these systems. An occupancy grid with a single frame of reference is the primary representation for the spatial structure of the environment for RHINO, CARMEL, and XAVIER. DERVISH uses a purely topological map, using ....

S. Koenig and R. G. Simmons. Xavier: a robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots: Case Studies of Successful Robot Systems, pages 91--122. AAAI Press/The MIT Press, Menlo Park, CA, 1998.

....environment. Both geometric and topological approaches represent only a single pose that is believed to be the current pose of the robot. If this proves to be incorrect, the robot is lost and has to relocalize itself. A navigation system using a partially observable Markov decision process model [21, 22] is able to explicitly account for various forms of uncertainty. Partially observable Markov decision processes integrate topological and approximate metric information, utilizing both action and sensor reports in determining the robot pose. A partially observable Markov decision process ....

....of the world, it is not possible for the agent to purposely delude itself into believing that it is in a good state when it is not. 25 The partially observable Markov decision process planning problem can be transformed in a planning problem for a completely observable Markov decision process [21, 22]. Since the decision maker can never be sure of the state of the partially observable Markov decision process, the set of executable actions A must be the same for every state s. This belief Markov decision process is such that an optimal policy for it will give rise to optimal behavior for the ....

[Article contains additional citation context not shown here]

S. Koenig and R. Simmons, "Xavier: A robot navigation architecture based on partially observable Markov decision process models," in AI-based Mobile Robots: Case-studies of Successful Robot Systems (D. Kortenkamp, P. Bonasso, and R. Murphy, eds.), MIT Press, 1997.

....from one another, which introduces perceptual aliasing and hidden state. Decision theoretic planning (Boutilier et al. 1999) has become the preferred approach to robot control in such situations and has been implemented on several research prototypes of mobile robots (Nourbakhsh, 1998; Koenig Simmons, 1998). A decision theoretic planner is usually provided with a stochastic model of the environment, which reflects the uncertainty in the robot s sensing and actions. Hand crafting such a model is typically time consuming and error prone and might require blueprints and or CAD models of the buildings ....

.... algorithms exist for finding the optimal control policy in POMDPs (Kaelbling et al. 1996) However, since this problem has been proven to be PSPACE hard, we are considering approximate solutions such as assumptive planning and MDP based approaches (Cassandra et al. 1996; Nourbakhsh, 1998; Koenig Simmons, 1998). Assumptive planning is a heuristic strategy proposed by Nourbakhsh (1998) which performs full belief updating of its belief state based on the transition and emission probabilities, but makes several simplifying assumptions when choosing an action. First, it assumes that it is in the most ....

[Article contains additional citation context not shown here]

Koenig, S., & Simmons, R. (1998). Xavier: a robot navigation architecture based on partially observable Markov decision process models. In D. Kortenkamp, R. Bonasso and R. Murphy (Eds.), Artificial intelligence and mobile robots, 91--122. Cambridge: MIT Press.

....return they are robust towards a certain degree of sensor noise and model uncertainty. Unfortunately it is difficult to evaluate exactly what happens in the presence of noise uncertainty. However, recently approaches explicitly dealing with sensing, model, and movement uncertainty have appeared [ Koenig and Simmons, 1998; Simmons and Koenig, 1995; Kaelbling et al. 1996; Thrun et al. 1998; Fox et al. 1999b ] Common to these approaches is that they use a probabilistic formulation to represent and update the pose of the robot which has the advantage of enabling them to handle uncertainty in a natural and ....

....additional pose uncertainty introduced by the move step. This fact stresses the importance of having a sensible movement sensing strategy, i.e. to do active sensing, since moving randomly in general does not bear the promise of gaining evidence efficiently enough. The approaches described in [ Koenig and Simmons, 1998; Simmons and Koenig, 1995; Kaelbling et al. 1996 ] and in [ Thrun et al. 1998; Fox et al. 1999b ] respectively, mainly differ in the chosen discrete pose representation. The former approaches use a coarse, topological representation (cell size approx. 1 meter) whereas the latter ones use a ....

[Article contains additional citation context not shown here]

Sven Koenig and Reid G. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In David Kortenkamp, R. Peter Bonasso, and Robin Murphy, editors, Artificial Intelligence and Mobile Robots, pages 91--122. AAAI Press/The MIT Press, 1998.

....the environment, and a major paradigm shift has been necessary. A framework that can handle stochastic, partially observable worlds with unknown dynamics, is that of partially observable Markov decision processes (POMDP) While there has been considerable work on reasoning and planning with POMDPs [KLC96, Nou98, KS98], there has been relatively little progress on learning their structure from data with the explicit purpose of reasoning and planning with them thereafter. This is exactly the focus of this paper. We apply two algorithms to the problem of learning POMDPs from data the traditional Baum Welch ....

....over all states in S. 2.1 Belief updating At all times, the agent maintains a belief distribution over all possible states it can be in, and updates it according to the actions it takes and the percepts it observes. This process of belief updating follows a twostaged prediction estimation cycle [KS98]. During prediction, the agent estimates how the belief state at time t will change if a particular action a t Gamma1 is performed at time t Gamma 1, based on the belief state before the action was performed and general knowledge about how actions affect states: Bel(s t ) X s t Gamma1 P (s ....

[Article contains additional citation context not shown here]

Sven Koenig and Reid Simmons. Xavier: a robot navigation architecture based on partially observable Markov decision process models. In D. Kortenkamp, R.P. Bonasso, and R. Murphy, editors, Artificial Intelligence and Mobile Robots, pages 91--122. MIT Press, Cambridge, 1998.

.... example, vertical structures such as door posts and poles (Chenavier and Crowley, 1992) large planes (Horn and Schmidt, 1995) geometric beacons (Leonard, 1992) or regions classified by type (e.g. corridor, intersection, doorway, room) aided by assumptions about the geometry of such structures (Koenig and Simmons, 1998). Using specific landmarks often requires the robot to perform special maneuvers in order to locate or recognize these landmarks (Bauer, 1995) In our work, such maneuvers are unnecessary. Because our method uses all available sensor data without the requirement of specific features in the ....

Koenig, S. and Simmons, R., 1998 "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models," Artificial Intelligence and Mobile Robots, Kortenkamp, Bonasso, and Murphy eds., AAAI Press, 1998 p91-122.

No context found.

Koenig, S., and Simmons, R. 1998b. Xavier: A robot navigation architecture based on partially observable Markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press. 91--122.

No context found.

Koenig, S. and Simmons, R.G. 1998b. Xavier: A robot navigation architecture based on partially observable Markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., editors 1998b, Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press. 91--122.

No context found.

S. Koenig and R. Simmons, "Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models," in Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MIT Press, 1998, pp. 91 -- 122.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, R. Bonasso, and R. Murphy, editors, Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, pages 91--122. MIT Press, 1998.

No context found.

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MITPress. 91--122.

No context found.

Koenig, S., and Simmons, R. 1998. Xavier: A robot navigation architecture based on partially observable markov decision process models. In Kortenkamp, D.; Bonasso, R.; and Murphy, R., eds., Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems. MITPress. 91--122.

No context found.

S. Koenig and R. G. Simmons, "Xavier: a robot navigation architecture based on partially observable Markov decision process models," Artificial Intelligence Based Mobile Robotics: Case Studies of Successful Robot Systems, Kortenkamp, D., Bonasso, R., and Murphy, R.(editor), pp. 91--122, 1998.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

No context found.

S. Koenig and R. Simmons. Xavier: A robot navigation architecture based on partially observable markov decision process models. In D. Kortenkamp, P. Bonasso, and Murphy. R., editors, AI-based Mobile Robots: Case-studies of Successful Robot Systems. MIT Press, 1997.

No context found.

Koenig, S., Simmons, R.G.: Xavier: A Robot Navigation Architecture Based on Partially Observable Markov Decision Process Models. In: Kortenkamp, D., Bonasso, R.P., Murphy, R.: Artificial Intelligence and Mobile Robots. AAAI Press/The MIT Press (1998) 91-122

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC