In 1960, R.E. Kalman published his famous paper describing a recursive solution to the discrete-data linear filtering problem. Since that time, due in large part to advances in digital computing, the Kalman filter has been the subject of extensive research and application, particularly in the area of autonomous or assisted navigation. The Kalman filter is a set of mathematical equations that provides an efficient computational (recursive) means to estimate the state of a process, in a way that minimizes the mean of the squared error. The filter is very powerful in several aspects: it supports estimations of past, present, and even future states, and it can do so even when the precise nature of the modeled system is unknown. The purpose of this paper is to provide a practical introduction to the discrete Kalman filter. This introduction includes a description and some discussion of the basic discrete Kalman filter, a derivation, description and some discussion of the extended Kalman filter, and a relatively simple (tangible) example with real numbers & results.

In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable mdps (pomdps). We then outline a novel algorithm for solving pomdps offline and show how, in some cases, a finite-memory controller can be extracted from the solution to a pomdp. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions.

Mobile robot localization is the problem of determining a robot's pose from sensor data. This article presents a family of probabilistic localization algorithms known as Monte Carlo Localization (MCL). MCL algorithms represent a robot's belief by a set of weighted hypotheses (samples), which approximate the posterior under a common Bayesian formulation of the localization problem. Building on the basic MCL algorithm, this article develops a more robust algorithm called MixtureMCL, which integrates two complimentary ways of generating samples in the estimation. To apply this algorithm to mobile robots equipped with range finders, a kernel density tree is learned that permits fast sampling. Systematic empirical results illustrate the robustness and computational efficiency of the approach.

Schultz, Wolfram. Predictive reward signal of dopamine neurons. is called rewards, which elicit and reinforce approach behav-J. Neurophysiol. 80: 1–27, 1998. The effects of lesions, receptor ior. The functions of rewards were developed further during blocking, electrical self-stimulation, and drugs of abuse suggest the evolution of higher mammals to support more sophistithat midbrain dopamine systems are involved in processing reward cated forms of individual and social behavior. Thus biologiinformation and learning approach behavior. Most dopamine neucal and cognitive needs define the nature of rewards, and rons show phasic activations after primary liquid and food rewards and conditioned, reward-predicting visual and auditory stimuli. the availability of rewards determines some of the basic They show biphasic, activation-depression responses after stimuli parameters of the subject’s life conditions. that resemble reward-predicting stimuli or are novel or particularly Rewards come in various physical forms, are highly variable salient. However, only few phasic activations follow aversive stim-in time and depend on the particular environment of the subject. uli. Thus dopamine neurons label environmental stimuli with appe- Despite their importance, rewards do not influence the brain titive value, predict and detect rewards and signal alerting and motivating events. By failing to discriminate between different

Scientists need new tools to explore and browse large collections of scholarly literature. Thanks to organizations such as JSTOR, which scan and index the original bound archives of many journals, modern scientists can search digital libraries spanning hundreds of years. A scientist, suddenly

Abstract—An algorithm for tracking multiple targets In a cluttered algorithms. Clustering is the process of dividing the entire environment Is developed. The algorithm Is capable of Initiating tracks, set of targets and measurements into independent groups accounting for false or m[~clngreports, and groom sets of dSPeU&IIt ~or clusters ~ Instead of solvin one lar e roblem a reports~As each measurement Is received, probabilities are ~*uI~ted for ~ g g ~‘ thehypotheses that the measurement ~ f ~ ~ IUM)Wfl ~ number of smaller problems are solved in parallel. Fiin a target file, or from a new target, or that the measurement Is false. nally, it is desirable for an algorithm to be recursive so Target states are estimated from each such da*a-as.soclatloo hypothesis that all the previous data do not have to be reprocessed using a 1C~InlQnfilter. As mere measurements are received, the probabill- whenever a new data set is received. ties of Joint hypotheses are calculated recursively using all available The a! onthm can easurements fro inforv~~Hnsuch as density of wiknown targets, density of false ~ g use m rn two ~.~erprobability of ietectlon, ami location ~rtainty. mis ~iciiing tecii- cut generic types of sensors. The first type is capable of nique allows correlation of a asurement with Its a based on sending information which can be used to infer the numsubsequent, as well as previous, data. To keep the number of hypotheses ber of targets within the area of coverage of the sensor reasonable, unlikely hypotheses areeliminated andhypotheses with similar Radar is an example of this type of sensor Tius type of target estimates are combined. To mlnhnlie CO put5tlOnal ~ sensor enerates a data set consistin of

A nonlinear differential equation of the Riccati type is derived for the covariance matrix of the optimal filtering error. The solution of this "variance equation " completely specifies the optimal filter for either finite or infinite smoothing intervals and stationary or nonstationary statistics. The variance equation is closely related to the Hamiltonian (canonical) differential equations of the calculus of variations. Analytic solutions are available in some cases. The significance of the variance equation is illustrated by examples which duplicate, simplify, or extend earlier results in this field. The Duality Principle relating stochastic estimation and deterministic control problems plays an important role in the proof of theoretical results. In several examples, the estimation problem and its dual are discussed side-by-side. Properties of the variance equation are of great interest in the theory of adaptive systems. Some aspects of this are considered briefly.

Important inference problems in statistical physics, computer vision, error-correcting coding theory, and artificial intelligence can all be reformulated as the computation of marginal probabilities on factor graphs. The belief propagation (BP) algorithm is an efficient way to solve these problems that is exact when the factor graph is a tree, but only approximate when the factor graph has cycles. We show that BP fixed points correspond to the stationary points of the Bethe approximation of the free energy for a factor graph. We explain how to obtain regionbased free energy approximations that improve the Bethe approximation, and corresponding generalized belief propagation (GBP) algorithms. We emphasize the conditions a free energy approximation must satisfy in order to be a “valid ” or “maxent-normal ” approximation. We describe the relationship between four different methods that can be used to generate valid approximations: the “Bethe method, ” the “junction graph method, ” the “cluster variation method, ” and the “region graph method.” Finally, we explain how to tell whether a region-based approximation, and its corresponding GBP algorithm, is likely to be accurate, and describe empirical results showing that GBP can significantly outperform BP.

The extended Kalman filter (EKF) is probably the most widely used estimation algorithm for nonlinear systems. However, more than 35 years of experience in the estimation community has shown that is difficult to implement, difficult to tune, and only reliable for systems that are almost linear on the time scale of the updates. Many of these difficulties arise from its use of linearization. To overcome this limitation, the unscented transformation (UT) was developed as a method to propagate mean and covariance information through nonlinear transformations. It is more accurate, easier to implement, and uses the same order of calculations as linearization. This paper reviews the motivation, development, use, and implications of the UT.

. This paper addresses the problem of building large-scale geometric maps of indoor environments with mobile robots. It poses the map building problem as a constrained, probabilistic maximum-likelihood estimation problem. It then devises a practical algorithm for generating the most likely map from data, alog with the most likely path taken by the robot. Experimental results in cyclic environments of size up to 80 by 25 meter illustrate the appropriateness of the approach. Keywords: Bayes rule, expectation maximization, mobile robots, navigation, localization, mapping, maximum likelihood estimation, positioning, probabilistic reasoning 1. Introduction Over the last two decades or so, the problem of acquiring maps in indoor environments has received considerable attention in the mobile robotics community. The problem of map building is the problem of determining the location of entities-of-interest (such as: landmarks, obstacles), often relative to a global frame of reference (such as ...