Visual servoing, i.e. the use of the vision sensor in feedback control, has been of increasing interest. A fair amount of work has been done in applications in autonomous driving, manipulation, mobile robot navigation and surveillance. However, the theoretical and analytical aspects of the problem have not received much attention. Furthermore, the problem of estimation from the vision measurements has been considered separately from the design of the control strategies. Instead of addressing the pose estimation and control problems separately, we attempt to characterize the types of control tasks which can be achieved using only quantities directly measurable in the image, bypassing the pose estimation phase. We consider the navigation task for a nonholonomic ground mobile base tracking an arbitrarily shaped continuous ground curve. This tracking problem is formulated as one of controlling the shape of the curve in the image plane. We study the controllability of the system characteriz...

Some issues in markerless tracking of human body motion are addressed. Extended Kalman filters have commonly been applied to kinematic variables, to combine predictions consistent with plausible motion, with the incoming stream of visual measurements. Kalman filtering is applicable only when the underlying distribution is approximately Gaussian. Often, this assumption proves remarkably robust. There are two

This thesis deals with the use of logic-based switching in the control of imprecisely modeled nonlinear systems. Each control system considered consists of a continuous-time dynamical process to be controlled, a family of candidate controllers, and an event-driven switching logic. The need for switching arises when no single candidate controller is capable, by itself, of guaranteeing good performance when connected with a poorly modeled process. In this thesis we develop provably correct switching strategies capable of determining in real-time which candidate controller should be put in feedback with a process so as to achieve a desired closed-loop performance. The resulting closed-loop systems are hybrid in the sense that in each case, continuous dynamics interact with event-driven logic. In the process of designing these switching algorithms, we develop several tools for the analysis and synthesis o...

A number of methods have been proposed in the literature for estimating scenestructure and ego-motion from a sequence of images using dynamical models. Despite the fact that all methods may be derived from a "natural " dynamical model within a unified framework, from an engineering perspective there are a number of trade-offs that lead to different strategies depending upon the applications and the goals one is targeting. We want to characterize and compare the properties of each model such that the engineer may choose the one best suited to the specific application. We analyze the properties of filters derived from each dynamical model under a variety of experimental conditions, assess the accuracy of the estimates, their robustness to measurement noise, sensitivity to initial conditions and visual angle, effects of the bas-relief ambiguity and occlusions, dependence upon the number of image measurements and their sampling rate.

We present an algorithm that performs recursive estimation of ego-motion and ambient structure from a stream of monocular perspective images of a number of feature points. The algorithm is based on an Extended Kalman Filter (EKF) that integrates over time the instantaneous motion and structure measurements computed by a 2-perspective-views step. Key features of our filter are (1) global observability of the model, (2) complete on-line characterization of the uncertainty of the measurements provided by the two-views step. The filter is thus guaranteed to be well-behaved regardless of the particular motion undergone by the observer. Regions of motion space that do not allow recovery of structure (e.g. pure rotation) may be crossed while maintaining good estimates of structure and motion; whenever reliable measurements are available they are exploited. The algorithm works well for arbitrary motions with minimal smoothness assumptions and no ad hoc tuning. Simulations are presented that il...

We have developed a prototype miniaturized active vision system whose sensor architecture is based on a logarithmic structured space-variant pixel geometry. This system integrates a CCD sensor, mini pan-tilt actuator, controller, general purpose processor and display. Due to the ability of space-variant sensors to cover large work-spaces yet provide high acuity with an extremely small number of pixels, space-variant active vision system architectures provide the potential for radical reductions in system size and cost. In this paper, we describe a prototype space-variant active vision system which performs such tasks as connected components analysis and motion tracking. The potential application domains for systems of this type include vision systems for mobile robots and robot manipulators, traffic monitoring systems, security and surveillance, and consumer video communications. Submitted to 11th ICPR specialty conference on Architectures for Vision and Pattern Recognition. Support...

Abstract not found

This thesis establishes a stochastic framework for tracking curves in visual clutter, using a Bayesian random-sampling algorithm. The approach is rooted in ideas from statistics, control theory and computer vision. The problem is to track outlines and features of foreground objects, modelled as curves, as they move in substantial clutter, and to do it at, or close to, video frame-rate. The algorithm, named Condensation, for Conditional density propagation, has recently been derived independently by several researchers, and is generating signi cant interest in the statistics and signal processing communities. This thesis contributes to the literature on Condensation-like lters by presenting some novel applications of and extensions to the basic algorithm, and contributes to the visual motion estimation literature by demonstrating high tracking performance in cluttered environments. Despite its power the Condensation algorithm has a remarkably simple form and this allows the use of non-linear motion models which combine characteristics of discrete Hidden Markov Models with the continuous Auto-Regressive Process motion models traditionally used in Kalman lters. These mixed discrete-continuous models have promising applications to the emerging eld of perception of action. This thesis also implements two algorithms to smooth the output of the Condensation lter which improves the accuracy of motion estimation in a batch-mode procedure after tracking is complete.

The causal estimation of three-dimensional motion from a sequence of two-dimensional images can be posed as a nonlinear filtering problem. We describe the implementation of an algorithm whose uniform observability, minimal realization and stability have been proven analytically in [5]. We discuss a scheme for handling occlusions, drift in the scale factor and tuning of the lter. We also present an extension to partially calibrated camera models and prove its observability. We report the performance of our implementation on a few long sequences of real images. More importantly, however, we have made our real-time implementation -- which runs on a personal computer -- available to the public for first-hand testing.

This article studies the following question: "When is it possible to decide, on the basis of images of point-features observed by an imprecisely modeled two-camera stereo vision system, whether or not a prescribed robot positioning task has been precisely accomplished?" Results are shown for three camera model classes: injective cameras, weakly calibrated projective cameras, and uncalibrated projective cameras. In particular, given a weakly calibrated stereo pair, it is shown that a positioning task can be precisely accomplished if and only if the task specification is invariant to projective transformations. It is shown that injective and uncalibrated projective cameras can accomplish fewer tasks, but are still able to accomplish tasks involving point coincidences. The same formal framework is applied to the problem of determining the set of tasks which can be precisely accomplished with the well-known position-based control architecture. It is shown that, for any class of camera mod...