Computer vision is embracing a new research focus in which the aim is to develop visual skills for robots that allow them to interact with a dynamic, realistic environment. To achieve this aim, new kinds of vision algorithms need to be developed which run in real time and subserve the robot's goals. Two fundamental goals are determin-ing the location of a known object. Color can be successfully used for both tasks. This article demonstrates that color histograms of multicolored objects provide a robust, efficient cue for index-ing into a large database of models. It shows that color histograms are stable object representations in the presence of occlusion and over change in view, and that they can differentiate among a large number of objects. For solving the identification problem, it introduces a technique called Histogram Intersection, which matches model and im-age histograms and a fast incremental version of Histogram Intersection, which allows real-time indexing into a large database of stored models. For solving the location problem it introduces an algorithm called Histogram Backprojection, which performs this task efficiently in crowded scenes. 1

. We describe a novel method of obtaining a fixation point on a moving object for a real-time gaze control system. The method makes use of a real-time implementation of a corner detector and tracker and reconstructs the image position of the desired fixation point from a cluster of corners detected on the object using the affine structure available from two or three views. The method is fast, reliable, viewpoint invariant, and insensitive to occlusion and/or individual corner dropout or reappearance. We compare two- and three-dimensional forms of the algorithm, present results for the method in use with a high performance head/eye platform, and compare the results with two naive fixation methods. Keywords: active vision, head/eye platform, gaze control, tracking, fixation, affine structure 1. Introduction One of the tenets of the active vision paradigm is that fixation over time -- the process of centering and holding gaze on a point in the environment -- can make many problems in co...

. Within the context of active vision, scant attention has been paid to the execution of motion saccades --- rapid re-adjustments of the direction of gaze to attend to moving objects. In this paper we first develop a methodology for, and give real-time demonstrations of, the use of motion detection and segmentation processes to initiate "capture saccades" towards a moving object. The saccade is driven by both position and velocity of the moving target under the assumption of constant target velocity, using prediction to overcome the delay introduced by visual processing. We next demonstrate the use of a first order approximation to the segmented motion field to compute bounds on the time-to-contact in the presence of looming motion. If the bound falls below a safe limit, a "panic saccade" is fired, moving the camera away from the approaching object. We then describe the use of image motion to realize smooth pursuit, tracking using velocity information alone, where the camera is moved...

This paper examines the problem of a moving robot tracking a moving object with its cameras, without requiring the ability to recognize the target to distinguish it from distracting surroundings. A novel aspect of the approach taken is the use of controlled camera movements to simplify the visual processing necessary to keep the cameras locked on the target. A gaze holding system implemented on a robot's binocular head demonstrates this approach. Even while the robot is moving, the cameras are able to track an object that rotates and moves in three dimensions. The central idea is that localizing attention in 3-D space makes precategorical visual processing sufficient to hold gaze. Visual fixation can help separate the target object from distracting surroundings. Converged cameras produce a horopter (surface of zero stereo disparity) in the scene. Binocular features with no disparity can be located with a simple filter, showing the object's location in the image. Similarly, an object th...

this document. I thank my family for the support and encouragement they have always given me. I thank my parents Les and Molly for their love and guidance. I thank my siblings, Jean, Joe, and John, or teaching me to love in the midst o stfie. I thank Robert Dodd or his continual support and encouragement and his calm unbiased eye

Today's robotics applications require complex, real-time, high-bandwidth sensor systems. Although many such systems have been developed, integrating them into an autonomous agent architecture remains an area of active research. We have integrated an active stereo vision system with an autonomous agent architecture using a system of perceptual memory. Perceptual memory is an important class of memory because it is designed for the "behavior-based" portion of the agent's architecture, and not the deliberative portion. This memory maintains current and recent task-dependent perceptual information, as well as expectations about the agent's immediate environment. Our system of perceptual memory is composed of visual primitives from our stereo system, called proximity spaces. Each proximity space represents a virtual fovea or locus of the agent's attention. As an application, we present a robot that uses our system of perceptual memory and proximity spaces to "attend to" multiple humans in a...

Two important problems in camera control are how to keep a moving camera xated on a target point, and how to precisely aim a camera, whose approximate pose is known, towards a given 3D position. This paper describes how electronic image alignment techniques can be used to solve these problems, as well as provide other bene ts such as stabilized video. Hence, stabilized, xated imagery is obtained despite large latencies in the control loop, even for simple control strategies. These techniques have been tested using an airborne camera and real-time a ne image alignment. 1

This paper describes an algorithm for stereo tracking using 3D affine transfer of a body-centred fixation point via the recovery of affine structure. Transfer of fixation and structure is founded on point positions in the image recovered using a corner detector, and matched over time and in stereo. The paper shows how affine transfer can be based on all the available data, not just a minimal basis set, providing further immunity to measurement noise and poor choices of basis corners. Robustness has been built in to the algorithm by enabling transfer to continue by 2D planar transfer and independent transfer if fewer spatio-temporal matches are available. The paper also shows that affine transfer can proceed without explicit calculation of the structure, allowing a fixed-latency tracking feedback loop. The method is implemented at video rates in the visual feedback loop of a four axis active camera platform, and experimental results are shown from stereo tracking of moving objects, with...

This paper describes a new binocular tracking method using Log Polar Mapping (LPM) which approximately represents the mapping of the retina into the visual cortex in primate vision. Using LPM makes it possible not only to obtain both a high central resolution and a wide field of view, but also to significantly reduce processing image data. In this paper, LPM is performed in software by lookup table method. Our tracking method utilizes zero disparity filter (ZDF) for extracting the target object and virtual horopter method for estimating binocular disparities, respectively. The performance of both target extraction and disparity estimation is improved in comparison with the conventional methods, by using LPM. Some experimental results are also shown to demonstrate the effectiveness of the proposed method.

The interest in camera heads has been ever increasing over the past years. This interest in agile sensor systems seems promising for guiding the vision research in new useful directions. At Aalborg University, we have implemented the second and much improved version of a camera head, which specifically has lead to research in how to control the gaze mechanism. We propose a combination of multiple visual cues for controlling the camera head, enabling it to cope with less restricted scenarios than otherwise possible. By integrating accommodation and disparity cues we are able to control the vergence of the camera head separate from the version mechanism, enabling reliable smooth pursuit and gaze shift. 1 Introduction Since the late sixties much research have been concentrated on extraction of robust and accurate 3D information. Despite the intensive efforts put into extraction of 3D information from visual input, no one has so far been able to come up with an algorithm, acquiring the r...