A snake is an energy-minimizing spline guided by external constraint forces and influenced by image forces that pull it toward features such as lines and edges. Snakes are active contour models: they lock onto nearby edges, localizing them accurately. Scale-space continuation can be used to enlarge the cap-ture region surrounding a feature. Snakes provide a unified account of a number of visual problems, in-cluding detection of edges, lines, and subjective contours; motion tracking; and stereo matching. We have used snakes successfully for interactive interpretation, in which user-imposed constraint forces guide the snake near features of interest.

Binocular stereo is the process of obtaining depth information from a pair of cameras. In the past, stereo algorithms have had problems at occlusions and have tended to fail there (though sometimes post-processing has been added to mitigate the worst effects). We show that, on the contrary, occlusions can help stereo computation by providing cues for depth discontinuities. We describe a theory for stereo based on the Bayesian approach, using adaptive windows and a prior weak smoothness constraint, which incorporates occlusion. Our model assumes that a disparity discontinuity, along the epipolar line, in one eye always corresponds to an occluded region in the other eye thus, leading to an occlusion constraint. This constraint restricts the space of possible disparity values, thereby simplifying the computations. An estimation of the disparity at occluded features is also discussed in light of psychophysical experiments. Using dynamic programming we can find the optimal solution to our s...

Abstract. Binocular stereo is the process of obtaining depth information from a pair of left and right views of a scene. We present a new approach to compute the disparity map by solving a global optimization problem that models occlusions, discontinuities, and epipolar-line interactions. In the model, geometric constraints require every disparity discontinuity along the epipolar lineinoneeyetoalways correspond to an occluded region in the other eye, while at the same time encouraging smoothness across epipolar lines. Smoothing coefficients are adjusted according to the edge and junction information. For some well-defined set of optimization functions, we can map the optimization problem to a maximum-flow problem on a directed graph in a novel way, which enables us to obtain a global solution in a polynomial time. Experiments confirm the validity of this approach. 1

We present two extensions to the Space Carving framework. The first is a progressive scheme to better reconstruct surfaces lacking sufficient textures. The second is a novel photo-consistency measure that is valid for both specular and diffuse surfaces, under unknown lighting conditions. 1

We have developed a model for the representation of stereo disparity by a population of neurons that is based on tuning curves similar in shape to those measured physiologically (Poggio and Fischer, 1977). Signal detection analysis was applied to the model to generate predictions of depth dis-crimination thresholds. Agreement between the model and human psychophysical data was possible in this model only when the population size representing disparity in a small patch of visual field was in the range of about 20-200 units. Interval encoding and rate encoding were found to be in-consistent with these data. Psychophysical data on stereo interpolation (Westheimer, 1988a) suggest that there are short-range excitatory and long-range inhibitory interactions between disparity-tuned units at nearby spatial locations. We extended our population model of disparity coding at a

This paper presents a quasi-dense matching algorithm between images based on match propagation principle. The algorithm starts from a set of sparse seed matches, then propagates to the neighboring pixels by the best- rst strategy, and produces a quasidense disparity map. The quasi-dense matching aims at broad modeling and visualization applications which rely heavily on matching information. Our algorithm is robust to initial sparse match outliers due to the best- rst strategy; It is ecient in time and space as it is only output sensitive; It handles half-occluded areas because of the simultaneous enforcement of newly introduced discrete 2D gradient disparity limit and the uniqueness constraint. The properties of the algorithm are discussed and empirically demonstrated.

Brute-force dense matching is usually not satisfactory because the same search range is used for the entire image, yielding potentially many false matches. In this paper, we propose a progressive scheme for stereo matching which uses two fundamental concepts: the disparity gradient limit principle and the least commitment strategy. The first states that the disparity should vary smoothly almost everywhere, and the disparity gradient should not exceed a certain limit. The second states that we should first select only the most reliable matches and therefore postpone unreliable decisions until enough confidence is accumulated. Our technique starts with a few reliable point matches obtained automatically via feature correspondence or through user input. New matches are progressively added during an iterative matching process. At each stage, the current reliable matches constrain the search range for their neighbors according to the disparity gradient limit, thereby reducing potential matching ambiguities of those neighbors. Only unambiguous matches are selected and added to the set of reliable matches in accordance with the least commitment strategy. In addition, a correlation match measure that allows rotation of the match template is used to provide a more robust estimate. The entire process is cast within a Bayesian inference framework. Experimental results illustrate the robustness of our proposed dense stereo matching approach.

Abstract. The lack of eye contact in desktop video teleconferencing substantially reduces the effectiveness of video contents. While expensive and bulky hardware is available on the market to correct eye gaze, researchers have been trying to provide a practical software-based solution to bring video-teleconferencing one step closer to the mass market. This paper presents a novel approach that is based on stereo analysis combined with rich domain knowledge (a personalized face model). This marriage is mutually beneficial. The personalized face model greatly improved the accuracy and robustness of the stereo analysis by substantially reducing the search range; the stereo techniques, using both feature matching and template matching, allow us to extract 3D information of objects other than the face and to determine the head pose in a much more reliable way than if only one camera is used. Thus we enjoy the versatility of stereo techniques without suffering from their vulnerability. By emphasizing a 3D description of the scene on the face part, we synthesize virtual views that maintain eye contact using graphics hardware. Our current system is able to generate an eye-gaze corrected video stream at about 5 frames per second on a commodity PC. Keywords: Stereoscopic vision, Eye-gaze correction, Structure from motion. 1

Through vision, we derive a rich understanding... This article reviews some computational studies of vision, focusing on edge detection, binocular stereo, motion analysis, intermediate vision and object recognition.

AbstractÐWe present an integrated approach to the derivation of scene descriptions from a pair of stereo images, where the steps of feature correspondence and surface reconstruction are addressed within the same framework. Special attention is given to the development of a methodology with general applicability. In order to handle the issues of noise, lack of image features, surface discontinuities, and regions visible in one image only, we adopt a tensor representation for the data and introduce a robust computational technique called tensor voting for information propagation. The key contributions of this paper are twofold: First, we introduce ªsaliencyº instead of correlation scores as the criterion to determine the correctness of matches and the integration of feature matching and structure extraction. Second, our tensor representation and voting as a tool enables us to perform the complex computations associated with the formulation of the stereo problem in three dimensions at a reasonable computational cost. We illustrate the steps on an example, then provide results on both random dot stereograms and real stereo pairs, all processed with the same parameter set. Index TermsÐBinocular stereo, tensor voting, perceptual grouping, surface inference. 1