Spatiotemporal energy models for the perception of motion. (1985)

by E H Adelson, J R Bergen
Venue:In Proceedings of the Workshop on Motion: Representation and Analysis
Add To MetaCart
Results 1 - 10 of 904

Performance of optical flow techniques

by J. L. Barron, D. J. Fleet, S. S. Beauchemin - INTERNATIONAL JOURNAL OF COMPUTER VISION , 1994
"... While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, ..."
Abstract - Cited by 1325 (32 self) - Add to MetaCart
While different optical flow techniques continue to appear, there has been a lack of quantitative evaluation of existing methods. For a common set of real and synthetic image sequences, we report the results of a number of regularly cited optical flow techniques, including instances of differential, matching, energy-based and phase-based methods. Our comparisons are primarily empirical, and concentrate on the accuracy, reliability and density of the velocity measurements; they show that performance can differ significantly among the techniques we implemented.
(Show Context)

The Design and Use of Steerable Filters

by William T. Freeman, Edward H. Adelson - IEEE Transactions on Pattern Analysis and Machine Intelligence , 1991
"... Oriented filters are useful in many early vision and image processing tasks. One often needs to apply the same filter, rotated to different angles under adaptive control, or wishes to calculate the filter response at various orientations. We present an efficient architecture to synthesize filters of ..."
Abstract - Cited by 1089 (11 self) - Add to MetaCart
Oriented filters are useful in many early vision and image processing tasks. One often needs to apply the same filter, rotated to different angles under adaptive control, or wishes to calculate the filter response at various orientations. We present an efficient architecture to synthesize filters of arbitrary orientations from linear combinations of basis filters, allowing one to adaptively "steer" a filter to any orientation, and to determine analytically the filter output as a function of orientation.

The Plenoptic Function and the Elements of Early Vision

by Edward H. Adelson, James R. Bergen - Computational Models of Visual Processing , 1991
"... experiment. Electrophysiologists have described neurons in striate cortex that are selectively sensitive to certain visual properties; for reviews, see Hubel (1988) and DeValois and DeValois (1988). Psychophysicists have inferred the existence of channels that are tuned for certain visual properties ..."
Abstract - Cited by 565 (4 self) - Add to MetaCart
experiment. Electrophysiologists have described neurons in striate cortex that are selectively sensitive to certain visual properties; for reviews, see Hubel (1988) and DeValois and DeValois (1988). Psychophysicists have inferred the existence of channels that are tuned for certain visual properties; for reviews, see Graham (1989), Olzak and Thomas (1986), Pokorny and Smith (1986), and Watson (1986). Researchers in perception have found aspects of visual stimuli that are processed pre-attentively (Beck, 1966; Bergen & Julesz, 1983; Julesz & Bergen, Motion Color Binocular disparity Retinal processing Early vision Memory Higher-level vision Etc... Retina More processing Still more processing Orientation Fig.1.1 A generic diagram for visual processing. In this approach, early vision consists of a set of parallel pathways, each analyzing some particular aspect of the visual stimulus. 1983; Treisman, 1986; Treisman & Gelade, 1980). And in computational

Spectral grouping using the Nyström method

by Charless Fowlkes, Serge Belongie, Fan Chung, Jitendra Malik - IEEE Transactions on Pattern Analysis and Machine Intelligence , 2004
"... Spectral graph theoretic methods have recently shown great promise for the problem of image segmentation. However, due to the computational demands of these approaches, applications to large problems such as spatiotemporal data and high resolution imagery have been slow to appear. The contribution ..."
Abstract - Cited by 316 (1 self) - Add to MetaCart
Spectral graph theoretic methods have recently shown great promise for the problem of image segmentation. However, due to the computational demands of these approaches, applications to large problems such as spatiotemporal data and high resolution imagery have been slow to appear. The contribution of this paper is a method that substantially reduces the computational requirements of grouping algorithms based on spectral partitioning making it feasible to apply them to very large grouping problems. Our approach is based on a technique for the numerical solution of eigenfunction problems knownas the Nyström method. This method allows one to extrapolate the complete grouping solution using only a small number of "typical" samples. In doing so, we leverage the fact that there are far fewer coherent groups in a scene than pixels.
(Show Context)

The Computation of Optical Flow

by S.S. Beauchemin, J.L. Barron , 1995
"... Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-ordered images allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image dis ..."
Abstract - Cited by 295 (10 self) - Add to MetaCart
Two-dimensional image motion is the projection of the three-dimensional motion of objects, relative to a visual sensor, onto its image plane. Sequences of time-ordered images allow the estimation of projected two-dimensional image motion as either instantaneous image velocities or discrete image displacements. These are usually called the optical flow field or the image velocity field. Provided that optical flow is a reliable approximation to two-dimensional image motion, it may then be used to recover the three-dimensional motion of the visual sensor (to within a scale factor) and the three-dimensional surface structure (shape or relative depth) through assumptions concerning the structure of the optical flow field, the three-dimensional environment and the motion of the sensor. Optical flow may also be used to perform motion detection, object segmentation, time-to-collision and focus of expansion calculations, motion compensated encoding and stereo disparity measurement. We investiga...
(Show Context)

Epipolarplane image analysis: An approach to determining structure from motion

by Robert C. Bolles, H. Harlyn Baker, David H. Marimont - INTERN..1. COMPUTER VISION , 1987
"... We present a technique for building a three-dimensional description of a static scene from a dense sequence of images. These images are taken in such rapid succession that they form a solid block of data in which the temporal continuity from image to image is approximately equal to the spatial conti ..."
Abstract - Cited by 253 (3 self) - Add to MetaCart
We present a technique for building a three-dimensional description of a static scene from a dense sequence of images. These images are taken in such rapid succession that they form a solid block of data in which the temporal continuity from image to image is approximately equal to the spatial continuity in an individual image. The technique utilizes knowledge of the camera motion to form and analyze slices of this solid. These slices directly encode not only the three-dimensional positions of objects, but also such spatiotemporal events as the occlusion of one object by another. For straight-line camera motions, these slices have a simple linear structure that makes them easier to analyze. The analysis computes the threedimensional positions of object features, marks occlusion boundaries on the objects, and builds a threedimensional map of "free space." In our article, we first describe the application of this technique to a simple camera motion, and then show how projective duality is used to extend the analysis to a wider class of camera motions and object types that include curved and moving objects.
(Show Context)

A Multi-body Factorization Method for Independently Moving Objects

by João Costeira, Takeo Kanade - INTERNATIONAL JOURNAL OF COMPUTER VISION , 1997
"... ... In this paper we present & new method for separating and recovering the motion and shape of multiple independently moving objects in sequence of images. The method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at th ..."
Abstract - Cited by 244 (8 self) - Add to MetaCart
... In this paper we present & new method for separating and recovering the motion and shape of multiple independently moving objects in sequence of images. The method does not require prior knowledge of the number of objects, nor is dependent on any grouping of features into an object at the image level. For this purpose, we introduce a mathematical construct of object shapes, called the shape interaction matrix, which is invariant to both the object motions and the selection of coordinate systems. This invariant structure is computable solely from the observed trajectories of image features without grouping them into individual objects. Once the

A biologically inspired system for action recognition

by H. Jhuang, T. Serre, L. Wolf, T. Poggio - In ICCV , 2007
"... We present a biologically-motivated system for the recognition of actions from video sequences. The approach builds on recent work on object recognition based on hierarchical feedforward architectures [25, 16, 20] and extends a neurobiological model of motion processing in the visual cortex [10]. Th ..."
Abstract - Cited by 238 (15 self) - Add to MetaCart
We present a biologically-motivated system for the recognition of actions from video sequences. The approach builds on recent work on object recognition based on hierarchical feedforward architectures [25, 16, 20] and extends a neurobiological model of motion processing in the visual cortex [10]. The system consists of a hierarchy of spatio-temporal feature detectors of increasing complexity: an input sequence is first analyzed by an array of motiondirection sensitive units which, through a hierarchy of processing stages, lead to position-invariant spatio-temporal feature detectors. We experiment with different types of motion-direction sensitive units as well as different system architectures. As in [16], we find that sparse features in intermediate stages outperform dense ones and that using a simple feature selection approach leads to an efficient system that performs better with far fewer features. We test the approach on different publicly available action datasets, in all cases achieving the highest results reported to date. 1.
(Show Context)

Bayesian Modeling of Uncertainty in Low-Level Vision

by Richard Szeliski , 1990
"... The need for error modeling, multisensor fusion, and robust algorithms i becoming increasingly recognized in computer vision. Bayesian modeling is a powerful, practical, and general framework for meeting these requirements. This article develops a Bayesian model for describing and manipulating the d ..."
Abstract - Cited by 204 (17 self) - Add to MetaCart
The need for error modeling, multisensor fusion, and robust algorithms i becoming increasingly recognized in computer vision. Bayesian modeling is a powerful, practical, and general framework for meeting these requirements. This article develops a Bayesian model for describing and manipulating the dense fields, such as depth maps, associated with low-level computer vision. Our model consists of three components: a prior model, a sensor model, and a posterior model. The prior model captures a priori information about he structure of the field. We construct this model using the smoothness constraints from regularization to define a Markov Random Field. The sensor model describes the behavior and noise characteristics of our measurement system. We develop a number of sensor models for both sparse and dense measurements. The posterior model combines the information from the prior and sensor models using Bayes ' rule. We show how to compute optimal estimates from the posterior model and also how to compute the uncertainty (variance) in these estimates. To demonstrate the utility of our Bayesian framework, we present three examples of its application to real vision problems. The first application is the on-line extraction of depth from motion. Using a two-dimensional generalization of the Kalman filter, we develop an incremental algorithm that provides a dense on-line estimate of depth whose accuracy improves over time. In the second application, we use a Bayesian model to determine observer motion from sparse depth (range) measurements. In the third application, we use the Bayesian interpretation f regularization to choose the optimal smoothing parameter for interpolation. The uncertainty modeling techniques that we develop, and the utility of these techniques invarious applications, support our claim that Bayesian modeling is a powerful and practical framework for low-level vision.

Motion Segmentation and Tracking Using Normalized Cuts

by Jianbo Shi, Jitendra Malik , 1998
"... We propose a motion segmentation algorithm that aims to break a scene into its most prominent moving groups. A weighted graph is constructed on the ira. age sequence by connecting pixels that arc in the spatio-temporal neighborhood of each other. At each pizel, we define motion profile vectors which ..."
Abstract - Cited by 179 (6 self) - Add to MetaCart
We propose a motion segmentation algorithm that aims to break a scene into its most prominent moving groups. A weighted graph is constructed on the ira. age sequence by connecting pixels that arc in the spatio-temporal neighborhood of each other. At each pizel, we define motion profile vectors which capture the probability distribution of the image veloczty. The distance between motion profiles is used to assign a weight on the graph edges. 5rsmg normalized cuts we find the most salient partitions of the spatiotemporaI graph formed by the image sequence. For swmenting long image sequences,' we have developed a recursire update procedure that incorporates knowledge of segmentation in previous frames for efficiently finding the group correspondence in the new frame.
(Show Context)