#### DMCA

## A database and evaluation methodology for optical flow (2007)

### Cached

### Download Links

- [www.cs.brown.edu]
- [www.cs.brown.edu]
- [cs.brown.edu]
- [research.microsoft.com]
- [www.visinf.tu-darmstadt.de]
- [www.cs.brown.edu]
- [www.cs.brown.edu]
- [cs.brown.edu]
- [files.is.tue.mpg.de]
- [scribblethink.org]
- [vision.middlebury.edu]
- [vision.middlebury.edu]
- [www.gris.informatik.tu-darmstadt.de]
- [www.gris.tu-darmstadt.de]
- [www.cs.brown.edu]
- [www.idiom.com]
- [research.microsoft.com]
- [cs.brown.edu]
- [cs.brown.edu]
- [scribblethink.org]
- [cs.brown.edu]
- [www.cs.brown.edu]
- [www.gris.informatik.tu-darmstadt.de]
- [www.gris.tu-darmstadt.de]
- [www.gris.informatik.tu-darmstadt.de]
- [www.visinf.tu-darmstadt.de]
- [vision.middlebury.edu]
- [vision3d.iro.umontreal.ca]
- DBLP

### Other Repositories/Bibliography

Venue: | In Proceedings of the IEEE International Conference on Computer Vision |

Citations: | 403 - 21 self |

### Citations

2848 | An iterative image registration technique with an application in stereo vision.
- Lucas, Kanade
- 1981
(Show Context)
Citation Context ...rt on the database and allow researchers to get a sense of what is good performance on the data. To this end, we compared 5 algorithms: Pyramid LK: An implementation [5] of the Lucas-Kanade algorithm =-=[11]-=- on a pyramid, subsequently refined at Microsoft Research. This implementation performs significantly better than the Lucas-Kanade code in Barron et al. [2]. It is included to give an idea of how the ... |

2367 | Determining optical flow,
- Horn, Schunck
- 1981
(Show Context)
Citation Context ... and (4) both provide one error per pixel, which leads to the question of how these errors are aggregated over the image. A baseline approach is to use an L2 norm as in the Horn and Schunck algorithm =-=[33]-=-: EData = ∑ [ x,y u ∂I ∂I ∂I + v + ∂x ∂y ∂t ] 2 . (5) If Equation (5) is interpreted probabilistically, the use of the L2 norm means that the errors in the Optical Flow Constraint are assumed to be Ga... |

2091 | Fast approximate energy minimization via graph cuts.
- Boykov, Veksler, et al.
- 2001
(Show Context)
Citation Context ...d energy function is reached. 2.4 Discrete Optimization Algorithms A number of recent approaches use discrete optimization algorithms, similar to those employed in stereo matching, such as graph cuts =-=[14]-=- and belief propagation [71]. Discrete optimization methods approximate the continuous space of solutions with a greatly simplified problem. The hope is that this will enable a more thorough and compl... |

1706 |
Robot vision",
- Horn
- 1986
(Show Context)
Citation Context ...l [16], Mitiche and Bouthemy [14], and Stiller and Konrad [23]. Instead we focus here on the evaluation of optical flow algorithms. We must first define what we mean by optical flow. Following Horn’s =-=[10]-=- taxonomy, the motion field is the 2D projection of the 3D motion of surfaces in the world, whereas the optical flow is the apparent motion of the brightness patterns in the image. These two are not a... |

1525 | A taxonomy and evaluation of dense two-frame stereo correspondence algorithms.
- Scharstein, Szeliski, et al.
- 2001
(Show Context)
Citation Context .... 1. Introduction As a subfield of computer vision matures, datasets for quantitatively evaluating algorithms are essential to ensure continued progress. Many areas of computer vision, such as stereo =-=[19]-=-, face recognition [17], and object recognition [8], have challenging datasets to track the progress made by leading algorithms and to stimulate new ideas. Optical flow was actually one of the first a... |

1315 | Performance of optical flow techniques.
- Barron, Fleet, et al.
- 1994
(Show Context)
Citation Context ...ging datasets to track the progress made by leading algorithms and to stimulate new ideas. Optical flow was actually one of the first areas to have such benchmark datasets for quantitative comparison =-=[2]-=-. The field benefited greatly from this study, which led to rapid and measurable progress. When the Barron et al. [2] evaluation first appeared, the state of the art was quite poor (a) First Frame (b)... |

887 | Visual Reconstruction.
- Blake, Zisserman
- 1987
(Show Context)
Citation Context ...ion. Another common cause of nonlinearity is the use of a robust penalty function (see Sections 2.1.2 and 2.2.2). A common approach to improve robustness in this case is Graduated Non-Convexity (GNC) =-=[11, 13]-=-. During GNC, the problem is first converted into a convex approximation that is more easily solved. The energy function is then made incrementally more non-convex and the solution is refined, until t... |

742 | From few to many: Illumination cone models for face recognition under variable lighting and pose", - Georghiades - 2001 |

693 | Lucas-Kanade 20 years on: A unifying framework,
- Baker, Matthews
- 2004
(Show Context)
Citation Context ...om concatenating the horizontal and vertical components of the flow at every pixel. The goal is then to optimize EGlobal with respect to f. The simplest gradient descent algorithm is steepest descent =-=[5]-=-, which takes steps in the direction of the negative gradient − ∂EGlobal . An important question with steepest descent is how big the ∂f step size should be. One approach is to adjust the step size it... |

665 |
The robust estimation of multiple motions: parametric and piecewise-smooth flow fields
- Black, Anandan
- 1996
(Show Context)
Citation Context ... Barron et al. [2]. It is included to give an idea of how the algorithms in [2] perform when implemented to today’s standards. Black and Anandan: We used the authors’ implementation of this algorithm =-=[4]-=- with the default parameters. Bruhn et al.: We implemented this highly regarded algorithm [6] ourselves. We (roughly) reproduced the results obtained by that algorithm on the Yosemite sequence (includ... |

661 | Hierarchical model-based motion estimation.
- Bergen, Anandan, et al.
- 1992
(Show Context)
Citation Context ...r video containing two or more frames. We exclude transparency which requires multiple motions per pixel. We also exclude more global representations of the motion such as parametric motion estimates =-=[9]-=-. Most existing optical flow algorithms pose the problem as the optimization of a global energy function that is the weighted sum of two terms: EGlobal = EData + λEPrior. (1) The first term EData is t... |

623 | The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results
- Everingham, Gool, et al.
(Show Context)
Citation Context ...datasets for quantitatively evaluating algorithms are essential to ensure continued progress. Many areas of computer vision, such as stereo [63], face recognition [28, 55, 68], and object recognition =-=[23, 24]-=-, have challenging datasets to track the progress made by leading algorithms and to stimulate new ideas. Optical flow was actually one of the first areas to have such a benchmark, introduced by Barron... |

525 | A computational framework and an algorithm for the measurement of visual motion. - Anandan - 1989 |

524 | A comparison and evaluation of multi-view stereo reconstruction algorithms
- Seitz, Curless, et al.
- 2006
(Show Context)
Citation Context ...ruth flow fields. The presence of nonrigid or independent motion makes collecting a ground-truth dataset for optical flow far harder than for stereo, say, where structured-light [19] or rangescanning =-=[21]-=- can be used to obtain ground truth. Our solution is to collect four different datasets, each of which satisfies a different subset of the desirable properties above. The combination of these dataset... |

505 | High accuracy optical flow estimation based on a theory for warping, in: - Brox, Bruhn, et al. - 2004 |

451 | Layered depth images.
- Shade, Gortler, et al.
- 1998
(Show Context)
Citation Context ...ver, use both frames to generate the actual intensity values. In all the experiments in this paper t = 0.5. Our algorithm is closely related to previous algorithms for depth-based frame interpolation =-=[67, 85]-=-: 1. Forward-warp the flow u0 to time t to give u1 where: ut(round(x + tu0(x))) = u0(x). (19) 20Backyard frame 0 Backyard frame 1 GT interpolated frame Basketball frame 0 Basketball frame 1 GT interp... |

420 |
Feature-based image metamorphosis
- Beier, Neely
- 1992
(Show Context)
Citation Context ...ion. Let x0 = x − tut(x) and x1 = x + (1 − t)ut(x) denote the locations of the two “source” pixels in the two images. If both pixels are visible, i.e., O0(x0) = 0 and O1(x1) = 0, blend the two images =-=[8]-=-: It(x) = (1 − t)I0(x0) + tI1(x1). (21) Otherwise, only sample the non-occluded image, i.e., set It(x) = I0(x0) if O1(x1) = 1 and vice versa. In order to avoid artifacts near object boundaries, we dil... |

368 |
High-quality video view interpolation using a layered representation
- Zitnick, Kang, et al.
- 2004
(Show Context)
Citation Context ...ver, use both frames to generate the actual intensity values. In all the experiments in this paper t = 0.5. Our algorithm is closely related to previous algorithms for depth-based frame interpolation =-=[67, 85]-=-: 1. Forward-warp the flow u0 to time t to give u1 where: ut(round(x + tu0(x))) = u0(x). (19) 20Backyard frame 0 Backyard frame 1 GT interpolated frame Basketball frame 0 Basketball frame 1 GT interp... |

361 | The CMU pose, illumination, and expression database
- Sim, Baker, et al.
- 2003
(Show Context)
Citation Context ...subfield of computer vision matures, datasets for quantitatively evaluating algorithms are essential to ensure continued progress. Many areas of computer vision, such as stereo [63], face recognition =-=[28, 55, 68]-=-, and object recognition [23, 24], have challenging datasets to track the progress made by leading algorithms and to stimulate new ideas. Optical flow was actually one of the first areas to have such ... |

358 | One-shot learning of object categories.
- Fei-Fei, Fergus, et al.
- 2006
(Show Context)
Citation Context ...atures, datasets for quantitatively evaluating algorithms are essential to ensure continued progress. Many areas of computer vision, such as stereo [19], face recognition [17], and object recognition =-=[8]-=-, have challenging datasets to track the progress made by leading algorithms and to stimulate new ideas. Optical flow was actually one of the first areas to have such benchmark datasets for quantitati... |

357 |
Computation of component image velocity from local phase information
- Fleet, Jepson
- 1990
(Show Context)
Citation Context ...taking the inverse cosine of their dot product. The popularity of this measure is based on the seminal survey by Barron et al. [2], although the measure itself dates to prior work by Fleet and Jepson =-=[9]-=-. The goal of the AE is to provide a relative measure of performance that avoids the “divide by zero” problem for zero flows. Errors in large flows are penalized less in AE than errors in small flows.... |

345 | Stereo matching using belief propagation.
- Sun, Shum, et al.
- 2002
(Show Context)
Citation Context .... 2.4 Discrete Optimization Algorithms A number of recent approaches use discrete optimization algorithms, similar to those employed in stereo matching, such as graph cuts [14] and belief propagation =-=[71]-=-. Discrete optimization methods approximate the continuous space of solutions with a greatly simplified problem. The hope is that this will enable a more thorough and complete search of the state spac... |

303 | High-accuracy stereo depth maps using structured light, in:
- Scharstein, Szeliski
- 2003
(Show Context)
Citation Context ... v1) 2 ] and used in [16]. For image interpolation, we use the (square root of the) Venus Moebius (a) First Frame (b) Ground-Truth Flow Figure 6. We cropped the stereo datasets Venus [19] and Moebius =-=[20]-=- to convert the asymmetric stereo disparity ranges into roughly symmetric flow fields. One important reason for including this dataset was to allow direct comparison with state of the art stereo algor... |

277 |
Determining three-dimensional motion and structure from optical flow generated by several moving objects,
- Adiv
- 1985
(Show Context)
Citation Context ...uthors have explored rigidity or fundamental matrix priors which, in the absence of other evidence, favor flows that are parallel to epipolar lines. These constraints have both been strictly enforced =-=[1, 29, 52]-=- and added as a soft prior [78, 79]. 7 y0 y02.3 Continuous Optimization Algorithms The two most commonly used continuous optimization techniques in optical flow are: (1) gradient descent algorithms (... |

274 |
An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences
- Nagel, Enkelmann
- 1986
(Show Context)
Citation Context ...Smoothness In Equation (10) the weighting function is isotropic, treating all directions equally. A variety of approaches weight the smoothness prior anisotropically. For example, Nagel and Enkelmann =-=[50]-=- and Werlberger et al. [82] weight the direction along the image gradient less than the direction orthogonal to it, and Sun et al. [70] learn a Steerable Random field to define the weighting. Zimmer e... |

224 | Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods
- Bruhn, Weickert, et al.
(Show Context)
Citation Context ...implemented to today’s standards. Black and Anandan: We used the authors’ implementation of this algorithm [4] with the default parameters. Bruhn et al.: We implemented this highly regarded algorithm =-=[6]-=- ourselves. We (roughly) reproduced the results obtained by that algorithm on the Yosemite sequence (included in the results webpage). MediaPlayer TM : As a baseline for interpolation, we obtained res... |

174 | Layered representation for motion analysis.
- Wang, Adelson
- 1993
(Show Context)
Citation Context ...ted above. 2.5.3 Layers Motion transparency has been extensively studied and is not considered in detail here. Most methods have focused on the use of parametric models that estimate motion in layers =-=[34,77]-=-. The regularization of transparent motion in the framework of global energy minimization, however, has received little attention with the exception of [36, 81]. 2.5.4 Sparse-to-Dense Approaches The c... |

171 | Smoothness in layers: Motion segmentation using nonparametric mixture estimation
- Weiss
- 1997
(Show Context)
Citation Context ...ic models that estimate motion in layers [34,77]. The regularization of transparent motion in the framework of global energy minimization, however, has received little attention with the exception of =-=[36, 81]-=-. 2.5.4 Sparse-to-Dense Approaches The coarse-to-fine methods described above have difficulty dealing with long-range motion of small objects. In contrast, there exist many methods to accurately estim... |

167 | Three-dimensional scene flow,
- Vedula, Baker, et al.
- 2005
(Show Context)
Citation Context ...oes not change. This assumption combines a number of assumptions about the reflectance properties of the 3scene (e.g., that it is Lambertian), the illumination in the scene (e.g., that it is uniform =-=[76]-=-) and about the image formation process in the camera (e.g., that there is no vignetting). If I(x, y, t) is the intensity of a pixel (x, y) at time t and the flow is (u(x, y, t), v(x, y, t)), Brightne... |

157 |
Scene segmentation from visual motion using global optimization
- D, Buxton
- 1987
(Show Context)
Citation Context ...tion of Equations (5) and (9) defines the energy used by Horn and Schunck [33]. Given more than two frames in the video, it is also possible to add temporal smoothness terms ∂u ∂v and to Equation (9) =-=[10, 49, 54]-=-. Note, however, that the temporal terms need ∂t ∂t to be weighted differently from the spatial ones. 2.2.2 Choice of the Penalty Function As for the data term in Section 2.1.2, under a probabilistic ... |

156 | Mixture models for optical flow computation - Jepson, Black - 1993 |

137 | Sift flow: dense correspondence across difference scenes
- Liu, Yuen, et al.
- 2008
(Show Context)
Citation Context ...lization of the resulting data term. It is also possible to use more complicated features than the gradient. For example a Field-of-Experts formulation was used in [70] and SIFT features were used in =-=[43]-=-. 2.1.4 Modeling Illumination, Blur, and Other Appearance Changes The motivation for using features is to increase robustness to illumination and other appearance changes. Another approach is to estim... |

128 | Robust dynamic motion estimation over time, in:
- Black, Anandan
(Show Context)
Citation Context ...tion of Equations (5) and (9) defines the energy used by Horn and Schunck [33]. Given more than two frames in the video, it is also possible to add temporal smoothness terms ∂u ∂v and to Equation (9) =-=[10, 49, 54]-=-. Note, however, that the temporal terms need ∂t ∂t to be weighted differently from the spatial ones. 2.2.2 Choice of the Penalty Function As for the data term in Section 2.1.2, under a probabilistic ... |

126 |
Computation and analysis of image motion: A synopsis of current problems and methods.
- Mitiche, Bouthemy
- 1996
(Show Context)
Citation Context ...al flow algorithms is beyond the scope of this paper. Interested readers are referred to previous surveys by Aggarwal and Nandhakumar [1], Barron et al. [2], Otte and Nagel [16], Mitiche and Bouthemy =-=[14]-=-, and Stiller and Konrad [23]. Instead we focus here on the evaluation of optical flow algorithms. We must first define what we mean by optical flow. Following Horn’s [10] taxonomy, the motion field i... |

107 | Highly accurate optic flow computation with theoretically justified warping
- Papenberg, Bruhn, et al.
(Show Context)
Citation Context ...timization algorithms (see Section 2.3), which often involves the use of a Taylor expansion to linearize the energies. The two constraints are therefore essentially equivalent in practical algorithms =-=[54]-=-. An alternative to the assumption of “constancy” is that the signals (images) at times t and t + 1 are highly correlated [17,57]. Various correlation constraints can be used for computing dense flow ... |

100 | Estimating optical flow in segmented images using variable-order parametric models with local deformations
- Black, Jepson
- 1996
(Show Context)
Citation Context ...en challenging to compute is that the flow and its segmentation must be computed together. 12Several methods first segment the scene using non-motion cues and then estimate the flow in these regions =-=[12,83]-=-. Within each image segment, Black and Jepson [12] use a parametric model (e.g., affine) [9], which simplifies the problem by reducing the number of parameters to be estimated. The flow is then refine... |

100 | Computing optical flow with physical models of brightness variation
- Haussecker, Fleet
- 2001
(Show Context)
Citation Context ... the illumination change model g(x, y) and b(x, y) [51, 65]. Explicit illumination modeling can be generalized in several ways, for example to model the changes physically over a longer time interval =-=[30]-=- or to model blur [65]. 52.2 Prior Term The data term alone is ill-posed with fewer constraints than unknowns. It is therefore necessary to add a prior to favor one possible solution over another. Ge... |

100 | On the spatial statistics of optical flow
- Roth, Black
- 2005
(Show Context)
Citation Context ...he first approach [11], while others use the second [16, 54, 80]. Note that some penalty (log probability) functions have probabilistic interpretations related to the distribution of flow derivatives =-=[61]-=-. 2.2.3 Spatial Weighting One popular refinement for the prior term is one that weights the penalty function with a spatially varying function. One particular example is to vary the weight depending o... |

91 | Learning conditional random fields for stereo
- Scharstein, Pal
- 2007
(Show Context)
Citation Context ...dified Stereo Data for Rigid Scenes Our final dataset consists of modified stereo data. Specifically we use the Venus dataset obtained by registering planes in the scene [19], and the Moebius dataset =-=[18]-=-, which was obtained using structured lighting [20]. These datasets have an asymmetric disparity range [0, dmax] that is appropriate for stereo, but not for optical flow. We crop different subregions ... |

74 |
Revised definition of optical flow: Integration of radiometric and geometric cues for dynamic scene analysis
- Negahdaripour
- 1998
(Show Context)
Citation Context ... than Equation (2), with four unknowns per pixel rather than two. It can, however, be solved by putting an appropriate prior on the two components of the illumination change model g(x, y) and b(x, y) =-=[51, 65]-=-. Explicit illumination modeling can be generalized in several ways, for example to model the changes physically over a longer time interval [30] or to model blur [65]. 52.2 Prior Term The data term ... |

73 | Skin and Bones: Multi-layer, Locally Affine, Optical Flow and Regularization with Transparency
- Ju, Black, et al.
- 1996
(Show Context)
Citation Context ...an be replaced with priors that encourage the secondorder derivatives ( ∂2u ∂x2 , ∂2u ∂y2 , ∂2u ∂x∂y , ∂2v ∂x2 , ∂2v ∂y2 , ∂2v ) to be small [4, 75]. ∂x∂y A related approach is to use an affine prior =-=[35, 36, 52, 65]-=-. One approach is to overparameterize the flow [52]. Instead of solving for two flow vectors (u(x, y, t), v(x, y, t)) at each pixel, the algorithm in [52] solves for 6 affine parameters ai(x, y, t), i... |

70 |
Optical flow estimation: advances and comparisons.
- Otte, Nagel
- 1994
(Show Context)
Citation Context ...Work A full review of optical flow algorithms is beyond the scope of this paper. Interested readers are referred to previous surveys by Aggarwal and Nandhakumar [1], Barron et al. [2], Otte and Nagel =-=[16]-=-, Mitiche and Bouthemy [14], and Stiller and Konrad [23]. Instead we focus here on the evaluation of optical flow algorithms. We must first define what we mean by optical flow. Following Horn’s [10] t... |

68 |
On the computation of motion from sequences of images—a review
- Aggarwal, Nandhakumar
- 1988
(Show Context)
Citation Context ...rs of the scoring measures. 2. Related Work A full review of optical flow algorithms is beyond the scope of this paper. Interested readers are referred to previous surveys by Aggarwal and Nandhakumar =-=[1]-=-, Barron et al. [2], Otte and Nagel [16], Mitiche and Bouthemy [14], and Stiller and Konrad [23]. Instead we focus here on the evaluation of optical flow algorithms. We must first define what we mean ... |

67 | On benchmarking optical flow
- McCane, Novins, et al.
- 2001
(Show Context)
Citation Context ...Otte and Nagel [16] introduced ground truth for a real scene consisting of polyhedral objects. While this provided real image data, the images still were extremely simple. Most recently McCane et al. =-=[12]-=- provided more ground truth for real polyhedral scenes as well as graphics scenes of varying realism. Here we go beyond these studies in several important ways. First, we provide ground-truth motion f... |

62 | Large displacement optical flow
- Brox, Bregler, et al.
- 2009
(Show Context)
Citation Context ...o accurately estimate sparse feature correspondences even when the motion is large. Such sparse matching method can be combined with the continuous energy minimization approaches in a variety of ways =-=[15,43,60,83]-=-. 2.5.5 Visibility and Occlusion Occlusions and visibility changes can cause major problems for optical flow algorithms. The most common solution is to model such effects implicitly using a robust pen... |

60 |
Anisotropic huber-l1 optical flow
- Werlberger, Trobin, et al.
- 2009
(Show Context)
Citation Context ... the weighting function is isotropic, treating all directions equally. A variety of approaches weight the smoothness prior anisotropically. For example, Nagel and Enkelmann [50] and Werlberger et al. =-=[82]-=- weight the direction along the image gradient less than the direction orthogonal to it, and Sun et al. [70] learn a Steerable Random field to define the weighting. Zimmer et al. [84] perform a simila... |

59 | Local Correlation Measures for Motion Analysis - Burt, Yen, et al. - 1982 |

57 |
Consistent segmentation for optical flow estimation
- Zitnick, Jojic, et al.
- 2005
(Show Context)
Citation Context ...r interpolation, we obtained results using the real-time flow algorithm used in Microsoft MediaPlayer 9 for video smoothing [13]. Zitnick et al.: We used the author’s implementation of this algorithm =-=[28]-=- that uses consistent segmentation. The results for all of these algorithms are available on the evaluation website. We include results for all four measures (AE, EP, SSD, and normalized SSD), all the... |

57 | A multigrid platform for real-time motion computation with discontinuity-preserving variational methods. InternationalJournalofComputerVision,70(3):257–277,December2006 - Bruhn, Weickert, et al. |

56 | Fusionflow: Discrete-continuous optimization for optical flow estimation - Lempitsky, Roth, et al. |

52 |
Correlation Techniques of Image Registration
- Pratt
- 1974
(Show Context)
Citation Context ... constraints are therefore essentially equivalent in practical algorithms [54]. An alternative to the assumption of “constancy” is that the signals (images) at times t and t + 1 are highly correlated =-=[17,57]-=-. Various correlation constraints can be used for computing dense flow including normalized cross correlation and Laplacian correlation [18, 26]. 2.1.2 Choice of the Penalty Function Equations (2) and... |

52 | An improved algorithm for TV-L1 optical flow
- Wedel, Pock, et al.
- 2009
(Show Context)
Citation Context .... Modeling/Norm. L1/TV Norm Other Robust Penalty Fn Algorithm Adaptive [78] X X X X X X X Complementary OF [84] X X X X X X X X Aniso. Huber-L1 [82] X X X X X X DPOF [39] X X X X X X X TV-L1-improved =-=[80]-=- X X X X CBF [74] X X X X X Brox et al. [54] X X X X X F-TV-L1 [79] X X X X Second-order prior [75] X X X Fusion [40] X X X X X X X Dynamic MRF [27] X X X Seg OF [83] X X X X X X Learning Flow [70] X ... |

51 |
Direct multi-resolution estimation of ego-motion and structure from motion
- Hanna
(Show Context)
Citation Context ...uthors have explored rigidity or fundamental matrix priors which, in the absence of other evidence, favor flows that are parallel to epipolar lines. These constraints have both been strictly enforced =-=[1, 29, 52]-=- and added as a soft prior [78, 79]. 7 y0 y02.3 Continuous Optimization Algorithms The two most commonly used continuous optimization techniques in optical flow are: (1) gradient descent algorithms (... |

49 |
Prediction error as a quality metric for motion and stereo.
- Szeliski
- 1999
(Show Context)
Citation Context ...-view generation and motion-compensated compression, what is important is not how well the flow field matches the ground-truth motion, but how well intermediate frames can be predicted using the flow =-=[26]-=-. 4. Real stereo imagery of rigid scenes, where dense ground-truth is captured using the procedures in [19, 20]. The data is then modified for optical flow. Our focus in this paper is on developing th... |

48 | Moving gradients: a path-based method for plausible image interpolation - Mahajan, Huang, et al. |

48 | Production-ready global illumination - Landis - 2002 |

46 | Learning optical flow
- Sun, Roth, et al.
- 2008
(Show Context)
Citation Context ...training set (12 datasets) and a final evaluation set (12 datasets). The training set includes the ground truth and is meant to be used for debugging, parameter estimation, and possibly even learning =-=[41, 70]-=-. The ground truth for the final evaluation set is not publicly available (with the exception of the Yosemite sequence, which is included in the test set to allow some comparison with algorithms publi... |

45 | Estimating Motion in Image Sequence: A Tutorial on Modeling and
- Stiller, Konrad
- 1999
(Show Context)
Citation Context ...the scope of this paper. Interested readers are referred to previous surveys by Aggarwal and Nandhakumar [1], Barron et al. [2], Otte and Nagel [16], Mitiche and Bouthemy [14], and Stiller and Konrad =-=[23]-=-. Instead we focus here on the evaluation of optical flow algorithms. We must first define what we mean by optical flow. Following Horn’s [10] taxonomy, the motion field is the 2D projection of the 3D... |

44 |
Pyramidal implementation of the Lucas-Kanade feature tracker, Open CV Documentation,
- Bouguet
- 1999
(Show Context)
Citation Context ...sults to define the state of the art on the database and allow researchers to get a sense of what is good performance on the data. To this end, we compared 5 algorithms: Pyramid LK: An implementation =-=[5]-=- of the Lucas-Kanade algorithm [11] on a pyramid, subsequently refined at Microsoft Research. This implementation performs significantly better than the Lucas-Kanade code in Barron et al. [2]. It is i... |

44 |
Structure- and motion-adaptive regularization for high accuracy optic flow
- Wedel, Cremers, et al.
- 2009
(Show Context)
Citation Context ...mental matrix priors which, in the absence of other evidence, favor flows that are parallel to epipolar lines. These constraints have both been strictly enforced [1, 29, 52] and added as a soft prior =-=[78, 79]-=-. 7 y0 y02.3 Continuous Optimization Algorithms The two most commonly used continuous optimization techniques in optical flow are: (1) gradient descent algorithms (Section 2.3.1) and (2) extremal or ... |

42 | Humanassisted motion annotation
- Liu, Freeman, et al.
- 2008
(Show Context)
Citation Context ... hidden markers was also used in [58] to provide a sparse ground-truth alignment (or flow) of face images. Finally, Liu et al. recently proposed a method to obtain ground-truth using human annotation =-=[42]-=-. • Realistic Synthetic Imagery: We address the limitations of simple synthetic sequences such as Yosemite [7] by rendering more complex scenes with larger motion ranges, more realistic texture, indep... |

41 | A multi-view approach to motion and stereo
- Szeliski
- 1999
(Show Context)
Citation Context ...chers to develop their own interpolation algorithms and submit interpolated images for direct comparison with the ground truth; for example, by looking at more than pairs of frames to estimate motion =-=[25, 24]-=-.sAcknowledgments MJB and SR were supported by NSF grants IIS-0535075 and IIS-0534858, and a gift from Intel Corporation. DS was supported by NSF grant IIS-0413169. Ludwig von Reiche of Mental Images ... |

34 | Computing optical flow across multiple scales: an adaptive coarse-to-fine strategy - Battiti, Amaldi, et al. - 1991 |

33 | Over-parameterized variational optical flow
- Nir, Bruckstein, et al.
- 2008
(Show Context)
Citation Context ...an be replaced with priors that encourage the secondorder derivatives ( ∂2u ∂x2 , ∂2u ∂y2 , ∂2u ∂x∂y , ∂2v ∂x2 , ∂2v ∂y2 , ∂2v ) to be small [4, 75]. ∂x∂y A related approach is to use an affine prior =-=[35, 36, 52, 65]-=-. One approach is to overparameterize the flow [52]. Instead of solving for two flow vectors (u(x, y, t), v(x, y, t)) at each pixel, the algorithm in [52] solves for 6 affine parameters ai(x, y, t), i... |

32 | Multi-resolution flow-through motion analysis - Burt, Yen, et al. - 1983 |

32 | Motion from color - Golland, Bruckstein - 1997 |

30 | Surface stereo with soft segmentation - Bleyer, Rother, et al. |

26 | Estimating intrinsic component images using non-linear regression
- TAPPEN, ADELSON, et al.
- 1992
(Show Context)
Citation Context ... each point capturing separate test images (in visible light) and ground-truth images (in UV light). Note that a related technique is being used commercially for motion capture [48] and Tappen et al. =-=[73]-=- recently used certain wavelengths to hide ground truth in intrinsic images. Another form of hidden markers was also used in [58] to provide a sparse ground-truth alignment (or flow) of face images. F... |

23 | Optical flow estimation with uncertainties through dynamic MRFs
- Glocker, Paragios, et al.
- 2008
(Show Context)
Citation Context ...sult. An early use of this idea for flow estimation employed simulated annealing with a state space that adapted based on the local shape of the objective function [10]. More recently, Glocker et al. =-=[27]-=- initially use a sparse sampling of possible motions on a coarse version of the problem. As the algorithm runs from coarse to fine, the spatial density of motion states (which are interpolated with a ... |

22 |
Scene Matching by Hierarchical Correlation
- Glazer, Reynolds, et al.
- 1983
(Show Context)
Citation Context ...nals (images) at times t and t + 1 are highly correlated [17,57]. Various correlation constraints can be used for computing dense flow including normalized cross correlation and Laplacian correlation =-=[18, 26]-=-. 2.1.2 Choice of the Penalty Function Equations (2) and (4) both provide one error per pixel, which leads to the question of how these errors are aggregated over the image. A baseline approach is to ... |

22 | A segmentation based variational model for accurate optical flow estimation
- Xu, Chen, et al.
- 2008
(Show Context)
Citation Context ...en challenging to compute is that the flow and its segmentation must be computed together. 12Several methods first segment the scene using non-motion cues and then estimate the flow in these regions =-=[12,83]-=-. Within each image segment, Black and Jepson [12] use a parametric model (e.g., affine) [9], which simplifies the problem by reducing the number of parameters to be estimated. The flow is then refine... |

19 | Complementary optic flow - Zimmer, Bruhn, et al. - 2009 |

16 |
Introducing a smoothness constraint in a matching approach for the computation of optical flow fields
- Anandan, Weiss
- 1985
(Show Context)
Citation Context ...gher-Order Priors The first-order priors in Section 2.2.1 can be replaced with priors that encourage the secondorder derivatives ( ∂2u ∂x2 , ∂2u ∂y2 , ∂2u ∂x∂y , ∂2v ∂x2 , ∂2v ∂y2 , ∂2v ) to be small =-=[4, 75]-=-. ∂x∂y A related approach is to use an affine prior [35, 36, 52, 65]. One approach is to overparameterize the flow [52]. Instead of solving for two flow vectors (u(x, y, t), v(x, y, t)) at each pixel,... |

14 | Local grouping for optical flow
- Ren
- 2008
(Show Context)
Citation Context ...o accurately estimate sparse feature correspondences even when the motion is large. Such sparse matching method can be combined with the continuous energy minimization approaches in a variety of ways =-=[15,43,60,83]-=-. 2.5.5 Visibility and Occlusion Occlusions and visibility changes can cause major problems for optical flow algorithms. The most common solution is to model such effects implicitly using a robust pen... |

13 | Optical flow estimation on coarse-to-fine region-trees using discrete optimization
- Lei, Yang
- 2009
(Show Context)
Citation Context ...e interpolated with a spline) and the density of possible flows at any given control point are chosen based on the uncertainty in the solution from the previous iteration. The algorithm of Lei et al. =-=[39]-=- also sparsely allocates states across space and for the possible flows at each spatial location. The spatial allocation uses a hierarchy of segmentations, with a single possible flow for each segment... |

13 | Overview of the face recognition grand challenge - Philips, Flynn, et al. |

13 | Region-based optical flow estimation - Fuh, Maragos - 1989 |

12 | An unbiased second-order prior for high-accuracy motion estimation
- Trobin, Pock, et al.
- 2008
(Show Context)
Citation Context ...tary OF [84] X X X X X X X X Aniso. Huber-L1 [82] X X X X X X DPOF [39] X X X X X X X TV-L1-improved [80] X X X X CBF [74] X X X X X Brox et al. [54] X X X X X F-TV-L1 [79] X X X X Second-order prior =-=[75]-=- X X X Fusion [40] X X X X X X X Dynamic MRF [27] X X X Seg OF [83] X X X X X X Learning Flow [70] X X X X X X Filter Flow [65] X X X X X X X Graph Cuts [20] X X X X Black & Anandan [11] X X X SPSA-le... |

10 | H (2008) Continuous energy minimization via repeated binary fusion
- Trobin, Pock, et al.
(Show Context)
Citation Context ... subsequent use of Equation (17) is that it reduces the restrictions on the functional form of EGlobal, important in learning-based approaches [70]. 2.3.3 Other Continuous Algorithms Another approach =-=[74, 80]-=- is to decouple the data and prior terms through the introduction of two sets of flow parameters, say (udata, vdata) for the data term and (uprior, vprior) for the prior: EGlobal = EData(udata, vdata)... |

9 | Toward global minimum through combined local minima
- Jung, Lee, et al.
- 2008
(Show Context)
Citation Context ... limit the power of the discrete algorithms to avoid local minima. The few methods proposed to date can be divided into two main approaches described below. 2.4.1 Fusion Approaches Algorithms such as =-=[37, 40, 74]-=- assume that a number of candidate flow fields have been generated by running standard algorithms such as Lucas-Kanade [44] and Horn-Schunck [33], possibly multiple times with a number of different pa... |

9 | Learning for optical flow using stochastic optimization
- Li, Huttenlocher
- 2008
(Show Context)
Citation Context ...training set (12 datasets) and a final evaluation set (12 datasets). The training set includes the ground truth and is meant to be used for debugging, parameter estimation, and possibly even learning =-=[41, 70]-=-. The ground truth for the final evaluation set is not publicly available (with the exception of the Yosemite sequence, which is included in the test set to allow some comparison with algorithms publi... |

8 |
Motion estimation based on optical flow with adaptive gradients
- Sun, Haynor, et al.
- 2000
(Show Context)
Citation Context ...chers to develop their own interpolation algorithms and submit interpolated images for direct comparison with the ground truth; for example, by looking at more than pairs of frames to estimate motion =-=[25, 24]-=-.sAcknowledgments MJB and SR were supported by NSF grants IIS-0535075 and IIS-0534858, and a gift from Intel Corporation. DS was supported by NSF grant IIS-0413169. Ludwig von Reiche of Mental Images ... |

8 |
Algorithmic differentiation: Application to variational problems in computer vision
- Pock, Pock, et al.
(Show Context)
Citation Context ...opriately and the next iteration applied. One disadvantage of variational algorithms is that the discretization of the Euler-Lagrange equations is not always exact with respect to the original energy =-=[56]-=-. Another extremal approach [70], closely related to the variation algorithms is to use: ∂EGlobal ∂f = 0 (17) rather than the Euler-Lagrange equations. Otherwise, the approach is similar. Equation (17... |

7 |
Dense optical flow by iterative local window registration
- Besnerais, Champagnat
- 2005
(Show Context)
Citation Context ...d inverting the Hessian, a 2n × 2n matrix where there are n pixels in the image. These algorithms are applicable to problems with fewer parameters such as the Lucas-Kanade algorithm [44] and variants =-=[38]-=-, which solve for a single flow vector (2 unknowns) independently for each block of pixels. Another set of examples are parametric motion algorithms [9], which also just solve for a small number of un... |

6 |
Occlusion reasoning for temporal interpolation using optical flow
- Herbst, Seitz, et al.
- 2009
(Show Context)
Citation Context ... 1 using the same approach as in Step 1 to give u1(x). Any pixel x in u1(x) that is not targeted by this splatting has no corresponding pixel in I0 and thus we set O1(x) = 1 for all such pixels. (See =-=[31]-=- for a bidirectional algorithm that performs this reasoning at time t.) In order to compute O0(x), we cross-check the flow vectors, setting O0(x) = 1 if |u0(x) − u1(x + u0(x))| > 0.5. (20) 4. Compute ... |

6 |
Increasing the density of Active Appearance Models
- Ramnath, Baker, et al.
- 2008
(Show Context)
Citation Context ...e is being used commercially for motion capture [48] and Tappen et al. [73] recently used certain wavelengths to hide ground truth in intrinsic images. Another form of hidden markers was also used in =-=[58]-=- to provide a sparse ground-truth alignment (or flow) of face images. Finally, Liu et al. recently proposed a method to obtain ground-truth using human annotation [42]. • Realistic Synthetic Imagery: ... |

6 | Filter flow
- Seitz, Baker
- 2009
(Show Context)
Citation Context ...BF [74] 69 Brox et al. [54] 18 Rannacher [59] 0.12 F-TV-L1 [79] 8 Second-order prior [75] 14 Fusion [40] 2,666 Dynamic MRF [27] 366 Algorithm Runtime Seg OF [83] 60 Learning Flow [70] 825 Filter Flow =-=[65]-=- 34,000 Graph Cuts [20] 1,200 Black & Anandan [11] 328 SPSA-learn [41] 200 Group Flow [60] 600 2D-CLG [16] 844 Horn & Schunck [33] 49 TI-DOFE [19] 260 FOLKI [38] 1.4 Pyramid LK [44] 11.9 Table 1: Repo... |

6 | Multi-PIE - Matthews, Cohn, et al. - 2008 |

5 |
Investigations of multigrid algorithms for the estimation of optical flow fields in image sequences
- Enkelman
(Show Context)
Citation Context ...he likelihood of falling into a local minimum. One component in many algorithms is a coarse-to-fine strategy. The most common approach is to build image pyramids by repeated blurring and downsampling =-=[3, 11, 18, 22, 26, 44]-=-. Optical flow is first computed on the top level (fewest pixels) and then upsampled and used to initialize the estimate at the next level. Computation at the higher levels in the pyramid involves far... |

4 |
Rendering with mental ray
- Driemeyer
- 2001
(Show Context)
Citation Context ...ditionally, the tree bark has significant 3D texture. The scenes are rigid and the camera motions include camera rotation and 3D translation. These scenes were generated using the Mental Ray renderer =-=[7]-=-. Each scene is generated with and without motion blur. For the scenes with blur, the motion is sampled at the virtual shutter open and close times and hence is assumed linear during the open shutter ... |

4 | Two-frame optical flow formulation in an unwarping multiresolution scheme
- Cassisa, Simoens, et al.
(Show Context)
Citation Context ...ime Seg OF [83] 60 Learning Flow [70] 825 Filter Flow [65] 34,000 Graph Cuts [20] 1,200 Black & Anandan [11] 328 SPSA-learn [41] 200 Group Flow [60] 600 2D-CLG [16] 844 Horn & Schunck [33] 49 TI-DOFE =-=[19]-=- 260 FOLKI [38] 1.4 Pyramid LK [44] 11.9 Table 1: Reported runtimes on the Urban sequence in seconds. We do not normalize for the programming environment, CPU speed, number of cores, or other hardware... |

3 |
Contour reality capture. http://www.mova.com
- LLC
(Show Context)
Citation Context ...scene, at each point capturing separate test images (in visible light) and ground-truth images (in UV light). See Figure 1. Note that a related technique is being used commercially for motion capture =-=[15]-=-. 2. Realistic synthetic imagery. We address the limitations of sequences such as Yosemite [2] by rendering more complex scenes with significant motion discontinuities and textureless regions. 3. Imag... |

3 |
Realtime 3d motion estimation on graphics hardware,” Undergraduate Thesis
- Rannacher
- 2009
(Show Context)
Citation Context ...r the selected metric/statistic. 27Algorithm Runtime Adaptive [78] 9.2 Complementary OF [84] 44 Aniso. Huber-L1 [82] 2 DPOF [39] 261 TV-L1-improved [80] 2.9 CBF [74] 69 Brox et al. [54] 18 Rannacher =-=[59]-=- 0.12 F-TV-L1 [79] 8 Second-order prior [75] 14 Fusion [40] 2,666 Dynamic MRF [27] 366 Algorithm Runtime Seg OF [83] 60 Learning Flow [70] 825 Filter Flow [65] 34,000 Graph Cuts [20] 1,200 Black & Ana... |

3 | Does color really help in dense stereo matching - Bleyer, Chambon |

2 | Estimating image motion in layers: the skin and bones model - Ju - 1998 |

1 |
Media player 9 video quality demos. http://www.microsoft.com/windows/windowsmedia/ demos/video quality demos.aspx
- Corporation
(Show Context)
Citation Context ...equence (included in the results webpage). MediaPlayer TM : As a baseline for interpolation, we obtained results using the real-time flow algorithm used in Microsoft MediaPlayer 9 for video smoothing =-=[13]-=-. Zitnick et al.: We used the author’s implementation of this algorithm [28] that uses consistent segmentation. The results for all of these algorithms are available on the evaluation website. We incl... |

1 |
Two applications of graph-cuts to image processing
- Cooke
- 2008
(Show Context)
Citation Context ...the segmentation hierarchy, first a sparse sampling of the possible flows is used, followed by a denser sampling with a reduced range around the solution from the previous iteration. The algorithm in =-=[20]-=- iteratively alternates between two steps. In the first step, all the states are allocated to the horizontal motion, which is estimated similarly to stereo, assuming the vertical motion is zero. In th... |

1 |
3Delight rendering software
- Research
(Show Context)
Citation Context ...ed textures and some surfaces are slightly reflective. There are cast shadows as well as a few independently moving “cars”. These scenes were generated using the 3Delight Renderman-compliant renderer =-=[21]-=- at a resolution of 640x480 pixels using linear gamma. The images are antialiased, mimicking the effect of sensors with finite area. Current rendered scenes do not use full global illumination but use... |

1 |
Fusion flow
- Lempitsky, Roth, et al.
- 2008
(Show Context)
Citation Context ...xtured cloth. This region is difficult for many flow algorithms because the difference in motions is small and the color difference is not great either. Only a few algorithms (e.g., DPOF [39], Fusion =-=[40]-=-, and Dynamic MRF [27]) perform well in this region. Getting this region correct is more important in the interpolation study than in the flow error study because: (1) the background is quite highly t... |

1 | The PASCAL visual object classes challenge 2009. http:// www.pascal-network.org/challenges/VOC/voc2009/workshop/ index.html Fei-Fei - Fergus, R, et al. - 2009 |