This paper presents a method for extracting distinctive invariant features from images, which can be used to perform reliable matching between different images of an object or scene. The features are invariant to image scale and rotation, and are shown to provide robust matching across a a substantial range of affine distortion, addition of noise, change in 3D viewpoint, and change in illumination. The features are highly distinctive, in the sense that a single feature can be correctly matched with high probability against a large database of features from many images. This paper also describes an approach to using these features for object recognition. The recognition proceeds by matching individual features to a database of features from known objects using a fast nearest-neighbor algorithm, followed by a Hough transform to identify clusters belonging to a single object, and finally performing verification through leastsquares solution for consistent pose parameters. This approach to recognition can robustly identify objects among clutter and occlusion while achieving near real-time performance.

Two images of a single scene/object are related by the epipolar geometry, which can be described by a 3×3 singular matrix called the essential matrix if images' internal parameters are known, or the fundamental matrix otherwise. It captures all geometric information contained in two images, and its determination is very important in many applications such as scene modeling and vehicle navigation. This paper gives an introduction to the epipolar geometry, and provides a complete review of the current techniques for estimating the fundamental matrix and its uncertainty. A well-founded measure is proposed to compare these techniques. Projective reconstruction is also reviewed. The software which we have developed for this review is available on the Internet.

This paper describes the extraction of 3D geometrical data from image sequences, for the purpose of creating 3D models of objects in the world. The approach is uncalibrated - camera internal parameters and camera motion are not known or required. Processing an image sequence is underpinned by token correspondences between images. We utilise matching techniques which are both robust (detecting and discarding mismatches) and fully automatic. The matched tokens are used to compute 3D structure, which is initialised as it appears and then recursively updated over time. We describe a novel robust estimator of the trifocal tensor, based on a minimum number of token correspondences across an image triplet; and a novel tracking algorithm in which corners and line segments are matched over image triplets in an integrated framework. Experimental results are provided for a variety of scenes, including outdoor scenes taken with a hand-held camcorder. Quantitative statistics are included to asses...

In this paper the theoretical and practical feasibility of self-calibration in the presence of varying intrinsic camera parameters is under investigation. The paper’s main contribution is to propose a self-calibration method which efficiently deals with all kinds of constraints on the intrinsic camera parameters. Within this framework a practical method is proposed which can retrieve metric reconstruction from image sequences obtained with uncalibrated zooming/focusing cameras. The feasibility of the approach is illustrated on real and synthetic examples. Besides this a theoretical proof is given which shows that the absence of skew in the image plane is sufficient to allow for self-calibration. A counting argument is developed which—depending on the set of constraints—gives the minimum sequence length for self-calibration and a method to detect critical motion sequences is proposed.

This paper presents a linear algorithm for recovering 3D affine shape and motion from line correspondences with uncalibrated affine cameras. The algorithm requires a minimum of seven line correspondences over three views. The key idea is the introduction of a one-dimensional projective camera. This converts 3D affine reconstruction of "line directions" into 2D projective reconstruction of "points". In addition, a line-based factorisation method is also proposed to handle redundant views. Experimental results both on simulated and real image sequences validate the robustness and the accuracy of the algorithm.

We present an algorithm to estimate the parameters of a linear model in the presence of heteroscedastic noise, i.e., each data point having a different covariance matrix.

Modelling of 3D objects from image sequences is a challenging problem and has been an important research topic in the areas of photogrammetry and computer vision for many years. In this paper, a system is presented which automatically extracts a textured 3D surface model from a sequence of images of a scene. The system can deal with unknown camerasettings. In addition, the parameters of this camera are allowed to change during acquisition (e.g. by zooming or focussing). No prior knowledge about the scene is necessary to build the 3D models. Therefore, this system offers a high degree of flexibility. The system is based on state-of-the-art algorithms recently developed in computer vision. The 3D modelling task is decomposed into a number of succesive steps. Gradually, more knowledge of the scene and the camera setup is retrieved. At this point, the obtained accuracy is not yet at the level required for most metrology applications, but the visual quality is very convincing. This system has been applied to a number of applications in archaeology. The Roman site of Sagalassos (south-west Turkey) was used as a test case to illustrate the potential of this new approach. Key words: 3D reconstruction; self-calibration; image matching; virtual reality; uncalibrated camera; image sequences; archaeology # Corresponding author. Tel.: +32-16-321064; Fax: +32-16-321723; E-mail: Marc.Pollefeys@esat.kuleuven.ac.be (M. Pollefeys). 1 1.

Abstract—We propose a new methodology for reliably solving the correspondence problem between sparse sets of points of two or more images. This is a key step in most problems of computer vision and, so far, no general method exists to solve it. Our methodology is able to handle most of the commonly used assumptions in a unique formulation, independent of the domain of application and type of features. It performs correspondence and outlier rejection in a single step and achieves global optimality with feasible computation. Feature selection and correspondence are first formulated as an integer optimization problem. This is a blunt formulation, which considers the whole combinatorial space of possible point selections and correspondences. To find its global optimal solution, we build a concave objective function and relax the search domain into its convex-hull. The special structure of this extended problem assures its equivalence to the original one, but it can be optimally solved by efficient algorithms that avoid combinatorial search. This methodology can use any criterion provided it can be translated into cost functions with continuous second derivatives. Index Terms—Correspondence problem, linear and concave programming, sparse stereo. 1

Camera calibration is essential to many computer vision applications. In practice this often requires cumbersome calibration procedures to be carried out regularly. In the last few years a lot of work has been done on selfcalibration of cameras, ranging from weak calibration to metric calibration. It has been shown that a metric calibration of the camera setup (up to scale) was possible based on the rigidity of the scene only. In this paper a stratified approach is proposed which gradually retrieves the metric calibration of the camera setup. Starting from an uncalibrated image sequence the projective calibration is retrieved first. In projective space the plane at infinity is then identified yielding the affine calibration. This is achieved using a constraint which can be formulated between any two arbitrary images of the sequence. Once the affine calibration is known the upgrade to metric is easily obtained through linear equations. 1

A robust approach for super resolution is presented, which is especially valuable in the presence of outliers. Such outliers may be due to motion erros, inaccurate blur models, noise, moving objects, motion blur etc. This tobusiness is needed since super-resolution methods are very sensitive to such errors.