SHASHUA, A., AND NAVAB, N. 1994. Relative affine structure: theory and application to 3d reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, 483--489.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....view synthesis from a single image; in this case, additional constraints must be used. For example, the symmetry of human faces is exploited in [14] We use the knowledge on the soccer ground structure and the fact that the players are in vertical position. We follow the relative affine structure [16] approach, which will be reviewed in the next section. The rest of the paper is structured as follows. Section 4 introduces our first contribution. It is subdivided into two subsections. The first (Sec. 4.1) describe how the relative affine structure is recovered, the second (Sec. 4.2) deals the ....

....then the generation of extrapolated views is described (Sec. 5.2) 2. Background In this section we review some background notions needed to understand the paper. A complete discussion and formulation of the relative affine structure theory, and its close relative plane parallax, can be found in [16, 17]. A more general reference on the geometry of multiple views is [4] Two views of a planar set of points are related via a homography, i.e, a non singular linear transformation of the projective plane into itself. The most general homography is represented by a non singular 3 3 matrix H . If m ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3-D reconstruction from perspective views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 483--489, 1994.

.... 3D scene points have also been derived [2, 20] Alternatively, multiple uncalibrated images can be handled using the plane parallax (P P) approach, which analyzes the parallax displacements of a point between two views relative to a (real or virtual) physical planar surface Pi in the scene [16, 12, 11]. The magnitude of the parallax displacement is called the relative affine structure in [16] 12] shows that this quantity depends both on the Height H of P from Pi and its depth Z relative to the reference camera. Since the relative affine structure measure is relative to both the ....

.... using the plane parallax (P P) approach, which analyzes the parallax displacements of a point between two views relative to a (real or virtual) physical planar surface Pi in the scene [16, 12, 11] The magnitude of the parallax displacement is called the relative affine structure in [16]. 12] shows that this quantity depends both on the Height H of P from Pi and its depth Z relative to the reference camera. Since the relative affine structure measure is relative to both the reference frame (through Z) and the reference plane (through H) we refer to the P P framework also as ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, pages 483--489, Seattle, Wa., June 1994.

....from the estimated displacement fields, since these include camera egomotion. To solve the problem we use the plane Fig. 1. Two frames from a test flight at Revinge. a) b) Fig. 2. Estimated displacement field (a) and residual displacement (b) subsampled and magnified. parallax approach [5, 6, 7]. The idea is that the background can be approximated by a reference plane, the displacement field of which can be fit to a parametric model. After subtracting this we obtain a residual parallax displacement field where moving objects turn up and can be identified. Unfortunately also structures ....

A. Shashua and N. Navab, "Relative affine structure: Theory and application to 3d reconstruction from perspective views," in IEEE Conference on Computer Vision and Pattern Recognition, June 1994, pp. 483--489.

....The residual parallax field is an epipolar field (see Section 2) and is quasi parametric# it is also estimated using a direct method. The parallax magnitudes, when normalized to cancel out a scale factor that depends on the magnitude of camera translation are relativeaffineinvariants 1 [Shashua94a] and [Sawhney94] From the parallax maps, a projective 3D reconstruction [Faugeras92] of the scene can be made. This projective3Dreconstruction is related to the euclidean 3D construction by a collineation (4 Theta 4) matrix. Our work is related to the recent work using projective geometry of ....

....and [Sawhney94] From the parallax maps, a projective 3D reconstruction [Faugeras92] of the scene can be made. This projective3Dreconstruction is related to the euclidean 3D construction by a collineation (4 Theta 4) matrix. Our work is related to the recent work using projective geometry of [Hartley93, Sawhney94, Shashua94a, Szeliski94] and motion stabilization of [Irani94a] It also extends the work of [Carlson90] in obstacle detection by removing the requirement placed by their technique that the camera translation should be parallel to the reference plane. It differs from previous work in several ways: First, our derivation ....

[Article contains additional citation context not shown here]

Shashua, A. and Navab, N., "Relative Affine Structure: Theory and Application to 3D reconstruction from perspective views," Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Seattle, June 1994.

....and in its robustness. No prior detection and matching are assumed, it requires solving only small sets of linear equations, and each computational step is stated as an overdetermined highly constrained problem which is numerically stable. Similar approaches are described in [17] 25] 18] [26], 14] and are often referred to by the name plane plus parallax, since the estimated 2D parametric transformation frequently corresponds to the induced homography of a 3D planar surface in the scene. 2E GO MOTION FROM 2D IMAGE MOTION 2.1 Basic Model and Notations Let (X, Y, Z) denote the ....

....from the FOE (taking the rotations into account) The depth map is computed from the magnitude of these displacements. In Fig. 2d, the computed inverse depth map of the scene 1 Zxy , ch F H I K is displayed. Similar approaches to 3D shape recovery have since been suggested by [25] 18] [26], 14] Fig. 3 shows an example where the ego motion estimation was used to electronically stabilize (i.e. remove camera jitter) a sequence obtained by a hand held camera. 3C OMPUTING A 2D PARAMETRIC MOTION We use the method described in [16] to detect a 2D parametric transformation of an image ....

A. Shashua and N. Navab, "Relative Affine Structure: Theory and Application to 3d Reconstruction from Perspective Views," IEEE Conf. Computer Vision and Pattern Recognition, pp. 483--489, Seattle, June 1994.

....results and insights formulated in this paper are influenced bymany authors. Mentioning all publications to which this paper is indebted would amount to citing all the references listed at the end. However, the following articles have greatly influenced the presentation (in alphabetical order) [12, 13, 15, 35, 39, 48, 52]. 2 The Perspective Camera Model In this paper, the image formation process in a camera is modeled as a perspective projection of the scene onto an image plane. In mathematical terms, the scene is defined as a collection of points, lines and surfaces in Euclidean 3 space IR 3 . The mathematical ....

....the first view, gives the homogeneous coordinates Ap of the vanishing point in the second view of the ray of sight observing p in the first camera. In terms of projective geometry, A is a matrix of the homography that maps the first image onto the second one via the plane at infinity of the scene [39]. Formula (10) can now be rewritten as ae 0 p 0 = aeAp ae 0 e e 0 : 12) Observe that equation (12) algebraically expresses that, for a given point p in one view, the corresponding point p 0 in the other view lies on the line l 0 through the epipole e 0 and the vanishing point Ap ....

A. Shashua and N. Navab, Relative affine structure: Theory and application to 3D reconstruction from perspective views, Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR'94), Seattle, WA, IEEE Computer Society Press, Los Alamitos, CA, 1994, pp. 483--489.

....In this paper, we present a direct method that is non iterative, linear, and capable of reconstructing from arbitrarily many images. Previous direct methods either were limited to a small number of images (two or three [25] required strong assumptions about the scene usually planarity [27][24][14] 12] 23] or employed iterative optimization and required a starting estimate [7] 16] 10] 3] We present algorithms for two motion scenarios: 1) general motion, where the camera positions do not lie on a single plane, and 2) linear motion, where the camera moves roughly along a line ....

A. Shashua, N. Navab, "Relative Affine Structure: Theory and Application to 3D Reconstruction from Perspective Views," CVPR 483--489, 1994.

....computing correspondences as an intermediate step. While [3, 16, 15] recover 3D information relative to a camera centered coordinate system, an alternative approach has been proposed for recovering 3D structure in a scene centered coordinate system. In particular, the Plane Parallax approach [14, 11, 13, 7, 9, 8], which analyzes the parallax displacements of points relative to a (real or virtual) physical planar surface in the scene (the reference plane ) The underlying concept is that after the alignment of the reference plane, the residual image motion is due only to the translational motion of the ....

....(as opposed to two frames) are discussed in Section 4. Section 5 shows some results of applying the algorithm to real data. Section 6 concludes the paper. 2 The Plane Parallax Decomposition The induced 2D image motion of a 3D scene point between two images can be decomposed into two components [9, 7, 10, 11, 13, 14, 8, 2]: i) the image motion of a reference planar surface Pi (i.e. a homography) and (ii) the residual image motion, known as planar parallax . This decomposition is described below. To set the stage for the algorithm described in this paper, we begin with the derivation of the plane parallax ....

Shashua A. and Navab N., Relative affine Structure: Theory and Application to 3D Reconstruction From Perspective Views, In IEEE Conference on Computer Vision and Pattern Recognition, pages 483-489, 1994.

....lie at this depth. This is based on the idea that for the points which lie at a fixed depth, there is an analytical relation between their projections in multiple images, as exploited for stereo in the calibrated case [15] and for relative positioning of pairs of points in the uncalibrated case [16, 17]. By sweeping 3D space with planes at a set of different depths, a representation of the terrain is obtained. The idea of the sweeping plane method was presented in [2] where it was argued that such a technique makes a full and efficient use of multiple images. Our work extends these ideas in two ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proc. Conference on Computer Vision and Pattern Recognition, pages 483--489, Seattle, WA, 1994.

....to estimate a relevant reference plane [9, 12] The planar structure assumption greatly simplifies the problem, since the number of degrees of freedom no longer depends on the image size. Given point and or line correspondences, the discrete time motion problem has been solved by several authors [6, 7, 11, 13, 16]. For instantaneous representations, excellent work has been done in using multiscale estimation techniques to couple the flow and motion estimation problems to provide a direct method This document is GRASP Laboratory s Technical Report #410. Research supported by ARO grant DAAH04 96 1 0007, ....

A. Shashua and N. Navab. Relative Affine Structure: Theory and Application to 3D Reconstruction from Perspective Views. Proceedings IEEE Conference on Computer Vision and Pattern Recognition, Seattle, June 1994.

.... we now list a number of recent papers which have studied tasks to be performed when the only information relating the cameras are the Fundamental matrix (or matrices) ffl Recovery of the 3D projective structure of a scene from point matches [13, 23, 68, 59] and of the relative affine structure [13, 67, 69], ffl Obtention of projective invariants [67, 24, 22] ffl Prediction of image features in an image from image features in two other images [2, 54, 9] positions) 19] positions,orientations,curvatures) ffl Synthesis of an image from several images [35] ffl Convex hull computation and ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proc. Conference on Computer Vision and Pattern Recognition, pages 483--489, Seattle, WA, 1994.

....techniques typically make the unrealistic assumption that the flow field is smooth. In many situations, a more plausible assumption is that of a rigid world. Given point and or line correspondences, the discrete time rigid motion problem has been studied and solved by a number of authors (e.g. [6, 7, 11, 14, 17]) For instantaneous representations, multi scale estimation techniques have been used to couple the flow and motion estimation problems to provide a direct method for planar surfaces [4, 8] These methods use the multi scale technique to capture large motions while significantly constraining the ....

A. Shashua and N. Navab. Relative Affine Structure: Theory and Application to 3D Reconstruction from Perspective Views. Proc. IEEE CVPR, Seattle, June 1994.

....optimization via Levenberg Marquardt (LM) this algorithm however is unreliable due to its problem with local minima 2 . Other algorithms either require small perspective effects as in Tomasi s approach, or else use a small amount of the available information (typically 2 or 3 image frames) [21, 22, 18, 13, 5, 6, 24] and thus are not robust when more is needed. Our algorithm is also fast. The computation time for a sequence of 15 images with 30 tracked points on an SGI Indigo II is about 1.6 seconds for the first iteration; a single iteration often gives adequate accuracy. This contrast with LM, which ....

....to the next, as long as the basic condition of moderate translational motion is valid. Our algorithm is not limited to infinitesimal motions but gives exact answers for finite motions. Recently, much research in structure from motion has focused on reconstruction from two or three image frames [21, 22, 18, 13, 5, 6], or else from a small number of feature points tracked across many images [24] We call such methods frugal, because they use just a fraction of the information potentially available in MFSFM when there are many (AE 3) images. The advantage of these methods is that the algorithms are ....

A. Shashua, R. Navab, "Relative Affine Structure: Theory and Applications to 3D Reconstruction From Perspective Views," CVPR 483489, Seattle 1994.

.... unknown projective transformation, see for example [10] and [1] Some present reconstruction algorithms rely on particular choices of coordinates in the images and the (unknown) object, e.g. projective or affine coordinates, where some points are sorted out in order to build up a basis, see [1] [8] and [3] This work has been done within the ESPRIT Reactive LTR project 21914, CUMULI and the Swedish Research Council for Engineering Sciences (TFR) project 95 64 222 Other algorithms rely on the so called multilinear constraints. The drawback of the reconstruction algorithms relying on ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In Conf. Computer Vision and Pattern Recognition, pages 483--489, 1994.

.... parallax motion (in the direction of the epipole) This formulation has formed the basis of both our projective structure from motion algorithms [15] and our projective dense depth estimation algorithm [14] More recently, it been used by other researchers under the names of affine depth [12] and planar parallax [10, 7] see Section 4.2 for a more detailed discussion of projective depth) The above formulation extends naturally to multiframe depth recovery by simply associating a separate M j and t j with each frame and minimizing the summed intensity error E = X j 6=0 X i ....

A. Shashua and N. Navab. Relative affine structure: Theory and applications to 3D reconstruction from perspective views. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'94), pages 483--489, Seattle, Washington, June 1994. IEEE Computer Society.

....57, 49] However, the constraints provided by projective geometry have sometimes proven quite weak for some applications. Affine geometry has been found to provide an interesting framework (see for instance [28] where no less than six papers about affine structure can be found, and for example [52, 9, 56, 42, 4, 60, 47]) borrowing some nice characteristics from both Euclidean geometry and projective geometry. However, one can remark that the representations adopted in the literature are of very disparate nature, and that often they are not even minimal. The relationships between different levels of ....

....from the affine depth by applying the transformation TA in (50) whereas the affine depth obtained from the projective depth by applying the transformation TP . It can be noted that the canonical decomposition gives a simple algebraic account of the geometric construction described by Shashua [58, 57, 60]. Using the form of the first projection matrix of the invariant description [I 3 0] we see that given a measurement m in R, the possible 3D points M can be written [m; Depending on which canonical representation we choose, is respectively Shashua s projective, relative affine (see ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proc. Conference on Computer Vision and Pattern Recognition, pages 483--489, Seattle, WA, 1994.

.... enables us to extend the 2D parametric registration approach to general 3D scenes is the following: the plane registration process (using the dominant 2D parametric transformation) removes all effects of camera rotation, zoom, and calibration, without explicitly computing them [15] 18] 28] [29]. The residual image motion after the plane registration is due only to the translational motion of the camera and to the deviations of the scene structure from the planar surface. Hence, the residual motion is an epipolar flow field. This observation has led to the so called plane parallax ....

....is due only to the translational motion of the camera and to the deviations of the scene structure from the planar surface. Hence, the residual motion is an epipolar flow field. This observation has led to the so called plane parallax approach to 3D scene analysis [17] 15] 18] 28] [29]. It can be shown (see [19] 15] 28] 29] that the displacement u of a pixel can be decomposed as follows: u = u ; 4) where u denotes the planar part of the 2D image motion (which aligns a reference plane Pi in the scene) As noted earlier, u can be described by a ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, pages 483--489, Seattle, Wa., June 1994.

....plane and in 3 space, with respect to the first 3 and 4 points, respectively. Thus Y i = Theta I 3 a i A i ; i = 1; m: X 1 = Theta I 4 B ; 17) where I 3 and I 4 are 3 Theta 3 and 4 Theta 4 identity matrices, respectively. Similar coordinates have been used in [20] and [28]. Here and in the sequel we have implicitly made the following assumption. Assumption 2. It is possible to select three non collinear points in the first image, such that in each image, the corresponding points are noncollinear. It is not necessary to use the same three points throughout the ....

Shashua, A., Navab, N., Relative Affine Structure: Theory and Application to 3D Reconstruction from Perspective Views, Proc. Conf. Computer Vision and Pattern Recognition, 1994, pp. 483-489.

....terms the relative affine result requires fewer corresponding points and fewer calculations than the projective framework, and is the only next general framework after projectivewhen working with perspective views. Parts of this work, as it evolved, have been presented in the meetings found in [33,38], and in [27] 2 Notation We consider object space to be the three dimensional projective space P , and image space to be the twodimensional projectivespaceP . An object (or scene) is modeled by a set of points and let i aeP denote views (arbitrary) indexed by i, of the object. ....

....is part of the reference frame and are called relative affine structure . This statement, that all the available degrees of freedom are captured by four points and one projective transformation, was also recently presented in [40] using different notations and tools than those used here and in [33,38]. This middle ground approachhasseveral advantages. First, the results are sharper than a full projective reconstruction approach( 7, 13] where five scene points are needed. The increased sharpness translates to a remarkably simple framework captured by a single equation (Equation 1) Second, ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, 1994.

....in general position. In a dual manner, in affine space a reference plane is minimally necessary for shape representation; in projective space we have the tetrahedron of reference. Work along the lines of representing shape using minimalframe configurations and recovery from views can be found in [9, 8, 4, 27, 28], and in further references therein. As long as we use the minimal configuration of points for representing shape, there is no practical reason to distinguish between reference frames and reference surfaces. The distinction becomes useful, as we shall see later, when we choose non minimal frames; ....

.... in active vision applications [29, 30, 13] as well as in infinitesimal motion models for visual reconstruction [7, 10, 23] A planar reference surface corresponds to the dual case of shape representation under parallel projection (cf. 9] or relative affine structure under perspective projection [26, 28]. In other words, a nominal transformation is either a 2D affine transformation or a 2D projective transformation, depending on whether we assume an orthographic or perspective model of projection. The magnitude of the residual field is thus small in image regions that correspond to object points ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, 1994.

....terms the relative affine result requires fewer corresponding points and fewer calculations than the projective framework, and is the only next general framework after projective when working with perspective views. Parts of this work, as it evolved, have been presented in the meetings found in [33, 38], and in [27] 2 Notation We consider object space to be the three dimensional projective space P 3 , and image space to be the twodimensional projective space P 2 . An object (or scene) is modeled by a set of points and let i ae P 2 denote views (arbitrary) indexed by i, of the object. ....

....is part of the reference frame and are called relative affine structure . This statement, that all the available degrees of freedom are captured by four points and one projective transformation, was also recently presented in [40] using different notations and tools than those used here and in [33, 38]. This middle ground approach has several advantages. First, the results are sharper than a full projective reconstruction approach ( 7, 13] where five scene points are needed. The increased sharpness translates to a remarkably simple framework captured by a single equation (Equation 1) ....

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3D reconstruction from perspective views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Washington, 1994.

....equivalence (mostly of the plane) ii) linear combination of points along a line. We do not use the concepts of internal and external camera parameters, and camera transformation matrix, and provide instead a much simpler and intuitive framework that can be used for other problems (for example, [33, 37, 36]) Most of the theoretical work described here appeared previously in [32, 31] with additional refinements described in [34, 33] II. Notations We consider object space to be the three dimensional projective space P 3 , and image space to be the two dimensional projective space P 2 . Let ....

.... in case we do not have the fifth point, one can choose a different cross ratio and obtain an invariant that is independent of the position of the second camera, that is, the cross ratio computed between the first view and any other view remains fixed [34] This kind of invariance is referred to in [33, 36] as a relative affine invariant and can be made simpler (one reference plane is sufficient) than in the construction presented here. Finally, consider the case of orthographic views. Generally speaking, a projective framework should not make any distinction between the two projections (any point ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, Seattle, Washington, 1994.

....p 2 1 and p 0 2 2 coming from an arbitrary point P 2 P 3 , we have p 0 = Ap kv 0 : The coefficient k is independent of 2 , i.e. is invariant to the choice of the second view. The lemma, its proof and its theoretical and practical implications are discussed in detail in [28, 32]. Note that the particular case where the homography A is affine, and the epipole v 0 is on the line at infinity, corresponds to the construction of affine structure from two orthographic views [17] In a nutshell, a representation R 0 of P 3 (tetrad of coordinates) can always be chosen such ....

....can be shown to be related to R 0 by an element of the affine group. Thus, the scalar k is an affine invariant within a projective framework, and is called a relative affine invariant . A ratio of two such invariants, each corresponding to a different reference plane, is a projective invariant [32]. For our purposes, there is no need to discuss the methods for recovering k all we need is to use the existence of a relative affine invariant k associated with some arbitrary reference plane which, in turn, gives rise to a homography A. Definition 1 Homographies A i 2 PGL 3 from 1 7 ....

[Article contains additional citation context not shown here]

A. Shashua and N. Navab. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In Proceedings IEEE Conf. on Computer Vision and Pattern Recognition, Seattle, Washington, 1994.

No context found.

SHASHUA, A., AND NAVAB, N. 1994. Relative affine structure: theory and application to 3d reconstruction from perspective views. In IEEE Conference on Computer Vision and Pattern Recognition, 483--489.

No context found.

SHASHUA, A., AND NAVAB, N. Relative affine structure: Theory and application to 3d reconstruction from perspective views. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (1994), pp. 483--489.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC