Results 11 - 20
of
138
Segment-based stereo matching using graph cuts
, 2004
"... In this paper, we present a new segment-based stereo matching algorithm using graph cuts. In our approach, the reference image is divided into non-overlapping homogeneous segments and the scene structure is represented as a set of planes in the disparity space. The stereo matching problem is formula ..."
Abstract
-
Cited by 79 (0 self)
- Add to MetaCart
(Show Context)
In this paper, we present a new segment-based stereo matching algorithm using graph cuts. In our approach, the reference image is divided into non-overlapping homogeneous segments and the scene structure is represented as a set of planes in the disparity space. The stereo matching problem is formulated as an energy minimization problem in the segment domain instead of the traditional pixel domain. Graph cuts technique is used to fast approximate the optimal solution, which assigns the corresponding disparity plane to each segment. Experiments demonstrate that the performance of our algorithm is comparable to the state-ofthe-art stereo algorithms on various data sets. Furthermore, strong performance is achieved in the conventionally difficult areas such as: textureless regions, disparity discontinuous boundaries and occluded portions. 1.
Surfaces with occlusions from layered stereo
- IEEE Trans. on PAMI
, 2004
"... Abstract—We propose a new binocular stereo algorithm that estimates scene structure as a collection of smooth surface patches. The disparities within each patch are modeled by a continuous-valued spline, while the extent of each patch is represented via a pixelwise partitioning of the images. Dispar ..."
Abstract
-
Cited by 76 (2 self)
- Add to MetaCart
Abstract—We propose a new binocular stereo algorithm that estimates scene structure as a collection of smooth surface patches. The disparities within each patch are modeled by a continuous-valued spline, while the extent of each patch is represented via a pixelwise partitioning of the images. Disparities and extents are alternately estimated in an iterative, energy minimization framework. Experimental results demonstrate that, for scenes consisting of smooth surfaces, the proposed algorithm significantly improves upon the state of the art. Index Terms—Binocular stereo vision, energy minimization, graph cuts, hybrid system, smooth surfaces, surface fitting, boundary localization, sharp discontinuities, quantitative comparison. 1
Manhattan-world Stereo
"... Multi-view stereo (MVS) algorithms now produce reconstructions that rival laser range scanner accuracy. However, stereo algorithms require textured surfaces, and therefore work poorly for many architectural scenes (e.g., building interiors with textureless, painted walls). This paper presents a nove ..."
Abstract
-
Cited by 73 (7 self)
- Add to MetaCart
(Show Context)
Multi-view stereo (MVS) algorithms now produce reconstructions that rival laser range scanner accuracy. However, stereo algorithms require textured surfaces, and therefore work poorly for many architectural scenes (e.g., building interiors with textureless, painted walls). This paper presents a novel MVS approach to overcome these limitations for Manhattan World scenes, i.e., scenes that consists of piece-wise planar surfaces with dominant directions. Given a set of calibrated photographs, we first reconstruct textured regions using an existing MVS algorithm, then extract dominant plane directions, generate plane hypotheses, and recover per-view depth maps using Markov random fields. We have tested our algorithm on several datasets ranging from office interiors to outdoor buildings, and demonstrate results that outperform the current state of the art for such texture-poor scenes. 1.
Piecewise Planar Stereo for Image-based Rendering
"... We present a novel multi-view stereo method designed for image-based rendering that generates piecewise planar depth maps from an unordered collection of photographs. First a discrete set of 3D plane candidates are computed based on a sparse point cloud of the scene (recovered by structure from moti ..."
Abstract
-
Cited by 54 (7 self)
- Add to MetaCart
(Show Context)
We present a novel multi-view stereo method designed for image-based rendering that generates piecewise planar depth maps from an unordered collection of photographs. First a discrete set of 3D plane candidates are computed based on a sparse point cloud of the scene (recovered by structure from motion) and sparse 3D line segments reconstructed from multiple views. Next, evidence is accumulated for each plane using 3D point and line incidence and photo-consistency cues. Finally, a piecewise planar depth map is recovered for each image by solving a multi-label Markov Random Field (MRF) optimization problem using graph-cuts. Our novel energy minimization formulation exploits high-level scene information. It incorporates geometric constraints derived from vanishing directions, enforces free space violation constraints based on ray visibility of 3D points and 3D lines and imposes smoothness priors specific to planes that intersect. We demonstrate the effectiveness of our approach on a wide variety of outdoor and indoor datasets. The view interpolation results are perceptually pleasing, as straight lines are preserved and holes are minimized even for challenging scenes with non-Lambertian and textureless surfaces. 1.
Spatially coherent clustering using graph cuts
- In CVPR (2
, 2004
"... Feature space clustering is a popular approach to image segmentation, in which a feature vector of local properties (such as intensity, texture or motion) is computed at each pixel. The feature space is then clustered, and each pixel is labeled with the cluster that contains its feature vector. A ma ..."
Abstract
-
Cited by 52 (1 self)
- Add to MetaCart
(Show Context)
Feature space clustering is a popular approach to image segmentation, in which a feature vector of local properties (such as intensity, texture or motion) is computed at each pixel. The feature space is then clustered, and each pixel is labeled with the cluster that contains its feature vector. A major limitation of this approach is that feature space clusters generally lack spatial coherence (i.e., they do not correspond to a compact grouping of pixels). In this paper, we propose a segmentation algorithm that operates simultaneously in feature space and in image space. We define an energy function over both a set of clusters and a labeling of pixels with clusters. In our framework, a pixel is labeled with a single cluster (rather than, for example, a distribution
A Layered Stereo Algorithm Using Image Segmentation And Global Visibility Constraints
- in IEEE International Conference on Image Processing
, 2004
"... We propose a new stereo algorithm which uses colour segmentation to allow the handling of large untextured regions and precise localization of depth boundaries. Each segment is modelled as a plane. Robustness of the depth representation is achieved by the use of a layered model. Layers are extracted ..."
Abstract
-
Cited by 44 (4 self)
- Add to MetaCart
(Show Context)
We propose a new stereo algorithm which uses colour segmentation to allow the handling of large untextured regions and precise localization of depth boundaries. Each segment is modelled as a plane. Robustness of the depth representation is achieved by the use of a layered model. Layers are extracted by mean-shift-based clustering of depth planes. For layer assignment a global cost function is defined. The quality of the disparity map is measured by warping the reference image to the second view and comparing it with the real image. Z-buffering enforces visibility and allows the explicit detection of occlusions. An efficient greedy algorithm searches for a local minimum of the cost function. Layer extraction and assignment are alternately applied. Results obtained for benchmark and self-recorded images indicate that the proposed algorithm can compete with the state-of-the-art.
Graph Cuts in Vision and Graphics: Theories and Applications
- “MATH. MODELS OF C.VISION: THE HANDBOOK”, EDTS. PARAGIOS, CHEN, FAUGERAS
"... Combinatorial min-cut algorithms on graphs emerged as an increasingly useful tool for problems in vision. Typically, the use of graphcuts is motivated by one of the following two reasons. Firstly, graph-cuts allow geometric interpretation; under certain conditions a cut on a graph can be seen as a ..."
Abstract
-
Cited by 38 (2 self)
- Add to MetaCart
Combinatorial min-cut algorithms on graphs emerged as an increasingly useful tool for problems in vision. Typically, the use of graphcuts is motivated by one of the following two reasons. Firstly, graph-cuts allow geometric interpretation; under certain conditions a cut on a graph can be seen as a hypersurface in N-D space embedding the corresponding graph. Thus, many applications in vision and graphics use min-cut algorithms as a tool for computing optimal hypersurfaces. Secondly, graphcuts also work as a powerful energy minimization tool for a fairly wide class of binary and non-binary energies that frequently occur in early vision. In some cases graph cuts produce globally optimal solutions. More generally, there are iterative graph-cut based techniques that produce provably good approximations which (were empirically shown to) correspond to high-quality solutions in practice. Thus, another large group of applications use graph-cuts as an optimization technique for low-level vision problems based on global energy formulations. This chapter is intended as a tutorial illustrating these two aspects of graph-cuts in the context of problems in computer vision and graphics. We explain general theoretical properties that motivate the use of graph cuts, as well as, show their limitations.
Shape and the stereo correspondence problem
- International Journal of Computer Vision
, 2005
"... We examine the implications of shape on the process of finding dense correspondence and half-occlusions for a stereo pair of images. The desired property of the disparity map is that it should be a piecewise continuous function which is consistent with the images and which has the minimum number of ..."
Abstract
-
Cited by 35 (1 self)
- Add to MetaCart
(Show Context)
We examine the implications of shape on the process of finding dense correspondence and half-occlusions for a stereo pair of images. The desired property of the disparity map is that it should be a piecewise continuous function which is consistent with the images and which has the minimum number of discontinuities. To zeroth order, piecewise continuity becomes piecewise constancy. Using this approximation, we first discuss an approach for dealing with such a fronto-parallel shapeless world, and the problems involved therein. We then introduce horizontal and vertical slant to create a first order approximation to piecewise continuity. In particular, we emphasize the following geometric fact: a horizontally slanted surface (i.e., having depth variation in the direction of the separation of the two cameras) will appear horizontally stretched in one image as compared to the other image. Thus, while corresponding two images, N pixels on a scanline in one image may correspond to a different number of pixels M in the other image. This leads to three important modifications to existing stereo algorithms: (a) due to unequal sampling, existing intensity matching metrics must be modified, (b) unequal numbers of pixels in the two images must be allowed to correspond to each other, and (c) the uniqueness constraint, which is often used for detecting occlusions, must be changed to an interval uniqueness constraint. We also discuss the asymmetry between vertical and horizontal slant, and the central role of non-horizontal edges in the context of vertical slant. Using experiments, we discuss cases where existing algorithms fail, and how the incorporation of these new constraints provides correct results. 1.
A Layered Stereo Matching Algorithm using Image Segmentation and Global Visibility Constraints,”
- ISPRS Journal of Photogrammetry and Remote Sensing,
, 2005
"... Abstract This work describes a stereo algorithm that takes advantage of image segmentation, assuming that disparity varies smoothly inside a segment of homogeneous colour and depth discontinuities coincide with segment borders. Image segmentation allows our method to generate correct disparity esti ..."
Abstract
-
Cited by 35 (6 self)
- Add to MetaCart
(Show Context)
Abstract This work describes a stereo algorithm that takes advantage of image segmentation, assuming that disparity varies smoothly inside a segment of homogeneous colour and depth discontinuities coincide with segment borders. Image segmentation allows our method to generate correct disparity estimates in large untextured regions and precisely localize depth boundaries. The disparity inside a segment is represented by a planar equation. To derive the plane model, an initial disparity map is generated. We use a window-based approach that exploits the results of segmentation. The size of the match window is chosen adaptively. A segment's planar model is then derived by robust least squared error fitting using the initial disparity map. In a layer extraction step, disparity segments that are found to be similar according to a plane dissimilarity measurement are combined to form a single robust layer. We apply a modified mean-shift algorithm to extract clusters of similar disparity segments. Segments of the same cluster build a layer, the plane parameters of which are computed from its spatial extent using the initial disparity map. We then optimize the assignment of segments to layers using a global cost function. The quality of the disparity map is measured by warping the reference image to the second view and comparing it with the real image. Z-buffering enforces visibility and allows the explicit detection of occlusions. The cost function measures the colour dissimilarity between the warped and real views, and penalizes occlusions and neighbouring segments that are assigned to different layers. Since the problem of finding the assignment of segments to layers that minimizes this cost function is N P-complete, an efficient greedy algorithm is applied to find a local minimum. Layer extraction and assignment are alternately applied. Qualitative and quantitative results obtained for benchmark image pairs show that the proposed algorithm outperforms most state-of-the-art matching algorithms currently listed on the Middlebury stereo evaluation website. The technique achieves particularly good results in areas with depth discontinuities and related occlusions, where missing stereo information is substituted from surrounding regions. Furthermore, we apply the algorithm to a self-recorded image set and show 3D visualizations of the derived results. D
Stereo correspondence with slanted surfaces: Critical implications of horizontal slant
- In Proc. CVPR
, 2004
"... We examine the stereo correspondence problem in the presence of slanted scene surfaces. In particular, we highlight a previously overlooked geometric fact: a horizontally slanted surface (i.e. having depth variation in the direction of the separation of the two cameras) will appear horizontally stre ..."
Abstract
-
Cited by 34 (4 self)
- Add to MetaCart
(Show Context)
We examine the stereo correspondence problem in the presence of slanted scene surfaces. In particular, we highlight a previously overlooked geometric fact: a horizontally slanted surface (i.e. having depth variation in the direction of the separation of the two cameras) will appear horizontally stretched in one image as compared to the other image. Thus, while corresponding two images, N pixels on a scanline in one image may correspond to a different number of pixels M in the other image. This leads to three important modifications to existing stereo algorithms: (a) due to unequal sampling, intensity matching metrics such as the popular Birchfield-Tomasi procedure must be modified, (b) unequal numbers of pixels in the two images must be allowed to correspond to each other, and (c) the uniqueness constraint, which is often used for detecting occlusions, must be changed to a 3D uniqueness constraint. This paper discusses these new constraints and provides a simple scanline based matching algorithm for illustration. We experimentally demonstrate test cases where existing algorithms fail, and how the incorporation of these new constraints provides correct results. Experimental comparisons of the scanline based algorithm with standard data sets are also provided. 1.
