Results 1 - 10
of
145
High Quality Depth Map Upsampling for 3D-TOF Cameras
"... This paper describes an application framework to perform high quality upsampling on depth maps captured from a low-resolution and noisy 3D time-of-flight (3D-ToF) camera that has been coupled with a high-resolution RGB camera. Our framework is inspired by recent work that uses nonlocal means filteri ..."
Abstract
-
Cited by 36 (2 self)
- Add to MetaCart
(Show Context)
This paper describes an application framework to perform high quality upsampling on depth maps captured from a low-resolution and noisy 3D time-of-flight (3D-ToF) camera that has been coupled with a high-resolution RGB camera. Our framework is inspired by recent work that uses nonlocal means filtering to regularize depth maps in order to maintain fine detail and structure. Our framework extends this regularization with an additional edge weighting scheme based on several image features based on the additional high-resolution RGB input. Quantitative and qualitative results show that our method outperforms existing approaches for 3D-ToF upsampling. We describe the complete process for this system, including device calibration, scene warping for input alignment, and even how the results can be further processed using simple user markup. 1.
Local Laplacian Filters: Edge-aware Image Processing with a Laplacian Pyramid
"... The Laplacian pyramid is ubiquitous for decomposing images into multiple scales and is widely used for image analysis. However, because it is constructed with spatially invariant Gaussian kernels, ..."
Abstract
-
Cited by 28 (2 self)
- Add to MetaCart
The Laplacian pyramid is ubiquitous for decomposing images into multiple scales and is widely used for image analysis. However, because it is constructed with spatially invariant Gaussian kernels,
A non-local cost aggregation method for stereo matching
, 2012
"... Matching cost aggregation is one of the oldest and still popular methods for stereo correspondence. While effec-tive and efficient, cost aggregation methods typically aggregate the matching cost by summing/averaging over a user-specified, local support region. This is obviously on-ly locally-optima ..."
Abstract
-
Cited by 19 (2 self)
- Add to MetaCart
Matching cost aggregation is one of the oldest and still popular methods for stereo correspondence. While effec-tive and efficient, cost aggregation methods typically aggregate the matching cost by summing/averaging over a user-specified, local support region. This is obviously on-ly locally-optimal, and the computational complexity of the full-kernel implementation usually depends on the region size. In this paper, the cost aggregation problem is re-examined and a non-local solution is proposed. The match-ing cost values are aggregated adaptively based on pixel similarity on a tree structure derived from the stereo im-age pair to preserve depth edges. The nodes of this tree are all the image pixels, and the edges are all the edges between the nearest neighboring pixels. The similarity between any two pixels is decided by their shortest distance on the tree. The proposed method is non-local as every n-ode receives supports from all other nodes on the tree. As can be expected, the proposed non-local solution outper-forms all local cost aggregation methods on the standard (Middlebury) benchmark. Besides, it has great advantage in extremely low computational complexity: only a total of 2 addition/subtraction operations and 3 multiplication oper-ations are required for each pixel at each disparity level. It is very close to the complexity of unnormalized box filtering using integral image which requires 6 addition/subtraction operations. Unnormalized box filter is the fastest local cost aggregation method but blurs across depth edges. The pro-posed method was tested on a MacBook Air laptop comput-er with a 1.8 GHz Intel Core i7 CPU and 4 GB memory. The average runtime on the Middlebury data sets is about 90 milliseconds, and is only about 1.25 × slower than unnor-malized box filter. A non-local disparity refinement method is also proposed based on the non-local cost aggregation method.
Image fusion with guided filtering
- IEEE Trans. Image Process
, 2013
"... Abstract — A fast and effective image fusion method is proposed for creating a highly informative fused image through merging multiple images. The proposed method is based on a two-scale decomposition of an image into a base layer containing large scale variations in intensity, and a detail layer ca ..."
Abstract
-
Cited by 14 (1 self)
- Add to MetaCart
(Show Context)
Abstract — A fast and effective image fusion method is proposed for creating a highly informative fused image through merging multiple images. The proposed method is based on a two-scale decomposition of an image into a base layer containing large scale variations in intensity, and a detail layer capturing small scale details. A novel guided filtering-based weighted average technique is proposed to make full use of spatial consistency for fusion of the base and detail layers. Experimental results demonstrate that the proposed method can obtain state-of-the-art performance for fusion of multispectral, multifocus, multimodal, and multiexposure images. Index Terms — Guided filter, image fusion, spatial consistency, two-scale decomposition. I.
M.: Patch match filter: Efficient edge-aware filtering meets randomized search for fast correspondence field estimation
- In: CVPR (2013) 16
"... Though many tasks in computer vision can be formu-lated elegantly as pixel-labeling problems, a typical chal-lenge discouraging such a discrete formulation is often due to computational efficiency. Recent studies on fast cost volume filtering based on efficient edge-aware filters have provided a fas ..."
Abstract
-
Cited by 10 (3 self)
- Add to MetaCart
(Show Context)
Though many tasks in computer vision can be formu-lated elegantly as pixel-labeling problems, a typical chal-lenge discouraging such a discrete formulation is often due to computational efficiency. Recent studies on fast cost volume filtering based on efficient edge-aware filters have provided a fast alternative to solve discrete labeling prob-lems, with the complexity independent of the support win-dow size. However, these methods still have to step through the entire cost volume exhaustively, which makes the solu-tion speed scale linearly with the label space size. When the label space is huge, which is often the case for (subpixel-accurate) stereo and optical flow estimation, their compu-tational complexity becomes quickly unacceptable. Devel-oped to search approximate nearest neighbors rapidly, the PatchMatch method can significantly reduce the complex-ity dependency on the search space size. But, its pixel-wise randomized search and fragmented data access within the 3D cost volume seriously hinder the application of efficient cost slice filtering. This paper presents a generic and fast computational framework for general multi-labeling prob-lems called PatchMatch Filter (PMF). For the very first time, we explore effective and efficient strategies to weave together these two fundamental techniques developed in isolation, i.e., PatchMatch-based randomized search and ef-ficient edge-aware image filtering. By decompositing an im-age into compact superpixels, we also propose superpixel-based novel search strategies that generalize and improve the original PatchMatch method. Focusing on dense cor-respondence field estimation in this paper, we demonstrate PMF’s applications in stereo and optical flow. Our PMF methods achieve state-of-the-art correspondence accuracy but run much faster than other competing methods, often giving over 10-times speedup for large label space cases. 1.
C.: Depth recovery using an adaptive color-guided auto-regressive model
- In ECCV
, 2012
"... Abstract. This paper proposes an adaptive color-guided auto-regressive (AR) model for high quality depth recovery from low quality measure-ments captured by depth cameras. We formulate the depth recovery task into a minimization of AR prediction errors subject to measurement con-sistency. The AR pre ..."
Abstract
-
Cited by 7 (1 self)
- Add to MetaCart
(Show Context)
Abstract. This paper proposes an adaptive color-guided auto-regressive (AR) model for high quality depth recovery from low quality measure-ments captured by depth cameras. We formulate the depth recovery task into a minimization of AR prediction errors subject to measurement con-sistency. The AR predictor for each pixel is constructed according to both the local correlation in the initial depth map and the nonlocal similarity in the accompanied high quality color image. Experimental results show that our method outperforms existing state-of-the-art schemes, and is versatile for both mainstream depth sensors: ToF camera and Kinect.
Importance Filtering for Image Retargeting
"... Content-aware image retargeting has attracted a lot of interests recently. The key and most challenging issue for this task is how to balance the tradeoff between preserving the important contents and minimizing the visual distortions on the consistency of the image structure. In this paper we prese ..."
Abstract
-
Cited by 7 (0 self)
- Add to MetaCart
(Show Context)
Content-aware image retargeting has attracted a lot of interests recently. The key and most challenging issue for this task is how to balance the tradeoff between preserving the important contents and minimizing the visual distortions on the consistency of the image structure. In this paper we present a novel filtering-based technique to tackle this issue, called ”importance filtering”. Specifically, we first filter the image saliency guided by the image itself to achieve a structure-consistent importance map. We then use the pixel importance as the key constraint to compute the gradient map of pixel shifts from the original resolution to the target. Finally, we integrate the shift gradient across the image using a weighted filter to construct a smooth shift map and render the target image. The weight is again controlled by the pixel importance. The two filtering processes enforce to maintain the structural consistency and yet preserve the important contents in the target image. Furthermore, the simple nature of filter operations allows highly efficient implementation for real-time applications and easy extension to video retargeting, as the structural constraints from the original image naturally convey the temporal coherence between frames. The effectiveness and efficiency of our importance filtering algorithm are confirmed in extensive experiments. 1.
Multi-View Hair Capture using Orientation Fields
"... We begin with many high-resolution photographs (with unconstrained lighting), compute an orientation field for each, and perform multi-view stereo matching using a metric based on orientation similarity. The resulting depth maps show high-resolution details of hair strands and we integrate them into ..."
Abstract
-
Cited by 6 (4 self)
- Add to MetaCart
We begin with many high-resolution photographs (with unconstrained lighting), compute an orientation field for each, and perform multi-view stereo matching using a metric based on orientation similarity. The resulting depth maps show high-resolution details of hair strands and we integrate them into a single merged model. In contrast, conventional multi-view stereo algorithms and merging techniques [4, 7] fail at capturing the fine hair structures. Reconstructing realistic 3D hair geometry is challenging due to omnipresent occlusions, complex discontinuities and specular appearance. To address these challenges, we propose a multi-view hair reconstruction algorithm based on orientation fields with structure-aware aggregation. Our key insight is that while hair’s color appearance is viewdependent, the response to oriented filters that captures the local hair orientation is more stable. We apply the structure-aware aggregation to the MRF matching energy to enforce the structural continuities implied from the local hair orientations. Multiple depth maps from the MRF optimization are then fused into a globally consistent hair geometry with a template refinement procedure. Compared to the state-of-the-art color-based methods, our method faithfully reconstructs detailed hair structures. We demonstrate the results for a number of hair styles, ranging from straight to curly, and show that our framework is suitable for capturing hair in motion. 1
Linear stereo matching
"... Recent local stereo matching algorithms based on an adaptive-weight strategy achieve accuracy similar to global approaches. One of the major problems of these algorithms is that they are computationally expensive and this complexity increases proportionally to the window size. This paper proposes a ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
(Show Context)
Recent local stereo matching algorithms based on an adaptive-weight strategy achieve accuracy similar to global approaches. One of the major problems of these algorithms is that they are computationally expensive and this complexity increases proportionally to the window size. This paper proposes a novel cost aggregation step with complexity independent of the window size (i.e. O(1)) that outperforms state-of-the-art O(1) methods. Moreover, compared to other O(1) approaches, our method does not rely on integral histograms enabling aggregation using colour images instead of grayscale ones. Finally, to improve the results of the proposed algorithm a disparity refinement pipeline is also proposed. The overall algorithm produces results comparable to those of state-of-the-art stereo matching algorithms. 1.
Joint Geodesic Upsampling of Depth Images
"... We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
(Show Context)
We propose an algorithm utilizing geodesic distances to upsample a low resolution depth image using a registered high resolution color image. Specifically, it computes depth for each pixel in the high resolution image using geodesic paths to the pixels whose depths are known from the low resolution one. Though this is closely related to the all-pair-shortest-path problem which has O(n2 log n) complexity, we develop a novel approximation algorithm whose com-plexity grows linearly with the image size and achieve real-time performance. We compare our algorithm with the state of the art on the benchmark dataset and show that our approach provides more accurate depth upsampling with fewer artifacts. In addition, we show that the proposed al-gorithm is well suited for upsampling depth images using binary edge maps, an important sensor fusion application. 1.
