In this paper, we make contact with the field of nonparametric statistics and present a development and generalization of tools and results for use in image processing and reconstruction. In particular, we adapt and expand kernel regression ideas for use in image denoising, upscaling, interpolation, fusion, and more. Furthermore, we establish key relationships with some popular existing methods and show how several of these algorithms, including the recently popularized bilateral filter, are special cases of the proposed framework. The resulting algorithms and analyses are amply illustrated with practical examples.

Effective image prior is necessary for image super resolution, due to its severely under-determined nature. Although the edge smoothness prior can be effective, it is generally difficult to have analytical forms to evaluate the edge smoothness, especially for soft edges that exhibit gradual intensity transitions. This paper finds the connection between the soft edge smoothness and a soft cut metric on an image grid by generalizing the Geocuts method [5], and proves that the soft edge smoothness measure approximates the average length of all level lines in an intensity image. This new finding not only leads to an analytical characterization of the soft edge smoothness prior, but also gives an intuitive geometric explanation. Regularizing the super resolution problem by this new form of prior can simultaneously minimize the length of all level lines, and thus resulting in visually appealing results. In addition, this paper presents a novel combination of this soft edge smoothness prior and the alpha matting technique for color image super resolution, by normalizing edge segments with their alpha channel description, to achieve a unified treatment of edges with different contrast and scale. 1.

Theoretical and practical limitations usually constrain the achievable resolution of any imaging device. Super-Resolution (SR) methods are developed through the years to go beyond this limit by acquiring and fusing several low-resolution (LR) images of the same scene, producing a high-resolution (HR) image. The early works on SR, although occasionally mathematically optimal for particular models of data and noise, produced poor results when applied to real images. In this paper, we discuss two of the main issues related to designing a practical SR system, namely reconstruction accuracy and computational efficiency. Reconstruction accuracy refers to the problem of designing a robust SR method applicable to images from different imaging systems. We study a general framework for optimal reconstruction of images from grayscale, color, or color filtered (CFA) cameras. The performance of our proposed method is boosted by using powerful priors and is robust to both measurement (e.g. CCD read out noise) and system noise (e.g. motion estimation error). Noting that the motion estimation is often considered a bottleneck in terms of SR performance, we introduce the concept of “constrained motions” for enhancing the quality of super-resolved images. We show that using such constraints will enhance the quality of the motion estimation and therefore results in more accurate reconstruction of the HR images. We also justify some practical assumptions that greatly reduce the computational complexity and memory requirements of the proposed methods. We use efficient approximation of the Kalman Filter (KF) and adopt a dynamic point of view to the SR problem. Novel methods for addressing these issues are accompanied by experimental results on real data. 1.

Abstract—Single sensor digital color still/video cameras capture images using a color filter array (CFA) and require color interpolation (demosaicking) to reconstruct full color images. The color reproduction has to combat sensor noises which are channel dependent. If untreated in demosaicking, sensor noises can cause color artifacts that are hard to remove later by a separate denoising process, because the demosaicking process complicates the noise characteristics by blending noises of different color channels. This paper presents a joint demosaicking-denoising approach to overcome this difficulty. The color image is restored from noisy mosaic data in two steps. First, the difference signals of color channels are estimated by linear minimum mean square-error estimation. This process exploits both spectral and spatial correlations to simultaneously suppress sensor noise and interpolation error. With the estimated difference signals, the full resolution green channel is recovered. The second step involves in a wavelet-based denoising process to remove the CFA channel-dependent noises from the reconstructed green channel. The red and blue channels are subsequently recovered. Simulated and real CFA mosaic data are used to evaluate the performance of the proposed joint demosaicking-denoising scheme and compare it with many recently developed sophisticated demosaicking and denoising schemes. Index Terms—Bayer pattern, color demosaicking, color filter array (CFA), denoising, wavelet. I.

We address the dynamic super-resolution (SR) problem of reconstructing a high-quality set of monochromatic or color superresolved images from low-quality monochromatic, color, or mosaiced frames. Our approach includes a joint method for simultaneous SR, deblurring, and demosaicing, this way taking into account practical color measurements encountered in video sequences. For the case of translational motion and common space-invariant blur, the proposed method is based on a very fast and memory efficient approximation of the Kalman filter (KF). Experimental results on both simulated and real data are supplied, demonstrating the presented algorithms, and their strength. Copyright © 2006 Hindawi Publishing Corporation. All rights reserved. 1.

In this paper we propose a maximum a posteriori (MAP) framework for the super resolution problem, i.e. reconstructing high-resolution images from shifted, rotated, low-resolution degraded observations. The main contributions of this work are two; first, the use of a new locally adaptive edge preserving prior for the super resolution problem. Second an efficient two-step reconstruction methodology that includes first an initial registration using only the low resolution degraded observations. This is followed by a fast iterative algorithm implemented in the discrete Fourier transform domain in which the restoration, interpolation and the registration subtasks of this problem are preformed simultaneously. We present examples with both synthetic and real data that demonstrate the advantages of the proposed framework.

We address the problem of estimating the relative motion between the frames of a video sequence. In comparison with the commonly applied pairwise image registration methods, we consider global consistency conditions for the overall multi-frame motion estimation problem, which is more accurate. We review the recent work on this subject and propose an optimal framework, which can apply the consistency conditions as both hard constraints in the estimation problem, or as soft constraints in the form of stochastic (Bayesian) priors. The framework is applicable to virtually any motion model and enables us to develop a robust approach, which is resilient against the effects of outliers and noise. The effectiveness of the proposed approach is confirmed by a super-resolution application on synthetic and real data sets. 1.

Most digital cameras capture color pictures in the form of an image mosaic, recording only one color channel at each pixel position. Therefore, an interpolation algorithm needs to be applied to reconstruct the missing color information. In this paper we present a novel Bayer pattern demosaicing approach, employing stochastic global optimization performed on a pixel neighborhood. We are minimizing a newly developed cost function that increases smoothness along one-dimensional image features. While previous algorithms have been developed focusing on LDR images only, our optimization scheme and the underlying cost function are designed to handle both LDR and HDR images, creating less demosaicing artifacts, compared to previous approaches. 1.

Abstract not found

Abstract—Existing learning-based face super-resolution (hallucination) techniques generate high-resolution images of a single facial modality (i.e., at a fixed expression, pose and illumination) given one or set of low-resolution face images as probe. Here, we present a generalized approach based on a hierarchical tensor (multilinear) space representation for hallucinating high-resolution face images across multiple modalities, achieving generalization to variations in expression and pose. In particular, we formulate a unified tensor which can be reduced to two parts: a global image-based tensor for modeling the mappings among different facial modalities, and a local patch-based multiresolution tensor for incorporating high-resolution image details. For realistic hallucination of unregistered low-resolution faces contained in raw images, we develop an automatic face alignment algorithm capable of pixel-wise alignment by iteratively warping the probing face to its projection in the space of training face images. Our experiments show not only performance superiority over existing benchmark face super-resolution techniques on single modal face hallucination, but also novelty of our approach in coping with multimodal hallucination and its robustness in automatic alignment under practical imaging conditions. Index Terms—Face hallucination, super-resolution, tensor. I. PROBLEM STATEMENT