#### DMCA

## Multi-frame demosaicing and super-resolution of color images (2006)

### Cached

### Download Links

- [cs.ucsc.edu]
- [www.cse.ucsc.edu]
- [www.duke.edu]
- [users.soe.ucsc.edu]
- [www.soe.ucsc.edu]
- [users.soe.ucsc.edu]
- [www.cs.technion.ac.il]
- [www.cse.ucsc.edu]
- [www.cse.ucsc.edu]
- [cs.ucsc.edu]
- [www.duke.edu]
- [www.cs.technion.ac.il]
- [users.soe.ucsc.edu]
- [www.soe.ucsc.edu]
- [www-personal.umich.edu]
- [users.soe.ucsc.edu]
- [www.dtic.mil]

Venue: | IEEE Trans. on Image Processing |

Citations: | 52 - 8 self |

### Citations

2267 | Nonlinear total variation based noise removal algorithms
- Rudin, Osher, et al.
- 1992
(Show Context)
Citation Context ...izes energy in the higher frequencies of the solution, opting for a smooth and, hence, blurry image. To achieve reconstructed images with sharper edges, in the spirit of the total variation criterion =-=[32]-=-, [33] and a related method called the bilateral filter9 [34], [35], a robust regularizer called bilateral-TV (B-TV) was introduced in [3]. The B-TV regularizing function looks like 9 Note that by ado... |

1153 | Bilateral Filtering for Gray and Color Images
- Tomasi, Manduchi
- 1998
(Show Context)
Citation Context ... for a smooth and, hence, blurry image. To achieve reconstructed images with sharper edges, in the spirit of the total variation criterion [32], [33] and a related method called the bilateral filter9 =-=[34]-=-, [35], a robust regularizer called bilateral-TV (B-TV) was introduced in [3]. The B-TV regularizing function looks like 9 Note that by adopting a different realization of the bilateral filter, [27] h... |

662 | Hierarchical model-based motion estimation
- Bergen, Anandan, et al.
- 1992
(Show Context)
Citation Context ...the relative motion between these images followed the translational model, we only needed to estimate the motion between the luminance components of these images [43]. We used the method described in =-=[44]-=- to computed the motion vectors. The shift-and-add result (resolution enhancement factor of 4) is shown in Fig. 8(b) [zoomed in Fig. 8(f)]. In Fig. 8(c) [zoomed in Fig. 8(g)], the method of [3] is use... |

366 |
Super-resolution image reconstruction, a technical overview
- Park, Park, et al.
- 2003
(Show Context)
Citation Context ...mit can be achieved by acquiring and fusing several low-resolution (LR) images of the same scene, producing high-resolution (HR) images; this is the basic idea behind super-resolution techniques [1], =-=[2]-=-, [3], [4]. In the last two decades a variety of super-resolution methods have been proposed for estimating the HR image from a set of LR images. Early works on SR showed that the aliasing effects in ... |

346 |
Improving resolution by image registration
- Irani, Peleg
- 1991
(Show Context)
Citation Context ...y of motion estimation. Another approach is transforming the problem to a different color space, where chrominance layers are separated from luminance, and SR is applied only to the luminance channel =-=[7]-=-. Both of these methods are suboptimal as they do not fully exploit the correlation across the color bands. In Section VI, we show that ignoring the relation between different color channels will resu... |

270 | Fast and robust multi-frame super-resolution
- Farsiu, Robinson, et al.
- 2004
(Show Context)
Citation Context ...ion of SR in that early work, in general, SR is a computationally complex and numerically ill-behaved problem in many instances [6]. In recent years, more sophisticated SR methods were developed (see =-=[3]-=- and [6]–[10] as representative works). 2 Three CCD cameras which measure each color field independently tend to be relatively more expensive. Note that almost all super-resolution methods to date hav... |

269 |
Multi-frame image restoration and registration
- Huang, Tsai
- 1984
(Show Context)
Citation Context .... Early works on SR showed that the aliasing effects in the LR images enable the recovery of the HR-fused image, provided that a relative subpixel motion exists between the under-sampled input images =-=[5]-=-. However, in contrast to the clean and practically naive frequency domain description of SR in that early work, in general, SR is a computationally complex and numerically ill-behaved problem in many... |

265 | Restoration of single super-resolution image from several blurred, noisy and down-sampled measured images
- Elad, Feuer
- 1997
(Show Context)
Citation Context ...super-resolution and demosaicing models are special cases of the general model presented above. In particular, in the super-resolution literature the effect of color-filtering is usually ignored [3], =-=[9]-=-, [10] and, therefore, the model is simplified to In this model, the LR images and the HR image are assumed to be monochromatic. On the other hand, in the demosaicing literature only single frame reco... |

199 |
On the bilateral filter and ways to improve it
- Elad
- 2002
(Show Context)
Citation Context ... smooth and, hence, blurry image. To achieve reconstructed images with sharper edges, in the spirit of the total variation criterion [32], [33] and a related method called the bilateral filter9 [34], =-=[35]-=-, a robust regularizer called bilateral-TV (B-TV) was introduced in [3]. The B-TV regularizing function looks like 9 Note that by adopting a different realization of the bilateral filter, [27] has pro... |

176 | Color plane interpolation using alternating projections
- Gunturk, Altunbasak, et al.
- 2002
(Show Context)
Citation Context ...inance interpolation is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are [20], [21], =-=[22]-=-, [23], [24], [25], and [26]. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions: 1) In the constructed image with the mosaicing pattern, there are ... |

152 | The digital TV filter and nonlinear denoising
- Chan, Osher, et al.
- 2001
(Show Context)
Citation Context ...nergy in the higher frequencies of the solution, opting for a smooth and, hence, blurry image. To achieve reconstructed images with sharper edges, in the spirit of the total variation criterion [32], =-=[33]-=- and a related method called the bilateral filter9 [34], [35], a robust regularizer called bilateral-TV (B-TV) was introduced in [3]. The B-TV regularizing function looks like 9 Note that by adopting ... |

111 | Demosaicing: Image reconstruction from color CCD samples
- Kimmel
(Show Context)
Citation Context ...the green channel and the first derivative of the red (or blue) channels are used to estimate the edge direction in the green channel. The smooth hue and gradient-based methods were later combined in =-=[16]-=-. In this iterative method, the smooth hue interpolation is done with respect to the local gradients computed in eight directions about a pixel of interest. A second stage using anisotropic inverse di... |

106 | Adaptive homogeneity-directed demosaicing algorithm
- Hirakawa, Parks
- 2003
(Show Context)
Citation Context ...erpolation step, while adaptive median filtering is used as the enhancement step. A different iterative implementation of the median filters is used as the enhancement step of the method described in =-=[18]-=- that take advantage of a homogeneity assumption in the neighboring pixels. Iterative MAP methods form another important category of demosaicing methods. A MAP algorithm with a smooth chrominance prio... |

90 | Multi-sensor super resolution
- Zomet, Peleg
- 2001
(Show Context)
Citation Context ...ly where the scalar is the step size. (17) V. RELATED METHODS As mentioned earlier, there has been very little work on the problem we have posed here. One related paper is the work of Zomet and Peleg =-=[39]-=-, who have recently proposed a novel method for combining the information from multiple sensors, which can also be used for demosaicing purposes. Although their method has produced successful results ... |

87 | Demosaicking methods for the Bayer color arrays
- Ramanath, Snyder, et al.
- 2002
(Show Context)
Citation Context ...those measured images looks like, one of these Bayer filtered LR images is reconstructed by the method in [14] and shown in Fig. 4(b). The above method is implemented on Kodak DCS-200 digital cameras =-=[41]-=-, so each LR image may be thought of as one picture taken with this 12 SNR is defined as IH ��� @' a' A, where ' , ' are variance of a clean frame and noise, respectively.s152 IEEE TRANSACTIONS ON IMA... |

77 | Effective color interpolation in CCD color filter arrays using signal correlation
- Pei, Tam
- 2003
(Show Context)
Citation Context ... luminance component of the image than the details in chrominance component [20]. 6) The human eye is more sensitive to chromatic changes in the low spatial frequency region than the luminance change =-=[24]-=-. 7) Interpolation should be preformed along and not across the edges. 8) Different color bands are correlated with each other. 9) Edges should align between color channels. Note that even the most po... |

67 |
Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients,” United States Patent 5,373,322
- Laroche, Prescott
- 1994
(Show Context)
Citation Context ...nsition assumption is logical for smooth regions of the reconstructed image, it is not successful in the high-frequency (edge) areas. Considering this fact, gradient-based methods, first addressed in =-=[14]-=-, do not preform interpolation across the edges of an image. This non-iterative method uses the second derivative of the Red and Blue channels to estimate the edge direction in the Green channel. Late... |

60 | Bayesian image super-resolution
- Tipping, Bishop
- 2002
(Show Context)
Citation Context ... common cost functions to measure the closeness of the final solution to the measured data is the least-squares (LS) cost function, which minimizes the norm of the residual vector (see [9], [10], and =-=[36]-=- as representative works). For the case where the noise is additive white, zero mean Gaussian, this approach has the interpretation of providing the maximum likelihood (ML) estimate of [9]. However, a... |

60 | Super-resolution from image sequences - a review
- Borman, Stevenson
- 1998
(Show Context)
Citation Context ...is limit can be achieved by acquiring and fusing several low-resolution (LR) images of the same scene, producing high-resolution (HR) images; this is the basic idea behind super-resolution techniques =-=[1]-=-, [2], [3], [4]. In the last two decades a variety of super-resolution methods have been proposed for estimating the HR image from a set of LR images. Early works on SR showed that the aliasing effect... |

52 |
Color filter array demosaicking: new method and performance measures
- Chang, Tan
- 2002
(Show Context)
Citation Context ...pic inverse diffusion will further enhance the quality of the reconstructed image. This two step approach of interpolation followed by an enhancement step has been used in many other publications. In =-=[16]-=-, spatial and spectral correlations among neighboring pixels are exploited to define the interpolation step, while adaptive median filtering is used as the enhancement step. A different iterative impl... |

46 | Primary-consistent soft-decision color demosaic for digital cameras
- Wu, Zhang
- 2003
(Show Context)
Citation Context ...on is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are [20], [21], [22], [23], [24], =-=[25]-=-, and [26]. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions: 1) In the constructed image with the mosaicing pattern, there are more green sensors... |

44 |
Improving image resolution using subpixel motion,” CVGIP:Graph
- Peleg, Keren, et al.
- 1992
(Show Context)
Citation Context ...that early work, in general SR is a computationally complex and numerically ill-behaved problem in many instances [6]. In recent years more sophisticated SR methods were developed (See [3], [6], [7], =-=[8]-=-, [9], [10] as representative works). Note that almost all super-resolution methods to date have been designed to increase the resolution of a single channel (monochromatic) image. A related problem, ... |

42 | Efficient generalized cross-validation with applications to parametric image restoration and resolution enhancement
- Nguyen, Milanfar, et al.
- 2001
(Show Context)
Citation Context ...inst the second term (regularization cost). Generally speaking, choosing could be either done manually, using visual inspection, or automatically using methods like generalized cross-validation [29], =-=[30]-=-, L-curve [31], or other techniques. How to choose such regularization parameters is in itself a vast topic, which we will not treat in the present paper. C. Monochromatic Spatial Regularization Tikho... |

39 | Resolution enhancement of monochrome and color video using motion compensation
- Tom, Katsaggelos
- 2001
(Show Context)
Citation Context ...resolution. To date, there is very little work addressing the problem of color SR. The typical solution involves applying monochromatic SR algorithms to each of the color channels independently [11], =-=[12]-=-, while using the color information to improve the accuracy of motion estimation. Another approach is transforming the problem to a different color space, where chrominance layers are separated from l... |

39 | Adaptive demosaicking
- Ramanath, Snyder
- 2003
(Show Context)
Citation Context ...The luminance interpolation is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are [21]–=-=[27]-=-. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions. 1) In the constructed image with the mosaicing pattern, there are more green sensors with regu... |

33 |
Resolution enhancement of color video sequences
- Shah, Zakhor
- 1999
(Show Context)
Citation Context ...atial resolution. To date, there is very little work addressing the problem of color SR. The typical solution involves applying monochromatic SR algorithms to each of the color channels independently =-=[11]-=-, [12], while using the color information to improve the accuracy of motion estimation. Another approach is transforming the problem to a different color space, where chrominance layers are separated ... |

33 |
Improving resolution by image registration,” CVGIP:Graph
- Irani, Peleg
- 1991
(Show Context)
Citation Context ...R in that early work, in general SR is a computationally complex and numerically ill-behaved problem in many instances [6]. In recent years more sophisticated SR methods were developed (See [3], [6], =-=[7]-=-, [8], [9], [10] as representative works). Note that almost all super-resolution methods to date have been designed to increase the resolution of a single channel (monochromatic) image. A related prob... |

33 | Motion from color
- Golland, Bruckstein
- 1997
(Show Context)
Citation Context ...h standard deviation equal to one. As the relative motion between these images followed the translational model, we only needed to estimate the motion between the luminance components of these images =-=[42]-=-. We used the method described in [43] to computed the motion vectors. The Shift-And-Add result (resolution enhancement factor of 4) is shown in Figure 8.b (zoomed in Figure 8.f). In Figure 8.c (zoome... |

32 |
Adaptive color plan interpolation in single sensor color electronic camera,” United States Patent 5,629,734
- Hamilton, Adams
- 1997
(Show Context)
Citation Context ... values in the Red and Blue channels. 3 This is the scenario for the more expensive 3-CCD cameras. 4 This is the scenario for cheaper 1-CCD cameras. 5sA variation of this method was later proposed in =-=[15]-=-, where the second derivative of the Green channel and the first derivative of the Red (or Blue) channels are used to estimate the edge direction in the Green channel. The smooth hue and gradient base... |

31 |
Super-resolution from image sequences–A review
- Borman, Stevenson
- 1998
(Show Context)
Citation Context ...is limit can be achieved by acquiring and fusing several low-resolution (LR) images of the same scene, producing high-resolution (HR) images; this is the basic idea behind super-resolution techniques =-=[1]-=-–[4]. In the last two decades, a variety of super-resolution methods have been proposed for estimating the HR image from a set of LR images. Early works on SR showed that the aliasing effects in the L... |

30 |
Generalized Wiener reconstruction of images from colour sensor data using a scale invariant prior
- Taubman
- 2000
(Show Context)
Citation Context ...ing. The luminance interpolation is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are =-=[21]-=-–[27]. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions. 1) In the constructed image with the mosaicing pattern, there are more green sensors with... |

27 | Efficient super-resolution and applications to mosaics
- Zomet, Peleg
- 2000
(Show Context)
Citation Context ... that early work, in general, SR is a computationally complex and numerically ill-behaved problem in many instances [6]. In recent years, more sophisticated SR methods were developed (see [3] and [6]–=-=[10]-=- as representative works). 2 Three CCD cameras which measure each color field independently tend to be relatively more expensive. Note that almost all super-resolution methods to date have been design... |

25 | Robust shift and add approach to super-resolution
- Farsiu, Robinson, et al.
- 2003
(Show Context)
Citation Context ... multiframe image fusion techniques, suggests that heavytailed noise distributions such as Laplacian are more appropriate models (especially in the presence of the inevitable motion estimation error) =-=[37]-=-. In [3], an alternate data fidelity term based on the norm is recently used, which has been shown to be very robust to data outliers Note that the norm is the ML estimate of data in the presence of L... |

24 |
A computationally efficient image superresolution algorithm
- Nguyen, Milanfar, et al.
- 2001
(Show Context)
Citation Context ...in contrast to the clean and practically naive frequency domain description of SR in that early work, in general, SR is a computationally complex and numerically ill-behaved problem in many instances =-=[6]-=-. In recent years, more sophisticated SR methods were developed (see [3] and [6]–[10] as representative works). 2 Three CCD cameras which measure each color field independently tend to be relatively m... |

23 | Color image quality metric S-CIELAB and its application on halftone texture visibility
- Zhang, Silverstein, et al.
- 1997
(Show Context)
Citation Context ...Y ˆA aIH��� PSS 2 R� w �ˆ 0 ˆ� 18 The S-CIELAB measure is a perceptual color fidelity measure that measures how accurate the reproduction of a color is to the original when viewed by a human observer =-=[42]-=-. In our experiments, we used the code with default parameters used in the implementation of this measure available at http://white.stanford.edu/~brian/scielab/scielab.html. XsFARSIU et al.: MULTIFRAM... |

19 | Color demosaicing by estimating luminance and opponent chromatic signals in the Fourier domain
- Alleysson, Süsstrunk, et al.
- 2002
(Show Context)
Citation Context ...polation is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are [20], [21], [22], [23], =-=[24]-=-, [25], and [26]. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions: 1) In the constructed image with the mosaicing pattern, there are more green s... |

18 | Restoring subsampled color images
- Keren, Osadchy
- 1999
(Show Context)
Citation Context ...ge of a homogeneity assumption in the neighboring pixels. Iterative MAP methods form another important category of demosaicing methods. A MAP algorithm with a smooth chrominance prior is discussed in =-=[19]-=-. The smooth chrominance prior is also used in [20], where the original image is transformed to YIQ representation. The chrominance interpolation is preformed using isotropic smoothing. The luminance ... |

18 | Demosaicing of color images using steerable wavelets
- Hel-Or, Keren
(Show Context)
Citation Context ...ixels. Iterative MAP methods form another important category of demosaicing methods. A MAP algorithm with a smooth chrominance prior is discussed in [19]. The smooth chrominance prior is also used in =-=[20]-=-, where the original image is transformed to YIQ representation. The chrominance interpolation is preformed using isotropic smoothing. The luminance interpolation is done using edge directions compute... |

16 |
Signal processing method and apparatus for sampled image signals,” United States Patent 4,630,307
- Cok
- 1987
(Show Context)
Citation Context ...ds. This property, combined with the assumption that the red/green and blue/green ratios are similar for the neighboring pixels, make the basics of the smooth hue transition method first discussed in =-=[13]-=-. Note that there is a negligible correlation between the values of neighboring pixels located on the different sides of an edge. Therefore, although the smooth hue transition assumption is logical fo... |

14 |
Optimal recovery demosaicing
- Muresan, Parks
- 2002
(Show Context)
Citation Context ...he luminance interpolation is done using edge directions computed in a steerable wavelet pyramidal structure. Other examples of popular demosaicing methods available in published literature are [20], =-=[21]-=-, [22], [23], [24], [25], and [26]. Almost all of the proposed demosaicing methods are based on one or more of these following assumptions: 1) In the constructed image with the mosaicing pattern, ther... |

13 |
Direct super-resolution and registration using raw CFA images
- Gotoh, Okutomi
- 2004
(Show Context)
Citation Context ...also exploits the difference in sensitivity of the human eye to the frequency content and outliers in the luminance and chrominance components of the image. In parallel to our work, Gotoh and Okotumi =-=[40]-=- are proposing another MAP estimation method for solving the same joint demosaicing/super-resolution problem. While their algorithm and ours share much in common, there are fundamental differences bet... |

10 |
Restoring subsampled color images,” Machine Vision and applications
- Keren, Osadchy
- 1999
(Show Context)
Citation Context ...ge of a homogeneity assumption in the neighboring pixels. Iterative MAP methods form another important category of demosaicing methods. A MAP algorithm with a smooth chrominance prior is discussed in =-=[18]-=-. The smooth chrominance prior is also used in [19], where the original image is transformed to YIQ representation. The chrominance interpolation is preformed using isotropic smoothing. The luminance ... |

9 | Multi-frame demosaicing and super-resolution from under-sampled color images
- Farsiu, Elad, et al.
- 2004
(Show Context)
Citation Context ... has been a two-step process. The first step 7 It is convenient to think of h @�A ae @�Ah@�A, where h@�A models the incoherent down-sampling effect of the CCD and e @�A models the color-filter effect =-=[28]-=-. 8 Note that color super resolution by itself is a special case of this model, where vectors † @�A and ‰ @�A are of size ‘Rw 2 I“ and matrices „ @�A and h @�A are of size ‘Rw 2 R� w “ for any color b... |

8 |
Asymptotic optimality of generalized cross-validation for choosing the regularization parameter
- Lukas
- 1993
(Show Context)
Citation Context ...t) against the second term (regularization cost). Generally speaking, choosing could be either done manually, using visual inspection, or automatically using methods like generalized cross-validation =-=[29]-=-, [30], L-curve [31], or other techniques. How to choose such regularization parameters is in itself a vast topic, which we will not treat in the present paper. C. Monochromatic Spatial Regularization... |

7 |
Apparatus and method for adaptive for adaptively interpolating a full color image utilizing chrominance gradients,” United States Patent 5,373,322
- Prescott
- 1994
(Show Context)
Citation Context ...ensive 3-CCD cameras. 4 This is the scenario for cheaper 1-CCD cameras.sFARSIU et al.: MULTIFRAME DEMOSAICING AND SUPER-RESOLUTION OF COLOR IMAGES 143 fact, gradient-based methods, first addressed in =-=[14]-=-, do not perform interpolation across the edges of an image. This noniterative method uses the second derivative of the red and blue channels to estimate the edge direction in the green channel. Later... |

7 |
Digital image processing, 3rd ed
- Pratt
- 2001
(Show Context)
Citation Context ...ng B-TV regularization to the luminance component will result in this desired property [3]. The luminance image can be calculated as the weighted sum X L =0.299X R +0.597X G +0.114X B as explained in =-=[37]-=-. The luminance regularization term is then defined as: J1(X) = C. Spatial Chrominance Penalty Term P� P� l=−P m=−P α |m|+|l| �X L − S l x Sm y X L�1. (12) Spatial regularization is required also for ... |

6 |
The use of the L-curve in the regularization of ill-posed problems
- Hansen, O’Leary
- 1993
(Show Context)
Citation Context ...term (regularization cost). Generally speaking, choosing λ could be either done manually, using visual inspection, or automatically using methods like Generalized Cross-Validation [28], [29], L-curve =-=[30]-=-, or other techniques. How to choose such regularization parameters is in itself a vast topic, which we will not treat in the present paper. 11sC. Monochromatic Spatial Regularization Tikhonov regular... |

2 |
Motion from color,” Comput
- Golland, Bruckstein
- 1997
(Show Context)
Citation Context ...h standard deviation equal to one. As the relative motion between these images followed the translational model, we only needed to estimate the motion between the luminance components of these images =-=[43]-=-. We used the method described in [44] to computed the motion vectors. The shift-and-add result (resolution enhancement factor of 4) is shown in Fig. 8(b) [zoomed in Fig. 8(f)]. In Fig. 8(c) [zoomed i... |

2 | and challenges in super-resolution - “Advances - 2004 |

2 | Super-resolution from image sequences -areview,”inIn - Borman, Stevenson - 1998 |

2 | and challenges in super-resolution,” Invited paper: to appear - “Advances - 2004 |

1 | and challenges in super-resolution,” Int - “Advances - 2004 |

1 | Adaptive Color Plan Interpolation - Hamilton, Adams - 1997 |

1 | Optimal recovery demosaicing,” presented at - Muresan, Parks - 2002 |

1 |
The use of the 1-curve in the regularization of ill-posed problems
- Hansen, O’Leary
- 1993
(Show Context)
Citation Context ...d term (regularization cost). Generally speaking, choosing could be either done manually, using visual inspection, or automatically using methods like generalized cross-validation [29], [30], L-curve =-=[31]-=-, or other techniques. How to choose such regularization parameters is in itself a vast topic, which we will not treat in the present paper. C. Monochromatic Spatial Regularization Tikhonov regulariza... |