Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2002) 1167--1183  Home/Search   Document Details and Download   Summary   Related Articles   Check

....solution, where the regularization coecients may be set by cross validation. The regularization terms are often motivated in terms of a prior distribution over the high resolution image, in which case the solution can be interpreted as a MAP (maximum a posteriori) optimization. Baker and Kanade [5] have tried to improve the performance of super resolution algorithms by developing domain speci c image priors, applicable to faces or text for example, which are learned from data. In this case the algorithm is e ectively hallucinating perceptually plausible high frequency features. Here we ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. Technical report, Carnegie Mellon University, 2002. submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence.

....implementation. Section IV gives the experimental results. Section V concludes the paper. II. PREVIOUS WORK In this section, a brief summary of objective analysis, Bayesian models, and the Gibbs sampler is provided. Introduction to traditional super resolution restoration can be found in [6] [8]. A. Objective analysis Objective analysis (OA) is used extensively by meteorologists [9] and oceanographers [1] for estimating the values of geophysical variables at a grid of interpolation points from Fig. 1. Sample sea surface temperature (SST) data with no interpolation. irregularly ....

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. V24, 2002.

....recognition. First, the temporal information of faces can be utilized to facilitate the recognition task. For example, the person specific dynamic characteristics can help the recognition [5] Secondly, more effective representations, such as a 3D face model [9] or super resolution images [10], can be obtained from the video sequence and used to improve recognition results. Finally, video based recognition allows learning or updating the subject model over time. Liu et at. proposed an updating during recognition scheme, where the current and past frames in a video sequence are used to ....

S. Baker and T. Kanade, "Limits on Super-Resolution and How to Break Them", 1EEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 24, No. 9, September 2002, pp. 1167-1183.

....in the paper texture. We want to ensure that for any zoom level, the texture looks like paper. This is related to self similar fractals, e.g. Hut81, Bar88] or to super Figure 8: Successive octaves in a Perlin noise. On the right we show the summation of the four octaves. resolution, e.g. BK00] We also need to be able to apply arbitrary rotations and translations. The latter can be obtained using tileable texture. We present a solution based on procedural solid textures [Per85] We show that an infinite zoom can be obtained by cyclically shifting the frequency spectrum of a Perlin ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proc. of CVPR, 2000.

....general images. However, our approach is limited to capturing styles that are local, and at the fragment level, the model is simple and fixed. 2. Previous work Many operations ranging from low level vision tasks to high end graphics applications have been efficiently performed based on examples [3, 6, 8, 9, 10, 15, 16, 20, 21, 27, 29]. The idea of defining and factoring image style and content using a bilinear model was introduced to computer vision by Freeman and Tenenbaum [17] Given a training set of aligned face images of different people under varying illumination, Tenenbaum and Freeman [29] refer to the identity of a ....

....as this results in the loss of image detail. Freeman et al. 15, 16] derive a model used for performing super resolution by example. The technique is based on examining many pairs of high resolution and low resolution versions of image patches from several training images. Baker and Kanade [3] apply this technique restrictively, to a class of face images. Given a new low resolution face image its corresponding high resolution image is inferred by re using the existing mapping between individual lowresolution and high resolution face patches. The term image quilting [13] was coined for ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

....to be the most common image enhancement approach. In the super resolution literature there are many papers which do not deal with the simple case of reconstruction based on a single image. Many authors are interested in reconstruction based on multiple slightly perturbed subsamples from an image [3, 2] . This is useful for photographic scanners for example. In a similar manner other authors utilise the information from a number of frames in a temporal sequence [4] In other situations highly substantial prior information is given, such as the ground truth for a part of the image. Sometimes ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of CVPR 00, pages 372--379, 2000.

....sharp edges and image details. Figure 15 shows an example where our low level training set al..one is not enough to distinguish JPEG compression noise from correct image data; the algorithm interprets the artifacts as image data and enhances them. Extensions of specialized high level models such as [2] could be needed. In a simple exploration of the relation of the zoomed image to the training images (see [6] for others) we enlarged the image of Fig. 8 using a pathological training set of images of text. Nonetheless, the algorithm does its best to explain the observed low resolution image in ....

....in the image, an image extrapolation based on local image evidence alone will not produce the new details that the viewer expects. Very small face images are suseptable to this problem. To address these properly, higherlevel reasoning would have to be added to the algorithm. Baker and Kanade [2] have recently explored super resolution algorithms tuned to a particular class of images, such as faces or text. In the zoomed up images, low contrast details next to high contrast edges may be lost, due to the contrast normalization fixing on the level of the high contrast edge. Independent ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In IEEE Conf. Computer Vision and Pattern Recognition, 2000.

....the constrained model is a subspace which is computed using principal component analysis of a set of registered face images. This sub space is also used to define a spatially varying prior within a MAP estimator. The methods of this paper are similar in spirit to the approach of Baker and Kanade [1] who also examined the use of spatially varying priors for face images. Their input consists of images which are all at the same scale and are related by translations. The novelty of our method is two fold: first, the low resolution images need not be at the same resolution and indeed the ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proc. CVPR, 2000.

....[23 Apr 2001] 1 We assume familiarity with basic sampling theory. Sampling a continuous signal on a discrete grid folds ( aliases ) high spatial frequencies down to their fractional parts (in cycles per grid unit) and thus confuses the signal. Signals limited to the Nyquist frequency band [ 1 2 , 1 2 ] (no wavelengths less than 2 pixels) have no aliasing and hence can be reconstructed exactly. Bandwidth limitation band limited signal reconstruction can be implemented by continuous discrete convolution against a sinc function sinc(#x) # sin(#x) #x) whose abruptly truncated Fourier ....

....the original camera would return if shifted, not an enhanced image. An illuminating 1D analytic study complementary to our 2D empirical one is [15] For a unified information optimizing approach to sampling and reconstruction, see [7] For subpixel reconstruction based on learned codebooks see [6, 1]. Our main aim was to establish good working practice for accurate subpixel image manipulation, side stepping the bewildering range of methods available for filter design [9, 10] Forms like splines are essentially heuristic, and we do not accept that strict band limitation and sinc ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Int. Conf. Computer Vision & Pattern Recognition, volume 2, pages 372--379, 2000.

....virtually, input images are rendered onto 3D model[11] Because the virtual viewpoint is not generally at the same point, the synthesized image at virtual view does not have sufficient resolution, if the virtual viewpoint is closer to the object than real viewpoint. Super resolution techniques[1,2,3,4], which are studied for obtaining higher resolution images than the original images, will be one solution for improving the resolution of the virtual viewpoint images. In most of the conventional approaches, however, only 2D geometric cases are mostly studied although 3D geometrical analysis is ....

S.Baker and T.Kanade, "Limits on Super-Resolution and How to Break Them," Proc.CVPR00, 2000.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167 -- 1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167 -- 1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167 -- 1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proc. of CVPR, 2000.

....approximation to the Hessian can then be used directly in the Newton algorithm. See Figure 13 for a summary. This approximation is commonly used in optimization problems with a large number of parameters. Examples of this diagonal approximation in vision include stereo [17] and super resolution [1]. 38 The Diagonal Hessian Inverse Compositional Algorithm (4) Evaluate ### # # ### # ### to compute################# (2) Compute the error (6) Compute the Diagonal Hessian matrix ### ### ] ########################### (9) Update ....

....dramatically when this step is added. See Figure 17 for a comparison of the performance of the diagonal Hessian algorithms with and without the step size estimation step. The diagonal approximation to the Hessian is used in various vision algorithms such as stereo [17] and super resolution [1]. Our results indicate that these algorithms may be under performing because of the optimization algorithm used, especially if the step size is not chosen correctly. 4.6.4 Importance of Parameterization Even after correcting for the step size the performance of steepest descent and the diagonal ....

[Article contains additional citation context not shown here]

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 372--379, 2000.

....approximation to the Hessian can then be used directly in the Newton algorithm. See Figure 12 for a summary. This approximation is commonly used in optimization problems with a large number of parameters. Examples of this diagonal approximation in vision include stereo [15] and super resolution [1]. 4.4.1 Computational Cost of the Diagonal Approximation to the Hessian The diagonal approximation to the Hessian makes the Newton inverse compositional algorithm far more efficient. Because there are only n elements on the leading diagonal, compared to n elements in the entire Hessian ....

....dramatically when this step is added. See Figure 16 for a comparison of the performance of the diagonal Hessian algorithms with and without the step size estimation step. The diagonal approximation to the Hessian is used in various vision algorithms such as stereo [15] and super resolution [1]. Our results indicate that these algorithms may be under performing 39 0 0.5 1 1.5 2 2.5 Point Diag GN Diag GN (SS) SD Diag N Diag N (SS) 1 2 3 4 5 6 7 8 9 10 0 10 20 30 40 50 60 70 80 90 100 Diag GN Diag GN (SS) SD Diag N Diag N (SS) Figure 16: The performance of ....

[Article contains additional citation context not shown here]

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2000.

....Shree Nayar, Steve Seitz, Sundar Vedula, and everyone in the Face Group at CMU. Finally we 31 would like to thank the anonymous reviewers for their comments and suggestions. The research described in this paper was supported by US DOD Grant MDA 904 98 C A915. A preliminary version of this paper [4] appeared in June 2000 in the IEEE Conference on Computer Vision and Pattern Recognition. Additional experimental results can be found in the technical report [1] ....

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the

....problem. These reconstruction constraints have been used by numerous authors since first studied by Peleg et al. 11] and can easily be embedded in a Bayesian framework incorporating a (smoothness) prior on the super resolution image [4, 14, 9, 7] Numerous other refinements are also possible. See [2] for examples. Is that all there is to super resolution, pose it as a reconstruction problem, write down the reconstruction constraints, add a smoothness prior, and then solve In this paper we present two complementary results which show that there is far more. First we analyze the ....

....low resolution inputs as we want, could we not recover a super resolution image with arbitrary resolution We show that this is not the case. If the input images are quantized to 8 bit values (grey levels in the range 0 255) there are limits on how well we can reconstruct a super resolution image [2]. Ironically, noise can theoretically be used to increase the number of bits per pixel Research supported by US DOD Grant MDA 904 98 C A915. by averaging, but the analysis of the noise free case still yields important insights into super resolution. In particular, we show that for large ....

[Article contains additional citation context not shown here]

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proc. of CVPR, 2000.

No context found.

Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2002) 1167--1183

No context found.

Baker, S., and Kanade, T. 2002. Limits on super-resolution and how to break them. IEEE Trans. on PAMI 24(9):1167-- 1183.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167--1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167 -- 1183, September 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167 -- 1183, September 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. Technical report, Carnegie Mellon University, 2002. submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2000.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

No context found.

Baker, S., Kanade, T.: Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence 24 (2002) 1167 -- 1183

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 2000. IEEE Computer Society.

No context found.

S. Baker, T. Kanade, "Limits on super-resolution and how to break them," CVPR, Vol.2, pp.372-379, 2000.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Conf. Computer Vision and Pattern Recognition, 2000.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, 2000, pp. 372--379.

No context found.

S. Baker and T. Kanade, Limits on super-resolution and how to break them, IEEE Trans. Pattern Analysis and Machine Intelligence, 24 (2002), pp. 1167--1183.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Trans. Pattern Anal. Machine Intell., vol. 24, pp. 1167-1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE PAMI, 24(9):1167--1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Trans. PAMI, 24(9):1167--1183, September 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, Hilton Head Island, South Carolina, June 2000. IEEE Computer Society.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence 24, pp. 1167--1183, September 2002.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," IEEE Transactions on Pattern Analysis and Machine Intelligence 24, pp. 1167--1183, September 2002.

No context found.

S. Baker and T. Kanade, "Limits on super-resolution and how to break them," in Proc. Computer Vision and Pattern Recognition 2000.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Conf. Computer Vision and Pattern Recognition, 2000.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(9):1167--1183, 2002.

No context found.

S. Baker and T. Kanade. Limits on super-resolution and how to break them. PAMI, 24(9):1167--1183, September 2002.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC