The thesis investigates the problem of how information contained in multiple, overlapping images of the same scene may be combined to produce images of superior quality. This area, generically titled frame fusion, offers the possibility of reducing noise, extending the field of view, removal of moving objects, removing blur, increasing spatial resolution and improving dynamic range. As such, this research has many applications in fields as diverse as forensic image restoration, computer generated special effects, video image compression, and digital video editing. An essential enabling step prior to performing frame fusion is image registration, by which an accurate estimate of the point-to-point mapping between views is computed. A robust and efficient algorithm is described to automatically register multiple images using only information contained within the images themselves. The accuracy of this method, and the statistical assumptions upon which it relies, are investigated empirically. Two forms of frame-fusion are investigated. The first is image mosaicing, which is the alignment of multiple images into a single composition representing part of a 3D scene.

Many vision applications require precise measurement of scene radiance. The function relating scene radiance to image brightness is called the camera response. We analyze the properties that all camera responses share. This allows us to find the constraints that any response function must satisfy. These constraints determine the theoretical space of all possible camera responses. We have collected a diverse database of real-world camera response functions (DoRF). Using this database we show that real-world responses occupy a small part of the theoretical space of all possible responses. We combine the constraints from our theoretical space with the data from DoRF to create a low-parameter Empirical Model of Response (EMoR). This response model allows us to accurately interpolate the complete response function of a camera from a small number of measurements obtained using a standard chart. We also show that the model can be used to accurately estimate the camera response from images of an arbitrary scene taken using different exposures. The DoRF database and the EMoR model can be downloaded at

Abstract. Brightness values of pixels in an image are related to image irradiance by a non-linear function, called the radiometric response function. Recovery of this function is important since many algorithms in computer vision and image processing use image irradiance. Several investigators have described methods for recovery of the radiometric response, without using charts, from multiple exposures of the same scene. All these recovery methods are based solely on the correspondence of gray-levels in one exposure to gray-levels in another exposure. This correspondence can be described by a function we call the brightness transfer function. We show that brightness transfer functions, and thus images themselves, do not uniquely determine the radiometric response function, nor the ratios of exposure between the images. We completely determine the ambiguity associated with the recovery of the response function and the exposure ratios. We show that all previous methods break these ambiguities only by making assumptions on the form of the

Abstract. Two central issues in stereo algorithm design are the matching criterion and the underlying smoothness assumptions. In this paper we propose a new stereo algorithm with novel approaches to both issues. We start with a careful analysis of the properties of the continuous disparity space image (DSI), and derive a new matching cost based on the reconstructed image signals. We then use a symmetric matching process that employs visibility constraints to assign disparities to a large fraction of pixels with minimal smoothness assumptions. While the matching operates on integer disparities, sub-pixel information is maintained throughout the process. Global smoothness assumptions are delayed until a later stage in which disparities are assigned in textureless and occluded areas. We validate our approach with experimental results on stereo images with ground truth. 1

A method is presented for computing the radiometric response function of a camera from a single grayscale image. While most previous techniques require a set of registered images with different exposures to obtain response data, our approach capitalizes on a statistical feature of graylevel histograms at edge regions to gain information for radiometric calibration. Appropriate edge regions are automatically determined by our technique, and a prior model of radiometric response functions is employed to deal with incomplete data. With this single-image method, radiometric calibration becomes possible to perform in many instances where the camera is unknown. 1

The increasing photorealism for computer graphics has made computer graphics a convincing form of image forgery. Therefore, classifying photographic images and photorealistic computer graphics has become an important problem for image forgery detection. In this paper, we propose a new geometrybased image model, motivated by the physical image generation process, to tackle the above-mentioned problem. The proposed model reveals certain physical differences between the two image categories, such as the gamma correction in photographic images and the sharp structures in computer graphics. For the problem of image forgery detection, we propose two levels of image authenticity definition, i.e., imaging-process authenticity and scene authenticity, and analyze our technique against these definitions. Such definition is important for making the concept of image authenticity computable. Apart from offering physical insights, our technique with a classification accuracy of 83.5 % outperforms those in the prior work, i.e., wavelet features at 80.3 % and cartoon features at 71.0%. We also consider a recapturing attack scenario and propose a counter-attack measure. In addition, we constructed a publicly available benchmark dataset with images of diverse content and computer graphics of high photorealism.

Outdoor scene classification is challenging due to irregular geometry, uncontrolled illumination, and noisy reflectance distributions. This paper discusses a Bayesian approach to classifying a color image of an outdoor scene. A likelihood model factors in the physics of the image formation process, the sensor noise distribution, and prior distributions over geometry, material types, and illuminant spectrum parameters. These prior distributions are learned through a training process that uses color observations of planar scene patches over time. An iterative linear algorithm estimates the maximum likelihood reflectance, spectrum, geometry, and object class labels for a new image. Experiments on images taken by outdoor surveillance cameras classify known material types and shadow regions correctly, and flag as outliers material types that were not seen previously. 1.

A central issue in stereo algorithm design is the choice of matching cost. Many algorithms simply use squared or absolute intensity differences based on integer disparity steps. In this paper we address potential problems with such approaches. We begin with a careful analysis of the properties of the continuous disparity space image (DSI) and propose several new matching cost variants based on symmetrically matching interpolated image signals. Using stereo images with ground truth, we empirically evaluate the performance of the different cost variants and show that proper sampling can yield improved matching performance. 1

Effective change detection under dynamic illumination conditions is an active research topic. Most research has concentrated on adaptive statistical representations for the appearance of the background scene. There is limited work that develops the statistical models for background representation by taking into account an explicit model for the camera response function, the camera noise model, and illumination priors. Assuming a monotone but nonlinear camera response function, a Phong shading model for the surface material, and a locally constant but spatially varying illumination, we show that the sign of the difference between two pixel measurements is maintained across global illumination changes. We use this result along with a statistical model for the camera noise to develop a change detection algorithm that deals with sudden changes in illumination. The performance evaluation of the algorithm is done through simulations and on real data.

The mapping that relates the image irradiance to the image brightness (intensity) is known as the Radiometric Response Function or Camera Response Function. This usually unknown mapping is nonlinear and varies from one color channel to another. In this paper, we present a method to estimate the radiometric response functions (of R, G and B channels) of a color camera directly from the images of an arbitrary scene taken under different illumination conditions (The illumination conditions are not assumed to be known). The response function of a channel is modeled as a gamma curve and is recovered by using a constrained nonlinear minimization approach by exploiting the fact that the material properties of the scene remain constant in all the images. The performance of the proposed method is demonstrated experimentally.