Abstract. We consider the problem of estimating the 3D shape and reflectance properties of an object made of a single material from a set of calibrated views. To model the reflectance, we propose to use the View Independent Reflectance Map (VIRM), which is a representation of the joint effect of the diffuse+specular Bidirectional Reflectance Distribution Function (BRDF) and the environment illumination. The object shape is parameterized using a triangular mesh. We pose the estimation problem as minimizing the cost of matching input images, and the images synthesized using the shape and VIRM estimates. We show that by enforcing a constant value of VIRM as a global constraint, we can minimize the cost function by iterating between the VIRM and shape estimation. Experimental results on both synthetic and real objects show that our algorithm can recover both the 3D shape and the diffuse/specular reflectance information. Our algorithm does not require the light sources to be known or calibrated. The estimated VIRM can be used to predict the appearances of objects with the same material from novel viewpoints and under transformed illumination. Keywords: reflectance model, 3d reconstruction, shape from shading, illumination model, BRDF

Helmholtz stereopsis has been introduced recently as a surface reconstruction technique that does not assume a model of surface reflectance. In the reported formulation, correspondence was established using a rank constraint, necessitating at least three viewpoints and three pairs of images. Here, it is revealed that the fundamental Helmholtz stereopsis constraint defines a nonlinear partial differential equation, which can be solved using only two images. It is shown that, unlike conventional stereo, binocular Helmholtz stereopsis is able to establish correspondence (and thereby recover surface depth) for objects having an arbitrary and unknown BRDF and in textureless regions (i.e., regions of constant or slowly varying BRDF). An implementation and experimental results validate the method for specular surfaces with and without texture. 1

Abstract We propose a generative model based method for recovering both the shape and the reflectance of the surface(s) of a scene from multiple images, assuming that illumination conditions and cameras calibration are known in advance. Based on a variational framework and via gradient descents, the algorithm minimizes simultaneously and consistently a global cost functional with respect to both shape and reflectance. The motivations for our approach are threefold. (1) Contrary to previous works which mainly consider specific individual scenarios, our method applies indiscriminately to a number of classical scenarios; in particular it works for classical stereovision, multiview photometric stereo and multiview shape from shading. It works with changing as well as static illumination. (2) Our approach naturally combines stereo, silhouette and shading cues in a single framework. (3) Moreover, unlike most previous methods dealing with only Lambertian surfaces, the proposed

Abstract. In this paper, we present a new method to deal with specular highlights in correspondence search. The proposed method is essentially based on the specular-free two-band image that we introduce to deal with specular reflection. For given input images, specular-free two-band images are generated using simple pixel-wise computations in real-time. Specular-free two-band images are then used to compute per-pixel raw matching costs. By using the specular-free two-band images instead of input images, reliable raw matching costs that are independent of the specularities of image pixels are obtained. As a result, we can find correct correspondences even in the presence of specular highlights. Experimental results show that the proposed method successfully produces accurate disparity maps for stereo images with specular highlights. 1

Real scenes are full of specularities (highlights and reflections), and yet most vision algorithms ignore them. In order to capture the appearance of realistic scenes, we need to model specularities as separate layers. In this paper, we study the behavior of specularities in static scenes as the camera moves, and describe their dependence on varying surface geometry, orientation, and scene point and camera locations. For a rectilinear camera motion with constant velocity, we study how the specular motion deviates from a straight trajectory (disparity deviation) and how much it violates the epipolar constraint (epipolar deviation). Surprisingly, for surfaces that are convex or not highly undulating, these deviations are usually quite small. We also study the appearance of specularities, i.e., how they interact with the body reflection, and with the usual occlusion ordering constraints applicable to di#use opaque layers. We present a taxonomy of specularities based on their photometric properties as a guide for designing separation techniques. Finally, we propose a technique to extract specularities as a separate layer, and demonstrate it using an image sequence of a complex scene.

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features ’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces ’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation. 1

In this work, we consider the dense reconstruction of specular objects. We propose the use of a specularity constraint, based on surface normal/depth consistency, to define a matching cost function that can drive standard stereo reconstruction methods. We discuss the types of ambiguity that can arise, and suggest an aggregation method based on anisotropic diffusion that is particularly suitable for this matching cost function. We also present a controlled illumination setup that includes a pair of cameras and one LCD monitor, which is used as a calibrated, variable-position light source. We use this setup to evaluate the proposed method on real data, and demonstrate its capacity to recover high-quality depth and orientation from specular objects. 1.

We present a system for reconstructing water surfaces using an indirect refractive stereo reconstruction method. Our work builds on previous work on image-based water recon-struction that uses single view refractive reconstruction techniques. We combine this approach with a stereo matching algorithm. Depth determination relies upon the refrac-tive disparity of points on a plane below the water. We describe how the location of points on the water surface can be determined by hypothesizing a depth from the refrac-tive disparity of one camera view. Then the second camera view is used to verify the depth. We compare two potential metrics for this matching process. We then present re-sults from our algorithm using both simulated and empirical input, analyzing the results to determine the primary factors that contribute toward accurate surface point determi-nation. We also show how this process can be used to reconstruct sequences of dynamic water and present several result sets. ii Acknowledgements I would like to acknowledge the insightful support given to me by my supervisor Kiriakos

apport de recherche

Techniques developed in the field of computer graphics and virtual reality are applied in many situations today, with the result that methods of measuring three-dimensional shapes of real objects are seen to be more and more important. However, few methods are proposed to measure three-dimensional shapes of transparent objects such as those made of glass and acrylic. In this thesis, I propose three methods for estimating the surface shapes of transparent objects by using polarization analysis. There is an ambiguity problem that results from using polarization data obtained by observing reflected light from one view. The first method resolves the ambiguity problem by the knowledge established in the research field of thermodynamics. The polarization analysis of reflected light in the visible light domain results in an ambiguity because two candidates for surface normal will be suggested. However, correct surface normal can be chosen by polarization analysis of thermal radiation in an infrared light domain. The second method resolves the ambiguity by using theory established in the research field of differential geometry. This method resolves the ambiguity problem, first, by observing the reflected visible light of a transparent object from two different views, and second, by analyzing