In this paper, we present an approach to color image understanding that can be used to segment and analyze sur- faces with color variations due to highlights and shading. The work is based on a theory-the Dichromatic Reflec- tion Model-which describes the color of the reflected light as a mixture of light from surface reflection (highlights) and body reflection (object color). In the past, we have shown how the dichromatic theory can be used to separate a color image into two intrinsic reflection images: an image of just the highlights, and the original image with the highlights removed. At that time, the algorithm could only be applied to hand-segmented images. This paper shows how the same reflection model can be used to include color image segmentation into the image analysis. The result is a color image understanding system, capable of generating physical descriptions of the reflection processes occurring in the scene. Such descriptions include the intrinsic reflection images, an image segmenta- tion, and symbolic information about the object and highlight colors. This line of research can lead to based image understanding methods that are both more reliable and more useful than traditional methods.

In this paper, we present anapproach to colorimage understandingthat accountsforcolorvariationsdue to highlights and shading. We demonstrate that the reflected light from every point on a dielectric object. such as plastic, can be described asa linearcombination of the object color and the highlight color. The colors of all light rays reflected from one object then form a planar cluster in the color space.The shapeof this cluster is determined by the object and highlight colors and by the object shape and illumination geometry. We present a method that exploits the difference between object color and highlight color to separate the color of every pixel into a matte component and a highlight component.This generates two intrinsic images, one showing the scene without highlights, and the other one showing only the highlights. The intrinsic images may be a useful tool for a variety of algorithms in computer vision. such as stereo vision, motion analysis, shape from shading,and shapefrom highlights. Ourmethod combines the analysis of matte and highlight reflection with a sensor model that accounts for camera limitations. This enables us to successfully run our algorithm on real images taken in a laboratory setting. We show and discuss the results.

A theoretical framework is introduced for the perception of specular surface geometry. When an observer moves in three-dimensional space, real scene features, such as surface markings, remain stationary with respect to the surfaces they belong to. In contrast, a virtual feature, which is the specular reflection of a real feature, travels on the surface. Based on the notion of caustics, a novel feature classification algorithm is developed that distinguishes real and virtual features from their image trajectories that result from observer motion. Next, using support functions of curves, a closedform relation is derived between the image trajectory of a virtual feature and the geometry of the specular surface it travels on. It is shown that in the 2D case where camera motion and the surface profile are coplanar, the profile is uniquely recovered by tracking just two unknown virtual features. Finally, these results are generalized to the case of arbitrary 3D surface profiles that are trav...

In this paper, we present an extremely simple algorithm for shape from shading, which can be implemented in 25 lines of C code 1 . The algorithm is very fast, taking .2 seconds on a Sun SparcStation-1 for a 128 \Theta 128 image, and is purely local and highly parallelizable (parallel implementation included). In our approach, we employ a linear approximation of the reflectance function, as used by others. However, the major difference is that we first use the discrete approximations for surface normal, p and q, using finite differences, and then linearize the reflectance function in depth, Z(x; y), instead of p and q. The algorithm has been tested on several synthetic and real images of both Lambertian and specular surfaces, and good results have been obtained. The research reported here was supported by the Florida HiTech Council under grant 65 02 731, and the National Science Foundation under grant number CDA 9100898. 1 C code and some images can obtained by anonymous ftp from ...

Abstract: We describe a system for capturing bump maps from a series of images of an object from the same view point, but with varying, known, illumination. Using the illumination information we can reconstruct the surface normals for a variety of, but not all, surface nishes and geometries. The system allows an existing object to be rerendered with new lighting and surface nish without explicitly reconstructing the object geometry. 1

We present an novel algorithm that reconstructs voxels of a general 3D specular surface from multiple images of a calibrated camera. A calibrated scene (i.e. points whose 3D coordinates are known) is reflected by the unknown specular surface onto the image plane of the camera. For every viewpoint, surface normals are associated to the voxels traversed by each projection ray formed by the reflection of a scene point. A decision process then discards voxels whose associated surface normals are not consistent with one another. The output of the algorithm is a collection of voxels and surface normals in 3D space, whose quality and size depend on user-set thresholds. The method has been tested on synthetic and real images. Visual and quantified experimental results are presented.

We explore the geometry linking the shape of a curved mirror surface to the distortions it produces on a scene it reflects. Our analysis is local and differential. We assume a simple calibrated scene composed of lines passing through a point. We demonstrate that local information about the geometry of the surface may be recovered up to the second order from either the orientation and curvature of the images of two intersecting lines, or from the orientation of the images of three or more intersecting lines. An explicit solution for calculating shape and position of spherical mirror surfaces is given.

Traditional intensity imaging does not offer a general approach for the perception of textureless and specular reflecting surfaces. Intensity based methods for shape reconstruction of specular surfaces rely on virtual (i.e. mirrored) features moving over the surface under viewer motion. We present a novel method based on polarization imaging for shape recovery of specular surfaces. This method overcomes the limitations of the intensity based approach, because no virtual features are required. It recovers whole surface patches and not only single curves on the surface. The presented solution is general as it is independent of the illumination. The polarization image encodes the projection of the surface normals onto the image and therefore provides constraints on the surface geometry. Taking polarization images from multiple views produces enough constraints to infer the complete surface shape. The reconstruction problem is solved by an optimization scheme where the surface geometry is modelled by a set of hierarchical basis functions. The optimization algorithm proves to be well converging, accurate and noise resistant. The work is substantiated by experiments on synthetic and real data.

The problem of accurate depth estimation using stereo in the presence of specular reflection is addressed. Specular reflection is viewpoint dependent and can cause large intensity differences at corresponding points. Hence, mismatches can result causing significant depth errors. Current stereo algorithms largely ignore specular reflection which is a fundamental reflection phenomenon from surfaces, both smooth and rough. We analyzed the physics of specular reflection and the geometry of stereopsis which led us to an interesting relationship between stereo vergence, surface roughness, and the likelihood of a correct match. Given the lower bound on surface roughness, an optimal binocular stereo configuration can be determined which maximizes precision in depth estimation despite specular reflection. However, surface roughness is difficult to estimate in unstructured environments. Therefore, multiple view configurations independent of surface roughness are determined such that at each scen...

This paper examines the construction of a 3-D surface model of an object rotating in front of a camera. Previous research in depth from motion has demonstrated the power of using an incremental approach to depth estimation. In this paper, we extend this approach to more general motion and use a full 3-D surface model instead of a 2 1 = 2 -D sketch. The algorithm starts with a flow field computed using local correlation. It then projects individual measurements into 3-D points with associated uncertainties. Nearby points from successive frames are merged to improve the position estimates. These points are then used to construct a finite element surface model, which is itself refined over time. We demonstrate the application of our new techniques to several real image sequences. Keywords: Computer vision, 3-D model construction, image sequence (motion) analysis, optic flow, Kalman filter, surface interpolation, computer aided design, computer graphics animation. c flDigital Equipment C...