Results 1  10
of
30
Shape and motion under varying illumination: Unifying structure from motion, photometric stereo, and multiview stereo
 In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition
, 2003
"... This paper presents an algorithm for computing optical flow, shape, motion, lighting, and albedo from an image sequence of a rigidlymoving Lambertian object under distant illumination. The problem is formulated in a manner that subsumes structure from motion, multiview stereo, and photometric ster ..."
Abstract

Cited by 68 (3 self)
 Add to MetaCart
(Show Context)
This paper presents an algorithm for computing optical flow, shape, motion, lighting, and albedo from an image sequence of a rigidlymoving Lambertian object under distant illumination. The problem is formulated in a manner that subsumes structure from motion, multiview stereo, and photometric stereo as special cases. The algorithm utilizes both spatial and temporal intensity variation as cues: the former constrains flow and the latter constrains surface orientation; combining both cues enables dense reconstruction of both textured and textureless surfaces. The algorithm works by iteratively estimating affine camera parameters, illumination, shape, and albedo in an alternating fashion. Results are demonstrated on videos of handheld objects moving in front of a fixed light and camera. 1.
A SignalProcessing Framework for Reflection
 ACM TRANSACTIONS ON GRAPHICS
, 2004
"... ... In this paper, we formalize these notions, showing that the reflected light field can be thought of in a precise quantitative way as obtained by convolving the lighting and BRDF, i.e. by filtering the incident illumination using the BRDF. Mathematically, we are able to express the frequencyspac ..."
Abstract

Cited by 47 (4 self)
 Add to MetaCart
... In this paper, we formalize these notions, showing that the reflected light field can be thought of in a precise quantitative way as obtained by convolving the lighting and BRDF, i.e. by filtering the incident illumination using the BRDF. Mathematically, we are able to express the frequencyspace coe#cients of the reflected light field as a product of the spherical harmonic coe# cients of the illumination and the BRDF. These results are of practical importance in determining the wellposedness and conditioning of problems in inverse renderingestimation of BRDF and lighting parameters from real photographs. Furthermore, we are able to derive analytic formulae for the spherical harmonic coe#cients of many common BRDF and lighting models. From this formal analysis, we are able to determine precise conditions under which estimation of BRDFs and lighting distributions are well posed and wellconditioned. Our mathematical analysis also has implications for forward renderingespecially the e#cient rendering of objects under complex lighting conditions specified by environment maps. The results, especially the analytic formulae derived for Lambertian surfaces, are also relevant in computer vision in the areas of recognition, photometric stereo and structure from motion.
Shape from Varying Illumination and Viewpoint
 In Proc. of Intl. Conf. on Comp. Vision
, 2007
"... We address the problem of reconstructing the 3D shape of a Lambertian surface from multiple images acquired as an object rotates under distant and possibly varying illumination. Using camera projection matrices estimated from point correspondences across views, the algorithm computes a dense corres ..."
Abstract

Cited by 25 (0 self)
 Add to MetaCart
(Show Context)
We address the problem of reconstructing the 3D shape of a Lambertian surface from multiple images acquired as an object rotates under distant and possibly varying illumination. Using camera projection matrices estimated from point correspondences across views, the algorithm computes a dense correspondence map by minimizing a multiocular photometric constraint. Once correspondence across views is established, photometric stereo is applied to estimate a surface normal field and 3D surface. Conceptually, the algorithm merges multiview stereo and photometric stereo and uses aspects of both methods to recover shape. The method is straightforward to implement and relies on established principles from the two stereo methods. We empirically validate the method on images of a number of objects and show that it outperforms previous methods. 1.
Accuracy of spherical harmonic approximations for images of lambertian objects under far and near lighting
 In ECCV. I–574–I–587
, 2004
"... Abstract. Various problems in Computer Vision become difficult due to a strong influence of lighting on the images of an object. Recent work showed analytically that the set of all images of a convex, Lambertian object can be accurately approximated by the lowdimensional linear subspace constructed ..."
Abstract

Cited by 21 (5 self)
 Add to MetaCart
(Show Context)
Abstract. Various problems in Computer Vision become difficult due to a strong influence of lighting on the images of an object. Recent work showed analytically that the set of all images of a convex, Lambertian object can be accurately approximated by the lowdimensional linear subspace constructed using spherical harmonic functions. In this paper we present two major contributions: first, we extend previous analysis of spherical harmonic approximation to the case of arbitrary objects; second, we analyze its applicability for near light. We begin by showing that under distant lighting, with uniform distribution of light sources, the average accuracy of spherical harmonic representation can be bound from below. This bound holds for objects of arbitrary geometry and color, and for general illuminations (consisting of any number of light sources). We further examine the case when light is coming from above and provide an analytic expression for the accuracy obtained in this case. Finally, we show that lowdimensional representations using spherical harmonics provide an accurate approximation also for fairly near light. Our analysis assumes Lambertian reflectance and accounts for attached, but not for cast shadows. We support this analysis by simulations and real experiments, including an example of a 3D shape reconstruction by photometric stereo under very close, unknown lighting. 1
Modeling Illumination Variation With Spherical Harmonics
 In Face Processing: Advanced Modeling and Methods
, 2005
"... Illumination can have a significant impact on the appearance of surfaces, as the patterns of shading, specularities and shadows change. For instance, some images of a face under different lighting conditions are shown in figure 1. Differences in lighting can often play a much greater role in image v ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
(Show Context)
Illumination can have a significant impact on the appearance of surfaces, as the patterns of shading, specularities and shadows change. For instance, some images of a face under different lighting conditions are shown in figure 1. Differences in lighting can often play a much greater role in image variability of human faces than differences between individual people. Lighting designers in
Photometric stereo for dynamic surface orientations
 In ECCV
, 2010
"... Abstract. We present a photometric stereo method for nonrigid objects of unknown and spatially varying materials. The prior art uses timemultiplexed illumination but assumes constant surface normals across several frames, fundamentally limiting the accuracy of the estimated normals. We explicitly ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We present a photometric stereo method for nonrigid objects of unknown and spatially varying materials. The prior art uses timemultiplexed illumination but assumes constant surface normals across several frames, fundamentally limiting the accuracy of the estimated normals. We explicitly account for timevarying surface orientations, and show that for unknown Lambertian materials, five images are sufficient to recover surface orientation in one frame. Our optimized system implementation exploits the physical properties of typical cameras and LEDs to reduce the required number of images to just three, and also facilitates frametoframe image alignment using standard optical flow methods, despite varying illumination. We demonstrate the system’s performance by computing surface orientations for several different moving, deforming objects. 1
A theory of spherical harmonic identities for BRDF/lighting transfer and image consistency
 In European Conference on Computer Vision
, 2006
"... , where the subscripts refer to the spherical harmonic indices, and the superscripts to the lighting (1 or 2) and object or material (again 1 or 2). We derive a basic identity, B 1,1 lm B2,2 lm = B1,2 lm B2,1 lm, independent of the specific lighting configurations or BRDFs. While this paper is prima ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
, where the subscripts refer to the spherical harmonic indices, and the superscripts to the lighting (1 or 2) and object or material (again 1 or 2). We derive a basic identity, B 1,1 lm B2,2 lm = B1,2 lm B2,1 lm, independent of the specific lighting configurations or BRDFs. While this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image, to detect tampering or image splicing. Abstract. We develop new mathematical results based on the spherical harmonic convolution framework for reflection from a curved surface. We derive novel identities, which are the angular frequency domain analogs to common spatial domain invariants such as reflectance ratios. They apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. Denote the spherical harmonic coefficients by B light,material lm 1
A Theory of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency
"... Abstract—This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Abstract—This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical harmonic convolution framework for reflection from a curved surface. Our identities apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. For this case, we derive a novel identity, independent of the specific lighting configurations or BRDFs, that allows us to directly estimate the fourth image if the other three are available. The identity can also be used as an invariant to detect tampering in the images. Although this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image to detect tampering or image splicing.
Photometric stereo beyond glass: Active separation of transparent layer and fivelight photometric stereo with Mestimator using laplace distribution for a virtual museum
 Proceedings of the IEEE Workshop on Photometric Analysis For Computer Vision (PACV'07, in conjunction with ICCV
, 2007
"... One of the necessary techniques for constructing a virtual museum is to digitize the artwork arranged inside a glass or acrylic display case without bringing the artwork out of the display case. By using photometric stereo, we estimate the shape (surface normal) and the reflectance (albedo) of the ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
One of the necessary techniques for constructing a virtual museum is to digitize the artwork arranged inside a glass or acrylic display case without bringing the artwork out of the display case. By using photometric stereo, we estimate the shape (surface normal) and the reflectance (albedo) of the artwork arranged inside a transparent display case. If we illuminate the display case, the light will reflect at its surface; thus, we cannot apply conventional photometric stereo as is. In this paper, we propose a fivelight photometric stereo that estimates the shape and the reflectance of an object that has specularities under the circumstances that the light is reflected at the surface of the display case. 1.
A twoframe theory of motion, lighting and shape
 In CVPR
, 2008
"... This paper explores how shape, motion, and lighting interact in the case of a twoframe motion sequence. We consider a rigid object with Lambertian reflectance properties undergoing small motion with respect to both a camera and a stationary point light source. Assuming orthographic projection, we d ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
This paper explores how shape, motion, and lighting interact in the case of a twoframe motion sequence. We consider a rigid object with Lambertian reflectance properties undergoing small motion with respect to both a camera and a stationary point light source. Assuming orthographic projection, we derive a single, first order quasilinear partial differential equation that relates shape, motion, and lighting, while eliminating out the albedo. We show how this equation can be solved, when the motion and lighting parameters are known, to produce a 3D reconstruction of the object. A solution is obtained using the method of characteristics and can be refined by adding regularization. We further show that both smooth bounding contours as well as surface markings can be used to derive Dirichlet boundary conditions. Experimental results demonstrate the quality of this reconstruction. 1.