An improved technique for 3D head tracking under varying illumination conditions is proposed. The head is modeled as a texture mapped cylinder. Tracking is formulated as an image registration problem in the cylinder's texture map image. The resulting dynamic texture map provides a stabilized view of the face that can be used as input to many existing 2D techniques for face recognition, facial expressions analysis, lip reading, and eye tracking. To solve the registration problem in the presence of lighting variation and head motion, the residual error of registration is modeled as a linear combination of texture warping templates and orthogonal illumination templates. Fast and stable on-line tracking is achieved via regularized, weighted least squares minimization of the registration error. The regularization term tends to limit potential ambiguities that arise in the warping and illumination templates. It enables stable tracking over extended sequences. Tracking does not require a precise initial fit of the model; the system is initialized automatically using a simple 2D face detector. The only assumption is that the target is facing the camera in the first frame of the sequence. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PC's, and game consoles. The non-optimized implementation runs at about 15 frames per second on a SGI O2 graphic workstation. Extensive experiments evaluating the effectiveness of the formulation are reported. The sensitivity of the technique to illumination, regularization parameters, errors in the initial positioning and internal camera parameters are analyzed. Examples and applications of tracking are reported. 1 1
|
606
|
Neural networkbased face detection
– Rowley, Baluja, et al.
- 1998
|
|
604
|
Active Appearance Models
– Cootes, Edwards, et al.
- 1998
|
|
427
|
Probabilistic visual learning for object representation
– Moghaddam, Pentland
- 1997
|
|
409
|
P.W.: Feature extraction from faces using deformable templates
– Yuille, Cohen, et al.
- 1994
|
|
372
|
Closed-form solution of absolute orientation using orthonormal matrices
– Horn, Hilden, et al.
- 1988
|
|
337
|
EigenTracking: Robust matching and tracking of articulated objects using a view-based representation
– Black, Jepson
- 1998
|
|
256
|
Efficient region tracking with parametric models of geometry and illumination
– Hager, Belhumeur
- 1998
|
|
200
|
Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion
– Black, Yacoob
- 1995
|
|
184
|
Coding analysis interpretation recognition of facial expressions
– Essa, Pentland
- 1997
|
|
162
|
Rank-deficient and discrete ill-posed problems
– Hansen
- 1998
|
|
154
|
Analysis and synthesis of facial image sequences using physical and anatomical models
– �Terzopoulos, Waters
- 1993
|
|
120
|
A mixed-state CONDENSATION tracker with automatic model-switching
– Isard, Blake
- 1998
|
|
116
|
3D motion estimation in model-based facial image coding
– Li, Roivainen, et al.
- 1993
|
|
109
|
Recognizing facial expressions in image sequences using local parameterized models of image motion
– Yacoob
- 1997
|
|
99
|
Motion regularization for model-based head tracking
– Basu, Essa, et al.
- 1996
|
|
99
|
Image analysis using multigrid relaxation methods
– Terzopoulos
- 1986
|
|
89
|
A low-dimensional representation of human faces for arbitrary lighting conditions
– Hallinan
- 1994
|
|
88
|
Multi-modal tracking of faces for video communications
– Crowley, Berard
- 1997
|
|
86
|
The integration of optical flow and deformable models with applications to human face shape and motion estimation
– DeCarlo
- 1996
|
|
80
|
Computing spatio-temporal representations of human faces
– Yacoob, Davis
- 1994
|
|
78
|
Visually controlled graphics
– Azarbayejani, Starner, et al.
- 1993
|
|
76
|
Active blobs
– Sclaroff, Isidoro
- 1998
|
|
73
|
Multiresolution Image Processing and Analysis
– Rosenfeld
- 1984
|
|
61
|
Geometry and photometry in 3d visual recognition
– Shashua
- 1992
|
|
61
|
Parametrized structure from motion for 3d adaptive feedback tracking of faces
– Jebara, Pentland
- 1997
|
|
55
|
reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3D models
– Fast
|
|
49
|
D.: Color-based tracking of heads and other mobile objects at video frame rates
– Fieguth, Terzopoulos
- 1997
|
|
47
|
Lafter: Lips and face real time tracker
– Oliver, Pentland, et al.
- 1997
|
|
46
|
A.: Parameterized structure from motion for 3D adaptive feedback tracking of faces
– Jebara, Pentland
- 1997
|
|
40
|
Computing 3D head orientation from a monocular image sequence
– Horprasert, Yacoob, et al.
|
|
39
|
Projective registration with difference decomposition
– Gleicher
- 1997
|
|
34
|
Deformable templates for feature extraction from medical images
– Lipson, Yuille, et al.
- 1990
|
|
25
|
Head tracking using a textured polygonal model
– Schodl, Haro, et al.
- 1998
|
|
22
|
Head Tracking via Robust Registration in Texture Map Images
– Cascia, Sclaroff
- 1998
|
|
22
|
Incremental focus of attention for robust visionbased tracking
– Toyama, Hager
- 1999
|
|
20
|
Jacobian images of super-resolved texture maps for model-based motion estimation and tracking
– Dellaert, Thrun, et al.
- 1998
|
|
16
|
Super-resolved texture tracking of planar surface patches
– Dellaert, Thorpe, et al.
- 1998
|
|
14
|
An elliptical head tracker
– Birchfield
- 1997
|
|
10
|
Color based tracking of heads and other mobile objects at video frame rates
– Fieguth, Terzopoulos
- 1997
|
|
8
|
Lafter: Lips and face real time tracker. CVPR
– Olivier, Pentland, et al.
- 1997
|
|
7
|
Multi-modal tracking of faces for video communications. CVPR
– Crowley, Berard
- 1997
|
|
6
|
Active appearance model. ECCV
– Cootes, Edwards, et al.
- 1998
|
|
6
|
ªNeural Network-Based Face Detection,º
– Rowley, Baluja, et al.
- 1998
|
|
5
|
ªProbabilistic Visual Learning for Object Representation,º
– Moghaddam, Pentland
- 1997
|
|
4
|
ªGeometry and Photometry in 3D Visual Recognition,º
– Shashua
- 1992
|
|
3
|
ªActive Appearance Models,º
– Cootes, Edwards, et al.
- 1998
|
|
2
|
ªMotion Regularization for ModelBased Head Tracking,º
– Basu, Essa, et al.
- 1996
|
|
2
|
ªMulti-Modal Tracking of Faces for Video Communications,º
– Crowley, Berard
- 1997
|
|
2
|
ªProjective Registration with Difference Decomposition,º
– Gleicher
- 1997
|
|
2
|
ªComputing 3-D Head Orientation from a Monocular Image
– Horprasert, Yacoob, et al.
- 1996
|
