In this work we consider face recognition from Face Motion Manifolds (FMMs). The use of the Resistor-Average Distance (RAD) as a dissimilarity measure between densities confined to FMMs is motivated in the proposed information-theoretic approach to modelling face appearance. We introduce a kernel-based algorithm that makes use of the simplicity of the closed-form expression for RAD between two Gaussian densities, while allowing for modelling of complex and nonlinear, but intrinsically low-dimensional manifolds. Additionally, it is shown how geodesically local FMM structure can be modelled, naturally leading to a stochastic algorithm for generalizing to unseen modes of data variation. Recognition performance of our method is demonstrated experimentally and is shown to exceed that of state-of-the-art algorithms. Recognition rate of 98% was achieved on a database of 100 people under varying illumination.

Illumination invariance remains the most researched, yet the most challenging aspect of automatic face recognition. In this paper we propose a novel, general recognition framework for efficient matching of individual face images, sets or sequences. The framework is based on simple image processing filters that compete with unprocessed greyscale input to yield a single matching score between individuals. It is shown how the discrepancy between illumination conditions between novel input and the training data set can be estimated and used to weigh the contribution of two competing representations. We describe an extensive empirical evaluation of the proposed method on 171 individuals and over 1300 video sequences with extreme illumination, pose and head motion variation. On this challenging data set our algorithm consistently demonstrated a dramatic performance improvement over traditional filtering approaches. We demonstrate a reduction of 50--75% in recognition error rates, the best performing method-filter combination correctly recognizing 96% of the individuals.

Abstract. This paper proposes the kernel orthogonal mutual subspace method (KOMSM) for 3D object recognition. KOMSM is a kernel-based method for classifying sets of patterns such as video frames or multi-view images. It classifies objects based on the canonical angles between the nonlinear subspaces, which are generated from the image patterns of each object class by kernel PCA. This methodology has been introduced in the kernel mutual subspace method (KMSM). However, KOMSM is different from KMSM in that nonlinear class subspaces are orthogonalized based on the framework proposed by Fukunaga and Koontz before calculating the canonical angles. This orthogonalization provides a powerful feature extraction method for improving the performance of KMSM. The validity of KOMSM is demonstrated through experiments using face images and images from a public database. 1

Our goal is to automatically determine the cast of a feature-length film. This is challenging because the cast size is not known, with appearance changes of faces caused by extrinsic imaging factors (illumination, pose, expression) often greater than due to differing identities. The main contribution of this paper is an algorithm for clustering over face appearance manifolds. Specifically: (i) we develop a novel algorithm for exploiting coherence of dissimilarities between manifolds, (ii) we show how to estimate the optimal dataset-specific discriminant manifold starting from a generic one, and (iii) we describe a fully automatic, practical system based on the proposed algorithm. The performance of the system is evaluated on well-known featurelength films and situation comedies on which it is shown to produce good results. 1.

In pattern recognition, feature vectors are occasionally subject to non-negative constraints. This characteristic can be expressed by a cone in feature vector space. In this paper, we propose cone-restricted subspace methods. The proposed methods admit the scaling and additivity of vectors as well as ordinary subspace methods; in addition, vectors can be strictly classified at the boundary of the cone. Some experimental results for face and person detection demonstrate the effectiveness of the proposed methods. 1.

Object recognition using image-set or video sequence as input tends to be more robust since image-set or video sequence provides much more information than single snap-shot about the variability in the appearance of the target subject. Constrained Mutual Subspace Method (CMSM) is one of the state-of-the-art algorithms for imageset based object recognition by first projecting the image-set patterns onto the so-called generalized difference subspace then classifying based on the principal angle based mutual subspace distance. By treating the subspace bases for each image-set patterns as basic elements in the grassmann manifold, this paper presents a framework for robust image-set based recognition by CMSMbased ensemble learning in a boosting way. The proposed Boosting Constrained Mutual Subspace Method(BCMSM) improves the original CMSM in the following ways: a) The proposed BCMSM algorithm is insensitive to the dimension of the generalized differnce subspace while the performance of the original CMSM algorithm is quite dependent on the dimension and the selecting of optimum choice is quite empirical and case-dependent; b) By taking advantage of both boosting and CMSM techniques, the generalization ability is improved and much higher classification performance can be achieved. Extensive experiments on real-life data sets (two face recognition tasks and one 3D object category classification task) show that the proposed method outperforms the previous state-ofthe-art algorithms greatly in terms of classification accuracy. 1

Non-linear subspaces derived using kernel methods have been found to be superior compared to linear subspaces in modeling or classification tasks of several visual phenomena. Such kernel methods include Kernel PCA, Kernel DA, Kernel SVD and Kernel QR. Since incremental computation algorithms for these methods do not exist yet, the practicality of these methods on large datasets or online video processing is minimal. We propose an approximate incremental Kernel SVD algorithm for computer vision applications that require estimation of non-linear subspaces, specifically face recognition by matching image sets obtained through long-term observations or video recordings. We extend a well-known linear subspace updating algorithm to the nonlinear case by utilizing the kernel trick, and apply a reduced set construction method to produce sparse expressions for the derived subspace basis so as to maintain constant processing speed and memory usage. Experimental results demonstrate the effectiveness of the proposed method. 1.

In this paper we consider face recognition from sets of face images and, in particular, recognition invariance to illumination. The main contribution is an algorithm based on the novel concept of Maximally Probable Mutual Modes (MMPM). Specifically: (i) we discuss and derive a local manifold illumination invariant and (ii) show how the invariant naturally leads to a formulation of "common modes" of two face appearance distributions. Recognition is then performed by finding the most probable mode, which is shown to be an eigenvalue problem. The effectiveness of the proposed method is demonstrated empirically on a challenging database containing the total of 700 video sequences of 100 individuals.

The objective of this work is to recognize faces using video sequences both for training and novel input, in a realistic, unconstrained setup in which lighting, pose and user motion pattern have a wide variability and face images are of low resolution. There are three major areas of novelty: (i) illumination generalization is achieved by combining coarse histogram correction with fine illumination manifold-based normalization; (ii) pose robustness is achieved by decomposing each appearance manifold into semantic Gaussian pose clusters, comparing the corresponding clusters and fusing the results using an RBF network; (iii) we describe a fully automatic recognition system based on the proposed method and an extensive empirical evaluation on 600 head motion video sequences with extreme illumination, pose and motion pattern variation. On this challenging data set our system consistently demonstrated a very high recognition rate (95% on average).

Abstract—We consider the problem of classification of an object given multiple observations that possibly include different transformations. The possible transformations of the object generally span a low-dimensional manifold in the original signal space. We propose to take advantage of this manifold structure for the effective classification of the object represented by the observation set. In particular, we design a low complexity solution that is able to exploit the properties of the data manifolds with a graph-based algorithm. Hence, we formulate the computation of the unknown label matrix as a smoothing process on the manifold under the constraint that all observations represent an object of one single class. It results into a discrete optimization problem, which can be solved by an efficient and low complexity algorithm. We demonstrate the performance of the proposed graph-based algorithm in the classification of sets of multiple images. Moreover, we show its high potential in video-based face recognition, where it outperforms state-of-the-art solutions that fall short of exploiting the manifold structure of the face image data sets. Index Terms—Graph-based classification, multiple observations sets, video face recognition, multi-view object recognition. I.