Computer vision and image recognition research have a great interest in dimensionality reduction techniques. Generally these techniques are independent of the classifier being used and the learning of the classifier is carried out after the dimensionality reduction is performed, possibly discarding valuable information. In this paper we propose an iterative algorithm that simultaneously learns a linear projection base and a reduced set of prototypes optimized for the Nearest-Neighbor classifier. The algorithm is derived by minimizing a suitable estimation of the classification error probability. The proposed approach is assessed through a series of experiments showing a good behavior and a real potential for practical applications. 1.

Dimensionality reduction plays a fundamental role in data processing, for which principal component analysis (PCA) is widely used. In this paper, we develop the Laplacian PCA (LPCA) algorithm which is the extension of PCA to a more general form by locally optimizing the weighted scatter. In addition to the simplicity of PCA, the benefits brought by LPCA are twofold: the strong robustness against noise and the weak metric-dependence on sample spaces. The LPCA algorithm is based on the global alignment of locally Gaussian or linear subspaces via an alignment technique borrowed from manifold learning. Based on the coding length of local samples, the weights can be determined to capture the local principal structure of data. We also give the exemplary application of LPCA to manifold learning. Manifold unfolding (non-linear dimensionality reduction) can be performed by the alignment of tangential maps which are linear transformations of tangent coordinates approximated by LPCA. The superiority of LPCA to PCA and kernel PCA is verified by the experiments on face recognition (FRGC version 2 face database) and manifold (Scherk surface) unfolding. 1.

In this paper, we develop a geometric framework for linear or nonlinear discriminant subspace learning and classification. In our framework, the structures of classes are conceptualized as a semi-Riemannian manifold which is considered as a submanifold embedded in an ambient semi-Riemannian space. The class structures of original samples can be characterized and deformed by local metrics of the semi-Riemannian space. Semi-Riemannian metrics are uniquely determined by the smoothing of discrete functions and the nullity of the semi-Riemannian space. Based on the geometrization of class structures, optimizing class structures in the feature space is equivalent to maximizing the quadratic quantities of metric tensors in the semi-Riemannian space. Thus supervised discriminant subspace learning reduces to unsupervised semi-Riemannian manifold learning. Based on the proposed framework, a novel algorithm, dubbed as Semi-Riemannian Discriminant Analysis (SRDA), is presented for subspace-based classification. The performance of SRDA is tested on face recognition (singular case) and handwritten capital letter classification (nonsingular case) against existing algorithms. The experimental results show that SRDA works well on recognition and classification, implying that semi-Riemannian geometry is a promising new tool for pattern recognition and machine learning. 1.

I would like to thank my thesis advisor, Sridhar Mahadevan. Sridhar has been such a wonderful advisor, and every aspect of this thesis has benefitted from his guidance and support throughout my graduate studies. I also like to thank Sridhar for giving me the flexibility to explore many different ideas and research topics. I am appreciative of the support offered by my other thesis committee members, Andrew McCallum, Erik Learned-Miller, and Weibo Gong. Andrew helped me on CRF, MALLET and topic modeling. Erik helped me on computer vision. Weibo has many brilliant ideas on how brains work. I got a lot of inspirations from him. I am grateful for many other professors and staff members, who helped me along. Andy Barto offered me many insightful comments and advice on my research. David Kulp and Oliver Brock helped me on bioinformatics. Stephen Scott brought me to this country, taught me machine learning/bioinformatics and offers me constant support. Vadim Gladyshev helped me on biochemistry. Mauro Maggioni helped me on diffusion wavelets. I also thank Gwyn Mitchell and Leanne Leclerc for their help with my questions over the years. I am deeply thankful to my Master thesis advisor, Zhuzhi Yuan and other teachers in Nankai University for guiding my development as

Abstract. This paper describes a novel framework for learning discriminative features, where both labeled and unlabeled data are used to map the data instances to a lower dimensional space, preserving both class separability and data manifold topology. In contrast to linear discriminant analysis (LDA) and its variants (like semi-supervised discriminant analysis), which can only return c−1featuresforaproblemwithc classes, the proposed approach can generate d features, where d is bounded only by the dimensionality of the original problem. The proposed framework can be used with both two class and multiple class problems. It can also be adapted to problems where class labels are continuous. We describe and evaluate the new approach both theoretically and experimentally, and compare its performance with other state of the art methods.

Most text mining tasks, including clustering and topic detection, are based on statistical methods that treat text as bags of words. Semantics in the text is largely ignored in the mining process, and mining results often have low interpretability. One particular challenge faced by such approaches lies in short text understanding, as short texts lack enough content from which statistical conclusions can be drawn easily. In this paper, we improve text understanding by using a probabilistic knowledgebase that is as rich as our mental world in terms of the concepts (of worldly facts) it contains. We then develop a Bayesian inference mechanism to conceptualize words and short text. We conducted comprehensive experiments on conceptualizing textual terms, and clustering short pieces of text such as Twitter messages. Compared to purely statistical methods such as latent semantic topic modeling or methods that use existing knowledgebases (e.g., WordNet, Freebase and Wikipedia), our approach brings significant improvements in short text understanding as reflected by the clustering accuracy. 1

Maximum Margin Criterion (MMC) based Feature Extraction method is more efficient than LDA for calculating the discriminant vectors since it does not need to calculate the inverse within-class scatter matrix. However, MMC ignores the discriminative information within the local structures of samples. In this paper, we develop a novel criterion to address the issue, namely Local Maximum Margin Criterion (Local MMC). We define the total laplacian matrix, within-class laplacian matrix and between-class laplacian matrix using the samples similar weighting. Local MMC gets the discriminant vectors by maximizing the difference between between-class laplacian matrix and within-class laplacian matrix. Experiments on FERET face database show the effectiveness of the proposed Local MMC based feature extraction method. 1.

This paper describes a novel framework to jointly learn data-dependent label and locality-preserving projections. Given a set of data instances from multiple classes, the proposed approach can automatically learn which classes are more similar to each other, and construct discriminative features using both labeled and unlabeled data to map similar classes to similar locations in a lower dimensional space. In contrast to linear discriminant analysis (LDA) and its variants, which can only return c − 1 features for a problem with c classes, the proposed approach can generate d features, where d is bounded only by the number of the input features. We describe and evaluate the new approach both theoretically and experimentally, and compare its performance with other state of the art methods.

Discriminant feature extraction plays a central role in pattern recognition and classification. Linear Discriminant Analysis (LDA) is a traditional algorithm for supervised feature extraction. Recently, unlabeled data have been utilized to improve LDA. However, the intrinsic problems of LDA still exist and only the similarity among the unlabeled data is utilized. In this paper, we propose a novel algorithm, called Semisupervised Semi-Riemannian Metric Map (S³RMM), following the geometric framework of semi-Riemannian manifolds. S³RMM maximizes the discrepancy of the separability and similarity measures of scatters formulated by using semi-Riemannian metric tensors. The metric tensor of each sample is learned via semisupervised regression. Our method can also be a general framework for proposing new semisupervised algorithms, utilizing the existing discrepancy-criterion-based algorithms. The experiments demonstrated on faces and handwritten digits show that S 3 RMM is promising for semisupervised feature extraction.