Learning vector quantization (LVQ) schemes constitute intuitive, powerful classification heuristics with numerous successful applications but, so far, limited theoretical background. We study LVQ rigorously within a simplifying model situation: two competing prototypes are trained from a sequence of examples drawn from a mixture of Gaussians. Concepts from statistical physics and the theory of on-line learning allow for an exact description of the training dynamics in highdimensional feature space. The analysis yields typical learning curves, convergence properties, and achievable generalization abilities. This is also possible for heuristic training schemes which do not relate to a cost function. We compare the performance of several algorithms, including Kohonen’s LVQ1 and LVQ+/-, a limiting case of LVQ2.1. The former shows close to optimal performance, while LVQ+/- displays divergent behavior. We investigate how early stopping can overcome this difficulty. Furthermore, we study a crisp version of robust soft LVQ, which was recently derived from a statistical formulation. Surprisingly, it exhibits relatively poor generalization. Performance improves if a window for the selection of data is introduced; the resulting algorithm corresponds to cost function based LVQ2. The dependence of these results on the model parameters, for example, prior class probabilities, is investigated systematically, simulations confirm our analytical findings. Keywords: prototype based classification, learning vector quantization, Winner-Takes-All algorithms, on-line learning, competitive learning 1.

We propose a new matrix learning scheme to extend relevance learning vector quantization (RLVQ), an efficient prototype-based classification algorithm, towards a general adaptive metric. By introducing a full matrix of relevance factors in the distance measure, correlations between different features and their importance for the classification scheme can be taken into account and automated, general metric adaptation takes place during training. In comparison to the weighted Euclidean metric used in RLVQ and its variations, a full matrix is more powerful to represent the internal structure of the data appropriately. Large margin generalization bounds can be transfered to this case leading to bounds which are independent of the input dimensionality. This also holds for local metrics attached to each prototype which corresponds to piecewise quadratic decision boundaries. The algorithm is tested in comparison to alternative LVQ schemes using an artificial data set, a benchmark multi-class problem from the UCI repository, and a problem from bioinformatics, the recognition of splice sites for C.elegans.

A large variety of supervised or unsupervised learning algorithms is based on a metric or similarity measure of the patterns in input space. Often, the standard euclidean metric is not sufficient and much more efficient and powerful approximators can be constructed based on more complex similarity calculations such as kernels or learning metrics. This procedure is benefitial for data in euclidean space and it is crucial for more complex data structures such as occur in bioinformatics or natural language processing. In this article, we review similarity based methods and its combination with similarity measures which go beyond the standard Euclidian metric. Thereby, we focus on general unifying principles of learning using non-standard metrics and metric adaptation.

We present a new variant of Generalized Learning Vector Quantization (GRLVQ) in a computer vision scenario. A version with incrementally added prototypes is used for the non-trivial case of high-dimensional object recognition. Training is based upon a generic set of standard visual features, the learned input weights are used for iterative feature pruning. Thus, prototypes and input space are altered simultaneously, leading to very sparse and task-specific representations. The effectiveness of the approach and the combination of the incremental variant together with pruning was tested on the Coil100 database. It exhibits excellent performance with regard to codebook size, feature selection and recognition accuracy. Key words: object recognition, relevance learning, feature selection, incremental learning vector quantization, adaptive metric 1

Clinical proteomics opens the way towards new insights into many diseases on a level of detail not available before. One of the most promising measurement techniques supporting this approach is mass spectrometry based clinical proteomics. The analysis of the high dimensional data obtained from mass spectrometry asks for sophisticated, problem adequate preprocessing and data analysis approaches. Ideally, automatic analysis tools provide insight into their behavior and the ability to extract further information, relevant for an understanding of the clinical data or applications such as biomarker discovery. Prototype based algorithms constitute efficient, intuitive and powerful machine learning methods which are very well suited to deal with high dimensional data and which allow good insight into their behavior by means of prototypical data locations. They have already successfully been applied to various problems in bioinformatics. The goal of this thesis is to extend prototype based methods, in such a way that they become suitable machine learning tools for typical problems in clinical proteomics. To achieve better adapted classification borders, tailored to the specific data distributions

The paper deals with the concept of relevance learning in learning vector quantization.