. This paper is an overview of the program package LVQ PAK, which has been developed for convenient and effective application of Learning Vector Quantization algorithms. Two new features are included: fast conflict-free initial distribution of codebook vectors into the class zones, and the optimized-learning-rate algorithm OLVQ1. 1 Introduction Statistical classification or pattern recognition of stochastic vectorial samples can be performed by the Learning Vector Quantization (LVQ) algorithms. Description of their three main variants, LVQ1, LVQ2, and LVQ3 can be found in Ref. [1]. There are certain delicate details in the application of these algorithms; if they are not taken into account, the recognition results and the learning speed may vary significantly. In particular, the following problems have to be solved properly: ffl assignment of an approximately optimal number of codebook vectors to each class and setting of their initial values, ffl selection of a proper learning ra...

The self-organizing feature maps introduced by T. Kohonen use a cell array of fixed size and structure. In many cases this array is not able to model a given signal distribution properly. We present a method to construct two-dimensional cell structures during a self-organization process which are specially adapted to the underlying distribution: Starting with a small number of cells new cells are added successively. Thereby signal vectors according to the (usually not explicitly known) probability distribution are used to determine where to insert or delete cells in the current structure. This process leads to problem dependent cell structures which model the given distribution with arbitrary high accuracy.

This work presents experiments to recognize pattern sequences using hidden Markov models (HMMs). The pattern sequences in the experiments are computed from speech signals and the recognition task is to decode the corresponding phoneme sequences. The training of the HMMs of the phonemes using the collected speech samples is a difficult task because of the natural variation in the speech. Two neural computing paradigms, the Self-Organizing Map (SOM) and the Learning Vector Quantization (LVQ) are used in the experiments to improve the recognition performance of the models. A HMM consists of sequential states which are trained to model the feature changes in the signal produced during the modeled process. The output densities applied in this work are mixtures of Gaussian density functions. SOMs are applied to initialize and train the mixtures to give a smooth and faithful presentation of the feature vector space defined by the corresponding training samples. The SOM maps similar feature vect...

The development of mechanisms that enable robot teams to autonomously generate cooperative behaviours is one of the most interesting issues in dis- tributed and autonomous robotic systems. In this paper, the application of reinforcement learning techniques to robot teams is studied, enabling the robot to learn cooperative behaviours based only on local information.

This dissertation addresses the problem of unsupervised learning for pattern classification or category learning. A model that is based on gross cortical anatomy and implements biologically plausible computations is developed and shown to have classification power approaching that of a supervised discriminant algorithm. The advantage of supervised learning is that the final error metric is available during training. Unfortunately, when modeling human category learning, or in constructing classifiers for autonomous robots, one must deal with not having an omniscient entity labeling all incoming sensory patterns. We show that we can substitute for the labels by making use of structure between the pattern distributions to different sensory modalities. For example the co-occurrence of a visual image of a cow with a "moo" sound can be used to simultaneously develop appropriate visual features for distinguishing the cow image and appropriate auditory features for recognizing the moo. We mode...

This paper is an continuation of the accompanying paper with the same main title. The first paper reviewed instance selection algorithms, here results of empirical comparison and comments are presented. Several test

Anomaly detection involves characterizing the behaviors of individuals or systems and recognizing behavior that is outside the norm. This paper describes some preliminary results concerning the robustness and generalization capabilities of machine learning methods in creating user profiles based on the selection and subsequent classification of command line arguments. We base our method on the belief that legitimate users can be classified into categories based on the percentage of commands they use in a specified period. The hybrid approach we employ begins with the application of expert rules to reduce the dimensionality of the data, followed by an initial clustering of the data and subsequent refinement of the cluster locations using a competitive network called Learning Vector Quantization. Since Learning Vector Quantization is a nearest neighbor classifier, and new record presented to the network that lies outside a specified distance is classified as a masquerader. Thus, this system does not require anomalous records to be included in the training set. 1.

This paper proposes a technique for jointly quantizing continuous features and the posterior distributions of their class labels based on minimizing empirical information loss, such that the index K of the quantizer region to which a given feature X is assigned approximates a sufficient statistic for its class label Y. We derive an alternating minimization procedure for simultaneously learning codebooks in the Euclidean feature space and in the simplex of posterior class distributions. The resulting quantizer can be used to encode unlabeled points outside the training set and to predict their posterior class distributions, and has an elegant interpretation in terms of lossless source coding. The proposed method is extensively validated on synthetic and real datasets, and is applied to two diverse problems: learning discriminative visual vocabularies for bag-of-features image classification, and image segmentation.

Abstract. We study an extension of learning vector quantization that draws on ideas from fuzzy clustering, enabling us to find fuzzy clusters of ellipsoidal shape with a competitive learning scheme. This approach may be seen as a kind of online fuzzy clustering, which can have advantages w.r.t. the execution time of the clustering algorithm. We demonstrate the usefulness of our approach by applying it to web page collections, which are, in general, difficult to cluster due to the high number of dimensions and the special distribution characteristics of the data. 1

This work presents training methods and recognition experiments for phoneme-wise tied mixture densities in hidden Markov models (HMM). The system trains speaker dependent, but vocabulary independent, phoneme models for the recognition of Finnish words. The Learning Vector Quantization (LVQ) methods are applied to increase the discrimination between the phoneme models. A segmental LVQ3 training is proposed to substitute the LVQ2 based corrective tuning as a parameter estimation method. The experiments indicate that the new method can provide the corresponding recognition accuracy, but with less training and more robustness over the initial models. Experiments to upscale the current system by introducing context vectors and larger mixture pools show up to 40 % reduction of recognition errors compared to the earlier results in [10]. 1