Finding structures in vast multidimensional data sets, be they measurement data, statistics, or textual documents, is difficult and time-consuming. Interesting, novel relations between the data items may be hidden in the data. The selforganizing map (SOM) algorithm of Kohonen can be used to aid the exploration: the structures in the data sets can be illustrated on special map displays. In this work, the methodology of using SOMs for exploratory data analysis or data mining is reviewed and developed further. The properties of the maps are compared with the properties of related methods intended for visualizing highdimensional multivariate data sets. In a set of case studies the SOM algorithm is applied to analyzing electroencephalograms, to illustrating structures of the standard of living in the world, and to organizing full-text document collections. Measures are proposed for evaluating the quality of different types of maps in representing a given data set, and for measuring the robu...

Neural maps project data given in a (possibly high-dimensional) input space onto a neuron position in a (usually low-dimensional) output space grid. An important property of this projection is the preservation of neighborhoods; neighboring neurons in output space respond to neighboring data points in input space. To achieve this preservation in an optimal way during learning, the topology of the output space has to roughly match the effective structure of the data in the input space. We here present a growth algorithm, called the GSOM, which enhances a widespread map self-organization process, Kohonen's Self-Organizing Feature Map (SOFM), by an adaptation of the output space grid during learning. During the procedure the output space structure is restricted to a general hypercubical shape, with the overall dimensionality of the grid and its extensions along the different directions being subject of the adaptation. This constraint distinguishes the present algorithm from other, less or ...

Abstract. The Self-Organizing Map (SOM) and Learning Vector Quantization (LVQ) algorithms are constructed in this work for variable-length and warped feature sequences. The novelty is to associate an entire feature vector sequence, instead of a single feature vector, as a model with each SOM node. Dynamic time warping is used to obtain time-normalized distances between sequences with different lengths. Starting with random initialization, ordered feature sequence maps then ensue, and Learning Vector Quantization can be used to fine tune the prototype sequences for optimal class separation. The resulting SOM models, the prototype sequences, can then be used for the recognition as well as synthesis of patterns. Good results have been obtained in speaker-independent speech recognition.

Neural maps combine the representation of data by codebook vectors, like a vector quantizer, with the property of topography, like a continuous function. While the quantization error is simple to compute and to compare between different maps, topography of a map is difficult to define and to quantify. Yet, topography of a neural map is an advantageous property, e.g. in the presence of noise in a transmission channel, in data visualization, and in numerous other applications. In this paper we review some conceptual aspects of definitions of topography, and some recently proposed measures to quantify topography. We apply the measures first to neural maps trained on synthetic data sets, and check the measures for properties like reproducability, scalability, systematic dependence of the value of the measure on the topology of the map etc. We then test the measures on maps generated for four real-world data sets, a chaotic time series, speech data, and two sets of image data. The measures ...

In this paper a Recurrent Self-Organizing Map (RSOM) algorithm is proposed for temporal sequence processing. The RSOM algorithm is close in nature to the Kohonen's Self-Organizing Map, except that in the RSOM context of the temporal sequence is involved both in the best matching unit finding and in the adaptation of the weight vectors of the map via an introduced recursive difference equation associated for each unit of the map. The experimental results in the paper demonstrate that the RSOM is able to learn and distinguish temporal sequences, and that the RSOM algorithm can be utilized, for instance, in electroencephalogram (EEG) based epileptic activity detection.

This paper argues for the development of synthetic approaches towards the study of brain and behavior as a complement to the more traditional empirical mode of research. As an example we present our own work on learning and problem solving which relates to the behavioral paradigms of classical and operant conditioning. We de ne the concept of learning in the context of behavior and lay out the basic methodological requirements a model needs to satisfy, which includes evaluations using robots. In addition, we de ne a number of design principles neuronal models should obey to be considered relevant. We present in detail the construction of a neural model of short- and long-term memory which can be applied to an arti cial behaving system. The presented model (DAC4) provides a novel self-consistent implementation of these processes, which satis es our principles. This model will be interpreted towards the present understanding of the neuronal substrate of memory.

A newly proposed Recurrent Self-Organizing Map (RSOM) is studied in time series prediction. In this approach RSOM is used to cluster the data to local data sets and local linear models corresponding each of the map units are then estimated based on the local data sets. A traditional way of clustering the data is to use a windowing technique to split it to input vectors of certain length. In this procedure, the temporal context between the consecutive vectors is lost. In RSOM the map units keep track of the past input vectors with a recurrent dioeerence vector in each unit. The recurrent structure allows the map to store information concerning the change in the magnitude and direction of the input vector. RSOM can thus be used to cluster the temporal context in the time series. This allows a dioeerent local model to be selected based on the context and the current input vector of the model. The studied cases show promising results.

Vector quantization consists in nding a discrete approximation of a continuous input. One of the most popular neural algorithms related to vector quantization is the, so called, Kohonen map. In this paper we generalize vector quantization to temporal data, introducing context quantization. We propose a recurrent network inspired by the Kohonen map, the Contextual Self-Organizing Map, that develops near-optimal representations of context. We demonstrate quantitatively that this algorithm shows better performance than the other neural methods proposed so far. 1. Introduction The temporal context present in sequential data is crucial for sequence processing and prediction of future events. An element of a sequence is context-dependent when it cannot be predicted from only one previous element, but also needs additional information provided by the preceding inputs [13]. Most neural techniques for sequence learning are based on recurrent networks, where the context is represented by the ...

A novel neural model made up of two self-organizing map nets --- one on top of the other --- is introduced and analysed experimentally. The model makes an effective use of context information, and that enables it to perform sequence classification and discrimination efficiently. It was successfully applied to real sequences, taken from the third voice of the sixteenth four-part fugue in G minor of the Well-Tempered Clavier (vol. I) of J. S. Bach. The model has application in domains which require pattern recognition, or more specifically, which demand recognizing either a set of sequences of vectors in time or sub-sequences into a unique and large sequence of vectors in time. 1 Introduction Several researchers have extended the Kohonen's self-organizing feature map model [1] to recognize sequential information. The problem involves either recognizing a set of sequences of vectors in time or recognizing sub-sequences inside a large and unique sequence. Some approaches are described be...

Recurrent Self-Organizing Map (RSOM) is studied in temporal sequence processing. RSOM includes a recurrent difference vector in each unit of the map, which allows storing temporal context from consecutive input vectors fed to the map. RSOM is a modification of the Temporal Kohonen Map (TKM). It is shown that RSOM learns a correct mapping from temporal sequences of a simple synthetic data, while TKM fails to learn this mapping. In addition, two case studies are presented, in which RSOM is applied to EEG based epileptic activity detection and to time series prediction with local models. Results suggest that RSOM can be efficiently used in temporal sequence processing.