Martinetz, T., Berkovich, S., and Schulten, K. (1993). "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE-Transactions on Neural Networks 4(4): 558-569.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....from a database collected by USPS in a real postal plant [2] our database distribution re ects the prior distribution of that site. For this reason some letters are less represented than others or almost absent. Clustering performed with Self Organizing Maps (SOM) 10] and Neural Gas (NG) [12], showed that, for some letters, the vectors corresponding to the upper and lower case version are distributed in di erent regions of the feature space. This can happen when upper and lower case characters are di erent in shape (e.g. g and G) In this case, it was useful to consider the two ....

T. Martinetz, S. Berkovich, K. Schulten, "Neural Gas" network for vector quantization and its application to time-series prediction, IEEE Transactions on Neural Networks, Vol. 4, No. 4, 558-569, 1993.

....a given population of individuals into similarity groups has many applications in science and business. Many clustering algorithms have been suggested and used in the literature to partition data into clusters (see [1] 2] 3] 4] 5] 6] 7] 8] 9] 10] 11] 12] 13] 14] 15] 16] [17]) When partitioning individuals into plausible subgroups, due to the origin of the data sets and also to the algorithms themselves, some main problems for this task are encountered. The true structure (ground truth) especially the number and shapes of the clusters, remains unknown. Different ....

Thomas M. Martinetz, Stanislav G. Berkovich, and Klaus J. Schulten. "Neural-Gas" network for vector quantization and its application to timeseries prediction. 4(4):558--569, July 1993.

....application of self organizing paradigms to non vectorial data, such as time series or more general graph structures, is not straightforward. A variety of self organizing models for time series processing has been proposed, a class of which using embeddings into a finite dimensional vector space [9, 12], but for which a standard rectangular lattice or the Euclidean metric is seldom appropriate for matching the possibly complex data topology. Other approaches model sequential data recursively [3, 4, 13, 14, 15] see e.g. 1] for an overview. Unsupervised sequence processors using recurrent ....

.... latter model operates with a hyperbolic lattice, i.e. a lattice with exponentially increasing neighborhood size [11] the former NG model adapts all neurons according to their rank with respect to the distance from the current data point, i.e. according to a data driven optimal lattice structure [9] without topology constraints. Several popular models extend SOM by means of recurrent connections: assume a sequence of data points in where a t , an entry t of the sequence at time t, is presented: TKM recursively computes the distance of neuron j from a t as d j (a t ) 1 #) d j (a ....

T. Martinetz, S. Berkovich, and K. Schulten. "Neuralgas " network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4:558--569, 1993.

....neural network. Section 3 explains the application of the model to a kinematic arm model and presents the results, which are discussed in Sec. 4. 2 Unsupervised learning and recall in an abstract RNN 2.1 Training The training algorithm is an extension of Neural Gas to local PCA. Neural Gas [4] is a robust vector quantization technique with soft competition between the units. It is an online method, where the model is updated after each presented pattern. A network contains N units, with unit index k = 1, N. In Neural Gas, each unit contains a center vector ck. For each ....

Martinetz, T. M., Berkovich, S. G., Schulten, K. J.: "Neural-Gas" Network for Vector Quantization and its Application to Time-Series Prediction. IEEE Transactions on Neural Networks 4 (1993) 558-569

....several important approaches proposed in the literature for SOMs with recurrence. The reported models are commonly trained with Hebbian learning. The general formulation allows to formalize Hebbian learning in a uniform manner and to immediately transfer alternatives like the neural gas algorithm [33] or vector quantization to the approaches. For standard vector based SOM and alternatives like neural gas, Hebbian Learning can be (approximately) interpreted as a stochastic gradient descent method on an appropriate error function [22, 33] One can uniformly formulate analogous cost functions for ....

....transfer alternatives like the neural gas algorithm [33] or vector quantization to the approaches. For standard vector based SOM and alternatives like neural gas, Hebbian Learning can be (approximately) interpreted as a stochastic gradient descent method on an appropriate error function [22, 33]. One can uniformly formulate analogous cost functions for the general framework for structural self organizing maps and investigate the connection to Hebbian learning. It turns out that Hebbian learning can be interpreted as an approximation of a gradient mechanism where contributions of ....

[Article contains additional citation context not shown here]

T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE-Transactions on Neural Networks 4(4): 558-569, 1993.

....1. Introduction Vector Quantization (VQ) techniques allow to quantify only interesting part of an input space by minimizing a cost function and placing the representing codebook vectors over the input data distribution respecting its density. Neural Gas networks (NGs) have been introduced in [1] to improve VQ techniques converging quickly to a lower distortion error than other methods and obeying a gradient descent on an energy surface. A NG is a set of neurons such as each one is located in an input space thanks to an input vector called the centrod of that neuron. An output vector can ....

....a correspondence between the input space and the output space. Any input data is associated to the output of the closest neuron in the sense of the Euclidean distance. All the input data which are closer to the same neuron determine its Vorono region in the input space. NGs have been used in [1] for function approximation tasks such as each neuron is in charge of the approximation of the output values associated to the input vectors of its own Vorono region. Their accuracy and speed in approximating a function is depending on mainly three parameters: the number of neurons used to ....

[Article contains additional citation context not shown here]

T.M. Martinetz, S.G. Berkovich and K.J. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4), 558-569, 1993.

....213 Annexe C. Quanti cation vectorielle et fonction d nergie 214 C.4. Conclusion section 4, we show that these results apply to the Kohonen rule used in SelfOrganizing Maps and in Fritzke s Growing Neural Gas (Fritzke 1995) and to other rules presented in the literature as the Neural Gas (Martinetz 1993), the Recruiting rule (Aupetit 2000) and the Observable Neighbors rule (Aupetit 2001a; Aupetit 2001b) 2 Behavior around Vorono region boundaries In their recent work (Lepetz 2001) Lepetz and Nemoz Gaillard prove the following theorem. Let D be a non empty part of R d (d 1) ....

....Self Organizing Maps and Growing Neural Gas verify the hypothesis of Theorem 2: their rule corresponds at least to a stochastic descent onto the na ve energy function E V , even if a datum v falls on a Vorono region boundary. 4.2 Martinetz s Neural Gas 4.2. 1 Theorem 1 In the Neural Gas (Martinetz 1993), the neighborhood function takes the form: i (w(t) v) h (k i (w; v) with k i (w(t) v) n X q=1 H(d 2 iq d 2 qq ) and =0 (19) where h is of the same form as in (15) and k i is the rank of the unit i such that k i (w; v) l 1 i i is the l th closest unit to v (note that ....

T.M. Martinetz, S.G.Berkovitch, K.J.Schulten, \NeuralGas " Network for Vector Quantization and its Application to Time-Series Prediction, In IEEE Trans. on Neural Networks, vol. 4, no.4, pp.558-569, 1993.

....mapping (9) using an algorithm designed especially for function approximating networks [7] The training data and the surface generated by the trained network are plotted in Figure 1. The rule extraction algorithm (2.1) was then applied to produce several sets of rules. The Neural Gas algorithm [8] was used as a clustering tool. After clustering the linear prototypes (2) and the consequence value (1) were calculated for each rule. The approximation algorithm (2.2) was subsequently used to reconstruct (9) using the obtained rule based models. For every model the root mean square error (RMSE) ....

T. M. Martinetz, S. Berkovich, and K. Schulten, ""Neural-gas" network for vector quantization and its application to time series prediction," IEEE Transactions on Neural Networks, vol. 4, pp. 558--569, 1993.

....# [1, N ] 1) The goal is to minimise the reconstruction error E = # d D xP (x)#x w i(x) # 2 (2) Here P (x) is the probability distribution of data vectors over the manifold #. A straightforward approach for clustering and VQ is the well known K means algorithm. Its on line version is [7]: #w i = # #(x w i ) if i = i(x) 0, otherwise (3) with # as the learning rate. Such an on line learning rule is likely to be trapped in local minima. A solution to this is to adopt some soft computing schemes in which not only the winner prototype is modified, but all reference ....

....for visualisation purpose, but it also limits its data modelling ability, as the data manifold can be rather complicated. The size of SOM is also fixed and therefore it is not an ideal choice for on line tasks. The constraint of a low dimensional map topology is removed in the neural gas model [7], with a learning rule similar to SOM, but the prototype vectors are organised in the original manifold of the input space. The weight updating rule is similar to that of SOM, but requiring the calculation of neighbourhood rank of the prototypes related to the current input, since no topology ....

T.M. Martinetz, S.G. Berkovich and K.J. Schulten: "Neural-Gas" network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4 (1993), 558-569.

....ffl kmeans: k means algorithm [6] Note that if you set the final neighborhood radius to zero during training a SOM, the final energy function equals that of the k means. ffl kmeans clusters: try and evaluate several k means partitionings ffl neural gas: neural gas vector quantization algorithm [16] 4.3.3 Modeling functions Although the SOM is not specifically meant for modeling, it is very easy to build local and nearest neighbor models on top of the SOM. Below are some functions which can be used for modeling. ffl lvq1: LVQ1 classification algorithm [12] ffl lvq3: LVQ3 classification ....

T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4):558--569, 1993.

....of the original data set using a certain number of cluster centers. The di erence between the algorithms is the way the cluster centers are updated on each adaptation step. Algorithm Notes k means [1] Only best matching (closest) cluster center of the sample vector is updated. maximum entropy [19, 25] All cluster centers are updated according to their distance to the sample vector. neural gas [19] All cluster centers are updated according to their ranking order in distance to the sample vector. SOM The cluster centers are updated according to their distance from the BMU of the sample ....

....is the way the cluster centers are updated on each adaptation step. Algorithm Notes k means [1] Only best matching (closest) cluster center of the sample vector is updated. maximum entropy [19, 25] All cluster centers are updated according to their distance to the sample vector. neural gas [19] All cluster centers are updated according to their ranking order in distance to the sample vector. SOM The cluster centers are updated according to their distance from the BMU of the sample vector on the map grid (h ck ) ization. The number of visual dimensions determines how many di erent ....

T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. \neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transaction on Neural Networks, 4(4):558-569, 1993.

....and more stable than the basic SOM. On the other hand, the prototypes no longer have well de ned positions on a low dimensional grid and thus visualization is more problematic. Neural gas. Neural gas is another variant of the SOM where the neighborhoods are adapatively de ned during training [77]. Neighborhoods are de ned by the ranking order of the distance of prototype vectors from the given training sample. 19 Growing Cell Structures. In growing cell structures algorithm the adaptability has been taken one step further [29, 30] Instead of having a xed number of prototype vectors, ....

T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. Neural-gas network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4):558569, 1993.

.... Recently several extensions to non Euclidean distances have been proposed [5] 6] Popular partitioning algorithms include classic methods like the k means algorithm and its online variants (which are often called hard competitive learning) More recent algorithms like the neural gas algorithm [7] or SOMs [8] also fall into this category, but add some regularization terms to Equation 1, which control the structure of the set of centers C K by enforcing neighborhood topologies among the centers. B. Hierarchical Methods Hierarchical methods do not try to nd a segmentation with a xed ....

T. M. Martinetz, S. G. Berkovich, and K. J. Schulten, \\NeuralGas " network for vector quantization and its application to time-series prediction," IEEE Transactions on Neural Networks, vol. 4, pp. 558-569, July 1993.

....is applied based on the ranking of the distance to the reference vectors, not on the distance to the winning PE in the lattice. The Neural Gas algorithm has been shown to converge quickly to low distortion errors which are smaller than k means, maximum entropy clustering or the SOM algorithm [19]. It has no predefined neighborhood structure as in the SOM and for this reason works better on disjoint or complicated input spaces. The dynamics of the GASTAD algorithm are the same as in the SOMTAD algorithm except that the neural gas algorithm does not have a predefined lattice structure. This ....

T.M. Martinetz, S.G. Berkovich, K.J. Schulten, "Neural-Gas" Network for Vector Quantization and its Application to Time-Series Prediction, IEEE Transactions on Neural Networks, Vol. 4, No. 4, July 1993, pp. 558-569.

....In a statistical framework Cherkassky [8] proposes a local linear extension to the SOM for the purpose of improving the approximation to the regression surface, which albeit similar to Ritter s work differs in the global nature of the regression. Previous applications of SOM for dynamic modeling [23], 48] only used the competitive properties of the neural model to create the codebook. Recently, Vesanto [55] proposed a scheme that essentially followed our topology [35] 5.1 Dynamic Learning in the SOM We propose below an improvement to Kohonen learning for the special task of dynamic ....

Martinetz, T.M., S.G. Berkovich and K.J. Schulten, ""Neural-Gas" network for vector quantization and its application to time-series prediction," IEEE Trans. on Neural Networks, Vol. 4, No. 4, July 1993.

....final deletion will always raise the error significantly above the desired error criterion, and the training of the network will be repeated indefinitely. 4. Experimental Results The power of large and non trivial networks ought to be tested on large and non trivial problems. Martinetz et al. [6] described an interesting problem to illustrate the Neural Gas algorithm. In an instance of this problem, illustrated in Figure 2(a) the input space is filled with a number of disjoint square shaped clusters, all of which are sufficiently far enough from one another to be clearly distinguishable. ....

T. M. Martinetz, S. G. Berkovich and K. J. Schulten, "Neural-Gas" Network for Vector Quantization and its Application to Time-Series Prediction, IEEE Transactions on Neural Networks, vol. 4, pp. 558-569, July 1993.

....values, and these confidence values are combined to obtain the final classification. Excellent recognition rates for several benchmark datasets are presented. 1 Introduction An innovative vector quantization technique called the Neural Gas algorithm was recently introduced by Martinetz et al. [9]. In this paper, a new hierarchical architecture, based on the neural gas algorithm, is presented. This architecture is called the hierarchical overlapped neural gas (HONG) network, and we demonstrate its applicability to labeled pattern classification by testing on several widely available ....

....neural gas algorithm and description of the HONG network architecture in Section 2, we present our results on the various datasets in Section 3. The paper is concluded with a brief discussion in Section 4. 2 The HONG Network Classifier 2. 1 The Neural Gas Algorithm The Neural Gas (NG) algorithm [9] is similar to the Kohonen s self organizing feature map [8] but adapts the reference vectors w i without any fixed topological arrangement of the neural units within the network. 1 Instead, it utilizes a neighborhood ranking of the reference vectors w i for a given data vector v. The reference ....

T. M. Martinetz, S. G. Berkovich, and K. J. Schulten. "Neural Gas" Network for Vector Quantization and its Application to Time-Series Prediction. IEEE Transactions on Neural Networks, 4(4):558--569, July 1993.

No context found.

Martinetz, T., Berkovich, S., and Schulten, K. (1993). "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE-Transactions on Neural Networks 4(4): 558-569.

No context found.

T.M. Martinetz, S.G. Berkovich, K.J. Schulten, "Neural-Gas" Network for Vector Quantization and its Applicat ion to TimeSeries Prediction, IEEE Transactions on Neural Networks, Vol. 4, No. 4, July 1993, pp. 558-569.

No context found.

Martinetz, T.M., Berkovich, S.G., Schulten, K.J., "Neural-Gas" Network for Vector Quantization and its Application to Time-Series Prediction, IEEE Transactions on Neural Networks, Vol. 4, No.4, pp.558-569, July, 1993.

No context found.

T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4):558--569, 1993.

No context found.

T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE-Transactions on Neural Networks 4(4): 558-569, 1993.

No context found.

T.M. Martinetz, S.G. Berkovich and K.J. Schulten: "Neural-Gas" network for vector quantization and its application to time-series prediction. IEEE Trans. on Neural Networks 4 (1993), 558-569.

No context found.

T. Martinetz, S. Berkovich, and K. Schulten. "Neuralgas " network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4:558--569, 1993.

No context found.

T. Martinetz, S. Berkovich, and K. Schulten. "Neural-gas" network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4):558--569, 1993.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC