T. Kohonen, , 1993, "Self-organizing maps: Optimization approaches, in Artificial Neural Networks" ,T. Kohonen, K.Makisara, O.Simula, and J.Kanga, eds., pp..1147-1156. IEEE, New York.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....are positions of neurons and on the SOM grid. Both and decrease monotonically with time. There is also a batch version of the algorithm where the adaptation coefficient is not used [2] In the case of a discrete data set and fixed neighborhood kernel, the error function of SOM can be shown to be [17] (6) where is number of training samples, and is the number of map units. Neighborhood kernel is centered at unit , which is the BMU of vector , and evaluated for unit . If neighborhood kernel value is one for the BMU and zero elsewhere, this leads to minimization of (1) the SOM reduces to ....

T. Kohonen, "Self-organizing maps: Optimization approaches," in Artificial Neural Networks, T. Kohonen, K. Mkisara, O. Simula, and J. Kangas, Eds. Amsterdam, The Netherlands: Elsevier, 1991, pp. 981--990.

....It has been shown that the basic SOM algorithm has no energy function in the general case [26] In case of a discrete data set and xed neighborhood kernel, the map distortion measure E = N X j=1 M X i=1 h bi jjx j m i jj 2 : 3. 8) can be shown to be a local energy function of the SOM [57]. However, when the BMU index b of any of the data samples x j changes, the energy function changes slightly, and thus the SOM only gives an approximate solution to Eq. 3.8. To obtain the exact solution, the de nition of winner (Eq. 3.1) should be changed to b = arg min k f P i h ki jjx j m i ....

Teuvo Kohonen. Self-Organizing Maps: Optimization Approaches. In Kohonen et al. [56], pages 981990.

....of neurons b and i on the SOM grid. Both (t) and (t) decrease monotonically with time. There is also a batch version of the algorithm where the adaptation coecient is not used [2] In the case of a discrete data set and xed neighborhood kernel, the error function of SOM can be shown to be [17] E = N X i=1 M X j=1 h bj jjx i m j jj 2 ; 6) where N is number of training samples and M is the number of map units. Neighborhood kernel h bj is centered at unit b, the BMU of vector x i , and evaluated for unit j. If neighborhood kernel value is one for the BMU and zero elsewhere, this ....

Teuvo Kohonen, \Self-Organizing Maps: Optimization Approaches, " In Kohonen et al. [50], pp. 981-990.

....quantization of unlabelled data seeks to minimize the reconstruction error. This can be accomplished with Competitive learning[Grossberg, 1976; Kohonen, 1982] an iterative learning algorithm for vector quantization that has been shown to perform gradient descent on the following energy function [Kohonen, 1991] Z kx w s (x) k 2 p(x)dx. where p(x) is the probability distribution of the input patterns and w s are the reference or codebook vectors and s (x) is defined by kx w s (x) k kx w i k (for all i) This minimizes the square reconstruction error of unlabelled data and may work ....

Teuvo Kohonen, "Self-Organizing Maps: Optimization Approaches," In T. Kohonen, K. Makisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, pages 981--990. Elsevier Science Publishers, 1991.

.... very nearly minimize (24) It would be interesting to find whether the self organizing algorithm would more efficiently order an initially disordered map if the update rule were modified to more closely follow a gradient descent on the energy function (24) Kohonen has recently taken this approach (Kohonen 1991), and has shown that for the case of a step neighborhood function one can achieve a self organizing algorithm which more nearly follows a gradient descent on the potential (24) by adding small additional terms to the update rule (3) These additional terms are similar to the terms in the ....

....of (24) or whether a general proof of ordering can be constructed for these algorithms. Although the original update rule does not perform a gradient descent of the energy function (24) the final mapping created by the algorithm does very nearly minimize (24) For the step neighborhood function, Kohonen s (1991) revised algorithm creates an ordered mapping in fewer time steps , but with a (slightly) increased computational demand for each step. For more general neighborhood functions, the revised algorithm might require more, rather than fewer, time steps, and the computational requirements per step are ....

Kohonen T (1991) Self-organizing maps: Optimization approaches. In: Kohonen T, et al.

....at least on a small open set. 7 The discrete case In this case, there is a finite number N of inputs f 1 ; 2 ; N g and the input distribution is discrete and uniform. It is the more useful setting for most practical applications, like classification or data analysis. The main result ([16], 24] is that for constant neighborhood (or for suitably decreasing ones [27] the algorithm derives from the potential V (x) 1 N n X i=1 X l 2C i (x) n X j=1 (i Gamma j)kx j Gamma l k 2 ) V is an intra class variance extended to the neighbor classes. But this potential ....

T.Kohonen, "Self-organizing maps : optimization approaches", in : T.Kohonen et al. (eds) Artificial neural networks, vol. II, North Holland, Amsterdam, 981-990, 1991.

....The neighborhood and the gain fl should slowly decrease in time) The convergence and the mathematical properties of this algorithm have been considered by several authors, e.g. 8] 16] 12] 2. 2 The SOM Optimization Problem The map algorithm was related to en energy function in [16] [10]. Let V b denote the set in the input space where Eq. 1) holds, i.e. the Voronoi compartment of unit b. Let p(x) denote the probability density of the inputs x. Define the cost or energy function E(m 1 ; mM ) X i Z V i X k h i;k kx Gamma m k k 2 p(x)dx: 3) The functional (3) ....

....learning rule: m i (t 1) m i (t) Gamma 1 2 fldE 1 =dm i (t) 9) m i (t) flh b;i (x(t) Gamma m i (t) 10) i = 1; n: 11) Thus the original SOFM algorithm (with constant neighborhood function) is a gradient descent method based on sample functions E 1 . It was shown by Kohonen [10] that, when the goal is to minimize the original function E of Eq. 3) extra terms appear in the algorithm due to the discontinuities at the set S. The minimization of E becomes straightforward in a special case when there is no neighborhood, h i;j = ffi i;j : 12) In this case, the learning ....

T. Kohonen, Self-Organizing Maps: optimization approaches, in T. Kohonen, K. Makisara, J. Kangas, and O. Simula (Eds.), Artificial Neural Networks, Vol. 2., North Holland, Amsterdam, 1991, pp. 981 - 990.

....in Section 5. 2. AN ENERGY FUNCTION Our notation is as follows. We have a network of n units ( neurons ) Each unit i has an m dimensional weight vector w i . We use W to denote the set of all n weight vectors. Given an input vector x, Kohonen s learning rule consists of the following steps [1,7,8]. 1. Find the winner : the unit (W; x) with the smallest distance to the input vector, i.e. W; x) argmin i k x Gamma w i k 2 : 1) 2. Update the weights according to Delta w j = jh (W; x) j ( x Gamma w j ) 2) with j a (usually small) learning parameter and h the ....

.... other way around: define an energy function, derive the corresponding learning procedure and check whether it has similar properties as the original (2) A first option would be to derive the learning algorithm that corresponds to the energy function (4) This, however, is quite complicated (see [7]) and, since it does not reproduce the original rule anyways, it might be better to look for simpler options. The choice we made in [11,12] is E(W ) min i e i (W; x) AE ; 6) or, equivalently, the energy function of (4) with a different definition for p(ijW; x) i.e. a different ....

[Article contains additional citation context not shown here]

T. Kohonen. Self-organizing maps: optimization approaches. In T. Kohonen, K. Makisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, pages 981--990, Amsterdam, 1991. North-Holland.

....array of output nodes and one dimensional array of input nodes, which employs an unsupervised learning algorithm known as Kohonen s algorithm. Kohonen s algorithm uses a learning or weight update rule called Kohonen s rule to update the connections weights in the SOM architecture. Kohonen [11] raises some questions about the self organization process, which have remained unanswered so far. Three of them are, 1. Do there exist several, possibly many, optimal algorithms that lead to a similar organization produced by Kohonen s algorithm 2. Does the basic recursive map algorithm ....

....fittest learning rules are found at the end of the experiments. Based on the fitness measure on the tasks, the performance of the learning rules on the tasks is comparable to that of the well known delta rule. 3 Experimental Framework 3. 1 The Self Organizing Map The Self Organizing Map (SOM) [9, 10, 11, 12] is an ANN in which the cells become specifically tuned to various input signal patterns or classes of patterns through an unsupervised learning process. The self organizing process can be considered as a mapping of the probability density function of the high dimensional input data onto the ....

Kohonen, T. (1991). Self-Organizing Maps : Optimization Approaches. In Proc. of the Int. Conf. on ANNs, pp. 981-990, Espoo, Finland.

....the weight vectors will converge is to consider the algorithm (2) as gradient descent minimization of a cost function, and solve for the extremal points of this function. For a continuous density of the input vectors x, an exact cost function that would be minimized by SOM training is not known [14]. It has been shown by Erwin et al. [6] for the one dimensional map, that the map algorithm is not a gradient descent step of any such cost function, although a set of cost functions, one for each neuron unit, can be defined. However, in clustering, the usual assumption is that the set to be ....

T. Kohonen, Self-organizing maps: optimization approaches, Proc. ICANN-91, Espoo, June 24 - 28, 1991, pp. 981 - 990.

....weight vector values m i are adapted (tuned) so that the match of the modified weight vectors in the active area (defined in the neighborhood of the best matching unit) and the input vector x(t) is improved. Some detailed derivations of the adaptation laws have been published (see, for example, Kohonen 1991a] Luttrell 1990] where some criterion function has been optimized. The resulting adaptation laws have the same general form (if Euclidean distance function has been used) m i (t 1) m i (t) ff(t)h(i; c; t) x(t) Gamma m i (t) 8i; 1:2) where ff(t) is a scalar adaptation gain, 0 ff(t) ....

....from some energy functions that are minimized. Traditional techniques (for example, the Liapunov function approach 1 ) could then be used to ensure the convergence properties of the algorithm. The possibility of the existence of energy functions has been studied in many papers, see e.g. Kohonen 1991a] Erwin et al. 1992a] and [Tolat 1990] In [Erwin et al. 1992a] it was shown that for a general case it is not possible to create any such global energy function. The best possible construction would be a system consisting of a set of energy functions, one for each weight value (proposed in ....

[Article contains additional citation context not shown here]

T. Kohonen. Self-organizing maps: Optimization approaches. In T. Kohonen, K. Makisara, O. Simula, J. Kangas, editors, Artificial Neural Networks, pages I--891--990. North-Holland, June 1991.

....Each map is representing a viewpoint. One important preliminary step was to build a categorization of the indexing vocabulary of the patents in order to establish different viewpoints on the patents. This categorization is the following: 1] plants, 2] plant parts, 3] pathogen agents, [4] transgenic techniques, 5] patenting firms. The four first categories correspond to subsets of the indexing vocabulary of the patents. The construction of these categories necessitates some manual labelling of the vocabulary due to the fact that it is only constituted by a flat list of index ....

Kohonen T., 1991, "Self-Organizing Maps: Optimization Approaches." In Artificial Neural Networks, Elsevier Science Publishers B.V, North Holland, p. 981--990,

....to neighboring positions on the map lattice gradually become similar. The process forms a neighborhood preserving mapping of the lattice into the input space. Although the algorithm looks deceptively simple it has turned out to be very difficult to analyze mathematically in the general case (cf. Kohonen, 1991; Kohonen, 1995) In practical applications the data set is always finite, however, and in that case there exists a cost function that the SOM algorithm tries to minimize with stochastic approximation (Robbins and Monro, 1951) The cost function for a fixed neighborhood function h 0 is (Ritter ....

.... In practical applications the data set is always finite, however, and in that case there exists a cost function that the SOM algorithm tries to minimize with stochastic approximation (Robbins and Monro, 1951) The cost function for a fixed neighborhood function h 0 is (Ritter and Schulten, 1988; Kohonen, 1991) 1 2 X k X i h 0 c(xk ) i kx k Gamma m i k 2 : 3) The update defined by equations (1) and (2) corresponds to one step towards the direction of a noisy negative gradient of the cost function. The noisy gradient is here the gradient that has been computed based on one randomly chosen ....

Kohonen, T. (1991). Self-organizing maps: optimization approaches. In Kohonen, T., Makisara, K., Simula, O., and Kangas, J., editors, Artificial Neural Networks, vol. II, pp. 981--990. North-Holland, Amsterdam.

.... at least the local structures of p(x) You might think of p(x) as a flower that is pressed ) Definition of such m i values, however, is far from trivial; a number of people have tried to define them as optima of some objective (energy) function (see e.g. Ritter et al. 1988] Luttrell 1989] [Kohonen 1991], and [Erwin et al. 1992] As the existence of a satisfactory definition is still unclear, we have restricted ourselves in this package to the stochasticapproximation type derivation [Kohonen 1991] that defines the original form of the SOM learning procedure. During learning, those nodes that ....

.... them as optima of some objective (energy) function (see e.g. Ritter et al. 1988] Luttrell 1989] Kohonen 1991] and [Erwin et al. 1992] As the existence of a satisfactory definition is still unclear, we have restricted ourselves in this package to the stochasticapproximation type derivation [Kohonen 1991] that defines the original form of the SOM learning procedure. During learning, those nodes that are topographically close in the array up to a certain distance will activate each other to learn from the same input. Without mathematical proof we state that useful values of the m i can be found as ....

Teuvo Kohonen. Self-organizing maps: Optimization approaches. In Proceedings of the International Conference on Artificial Neural Networks, pages 981--990, Espoo, Finland, June 1991.

No context found.

T. Kohonen, , 1993, "Self-organizing maps: Optimization approaches, in Artificial Neural Networks" ,T. Kohonen, K.Makisara, O.Simula, and J.Kanga, eds., pp..1147-1156. IEEE, New York.

No context found.

T. Kohonen, , 1993, "Self-organizing maps: Optimization approaches, in Artificial Neural Networks" ,T. Kohonen, K.Makisara, O.Simula, and J.Kanga, eds., pp..1147-1156. IEEE, New York.

No context found.

T. Kohonen. Self-Organizing Maps: Optimization approaches. In T. Kohonen, K. M akisara, O. Simula, and J. Kangas, editors, Artificial Neural Networks, volume II, pages 981--990, Amsterdam, Netherlands, 1991. North-Holland.

No context found.

T. Kohonen, Self-organizing maps: optimization approaches, in: T. Kohonen, K. Mikisara, J. Kangas, O. Simula (Eds.), Artificial Neural Networks, Vol. 2, pp. 981-990, North-Holland, Amsterdam, 1991.

No context found.

T. Kohonen, "Self-organizing maps: optimization approaches", in T. Kohonen, K. Makisara and J. Kangas, editors, Artificial Neural Networks, Elsevier Science Publishers, pp. 981-990, 1991.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC