V. Porto, D. Fogel, and L. Fogel, "Alternative Neural Network Training Methods," IEEE Expert, vol. 10, no. 3, pp. 16--22, 1995.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....a changing environment. In the literature, research into EANN has been taking one of three approaches; evolving the weights of the network, evolving the architecture, or evolving both simultaneously. The EANN approach uses either binary representation to evolve the weight matrix [12, 13] or real [7, 8, 9, 18, 19, 23]. There is not an obvious advantage of binary encoding in EANN over the real. However, with real encoding, there are more advantages including compact and natural representation. The key problem (other than being trapped in a local minimum) with BP and other traditional training algorithms is the ....

V.W. Porto, D.B. Fogel, and L.J. Fogel. Alternative neural network training methods. IEEE Expert, 10(3):16--22, 1995.

....a changing environment. In the literature, research into EANN has been taking one of three approaches; evolving the weights of the network, evolving the architecture, or evolving both simultaneously. The EANN approach uses either binary representation to evolve the weight matrix [10, 11] or real [6, 7, 16, 19]. There is not an obvious advantage of binary encoding in EANN over the real. However, with real encoding, there are more advantages including compact and natural representation. The key problem (other than being trapped in a local minimum) with BP and other traditional training algorithms is the ....

V.W. Porto, D.B. Fogel, and L.J. Fogel. Alternative neural network training methods. IEEE Expert, 10(3):16--22, 1995.

.... been applied to uni objective NN design, genetic algorithms (GAs) evolution strategies (ES) and particle swarm optimisation (PSO) GAs have previously be used for feature selection [8, 53] and topography selection [2, 5, 29, 35, 36, 38, 52] and ESs have been used for weight optimisation [21, 42, 45, 55], and adaptive topography selection [15, 37, 57] The recent EC technique of PSO [27] has also proved popular as a uni objective NN optimiser [10, 12, 13, 26, 48] 2 Multi objective evolutionary neural network flamework The use of evolutionary approaches to NN training (with a single error ....

....error measures, for which a derivative may be extremely costly to calculate, can be easily incorporated in EC approaches to training, as derivatives are not needed. Indeed the ability of these approaches to facilitate NN training beyond the Euclidean objective was highlighted by Porto et al. [42] (although not with multiple objectives) but taken no further in [42] In addition, they benefit by training a population of evolutionary neural networks (ENNs) in one run, making them highly compatible with the concepts of population based multi objective training from the MOEA literature. There ....

[Article contains additional citation context not shown here]

V.W. Porto, D.B. Fogel, and L.J. Fogel. Alternative Neural Network Training Methods. IEEE Expert, June:16- 21, 1995.

....Generally, a population of candidate solutions, ranked by their performance, are maintained and updated iteratively by evolutionary search. Each candidate solution represents one neural network in which the weights can be encoded as a string of binary [14,81,82] or floatingpoint numbers [24,52,66,71]. The performance of each solution is determined by the network error function which is to be optimized by evolutionary search. Unlike local search, evolutionary search maintains a population of potential solutions rather than a single solution. Therefore, the risk of getting stuck in local optima ....

V. W. Porto, D. B. Fogel, and L. J. Fogel. Alternative neural networks training methods. IEEE Expert, 10(3):16--22, 1995.

....and operators based on the concept of equivalent classes [115] 116] has given representations other than ary strings a more solid theoretical foundation. Real numbers have been proposed to represent connection weights directly, i.e. one real number per connection weight [27] 29] 30] 48] [63] [65] 74] 95] 96] 102] 110] 111] 117] 118] For example, a realnumber representation of the ANN given by Fig. 3(a) could be (4.0,10.0,2.0,0.0,7.0,3.0) As connection weights are represented by real numbers, each individual in an evolving population will be a real vector. ....

....of the major advantages of using mutation based EA s is that they can reduce the negative impact of the permutation problem. Hence the evolutionary process can be more efficient. There have been a number of successful examples of applying EP or ES to the evolution of ANN connection weights [29] [63] [65] 67] 68] 95] 96] 102] 106] 111] 117] 119] 120] In these examples, the primary search operator has been Gaussian mutation. Other mutation operators, such as Cauchy mutation [121] 122] can also be used. EP and ES also allow self adaptation of strategy parameters. ....

[Article contains additional citation context not shown here]

V. W. Porto, D. B. Fogel, and L. J. Fogel, "Alternative neural network training methods," IEEE Expert, vol. 10, pp. 16--22, Mar. 1995.

....of connection weights wherein each weight is represented by 4 bits. 1 5 2 3 Output 4 8 7 3 1 5 Genotype: 0100 1000 0111 0011 0001 0101 Figure 15. Connection weight chromosome encoding using binary representation Real numbers have been proposed to represent connection weights directly [66]. A representation of the ANN could be (2.0, 6.0, 5.0, 1.0, 4.0, 10.0) However proper genetic operators are to be chosen depending upon the representation used. Evolutionary Search of connection weights can be formulated as follows: 19 1) Generate an initial population of N weight chromosomes. ....

Porto V W, Fogel D B and Fogel L J (1995), Alternative neural Network training methods, IEEE Expert, volume 10, no.4, pp. 16-22.

....if the training process is considered as a global search of optimal connection weights wherein the architecture and learning rules of the ANN are pre defined and fixed during the search process. Connection weights may be represented as binary strings of a certain length or as real numbers directly [13]. The whole network is encoded by concatenation of all the connection weights of the network in the chromosome. A heuristic concerning the order of the concatenation is to put connection weights to the same node together. Figure 2 illustrates the binary representation of connection weights wherein ....

) Porto, V.W., Fogel, D.B. and Fogel, L.J. (1995): Alternative Neural Network Training Methods, IEEE Expert, volume 10, no.4, pp. 16-22.

....A heuristic concerning the order of the concatenation is to put connection weights to the same node together. Fig 4 illustrates the binary representation of connection weights wherein each weight is represented by 4 bits. Real numbers have been proposed to represent connection weights directly [10]. A representation of the ANN could be (4.0, 8.0, 7.0, 3.0, 1.0, 5.0) However proper genetic operators are to be chosen depending upon the representation used. Global search of connection weights using the hybrid heuristic can be formulated as follows: 1) Generate an initial population of N ....

Porto V.W., Fogel D.B., Fogel L.J.: Alternative Neural Network Training Methods, IEEE Expert, volume 10, no.4, pp. 16-22, (1995).

....multiple error measures. A method of error smoothing is also introduced as an attempt to solve the current stopping problem associated with ES (and GA) trained NNs. The use of Evolutionary and Genetic approaches to Neural Network training has received increasing attention in recent years [4, 5, 6, 7, 9, 12, 14]. Indeed the ability of these approaches to facilitate NN training beyond the Euclidean objective was highlighted by Porto et al. [9] but apparently taken no further. Other limited approaches to multi objective training do appear in the literature, but in the form of simultaneously choosing the ....

....(and GA) trained NNs. The use of Evolutionary and Genetic approaches to Neural Network training has received increasing attention in recent years [4, 5, 6, 7, 9, 12, 14] Indeed the ability of these approaches to facilitate NN training beyond the Euclidean objective was highlighted by Porto et al. [9], but apparently taken no further. Other limited approaches to multi objective training do appear in the literature, but in the form of simultaneously choosing the network topology as well as Euclidean training e.g. 7] 5 Diagram 1, generic population selection. The ES proposed for use in this ....

[Article contains additional citation context not shown here]

Porto, V.W., Fogel, D.B. and Fogel, L.J., "Alternative Neural Network Training Methods", IEEE Expert, June, pp16-21,1995.

.... [130, 257] Fuzzy Sets and Systems (Netherlands) 345] 11 12 Genetic algorithms and neural networks IEEE Aerospace and Electronic Systems Magazine, 186] IEEE Computer Society Technical Committee on Microprogramming and Microarchitecture, 890] IEEE Control Systems, 185] IEEE Expert, [276, 282, 355, 698, 943] IEEE Expert (USA) 489] IEEE Potentials, 699] IEEE Trans. Ind. Appl. USA) 564] IEEE Transaction on Neural Networks, 576] IEEE Transactions on Circuits and Systems I, Fundamental Theory and Applications, 647] IEEE Transactions on Evolutionary Computing, 504] IEEE Transactions ....

....Fariselli, Piero, 224, 558] Fekadu, Adhanom A. 761] Feldman, David S. 675] Ferguson, J. J. 676] Filelis, A. 871] Fleischhauer, T. 407] Fleming, Peter J. 589] Floreano, Dario, 677] Floyd, C. E. 160] Fogarty, Terence C. 241, 378, 436, 471, 474, 678] Fogel, David B. [83, 276, 282, 313, 315, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692] Fogel, Lawrence J. 276, 681, 685, 687, 691] Foo, Shou King, 170] Forst, C. V. 248] Fortuna, L. 639, 640] Foy, Mark, 693] Foy, M. 694] Franco, Aurali B. 888] Fredriksson, Kimmo, 496, 517] Freedman, M. T. 487] Freisleben, Bernd, 695] French, I. G. 171] Frenzel, James ....

[Article contains additional citation context not shown here]

Vincent W. Porto, David B. Fogel, and Lawrence J. Fogel. Alternative neural network training methods. IEEE Expert, 10(3):16--22, June 1995. ga95bPorto.

....EOS, 147] European Journal of Operational Research, 103, 205, 61, 29] Europhysics Letters, 237, 241] Guangxue Xuebao, 245] Helvetica Physica Acta, 141] IEE Proceedings C: Generation, Transmission and Distribution, 76, 187] IEE Proceedings Comput. Digital Tech. 223] IEEE Expert, [20] IEEE Trans. Parallel Distrib. Syst. USA) 125] IEEE Transaction on Information Technology in Biomedicine, 232] IEEE Transactions on Computer Aided Design of Integrated Circuits and Systems, 261, 264] IEEE Transactions on Magn. 102] IEEE Transactions on Magnetics, 201, 248] IEEE ....

....Ebeling, Werner, 237] Elazar, J. M. 121] El Keib, A. A. 154] Elmer, B. S. 140] Enokura, T. 133] Esbensen, Henrik, 168] Esfarjani, Keivan, 116] Evans, D. J. 17] Evans, G. 40] Fan, Alex, 136] Ferrier, G. D. 12] Finnerty, S. 11] Flann, N. S. 155, 125] Fogel, David B. [20] Fogel, Lawrence J. 20] Fox, B. L. 244] Fox, G. C. 13] 14 Genetic algorithms and simulated annealing Franconi, L. 109] Frazer, L. N. 147] Friesner, R. A. 157] Friesner, Richard A. 220] Fu, Rong Tang, 116] Fu, Yan, 233] Fujimoto, H. 172] Fushuan, Wen, 74] Gambardella, Luca ....

[Article contains additional citation context not shown here]

Vincent W. Porto, David B. Fogel, and Lawrence J. Fogel. Alternative neural network training methods. IEEE Expert, 10(3):16-22, June 1995. ga95bPorto.

....or even continuous since EAs do not depend on gradient information. Because EAs can treat large, complex, nondifferentiable and multimodal spaces, which are the typical case in the real world, considerable research and application has been conducted on the evolution of connection weights [24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112]. The evolutionary approach to weight training in ANNs consists of two major phases. The first phase is to decide the representation of connection weights, i.e. whether in the form of binary strings or not. The second one is the evolutionary process simulated by an EA, in which search operators ....

....6 Gamma Gamma Gamma Gamma Gamma Gamma Delta Delta Delta Delta Delta Delta A A A A A AK 2 10 4 3 7 0010 0000 0100 1010 0011 0111 Figure 4: a) An ANN which is equivalent to that given in Figure 3(a) b) Its binary representation under the same representation scheme. connection weight [27, 29, 30, 48, 117, 63, 64, 65, 95, 96, 74, 111, 102, 110, 118]. For example, a realnumber representation of the ANN given by Figure 3(a) could be (4.0,10.0,2.0,0.0,7.0,3.0) As connection weights are represented by real numbers, each individual in an evolving population will be a real vector. Traditional binary crossover and mutation can no longer be used ....

[Article contains additional citation context not shown here]

V. W. Porto, D. B. Fogel, and L. J. Fogel, "Alternative neural network training methods," IEEE Expert, vol. 10, no. 3, pp. 16--22, 1995.

.... value was sampled from this distribution and added to the weight: m 0 i = m i exp(Gaussian( 0; oe = 0:1) w 0 ij = w ij Gaussian( 0; oe = m 0 i ) for j = 1 : n Evolutionary algorithms such as this one have previously been shown to produce good results for training ANN s (e.g. [15], 5] 10] The minimum SSE in the population over time for both the training and test sets is shown for a typical run in figure 2. SSE was measured for the test set to observe the change of generalization performance over time, but only SSE on the training set was used for the purposes of the ....

Vincent W. Porto, David B. Fogel, and Lawrence J. Fogel. Alternative neural network training methods. IEEE Expert, 10(3):16--22, 1995.

....are trajectory methods [43] and branch and bound methods [124,249] The deficiency of these methods is that they can only solve small networks. Stochastic global optimization methods are more successful in solving the learning problem of neural networks. These include random search algorithms [10, 35, 198], simulated annealing [68, 230] and genetic algorithms [95, 114, 171, 273] They have shown improved performance with respect to local optimization algorithms. However, their execution time can be significantly longer. One thing that needs to be mentioned is that neural network learning has also ....

V. W. Porto, D. B. Fogel, and L. J. Fogel. Alternative neural network training methods. IEEE Expert, pages 16--22, June 1995.

....of the set of exemplars. Thus any operator that varies the weights must be a genetic, not a phenotypic, operator. Within evolution strategies and evolutionary programming, the general approach to neural network training is to vary the weights by a multivariate Gaussian random variable (e.g. [19]) just as is the normal procedure for simple real valued function optimization problems. But the rationale for selecting the Gaussian operator in the case of optimizing neural weights is much different. The focus of attention is on the changes that are generated to the network s output (i.e. its ....

V.W. Porto, D.B. Fogel, L.J. Fogel, "Alternative Neural Network Training Methods," IEEE Expert, Vol. 10;3, June, 1995, pp. 16-22.

No context found.

V. Porto, D. Fogel, and L. Fogel, "Alternative Neural Network Training Methods," IEEE Expert, vol. 10, no. 3, pp. 16--22, 1995.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC