N. Saravanan and D. Fogel, "Evolving Neural Control Systems," IEEE Expert, vol. 10, no. 3, pp. 23--27, 1998.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... been applied to uni objective NN design, genetic algorithms (GAs) evolution strategies (ES) and particle swarm optimisation (PSO) GAs have previously be used for feature selection [8, 53] and topography selection [2, 5, 29, 35, 36, 38, 52] and ESs have been used for weight optimisation [21, 42, 45, 55], and adaptive topography selection [15, 37, 57] The recent EC technique of PSO [27] has also proved popular as a uni objective NN optimiser [10, 12, 13, 26, 48] 2 Multi objective evolutionary neural network flamework The use of evolutionary approaches to NN training (with a single error ....

....(4) where w is the i tn weight of the n tn network in the population at the ktn epoch of training, 0, w) is i,k some multiplier and p takes a value (0, 1) with some probability. This can be seen as similar to the common approach taken in uni objective ENN optimisers for weight adjustment [21, 42, 45, 55], and was the only means of parameter adjustment used in [18, 30] 2.1.3 Weight addition deletion Finally connectivity (and therefore complexity) is adjusted within the Pareto ENN model by a GA type bit mu tator, where at each generation every weight has a small probability of being severed ....

N. Saravanan and D.B. Fogel. Evolving Neural Control Systems. IEEE Expert, June:23-27, 1998.

....Generally, a population of candidate solutions, ranked by their performance, are maintained and updated iteratively by evolutionary search. Each candidate solution represents one neural network in which the weights can be encoded as a string of binary [14,81,82] or floatingpoint numbers [24,52,66,71]. The performance of each solution is determined by the network error function which is to be optimized by evolutionary search. Unlike local search, evolutionary search maintains a population of potential solutions rather than a single solution. Therefore, the risk of getting stuck in local optima ....

N. Saravanan and D. B. Fogel. Evolving neural control systems. IEEE Expert, 10(3):23--27, 1995.

....[115] 116] has given representations other than ary strings a more solid theoretical foundation. Real numbers have been proposed to represent connection weights directly, i.e. one real number per connection weight [27] 29] 30] 48] 63] 65] 74] 95] 96] 102] 110] 111] [117], 118] For example, a realnumber representation of the ANN given by Fig. 3(a) could be (4.0,10.0,2.0,0.0,7.0,3.0) As connection weights are represented by real numbers, each individual in an evolving population will be a real vector. Traditional binary crossover and mutation can no longer be ....

....that they can reduce the negative impact of the permutation problem. Hence the evolutionary process can be more efficient. There have been a number of successful examples of applying EP or ES to the evolution of ANN connection weights [29] 63] 65] 67] 68] 95] 96] 102] 106] 111] [117], 119] 120] In these examples, the primary search operator has been Gaussian mutation. Other mutation operators, such as Cauchy mutation [121] 122] can also be used. EP and ES also allow self adaptation of strategy parameters. Evolving connection weights by EP can be implemented as ....

[Article contains additional citation context not shown here]

N. Saravanan and D. B. Fogel, "Evolving neural control systems, " IEEE Expert, vol. 10, pp. 23--27, Mar. 1995.

....problem of balancing two poles simultaneously without giving velocity inputs to the network. This problem is a known benchmark in the reinforcement learning literature, which makes it possible to compare NEAT to other methods. Pole balancing has been used in RL and NE research for over 30 years [1, 3, 6, 7, 9, 11, 15, 17 20]. It is also a good surrogate for real problems, in part because pole balancing in fact is a real task, and also because the difficulty can be adjusted. We present the hardest such problem, balancing two pole simultaneously without velocities, in order to show that NEAT performs well on a ....

....will explain how each ablation was performed and interpret the results. B. No growth Ablation In order to make no growth NEAT comparable to fixedtopology NE, it was allowed to start with a fully connected hidden layer of 10 hidden units, the same number as in past fixed topology NE experiments [15, 19]. Without growth, the system was only able to use weight differences to speciate the population. Given 1,000 generations to find a solution, the ablated system could only find a solution 20 of the time When it did find a solution, it took 8.5 times more evaluations than full NEAT. Clearly, ....

N. Saravanan and D. B. Fogel. Evolving neural control systems. IEEE Expert, pages 23--27, June 1995.

....approaches . Saravanan 15 Method Evaluations Generations No. Nets Ev. Programming 307,200 150 2048 Conventional NE 80,000 800 100 SANE 12,600 63 200 ESP 3,800 19 200 NEAT 3,600 24 150 Table 1: Double Pole Balancing with Velocity Information. Evolutionary programming results were obtained by Saravanan and Fogel (1995). Conventional neuroevolution data was reported by Wieland (1991) SANE and ESP results were reported by Gomez and Miikkulainen (1999) NEAT results are averaged over 120 experiments. All other results are averages over 50 runs. The standard deviation for the NEAT evaluations is 2,704 evaluations. ....

....are fixed start with a fully connected hidden layer of neurons (Wieland 1991) Therefore, to make the experiment fair, the no growth ablation was also allowed to start with a fully connected hidden layer. Every genome specified 10 hidden units like the fixed topology methods in this task (Saravanan and Fogel 1995; Wieland 1991) Without growth, the system was only able to use weight differences to speciate the population. Given 1,000 generations to find a solution, the ablated system could only find a solution 20 of the time When it did find a solution, it took 8.5 times more evaluations than full NEAT. ....

Saravanan, N., and Fogel, D. B. (1995). Evolving neural control systems. IEEE Expert, 23--27.

....multiple error measures. A method of error smoothing is also introduced as an attempt to solve the current stopping problem associated with ES (and GA) trained NNs. The use of Evolutionary and Genetic approaches to Neural Network training has received increasing attention in recent years [4, 5, 6, 7, 9, 12, 14]. Indeed the ability of these approaches to facilitate NN training beyond the Euclidean objective was highlighted by Porto et al. [9] but apparently taken no further. Other limited approaches to multi objective training do appear in the literature, but in the form of simultaneously choosing the ....

....approaches to multi objective training do appear in the literature, but in the form of simultaneously choosing the network topology as well as Euclidean training e.g. 7] 5 Diagram 1, generic population selection. The ES proposed for use in this multi objective learning model is that used in [4, 9, 12] (in the latter two it is referred to as Evolutionary Programming) and is shown in Eq. 3. Here the weight space of a network is perturbed by some values drawn from a known distribution each epoch (generation) Q = g k i n k j i n w w , 1 , 3) where w n,I,j,k is the weight between ....

[Article contains additional citation context not shown here]

Saravanan, N. and Fogel, D.B., "Evolving Neural Control Systems", IEEE Expert, June, pp23-27, 1995.

.... [130, 257] Fuzzy Sets and Systems (Netherlands) 345] 11 12 Genetic algorithms and neural networks IEEE Aerospace and Electronic Systems Magazine, 186] IEEE Computer Society Technical Committee on Microprogramming and Microarchitecture, 890] IEEE Control Systems, 185] IEEE Expert, [276, 282, 355, 698, 943] IEEE Expert (USA) 489] IEEE Potentials, 699] IEEE Trans. Ind. Appl. USA) 564] IEEE Transaction on Neural Networks, 576] IEEE Transactions on Circuits and Systems I, Fundamental Theory and Applications, 647] IEEE Transactions on Evolutionary Computing, 504] IEEE Transactions ....

....Fariselli, Piero, 224, 558] Fekadu, Adhanom A. 761] Feldman, David S. 675] Ferguson, J. J. 676] Filelis, A. 871] Fleischhauer, T. 407] Fleming, Peter J. 589] Floreano, Dario, 677] Floyd, C. E. 160] Fogarty, Terence C. 241, 378, 436, 471, 474, 678] Fogel, David B. [83, 276, 282, 313, 315, 679, 680, 681, 682, 683, 684, 685, 686, 687, 688, 689, 690, 691, 692] Fogel, Lawrence J. 276, 681, 685, 687, 691] Foo, Shou King, 170] Forst, C. V. 248] Fortuna, L. 639, 640] Foy, Mark, 693] Foy, M. 694] Franco, Aurali B. 888] Fredriksson, Kimmo, 496, 517] Freedman, M. T. 487] Freisleben, Bernd, 695] French, I. G. 171] Frenzel, James ....

[Article contains additional citation context not shown here]

N. Saravanan and David B. Fogel. Evolving neural control systems. IEEE Expert, 10(3):23--27, June 1995. ga95bSaravanan.

....6 Gamma Gamma Gamma Gamma Gamma Gamma Delta Delta Delta Delta Delta Delta A A A A A AK 2 10 4 3 7 0010 0000 0100 1010 0011 0111 Figure 4: a) An ANN which is equivalent to that given in Figure 3(a) b) Its binary representation under the same representation scheme. connection weight [27, 29, 30, 48, 117, 63, 64, 65, 95, 96, 74, 111, 102, 110, 118]. For example, a realnumber representation of the ANN given by Figure 3(a) could be (4.0,10.0,2.0,0.0,7.0,3.0) As connection weights are represented by real numbers, each individual in an evolving population will be a real vector. Traditional binary crossover and mutation can no longer be used ....

....One of the major advantages of using mutation based EAs is that they can reduce the negative impact of the permutation problem. Hence the evolutionary process can be more efficient. There have been a number of successful examples of applying EP or ES to the evolution of ANN connection weights [29, 117, 63, 64, 65, 95, 96, 119, 120, 67, 111, 102, 68, 106]. In these examples, the primary search operator has been Gaussian mutation. Other mutation operators, such as Cauchy mutation [121, 122] can also be used. EP and ES also allow self adaptation of strategy parameters. Evolving connection weights by EP can be implemented as follows: 1. Generate an ....

[Article contains additional citation context not shown here]

N. Saravanan and D. B. Fogel, "Evolving neural control systems," IEEE Expert, vol. 10, no. 3, pp. 23--27, 1995.

....from X to E. Evolutionary programming (EP) has been successfully applied in the design and optimization of neural networks for applications requiring nonlinear transformations for control (Saravanan and Fogel, 1994) time series prediction (Yao and Liu, 1995) and many other applications (Saravanan and Fogel, 1995; Fogel et al. 1997; Harrald and Kamstra, 1997; Heimes et al. 1997) In this study, feedforward neural networks with one hidden layer were considered. The output layer consisted of a single node that performed a linear combination of the inputs from the hidden layer to generate the output. Such ....

Saravanan N and Fogel DB (1995), "Evolving Neural Control Systems," IEEE Expert, vol.10, no.3, June, pp. 23-27.

....poles are very close in length, solutions to this system cannot be evolved directly by current methods. In order to control the system under these conditions, shaping (or incremental learning) techniques can be employed that increase the length of the shorter pole very gradually [Wieland, 1991; Saravanan and Fogel, 1995] This kind of approach is effective but can be extremely slow due to the limitations of the underlying evolutionary search method many generations are required to recover from minute changes to the environment. Using an incremental approach in conjunction with a local search technique ....

....in these approaches the complex task is broken into simpler components or subtasks that are each learned by separate systems (e.g. GAs or rule bases) and then combined to achieve the goal task. In contrast, in incremental evolution as proposed in this paper (and also used by Wieland [1991] and Saravanan and Fogel [1995] ) a single system learns a succession of tasks. Such an adaptation process is similar to continual (or lifelong) learning [Elman, 1991; Ring, 1994] and motivated by staged learning in real life. 4 Pole Balancing Experiments The starting point for our experiments is the more challenging ....

[Article contains additional citation context not shown here]

N. Saravanan and David B. Fogel. Evolving neural control systems. IEEE Expert, pages 23--27, June 1995.

....system are useful for the understanding of the system, a modeling method satisfying both the precision and the clarity of the resulting model is desirable. In this paper we address the knowledge acquisition with clear linguisitic meanings for the fuzzy modeling and propose a new approach using EP[2], which consists of modeling phase and re evaluation phase. This paper introduces an explanation degree for the re evaluation of the acquired fuzzy model. The acquired fuzzy rules are re evaluated for clarifying their linguistic meanings as the trade off with the precision of the model. EP is a ....

N. Saravanan and D. B. Fogel, "Evolving Neural Control Systems," IEEE EXPERT, 10(3), pp.2327, June 1995.

....these approaches the complex task is broken into simpler components or subtasks that are each learned by separate systems (e.g. GAs or rule bases) and then combined to achieve the goal task. In contrast, in incremental evolution as proposed in this paper and also used by Wieland (1990, 1991) and Saravanan and Fogel (1995), a single system learns a succession of tasks. Such an adaptation process is similar to continual (or lifelong) learning (Elman 1991; Ring 1994; Thrun 1996) and motivated by learning in real life. If, for instance, the goal task is that of driving a Formula 1 race car at Grand Prix level, we can ....

....problem in an average of 10 generations. ESP solved the same task in as many generations on average (over 10 simulations) but tested less than half as many networks (200 instead of 512 per generation) An even more challenging problem is to place a second pole next to the first. In this task, Saravanan and Fogel (1995) used Evolutionary Computation techniques to evolve a feed forward network with 10 hidden units to balance the poles (1m and 0.1m) in an average of 800 generations. ESP was able to perform the same task in an average of 45 generations (averaged over 10 simulations) These results show that ESP is ....

Saravanan, N., and Fogel, D. B. (1995). Evolving neural control systems. IEEE Expert, 23--27.

No context found.

N. Saravanan and D. Fogel, "Evolving Neural Control Systems," IEEE Expert, vol. 10, no. 3, pp. 23--27, 1998.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC