Abstract

Gradient descent techniques such as back propagation have been used effectively to train neural network connection weights; however, in some applications gradient information may not be available. Biologically inspired genetic algorithms provide an alternative. Unfortunately, early attempts to use genetic algorithms to train connection weights demonstrated that exchanging genetic material between two parents with the crossover operator often leads to low performance children. This occurs because the genetic material is removed from the context in which it was useful due to incompatible feature-detector mappings onto hidden units. This paper explores an approach in which a traditional genetic algorithm using standard two-point crossover and mutation is applied within the Cascade-Correlation learning architecture to train neural network connection weights. In the Cascade-Correlation architecture the hidden unit feature detector mapping is static; therefore, the possibility of the crossover operator shifting genetic material out of its useful context is reduced. 1 Problem Statement Although gradient descent techniques such as back propagation have been used effectively to train feedforward neural network connection weights, researchers have experimented with evolving an optimal set of connection weights with biologically inspired genetic algorithms. Genetic algorithms are adaptive search algorithms in which a population of individuals representing alternative solutions to a problem are allowed to propagate based on a measure of their fitness (Holland 1975). As the population evolves, the fitness of the individuals increases through the application of genetic recombination operators. Genetic algorithms have several advantages over the back propagation algorithm. First, because genetic algorithms work on a population of solutions in parallel there is less chance of converging to

Citations

2304	Learning Internal Representations by Error Propagation - Rumelhart, Hinton, et al. - 1986
2289	Adaptation in Natural and Artificial Systems - Holland - 1975
590	The cascade-correlation learning architecture - Fahlman, Lebiere - 1990
205	An Empirical Study of Learning Speed in Back-Propagation Networks - Fahlman - 1988
177	Training Feedforward Neural Networks Using Genetic Algorithms - Davis
124	Generalization of backpropagation to recurrent neural networks - Pineda - 1987
115	Genetic algorithms and neural networks: optimizing connections and connectivity - Starkweather, Bogart - 1990
67	Optimizing Neural Networks using Faster, more Accurate Genetic Search - Whitley, Hanson - 1989
26	Genetic Reinforcement Learning with Multilayer Neural Networks - Whitley, Dominic, et al.
1	The generalization of "Student's" problem when several different population variances are involved - Welch - 1947