Whitley, D., & Hanson, T. (1989). Optimizing neural networks using faster, more accurate genetic search. In Proc. of the Third Intl. Conf. on Genetic Algorithms (pp. 391--397).  Home/Search   Document Not in Database   Summary   Related Articles   Check

....faster than BP, they still suffer from same of the problems mentioned in Section 1 [11, 40] 3 Previous Work on the Evolution of Neural Network Learning Rules A considerable amount of work has been done on the evolution of the weights and or the topology of neural networks. See for example [20, 35, 36, 37, 54]. However only a relatively small amount of previous work has been reported on the evolution of learning rules for neural networks. Given the topology of the network, GAs have been used to find the optimum learning rules. For example, Montana [30] used GAs for training feedforward networks and ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, Georg Mason University,, pages 391-396. Morgan Kaufmann, 1989.

.... The only practical computational approach to these combinatorially complex problems is to use reverse biological engineering and simulate the natural dynamics with artificial neural nets [25,56] and natural selection in the form of genetic algorithms to evolve the connection weights in the nets [53]. There is no doubt that these techniques derived from life as we know it are of practical engineering value. However, I would call them virtual dynamical analogs implemented by programmed computers. Adlelman [ 1 ] has used real DNA molecules in a massively parallel chemical search for a ....

Whitley, D. and Hanson, T., 1989, Optimizing neural networks using faster, more accurate genetic search. In Proceedings of the ThirdInternational Conference on GAs, Morgan Kauffxnan, pp. 391-396.

....methods such as simulated annealing [24] or genetic algorithms [19, 13] Local methods are usually faster, but they can get trapped in local minima, whereas global methods are less sensitive to local minima but usually slower. Hybrid gradient descent genetic algorithms were recently suggested [9, 28]. In our experiments we used gradient descent, simulated annealing, as well as genetic algorithms. 2.7 Problem complexity It is interesting to discuss briefly the complexity of the above optimization problem. We already have some knowledge of the complexity of neural network learning. Deciding ....

D. Whitley and T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in Proceedings of the Third International Conference on Genetic Algorithms, San Mateo, CA, USA, 1989, Morgan Kaufmann, pp. 1888--1898. 29

....best performance of the evaluation function. Operators such as cross over are applied to the chromosomes of the parents to produce children that are inserted into the population. Domain knowledge can be exploited to create operators which improve the efficiency of the optimization procedure [Whit89]. It should be noted that interesting gradient descent genetic algorithm hybrids have been proposed and could be considered here (see for example [Davi89] or [Whit89] A coding scheme and a set of genetic operators for Deltaw( could be designed, based for example on those proposed in [Whit89] ....

....the population. Domain knowledge can be exploited to create operators which improve the efficiency of the optimization procedure [Whit89] It should be noted that interesting gradient descent genetic algorithm hybrids have been proposed and could be considered here (see for example [Davi89] or [Whit89]) A coding scheme and a set of genetic operators for Deltaw( could be designed, based for example on those proposed in [Whit89] to improve neural networks with genetic algorithms. The advantages of genetic algorithms is that they are quite resistant to the problem of local minima in the ....

[Article contains additional citation context not shown here]

Whitley D. and Hanson T. (1989). Optimizing neural networks using faster, more accurate genetic search. Proc. Third International Conference on Genetic Algorithms, J.D. Shafer ed., Morgan Kaufmann, pp. 391-396.

....as the median individual. See Whitley [50] for more explanation on the selection bias. For all the 1296 (8 6 3 3 3) possible combinations of the above parameters we carry out 10 executions of the algorithm. We use an algorithm based on the principles of the GENITOR algorithm (Whitley [50, 51]) in which the generation reproduction rate, i.e. the proportion of the created individuals in every iteration of the algorithm, corresponds to the inverse of the population size. Moreover, the reduction criterion is elitist. Therefore, in every iteration of the genetic algorithm, only one new ....

D. Whitley and T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in: J.J. Grefenstette, ed., Genetic Algorithms and Their Applications: Proceedings of the Second International Conference, Cambridge, MA (Lawrence Erlbaum, Hillsdale, New Yersey, 1987) 391-396.

....Networks. Behind these motivation lie, of course, the biological roots both approaches have in common. A main application of GAs to NNs has been the usage of the GA as a learning rule together with or instead of Backpropagation or related algorithms to train the weights of Feed Forward Networks [12]. They have also been used for searching optimal learning rules [3] and efficient net structure and topology [2] 12] It has been pointed out that a main target of combined GA NN research should be the search for a non explicit genotype phenotype coupling [1] a direct coding often proves ....

.... of GAs to NNs has been the usage of the GA as a learning rule together with or instead of Backpropagation or related algorithms to train the weights of Feed Forward Networks [12] They have also been used for searching optimal learning rules [3] and efficient net structure and topology [2] [12]. It has been pointed out that a main target of combined GA NN research should be the search for a non explicit genotype phenotype coupling [1] a direct coding often proves inappropriate and may lead to an unnecessarily large GA search space. With their particular ability to adapt to simply and ....

Darrell Whitley and Thomas Hanson. Optimizing neural networks using faster, more accurate genetic search. In David Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, pages 391--396, San Mateo, California, 1989.

....by encouraging unique strings and or checking the fecundity of particularly fit strings. Of the three, sharing has been shown to be superior [12, 24] A more extreme approach to selection is employed by rank based schemes such as work done by Baker [3] and Darrell Whitley s GENITOR algorithm [41, 42]. Under a rank based scheme, the absolute fitness of a string is unimportant. Instead, a method of determining if one string is more fit than another is required. The initial population is sorted according to this criterion, and then new children are inserted into the population, pushing out the ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. David Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, pages 391--398, George Mason University, June 1989.

....and interaction of their constituent schemata; and (2) that of the perceptron computation itself. Since the perceptron is a fundamental example of an artificial neural network, our investigation is related to several others in which network characteristics were evolved genetically. In some (e.g. [6 7]) the emphasis was on evolving connection weights. Others (e.g. 8 9] addressed the question of evolving general network structure and are thus more related to the present work. We shall offer some answers to the following questions: 1) Can genetic search over perceptron space indeed lead ....

D. Whitley and T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in: Proc. Third Internat. Conf. on Genetic Algorithms, J. D. Schaffer, ed. (Morgan Wilson 9 Perceptron Redux Kaufmann, San Mateo, CA, 1989).

....4) network pruning and reduction, and 5) gradient descent training. He demonstrated that the method provides an order of magnitude speed up with respect to other methods. Montana [13] used GAs for training feedforward networks and created a new method of training which is similar to SBP. Whitley [25] showed that GAs can make a positive and competitive contribution in the neural networks area, as, although they have trouble getting an exact solution, all the solution are nearly correct. Chalmers [2] applied GAs to discover supervised learning rules for single layer neural networks. He ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, pages 391--396, San Mateo, CA, 1989. Morgan Kaufmann.

....faster than SBP , they still suffer from same of the problems mentioned in Section 1 [11, 40] 3 Previous Work on the Evolution of Neural Network Learning Rules A considerable amount of work has been done on the evolution of the weights and or the topology of neural networks. See for example [20, 35, 36, 37, 54]. However only a relatively small amount of previous work has been reported on the evolution of learning rules for neural networks. Given the topology of the network, GAs have been used to Thetand the optimum learning rules. For example, Montana [30] used GAs for training feedforward networks and ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, Georg Mason University,, pages 391#396. Morgan Kaufmann, 1989.

....still suffers from many problems [9, 34] and can not train networks with step activation function. 3 Previous Work on the Evolution of Neural Network Learning Rules A considerable amount of work has been done on the evolution of the weights and or the topology of neural networks. See for example [16, 28, 29, 30, 48]. However only a relatively small amount of previous work has been reported on the evolution of learning rules for neural networks. Given the topology of the network, GAs have been used to Thetand the optimum learning rules. For example, Montana [23] used GAs for training feedforward networks and ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, Georg Mason University,, pages 391# 396. Morgan Kaufmann, 1989.

....faster than SBP , they still suffer from same of the problems mentioned in Section 1 [11, 41] 3 Previous Work on the Evolution of Neural Network Learning Rules A considerable amount of work has been done on the evolution of the weights and or the topology of neural networks. See for example [20, 36, 37, 38, 54]. However only a relatively small amount of previous work has been reported on the evolution of learning rules for neural networks. Given the topology of the network, GAs have been used to Thetand the optimum learning rules. For example, Montana [31] used GAs for training feedforward networks and ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, Georg Mason University,, pages 391#396. Morgan Kaufmann, 1989.

....4) network pruning and reduction, and 5) gradient descend training. He demonstrated that the method provides an order of magnitude speed up with respect to other methods. Montana [24] used GAs for training feedforward networks and created a new method of training which is similar to SBP. Whitley [35] showed that GAs can make a positive and competitive contribution in the neural networks area. As, although they have trouble getting an exact solution, all the solution are nearly correct. Chalmers [4] applied GAs to discover supervised learning rules for single layer neural networks. He ....

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. D. Schaffer, editor, Third International Conference on Genetic Algorithms, pages 391--396, San Mateo, CA, 1989. Morgan Kaufmann.

....Hamersma, H. 425] Hammer, Jurgen, 588] Han, Seung Soo, 344, 393, 459, 469, 546] Han, S. S. 78] Hancock, Peter J. B. 721, 722, 723, 724, 725, 796] Handroos, H. 543] Hanebeck, Uwe D. 345] Hansen, J. V. 346] Hansen, Kim Kortermand, 40] Hansen, L. K. 274] Hanson, Thomas, [947, 952] Happel, B. L. M. 726] Happel, Bart L. M. 41] Harget, A. 456] Haring, S. 503] Harp, Steven Alex, 727, 728, 729, 730, 731, 732, 733, 734] Harrald, P. G. 504] Harrington, Robert J. 428] Harrison, G. 505] Harrison, Leonard, 588] Harrison, R. F. 817] Hart, William Eugene, ....

....[560] Wenhui, Chen, 488] Werner, R. 884, 911] Westland, S. 776] Wezel, Michiel C. van, 148, 503] Whitaker, Kevin W. 944] White, David W. 27] White, D. 653] Whitehead, B. A. 418] Whitehead, Bruce A. 419] Whitfort, T. 514] Whitley, Darrell L. 341] Whitley, Darrell, [132, 303, 314, 908, 945, 946, 947, 948, 949, 950, 951, 952, 953, 954, 955, 956, 957, 958, 959] Wieland, Alexis P. 960] Wienholt, Willfried, 961, 962] Wilke, Peter, 963, 964] Wilke, P. 229] Williams, Bryn V. 631] Williams, G. J. 542] Williams, Tom, 220] Williamson, A. G. 288] Wilson, Stewart W. 902] Winfield, A. 378] Winkler, David A. 495] Wise, B. M. 149] ....

[Article contains additional citation context not shown here]

Darrell Whitley and Thomas Hanson. Optimizing neural networks using faster, more accurate genetic search. In Schaffer [1026], pages 391--396. ga:Whitley89a.

....to zero, approaches which encode the weight matrix on the chromosome will be considered fixed architecture models because of the explicit encoding on the chromosome of all the weights for a fixed number of neurons. Research has investigated performance on Boolean problems (e.g. Whitley and Hanson [78], Santos and Duro [65] Heistermann [27] Srinivas and Patnaik [72] and Maniezzo [47] polebalancing (Whitley et al. 76] temporal pattern learning (e.g. Fukuda et al. 19] and classification (e.g. Korning [37] and Montana and Davis [52] The utility of using the GA over BP for training ....

Darrell Whitley and Thomas Hanson. Optimizing neural networks using faster, more accurate genetic search. In J. David Schaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms, pages 391--396, George Mason University, Washington D.C., 4--7 June 1989. Morgan Kaufmann.

.... case is especially clear in the case of continuous problem spaces, where guarantees of local search methods within local neighborhoods are extended to seek solutions which are globally optimal across an entire domain, even when explicit gradient information is not available [Belew et al. 1991, Whitley and Hanson, 1989, Hart, 1994, Rosin et al. 1997, Land et al. 1997] In the case of discrete, combinatorial optimization definitions of local search neighborhoods must be in terms of operators which perturb one solution slightly to form alternatives. Classic examples include single bit changes performed by ....

Whitley, D. and Hanson, T. (1989). Optimizing neural networks using faster, more accurate genetic search. In Schaffer, J. D., editor, Proc. Third Intl. Conf. on Genetic Algorithms, pages 391--397, Washington, D.C. Morgan Kaufman.

....times, to reach a solution of a certain quality, or the total wall clock time to achieve the same goal. Indeed, this best so far graph as a function of the number of function evaluations is precisely the graph shown in most empirical papers on stochastic search. Thus, for example, when Whitley (1989) and Davis (1991b) advocate barring duplicates in the population of a genetic algorithm, it is perfectly possible that this will indeed prove beneficial over an entire ensemble of problems R S by reducing the likelihood of revisiting. The suggested benefits of enforcing uniqueness of solutions ....

....transmission would require that the child had either red or brown hair. It is easy to see that a recombination operator that transmits genes is respectful. However, with an allelic representation this is not necessarily the case. For example, the Edge Recombination Operator in its original form (Whitley et al. 1989) sought to transmit as many undirected edges from the two parents as possible, and was thus striving to achieve allele transmission, which it did typically with over 99 success. The operator was then modified to achieve strict respect by placing all edges common to the parents in the child at the ....

[Article contains additional citation context not shown here]

D. Whitley and T. Hanson, 1989. Optimizing neural networks using faster, more accurate genetic search. In Proceedings of the Third International Conference on Genetic Algorithms, pages 391--396. Morgan Kaufmann (San Mateo).

....simply used the set of features given by c4.5 while indigent and tnt indigent use this feature set as a springboard for further search. 3.5 Genetic Neural Networks Much work has been done on the genetic refinement of neural networks whose topologies are not knowledge based. Some early efforts [MD89, WGM89] used genetic algorithms to set and modify the weights of synapses in a neural network. These algorithms showed comparable performance to back propagation methods. They do not show marked improvement over back propagation [Yao93] and in some cases their performance lags in speed behind optimized ....

D. Whitley, S. Gordon, and K. Mathias. Optimizing neural networks using faster, more accurate genetic search. In Proceedings of the third international conference on genetic algorithms, pages 391--396, 1989.

....Nine Men s Morris [2] employing a (23 12 1010) topology, a (3 20 3) topology and a (120 60 20 21) topology, respectively. So at least the last one of them is much more complex than problems considered in publications on comparable topics ( 3] 4] 5] 6] 7] 8] 9] 10] 11] 12] [13], 14] 15] Arguing with Miller et.al. 9] the search space of possible network topologies is infinitely large, not differentiable, complex, noisy, deceptive and multimodal. These attributes make random, enumerative, gradient descent or heuristic search methods unpracticable. Genetic ....

....demands of the examined search space [16] Accordingly, there has been a lot of work concerning the use of genetic methods in order to evolve problem specific network architectures. On the other hand there have been attempts to achieve robuster learning techniques by using genetic methods ( 12] [13]) the most promising of them be while evolution do (1) select one rsp. two parents selection (2) generate one offspring mutation rsp. crossover (3) evaluate offspring learning evaluation (4) insert offspring in population survival of the fittest and delete the last population element ....

D. Whitley, T. Hanson, Optimizing neural networks using faster, more accurate genetic search, in: Proc. Third Internat. Conf. Genetic Algorithms (Morgan Kaufmann, San Mateo, CA, 1989)

....till the last resource are taken from the other parent. 3) As a last step, a mutation operator is applied. This operator might change some of the values in the ternary string obtained. This is typically a low probability operator whose probability linearly increases as the parents are more similar [17]. One should note that the genetic operators are not necessarily closed: the combination of two states from which solutions can be reached might result in a state from which no solution can be reached. Here again, expanding the child through the use of constraint propagation allows us to spot a ....

Whitley, D., (1989), Optimizing Neural Networks using Faster, more Accurate Genetic Search. Proc. Third Int. Conf. on Genetic Algorithms, Morgan Kaufmann.

....of distributed adaptive control [13, 10] can be seen as a model of the development of categorization. Others have applied genetic algorithms to neural networks, these attempts however were mostly concerned with issues of genetic algorithm design rather the modeling evolution as observed in nature [1, 14]. In another approach [11] it was attempted to model the interaction of individuals in a competitive environment. When we would take the above mentioned work on the combination of GA s and neural networks as a starting point in modeling the interaction between natural selection and adaptation of ....

Darrell Whitley and Thomas Hanson. Optimizing neural networks using faster, more accurate genetic search. In Third International Conference on Genetic Algorithms.

....using increased selective pressure coupled with a special mutation operator to sustain diversity. This combination resulted in optimization with greater accuracy (errors of 10 0 and below) and produced these results up to 10 times faster using a very small population of 50 genotypes (Whitley 1989). Creating Selective Pressure First, it should be noted that while more complex mechanisms for selective pressure could possibly be defined, a single variable is adequate. Over the past three year the GENITOR project has experimented with several modes of selective pressure, but we have found ....

....are currently further testing the use of specialized mutation and increased selective pressure to produce faster yet more accurate optimization. Selective mutation can help sustain genetic diversity, which in turn allows a higher level of exploitation to be achieved via higher selective pressure (Whitley 1989). ACKNOWLEDGMENTS This research was supported in part by a grant from the Colorado Institute of Artificial Intelligence (CIAI) CIAI is sponsored in part by the Colorado Advanced Technology Institute (CATI) an agency of the State of Colorado. CATI promotes advanced technology education and ....

Whitley, D., and Hanson, T. Optimizing Neural Networks Using Faster, More Accurate Genetic Search. 1989 Genetic Algorithm Conference. Morgan Kaufmann, Publishers. In Proc. International Joint Conference on Artificial Intelligence.

No context found.

Whitley, D., & Hanson, T. (1989). Optimizing neural networks using faster, more accurate genetic search. In Proc. of the Third Intl. Conf. on Genetic Algorithms (pp. 391--397).

No context found.

D. Whitley and T. Hanson. Optimizing neural networks using faster, more accurate genetic search. In J.D. Shaffer, editor, Proceedings of the Third International Conference on Genetic Algorithms and Their Applications, pages 391--396. Morgan Kaufmann, 1989.

No context found.

Whitley, D. and Hanson, T. Optimizing Neural Networks Using Faster, More Accurate Genetic Search, Proceedings of the Third International Conference on Genetic Algorithms, pp. 391-396. 1989.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC