J. Koza and J. Rice. Genetic generation of both the weights and architecture for a neural network. In IEEE International Joint Conference on Neural Networks, pages II-397 -- II-404, Seattle, WA, IEEE Press, 1991.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... to use of those evolutionary computation (EC) methods which have previously been applied to uni objective NN design, genetic algorithms (GAs) evolution strategies (ES) and particle swarm optimisation (PSO) GAs have previously be used for feature selection [8, 53] and topography selection [2, 5, 29, 35, 36, 38, 52] and ESs have been used for weight optimisation [21, 42, 45, 55] and adaptive topography selection [15, 37, 57] The recent EC technique of PSO [27] has also proved popular as a uni objective NN optimiser [10, 12, 13, 26, 48] 2 Multi objective evolutionary neural network flamework The use of ....

J.R. Koza and J.P. Rice. Genetic generation of both the weights and architecture for a neural network. In Proceedings of IJCNN'92, Seattle IEEE/INNS, volume II, pages 397-404, 1992.

....indirect encoding. After a representation scheme has been chosen, the evolution of architectures can progress according to the cycle shown in Fig. 6. The cycle stops when a satisfactory ANN is found. Considerable research on evolving ANN architectures has been carried out in recent years [33] [42], 45] 118] 127] 128] 130] 138] 149] 225] Most of the research has concentrated on the evolution of ANN topological structures. Relatively little has been done on the evolution of node transfer functions, let al..one the simultaneous evolution of both topological structures and node ....

....where indicates presence or absence of the connection from node to node . We can use to indicate a connection and to indicate no connection. In fact, can represent real valued connection weights from node to node so that the architecture and connection weights can be evolved simultaneously [37] [42], 45] 165] 166] 169] 171] Each matrix has a direct one to one mapping to the corresponding ANN architecture. The binary string representing an architecture is the concatenation of rows (or columns) of the matrix. Constraints on architectures being explored can easily be incorporated ....

[Article contains additional citation context not shown here]

J. R. Koza and J. P. Rice, "Genetic generation of both the weights and architecture for a neural network," in Proc. 1991 IEEE Int. Joint Conf. Neural Networks (IJCNN'91 Seattle), vol. 2, pp. 397--404.

....in local maxima[10] The fact that the search space for neural network architectures is in nitely large and nondi erentiable[11] makes the genetic algorithm approach a good candidate for success. Indeed, research into the evolution of neural network architectures has been largely successful[12, 13, 14, 15, 16, 17, 10]. 2.3 Transfer Functions The transfer function for all neurons of a neural networks is generally taken to be xed, although some attempts have been made to allow its adaptation over generations [18, 19] These schemes typically begin with a xed proportion of transfer functions, such as ....

....to evolve [20, 15] 2.5 Simultaneous Evolution One of the most interesting areas of evolutionary neural networks is the combination of several schemes which simultaneously evolve di erent aspects of the networks. One of the most important is the combination of architecture and weight evolution [11, 7, 12, 21, 22, 18, 23, 24, 25]. The advantage of combining these two basic elements of a neural network is that a completely functioning network can be evolved without any human interaction. Clearly, it might be advantageous to simultaneously evolve more neural network features, thus leading to more ecient and accurate ....

[Article contains additional citation context not shown here]

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. In International Joint Conference on Neural Networks, IJCNN-91, volume II, pages 397-404, Washington State Convention and Trade Center, Seattle, WA, USA, 8-12 1991. IEEE Computer Society Press.

....This requires a fixed maximal architecture, which is typically either fully connected or layered. Node based encoding An advantage over connection based encoding is that more flexibility can be obtained by using nodes as basic units. In this approach, the genome is a string or tree [14] of node information. The code for each node may include relative position, backward connectivity [24] weight values, threshold function [26] and more. Crossover and mutation is mostly restricted to cuts between node information. Layer based encoding With layer based encoding one can obtain ....

John R. Koza and James P. Rice. Genetic generation of both the weight and architecture for a neural network. In International Joint Conference on Neural Networks, pages II 397--404. IEEE, 1991.

....are possible. Crossing over now means term replacement. 4 Results Even very large neural networks can be represented in an intuitive and compact way. The ZIP code reader by Le Cun et al. 1] consisting of about 1000 Units, can be encoded in less than three lines. Compared to Koza and Rice [2], we achieve a higher level of abstraction. Our genetic algorithm found network topologies for multiple problems, e.g. the two spirals problem [4] and real valued functions. ....

J. R. Koza, J. P. Rice: "Genetic Generation of both the Weights and Architecture for a Neural Network"; IJCNN92, pp 397, 1991

....N. 358, 503] Kolarik, W. J. 392] Komata, Y. 552] Kong, Seong Gon, 320] Koppen, M. 530] Korkin, Michael, 523] Korning, Peter G. 263] Korousic Seljak, B. 25] Koskimies, Kai, 510] Kosugi, Y. 395] Kosugi, Yukio, 411] Kouchi, M. 791] Kovacs, S. 390] Koza, John R. [792] Kozek, Tibor, 647] Krejsa, Jir i, 579] Kremer, Stefan, 793] Krishnamraju, P. 21] Krishnan, Rajendra, 53] Ku, K. W. C. 518] Kuijpers, C. M. H, 527] Kuiper, Herman, 625] Kukreja, Basant, 28] Kumagai, Totu, 553, 574] Kumar, K. K. 54] Kuncheva, L. 359] Kuncheva, Ludmila ....

....Rastogi, Ravi, 28] Ray, K. S. 356] Reed, R. 883] Reeves, Colin R. 889, 890, 891, 892] Rehder, J. 229] Reidys, C. 248] Reilly, K. D. 351] Reilly, Kevin D. 21] Renders, Jean Michael, 275] Rennolls, Keith, 893] Reyneri, L. M. 293] Ribert, A. 479] Rice, James P. [792] Ridella, Sandro, 845] Riessen, G. A. 542] Ristov, Strahil, 348] Ritter, Helge, 49] Robbins, Philip, 893] Roberts, Stephen G. 208] Robillard, C. 467] Rocha, Armando Freitas da, 969] Rogers, David, 652] Rogers, Leah Lucille, 795] Romaniuk, Steve G. 77, 120, 209, 278, 383, ....

[Article contains additional citation context not shown here]

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, volume II, pages 397--404, Seattle, WA, 8.-12. July 1991. IEEE Press. * ga:Koza91g.

..... 12 2.7 Local neighbourhood operation used in image processing. 17 2.8 Local neighbourhood operation implemented as a single neural network. 17 3. 1 Example network and encoding of Koza et al. [41]. 30 3.2 Example network representation as used by Poli [60] 32 3.3 Initial network configuration for Cellular Encoding [23] 33 3.4 Operation of the PAR instruction in Cellular Encoding . ....

....the fact that the GP tree can be regarded as analogous to a feedforward neural network in which nodes of the tree are neurons, with the arcs representing the connections. However, the need to additionally specify connection weights necessarily complicates the encoding slightly. Koza et al. [41] represent a feedforward neural network as a GP tree by using functions P for a neuron and W for a weight. Each P node has W nodes as children in the tree. During evaluation, the W function multiplies the values represented by its child subtrees together. Each W node may have as its left child a ....

[Article contains additional citation context not shown here]

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. In IJCNN-91-Seattle: International Joint Conference on Neural Networks, volume 2, pages 397--404, Seattle, Washington, USA, 8--14 July 1991. IEEE Press. 140

....performance of the system is superior to a non modular version of the system. 1 Introduction Genetic programming (GP) is a modi cation of the genetic algorithm which evolves variable sized tree structures rather than xed length bitstrings. GP has been applied to neural network (NN) learning ([1, 3]) and can automatically determine the necessary number of network neurons and the connection weights to solve the problem, as well as being able to produce recurrent networks. The GP based Cellular Encoding system of Gruau [2] has shown the ability to exploit the decomposability of problems. This ....

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. In IJCNN-91-Seattle: International Joint Conference on Neural Networks, volume 2, pages 397-404, Seattle, Washington, USA, 8-14 July 1991. IEEE Press.

....or even continuous since EAs do not depend on gradient information. Because EAs can treat large, complex, nondifferentiable and multimodal spaces, which are the typical case in the real world, considerable research and application has been conducted on the evolution of connection weights [24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112]. The evolutionary approach to weight training in ANNs consists of two major phases. The first phase is to decide the representation of connection weights, i.e. whether in the form of binary strings or not. The second one is the evolutionary process simulated by an EA, in which search operators ....

....parents from the population based on their fitness. 5. Apply search operators to the parents and generate offspring which form the next generation. Figure 6: A typical cycle of the evolution of architectures. Considerable research on evolving ANN architectures has been carried out in recent years [33, 42, 45, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 149, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 138, 213, 214, 215, 216, 118, 130, 127, 217, 218, 219, 220, 221, 222, 223, 128, 224, 225]. Most of the research has concentrated on the evolution of ANN topological structures. Relatively little has been done on the evolution of node transfer functions, let al..one the simultaneous evolution of both topological structures and node transfer functions. In this paper, we will analyze the ....

[Article contains additional citation context not shown here]

J. R. Koza and J. P. Rice, "Genetic generation of both the weights and architecture for a neural network," in Proc. of 1991 IEEE International Joint Conference on Neural Networks (IJCNN'91 Seattle), vol. 2, pp. 397--404, IEEE Press, New York, NY, 1991.

....restrictions We constrain the structure of the programs in the population by requiring that all programs manipulated by SGOCE be well formed trees according to a given context free tree grammar. Such syntactic restrictions have already been used by Koza and Rice to evolve neural networks [9]. However, while in their application such constraints were imposed to obtain valid descriptions of neural networks, here we only use them to limit the sizes of the individual programs and of the problem space by concentrating the search on a restricted family of interesting programs. The ....

J. R. Koza and J. P. Rice. Genetic generation of both the weights and architecture for neural networks. In an IEEE International Joint Conference on Neural Networks, pages II--397--II--404, 1991.

....Kimbrough, Steven O. 279] Kimura, Masayuki, 301] Kinnear, Jr. Kenneth E. 104, 152, 166, 513, 514] Kirkwood, I. M. A. 443] Kiryati, Nahum, 314] Klebus, G. P. 423] Knight, Leslie, 292] Ko, Eun Joung, 190] Koivo, Heikki N. 298] Koppen, M. 310] Korkmaz, E. E. 177] Koza, John R. [9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75] Kozasa, Junji, 76] Kraft, Donald H. 105] Kuhn, Leslie A. 326] Kuscu, Ibrahim, 311] Kvasnicka, Vladim ir, 424, 436] Laane, Lisa A. 206] Laing, James D. 279] Lakner, R. 350] Lamas, Ricardo, 219] Lang, K. J. 307] Langdon, W. B. 312] Langdon, William Benjamin, 361] Langdon, ....

....[242, 272] Posp ichal, Jir i, 424, 436] Potvin, J. Y. 388] Prager, Richard, 82] Punch, William F. 326] Pyeatt, Larry, 288] Qureshi, Adil, 325] Raik, Simon, 122] Ray, T. S. 224] Raymer, Michael L. 326] Rettenmaier, H. 181] Reynolds, Craig W. 87, 144, 484] Rice, James P. [48, 49, 56, 57, 60, 65, 66, 67, 68, 70, 71, 74] Richter, Charles, 459] Riolo, Rick L. 318] Robinson, G. 243] Roll, C. 187] Rosca, Justinian P. 123, 155, 167, 178, 223, 440] Rosca, Justinian, 327, 464] Ross, Peter, 96, 285, 460] Ross, Steven J. 235, 268, 275, 328, 368] Ross, Steven, 394] Roughgarden, Jonathan, 67, 68] Rush, J. ....

[Article contains additional citation context not shown here]

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, volume II, pages 397--404, Seattle, WA, 8.-12. July 1991. IEEE Press. * ga:Koza91g.

....direct encoding strategies [48] 18] 37] 36] 10] 31] 26] the architecture of the network is directly encoded onto the chromosome representation. In the case of the generative encoding strategies, however, some sort of grammar which generates network architectures is used [27] 13] 14] 15] [25] [16] 17] 5] In the following sections we will start by describing direct encoding methods, and next, concentrate on some recent studies which use some sort of grammar in generating ANNs. Then, we will present some other research which uses evolutionary design of ANNs as a tool in pursuing ....

Koza J.R. and Rice J.P. Genetic generation of both the weights and the architecture for a neural network. pages 397--404, 1992.

....we have shown that computer programs can be genetically bred to solve a surprising variety of problems in many different areas [Koza 1989, 1990, 1992a] including . planning (e.g. navigating an artificial ant along a trail and developing a robotic plan for stacking blocks in to a desired order) [Koza 1989, 1991a] emergent behavior (e.g. discovering a computer program which, when executed by all the ants in an ant colony, enables the ants to locate food, pick it up, carry it to the nest, and drop pheromones along the way so as to produce cooperative emergent behavior) Koza 1991] evolution of ....

.... a desired order) Koza 1989, 1991a] emergent behavior (e.g. discovering a computer program which, when executed by all the ants in an ant colony, enables the ants to locate food, pick it up, carry it to the nest, and drop pheromones along the way so as to produce cooperative emergent behavior) [Koza 1991], evolution of subsumption (e.g. evolving a program for a wall following robot) Koza 1992b] machine learning of functions (e.g. learning the Boolean 11 multiplexer function) Koza 1991d] automatic programming (e.g. solving pairs of linear equations, solving quadratic equations for ....

[Article contains additional citation context not shown here]

Koza, John R. and Rice, James P. Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, Seattle, July 1991. IEEE Press. Volume II, Pages 397-404. 1991a.

....evolution and co evolution, automatic programming (e.g. discovering a computational procedure for solving pairs of linear equations, solving quadratic equations for complex roots, and discovering trigonometric identities) and . simultaneous architectural design and training of neural networks [Koza and Rice 1991]. Additional information and examples can be found in Koza [1992a] 11. Conclusions We have shown that many seemingly different problems in machine learning and artificial intelligence can be viewed as requiring the discovery of a computer program that produces some desired output for particular ....

Koza, John R. and Rice, James P. Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, Seattle, July 1991. IEEE Press. Volume II, Pages 397-404.

.... To appear in the Late breaking Papers of the Genetic Programming 96 (GP96) conference, Stanford, July 1996 Genetic programming (GP) techniqueshave also been used to evolve a variety of graph structures, including push down automata [Zomorodian 1995] and novel graph based programs [Teller 1996] [Koza and Rice 1991] used GP to directly encode the edges and nodes of a neural network. Gruau s rule based cellular encodingtechnique [Gruau 1992] uses tree like chromosomes and other elements of GP technique to evolve neural networks. 2 Cellular Encoding Cellular encoding [Gruau 1992] uses chromosomes consisting ....

Koza, J.R. and J.P. Rice. 1991. Genetic Generation of Both the Weights and Architecture for a Neural Network. In IEEE International Joint Conference onNeural Networks.

....then the backpropagation algorithm was used to conduct an efficient local search for fine tuning of weights and biases. Recently there have been a few studies on the use of genetic algorithms for designing neural networks without using back propagation (Marti, 1992; Hintz and Spofford, 1990; Koza and Rice, 1991). Our method is an alternative approach with distinct following feature. The Structured Genetic Algorithm (Dasgupta and McGregor, 1992d) defines the network configuration and its connection weights in its chromosome and both the sets of parameters are optimized simultaneously in single ....

Koza, J. R. and Rice, J. P. (1991). Genetic generation of both the weights and architecture for a neural network. In Intenational Joint conference on Neural Network(IJCNN).

....number of back propagation cycles and these changes are incorporated in the genome. Results are reported with synthetic mathematical problems, XOR and classification of FLIR data. Koza Koza applies his genetic programming paradigma to neural networks by choosing a nodebased encoding strategy [Koza, 1991b] The nodes are not encoded in a parameter string, but rather in a parameter tree. Figure 2.4 illustrates this at hand of an example. P represents a node ( processing element ) W is in place of a weight. The representation has to be read from the output. Each out P W 1.1 0.741 1.66 ....

....D0 1.387 D1 1.2 1.584 1.191 D1 0.989 D0 W W W W P P W (P (W ( 1.1 0.741) P (W 1.66 D0) W 1.387) W ( 1.2 1.584) P (W 1.191 D1) W 0.989 D0) Tree Representation LISP S Expression D0 D1 1.66 .989 1.191 1.387 1.1 0.741 1.2 1.584 Neural Network Figure 2. 4: Tree Encoding according to [Koza, 1991b] 24 put requires an own tree, so basically the output nodes are independent. A neuron can have any number of sub trees that start with an weight. A weight has two sub trees: one that contains the weight value, the other the source of the connection, which has to be a processing unit or an ....

[Article contains additional citation context not shown here]

: John R. Koza and James P. Rice: "Genetic Generation of Both the Weight and Architecture for a Neural Network", in: Proceedings of the International Joint Conference on Neural Networks, Vol. II, pp. 397-404, IEEE.

..... optimal control (e.g. centering a cart and balancing a broom on a moving cart in minimal time by applying a bang bang force to the cart [Koza and Keane 1990] and backing a tractor trailer truck to a loading dock [Koza 1992a] planning (e.g. navigating an artificial ant along a trail) [Koza 1991a] finding minimax strategies for games (e.g. differential pursuer evader games, discrete games in extensive form) by both evolution and co evolution [Koza 1991b] evolving robotic action plans in the style of the subsumption architecture (e.g. a wall following strategy for a robot with ....

.... and backing a tractor trailer truck to a loading dock [Koza 1992a] planning (e.g. navigating an artificial ant along a trail) Koza 1991a] finding minimax strategies for games (e.g. differential pursuer evader games, discrete games in extensive form) by both evolution and co evolution [Koza 1991b] evolving robotic action plans in the style of the subsumption architecture (e.g. a wall following strategy for a robot with sonar sensors in an irregular room) Koza 1992d] empirical discovery (e.g. rediscovering Kepler s Third Law, rediscovering the well known nonlinear econometric ....

[Article contains additional citation context not shown here]

Koza, John R., and Rice, James P. Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, Seattle, July 1991. Los Alamitos, CA: IEEE Press 1991. Volume II, Pages 397-404. 1991a.

No context found.

J. Koza and J. Rice. Genetic generation of both the weights and architecture for a neural network. In IEEE International Joint Conference on Neural Networks, pages II-397 -- II-404, Seattle, WA, IEEE Press, 1991.

No context found.

J. Koza and J. Rice, "Genetic generation of both the weights and architecture for a neural network," in Proceedings of IJCNN'92, Seattle IEEE/INNS, vol. II, 1992, pp. 397--404.

No context found.

John R. Koza and James P. Rice. Genetic generation of both the weights and architecture for a neural network. International Joint Conference on Neural Networks, IJCNN-91, II:397--404, 8-12 1991.

No context found.

Complex Systems, 4:461-476. Koza, J. and Rice, J. (1991). Genetic generation of both the weights and architecture for a neural network. In IEEE International Joint Conference on Neural Networks, pages II-397 - II-404, Seattle, WA, IEEE, Computer Society Press. U.S.A.

No context found.

Koza, J.R. & Rice, J.P. (1991). Genetic generation of both the weights and architecture for a neural network. Intern. Joint Conf. on Neural Networks, Seattle 92.

No context found.

J. R. Koza and J. P. Rice, "Genetic generation of both the weights and architecture for a neural network," in IJCNN-91Seattle: International Joint Conference on Neural Networks, pp. II--397--II--404, Piscataway, N.J.: IEEE, July 1991.

No context found.

: John R. Koza and James P. Rice: "Genetic Generation of Both the Weight and Architecture for a Neural Network", in: Proceedings of the - 92 - International Joint Conference on Neural Networks, Vol. II, pp. 397-404, IEEE.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC