S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992. 19  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and natural representation. The key problem (other than being trapped in a local minimum) with BP and other traditional training algorithms is the choice of a correct architecture (number of hidden nodes and connections) This problem has been tackled by the evolutionary approach in many studies [4, 14, 17, 21, 24, 39, 40, 41]. In some of these studies, weights and architectures are evolved simultaneously. The major disadvantage to the EANN approach is it is computationally expensive, as the evolutionary approach is normally slow. To overcome the slow convergence of the evolutionary approach to ANN, hybrid techniques ....

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992. 19

....encoding. After a representation scheme has been chosen, the evolution of architectures can progress according to the cycle shown in Fig. 6. The cycle stops when a satisfactory ANN is found. Considerable research on evolving ANN architectures has been carried out in recent years [33] 42] [45], 118] 127] 128] 130] 138] 149] 225] Most of the research has concentrated on the evolution of ANN topological structures. Relatively little has been done on the evolution of node transfer functions, let al..one the simultaneous evolution of both topological structures and node ....

....indicates presence or absence of the connection from node to node . We can use to indicate a connection and to indicate no connection. In fact, can represent real valued connection weights from node to node so that the architecture and connection weights can be evolved simultaneously [37] 42] [45], 165] 166] 169] 171] Each matrix has a direct one to one mapping to the corresponding ANN architecture. The binary string representing an architecture is the concatenation of rows (or columns) of the matrix. Constraints on architectures being explored can easily be incorporated into such ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz, "General asymmetric neural networks and structure design by genetic algorithms," Neural Networks, vol. 5, no. 2, pp. 327--334, 1992.

....to RNNs is currently quite limited, but seems to be increasing. While work on this thesis was in progress, some papers have appeared in the neural network literature dealing strictly with an evolutionary approach to synthesizing RNNs [Angeline, Gregory, and Jordan, 1994; Beer and Gallagher, 1992; Bornholdt and 6 Graudenz, 1992]. However, most of these papers describe approachs which use ad hoc operators for combining and mutating networks and do not use the much studied bit string representation for encoding and manipulating the networks. One of the difficulties in using a bit string representation is that it does not ....

Bornholdt S., & Graudenz D. (1992). General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5, 327334.

.... on the neural network directly, and rely exclusively on mutation [10, 11, 12, 13] or combine mutation with training [14] Methods based on genetic algorithms usually represent the structure and the weights of ANNs as a string of 1 bits or as a combination of bits, integers and real numbers [15, 16, 17, 18, 19, 20], and perform the crossover operation as if the network were a linear structure. However, neural networks cannot naturally be represented as vectors. They are oriented graphs, whose nodes are neurons and whose arcs are synaptic connections. Therefore, it is arguable that any efficient approach to ....

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992.

....776] Blanchet, Max, 623] Blekas, K. 587] Bluff, K. 482, 624] Bo, Z. Q. 566] Boers, Egber J. W. 625] Boers, Egbert J. W. 155] Bogart, Christopher, 949, 953, 954] Bohari, Abdul Rahman, 434] Borges, Newton Chaves Kras, 156] Born, Joachim, 17, 102, 626, 627] Bornholdt, Stefan, [628, 629] 16 Genetic algorithms and neural networks Borst, Marko V. 18, 155] Bos, M. 630] Bossomaier, Terry, 202] Boughton, Edward M. 313] Boullart, L. 400] Bounds, David G. 631] Boyd, R. 632] Branke, Jurgen, 19, 157, 237] Brassinne, P. de, la, 633] Braun, H. 217] Braun, ....

....Glass, C. 632] Glorennec, Pierre Yves, 715] Gokulakrishnan, S. 340] Golden, J. B. 175, 350] Golukakrishnan, S. 251] Gonzales Seco, Jose, 774] Goto, Fumiyoshi, 525] Gottvald, Ales, 366] Gough, N. E. 146] Goulermas, Yannis J. P. 264] Grabensek, L. 662] Graudenz, Dirk, [628, 629] Grauel, Adolf, 365] Greenwood, Daniel, 875] Greenwood, Garrison W. 176] Gronroos, Marko A. 585] Grossi, G. 319] Gruau, Fr ed eric C. 39, 111, 127, 303, 341, 716, 717, 718] Guak, Kyuh Wan, 453, 462] Guan, Ling, 445] Guan, Shanguchuan, 764] Gueriot, Didier, 446] Guha, ....

[Article contains additional citation context not shown here]

Stefan Bornholdt and Dirk Graudenz. General asymmetric neural networks and structure design by genetic algorithms: A learning rule for temporal patterns. In 1993, International Conference on Systems, Man and Cybernetics, volume 2, pages 595--600, Le Touquet (France), 17.-20. October 1993. IEEE, New York. ga:Bornholdt93a.

....contains references to all papers published as technical reports. The list is arranged in alphabetical order by the name of the institute. Academy of Sciences of the USSR, 636] Carnegie Mellon University, 787] Colorado State University, 945, 948, 949, 950] Deutsches Elektronen Synchrotron, [628] Ecole Normale Superiore, 772] Ecole Normale Sup erieure de Lyon, 716] Edinburgh Parallel Computing Centre, 886] Honeywell Corporate Systems, 727, 729] Institute of Psychology CNR, 834] Iowa State University, 150] LASPP FER, 662] NIBS Pte Ltd. 35] 14 Genetic algorithms and neural ....

....776] Blanchet, Max, 623] Blekas, K. 587] Bluff, K. 482, 624] Bo, Z. Q. 566] Boers, Egber J. W. 625] Boers, Egbert J. W. 155] Bogart, Christopher, 949, 953, 954] Bohari, Abdul Rahman, 434] Borges, Newton Chaves Kras, 156] Born, Joachim, 17, 102, 626, 627] Bornholdt, Stefan, [628, 629] 16 Genetic algorithms and neural networks Borst, Marko V. 18, 155] Bos, M. 630] Bossomaier, Terry, 202] Boughton, Edward M. 313] Boullart, L. 400] Bounds, David G. 631] Boyd, R. 632] Branke, Jurgen, 19, 157, 237] Brassinne, P. de, la, 633] Braun, H. 217] Braun, ....

[Article contains additional citation context not shown here]

Stefan Bornholdt and Dirk Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Technical Report DESY 91-046, Deutsches Elektronen-Synchrotron, Hamburg, 1991. y(BackBib) ga:Bornholdt91a.

....K. 123] Blume, Christian, 1043] Bommel, Patrick van, 124, 646] Bonarini, Andrea, 125] Bonelli, Pierre, 126] Bonnet, J erome, 64, 65] Booker, Lashon B. 383] Booker, Peter, 796] Boone, G. 127] 16 Genetic algorithms of 1993 Born, Joachim, 128, 129, 130, 131] Bornholdt, Stefan, [132] Botta, M. 371] Bounds, D. G. 133] Boyd, R. 134] Bradshaw, J. 719] Brandon, J. A. 866] Brassinne, P. de, la, 135] Brown, Christopher, 144, 145] Brown, D. R. 195, 239] Brown, Robert D. 373] Bruns, Ralf, 136, 137] Buckley, James J. 139] Bui, T. D. 601] Bui, Thang ....

....382, 383] Gonzalez, Carlos, 147] Goodman, Erik D. 256] Gordon, Diana F. 216] Gordon, Edward O. 240] Gordon, Vahl Scott, 1062] Gorne, Thomas, 634, 665] Goto, T. 320] Grabensek, L. 225] Graf, J. 384] Gra na, M. 615] Grand, Scott Michael Le, 619, 620] Graudenz, Dirk, [132] Graziani, S. 158] Greene, David Perry, 386] Grefenstette, John J. 387, 388, 389, 390, 391, 392] Grierson, D. E. 788] Gritz, Larry, 982] Groot, Claas de, 393, 394] Gruau, Fr ed eric C. 395, 396, 397, 398, 399, 400] Gruber, Herv e, 56] Guan, Shanguchuan, 627] Gubbi, Ananda ....

[Article contains additional citation context not shown here]

Stefan Bornholdt and Dirk Graudenz. General asymmetric neural networks and structure design by genetic algorithms: A learning rule for temporal patterns. In 1993, International Conference on Systems, Man and Cybernetics, volume 2, pages 595--600, Le Touquet (France), 17.-20. October 1993. IEEE, New York. ga:Bornholdt93a.

.... [88] Mathematical and Computer Modelling, 241, 339] Memoirs of the Faculty of Engineering, Fukui University, 670] Microprocessing and Microprogramming EURO Micro Journal, 167] Microprocessors and Microsystems (UK) 80] Network: Computation in Neural Systems, 489] Neural Networks, [108] New Generation Computing, 395] Optical Engineering, 605] OR Spektrum, 396] ORSA Journal on Computing, 391] Parallel Computing, 62] Parallel Processing Letters, 148] Physical Review Letters, 363] Physics of the Earth and Planetary Interiors, 612] Protein Engineering, 59] Rivista ....

....Jr. Joe L. 99] Blanton Jr. Joe L. 100] Blommers, Marcel J. J. 447] Blume, Christian, 266] Boers, Egber, 101] Bogardi, J. J. 102] Boggia, R. 434] Bohm, A. P. Wim, 693] Booker, A. 120] Booker, Lashon B. 103] Born, Joachim, 104, 105, 106, 107] Bornholdt, Stefan, [108] Borup, Liana, 544] Bowden, Royce O. 109] Bratko, Ivan, 191] Brill, Frank Z. 110] Brown, Donald E. 110, 135] Brown, J. 343] Bruns, Ralf, 111] Bukatova, Innesa L. 112, 113] Bullock, G. N. 545] Cai, H. 352] Callahan, K. J. 116] Campbell, J. A. 342] Caponetto, R. ....

[Article contains additional citation context not shown here]

Stefan Bornholdt and Dirk Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5(2):327--334, 1992. ga:Bornholdt92a.

.... operate on the neural network directly, and rely exclusively on mutation [10, 11, 12, 13] or combine mutation with training [14] Methods based on genetic algorithms usually represent the structure and the weights of ANNs as a string of bits or as a combination of bits, integers and real numbers [15, 16, 17, 18, 19, 20], and perform the crossover operation as if the network were a linear structure. However, neural networks cannot naturally be represented as vectors. They are oriented graphs, whose nodes are neurons and whose arcs are synaptic connections. Therefore, it is arguable that any efficient approach to ....

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992.

....or even continuous since EAs do not depend on gradient information. Because EAs can treat large, complex, nondifferentiable and multimodal spaces, which are the typical case in the real world, considerable research and application has been conducted on the evolution of connection weights [24, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112]. The evolutionary approach to weight training in ANNs consists of two major phases. The first phase is to decide the representation of connection weights, i.e. whether in the form of binary strings or not. The second one is the evolutionary process simulated by an EA, in which search operators ....

....parents from the population based on their fitness. 5. Apply search operators to the parents and generate offspring which form the next generation. Figure 6: A typical cycle of the evolution of architectures. Considerable research on evolving ANN architectures has been carried out in recent years [33, 42, 45, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 149, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 138, 213, 214, 215, 216, 118, 130, 127, 217, 218, 219, 220, 221, 222, 223, 128, 224, 225]. Most of the research has concentrated on the evolution of ANN topological structures. Relatively little has been done on the evolution of node transfer functions, let al..one the simultaneous evolution of both topological structures and node transfer functions. In this paper, we will analyze the ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz, "General asymmetric neural networks and structure design by genetic algorithms," Neural Networks, vol. 5, pp. 327--334, 1992.

.... neural networks [1] In addition to the obvious biological appeal of the idea, evolutionary techniques such as Genetic Algorithms have an advantage over more popular supervised learning techniques in that they can be used to train neural networks with unrestricted architectures and neuron types [2,3], and their blind search characteristic means that networks can be evolved to perform unsupervised learning tasks where no immediate error information can be provided about the network output. Performance depends on the existence of satisfactory networks within the search space of allowed ....

Bornholdt S., Graudenz D.;'General Asymmetric Neural Networks and Structure Design by Genetic Algorithms', Neural Networks Vol 5, 327-334, 1993

....very fast during the given number of learning passes. The following mutation operators are proposed in [Schiffmann, 1991] Removal of weak connections, addition of new units with weak connections and addition of weak connections between existing units. Similar mutation operators are proposed in [Bornholdt, 1992]. The crossover operator [Schiffmann, 1992, 1993] is illustrated in figure 2.3. 22 The crossover points can be only set between nodes, crossover does not split connectivity lists. The crossover point in the second parent has to be chosen adequately. Hidden units that have no input or output ....

: Stefan Bornholdt and Dirk Graudenz: "General Asymmetric Neural Networks and Structure Design by Genetic Algorithms", in: Neural Networks, Vol. 5, pp. 327334, Pergamon Press.

....since GAs do not depend on gradient information in search. Because GAs are good at dealing with large, complex, nondifferentiable and multimodal spaces which are the typical space defined by an error function or fitness function, a lot of work has been done on the evolution of connection weights [29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64]. The evolutionary approach to weight training in EANNs consists of two major stages. The first stage is to decide the genotype representation of connection weights, i.e. whether in the form of binary strings or not. The second one is the evolution itself simulated by a GA or other evolutionary ....

....the next generation. Figure 6: A typical cycle of the evolution of architectures. Reprinted with permission from Ref. 1. X. Yao: Evolutionary Artificial Neural Networks 19 Because of advantages of the evolutionary design of architectures, a lot of research has been carried out in recent years [37, 46, 49, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100]. However, almost all the research only deals with the topological structure of EANNs and little has been done on the evolution of node transfer functions let al..one the evolution of both topological structures and node transfer functions. We will concentrate on the evolution of topological ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992.

....with a minimal network and dynamically builds a suitable cascade structure by training and installing one hidden unit at a time until the problem is successfully learned. Thus the structure is not directly optimized by the evolutionary algorithm but rather a result of the cascade algorithm. In [4, 9, 64, 68, 70] the weights and topology of recurrent neural networks are determined, Zhang [82] optimizes Sigma Pi networks A quite unusual approach has been proposed by Oliker et al. 55] where the search for the optimal neural network is done separately for every single neuron, i.e. separately in different ....

....the chromosome. The aspect of encoding functional units together on the chromosome is addressed in Marti [48] where the ordering of the links on the genotype is optimized by an additional outer genetic algorithm. 4. 2 Performance Evaluation Optimizing recurrent networks, Bornholdt and Graudenz [9] included the number of update cycles until the network reaches a stable state in the evaluation of their genetically constructed networks. Kendall and Hall [36, 37] used an estimation for the description length (according to the Minimum Description Length Principle, 59] for evaluation. This ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992.

....its fitness. 4. Apply genetic operators, such as crossover and mutation, to each child individual generated above and obtain the next generation. Figure 1: A typical cycle of the evolution of connection weights. fitness function, a lot of work has been done on the evolution of connection weights [27, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62]. The evolution of connection weights provides an alternative approach to training EANNs. Such an evolutionary approach consists of two major stages. The first stage is to decide the genotype representation of connection weights, i.e. whether in the form of binary strings or not. The second one ....

....the children generated above, and obtain the next generation. Figure 3: A typical cycle of the evolution of architectures. X. Yao: Evolutionary Artificial Neural Networks 13 Because of advantages of the evolutionary design of architectures, a lot of research has been carried out in recent years [36, 45, 48, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96]. However, almost all the research only deals with the topological structure of EANNs, little has been done on the evolution of node transfer functions, let al..one the evolution of both topological structures and node transfer functions. We will concentrate on the evolution of topological ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. X. Yao: Evolutionary Artificial Neural Networks 38 Neural Networks, 5:327--334, 1992.

....The resulting nets each have two copies of similar hidden units, and do not cover the input space. While this problem is widely recognised (e.g. Belew et al. (1990) there have been few attempts to solve it. Indeed, a number of workers have simply removed crossover from the algorithm, e.g. (Bornholdt Graudenz, 1992; de Garis, 1990; Nolfi et al. 1990) which seems drastic given the centrality of recombination to the GA model. Whitley et al. (1991) report successful results for training the weights of nets by GA, using what they term a genetic hill climber, which is essentially mutation driven. This is ....

Bornholdt, S., & Graudenz, D. 1992. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5, 327--334.

....different information processing abilities and performances. ffl The surface is multimodal since EANNs with quite different architectures can have very similar capabilities. Because of advantages of the evolutionary design of architectures, a lot of research has been carried out in recent years [49, 50, 48, 51, 52, 53, 54, 55, 56, 57, 58, 59, 29, 60, 32, 61, 62, 63], which concentrates on the evolution of EANN s connectivity, i.e. the number of nodes in an EANN and the connection topology among these nodes. Little work, however, has been done on the evolution of node transfer functions except for a couple of non GA based approaches to it [64, 65] let al..one ....

....new generation. Figure 2: A typical cycle of the evolution of architectures. X. Yao: A Review of Evolutionary Artificial Neural Networks 12 3. 1 Direct Encoding Scheme for EANN s Connectivity In direct encoding scheme, each connection of an EANN is specified directly by its binary representation [52, 51, 48, 59, 60, 29, 62, 63]. Because the chromosomal representation of EANN s connectivity has specified all the detailed information about it, the developmental rule used to decode chromosomes into EANN s connectivity patterns is virtually degenerated into a one to one mapping, no real development at all. In general, an ....

[Article contains additional citation context not shown here]

S. Bornholdt and D. Graudenz. General asymmetric neural networks and structure design by genetic algorithms. Neural Networks, 5:327--334, 1992.

No context found.

S. Bornholdt and D. Graudenz, "General asymmetric neural networks and structure design by genetic algorithms," Neural Networks, vol. 5, no. 1, pp. 327--334, 1992.

No context found.

: Stefan Bornholdt and Dirk Graudenz: "General Asymmetric Neural Networks and Structure Design by Genetic Algorithms", in: Neural Networks, Vol. 5, pp. 327-334, Pergamon Press.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC