K. Balakrishnan and V. Honavar. Evolutionary Design of Neural Architectures: A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR 95-01, Department of Computer Science, Iowa State University, Ames, Iowa, 1995.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....quality of GA depending on the size. GAs have been used in various research projects to draw natural looking graphics [102] evolve virtual creatures [103] simulate a ant colony and their evolution [25] or generate robot configurations [68] Balakrishnan et al. use GA to design neural networks [7] and Yamauchi et al. explore the use of GA to evolve neural networks capable of sequential behavior and learning [127] 3.2.2 Simulated Annealing Simulated annealing can be considered a similar approach to find local extrema in a unknown multi dimensional space. Like genetic algorithms, it is a ....

.... algorithms to evolve continuous time recurrent NNs capable of sequential behavior and learning [127] Mathias replaces the batch model of learning by an interactive teaching model to obtain better performance in learning [74] Balakrishnan discusses evolutionary designs of neural architectures [7], and Boehse and Khang use simulated annealing on NN to show that the cooling strategy not necessarily decreases monotonically to zero [11] At last, Miltrup and Schnitger deepen into the memory complexity of NN [76] 3.2.4 Reinforcement Learning In contrast to neural networks, where the ....

K. Balakrishnan and V. Honavar. "Evolutionary Design of Neural Architectures: A Preliminary Taxonomy and Guide to Literature." Technical report, Department of Computer Science, Iowa State University, Ames, Iowa, 1995.

....is that, sometimes, the chosen strategies do not consider the interdependence between these parameters and may not necessarily find a good point in the complete space of parameters. A more unifying approach to define the architecture of neural networks is via the use of Genetic Algorithms [7]. Instead of treating each parameter in isolation, GAs are able to define, at same time, a large number of parameters, performing a global optimization in the search space. In [8] for example, GAs were successfully used to define the input variables, the number of hidden nodes, the activation ....

K. Balakrishnan & V. Honavar, Evolutionary Design of Neural Architectures: Preliminary Taxonomy and Guide to Literature, Technical Report CS TR95-01, Department of Computer Science, Iowa State University, 1995.

....approach is that, sometimes, the chosen strategies do not consider the interdependence between the parameters and may not necessarily find a good point in the complete space of parameters. A more unifying approach to define the architecture of neural networks is via the use of Genetic Algorithms [Balakrishnan, Honavar 1995]. Instead of treating each parameter in isolation, GAs are able to define, at same time, a large number of parameters, performing a global optimization in the search space of parameters. In [Hakkarainen et al. 1996] for example, GAs were successfully used to define the inputs variables, the number ....

Balakrishnan, K., Honavar, V. (1995), "Evolutionary Design of Neural Architectures: Preliminary Taxonomy and Guide to Literature", Tech. rept. CS TR95-01. Department of Computer Science, Iowa State University, Ames.

.... These latter two problems, premature convergence and parameter setting, have been approached using several optimization procedures, which can be divided into two groups: Incremental Decremental (see [8] by Alpaydim et al. for a review) or genetic algorithms, which are reviewed for instance in [9]. Incremental algorithms, are based on adding hidden neurons to a network of minimum size until the required precision is reached. These methods start with few hidden neurons and increase their number until the error is su ciently small. This approach is used in the Cascade Correlation ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary design of neural architectures { a preliminary taxonomy and guide to literature. Technical report, AI Research Group, January 1995. CS-TR 95-01.

....setting and optimization remains an open question. There are two ways of approaching the optimization of BP parameters for certain problem: Incremental decremental (see [10] by Alpaydim et al. for an interesting review) or genetic algorithms (see either Yao [11,12] or Balakrishnan et al. [13] for an interesting review) Incremental algorithms, such as Cascade Correlation by Fahlman and Lebi ere [9] and the Tiling and Perceptron Cascade Algorithm presented by Parekh et al. in [14] are based on adding hidden neurons to a network of minimum size until it reaches the required ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary design of neural architectures { a preliminary taxonomy and guide to literature. Technical report, AI Research Group, January 1995. CS-TR 95-01.

....K to the average number of connections for each element. Chapter 4 Evolutionary neural networks In this chapter, we give a short introduction to some of the most well known approaches for evolving neural network topologies. For more comprehensive overviews on evolutionary neural networks see (Balakrishnan and Honavar 1995; Branke 1995; Kodjabachian and Meyer 1995; Mitchell 1996) There are currently maybe dozens of dioeerent methods for evolving ANNs. We selected the rst two methods studied in this work (methods by Miller et al. and Kitano) mostly because they are so well known and easy to implement. The third ....

Balakrishnan, K. and V. Honavar (1995, January). Evolutionary design of neural architectures a preliminary taxonomy and guide to literature. Technical Report CS TR 9501, Department of Computer Science, Iowa State University, Ames, IA 50011.

.... State University, 945, 948, 949, 950] Deutsches Elektronen Synchrotron, 628] Ecole Normale Superiore, 772] Ecole Normale Sup erieure de Lyon, 716] Edinburgh Parallel Computing Centre, 886] Honeywell Corporate Systems, 727, 729] Institute of Psychology CNR, 834] Iowa State University, [150] LASPP FER, 662] NIBS Pte Ltd. 35] 14 Genetic algorithms and neural networks National Research Counsil (C. N. R. 271, 372] National University of Singapore, 896] Naval Command, 828] Ohio State University, 878] Oregon Graduate Center, 898] Politecnico di Milano, 812] ....

....Christopher T. 856] Atlan, Laurent, 772] Austin, Alan Scott, 14, 604] Austin, Scott, 603] Azevedo, Fernando M. de, 273] Baba, N. 605] Baba, Norio, 535] Back, Barbro, 148, 318, 430] Badii, A. 606] Baerdemaeker, J. 268] Baidyk, Tatyana N. 191] Balakrishnan, Karthik, [150] Balasekar, S. 426] Balicki, J. 236] Ball, A. D. 537] Ball, N. R. 323, 607, 608] Baluja, Shumeet, 151, 431] Banzhaf, Wolfgang, 65] Barham, John, 602] Barnard, S. T. 620] Barone, Dante Augusto Couto, 292] Barreto, Jorge M. 273] Barton, S. A. 58, 234] Bartscht, E. ....

[Article contains additional citation context not shown here]

Karthik Balakrishnan and Vasant Honavar. Evolutionary design of neural architectures -- preliminary taxonomy and guide to literature. Technical Report CS TR #95-01, Iowa State University, Artificial Intelligence Group, 1995. y([450] Branke) ga95aBalakrishnan.

....reasons it seems logical and attractive to apply genetic algorithms. Genetic algorithms may provide a useful tool for automating the design of neural networks. Various schemes for combining neural networks and genetic algorithms have been proposed. For an extended analysis in this subject see [4, 21, 123]) In most cases, genetic algorithms have been used to either train a network or to find a suitable topology. The study of the synergy of neural networks and genetic algorithms can be summarized to three general aspects: genetically training neural networks, genetic optimization of network ....

K. Balakrishnan and V. Honovar. Evolutionary Design of Neural Architectures - A Preliminary Taxonomy and Guide to Literature. Technical Report: ISU CS-TR 95-01, 1995.

....These problems, premature convergence and parameters setting, have been approached using several optimization procedures. These procedures applied to the MLP can be divided in two groups: Incremental decremental (see [3] by Alpaydim et al. for a good review) or genetic algorithms (see [4], by Balakrishnan et al. for a good review) ffl Incremental algorithms, such as Cascade Correlation by Fahlman and Lebi ere [5] the Tiling and Perceptron Cascade by Parekh et al. 6] or the methods proposed by Zhang [7] or Rathbun et al. 8] are based on adding hidden layer neurons to a ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary design of neural architectures -- a preliminary taxonomy and guide to literature. Technical report, AI Research Group, January 1995. CS-TR 95-01.

....achieved through generalizing previous encountered, known situations. The lack of knowledge in determining the appropriate topology of an artificial neural network, limits such capability. Evolutionary techniques (i.e. genetic algorithms) have been widely used for the design of these networks [2]. Essentially, the algorithm employs a population of neural networks, each encoded as a bit string genome ; evolution proceeds applying genetic operators to the population, such that in time better individuals (neural networks) emerge. Currently, these techniques are implemented as software ....

K. Balakrishnan. Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature. Technical Report CS-TR-95-01, Department of Computer Science, Iowa State University, Ames, USA, January 1995.

....most common paradigm employed in the artificial evolution of ANNs is the field of Genetic Algorithms (GA) in which pioneering work was done by J. H. Holland [Hol75] In order to structurize the various approaches to artificial evolution of ANNs we will use a slightly adapted taxonomy proposed in [BH95] The Genotype Encoding Scheme can be broadly classified into two categories: ffl Direct Encoding The number of neurons and the connections between specific neurons are explicitely encoded in the genotype which usually results in a connection matrix. The decoding effort to map the genotype to ....

....increases quadratically with each additional erroneous output. 4. 4 Parameters Following GA parameters have been used with all the experiments in this paper: Crossover probability p c = 0:6 Mutation probability p m = 0:005 Crossover: 2 point Selection Method: Tournament with 2 Design Taxonomy [BH95] Encoding: Direct Network Topology: Feed Forward Variables of Evolution: Evolving Topologies with Conventional Training Application Domain: Pattern Recognition Back propagation training: Number of epochs: 5000 (fixed) Learning parameter j = 0:2 No momentum term 5 A Simple Example This ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary design of neural architectures -- a preliminary taxonomy and guide to literature. Technical Report CS TR #95-01, Iowa State University, Department of Computer Science, Ames, Iowa 50011--1040, U.S.A., January 1995.

....machines. Performance reducing situations at the end of a generation due to overloaded machines are handled by bench penalties (i.e. the overloaded machine(s) take(s) a rest for a number of generations) 4 ANN Genotype Representation Out of numerous existing genotype representations of an ANN [BH95] we adopted a direct encoding method introduced by Miller et al. MTH89] Their original encoding in the following called Miller Matrix (MM) uses an adjacency matrix to represent the topology of an ANN. Each entry w ij 2 f0; Lg ( 0 . no connection, L . learnable connection) in the ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR #95-01, Iowa State University, Department of Computer Science, Ames, Iowa 50011--1040, U.S.A., January 1995.

....netGEN system we chose the latter approach by employing Genetic Algorithms. We restricted the genetic search to Feed Forward Networks which are trained by standard Error Back Propagation with a fixed learn rate and no momentum term. Out of numerous existing genotype representations of an ANN [11], we adopted a direct encoding method introduced by Miller Todd [12] Their original encoding in the following called Miller Matrix (MM) uses an adjacency matrix to represent the topology of an ANN. Each entry w ij 2 f0(no connection) L(learnable connection)g in the matrix at position (i; ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR #95-01, Iowa State University, Department of Computer Science, Ames, Iowa 50011--1040, U.S.A., January 1995.

....task setup and on the characteristics of a selected network type. A vast majority of attempts to automate neural networks high level parameter tuning is based on the evolutionary computation. Due to the space limits we do not include a review of such methods here (for up to date reviews refer to [1, 14]) instead we only express a general opinion that they are usually very expensive in terms of a number of network configurations to try before coming up with a satisfactory one. In contrast, new optimization techniques emerging from experimental design [2] and memory based learning [11] offer a ....

Balakrishnan K., Honavar V. Evolutionary Design of Neural Architectures - A Preliminary Taxonomy and Guide to Literature. Technical Reprort CS TR 95-01, Iowa State University, 1995.

....learning, resulting in new contexts which facilitate future learning. Hinton and Nowlan (1987) were among the first to consider computational ways in which evolution of strings and the learning of individuals might interact. Many additional examples can be found in Belew and Mitchell (1996) and Balakrishnan and Honavar (1995). Recently, Bala et al. (1995, 1996) Terano and Ishino (1995) and Turney (1995) have addressed the same issue in the 6 context of data analysis for pattern classification, marketing decisions on noisy questionnaire data, and medical data, respectively. As an indication of the wide applicability ....

Balakrishnan, K. and V. Honavar (1995). Evolutionary design of neural architectures: A preliminary taxonomy and guide to the literature. Artificial Intelligence Research Group, Department of Computer Science, Iowa State University, Technical Report CS TR #95-01.

....netGEN system we chose the latter approach by employing Genetic Algorithms. We restricted the genetic search to Feed Forward Networks which are trained by standard Error Back Propagation with a fixed learn rate and no momentum term. Out of numerous existing genotype representations of an ANN [12], we adopted a direct encoding method introduced by Miller Todd [13] Their original encoding in the following called Miller Matrix (MM) uses an adjacency matrix to represent the topology of an ANN. Each entry w ij 2 f0(no connection) L(learnable connection)g in the matrix at position (i; ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR #95-01, Iowa State University, Department of Computer Science, Ames, Iowa 50011--1040, U.S.A., January 1995.

....between the intended research other work which I know about and is apparently similar. ffl Genetic Programming has never been used to extract features of different types, or to solve general classification problems. Much research has been done on the design of ANNs using genetic algorithms [2], and [61] has used GP to evolve a variablesized feed forward network with summing or multiplying neurons. The novelty of the proposed work is that the nodes can contain one of around twenty different functions, from DFTs to If then else statements. GP has been used to classify target non target ....

Karthik Balakrishnan and Vasant Honavar. "Evolutionary Design of Neural Architectures - A Preliminary Taxonomy and Guide to Literature". TR CS 95-01, Artificial Intelligence Group, Iowa State University, Ames, Iowa 50011-1040 USA, January 1995.

....the Austrian National Park Hohe Tauern . Specifically, we consider the influence of an additional regularization term used with the GA fitness function. 2 Evolution of Artificial Neural Networks 2. 1 Genotype Representation of an ANN Out of numerous existing genotype representations of an ANN [BH95] we adopted a direct encoding method introduced by Miller Todd [MTH89] Their original encoding in the following called Miller Matrix (MM) uses an adjacency matrix to represent the topology of an ANN. Each entry w ij 2 f0(no connection) L(learnable connection)g in the matrix at position ....

Karthik Balakrishnan and Vasant Honavar. Evolutionary Design of Neural Architectures -- A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR #95-01, Iowa State University, Department of Computer Science, Ames, Iowa 50011--1040, U.S.A., January 1995.

....determining the network s topology is a difficult problem, limiting the usefullness of this approach. Evolutionary algorithms have been widely used for the design of artificial neural networks, and, specifically, their topology [18, 14, 24] A brief guide to literature can be found in Ref. [3]. Currently, these techniques are mostly implemented in software, since hardware implementation poses several obstacles, including large population sizes and computation intensive fitness evaluation. However, a special class of learning algorithms enables a hardware implemented learning network to ....

K. Balakrishnan. Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature. Technical Report CS-TR-95-01, Department of Computer Science, Iowa State University, Ames, USA, Jan. 1995.

....to introduce any new nomenclature. In this work we constrain the choice of activation functions to ones that are differentiable. This would enable such networks to be trained using gradient descent techniques like BP. Further, the functions should also have bounded output (e.g. in the interval [ 1, 1]) to prevent the weights from becoming unbounded themselves. Functions such as sigmoid, sine, cosine, tanh, and gaussian are suitable candidates for GNNs. Generalized Neural Networks 5 GNNs can be expected to produce compact mappings by drawing on the relative strengths of the different ....

.... the random searches become highly ineffective. Evolutionary approaches such as genetic algorithms (GAs) have been shown to produce near optimal results in vast, complex, and multimodal search spaces [8, 5] They are thus natural candidates for searching the space of network architectures [1]. These algorithms are models of processes that appear to be at work in biological evolution. Such systems work with populations of genotypes. In our application, a genotype is a specification of an ANN. Each genotype is itself a collection of structures (called genes) typically arranged in ....

K. Balakrishnan and V. Honavar, Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature, Tech. Rep. CS TR 95-01, Department of Computer Science, Iowa State University, Ames, IA - 50011, January 1995.

....environment and evolutionary processes in determining the structure and function of the resulting neural architectures. 1. Introduction Artificial neural networks offer an attractive paradigm for the design of behavior and control systems in robots and autonomous agents for a variety of reasons [7, 5, 2, 1] including: ability to adapt and learn, potential for resistance to noise, faults and component failures, potential for real time performance in dynamic environments (through massive parallelism and suitable hardware realization) etc. However, the task of designing a good neuro controller for a ....

K. Balakrishnan and V. Honavar, "Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature," Tech. Rep. CS TR 95-01, Department of Computer Science, Iowa State University, Ames, IA - 50011, January 1995.

No context found.

K. Balakrishnan and V. Honavar. Evolutionary Design of Neural Architectures: A Preliminary Taxonomy and Guide to Literature. Technical Report CS TR 95-01, Department of Computer Science, Iowa State University, Ames, Iowa, 1995.

No context found.

K. Balakrishnan and V. Honavar. Evolutionary design of neural architectures { a preliminary taxonomy and guide to literature. Technical Report CS TR 95-01, Department of Computer Science, Iowa State University, Ames,Ioma 50011-1040, USA, Jan 1995.

No context found.

K. Balakrishnan and V. Honavar, Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature, Tech. Report CS TR 95--01, Department of Computer Science, Iowa State University, Ames, IA 50011, January 1995.

No context found.

K. Balakrishnan and V. Honavar, Evolutionary design of neural architectures --- a preliminary taxonomy and guide to literature, Tech. Report CS TR 95--01, Department of Computer Science, Iowa State University, Ames, IA 50011, January 1995.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC