P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," Neural Computation, vol. 5, pp. 54--65, 1994.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....other main fields of computational intelligence, namely fuzzy logic and neural networks. An overview of the utilization of genetic algorithms to train and construct neural networks is given in [198] and of course other variants of evolutionary algorithms can also be used for this task (see e.g. [199] for an evolutionary programming, 200] for an evolution strategy example, and [97] 201] for genetic algorithm examples) Similarly, both the rule base and membership functions of fuzzy systems can be optimized by evolutionary algorithms, typically yielding improvements of the performance of ....

P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 54-- 65, 1994.

....based on evolutionary programming (EP) 19] 20] for designing feedforward ANN s [7] 9] The main structure of the system is shown in Fig. 1. EPNet does not use recombination operators in the simulated evolution in order to avoid the permutation (i.e. competing conventions) problem [21] [23]. It relies on novel mutations and a rank based selection scheme [24] EPNet evolves both architectures and connection weights of ANN s simultaneously in order to reduce the noise in fitness evaluation [7] In some of the previous studies in EANN s [25] the evolution of ANN architectures was ....

P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, no. 1, pp. 54--65, 1994.

....with a minimal number of hidden layers, nodes, and connections) and adds new layers, nodes, and connections if necessary during training, while a pruning algorithm does the opposite, i.e. deletes unnecessary layers, nodes, and connections during training. However, as indicated by Angeline et al. [10], Such structural hill climbing methods are susceptible to becoming trapped at structural local optima. In addition, they only investigate restricted topological subsets rather than the complete class of network architectures. Design of a near optimal ANN architecture can be formulated as a ....

....First, EPNet emphasises the evolution of ANN behaviors by EP and uses a number of techniques, such as partial training after each architectural mutation and node splitting, to maintain the behavioral link between a parent and its offspring effectively. While some of previous EP systems [3] [10], 12] 15] acknowledged the importance of evolving behaviors, few techniques have been developed to maintain the behavioral link between parents and their offspring. The common practice in architectural mutations was to add or delete hidden nodes or connections uniformly at random. In ....

[Article contains additional citation context not shown here]

P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 54--65, 1994.

....with which neural networks have been used in static problems. It has been proven that a RNN can approximate any known dynamical system [16] and several techniques for training RNNs have been developed [17] 18] 19] 20] 21] 22] 23] 24] 25] 26] 27] 28] 29] 30] 31] [32], 33] 34] 35] 36] 37] 38] 39] 40] A subset of these works directed to the trajectory generation problem have shown that a RNN can indeed be trained to produce desired trajectory behavior, and have demonstrated the success of their training algorithms in generating certain benchmark ....

.... with local minima and it is not always possible to find global solutions [30] Also, gradient descent approaches do not work well because the gradients tend to vanish as the dynamics of the neural networks evolve [31] To circumvent this problem, other approaches not based on gradient descent [32], 33] 34] 35] have been recently proposed. Still, they cannot provide satisfactory solutions in all cases and their convergence to useful results is not always guaranteed. Other techniques based on monotone systems theory work quite well for simple trajectories [36] 37] but how to scale ....

P.J. Angeline, G.M. Saunders, and J.B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. on Neural Networks, vol. 5, no. 1, pp. 54-65, January 1994.

....network shown in Figure . 4.2 racker problem To assess the ability of the method to evolve recurrent neural networks, it was applied to the complex problem of finding a control system for an agent whose objective is to track and clear a trail. The particular trail we used, the John Muir trail [16, 17], consists of 89 tiles in a 32x32 toroidal grid. The tracker starts in the upper left corner, and faces the first position of the trail. The only information available to the tracker is whether the position ahead belongs to the trail or not. Based on this information, at each time step, the ....

....moving) or turn left 90 (without moving) When the tracker moves to a posi tion of the trail, that position is immediately cleared. This is a variant of the well known ant problem often studied in the GP literature [4] Usually, the information to the tracker is given as a pair of input data [16, 17]: the pair is (1,0) if the po sition ahead of the current tracker position belongs to the trail, and (0,1) if it does not. The objective is to build a neural network that, at each time step, receives this information, returns the action to be carried out, and clears the maximum number of ....

[Article contains additional citation context not shown here]

P. J. Angeline, G. Saunders, and J. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):54-65, 1994.

....strategies [67,69,73] are typical examples of evolutionary search. Attempts at training feedforward neural networks by evolutionary search include the work of Fogel et al. 19] Yao and Liu [87] and Montana and Davis [57] There are also attempts to evolve recurrent networks, e.g. Angeline et al. [3] and McDonnell and Waagen [51] Applying evolutionary search to more complex types of neural networks (high order networks, for example) can be found in [33 35,85] and a good review of evolving neural networks is provided by [86] Back et al. 4] and Fogel [17] provided an introduction to ....

P. J. Angeline, G. M. Saunders, and J. B. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):54--65, 1994.

.... at least ressource wasting, that evolution will discover multiple instances of Of course, there also exists extensive unstructured representation, such as the Messy GA representation [22] di erent representations for the TSP [43] or extensive description of variable topology neural networks [5, 18] which will not be considered further in this paper in the light of the scalability issue. the same mechanism. But the to date ultimate research direction toward the evolution of complex solutions seems to lie in the so called morphogenetic approach: instead of evolving parts of solutions ....

P. J. Angeline, G. M. Saunders, and J. B. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(2):86-91, 1993.

....related anomalous completion. This suggested that brain activity is higher and longer the greater the anomaly [Caryl Harper 1996] Since in this stage myriads of possible explanations are generated, selection must be done. Much recent work is precisely there, in evolutionary recurrent networks [Angeline et al. 1994] [Batali 1995] But the selection rule is not clear. Usually a combination of interest, utility, and complexity is used, where MDL is the more popular candidate in the later case. We suggest SED instead or implementationoriented approximations. In our opinion, the next step is a re encounter ....

Angeline, P.J.; Saunders, G.M.; Pollac, J.B. "An Evolutionary Algorithm That Constructs Recurrent Neural Networks", IEEE Trans. Neur. Nets, Vol. 5, no. 1, pp 54-65, 1994.

....nodes, and connections) and adds new layers, nodes, and connections when necessary during training while a destructive algorithm does the opposite, i.e. starts with the maximal network and deletes unnecessary layers, nodes, and connections during training. However, as indicated by Angeline et al. [149], Such structural hill climbing methods are susceptible to becoming trapped at structural local optima. In addition, they only investigate restricted topological subsets rather than the complete class of network architectures [149] Design of the optimal architecture for an ANN can be ....

....during training. However, as indicated by Angeline et al. 149] Such structural hill climbing methods are susceptible to becoming trapped at structural local optima. In addition, they only investigate restricted topological subsets rather than the complete class of network architectures [149]. Design of the optimal architecture for an ANN can be formulated as a search problem in the architecture space where each point represents an architecture. Given some performance (optimality) criteria, e.g. lowest training error, lowest network complexity, etc. about architectures, the ....

[Article contains additional citation context not shown here]

P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. Neural Networks, vol. 5, pp. 54--65, Jan. 1994.

....network and successively removes connections and neurons until the network is no longer able to solve the problem, at which point the last move is undone. There are clearly problems with this kind of approach, which is in fact a form of hill climbing and is likely to become trapped in local maxima[10]. The fact that the search space for neural network architectures is in nitely large and nondi erentiable[11] makes the genetic algorithm approach a good candidate for success. Indeed, research into the evolution of neural network architectures has been largely successful[12, 13, 14, 15, 16, ....

....in local maxima[10] The fact that the search space for neural network architectures is in nitely large and nondi erentiable[11] makes the genetic algorithm approach a good candidate for success. Indeed, research into the evolution of neural network architectures has been largely successful[12, 13, 14, 15, 16, 17, 10]. 2.3 Transfer Functions The transfer function for all neurons of a neural networks is generally taken to be xed, although some attempts have been made to allow its adaptation over generations [18, 19] These schemes typically begin with a xed proportion of transfer functions, such as ....

Peter J. Angeline, Gregory M. Saunders, and Jordan P. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):54-65, January 1994.

....lacks expressive power. 2.1.2 The Genotype as the Structure Itself This approach makes no distinction between the genotype and the phenotype: the genes are components of the structure. Many structures are reported to be evolved this way: nite state machines [2, 17] graphs [18] neural networks [19], trees [4] The evolutionary algorithms involved are mainly Evolutionary Programming [2] and Genetic Programming (EP and GP) 4, 20] for trees. Structures can be built from scratch, which is a strong advantage over parameters optimization because it allows a wide search in the space of ....

P. J. Angeline, G. M. Saunders, and J. P. Pollack, An evolutionary algorithm that constructs recurrent neural networks, IEEE Transactions on Neural Networks, vol. 5, pp. 5465, January 1994.

....to other kinds of search in weight space, in which the operators for generating another candidate weight solution are not based on continuous gradients. Bengio et al. 6] investigate methods such as simulated annealing, multi grid random search, and discrete error propagation. Angeline et al. [1] (see also crossreference Chapter 15) propose a genetic approach that also avoids gradient computation. The simplest kind of search without gradient, however, simply randomly initializes all network weights until the resulting net happens to classify all training sequences correctly. In fact, as ....

P. J. Angeline, G. M. Saunders, and J. P. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):54--65, 1994.

No context found.

Peter J. Angeline, Gregory M. Saunders, and Jordan B. Pollack. An evolutionary algorithm that constructs recurrent networks. IEEE Transactions on Neural Networks, 5(1):54--65, 1994.

No context found.

Peter J. Angeline, Gregory M. Saunders, and Jordan B. Pollack. An evolutionary algorithm that constructs recurrent networks. IEEE Transactions on Neural Networks, 5(1):54--65, 1994.

No context found.

P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Transactions on Neural Networks, 5 (1), pp. 54-65, 1994.

No context found.

P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," Neural Computation, vol. 5, pp. 54--65, 1994.

No context found.

P. J. Angeline, G. M. Saunders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," Neural Computation, vol. 5, pp. 54--65, 1994. 32

No context found.

Angeline, P. J., Saunders, G. M., & Pollack, J. B. (1994). An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans. Neural Networks, 5, 54-- 65.

No context found.

P.J. Angeline, Gregory M. Saunders, Jordan B. Pollack, An evolutionary algorithm that constructs recurrent neural networks, IEEE Transactions on Neural Networks 5 (1994) 54--65.

No context found.

Angeline, P. J., Saunders, G. M., and Pollack, J. B. (1994): An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks 5(1):54-65.

No context found.

P. J. Angeline, G. M. Saunders, and J. P. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1):5465, January 1994.

No context found.

Peter J. Angeline, Gregory M. Saunders, and Jordan B. Pollack. An evolutionary algorithm that constructs recurrent networks, 1994.

No context found.

P. J. Angeline, G. M. Saunders, and J. B. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5:54-- 65, 1994.

No context found.

P. J. Angeline, G. M. Sauders, and J. B. Pollack, "An evolutionary algorithm that constructs recurrent neural networks," IEEE Trans. on Neural Networks, vol. 5, no. 1, pp. 54--65, 1994.

No context found.

P.J. Angeline, G.M. Saunders, and J.B. Pollack. An Evolutionary Algorithm that Constructs Recurrent Neural Networks. IEEE Transactions on Neural Networks, 5(1):54--64, 1994.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC