J. McDonnell and D. Waagen, "Evolving Recurrent Perceptrons for Time-Series Modeling," IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....the noisy fitness evaluation problem is to have a one to one mapping between genotypes and phenotypes. That is, both architecture and weight information are encoded in individuals and are evolved simultaneously. Although the idea of evolving both architectures and weights is not new [3] 10] [13], 26] few have explained why it is important in terms of accurate fitness evaluation. The simultaneous evolution of both architectures and weights can be summarized by Fig. 2. The evolution of ANN architectures in general suffers from the permutation problem [27] 28] or called competing ....

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. Neural Networks, vol. 5, pp. 24--38, 1994.

.... algorithms (GAs) evolution strategies (ES) and particle swarm optimisation (PSO) GAs have previously be used for feature selection [8, 53] and topography selection [2, 5, 29, 35, 36, 38, 52] and ESs have been used for weight optimisation [21, 42, 45, 55] and adaptive topography selection [15, 37, 57]. The recent EC technique of PSO [27] has also proved popular as a uni objective NN optimiser [10, 12, 13, 26, 48] 2 Multi objective evolutionary neural network flamework The use of evolutionary approaches to NN training (with a single error function) has received increasing attention in recent ....

J. McDonnell and D. Waagen. Evolving Recurrent Perceptrons for Time-Series Modeling. IEEE Transactions on Neural Networks, 5(1):24-38, 1994.

....examples of evolutionary search. Attempts at training feedforward neural networks by evolutionary search include the work of Fogel et al. 19] Yao and Liu [87] and Montana and Davis [57] There are also attempts to evolve recurrent networks, e.g. Angeline et al. 3] and McDonnell and Waagen [51]. Applying evolutionary search to more complex types of neural networks (high order networks, for example) can be found in [33 35,85] and a good review of evolving neural networks is provided by [86] Back et al. 4] and Fogel [17] provided an introduction to various evolutionary search ....

....variable d are selected for consideration of flipping. The weight with the largest absolute value of d in the subset is flipped, while others are flipped with very small probabilities. Gruau and Whitley showed that this simple local search can speed up the evolution process. McDonnell and Waagen [51] proposed a hybrid algorithm that combines the method of Solis and Wets [76] and evolutionary search for evolving RNNs. In each iteration of the evolutionary search, a set of offspring is generated by applying the Solis and Wets method and another set is generated by perturbing the parent ....

[Article contains additional citation context not shown here]

J. R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modelling. IEEE Transactions on Neural Networks, 5(1):24--38, 1994.

....hidden nodes differently have two different genotypical representations, the probability of producing a highly fit offspring by recombining them is often very low. Some researchers thus avoided crossover and adopted only mutations in the evolution of architectures [45] 128] 149] 179] 185] [197], 217] 223] although it has been shown that crossover may be useful and important in increasing the efficiency of evolution for some problems [48] 113] 212] 229] Hancock [113] suggested that the permutation problem might not be as severe as had been supposed with the population size ....

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. Neural Networks, vol. 5, pp. 24--38, Jan. 1994.

....and their graphical presentation in Section 3 show to what extent TAMSASS PECS is able to nd optimal or good solutions. 2. The TAMSASS PECS algorithm The TAMSASS PECS algorithm is based on the Threshold Accepting method [2] and on our modi ed version of SASS (Single Agent Stochastic Search) [4, 7, 8, 14, 15] (MSASS) The TAMSASS PECS algorithm has speci cally been built to solve the packing n equal circles in a square problem, but can be easily extended to pack equal circles in other regular shapes. The Threshold Accepting framework and the SASS local search algorithm are used for nding a local ....

....keeping the sideway moving with the same function value allowed. In [2] a more adaptive inner loop stopping criterion is worked with: it ends the inner loop after no improvements were made in the last steps. Random optimization is traditionally based on single agent stochastic search strategies [8]. S.S. Rao [14] and D.C. Karnop [4] used a uniform random variable as the move function, d(s) J. Matyas utilized Gaussian perturbations for the move function with a bias term to direct the search, N(b; I) 7] F.J. Solis and J.B. Wets [15] enhanced this approach by evaluating the objective ....

J.R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24-38, 1994.

....and their graphical presentation in Section 3 show to what extent TAMSASS PECS is able to nd optimal or good solutions. 2. The TAMSASS PECS algorithm The TAMSASS PECS algorithm is based on the Threshold Accepting method [2] and on our modi ed version of SASS (Single Agent Stochastic Search) [4, 7, 8, 14, 15] (MSASS) The TAMSASS PECS algorithm has speci cally been built to solve the packing n equal circles in a square problem, but can be easily extended to pack equal circles in other regular shapes. The Threshold Accepting framework and the SASS local search algorithm are used for nding a local ....

....level, keeping the sideway moves with the same function value allowed. In [2] a more adaptive inner loop stopping criterion is used: it ends the inner loop after no improvements were made in the last steps. Random optimization is traditionally based on single agent stochastic search strategies [8]. S.S. Rao [14] and D.C. Karnop [4] used a uniform random variable as the move function, d(s) J. Matyas utilized Gaussian perturbations for the move function with a bias term to direct the search, N(b; I) 7] F.J. Solis and J.B. Wets [15] enhanced this approach by evaluating the objective ....

J.R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24-38, 1994.

....to published ones. 3.1. Sunspots dataset This dataset contains the yearly number of dark spots on the sun from 1700 to 1979. The time series has a pseudo period of 10 to 11 years. Several models were evaluated for one step ahead predictions [11] 12] including feed forward [13] and recurrent [14] neural networks. The training set corresponds to the period 1700 1920 and two test sets were defined, 1921 1955 (test1) and 1956 1979 (test2) Test2 is considered to be more difficult because it has a larger variance. Table 1 compares the results obtained by various models applied to this ....

....error (NMSE) The Threshold AutoRegressive (TAR, 11] model employs a threshold to switch between two autoregressive models. The MLP has a time window of size 12 in the input layer and starts with 8 hidden neurons [13] a pruning algorithm reduces the number of hidden neurons to 3. The IIR MLP in [14] contains local feedbacks and delays and is obtained by an evolutionary algorithm. Model Parameters Learning Test1 Test2 Carbon Copy 0.289 0.427 0.966 TAR 18 0.097 0.097 0.280 MLP 43 0.082 0.086 0.350 IIR MLP 23 0.101 0.097 0.436 RNN with BPTT 155 0.064 0.084 0.300 RNN with CBPTT 15 0.098 0.092 ....

McDonnell, J.R. and D. Waagen, Evolving Recurrent Perceptrons for Time Series Modeling, IEEE Transactions on Neural Networks, 1994, 5(1): 24-38.

....to optimise the connection weights of nodes with a given network architecture which has been evolved by the PSO algorithm. In other words, the purpose of this process is to evaluate the quality of this given network architecture and maintain the behavioural link between a parent and its offspring [6]. 3 Experimental studies In order to evaluate the ability of PSONN in evolving ANN s, it was applied to product quality estimation where jet fuel endpoint variable is related to the thirteen variables. After 150 epoch off line learning, the maximum absolute mean error averaged 30 runs for jet ....

J. R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Tran. Neural Networks, vol. 5, no. 1, pp. 24-38, 1994. This work was supported by National 973 Fundamental Research Program of China.

.... approaches based on evolutionary algorithms, such as evolutionary programming (EP) 8] and genetic algorithms (GAs) 9] have been applied to the development of artificial neural networks (ANNs) Approaches based on EP operate on the neural network directly, and rely exclusively on mutation [10, 11, 12, 13] or combine mutation with training [14] Methods based on genetic algorithms usually represent the structure and the weights of ANNs as a string of 1 bits or as a combination of bits, integers and real numbers [15, 16, 17, 18, 19, 20] and perform the crossover operation as if the network were a ....

J. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modelling. IEEE Transactions on Neural Networks, 5(1):24--38, Jan. 1994.

.... [576] IEEE Transactions on Circuits and Systems I, Fundamental Theory and Applications, 647] IEEE Transactions on Evolutionary Computing, 504] IEEE Transactions on Fuzzy Systems, 275, 451, 486] IEEE Transactions on Industrial Electronics, 483] IEEE Transactions on Neural Networks, [45, 60, 117, 135, 232, 280, 376, 379, 418, 419, 426, 635, 688] IEEE Transactions on Pattern Analysis and Machine Intelligence, 389] IEEE Transactions on Power Systems, 545] IEEE Transactions on Semiconductor Manufacturing, 469, 546] IEEE Transactions on Systems, Man, and Cybernetics, 431, 455, 581] IEEE Transactions on Systems, Man, and ....

....[634, 635] Martinez, Tony R. 223, 235] Martins, Weber, 144] Masters, Timothy, 820] Matsushita, S. 500] Matthews, C. 514] May, G. S. 78, 344, 393] May, Gary S. 459, 469, 546] Mayer, Helmut A. 180, 369, 549] McCaskill, J. S. 821] McCullagh, J. 482, 624] McDonnell, John R. [61, 117, 822, 823, 824, 825, 826, 827, 828] McGregor, Douglas R. 829, 830, 831, 832, 833] McInerney, John, 614, 615, 616] McInerney, Michael, 805] McInerney, M. 62] Mecklenburg, Klaus, 910] Meeden, Lisa A. 455] Meisel, J. 417] Melsheimer, S. S. 55] Menczer, Filippo, 834, 835, 836] Mendoca, P. R. S. 480] Mendonca, ....

[Article contains additional citation context not shown here]

John R. McDonnell and Don E. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Transactions on Neural Networks, 5(1):24--38, January 1994. ga94bMcDonnell.

.... approaches based on evolutionary algorithms, such as evolutionary programming (EP) 8] and genetic algorithms (GAs) 9] have been applied to the development of artificial neural networks (ANNs) Approaches based on EP operate on the neural network directly, and rely exclusively on mutation [10, 11, 12, 13] or combine mutation with training [14] Methods based on genetic algorithms usually represent the structure and the weights of ANNs as a string of bits or as a combination of bits, integers and real numbers [15, 16, 17, 18, 19, 20] and perform the crossover operation as if the network were a ....

J. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modelling. IEEE Transactions on Neural Networks, 5(1):24--38, Jan. 1994.

....and their graphical presentation in Section 3 show to what extent TAMSASS PECS is able to nd optimal or good solutions. 2. The TAMSASS PECS algorithm The TAMSASS PECS algorithm is based on the Threshold Accepting method [2] and on our modi ed version of SASS (Single Agent Stochastic Search) [4, 7, 8, 14, 15] (MSASS) The TAMSASS PECS algorithm has speci cally been built to solve the packing n equal circles in a square problem, but can be easily extended to pack equal circles in other regular shapes. The Threshold Accepting framework and the SASS local search algorithm are used for nding a local ....

....keeping the sideway moving with the same function value allowed. In [2] a more adaptive inner loop stopping criterion is worked with: it ends the inner loop after no improvements were made in the last steps. Random optimization is traditionally based on single agent stochastic search strategies [8]. S.S. Rao [14] and D.C. Karnop [4] used a uniform random variable as the move function, d(s) J. Matyas utilized Gaussian perturbations for the move function with a bias term to direct the search, N(b; I) 7] F.J. Solis and J.B. Wets [15] enhanced this approach by evaluating the objective ....

J.R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24-38, 1994.

....0.398207310236850] 17 [0.306153985300327, 0.306153985300338] 19 [0.289541991994965, 0. 289541991994996] 11 5 The TAMSASS PECS algorithm The TAMSASS PECS algorithm is based on the Threshold Accepting method [5] and on our modi ed version of SASS (Single Agent Stochastic Search) [20, 28, 29, 49, 56] (MSASS) The TAMSASS PECS algorithm has speci cally been built to solve the packing n equal circles in a square problem, but can be easily extended to pack equal circles in other regular shapes. The Threshold Accepting framework and the SASS local search algorithm are used for nding a local ....

....level, keeping the sideway moves with the same function value allowed. In [5] a more adaptive inner loop stopping criterion is used: it ends the inner loop after no improvements were made in the last steps. Random optimization is traditionally based on single agent stochastic search strategies [29]. S.S. Rao [49] and D.C. Karnop [20] used a uniform random variable as the move function, d(s) J. Matyas utilized Gaussian perturbations for the move function with a bias term to direct the search, N(b; I) 28] F.J. Solis and J.B. Wets [56] enhanced this approach by evaluating the objective ....

J.R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24-38, 1994.

....parents from the population based on their fitness. 5. Apply search operators to the parents and generate offspring which form the next generation. Figure 6: A typical cycle of the evolution of architectures. Considerable research on evolving ANN architectures has been carried out in recent years [33, 42, 45, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 149, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 138, 213, 214, 215, 216, 118, 130, 127, 217, 218, 219, 220, 221, 222, 223, 128, 224, 225]. Most of the research has concentrated on the evolution of ANN topological structures. Relatively little has been done on the evolution of node transfer functions, let al..one the simultaneous evolution of both topological structures and node transfer functions. In this paper, we will analyze the ....

....Encoding Scheme Two different approaches have been taken in the direct encoding scheme. The first separates the evolution of architectures from that of connection weights [154, 153, 150, 24, 170, 169, 165, 167] The second approach evolves architectures and connection weights simultaneously [179, 180, 182, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 149, 198, 199, 200]. This section will focus on the first approach. The second approach will be discussed in Section 3.4. In the first approach, each connection of an architecture is directly specified by its binary representation [154, 153, 150, 24, 170, 169, 165, 167, 202] For example, an N Theta N matrix C = c ....

[Article contains additional citation context not shown here]

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.

....In this work a stochastic global optimization algorithm, called TAMSASS PECS (Threshold Accepting Modi ed Single Stochastic Search for Packing Equal Circles in a Square) has been designed. TAMSASSPECS algorithm is based on the Threshold Accepting method [3] and on our modi ed version of SASS [4, 5, 6, 7, 8] (MSASS) TAMSASS PECS is an algorithm which formally is very similar to the Simulating Annealing algorithm. TAMSASS PECS sets up and updates parameters for the MSASS procedure which is iteratively executed until stopping criterion is reached. MSASS is in charge of perturbing the current location ....

J.R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24-38, 1994.

....surface) is calculated, and weights are changed accordingly. The gradient information is therefore fully utilized. However, these learning methods are computationally expensive for large networks. Apart from the gradient descent algorithms, the algorithms from Solis and Wets [42] can also be used [32]. On the other hand, if binary representation is used, the learning methods [24] 30] will usually involve flipping some bits in a chromosome randomly in order to obtain a better chromoNovember 13, 1998 DRAFT 6 some. These bit flipping learning methods do not take the gradient information of ....

J. R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modelling. IEEE Transactions on Neural Networks, 5(1):24--38, 1994.

....which was shown to have better optimization capabilities than meta EP. From these two tables we observe that the recurrent bilinear perceptron is capable of performing better than the simple bilinear model in terms of NMSE and the model order. For comparisons, with other techniques please refer to (McDonnell and Waagen 1994). CONCLUSION In this paper, we presented a recurrent bilinear perceptron for time series prediction. Evolutionary programming, a multi agent stochastic search technique, has been successfully applied to optimize the parameters of a recurrent bilinear perceptron and used for prediction of ....

McDonnell J.R., and Waagen D., (1994). Evolving Recurrent perceptrons for time-series modeling, IEEE Trans. Neural Networks, vol. 5, no. 1, Jan.

....work on evolving ANNs in a number of aspects. First, EPNet emphasises the evolution of ANN behaviours and uses a number of techniques, such as partial training after each architectural mutation, to maintain the behavioural link between a parent and its offspring. While some of previous EP systems [6, 7, 8, 5], acknowledged the importance of evolving behaviours, few techniques have been developed to maintain the behavioural link between parents and their offspring. Second, EPNet encourages parsimony of evolved ANNs by attempting different mutations sequentially. That is, node or connection deletion is ....

....data points (starting from point 618) were used as testing data. The values of training and testing data were rescaled linearly to between 0:1 and 0:9. 3. 2 The Logistic Map The logistic map or the iterated quadratic x(t 1) 4x(t) 1 Gamma x(t) is known to be chaotic in the interval [0,1] [7]. Following McDonnell and Waagen [7] we shall use EPNet to evolve an ANN for predicting one step ahead. In the following experiments, the input to an ANN was x(t) and the true value of x(t 1) was used as the target value during training. 200 points were generated from x(0) 0:2. The training ....

[Article contains additional citation context not shown here]

J. R. McDonnell and D. Waagen. Evolving recurrent perceptrons for time-series modeling. IEEE Trans. on Neural Networks, 5(1):24--38, 1994.

....and evolves both architectures and weights at the same time in order to reduce the noise in fitness evaluation. The major steps of a typical EP algorithm can be described by Figure 1. A key component of EP is its mutation operators. Although there has been published work on evolving ANNs by EP [7, 8, 9, 10], the mutation operators used in this paper are quite different from others. In addition, this paper proposes a novel way of using validation sets in the evolution of ANNs. Our experiments have shown that the evolutionary ANNs can produce very competitive results in comparison with others. Section ....

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. on Neural Networks, 5(1):24--38, 1994.

....First, EPNet emphasises the evolution of ANN behaviours by EP and uses a number of techniques, such as partial training after each architectural mutation and node splitting, to maintain the behavioural link between a parent and its offspring effectively. While some of previous EP systems [12] [13], 10] 14] 15] 3] acknowledged the importance of evolving behaviours, few techniques have been developed to maintain the behavioural link between parents and their offspring. The common practice in architectural mutations was to add or delete hidden nodes or connections uniformly at random. ....

....the noisy fitness evaluation problem is to have a one to one mapping between genotypes and phenotypes. That is, both architecture and weight information are encoded in individuals and are evolved simultaneously. Although the idea of evolving both architectures and weights is not new [10] 26] [13], 3] few have explained why it is important in terms of accurate fitness evaluation. The simultaneous evolution of both architectures and weights can be summarised by Figure 2. 1. Evaluate each individual based on its error and or other performance criteria such as its complexity. 2. Select ....

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.

No context found.

J. McDonnell and D. Waagen, "Evolving Recurrent Perceptrons for Time-Series Modeling," IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.

No context found.

J. R. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Trans. on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.

No context found.

J. McDonnell and D. Waagen, "Evolving recurrent perceptrons for time-series modeling," IEEE Transactions on Neural Networks, vol. 5, no. 1, pp. 24--38, 1994.

No context found.

John R. McDonnell and Don E. Waagen. Evolving Recurrent Perceptrons for Time-Series Modeling. IEEE Transactions on Neural Networks, 5:24--38, 1994.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC