M. Moreira and E. Fiesler, "Neural networks with adaptive learning rate and momentum terms," Tech. Rep. 95-04, Institut Dalle Molle D'Intelligence Artificielle Perceptive, Case Postale 609, 1920. Home/Search Document Details and Download
Summary Related Articles Check |
This paper is cited in the following contexts:
Modelling Chaotic Systems with Neural Networks: Application to.. - van Zyl (Correct)
....optimisation, and heuristics utilising the sign of the local gradient, the angle between gradient direction, or peak learning rate values. We discuss the delta bar delta rule by Jacobs [28] and refer the reader to the literature for a survey of some of the other common techniques employed [45]. The delta bar delta adaptive learning rate technique defines a separate learning rate for each weight. It uses an estimation of the slope of the local error function to adjust the learning rate. This estimation is derived using the partial derivative of the error function with respect to the ....
M. Moreira and E. Fiesler, "Neural networks with adaptive learning rate and momentum terms," Tech. Rep. 95-04, Institut Dalle Molle D'Intelligence Artificielle Perceptive, Case Postale 609, 1920.
Quantization and Pruning of Multilayer Perceptrons: Towards.. - Lundin, Moerland (1997) (Correct)
.... to update the weights (as such it might be considered as a generalization of the LMS algorithm to multilayer networks) To improve this algorithm a variety of different training techniques, all based on the original backpropagation algorithm [Rumelhart 86] have been proposed in the literature [Moreira 95]. However, since none of them is showing an improved performance on a wide class of problems, the original on line backpropagation algorithm is used in this report. The backpropagation algorithm consists of two different phases: a forward pass and a backward pass. In the forward phase, a pattern ....
....[ 1,1] while the other benchmarks are classification problems with desired output values of Gamma1 and 1. For each run the network was initialized with different random weights in the interval [ 0.77,0.77] according to [Thimm 96] Furthermore, a learning rate of 0. 5 was chosen according to [Moreira 95]; only for the Digit benchmark a learning rate of 0.1 has been used. Moreover, a momentum term of 0.9, and a flat spot constant of 0.1 have been used. In all experiments, the network training was done with the on line backpropagation algorithm as described in section 1.3. The tables with ....
M. Moreira and E. Fiesler. Neural Networks with Adaptive Learning Rate and Momentum Terms. Technical report 95-04, IDIAP, Martigny, Switzerland, available via anonymous ftp: ftp://ftp.idiap.ch/pub/techreports/95-04.ps.Z.
A Class of Asymptotically Stable Algorithms for Learning-Rate.. - Rüger (Correct)
....n (the number of weights) updates to the gradient direction. Another reason for resetting the direction arises when g t does not have the minimal positive projection c, say 0:01, onto the normalised gradient. Comparisons of Salomon s algorithm to many other methods have already been published (Moreira and Fiesler 1995; Salomon and van Hemmen 1996; Pfister and Rojas 1996) This paper not only demonstrates that significant improvements are brought about by PolakRibi ere directions (10) but also gives comparisons to pure backpropagation and conjugate gradient with the Polak Ribi ere rule. The following problems ....
Moreira, M. and E. Fiesler (1995). Neural networks with adaptive learning rate and momentum terms. In Technical Report 95-04 Institut Dalle Molle d'Intelligence Artificielle Perceptive, Martigny.
Discrete All-Positive Multilayer Perceptrons for.. - Moerland, Fiesler.. (1998) Self-citation (Fiesler) (Correct)
....on XOR(2) and gives the best classification rate on the test set for the Wine problem. A comparison of these results with the ones obtained in a benchmarking study (also including Xor(2) Sonar, Wine, and Digit benchmarks) of different adaptive learning rate algorithms for ordinary MLPs [27], shows that the influence of the optical non idealities is as good as negligible and gives comparable results. 3.2.3 Experimental Results for Training Discrete Non Negative MLPs For most benchmarks, the number of discretization levels, d, used in the simulations is subsequently 2, 4, 6, 8, and ....
M. Moreira and E. Fiesler, Neural Networks with Adaptive Learning Rate and Momentum Terms. IDIAPRR 95-04, IDIAP, Martigny, Switzerland. ftp://ftp.idiap.ch/pub/reports/1995/95-04.ps.Z.
Online articles have much greater impact More about CiteSeer.IST Add search form to your site Submit documents Feedback
CiteSeer.IST - Copyright Penn State and NEC