A. Jacobs R. Increased Rates of Convergence Through Learning Rate Adaptation. Technical Report: UM-CS-1987-117. University of Massachusetts, Amherst, MA, 1987.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....and the error 1 Sip (s) and r is a parameter called learning rate. Z.2 Improvements to SBP Many methods have been proposed to improve generalisation performance and con vergence time of BP. Current research mostly concentrates on: the optimum setting of learning rates and momentum [5, 9, 18, 40, 45, 50, 51, 52]; the optimum setting of the initial weights [6, 24, 53] the enhancement of the contrast in the input patterns [23, 28, 1, 48, 57] changing he error function [1, 9, 17, 2, 44, 46] finding optimum architectures using pruning echniques [7, 15] In he following we will describe wo speed up methods ....

....learning rae coefficien implicitly by adding o he weigh change a fraction of he las weigh change as follows: where p is a parameter called momentum. This method decreases he oscillation which may occur wih large learning raes and accelerates he convergence. For a more derailed discussion see [9, 18, 50]. Rprop is one ofhe fases variations ofhe SBPalgorihm [40, 4, 56] Rprop stands for Resilien backpropagation . I is a local adaptive learning scheme, peorming supervised batch learning. The basic principle of Rprop is o eliminate he haful OE influence of he magnitude of he partial derivative o ....

Robert A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295-307, 1988.

....evolution could be considered as the first attempt of the evolution of learning rules [32] 152] 272] Harp et al. 152] encoded BP s parameters in chromosomes together with ANN s architecture. This evolutionary approach is different from the nonevolutionary one such as offered by Jacobs [273] because the simultaneous evolution of both algorithmic parameters and architectures facilitates exploration of interactions between the learning algorithm and architectures such that a near optimal combination of BP with an architecture can be found. Other researchers [32] 139] 213] 272] ....

R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, no. 3, pp. 295--307, 1988.

....basis functions. Neural networks controlled by fuzzy logic. Some basic theoretical aspects, detailed description of the characteristics of the methodological components, as well as the early adoptions of neural networks controlled by fuzzy logic, can be found in a series of publications [15] [16], 17] 18] 19] 20] 21] and [22] In [23] is addressed the concept of a fuzzy neural network to implement syllogistic fuzzy reasoning. In syllogistic fuzzy reasoning, the consequence of a rule in one reasoning stage is passed to the next stage as a fact. The approach is shown to be ....

Jacobs R.A. Increased rates of convergence through learning rate adaptation, Neural Networks Vol.1, 295-307, 1988

....algorithm and avoid oscillations in a steep direction of the error surface. However, it is well known that this approach tends to be inefficient. For example, this happens when the search space contains long ravines that are characterized by sharp curvature across them and a gently slopping floor [28], 62] Moreover, this approach introduces difficulties in obtaining convergence of BP training algorithms [33] 38] Nevertheless, there are theoretical results that guarantee the convergence of batch BP algorithms for a constant learning rate. In this case, the learning rate should be ....

.... keep gradient direction fairly constant, or rapidly decrease it, if the direction of the gradient varies greatly at each epoch [11] 3) For each weight, an individual learning rate is given, which increases if the successive changes in the weights are in the same direction and decreases otherwise [28], 54] 60] 63] 4) Use a closed formula to calculate a common learning rate for all the weights at each iteration [27] 42] 56] or a different learning rate for each weight [15] 43] Note that all the above mentioned strategies employ heuristic parameters in an attempt to enforce the ....

[Article contains additional citation context not shown here]

R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295--307, 1988.

.... constant, or rapidly decrease it if the direction of the gradient varies greatly [4] 3) use a local learning rate for each weight w i 2 (i = 1; 2; n) i.e. j 1 ;j n , which increases if the successive corrections of the weights are in the same direction and decreases otherwise [8], 19] 23] 27] This paper focuses on the last approach and particularly on the special class of first order adaptive training algorithms that employ local learning rates. These algorithms employ heuristic strategies to adapt the learning rates at each iteration and require fine tuning ....

....as well as to exploit the parallelism inherent in the evaluation of the error, E(w) and its gradient, rE(w) by the backpropagation (BP) algorithm, consists of using a different adaptive learning rate for each direction in weight space. Batch type BP training algorithms of this class [6] [8], 19] 23] 27] follow the iterative scheme 0 diagfj ) 1) and try to decrease the error by searching a local minimum with small weight steps. These steps are usually constrained by problem dependent heuristic parameters in order to avoid oscillations and ensure subminimization of ....

[Article contains additional citation context not shown here]

R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295--307, 1988.

....of implementing more than a neuron in a single PE. This allows to share the resources and to optimize the silicon area occupancy. Therefore the architecture can be implemented as mixed grain computing structure. Moreover more complex learning algorithms (momentum, the delta delta learning rule [5],etc. can be easily implemented. PROCEDURE ForwardMode (VAR PE:Neuron) wait for a valid input forward mode operations IF Right.Valid3=on THEN BEGIN BEGIN WITH PE DO State: State 1; BEGIN WaitStates: Positlon; IF LeftActivated THEN END; BEGIN IF State= i THEN signal busy state wait ....

R.A. Jacobs, "Increased Rates of Convergence Through Learning Rate Adaptation", Neural Networks, Vol. 1, No. 4, 1988, pp.295-307.

....information (conjugate gradient, Quasi Newton, second order calculation of the step size) stochastic optimisation, and heuristics utilising the sign of the local gradient, the angle between gradient direction, or peak learning rate values. We discuss the delta bar delta rule by Jacobs [28], and refer the reader to the literature for a survey of some of the other common techniques employed [45] The delta bar delta adaptive learning rate technique defines a separate learning rate for each weight. It uses an estimation of the slope of the local error function to adjust the learning ....

R. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295--307, 1988.

No context found.

A. Jacobs R. Increased Rates of Convergence Through Learning Rate Adaptation. Technical Report: UM-CS-1987-117. University of Massachusetts, Amherst, MA, 1987.

No context found.

R. A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1 (4):295--308, 1988.

No context found.

R.A. Jacobs, Increased rates of convergence through learning rate adaptation, Neural Network 1 (1988) 295--307.

No context found.

R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295--307, 1988.

No context found.

R. A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295--307, 1988.

No context found.

R. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, no. 1, pp. 295--307, 1989.

No context found.

R. A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295-307, 1988.

No context found.

R. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1(4), 1988.

No context found.

R.A. Jacobs, Increased rates of convergence through learning rate adaptation, Neural Networks 1 (1988) 295--307.

No context found.

R. A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295#307, 1988.

No context found.

R.A. Jacobs, Increased rates of convergence through learning rate adaptation, Neural Networks, 1, 295--307, (1988).

No context found.

Jacobs, R.A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1, 295--307.

No context found.

R. Jacobs, \Increased rates of convergence through learning rate adaptation", Neural Networks, 1:295-307, 1988.

No context found.

R. Jacobs, \Increased rates of convergence through learning rate adaptation", Neural Networks, 1:295-307, 1988.

No context found.

Robert A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1:295 -- 307, 1988.

No context found.

R. A. Jacobs. Increased rates of convergence through learning rate adaptation. Neural Networks, 1(4):295--307, 1988.

No context found.

R. A. Jacobs, "Increased Rates of Convergence through Learning Rate Adaptation," Neural Networks, vol. 1, pp. 295--307, 1988.

No context found.

Jacobs, R. A., Increased rates of convergence through learning rate adaptation. Neural Networks 1 (1988), 295--307.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC