S. Singhal and L. A. Wu, "Training multilayer perceptrons with the extended Kalman algorithm," Advances in Neural Information Processing Systems 1, pp. 133--140, San Mateo, CA: Morgan Kaufmann 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....to a nonlinear observation on w. The SPKF can then be applied directly as an efficient second order technique for learning the parameters. Four Regions Classification: In the parameter estimation example, we consider a benchmark pattern classification problem having four interlocking regions [37]. A three layer feedforward network (MLP) with 2 10 10 4 nodes is trained using inputs randomly drawn within the pattern space, S = 1, 1] x [1, 1] with the desired output value of 0.8 if the pattern fell within the assigned region and 0.8 otherwise. Figure 4 illustrates the classification ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman filter. In Advances in Neural Information Processing Systems 1, pages 133-140, San Mateo, CA, 1989. Morgan Kauffman.

.... Filtering While quickprop is an improvement over the standard backpropagation algorithm for adjusting the weights in the cascade network, it is still essentially a gradient descent based algorithm, which, although simple, can require many iterations until satis factory convergence is reached [15, 22]. Thus, we modify standard cascade learning by replacing the quickprop algorithm with node decoupled extended KaIman filtering (NDEKF) 23] which has been shown to have better convergence properties and faster training times than gradient descent techniques for multi layer feedforward net ....

....replacing the quickprop algorithm with node decoupled extended KaIman filtering (NDEKF) 23] which has been shown to have better convergence properties and faster training times than gradient descent techniques for multi layer feedforward net works. In general extended Kalman filtering (GEKF) [22], an m x m conditional error covariance matrix P, which stores the interdependence of each pair of m weights in a given neural network is explicitly generated. NDEKF reduces this computational and storage com plexity by as the name suggests decoupling weights by node, so that we consider only ....

S. Singhal and L. Wu, "Training Multilayer Perceptrons with the Extended Kalman Algorithm," Advances in Neural Information Processing Systems 1, ed. D.S. Touretzky, Morgan Kaufmann Publishers, pp. 133-140, 1989.

....1. Introdution Due to the slow nature of network training with the standard back propagation algorithm, a great deal of research effort has been expended in the linear adaptive filtering literature with the objetive of using Kalman filter approach to train feedforward neural networks ( 1] 2] [3], 4] 5] It is known that the speed of convergence of the back propagation algorithm is that of a gradient method based algorithm [6] On the other hand, Kalman filtering algorithms utilize information contained in the input data more effectively. In this case, the use of Kalman filtering ....

....contained in the input data more effectively. In this case, the use of Kalman filtering algorithms offers the desirable advantages of fast speed of convergence, a built in learning rate parameter and, insensitiveness to variations in the condition number of the input data. Singhal and Wu [3] have proposed the use of the recursive least squares RLS (Kalman) algorithm by defining a quadratic cost function and linearizing it with respect to synaptic weights, at each working point. The result of such an approach is a complex computational problem because it requires storage and updating ....

S. Singhal, and L. Wu. Training multilayer perceptrons with the extended Kalman algorithm. Advances in Neural Information Processing Systems, Vol. 1, pages 133-140, Morgan Kaufman Pub. Inc., 1989.

....fact does not require the parameter. Finally, the plot for batch size 50 shows how lowering serves to dampen oscillations observable (as a function of ) for high values of . 4. 2 The Four Regions Task As a more realistic and di cult test example we use the four regions classi cation task [7] to be solved by a multi layer perceptron, as shown in Figure 2 7 7 7 7 7 7 OO fl fl jjT T T T T TT T T T T T T T T T T T T T T T T T 7 7 7 UUU UU UUU UUU UUU 44 ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman lter. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems. Proceedings of the 1988 Conference, pages 133140, San Mateo, CA, 1989. Morgan Kaufmann.

....(x t x t ) 10) where x t = F(x t 1 ; w) R and R are the additive noise and innovations noise covariances, respectively. This interpretation will be useful when dealing with alternate forms of the Dual EKF in section 0.3.3. 0.2. 2 EKF Weight Estimation As proposed initially in [37], and further developed in [29, 28] the EKF can also be used for estimating the parameters of nonlinear models (i.e. training neural networks) from clean data. Consider the general problem of learning a mapping using a parameterized nonlinear function G(x k ; w) Typically, a training set is ....

Sharad Singhal and Lance Wu. Training multilayer perceptrons with the extended Kalman lter. In Advances in Neural Information Processing Systems 1, pages 133-140, San Mateo, CA, 1989. Morgan Kauman.

....this fashion for several epochs through the training data until a satisfactory neural model is achieved. In this research, the EKF training method is used due to its well known superior convergence speed and lower tendency to get stuck in local minima as compared to other gradient descent methods [7]. Once the neural model is built, both the open loop and closed loop methodologies, depicted in Figs. 1.2a and 1.2b, respectively, are used for model evaluation. In the open loop mode the network is provided with the true values of the signal at every time step and the goal is to predict one step ....

....every iteration of the Kalman filter. This is a significant overhead, especially when the state vector has a high dimension. 3. 4 Application of EKF to Neural Networks The extended Kalman filter algorithm was first applied to the training of feed forward multilayer perceptrons by Singhal and Wu [7]. They showed that the EKF algorithm converged in fewer iterations than the standard back propagation algorithm using a few artificial examples. They also showed that in some cases when the back propagation algorithm failed, the EKF converged to a good solution. Following this useful ....

[Article contains additional citation context not shown here]

Shared Singhal and Lance Wu, "Training multilayer perceptrons with the extended Kalman algorithm", Advances in Neural Information Processing Systems, pp. 133--140, 1989. 103

....extended the idea to recurrent neural network training [90] Gorinevsky [28] provided the persistence of excitation condition for using the recursive least squares method in neural network training. Along the same line of research, Matthews Moschytz [73] Iiguni et al. 37] Singhal Wu [97], and Shan et al. 95] independently applied an extended Kalman filter in training a multi layered perceptron and showed that its performance was superior to using a conventional backpropagation training method. Ruck et al. 93] gave a comparative analysis of the use of the extended Kalman filter ....

....layer feedforward neural network where y 2 R is the output, x 2 R m is the input and 2 R n is its parameter vector. Given a 25 Chapter 3 Techniques for Neural Learning set of training data fx(i) y(i)g N i=1 , the training of a neural network can be formulated as a filtering problem [2, 97] assuming that the data are generated by the following noisy signal model : t) t Gamma 1) v(t) 3.15) y(t) f(x(t) t) ffl(t) 3.16) where v(t) and ffl(t) are zero mean Gaussian noise with variance Q(t) and R(t) A good estimation of the system parameter can thus be obtained via ....

[Article contains additional citation context not shown here]

Singhal S. and L. Wu (1989). Training multilayer perceptrons with the extended Kalman algorithm, in Advances in Neural Information Processing Systems I, D.S.Touretzky Ed., 133-140.

....state transition matrix, driven by process noise r k (the choice of variance determines convergence and tracking performance) The output d k corresponds to a nonlinear observation on w k . The EKF can then be applied directly as an efficient second order technique for estimating the parameters [7, 8]. In this paper, we present new derivative free implementations of Kalman filtering for this purpose. This work was sponsored in part by NSF under grant ECS 0083106, and DARPA under grant F3361598 C 3516. Actual (sampling) Linearized (EKF) UT y = f (x) P y = A T P x A y = f ( x) f ( ....

....of the hand. The learning curves of the different filters are shown in Figure 2a. As expected, the performance of all filters are comparable. In the next example, we consider training a 2 10 10 4 network on a benchmark pattern classification problem having four interlocking regions (see [7] for details) Figure 2b illustrates learning curves for the different filters, and again shows the equivalent performance of the square root derivative free approaches relative the EKF. Finally, Figure 3 shows how the computational complexity of the different filters scale as a function of the ....

S. Singhal and L. Wu, "Training multilayer perceptrons with the extended Kalman filter," in Advances in Neural Information Processing Systems 1, San Mateo, CA, 1989, pp. 133--140.

....The technique is based on gradient descent in the weight space over an error surface created by the sum of the squared error at each output node over the entire training set. Another rule for adjusting the weights is the extended Kalman filtering algorithm initially proposed by Singhal and Wu [16]. The Kalman filtering approach considers finding the weight values in the network to be an estimation problem. The goal of a Kalman filter is to estimate the state of a system based on observations taken over time. In the multilayer perceptron case, the state of the system being estimated is the ....

SINGHAL, S., AND WU, L. Training multilayer perceptrons with the extended Kalman algorithm. In Advances in Neural Information Processing Systems 1 (1989), D. S. Touretzky, Ed., Morgan Kaufmann, pp. 133--140.

....then be applied directly as an efficient second order technique for learning the parameters. In the linear case, the relationship between the Kalman Filter (KF) and Recursive Least Squares (RLS) is given in [3] The use of the EKF for training neural networks has been developed by Singhal and Wu [9] and Puskorious and Feldkamp [8] Dual Estimation A special case of machine learning arises when the input x k is unobserved, and requires coupling both state estimation and parameter estimation. For these dual estimation problems, we again consider a discrete time nonlinear dynamic system, x ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman filter. In Advances in Neural Information Processing Systems 1, pages 133--140, San Mateo, CA, 1989. Morgan Kauffman.

....state distribution remains Gaussian. Such algorithms are known as extended Kalman lters (EKF) 15] 16] The EKF has been used both in the classical setting of state estimation for nonlinear dynamical systems, and also as a basis for online learning algorithms for feedforward neural networks [17] and radial basis function networks [18] 19] Another possibility is to propagate a set of discrete samples in state space through f and g, and to re weight them using the likelihood p(yjx) Algorithms which use this general strategy are known as particle lters [20] a particular form of which ....

S. Singhal and L. Wu, \Training multilayer perceptrons with the extended Kalman algorithm," in Advances in Neural Information Processing Systems, vol. 1, pp. 133-140. Morgan Kaufmann, 1989.

....all parameters. 3. Gradient descent and LSE : this is the proposed hybrid learning rule. 4. Sequential LSE only : the ANFIS is linearized w.r.t. all parameters and the extended kalman filter algorithm is employed to update all parameters. This has been proposed in the neural network literature [40, 39]. The choice of above methods should be based on the trade off between computation complexity and resulting performance. Our simulations presented in the next section are performed by the forward pass backward pass premise parameters fixed gradient descent consequent parameters least squares ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended kalman algorithm. In David S. Touretzky, editor, Advances in neural information processing systems I, pages 133--140. Morgan Kaufmann Publishers, 1989.

....with gradient descent training algorithms we look towards extended Kalman filtering (EKF) What makes EKF algorithms attractive is that, unlike backpropagation, they explicitly account for the pairwise interdependence of the weights in the neural network during training. Singhal and Wu [5] were the first to show how the EKF algorithm can be used for neural network training. While converging to better local minima in many fewer epochs than backpropagation, their global extended Kalman filtering (GEKF) approach, carries a heavy computational toll. GEKF s computational complexity is , ....

....function are the Gaussian function, Bessel functions, and sinusoidal functions of various frequency [6] 3. Node decoupled extended Kalman filtering While quickprop is an improvement over standard backpropagation it can still require many iterations until satisfactory convergence is reached [3, 5]. Thus, we modify standard cascade learning by replacing the quickprop algorithm with node decoupled extended Kalman filtering (NDEKF) which has been shown to have better convergence properties and faster training times than gradient descent techniques for fixed architecture multi layer ....

[Article contains additional citation context not shown here]

S. Singhal and L. Wu, "Training Multilayer Perceptrons with the Extended Kalman Algorithm," Advances in Neural Information Processing Systems 1, ed. Touretzky, D. S., Morgan Kaufmann Publishers, pp. 133-140, 1989.

....state transition matrix, driven by process noise u k : w k = w k 1 u k (11) y k = f(x k 1 ; w k ) v k n k : 12) The noisy measurement y k has been rewritten as an observation on w. This allows the use of an EKF for weight estimation (representing a second order optimization procedure) [7]. Two EKFs can now be run simultaneously for signal and weight estimation. At every time step, the current estimate of the weights is used in the signal filter, and the current estimate of the signal state is used in the weight filter. 4 The dual UF EKF algorithm is formed by simply replacing ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman filter. In Advances in Neural Information Processing Systems 1, pages 133--140, San Mateo, CA, 1989. Morgan Kauffman.

....state transition matrix, driven by process noise u k : w k = w k 1 u k (11) y k = f(x k 1 ; w k ) v k n k : 12) The noisy measurement y k has been rewritten as an observation on w. This allows the use of an EKF for weight estimation (representing a second order optimization procedure) [7]. Two EKFs can now be run simultaneously for signal and weight estimation. At every time step, the current estimate of the weights is used in the signal lter, and the current estimate of the signal state is used in the weight lter. The dual UF EKF algorithm is formed by simply replacing the EKF ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman lter. In Advances in Neural Information Processing Systems 1, pages 133-140, San Mateo, CA, 1989. Morgan Kauman.

.... , fR; Q; A; Pig given the measurements fx 1:N ; y 1:N g 1 , where and Pi denote the mean and covariance of the Gaussian prior p( 0 j ) 3 The extended Kalman smoother One of the earliest implementations of the extended Kalman filter (EKF) to train MLPs is due to Singhal and Wu (Singhal and Wu 1988). The algorithm s computational complexity is of the order O(cm 2 ) multiplications per time step. Shah, Palmieri and Datum (Shah, Palmieri and Datum 1992) and Puskorius and Feldkamp (Puskorius and Feldkamp 1991) have proposed various approximations to the weights covariance so as to simplify ....

Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133--140.

....of confidence intervals and of mixing coefficients, required to generate mixtures of models, has motivated us to train neural networks with the extended Kalman smoothing algorithm. 4. 2 Training MLPs with the EKF One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu (Singhal and Wu 1988). In their method, the network weights are grouped into a single vector w that is updated in accordance with the EKF equations. The entries of the Jacobian matrix are calculated by back propagating the m output values fy 1 (t) y 2 (t) Delta Delta Delta ; ym (t)g through the network. An ....

Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133--140.

....The E.K.F. algorithm might thus diverge, whereas the P.E. method will always provide the best predictor of a given arbitrary structure. As a consequence, P.E. methods are preferred for the training of neural networks. For a presentation of E.K.F. methods applied to neural network training, see [Singhal and Wu 1989, Matthews and Moschytz 90] In the framework of a P.E. method, a candidate network is trained by minimizing its MSPE on finite size input output training sequences. The best candidate is selected using test sequences: it is the predictor with the smallest MSPE on the test sequence, thus avoiding ....

Singhal S., Wu L. Training multilayer perceptrons with the extended Kalman algorithm, Advances in Neural Information Processing Systems I (Morgan Kaufmann, San Mateo 1989), pp. 133-140.

....that is amenable to the design of inference and learning algorithms. In the past, various authors have studied the problem of approximating the distribution of neural network weights with a Gaussian function. One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu (Singhal and Wu 1988). In their method, the network weights are grouped into a single vector w that is updated in accordance with the EKF equations. The entries of the Jacobian matrix (i.e. the derivative of the outputs with respect to the weights) are calculated by back propagating the output observations through the ....

Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133--140.

....of confidence intervals and of mixing coefficients, required to generate mixtures of models, has motivated us to train neural networks with the extended Kalman smoothing algorithm. 4. 2 Training MLPs with the EKF One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu [37]. In their method, the network weights are grouped into a single vector w that is updated in accordance with the EKF equations. The entries of the Jacobian matrix are calculated by back propagating the m output values fy 1 (t) y 2 (t) Delta Delta Delta ; y m (t)g through the network. An ....

S Singhal and L Wu. Training multilayer perceptrons with the extended Kalman algorithm. In D S Touretzky, editor, Advances in Neural Information Processing Systems, volume 1, pages 133--140, San Mateo, CA, 1988.

....the linear parameters. 5. Sequential (approximate) LSE only: The outputs of an adaptive network are linearized with respect to its parameters, and then the extended Kalman filter algorithm [21] is employed to update all parameters. This method has been proposed in the neural network literature [85], 84] 83] The choice of one of the above methods should be based on a trade off between computational complexity and performance. Moreover, the whole concept of fitting data to parameterized models is called regression in statistics literature, and there are a number of other techniques for ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended kalman algorithm. In David S. Touretzky, editor, Advances in neural information processing systems I, pages 133-- 140. Morgan Kaufmann Publishers, 1989.

.... whereas conjugate gradient methods are among the fastest [16, 17, 67, 143, 170] Other heuristic methods that have fast learning speed include methods that learn layer by layer [70] iterative methods [9] hybrid learning algorithms [97] and methods developed from the field of optimal filtering [213, 233]. Recurrent neural networks have also been trained by gradient based methods [26, 179,197,274] Local minimization algorithms have difficulties when the surface is flat (gradient close to zero) when gradients can be in a large range, or when the surface is very rugged. When gradients can vary ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman algorithm. In D. Z. Anderson, editor, Proc. Neural Information Processing Systems, pages 133--140, New York, 1988. American Inst. of Physics.

....The E.K.F. algorithm might thus diverge, whereas the P.E. method will always provide the best predictor of a given arbitrary structure. As a consequence, P.E. methods are preferred for the training of neural networks. For a presentation of E.K.F. methods applied to neural network training, see [Singhal and Wu 1989, Matthews and Moschytz 90] In the framework of a P.E. method, a candidate network is trained by minimizing its MSPE on input output training sequences. Since we are concerned with timeinvariant models, the MSPE on the training sequences can be minimized in a non recursive, iterative fashion ....

Singhal S., Wu L. Training multilayer perceptrons with the extended Kalman algorithm, Advances in Neural Information Processing Systems I (Morgan Kaufmann, San Mateo 1989), pp. 133-140.

....immediate availability of confidence intervals and of mixing coefficients, required to generate mixtures of models, has motivated us to train neural networks with the EKF algorithm. 4. 3 Training MLPs with the EKF One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu (Singhal and Wu 1988). In their method, the network weights are grouped into a single vector w that is updated in accordance with the EKF equations. The entries of the Jacobian matrix are calculated by back propagating the m output values fy 1 (t) y 2 (t) Delta Delta Delta ; ym (t)g through the network. An ....

.... from the adaptive estimation field, to improve the existing algorithms for training neural networks with the EKF algorithm (Kadirkamanathan and Niranjan 1992, Kadirkamanathan and Niranjan 1993, Puskorius and Feldkamp 1991, Puskorius and Feldkamp 1994, Puskorius et al. 1996, Shah et al. 1992, Singhal and Wu 1988, Williams 1992) We achieve this in a principled manner by adhering to a hierarchical Bayesian methodology. In doing so, we are able to place some of the heuristic algorithms in the estimation field within a proper theoretical framework. Furthermore, this framework serves to unify important ....

Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133--140.

.... of optimal parameters; both on line and off line learning paradigms were developed and reported in [3] Moreover, other advanced techniques in nonlinear regression and optimization, such as the Gauss Newton method, the Levenberg Marquardt method [8, 10] and the extended Kalman filter algorithm [14, 12] can also be applied here directly. The original ANFIS C codes and several examples can be retrieved via anonymous ftp in user ai areas fuzzy systems anfis at ftp.cs.cmu.edu (CMU Artificial Intelligence Repository) For MATLAB users, ANFIS is also available in the Fuzzy Logic Toolbox used with ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended kalman algorithm. In D. S. Touretzky, editor, Advances in neural information processing systems I, pages 133--140. Morgan Kaufmann, San Mateo, CA, 1989.

....used to determine the contribution of each estimator to the committee. In addition, the parameter covariances serve the purpose of placing confidence intervals on the output prediction. 3 TRAINING MLPs WITH THE EKF One of the earliest implementations of EKF trained MLPs is due to Singhal and Wu (Singhal and Wu 1988). In their method, the network weights are grouped into a single vector w that is updated in accordance with the EKF equations. The entries of the Jacobian matrix are calculated by back propagating the m output values through the network. The algorithm proposed by Singhal and Wu requires a ....

Singhal, S. and Wu, L. (1988). Training multilayer perceptrons with the extended Kalman algorithm, in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems, Vol. 1, San Mateo, CA, pp. 133--140.

....a sequential training strategy to deal with non stationarity in signals, so that information from the recent past is lent more credence than information from the distant past. One way to sequentially estimate neural network models is to use a state space formulation and the extended Kalman filter [7]. This involves local linearisation of the output equation, which can be easily performed, since we only need the derivatives of the output with respect to the unknown parameters. This approach has been employed by several authors, including ourselves. Recently, we demonstrated a number of ....

S Singhal and L Wu. Training multilayer perceptrons with the extended Kalman algorithm. In D S Touretzky, editor, Advances in Neural Information Processing Systems, volume 1, pages 133--140, San Mateo, CA, 1988.

....of output error with respect to network activity, is Rohwer s (1990) moving targets method. Recently, several authors have noted that the extended Kalman filter (EKF) well known in engineering circles, can also be used for the purpose of training networks to perform desired inputoutput mappings. Singhal and Wu (1989) and Puskorius and Feldkamp (1991) have successfully applied the EKF to feedforward network problems, and Matthews (1990) has studied its use in recurrent networks. This paper focuses on the use of the EKF as a recurrent network learning algorithm and examines its relationship to the simpler and ....

.... 3 Application of the EKF to Feedforward Networks Although the main focus of this report is on the use of the EKF as a recurrent network learning algorithm, it is useful to note here that it can be applied to the problem of finding the weights in a feedforward network in supervised learning tasks (Singhal and Wu, 1989; Puskorius Feldkamp, 1991) For this application, one treats the unknown weight matrix W as the state vector to be estimated, and the desired output vector plays the role of the measurement vector in the EKF formulation. In this case, the unknown weights are assumed to undergo the trivial ....

[Article contains additional citation context not shown here]

Singhal, S. & Wu, L. (1989). Training multilayer perceptrons with the extended Kalman filter algorithm. In D. S. Touretzky, Ed. Advances in Neural Information Processing Systems, 1, 133-140. San Mateo, CA: Morgan Kaufmann.

....theory is actually not intended. However, since the structural properties of the network, which has to be trained are well known, one takes advantage of the approximation ability of neural nets and directly applies control theory methods as learning algorithms. For this purpose, Singhal and Wu [SW89] suggested the use of the Extended Kalman Filter algorithm from the estimation theory for training of the nets. 5.1 State Model of a Neural Network The process behaviour is characterized by a state model with a transition and an observation function, cf. sec. 2.1. In case of a neural net, the ....

....the gradient information also the the dependencies among the weights and the estimation error of the weight parameters. Although this method is computationally complex, it yields a speed up in training time measured in number of pattern set presentation and obtains more acurate solutions, cf. [SW89]. Since the algorithm by Singhal and Wu considers the dependencies of all the weights with each other for the adaption of a single weight in the network, the algorithm is called the Global Extended Kalman Filter (GEKF) Other proposals consider only local, i.e. node level, dependencies, FPDY92, ....

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman algorithm. In D. Touretzky, editor, Advances in Neural Information Processing Systems I, pages 133--140. Morgan Kaufman, 1989.

....and 100 input nodes in a tapped delay line configuration, with 250 Hz sampling. Extended Kalman filtering (EKF) was used for learning in the neural network. EKF is an extension of the familiar Kalman filter [16] to deal with nonlinear systems via linearization about the current parameter estimates [17]. When EKF is used to train a neural network, learning is viewed as an identification problem for a nonlinear dynamic system [17] The neural network weights represent the state of the nonlinear system. The EKF theory is then used to derive a recursion for the weight updates [17] This method ....

....in the neural network. EKF is an extension of the familiar Kalman filter [16] to deal with nonlinear systems via linearization about the current parameter estimates [17] When EKF is used to train a neural network, learning is viewed as an identification problem for a nonlinear dynamic system [17]. The neural network weights represent the state of the nonlinear system. The EKF theory is then used to derive a recursion for the weight updates [17] This method offers greatly improved convergence results over backpropagation, but the computational load is considerable. The variation of EKF ....

[Article contains additional citation context not shown here]

S. Singhal and L. Wu, "Training multilayer perceptrons with the extended Kalman algorithm," in D. S. Touretzky, ed., Advances in Neural Information Processing Systems 1, Los Altos, Ca.: Morgan Kauffman, pp. 133140, 1989.

No context found.

S. Singhal and L. A. Wu, "Training multilayer perceptrons with the extended Kalman algorithm," Advances in Neural Information Processing Systems 1, pp. 133--140, San Mateo, CA: Morgan Kaufmann 1989.

No context found.

S. Singhal and L. Wu, "Training multilayer perceptrons with the extended Kalman filter," in Advances in Neural Information Processing Systems 1, San Mateo, CA, 1989, pp. 133--140.

No context found.

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman lter. In Advances in Neural Information Processing Systems 1, pages 133-140, San Mateo, CA, 1989. Morgan Kauman.

No context found.

S. Singhal and L. Wu. Training multilayer perceptrons with the extended Kalman algorithm. In D. S. Touretzky, editor, Advances in Neural Information Processing Systems - NIPS 1, pages 133--140, San Mateo, 1989. Morgan Kau#man.

No context found.

S. Singhal and L. Wu, "Training multilayer perceptrons with the extended Kalman filter algorithm," in Advances in Neural Information Processing Systems, 1, D. S. Touretzky, Ed., pp. 133--140. Morgan Kaufmann, San Mateo, CA, 1989.

No context found.

Singhal, S. and Wu, L. (1989). Training Multi-layer Perceptrons with the Extended Kalman Algorithm. In: Touretzky, D. S., ed., Advances in Neural Information Processing Systems 1, Morgan Kaufman, pp. 133-140.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC