C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1: 995--1019, 1987.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....1 every node looks equally like an ancestor, and all observable labels will be the same. In our algorithm, labeled data are constant sources that push out labels, and the system achieves equilibrium when t 1. There seems to be a resemblance between label propagation and mean field approximation [9] [10] In label propagation, upon convergence we have the equations (for unlabeled data) Y ic = 6) Consider the labeled unlabeled data graph as a conditional Markov random field F with pairwise interaction w ij between nodes i; j, and with labeled nodes clamped. Each unclamped ....

Carsten Peterson and James R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995--1019, 1987.

....one may find that a correlation is definitely between 0:4 and 0:6. An ordinary approximation might be more accurate, but in practical situations there is absolutely no warranty for that. The best known bound is probably the mean field bound, which has been described for Boltzmann machines in [1] and later for sigmoid belief networks in [2] Apart from its bounding properties, mean field theory is a commonly used approximation technique as well. Recently this first order bound was extended to a third order approximation for Boltzmann machines and sigmoid belief networks in [3] and [4] ....

C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex systems, 1:995--1019, 1987.

....one may find that a correlation is definitely between 0:4 and 0:6. An ordinary approximation might be more accurate, but in practical situations there is absolutely no warranty for that. The best known bound is probably the mean field bound, which has been described for Boltzmann machines in [1] and later for sigmoid belief networks in [2] Apart from its bounding properties, mean field theory is a commonly used approximation technique as well. Recently this first order bound was extended to a third order approximation for Boltzmann machines and sigmoid belief networks in [3] and [4] ....

C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex systems, 1:995--1019, 1987.

....an existing one, which is applied to a Boltzmann machine in section 3. Boltzmann machines are another type of graphical models. In contrast with belief networks the connections are symmetric and not directed [4] A mean field approximation for this type of neural networks was already described in [5]. An improvement of this approximation was found by Thouless, Anderson and Palmer in [6] which was applied to Boltzmann machines in [7] Unfortunately, this so called tap approximation is not a bound. We apply our method to the mean field approximation, which results in a third order bound. We ....

C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex systems, 1:995--1019, 1987.

....Z requires an exponential summation operation. Thus exact calculation of Z is intractable. Sampling based methods offer a possible way out. But these methods are computationally very expensive. Another alternative is to look for deterministic approaches, based on mean field theory as advocated by (Peterson Anderson, 1987; Saul et al. 1996; Kappen Rodriguez, 1998) The methods proposed by (Peterson Anderson, 1987; Kappen Rodriguez, 1998) are applicable to BMs, while Saul et al. 1996) developed a scheme which has been applied to both BNs and BMs. The next section will be devoted to Plefka s method. We intend ....

....based methods offer a possible way out. But these methods are computationally very expensive. Another alternative is to look for deterministic approaches, based on mean field theory as advocated by (Peterson Anderson, 1987; Saul et al. 1996; Kappen Rodriguez, 1998) The methods proposed by (Peterson Anderson, 1987; Kappen Rodriguez, 1998) are applicable to BMs, while Saul et al. 1996) developed a scheme which has been applied to both BNs and BMs. The next section will be devoted to Plefka s method. We intend to study it, understand its relations with existing theories and apply it to BNs. 93 Chiru ....

Peterson, C., & Anderson, J. R. (1987). A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995--1019.

....Moreover, at every run, we have to wait until the Markov chain has reached equilibrium, and many independent samples are produced. 2 Mean Field Boltzmann Machines An alternative to the slow Gibbs sampling is to approximate the averages using a fully factorized, mean field (MF) distribution (Peterson Anderson, 1987). Q(s) Y i m s i i (1 m i ) 1 s i (10) where the product is taken over all units s i . This MF distribution has one free parameter per unit, m i , describing the probability that this unit will be on . These parameters will be chosen such that the approximating distribution, Q, is as ....

Peterson, C.& Anderson, J. (1987). A Mean Field Theory Learning Algorithm for Neural Networks.

....distribution Q(T : that has tractable structure. Using this approximating distribution, we can define a lower bound on the likelihood P (o) The parameters that define Q are adapted by trying to maximize this lower bound. The simplest variational approximation is the mean field approximation [8, 9] that approximates the posterior distribution with a network in which all the random variables are independent. As such, it is unsuitable when there are strong dependencies in the posterior. Saul and Jordan [10] suggest to circumvent this problem by using structured variational approximation. This ....

C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995--1019, 1987.

....by Pineda [Pineda, 1988] The major disadvantage of this algorithm is that it requires an extensive amount of computation at each iteration. Another algorithm which has been shown to descend an error function and has been applied to RNNs is the deterministic Boltsman Machine learning rule [Peterson and Anderson, 1987]. Unlike backpropagation based learning which requires a different formulation for discrete and continuous time networks the Boltsman Machine learning rule can be applied equally well to both types of RNNs. The Boltsman Machine learning rule treats the system error of the RNN as an energy function ....

Peterson, C. & Anderson, J. (1987). A mean field theory learning algorithm for neural networks. Complex Systems, 1.

....by the parameters, and in principle exhaustive computation will give the values. However, the problem arises from the fact that exhaustive computation often requires huge amount of computational resources, and therefore some approximation scheme is needed in practice. Mean field approximation[1] provides a practically useful framework to the problem. The currently common way of formulating mean field approximation is that based on the variational principle: mean field approximation gives a factorized probability distribution as an approximation to the target probability distribution. ....

C. Peterson and J. R. Anderson, "A mean field theory learning algorithm for neural networks," Complex Syst., vol. 1, pp. 995--1019, 1987.

.... signals in an interactive network are naturally propagated (even through hidden layers) in just the right way to enable the correct error gradient to be simply and locally computed at each unit [25] The GeneRec analysis also showed that Boltzmann machine learning and its deterministic versions [19, 27, 28, 29] can be seen as variants of this more biologically plausible version of the backpropagation algorithm. This means that all of the existing approaches to error driven learning using activation based signals converge on essentially the same basic mechanism, making it more plausible that this is the ....

C. Peterson and J. R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995--1019, 1987.

.... The probabilistic semantics of these networks (Lauritzen, 1996; Neal, 1992; Pearl, 1988) differ in useful respects from those of symmetric neural networks, for which the attractor paradigm was first established (Cohen Grossberg, 1983; Hopfield, 1982; Geman Geman, 1984; Ackley et al. 1985; Peterson Anderson, 1987). Borrowing ideas from statistical mechanics, we have derived a mean field theory for approximate probabilistic inference. We have also exhibited an attractor dynamics that converges to solutions of the mean field equations and that generates the signals required for unsupervised learning. While ....

....dynamics to refine these guesses. Even without such enhancements, however, we believe that the attractor paradigm in directed graphical models is worthy of further investigation. Attractor neural networks have provided a viable approach to probabilistic inference in undirected graphical models (Peterson Anderson, 1987), particularly when combined with deterministic annealing. We attribute the lack of learning based applications for symmetric neural networks to their representational limitations for modeling causal processes (Pearl, 1988) and the peculiar instabilities arising from the sleep phase of Boltzmann ....

Peterson, C., & Anderson, J. R. (1987). A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995--1019.

....by a demonstration of the similarity between systems developed with alternative procedures. In this chapter, we replicate and extend the H S results using the more plausible gradient descent procedure of contrastive Hebbian learning in a deterministic Boltzmann Machine (DBM, Hinton, 1989; Peterson Anderson, 1987). The architecture of the network is broadly similar to the back propagation networks that have a phonological output system, except that there are no separate clean up units. There is also full connectivity within each of the orthographic, semantic, and phonological layers, and all sets of ....

Peterson, C. & Anderson, J. R. (1987). A mean field theory learning algorithm for neural nets. Complex Systems, 1:995--1019.

....the learning procedure, or the way responses are generated from semantic activity. A particularly relevant simulation in this regard involved an implementation of the full semantic pathway mapping orthography to phonology via semantics using a deterministic Boltzmann Machine (Hinton, 1989b; Peterson Anderson, 1987). Lesions throughout the network gave rise to both visual and semantic errors, with lesions prior to semantics producing a bias towards visual errors and lesions after semantics producing a bias towards semantic errors. Thus, the network replicated both the qualitative similarity and quantitative ....

Peterson, C., & Anderson, J. R. (1987). A mean field theory learning algorithm for neural nets. Complex Systems, 1, 995-- 1019.

....of the similarity between systems developed with alternative procedures. 6 In this section, we replicate the main results obtained thus far with back propagation, within the more plausible learning framework of contrastive Hebbian learning (CHL) in a deterministic Boltzmann Machine (DBM) [PA87, Hin89b]. In this framework, weights are changed in proportion to the difference in the product of unit states after settling with both inputs and outputs are clamped (the positive phase) and when settling after only the inputs are clamped (the negative phase) CHL is somewhat more biologically plausible ....

Carsten Peterson and James R. Anderson. A mean field theory learning algorithm for neural nets. Complex Systems, 1:995--1019, 1987. Word Reading in Damaged Connectionist Networks 24

....problem. How difficult it is to solve a particular problem depends on a statistical model one employs in solving the problem. For Boltzmann machines[1] for example, it is computationally very hard to evaluate expectations of state variables from the model parameters. Mean field approximation[2], which is originated in statistical physics, has been frequently used in practical situations in order to circumvent this difficulty. In the context of statistical physics several advanced theories have been known, such as the TAP approach[3] linear response theorem[4] and so on. For neural ....

Peterson, C., and Anderson, J. R. (1987) A mean field theory learning algorithm for neural networks. Complex Systems 1: 995--1019.

No context found.

C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1: 995--1019, 1987.

No context found.

C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995-- 1019, 1987.

No context found.

C. Peterson and J. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995-- 1019, 1987.

No context found.

C. Peterson and J.R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems, 1:995-- 1019, 1987.

No context found.

C. Peterson and J. R. Anderson, "A mean field theory learning algorithm for neural networks," Complex Syst., vol. 1, pp. 995--1019, 1987.

No context found.

C. Peterson and J. R. Anderson, "A mean field theory learning algorithm for neural nets," Complex Systems, vol. 1, pp. 995--1019, 1987.

No context found.

C. Peterson and J. R. Anderson, "A mean field theory learning algorithm for neural nets," Complex Systems, vol. 1, pp. 995--1019, 1987.

No context found.

Peterson, C.& Anderson, J. (1987). A Mean Field Theory Learning Algorithm for Neural Networks. Complex Systems, 1, 995--1019.

No context found.

C. Peterson and J. R. Anderson. A mean field theory learning algorithm for neural networks. Complex Systems 1:995--1019 (1987).

No context found.

C. Peterson, & J. R. Anderson. "A mean field theory learning algorithm for neural networks." Complex Systems, 1, 995-1019, 1987.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC