Nowlan, S. J. (1991). Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. CMU-CS-91-126, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....accounting for a single example. Models of this form are sometimes referred to as factorial, and have recently been considered in the literature. Dayan and Zemel [44] consider multiple cause models that learn cooperation between inputs to generate targets. Cooperation was also considered by Nowlan [159] and Jacobs [98] in the original form of the mixtures of experts, but not in the context of multiple causes. Experimental analyses by both Nowlan and Jacobs showed that competitive training was more effective than cooperative training for mixtures of experts. Factorial learning has also been ....

.... has also been considered by Fritsch [60] Other researchers have found the localised ME model to be successful, including Ramamurti and Ghosh [187] Classification using mixtures of experts Classification was considered in the original forms of the mixtures of experts model by Nowlan [159] and Jacobs, Jordan, Nowlan and Hinton [104] who applied the model to the Peterson Barney vowel data. 2.4. REVIEW OF RELATED LITERATURE 31 On this data they found that the model gave good predictive performance, outperforming an MLP in empirical comparisons [160] but also gave intuitive ....

Nowlan, S. J. [1991], Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures, PhD thesis, Carnegie Mellon University. CS--91--126.

....first suited for the job, this requires too much interaction with the billing system to be used in practice. To avoid this burdensome processing of data, we formulate our partial estimation procedure using on line estimation. The on line version of the EM algorithm was first introduced by Nowlan [7]. # ### ### # ## ### ### # # ##### (3) Remembering that the new maximum likelihood estimate for # ### is computed as the expected value of # ##### over the whole data set with the current parameter fit, we can easily formulate a recursive estimator for this expected value as can be seen in ....

S.J. Nowlan. Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, 1991.

....in the appropriate limit, to partitioning by dimension reduction error. These systems neither design transforms nor partition the signal space with the goal of minimizing compression distortion. This ad hoc construction contrasts sharply with the solid grounding of vector quantization. Nowlan [8] develops a probabilistic framework for VQ by demonstrating the correspondence between a VQ and a mixture of spherically symmetric Gaussians. In the limit that the mixture component variance goes to zero, the ExpectationMaximization (EM) procedure for tting the mixture model to data becomes ....

Steve Nowlan. Soft Competitive Adaptation: neural network learning algorithms based on tting statistical mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

....last two techniques are optimal for dimensionreduction, but not for coding. These systems neither design transforms nor partition the signal space with the goal of minimizing compression distortion. This ad hoc construction contrasts sharply with the solid grounding of vector quantization. Nowlan [9] develops a probabilistic framework for VQ by demonstrating the correspondence between a VQ and a mixture of spherically symmetric Gaussians. In the limit that the mixture component variance goes to zero, the ExpectationMaximization (EM) procedure for fitting the mixture model to data becomes ....

Steve Nowlan. Soft Competitive Adaptation: neural network learning algorithms based on fitting statistical mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

....bit allocation, entropy constrained quantizer design, and lossless entropy coder design, into a single framework. Adaptive transform coding should be grounded in a compelling probabilistic framework. This 17 framework currently exists for vector quantizers, but not for transform coders. Nowlan [18] develops the correspondence between vector quantizers and a mixture of Gaussians probability model in his thesis. In the limit where the mixture component variances approach zero, 1) the least squares estimators for the VQ parameters are the same as the maximum likelihood estimators for the ....

Steve Nowlan. Soft Competitive Adaptation: neural network learning algorithms based on tting statistical mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991. 19

....in the appropriate limit, to partitioning by dimension reduction error. These systems neither design transforms nor partition the signal space with the goal of minimizing compression distortion. This ad hoc construction contrasts sharply with the solid grounding of vector quantization. Nowlan [8] develops a probabilistic framework for VQ by demonstrating the correspondence between a VQ and a mixture of spherically symmetric Gaussians. In the limit that the mixture component variance goes to zero, the Expectation Maximization (EM) procedure for fitting the mixture model to data becomes ....

Steve Nowlan. Soft Competitive Adaptation: neural network learning algorithms based on fitting statistical mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

....The recent emphasis in the neural network literature on probabilistic models has led to increased interest in EM as a possible alternative to gradient based methods for optimization. EM has been used for variations on the traditional theme of Gaussian mixture modeling (Ghahramani Jordan, 1994; Nowlan, 1991; Xu Jordan, 1993a, b; Tresp, Ahmad Neuneier, 1994; Xu, Jordan Hinton, 1994) and has also been used for novel chain structured and treestructured architectures (Bengio Frasconi, 1995; Jordan Jacobs, 1994) The empirical results reported in these papers suggest that EM has considerable ....

Nowlan, S.J. (1991). Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures. Tech. Rep. CMU-CS-91-126, CMU, Pittsburgh, PA.

....with adaptive optimization approaches, the algorithm sometimes falls into local maxima. 1.4.2 Mixture Models Another natural extension is to consider mixtures of curved Gaussians. One of the most popular extensions of standard Gaussian models is to mixture models such as the mixture of Gaussians [Nowlan 1991], and in more recent work, mixtures of principal component or factor analyzers [Bregler and Omohundro 1995, Kambhatla and Leen 1997, Hinton et al. 1997, Roweis and Ghahramani 1999, Tipping 1997] Such mixture models are attractive because the expectation maximization algorithm (EM; Dempster et al. ....

Nowlan, S. J. 1991. Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures. Technical Report PhD Thesis, Department of Computer Science, Carnegie--Mellon University.

....is chosen. 2. Non parametric Methods. When no such assumptions can be done, the densities need be estimated directly from the data. These are also known as kernel based estimators [12, 45, 46] 3. Semi parametric Methods. The densities are written as a mixture model whose parameters are estimated [12, 40, 50, 36, 48]. In the case of normal mixtures, this approach is equivalent to cluster based classification strategies like LVQ of Kohonen [24] and is similar to Gaussian radial basis function networks [32] 5 A decision rule as given in Eq. 1) has the effect of dividing the input space into mutually ....

....instead there may be several. For example in character recognition while writing 7 one prototype may be a seven with a horizontal middle bar (European version) and one without (American version) A mixture density defines the class conditional density as a sum of a small number of densities [12, 36, 31]: p(xjC j ) h j X h=1 p(xj jh ; C j )P ( jh ) 13) 10 where the conditional densities p(xj jh ; C j ) are called the component densities and the prior probabilities P ( jh ) are called the mixing parameters. Note that here we have one mixture model for each class leading to an ....

[Article contains additional citation context not shown here]

Nowlan, S.J. (1991) Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures, PhD Thesis, School of Computer Science, Carnegie Mellon University.

....the codebook vectors so that they define good classification boundaries. Unsupervised methods of moving the codebook vectors are the Competitive learning and related Kohonen feature mapping [Grossberg, 1976, Kohonen, 1982, Rumelhart and Zipser, 1986] and soft Competitive learning[Nowlan, 1990, Nowlan, 1991] algorithms. In Competitive learning the codebook vectors move to minimize the distance from every input pattern to their closest codebook vector. This tends to move the codebook vectors to the centres of clusters. Kohonen feature mapping is similar except that the codebook vectors have specified ....

Nowlan, S. J. (1991). Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University.

....interrupt latency, and lock contention. MS Manners employs exponential averaging of sufficient statistics in its target calibration. This is a common technique in the discipline of artificial intelligence, used in various contexts by, for example, Spiegelhalter and Lauritzen [25] and Nowlan [18]. 11. Future work Future work should focus on addressing the limitations of progress based regulation, such as the assumption of symmetric performance impact from resource contention. Since this is the primary assumption of the technique, it seems an especially hard requirement to remove. ....

S. J. Nowlan. Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures, Ph.D. thesis, Carnegie Mellon University. CS-91-126, 1991.

....is made, resulting in many Markov chain runs. In practice, the chain converges very quickly, as the models change only slowly with each incoming observation. We are currently investigating the use of online EM, where only a single iteration is performed for each new available data point. Nowlan [ 1991 ] has proved that this approach should lead to locally maximum likelihood estimates in the limit, and our preliminary experimental results are encouraging. 4 Experimental results We have performed two experiments comparing our approach to that of Huang and Russell. Our data sets were created ....

S. J. Nowlan. Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

....of expert mappings and positions, experts are placed in the input space in such a way so as to minimize error. We discuss this in more detail in Section C. B. Competing Learners First proposed by Jacobs et al. 6] a measure that forces competition is to view the architecture as a mixture model [14]. The gating values are the mixture proportions and the expert perceptron outputs are the means. If we take gaussian components with equal variances, the likelihood of the sample point x is given as: E = log X r g r exp 2 4 X j y j log O jr 3 5 (13) Eq. 13) forces experts to compete ....

S. J. Nowlan, Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures, PhD Thesis, School of Computer Science, Carnegie Mellon University, 1990.

....of the variables is re calculated, it makes sense to immediately re estimate the parameters before performing the E step for the next unobserved variable, as this utilizes the new information immediately, speeding convergence. An incremental algorithm along these general lines was investigated by Nowlan (1991). However, such incremental variants of the EM algorithm have not previously received any formal justification. We present here a view of the EM algorithm in which it is seen as maximizing a joint function of the parameters and of the distribution over the unobserved variables that is analogous to ....

....be avoided in several ways one could use a fixed point representation of e s, in which addition and subtraction is exact, for example, or recompute e s non incrementally at infrequent intervals. An incremental variant of the EM algorithm somewhat similar to that of (9) was investigated by Nowlan (1991). His variant does not maintain strictly accurate sufficient statistics, however. Rather, it uses statistics computed as an exponentially decaying average of recently visited data points, with iterations of the following form: E Step: Select the next data item, i, for updating. Set e s (t) i = ....

Nowlan, S. J. (1991) Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures, Ph. D. thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh.

....all the images we have observed. This is clearly impractical for our application. Moreover, batch processing of the complete image sequence is not possible in a real time setting. We now describe an incremental variant of EM that does not require storing the data. This procedure was introduced by Nowlan [1991], and is best understood in terms of the results of Neal and Hinton [1993] Neal and Hinton show that we can think of the EM process as continually adjusting the sufficient statistics. In this view, on each iteration when we process an instance, we remove its previous contribution to the sum and ....

S. J. Nowlan. Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

No context found.

Nowlan, S. J. (1991). Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. CMU-CS-91-126, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.

No context found.

Nowlan, S.J. (1991). Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures. Tech. Rep. CMU-CS-91-126, CMU, Pittsburgh, PA.

No context found.

Nowlan, S. J. (1991). Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. CMU-CS-91-126, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA.

No context found.

Nowlan, S.J. 1991. "Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures". PhD Dissertation. Carnegie-Mellon University.

No context found.

S. J. Nowlan. Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, 1991.

No context found.

S. Nowlan, "Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures," Ph.D. dissertation, School of Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, 1991.

No context found.

Nowlan S.J. (1991): Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh.

No context found.

Nowlan, SJ (1991). Soft Competitive Adaptation: Neural Network Learning Algorithms based on Fitting Statistical Mixtures. CMU Technical Report CMU-CS-91126, Carnegie-Mellon University, Pittsburgh PA.

No context found.

Nowlan, S. J. (1991). Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures. CMU Technical Report, CS-91-126, Pittsburgh: Carnegie Mellon University.

No context found.

Nowlan, S. J. (1991). Soft competitive adaptation: neural network learning algorithms based on fitting statistical mixtures. CMU Technical Report, CS-91-126, Pittsburgh: Carnegie Mellon University.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC