| C. Rasmussen, "The infinite Gaussian mixture model," NIPS 12, 2000. |
....are far more computationally intensive than EM. MCMC can be used in two different ways: to implement model selection criteria to actually estimate k (e.g. 29] 30] 31] or in a more fully Bayesian way, to sample from the full a posteriori distribution with k considered unknown [32] 33] [34]. Despite their formal appeal, we think that MCMC based techniques are still far too computationally demanding to be useful in pattern recognition applications. For example, tests reported in [33] using small samples (n = 245, n = 155, and n = 82) of univariate data, require 100; 000 MCMC sweeps ....
C. Rasmussen, "The infinite Gaussian mixture model," in Advances in Neural Information Processing Systems 12 (S. Solla, T. Leen, and K.-R. Muller, eds.), pp. 554--560, MIT Press, 2000.
....instead we use the theory of Dirichlet processes (DPs) 2, 1] to implicitly integrate them out, leaving just three hyperparameters defining the prior over transition dynamics. The idea of using DPs to define mixture models with infinite number of components has been previously explored in [5] and [7]. This simple form of the DP turns out to be inadequate for HMMs. Because of this we have extended the notion of a DP to a two stage hierarchical process which couples transitions between different states. It should be stressed that Dirichlet distributions have been used extensively both as ....
C. E. Rasmussen. The infinite Gaussian mixture model. In Advances in Neural Information Processing Systems 12, Cambridge, MA, 2000. MIT Press.
....Process is not input dependent, but we will modify it to serve as a gating mechanism. We start from a symmetric Dirichlet distribution on proportions: p( 1 ; k j ) Dirichlet( k) k) k Y j =k 1 j ; where is the (positive) concentration parameter. It can be shown [Rasmussen, 2000] that the conditional probability of a single indicator when integrating over the j variables and letting k tend to infinity is given by: components where n i;j 0: p(c i = jjc i ; n i;j n 1 ; all other components combined: p(c i 6= c i 0 for all i 0 6= ijc i ; n 1 ....
....nmax data points assigned to it. This is easily implemented 1 by modifying the conditionals in the Gibbs sampler. The hyperparameter controls the prior probability of assigning a data point to a new expert, and therefore influences the total number of experts used to model the data. As in Rasmussen [2000], we give a vague inverse gamma prior to , and sample from its posterior using Adaptive Rejection Sampling (ARS) Gilks Wild, 1992] Allowing to vary makes it possible for the model more freely to infer the number of GPs to use for a particular dataset. 4 The Algorithm The individual GP ....
Rasmussen, C. E. (2000). The Infinite Gaussian Mixture Model, NIPS 12, S.A. Solla, T.K. Leen and K.-R. Muller (eds.), pp. 554--560, MIT Press.
....may want to select one as an approximation. 2 For some models, the limit of an infinite number of parameters is a simple model which can be treated tractably. Two examples are the Gaussian Process limit of Bayesian neural networks [Neal, 1996] and the infinite limit of Gaussian mixture models [Rasmussen, 2000]. too simple too complex just right All possible data sets P(Y M i ) Y Figure 1: Left panel: the evidence as a function of an abstract one dimensional representation of all possible datasets. Because the evidence must normalize , very complex models which can account for many datasets ....
Rasmussen, C. E. (2000) The Infinite Gaussian Mixture Model, in S. A. Solla, T. K. Leen and K.-R. Muller (editors.), Adv. Neur. Inf. Proc. Sys. 12, MIT Press, pp. 554--560.
No context found.
C. Rasmussen, "The infinite Gaussian mixture model," NIPS 12, 2000.
No context found.
C. Rasmussen, "The infinite Gaussian mixture model," in Advances in Neural Information Processing Systems 12. Cambridge, MA: MIT Press, 2000.
No context found.
Rasmussen, C. (2000), "The Infinite Gaussian Mixture Model," in S. Solla, T. Leen, and K.-R. M uller (eds.) Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, vol. 12.
No context found.
C.E. Rasmussen. The infinite Gaussian mixture model. In NIPS, volume 12, 2000.
No context found.
Rasmussen, C. (2000), "The Infinite Gaussian Mixture Model," in S. Solla, T. Leen, and K.-R. M uller (eds.) Advances in Neural Information Processing Systems, Cambridge, MA: MIT Press, vol. 12.
No context found.
Rasmussen, C. E. 2000. The infinite Gaussian mixture model. In NIPS 12.
No context found.
C. Rasmussen. The infinite Gaussian mixture model. In Advances in Neural Information Processing Systems 12. MIT Press, Cambridge, MA, 2000.
No context found.
C. E. Rasmussen. The infinite Gaussian mixture model. In Solla et al.
No context found.
C.E. Rasmussen, The Infinite Gaussian Mixture Model, in Advances in Neural Information Processing Systems, 14, 2000.
No context found.
Rasmussen, C.E. (2000) The Infinite Gaussian Mixture Model, in S.A. Solla, T.K. Leen K.-R. Muller (eds.) Advances in NIPS 12, MIT Press, pp. 554--560.
Online articles have much greater impact More about CiteSeer.IST Add search form to your site Submit documents Feedback
CiteSeer.IST - Copyright Penn State and NEC