Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. R. Statist. Soc. 56 (1994) 363--375  Home/Search   Document Not in Database   Summary   Related Articles   Check

....is quite different in its principle. The vectors Xt, Kt and rt are considered to be random variables with prior densities. Samples are then obtained iteratively from their joint posterior using a proper MCMC technique, namely the Gibbs Sampler. This method has been studied in [32] 33] 34] [35] or [36] for instance. It can be run sequentially at each time period. Gibbs Sampler is a special case of the Metropolis Hasting algorithm with the proposal densities being the conditional distributions, and the acceptance probability being consequently always equal to one. The interested reader ....

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363-375, 1994.

....belonging to the parameter space. The choice of the conjugate prior of the likelihood of the complete data as penalization term conducted to explicit EM algorithm re estimation formulas. While the role of conjugate priors is acknowledged in Bayesian sampling schemes, including in mixture problems [13], putting forward the link between conjugate priors and explicit penalized EM schemes is an original contribution, as far as we know. Numerical examples put in evidence the existence of the singularities and the e#ciency of the penalized solution. Concerning the asymptotic behavior of the ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, 56, (2), pp. 363--375, 1994.

.... P k is Gaussian, due to the Gaussianity of the error: 8 k = 1 p; P rob exp B C A We sample the new augmented variables t from the density p( Xj ; X 1 N ) This density is taken as a multinomial distribution M, usually used in mixing distribution problems [31]. t p X t jX t ; a ; b ; c = M(1; ff 1 ; ff p ) 4) with fi fi fi fi fi fi fi fi ff k P rob 8 k = 1 : p t [k] 1 (5) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the ....

....: p t [k] 1 (5) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the global augmented model is the derivation of the weights ff k . For more details on indicator variables in mixture or switching models, we refer to [31]. 2.3 Posterior step The second step of our process (given in eq. 3) deals with the sampling of the conditional posterior densities of the parameters. We start from the global posterior density derived from the classical Bayes rule: jX 1 N ; X 1 N ja; b; c; oe ; 6) ....

J. Diebolt and C.P. Robert. Estimation of finite mixture distributions through bayesian sampling. J. Royal Stat. Soc., 56(2):363--375, 1994. 23

....= 1 : p (3) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the global augmented model is the derivation of the weights ff k . For more details on indicator variables in mixture or switching models, we can refer to [8]. 3. SEM ALGORITHM FOR MODEL IDENTIFICATION With the description of the model made in the preceeding section, we now give the full conditional posterior densities of the parameters we want to estimate. We start from the global posterior density derived from the classical Bayes rule: X1 N ....

J. Diebolt and C.P. Robert. Estimation of finite mixture distributions through bayesian sampling. J. Royal Stat. Soc., 56(2):363--375, 1994.

....conditional mean estimates of the states and . The proposed conditional mean state estimator via the data augmentation algorithm is summarized in Fig. 1. Remark 3.1: Theoretically speaking, the DA algorithm does not have a stopping criterion. However, a reasonable choice (see, for example, [4]) is to terminate the algorithm when is less than some specified tolerance limit. Sampling Schemes: The DA algorithm presented in Fig. 1 requires us to compute samples from and . One possible scheme is the efficient forward filtering backward sampling recursions introduced by Carter and Kohn ....

....Thus, the sequence obtained from the DA sampling scheme is a Markov chain with invariant distribution and is ergodic. Ergodicity implies convergence of ergodic (sample) averages [23, Th. 3, p. 1717] Uniform ergodicity implies that the Law of Large Numbers and a central limit theorem also hold [4], 18] and [23, Th. 5, p. 1717] Corollary 3.1 Convergence of Ergodic Averages: For every real valued function , let us consider the time average of the first outputs of the Markov chain .If , then, for any initial distribution, 12) If , then a constant exists such that the distribution ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Stat. Soc. B, vol. 56, pp. 363--375, 1994.

....an infinite number of iterations to give the exact values of the MMSE and MMAP estimates. For all our simulations, we discard the first iterations to compute the MMSE estimates using the MCMC sampler. These first iterations correspond to the so called burn in period of the Markov chain. As in [9], the MCMC sampler algorithm is then iterated until the computed values of the ergodic averages are no longer modified. To ensure convergence toward the set of global Methods for determining the burn in period N are beyond the scope of this paper; see [29] for an overview of such methods. ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, vol. 56, pp. 363--375, 1994.

....we have the data y, a known relation between the unknown parameters a and y and finally the hyperparameters 1 and 0. The Bayesian estimation technique is now well established [1 7] and has been used since many years to resolve the inverse problems in signal and image reconstruction and restoration [10 14,17,18,20,21]. The first step before applying the Bayes rule is to assign the prior probability laws p(a 10) p(yla, fl) p(O) and p(fl) The next step is to determine the posterior laws and then to infer the unknowns. In this paper we are focusing more on the second step than on the first step. So we assume ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," Journal of Royal Statistical Society B, vol. 56, no. 2, pp. 363-375, 1994.

.... illustration of the Bayesian perspective on density estimation using Gaussian mixtures was recently provided by Roeder and Wasserman [15] The sampling approach to Bayesian inference in the context of Gaussian mixture models in the form used in this paper was first described by Diebolt [16]. An interesting extension of Bayesian sampling to cases where the number of Gaussian components is unknown has recently been suggested by Richardson and Green [17] Green [18] was also one of the first authors who used the EM algorithm for maximum penalized likelihood estimation. The first ....

....)d Theta: By using a conjugate prior p( Theta) we can obtain an analytically closed formulation of p(xjx ) Unfortunately, p(xjx ) is a sum of n m 1 terms and therefore is typically approximated. We use a stochastic approximation to p(xjx ) by employing the data augmentation method [16]. Data augmentation is an instantiation of Gibbs sampling, where one exploits the hierarchical structure of mixture models to generate a Markov chain ( Theta) t with stationary distribution p( Thetajx ) More specifically, one generates samples from the posterior of the parameters by the ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," Journal of the Royal Statistical Society B, vol. 56, no. 2, pp. 363--375, 1994.

....Tipping and Bishop (1999) proposed an MFA model with isotropic uniqueness matrices # k = # k I p , which is called the mixture of probabilistic PCA. Under these constraints, they have derived the ML estimation algorithms. 2. 2 Natural conjugate priors on parameters Many studies on mixture models (Diebolt and Robert, 1994) and factor analysis models (Press and Shigemasu, 1989) employ natural conjugate priors on their parameters, because such priors usually lead to simple Bayesian estimation algorithms. The natural conjugate prior on # is a Dirichlet distribution f(# #) D(# #) # m # k=1 # # 1 k (2.6) ....

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B, 56:363--375.

....same for all # to write the second equality in (9) It remains to estimate the association probabilities ## i t # i##; M , which can be seen as the stochastic coefficients of the ##component mixture. To estimate them we propose to use a Gibbs sampler whose principles are briefly recalled (see [1] or [9] for more details) For # ### t ## t # # t #, it consists in generating a Markov chain that converges to the stationary distribution ##### ##t # which cannot be sampled directly. Given a partition # # ###### P of #, one samples alternatively from the conditional posterior distribution ....

....the following simulations) # A subset of false measurements which number follows a Poisson distribution with mean ## . 3.3. 2 Results of the MTPF 3000 2500 2000 1500 1000 500 0 500 1000 1500 2000 1000 1000 2000 3000 4000 5000 6000 7000 y coordinate in meters x coordinate in meters tar[1] tar[3] tar[2] observer 1 2000 1500 1000 500 0 500 1000 1500 2000 1000 0 1000 2000 3000 4000 5000 6000 7000 y coordinate in meters x coordinate in meters tar[1] tar[3] tar[2] target trajectories estimated target trajectories 2 Figure 3: ### Trajectories of the three targets and of the ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363--375, 1994.

....on two examples. KEYWORDS: Bayesian, Classification, Clustering, Identifiability, MCMC, Mixture model, Multimodal posterior 1 Introduction The so called label switching problem arises when taking a Bayesian approach to parameter estimation and clustering using mixture models (see for example Diebolt and Robert, 1994; Richardson and Green, 1997) The term label switching was used by Redner and Walker (1984) to describe the invariance of the likelihood under relabelling of the mixture components. In a Bayesian context this invariance can lead to the posterior distribution of the parameters being highly ....

Diebolt, J. and Robert, C. P. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, series B, 56, 363--375.

.... and West, 1995; Lindsay, 1995; Roeder and Wasserman, 1997) In the Bayesian framework, estimation of the mixture model parameters can be done throughout simulation methods, involving either the Gibbs sampler or more elaborate forms of Markov Chain Monte Carlo simulations (MCMC) See, for example, Diebolt and Robert (1995); Escobar and West (1995) Richardson and Green (1998) Unfortunately, in the mixture model setting, it is not possible to perform default statistical analysis by directly using noninformative priors. In this work we explore the expected posterior prior approach as a default solutions to this ....

....model parameters 2 in an automatic way, or with minimal intervention. Note, however, that default priors are typically not proper, and, therefore, the marginal for the mixture model is not finite. Indeed, the marginal for any number of observations under an improper prior, N , is not finite (Diebolt and Robert, 1995; Shui, 1996; Roeder and Wasserman, 1997) In order to avoid this problem, Diebolt and Robert (1995) changed the probability distribution of z in (2) so that each component contains sufficient observations so that the marginal is finite. Unfortunately, this implies that the observations are no ....

[Article contains additional citation context not shown here]

Diebolt, J. and Robert, C. P. (1995), "Estimation of finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society, Series B, 56, 363--375.

....Beta binomials and Poisson gammas, typically achieve increased heterogeneity but are still limited to unimodality. Finite mixture distributions (Titterington, Smith and Makov, 1985) are more flexible and now more feasible to implement due to advances in simulation based model fitting. See, e.g. Diebolt and Robert (1994) and Richardson and Green (1997) Paradoxically, rather than handling the very large number of parameters resulting from finite mixture models with a large number of mixands, it may be easier to work with an infinite dimensional specification by assuming a random mixing distribution which is not ....

Diebolt, J., and Robert, C. P. (1994), "Estimation of Finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society, Ser. B, 56, 363-375.

....a few minor modi cations to the prior. The basic idea is to guarantee that the logistic basis functions of the hidden layer are linearly independent. We do this by putting certain restrictions on the parameters during the MCMC tting process, a method now common in the mixture model literature (Diebolt and Robert, 1994; Wasserman, 1998) First, a piece notation is helpful. Denote the outputs of the hidden layer as z ij = 1 exp j0 p X h=1 jh x ih # 1 (5) and let Z be the matrix with elements (z ij ) The tting of the vector is merely a least squares regression on the design matrix Z. What ....

Diebolt, J. and Robert, C. (1994). \Estimation of Finite Mixture Distributions Through Bayesian Sampling." Journal of the Royal Statistical Society B , 56, 363-375.

....that will be described shortly (equation (4) This is a data dependent prior where the data dependence will go to zero asymptotically. It approximates (2) with a sequence of at priors on increasing compact sets. The idea of using such a restricted prior comes from the mixture model literature (Diebolt and Robert 1994; Wasserman 1998) A heuristic justi cation is as follows: one can approximate a neural network with linear combinations of indicator functions, instead of logistic functions. In order to t the model, there must be at least one data point between the changepoints of the indicator functions. If ....

Diebolt, J. and Robert, C. (1994). \Estimation of Finite Mixture Distributions Through Bayesian Sampling." Journal of the Royal Statistical Society B , 56, 363-375.

No context found.

Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. R. Statist. Soc. 56 (1994) 363--375

No context found.

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B, 56, 363--375.

No context found.

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling.

No context found.

Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. of Royal Statist. Soc. B 56, No. 4, (1994) 363--375.

No context found.

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B, 2:363--375, 1994.

No context found.

Diebolt J., Robert C.P. (1994), Estimation of Finite Mixture Distributions through Bayesian Sampling, Journal of the Royal Statistical Society, Series B (Methodological), Vol. 56, Issue 2, pp. 363-375.

No context found.

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 56, pp. 363--375, 1994.

No context found.

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363--375, 1994.

No context found.

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, vol. 56, pp. 363--375, 1994.

No context found.

--175. Diebolt, J. and Robert, C. P. (1994), "Estimation of Finite Mixture Distributions Through Bayesian Sampling," Journal of the Royal Statistical Society, Series B, Methodological, 56,

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC