Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. R. Statist. Soc. 56 (1994) 363--375  Home/Search   Document Not in Database   Summary   Related Articles   Check

....is quite different in its principle. The vectors Xt, Kt and rt are considered to be random variables with prior densities. Samples are then obtained iteratively from their joint posterior using a proper MCMC technique, namely the Gibbs Sampler. This method has been studied in [32] 33] 34] [35] or [36] for instance. It can be run sequentially at each time period. Gibbs Sampler is a special case of the Metropolis Hasting algorithm with the proposal densities being the conditional distributions, and the acceptance probability being consequently always equal to one. The interested reader ....

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363-375, 1994.

....belonging to the parameter space. The choice of the conjugate prior of the likelihood of the complete data as penalization term conducted to explicit EM algorithm re estimation formulas. While the role of conjugate priors is acknowledged in Bayesian sampling schemes, including in mixture problems [13], putting forward the link between conjugate priors and explicit penalized EM schemes is an original contribution, as far as we know. Numerical examples put in evidence the existence of the singularities and the e#ciency of the penalized solution. Concerning the asymptotic behavior of the ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, 56, (2), pp. 363--375, 1994.

.... P k is Gaussian, due to the Gaussianity of the error: 8 k = 1 p; P rob exp B C A We sample the new augmented variables t from the density p( Xj ; X 1 N ) This density is taken as a multinomial distribution M, usually used in mixing distribution problems [31]. t p X t jX t ; a ; b ; c = M(1; ff 1 ; ff p ) 4) with fi fi fi fi fi fi fi fi ff k P rob 8 k = 1 : p t [k] 1 (5) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the ....

....: p t [k] 1 (5) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the global augmented model is the derivation of the weights ff k . For more details on indicator variables in mixture or switching models, we refer to [31]. 2.3 Posterior step The second step of our process (given in eq. 3) deals with the sampling of the conditional posterior densities of the parameters. We start from the global posterior density derived from the classical Bayes rule: jX 1 N ; X 1 N ja; b; c; oe ; 6) ....

J. Diebolt and C.P. Robert. Estimation of finite mixture distributions through bayesian sampling. J. Royal Stat. Soc., 56(2):363--375, 1994. 23

....= 1 : p (3) A random variable sampled from this density is then a p variate vector with only one nonzero component. The key point of the global augmented model is the derivation of the weights ff k . For more details on indicator variables in mixture or switching models, we can refer to [8]. 3. SEM ALGORITHM FOR MODEL IDENTIFICATION With the description of the model made in the preceeding section, we now give the full conditional posterior densities of the parameters we want to estimate. We start from the global posterior density derived from the classical Bayes rule: X1 N ....

J. Diebolt and C.P. Robert. Estimation of finite mixture distributions through bayesian sampling. J. Royal Stat. Soc., 56(2):363--375, 1994.

....conditional mean estimates of the states and . The proposed conditional mean state estimator via the data augmentation algorithm is summarized in Fig. 1. Remark 3.1: Theoretically speaking, the DA algorithm does not have a stopping criterion. However, a reasonable choice (see, for example, [4]) is to terminate the algorithm when is less than some specified tolerance limit. Sampling Schemes: The DA algorithm presented in Fig. 1 requires us to compute samples from and . One possible scheme is the efficient forward filtering backward sampling recursions introduced by Carter and Kohn ....

....Thus, the sequence obtained from the DA sampling scheme is a Markov chain with invariant distribution and is ergodic. Ergodicity implies convergence of ergodic (sample) averages [23, Th. 3, p. 1717] Uniform ergodicity implies that the Law of Large Numbers and a central limit theorem also hold [4], 18] and [23, Th. 5, p. 1717] Corollary 3.1 Convergence of Ergodic Averages: For every real valued function , let us consider the time average of the first outputs of the Markov chain .If , then, for any initial distribution, 12) If , then a constant exists such that the distribution ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Stat. Soc. B, vol. 56, pp. 363--375, 1994.

....an infinite number of iterations to give the exact values of the MMSE and MMAP estimates. For all our simulations, we discard the first iterations to compute the MMSE estimates using the MCMC sampler. These first iterations correspond to the so called burn in period of the Markov chain. As in [9], the MCMC sampler algorithm is then iterated until the computed values of the ergodic averages are no longer modified. To ensure convergence toward the set of global Methods for determining the burn in period N are beyond the scope of this paper; see [29] for an overview of such methods. ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, vol. 56, pp. 363--375, 1994.

....we have the data y, a known relation between the unknown parameters a and y and finally the hyperparameters 1 and 0. The Bayesian estimation technique is now well established [1 7] and has been used since many years to resolve the inverse problems in signal and image reconstruction and restoration [10 14,17,18,20,21]. The first step before applying the Bayes rule is to assign the prior probability laws p(a 10) p(yla, fl) p(O) and p(fl) The next step is to determine the posterior laws and then to infer the unknowns. In this paper we are focusing more on the second step than on the first step. So we assume ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," Journal of Royal Statistical Society B, vol. 56, no. 2, pp. 363-375, 1994.

.... illustration of the Bayesian perspective on density estimation using Gaussian mixtures was recently provided by Roeder and Wasserman [15] The sampling approach to Bayesian inference in the context of Gaussian mixture models in the form used in this paper was first described by Diebolt [16]. An interesting extension of Bayesian sampling to cases where the number of Gaussian components is unknown has recently been suggested by Richardson and Green [17] Green [18] was also one of the first authors who used the EM algorithm for maximum penalized likelihood estimation. The first ....

....)d Theta: By using a conjugate prior p( Theta) we can obtain an analytically closed formulation of p(xjx ) Unfortunately, p(xjx ) is a sum of n m 1 terms and therefore is typically approximated. We use a stochastic approximation to p(xjx ) by employing the data augmentation method [16]. Data augmentation is an instantiation of Gibbs sampling, where one exploits the hierarchical structure of mixture models to generate a Markov chain ( Theta) t with stationary distribution p( Thetajx ) More specifically, one generates samples from the posterior of the parameters by the ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," Journal of the Royal Statistical Society B, vol. 56, no. 2, pp. 363--375, 1994.

....Tipping and Bishop (1999) proposed an MFA model with isotropic uniqueness matrices # k = # k I p , which is called the mixture of probabilistic PCA. Under these constraints, they have derived the ML estimation algorithms. 2. 2 Natural conjugate priors on parameters Many studies on mixture models (Diebolt and Robert, 1994) and factor analysis models (Press and Shigemasu, 1989) employ natural conjugate priors on their parameters, because such priors usually lead to simple Bayesian estimation algorithms. The natural conjugate prior on # is a Dirichlet distribution f(# #) D(# #) # m # k=1 # # 1 k (2.6) ....

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B, 56:363--375.

....same for all # to write the second equality in (9) It remains to estimate the association probabilities ## i t # i##; M , which can be seen as the stochastic coefficients of the ##component mixture. To estimate them we propose to use a Gibbs sampler whose principles are briefly recalled (see [1] or [9] for more details) For # ### t ## t # # t #, it consists in generating a Markov chain that converges to the stationary distribution ##### ##t # which cannot be sampled directly. Given a partition # # ###### P of #, one samples alternatively from the conditional posterior distribution ....

....the following simulations) # A subset of false measurements which number follows a Poisson distribution with mean ## . 3.3. 2 Results of the MTPF 3000 2500 2000 1500 1000 500 0 500 1000 1500 2000 1000 1000 2000 3000 4000 5000 6000 7000 y coordinate in meters x coordinate in meters tar[1] tar[3] tar[2] observer 1 2000 1500 1000 500 0 500 1000 1500 2000 1000 0 1000 2000 3000 4000 5000 6000 7000 y coordinate in meters x coordinate in meters tar[1] tar[3] tar[2] target trajectories estimated target trajectories 2 Figure 3: ### Trajectories of the three targets and of the ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363--375, 1994.

....on two examples. KEYWORDS: Bayesian, Classification, Clustering, Identifiability, MCMC, Mixture model, Multimodal posterior 1 Introduction The so called label switching problem arises when taking a Bayesian approach to parameter estimation and clustering using mixture models (see for example Diebolt and Robert, 1994; Richardson and Green, 1997) The term label switching was used by Redner and Walker (1984) to describe the invariance of the likelihood under relabelling of the mixture components. In a Bayesian context this invariance can lead to the posterior distribution of the parameters being highly ....

Diebolt, J. and Robert, C. P. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, series B, 56, 363--375.

.... and West, 1995; Lindsay, 1995; Roeder and Wasserman, 1997) In the Bayesian framework, estimation of the mixture model parameters can be done throughout simulation methods, involving either the Gibbs sampler or more elaborate forms of Markov Chain Monte Carlo simulations (MCMC) See, for example, Diebolt and Robert (1995); Escobar and West (1995) Richardson and Green (1998) Unfortunately, in the mixture model setting, it is not possible to perform default statistical analysis by directly using noninformative priors. In this work we explore the expected posterior prior approach as a default solutions to this ....

....model parameters 2 in an automatic way, or with minimal intervention. Note, however, that default priors are typically not proper, and, therefore, the marginal for the mixture model is not finite. Indeed, the marginal for any number of observations under an improper prior, N , is not finite (Diebolt and Robert, 1995; Shui, 1996; Roeder and Wasserman, 1997) In order to avoid this problem, Diebolt and Robert (1995) changed the probability distribution of z in (2) so that each component contains sufficient observations so that the marginal is finite. Unfortunately, this implies that the observations are no ....

[Article contains additional citation context not shown here]

Diebolt, J. and Robert, C. P. (1995), "Estimation of finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society, Series B, 56, 363--375.

....Beta binomials and Poisson gammas, typically achieve increased heterogeneity but are still limited to unimodality. Finite mixture distributions (Titterington, Smith and Makov, 1985) are more flexible and now more feasible to implement due to advances in simulation based model fitting. See, e.g. Diebolt and Robert (1994) and Richardson and Green (1997) Paradoxically, rather than handling the very large number of parameters resulting from finite mixture models with a large number of mixands, it may be easier to work with an infinite dimensional specification by assuming a random mixing distribution which is not ....

Diebolt, J., and Robert, C. P. (1994), "Estimation of Finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society, Ser. B, 56, 363-375.

....a few minor modi cations to the prior. The basic idea is to guarantee that the logistic basis functions of the hidden layer are linearly independent. We do this by putting certain restrictions on the parameters during the MCMC tting process, a method now common in the mixture model literature (Diebolt and Robert, 1994; Wasserman, 1998) First, a piece notation is helpful. Denote the outputs of the hidden layer as z ij = 1 exp j0 p X h=1 jh x ih # 1 (5) and let Z be the matrix with elements (z ij ) The tting of the vector is merely a least squares regression on the design matrix Z. What ....

Diebolt, J. and Robert, C. (1994). \Estimation of Finite Mixture Distributions Through Bayesian Sampling." Journal of the Royal Statistical Society B , 56, 363-375.

....that will be described shortly (equation (4) This is a data dependent prior where the data dependence will go to zero asymptotically. It approximates (2) with a sequence of at priors on increasing compact sets. The idea of using such a restricted prior comes from the mixture model literature (Diebolt and Robert 1994; Wasserman 1998) A heuristic justi cation is as follows: one can approximate a neural network with linear combinations of indicator functions, instead of logistic functions. In order to t the model, there must be at least one data point between the changepoints of the indicator functions. If ....

Diebolt, J. and Robert, C. (1994). \Estimation of Finite Mixture Distributions Through Bayesian Sampling." Journal of the Royal Statistical Society B , 56, 363-375.

.... constraint oe 2 s = 1 makes closed form expressions for the study specific variances unavailable, so we use a Metropolis step (Tierney 1994) The normal mixture parameters for the study effects are updated by augmenting the data with a vector of latent mixture component indicators, as done by Diebolt and Roberts (1994) and West and Turner (1994) for their unconstrained cases. The constraint E( s ) 0 is linear in ff and so the full conditional distribution of ff is normal, truncated to the positive half line. At each iteration the latent variables are simulated from the distributions given in Section 2.3. We ....

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B, Methodological, 56:363--375.

....which causes label switching in the Gibbs sampler output and makes inferences for individual components of the mixture meaningless. A common practice is to impose identifiability constraints on the model parameters such as k s s s # 1 1 but this is often not a satisfactory solution (Diebolt and Robert, 1994). Stephens (1997) suggests a general solution which involves permuting samples from the parameter posterior density so as to remove as much multimodality as possible and allows interpretations for groups to be discovered rather than imposed. 3 The purpose of this paper is to present EM, ....

Diebolt, J., and C. P. Robert. 1994. Estimation of finite mixture distributions through Bayesian Analysis. J. Roy. Statist. Soc. B 56(2):363-375.

....these algorithms and their applications to predictive approaches to model selection. The EM algorithm for these problems follows from McLachlan and Jones (1988) We are adding the MCEM development in case the E step is hard. The MCMC for i.i.d. observations for mixture models has been developed by Diebolt and Robert (1994). We are extending it to censored data. The basic principle in the extension can be seen in Gelfand, Smith, and Lee (1992) and Kuo and Smith (1992) The EM algorithm proposed by Dempster, Laird, and Rubin (1977) allows us to consider a likelihood of a product of components model for i.i.d. ....

Diebolt. J., and Robert, C. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society. Series B 56, 363-375.

....(Eqs. 14) and (15) or the mixture estimators (Eqs. 16) and (17) Figure 1: Algorithm I : Conditional mean state estimator via the data augmentation algorithm. Remark 3.1 Theoretically speaking the DA algorithm does not have a stopping criterion. However, a reasonable choice (see for example [4]) is to terminate the algorithm when #r(N) r(N 1)# 2 is less than some specified tolerance limit. Sampling Schemes: The Data Augmentation Algorithm presented in Fig. 1, requires to compute samples from f # x y, r (k 1) # and f # r y, x (k) # . One possible scheme is the e#cient ....

....sampling scheme is a Markov chain with invariant distribution f ( r, x y) and is ergodic. Ergodicity implies convergence of ergodic (sample) averages ( 22] Th. 3, pp. 1717) Uniform ergodicity implies that a law of large numbers and a central limit theorem also holds ( 22] Th. 5, pp. 1717, [17, 4]) Convergence of Ergodic Averages) For every real valued function # : S T R (T 1)nx # R, let us consider the time average of the N first outputs of the Markov chain # N # = 1 N # N 1 k=0 #(r (k) x (k) If E f( r,x y) #(r, x) # then for any initial distribution, # N ....

[Article contains additional citation context not shown here]

J. Diebolt and C.P. Robert, "Estimation of Finite Mixture Distributions through Bayesian Sampling", J. R. Statis. Soc. B, Vol. 56, No. 2, pp. 363-375, 1994.

....quasi Newton methods, conjugate gradient methods, simulated annealing and downhill simplex. See Gill et al. 1981) for an introduction to practical optimization. Other approaches to mixture estimation also exists. Bernardo and Gir on (1988) give a Bayesian treatment of finite mixture models. Diebolt and Robert (1994) consider estimation of finite mixture distributions through Bayesian sampling and Crawford (1994) uses the Laplace method. We will concentrate on estimating the parameters and weights of mixture distributions based on the likelihood equations using an alternating likelihood estimation procedure, ....

Diebolt, J. and Robert, C. P. (1994), "Estimation of finite mixture distributions through Bayesian sampling", Journal of the Royal Statistical Society, Series B, 56, 363--375.

....mixture, m, is assumed to be known. Then the model can be estimated by MCMC byintroducing latent (unobserved) variables ffi j (1 j n) which indicate the label of the group from which observation j is drawn. Of course, a priori p(ffi j = i) i # for i =1#: #m: This model has been studied by Diebolt and Robert (1994) who proposed a data augmentation algorithm to carry out the estimation and proved that it converges geometrically. They also studied the convergence of Gibbs sampling. In practice the number of components in the mixture is unknown. Then wehave four possible approaches. The first one estimates m ....

Diebolt, J. and Robert, C. (1994). "Estimation of finite mixture distributions through Bayesian Sampling". Journal of the Royal Statistical Society, B, 56, 363--375.

....Note, however, that default priors are typically not proper, and, therefore, the marginal for the mixture model is not finite. Indeed, the marginal for any number of observations under an improper prior, p N , is not finite. Several approaches have been proposed in order to solve this problem ([22], 55] 40] 59] 60] Some of these approaches utilize a prior structure in which the components are linked to a common parameter with a flat prior, so that all observations contribute to its estimation. Shui [60] proposes use of resampling from the observations in order to produce versions of ....

....Shui [60] proposes use of resampling from the observations in order to produce versions of the Intrinsic Bayes Factor [8] and the Fractional Bayes Factor [45] which then can be used for model selection. Note that the use of vague priors does not solve this problem, as pointed out by many [22], 38] 60] While vague priors are proper densities, the results can be very sensitive to the way in which these priors are made vague . For the normal univariate case, 53] suggests the use of weak, data dependent, hierarchical priors for the component parameters. The hyperparameters of these ....

[Article contains additional citation context not shown here]

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling", J. Roy. Stat. Soc. B, 56, 363-375, 1994.

....m, is assumed to be known. Then the model can be estimated by MCMC by introducing latent (unobserved) variables ffi j (1 j n) which indicate the label of the group from which observation j is drawn. Of course, a priori p(ffi j = i) i ; for i = 1; m: This model has been studied by Diebolt and Robert (1994) who proposed a data augmentation algorithm to carry out the estimation and proved that it converges geometrically. They also studied the convergence of Gibbs sampling. In practice the number of components in the mixture is unknown. Then we have four possible approaches. The first one estimates m ....

Diebolt, J. and Robert, C. (1994). "Estimation of finite mixture distributions through Bayesian Sampling". Journal of the Royal Statistical Society, B, 56, 363--375.

....have the data y, a known relation between the unknown parameters x and y and finally the hyperparameters fi and . The Bayesian estimation technique is now well established [1 7] and has been used since many years to resolve the inverse problems in signal and image reconstruction and restoration [10 14,17,18,20,21]. The first step before applying the Bayes rule is to assign the prior probability laws p(xj ) p(yjx; fi) p( and p(fi) The next step is to determine the posterior laws and then to infer the unknowns. In this paper we are focusing more on the second step than on the first step. So we assume ....

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," Journal of Royal Statistical Society B, vol. 56, no. 2, pp. 363--375, 1994.

....for coping with these problems. Finally we give some results on the consistency of the method when the maximum number of components is allowed to grow with the sample size. KEY WORDS: Markov chain Monte Carlo, Normal mixture, Partially proper prior, Sieve. 1. INTRODUCTION. Many authors, including Diebolt and Robert (1994), Escobar and West (1995) Mengersen and Robert (1993) Nobile (1994) Raftery (1995) and West (1992) have shown that mixtures of normals provide a simple, effective basis for nonparametric Bayesian density estimation. The class is very flexible and Markov chain Monte Carlo methods make it easy ....

Diebolt, J. and Robert, C. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. R. Statist. Soc. B, 56, 363-375.

....observations into the underlying classes, and estimate the density of each class. Examples and more background on non Bayesian analysis of mixtures are given in the monologues by Titterington et al. 1985) and McLachlan and Basford (1988) Key papers on the Bayesian analysis of mixtures include Diebolt and Robert (1994), Escobar and West (1995) and Richardson and Green (1997) Formally we assume that x n = x 1 ; x n are independent observations from a mixture density with k (k assumed known and finite) components: p(x j ; OE; j) 1 f(x; OE 1 ; j) Delta Delta Delta k f(x; OE k ; j) 1) ....

.... Gamma F ( Theta) j x n Delta = Z F ( p( j x n ) d : 3) Accurate approximation of such integrals is now routine through the use of Markov chain Monte Carlo (MCMC) methods (see Gilks et al. 1996, for example) In particular, the application of Gibbs sampling to mixture models (as in Diebolt and Robert, 1994, for example) is straightforward for many parametric families ff( Delta; OE; j)g, provided suitable (conjugate) prior distributions are used. Such methods allow the construction of an ergodic Markov chain with stationary distribution p( j x n ) and integrals of the form (3) may be ....

Diebolt, J. and Robert, C. P. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, series B, 56, 363--375.

....covariate, but in the change point problem we use the normal mixture to model the measurement error. 3 Normal Mixtures For Modeling Measurement Error Many authors have shown that mixtures of normals with an unspecified number of components provide a simple, flexible family of distributions (Diebolt and Robert, 1994, Escobar and West, 1995, Mengersen and Robert, 1993, Nobile, 1994, Raftery, 1995, Roeder and Wasserman, 1997, West, 1992) We represent such a mixture by f( Deltaj ; oe; p) P k j=1 p j OE( Delta; j ; oe j ) where = 1 ; k ) oe = oe 1 ; oe k ) p = p 1 ; p k ....

....: X n ) be a sample from a mixture of k normals. Define the group membership vector G = G 1 ; G n ) where G i indicates which component of of the normal mixture from which X i arose. For example, if G 3 = 2, then the third observation came from the second component of the mixture. Diebolt and Robert (1994) and Verdinelli and Wasserman (1991) show that introducing G into the model makes it easy to simulate from the posterior. This is because the complete conditionals of the parameters given G are usually very simple distributions. In the errors in variables context one rarely has prior ....

Diebolt and Robert (1994). Estimation of finite mixture distributions through Bayesian sampling," Journal of the Royal Statistical Society Ser. 56, 363--375.

....investigators generally enlarge the number of models considered, thus perpetuating sensitivity to the prior distribution. Several extensions to these developments are natural and would involve no problems beyond normal technical difficulties in implementation. Following West (1984) Geweke (1993) Diebolt and Robert (1994), and others, the assumption that disturbances are normal may be weakened through appropriate use of mixture models. Nor is the procedure limited to truncated normal prior distributions for coefficients of included variables: since the Gibbs sampling algorithm is fully blocked essentially ....

Diebolt, J., and C.P. Robert, 1994, "Estimation of Finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society Series B 56: 363-376.

....this is determined, for example, by enumeration and cross validation. Furthermore, EM solutions to the maximum likelihood (ML) fitting problem are local maxima and saddle points, and are often very poor (fig 1) Markov chain Monte Carlo methods can more closely approximate the Bayesian solution [3,4], but these are exceedingly slow. The approach taken here is to modify the ML EM algorithm for GMMs to include easily obtainable priors in order to (i) estimate the GMM s complexity simultaneously with its continuous parameters, ii) to obtain smooth well generalising models, and (iii) to extend ....

Diebolt, J. and Robert, C. P., 1994, "Estimation of finite mixture distributions through Bayesian sampling ", J. Roy. Statistical Society, B, vol. 56, no. 2, pp. 363-375.

....setting h ( X n ; Y n ) X n , we can apply Proposition 4. We conclude that fX n g is forward sufficient for f(X n ; Y n )g, and that fX n Gamma1 g is backward sufficient for f(X n ; Y n )g. One example of the use of this property is in the Bayesian analysis of finite mixtures (see for example Diebolt and Robert, 1994). For these models, the space of missing data is finite, and therefore the Markov chain consisting of just the missing data is uniformly ergodic. Furthermore, the missing data is bi sufficient for the entire chain. Consequently, by Corollary 2, the data augmentation algorithm is also uniformly ....

....therefore the Markov chain consisting of just the missing data is uniformly ergodic. Furthermore, the missing data is bi sufficient for the entire chain. Consequently, by Corollary 2, the data augmentation algorithm is also uniformly ergodic. This observation was termed the duality principle by Diebolt and Robert (1994). 4. Pseudo finite Markov chains, or Markov chains of finite rank (Hoekstra and Steutel, 1984; Runnenburg and Steutel, 1962; Rosenthal, 1992) Here P (X n 1 2 Delta j X n = x) m X j=1 f j (x) Q j ( Delta) for some finite number m 2 N, where f i : X [0; 1] are deterministic functions ....

Diebolt, J. and Robert, C.P. (1994), Estimation of finite mixture distributions through Bayesian sampling. J. Royal Stat. Soc. Ser. B 56, 363--375.

....with an unknown number of components. Much previous work on finite mixture estimation, Bayesian or otherwise, has separated the issues of testing the number of components k from estimation with k fixed. For the fixed k case, a comprehensive Bayesian treatment using MCMC methods was presented in Diebolt and Robert (1994). Early approaches to the general case where k is unknown typically adopted a different style of modelling, treating the problem as an example of Bayesian nonparametrics , and basing prior distributions on the Dirichlet process; see Escobar and West (1995) for example. Other authors, like ....

....possible in a mixture context. Since there is always the possibility that no observations are allocated to one or more components, and so the data are uninformative about them, standard choices of independent improper non informative prior distributions for the component parameters cannot be used (Diebolt and Robert, 1994; Roeder and Wasserman, 1995) Some previous attempts to circumvent this problem, which involve dependent priors, are mentioned in Section 8.3. It seems to us that for most purposes of mixture modelling, there is a case for keeping to the simple independence prior structure for the j and oe ....

[Article contains additional citation context not shown here]

Diebolt, J. and Robert, C. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, B, 56, 163--175.

.... 1 Delta Delta Delta fl k ) Gamma(fl 1 ) Gamma(fl k ) fl 1 Gamma1 1 : fl k Gamma1 Gamma1 k Gamma1 (1 Gamma 1 Gamma Delta Delta Delta Gamma k Gamma1 ) fl k Gamma1 : We note that this general hierarchical model includes the specific models used by Diebolt and Robert (1994) and Richardson and Green (1997) in the context of mixtures of univariate normal distributions. For an alternative approach see Escobar and West (1995) who use a prior structure based on the Dirichlet process. Given data x n , Bayesian inference may be performed using MCMC methods, which involve ....

.... k (t) ig (N large) 6) and similarly the predictive density for a future observation may be estimated by p(x n 1 j x n ) 1 N N X t=1 p(x n 1 j (t) 7) More details, including details of the construction of a suitable Markov chain when k is fixed, can be found in the paper by Diebolt and Robert (1994), chapters of the books by Robert (1994) and Gelman et al. 1995) and the article by Robert (1996) Richardson and Green (1997) describe the construction of a suitable Markov chain when k is allowed to vary using the reversible jump methodology developed by Green (1995) We now describe an ....

Diebolt, J. and Robert, C. P. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, series B, 56, 363--375.

....modelling. The second focus is on estimation of the component parameters p 1 ; p 2 ; p k ; 1 ; 2 ; k ; oe 1 ; oe 2 ; oe k : Third, there is the question of classification, that is, determination of the component to which each of the x j ; j = 1; n belongs. As discussed in Diebolt and Robert (1994), this classification exposes a hidden structure in the model which may be viewed as missing data: each observation is associated with an unobserved indicator of the component from which it originated. This representation is exploited in the Bayesian framework adopted in this paper. Enjoyment of ....

....the ith component. Not only does this prevent the use of (independent) improper priors on ( i ; oe i ) although they are particularly attractive in this framework (Bernardo and Gir on, 1988) but it also generates trapping states for Markov chain Monte Carlo (MCMC) algorithms, as identified by Diebolt and Robert (1994). When the number of components k is unknown, the parameter space is simultaneously ill defined and of infinite dimension, and this additional difficulty prevents the use of classical testing procedures and priors. Tests about k have been proposed in the literature (Ghosh and Sen, 1985) but only ....

[Article contains additional citation context not shown here]

Diebolt, J. and Robert, C.P. (1994) Estimation of finite mixture distributions through Bayesian sampling. J. Royal Statist. Soc. (Ser. B) 56(2), 363-375 Escobar, M.D. (1994) Estimating normal means with a Dirichlet process prior. Journal of the American Statistical Association 89, 268--277.

.... X q (l) 2f0;1g K min q (l Gamma1) 2f0;1g K KPn 0 Gamma q (l Gamma1) q (l) Delta , we finally conclude that the Markov chain Phi q (l) l 2 N Psi satisfies [27] X q2f0;1g K jP l (q) Gamma P (q j y)j 2ae l : Applying the duality principle of Robert and Diebolt [12], we now show that the uniform geometric convergence of the chain Phi q (l) l 2 N Psi is transfered to the continuous statespace Markov chain Phi (l) l 2 N Psi Z Theta jp l ( Gamma p( j y)jd 2ae l Gamma1 : To prove this result, first note that p l (q; P (q j ....

J. Diebolt and C. P. Robert. Estimation of Finite Mixture Distributions through Bayesian Sampling. J. Roy. Stat. Soc. B, 56:363--375, 1994.

....with an unknown number of components. Much previous work on finite mixture estimation, Bayesian or otherwise, has separated the issues of testing the number of components k from estimation with k fixed. For the fixed k case, a comprehensive Bayesian treatment using MCMC methods was presented in Diebolt and Robert (1994). Early approaches to the general case where k is unknown typically adopted a different style of modelling, treating the problem as an example of Bayesian nonparametrics , and basing prior distributions on the Dirichlet process; see Escobar and West (1995) for example. Other authors, for example ....

....possible in a mixture context. Since there is always the possibility that no observations are allocated to one or more components, and so the data are uninformative about them, standard choices of independent improper non informative prior distributions for the component parameters cannot be used (Diebolt and Robert, 1994; Roeder and Wasserman, 1995) Some previous attempts to circumvent this problem, which involve dependent priors, are mentioned in Section 8.3. It seems to us that for most purposes of mixture modelling, there is a case for keeping to the simple independence prior structure for the j and oe ....

[Article contains additional citation context not shown here]

Diebolt, J. and Robert, C. (1994) Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, B, 56, 163--175.

.... those components, and in kernel estimation these quantities are fixed (but see, for example, Jones et al. 1994) In addition, only one point of data is assigned to each mixture component, occurring at the mode, and clearly trying to estimate the variance (or itself) by methods such as those of Diebolt and Robert (1994) say, is not possible, as the relevant integral will have infinite value. In kernel density estimation, one common way around this problem is to use crossvalidation. Each point of data is assumed to have originated from the kernel density based on all the other observations. We build a model based ....

Diebolt, J, and Robert, CP (1994) "Estimation of Finite Mixture Distributions through Bayesian Sampling," Journal of the Royal Statistical Society, B 56 363-375.

....Latent Data MCMC methods often involve introducing latent data z, that is such that when z is known, the complete data likelihood , pr(D; zj ) has a simple form. This is the idea underlying the EM algorithm (Dempster, Laird and Rubin, 1977) its stochastic generalizations (see the chapter by Diebolt and Ip, 1994), and its Bayesian analogue, the IP algorithm (Tanner and Wong, 1987) In MCMC methods, values of both z and are generated from their joint posterior distribution. The latent data can consist, for example, of individual random effects in a random effects or hierarchical model, of group ....

....and Raftery, Madigan and Hoeting (1993) In order to use the Gibbs sampler, I introduce the latent data z = z 1 ; z n ) where z i = j if y i N( j ; v j ) i.e. if y i belongs to the jth component of the mixture. The required conditional posterior distributions are then given by Diebolt and Robert (1994); see also the chapter by Robert (1994) The Gibbs sampler is initialized by dividing the data into J equal sized chunks of contiguous data points, using the resulting means and variances for and v, and setting ae j = 1=J (j = 1; J ) It proceeds by drawing first z, and then ae, v and ....

Diebolt, J. and Robert, C. (1994). Estimation of finite mixture distributions through Bayesian sampling. J. R. Statist. Soc., ser. B, 56, 363--376.

....as the Bayesian version of SEM. Alternatively, SEM can be recovered from the Data Augmentation algorithm by taking a suitable noninformative prior ( and replacing the simulation step of r from ( jy; z) by an imputation step where r is updated as r 1 = R ( jy; z r ) d . See Diebolt and Robert (1994) for more details in the mixture context. Finally, the SIP can be expected to provide a satisfactory data driven magnitude for the random perturbations of SEM. For instance, when the sample of observed data is small and contains little information about the true value of the parameter , the ....

Diebolt, J. and Robert, C. P. (1994), Estimation of Finite Mixture Distributions through Bayesian Sampling, Journal of the Royal Statistical Society B 56, 363--375.

No context found.

Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. R. Statist. Soc. 56 (1994) 363--375

No context found.

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B, 56, 363--375.

No context found.

Diebolt, J. and Robert, C. P. (1994). Estimation of finite mixture distributions through Bayesian sampling.

No context found.

Diebolt, J., Robert, C.P.: Estimation of Finite Mixture Distributions through Bayesian Sampling. J. of Royal Statist. Soc. B 56, No. 4, (1994) 363--375.

No context found.

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society B, 2:363--375, 1994.

No context found.

Diebolt J., Robert C.P. (1994), Estimation of Finite Mixture Distributions through Bayesian Sampling, Journal of the Royal Statistical Society, Series B (Methodological), Vol. 56, Issue 2, pp. 363-375.

No context found.

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Stat. Soc. Ser. B Stat. Methodol., vol. 56, pp. 363--375, 1994.

No context found.

J. Diebolt and C. P. Robert. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society series B, 56:363--375, 1994.

No context found.

J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through Bayesian sampling," J. R. Statist. Soc. B, vol. 56, pp. 363--375, 1994.

No context found.

--175. Diebolt, J. and Robert, C. P. (1994), "Estimation of Finite Mixture Distributions Through Bayesian Sampling," Journal of the Royal Statistical Society, Series B, Methodological, 56,

No context found.

Diebolt, J. and Robert C. (in press). "Estimation of finite mixture distributions through Bayesian sampling," Journal of the Royal Statistical Society, Series B. Gaver, D.P. and O'Muircheartaigh, I.G. (1987), "Robust empirical Bayes analysis of event rates," Technometrics, 29, 1-15.

No context found.

Diebolt, J. and Robert, C. (1994). Estimation of finite mixture distributions through Bayesian sampling, Jour. Roy. Stat. Soc., 56, 363-375.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC