Results 11  20
of
506
Simulating ratios of normalizing constants via a simple identity: A theoretical exploration
 Statistica Sinica
, 1996
"... Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. ..."
Abstract

Cited by 180 (3 self)
 Add to MetaCart
Abstract: Let pi(w),i =1, 2, be two densities with common support where each density is known up to a normalizing constant: pi(w) =qi(w)/ci. We have draws from each density (e.g., via Markov chain Monte Carlo), and we want to use these draws to simulate the ratio of the normalizing constants, c1/c2. Such a computational problem is often encountered in likelihood and Bayesian inference, and arises in fields such as physics and genetics. Many methods proposed in statistical and other literature (e.g., computational physics) for dealing with this problem are based on various special cases of the following simple identity: c1 c2 = E2[q1(w)α(w)] E1[q2(w)α(w)]. Here Ei denotes the expectation with respect to pi (i =1, 2), and α is an arbitrary function such that the denominator is nonzero. A main purpose of this paper is to provide a theoretical study of the usefulness of this identity, with focus on (asymptotically) optimal and practical choices of α. Using a simple but informative example, we demonstrate that with sensible (not necessarily optimal) choices of α, we can reduce the simulation error by orders of magnitude when compared to the conventional importance sampling method, which corresponds to α =1/q2. We also introduce several generalizations of this identity for handling more complicated settings (e.g., estimating several ratios simultaneously) and pose several open problems that appear to have practical as well as theoretical value. Furthermore, we discuss related theoretical and empirical work.
A multivariate technique for multiply imputing missing values using a sequence of regression models. Survey Methodology 27
, 2001
"... This article describes and evaluates a procedure for imputing missing values for a relatively complex data structure when the data are missing at random. The imputations are obtained by fitting a sequence of regression models and drawing values from the corresponding predictive distributions. The ty ..."
Abstract

Cited by 169 (8 self)
 Add to MetaCart
(Show Context)
This article describes and evaluates a procedure for imputing missing values for a relatively complex data structure when the data are missing at random. The imputations are obtained by fitting a sequence of regression models and drawing values from the corresponding predictive distributions. The types of regression models used are linear, logistic, Poisson, generalized logit or a mixture of these depending on the type of variable being imputed. Two additional common features in the imputation process are incorporated: restriction to a relevant subpopulation for some variables and logical bounds or constraints for the imputed values. The restrictions involve subsetting the sample individuals that satisfy certain criteria while fitting the regression models. The bounds involve drawing values from a truncated predictive distribution. The development of this method was partly motivated by the analysis of two data sets which are used as illustrations. The sequential regression procedure is applied to perform multiple imputation analysis for the two applied problems. The sampling properties of inferences from multiply imputed data sets created using the sequential regression method are evaluated through simulated data sets. Key Words: Item nonresponse ; Missing at random ; Multiple imputation ; Nonignorable missing mechanism; Regression ; Sampling properties and simulations.
inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts
 Journal of Business and Economic Statistics
, 1993
"... We examine autoregressive time series models that are subject to regime switching. These shifts are determined by the outcome of an unobserved twostate indicator variable that follows a Markov process with unknown transition probabilities. A Bayesian framework is developed in which the unobserved s ..."
Abstract

Cited by 147 (5 self)
 Add to MetaCart
(Show Context)
We examine autoregressive time series models that are subject to regime switching. These shifts are determined by the outcome of an unobserved twostate indicator variable that follows a Markov process with unknown transition probabilities. A Bayesian framework is developed in which the unobserved states, one for each time point, are treated as missing data and then analyzed via the simulation tool of Gibbs sampling. This method is expedient because the conditional posterior distribution f the parameters, given the states, and the conditional posterior distribution of the states, given the parameters, all have a form amenable to Monte Carlo sampling. The approach is straightforward and generates marginal posterior distributions for all parameters of interest. Posterior distributions of the states, future observations, and the residuals, averaged over the parameter space are also obtained. Several examples with real and artificial data sets and weak prior information illustrate the usefulness of the methodology.
Geophysical inversion with a neighbourhood algorithmöI. Searching a parameter space, Geophys
 J. Int
, 1999
"... etc., are often used to explore a ¢nitedimensional parameter space. They require the solving of the forward problem many times, that is, making predictions of observables from an earth model. The resulting ensemble of earth models represents all `information ' collected in the search process. ..."
Abstract

Cited by 144 (8 self)
 Add to MetaCart
(Show Context)
etc., are often used to explore a ¢nitedimensional parameter space. They require the solving of the forward problem many times, that is, making predictions of observables from an earth model. The resulting ensemble of earth models represents all `information ' collected in the search process. Search techniques have been the subject of much study in geophysics; less attention is given to the appraisal of the ensemble. Often inferences are based on only a small subset of the ensemble, and sometimes a single member. This paper presents a new approach to the appraisal problem. To our knowledge this is the ¢rst time the general case has been addressed, that is, how to infer information from a complete ensemble, previously generated by any search method. The essence of the new approach is to use the information in the available ensemble to guide a resampling of the parameter space. This requires no further solving of the forward problem, but from the new `resampled ' ensemble we are able to obtain measures of resolution and tradeo ¡ in the model parameters, or any combinations of them. The new ensemble inference algorithm is illustrated on a highly nonlinear waveform inversion problem. It is shown how the computation time and memory requirements scale with the dimension of the parameter space and size of the ensemble. The method is highly parallel, and may easily be distributed across several computers. Since little is assumed about the initial ensemble of earth models, the technique is applicable to a wide variety of situations. For example, it may be applied to perform `error analysis ' using the ensemble generated by a genetic algorithm, or any other direct search method. Key words: numerical techniques, receiver functions, waveform inversion. 1
Bayesian Treatment of the Independent Studentt Linear Model
 JOURNAL OF APPLIED ECONOMETRICS
, 1993
"... This article takes up methods for Bayesian inference in a linear model in which the disturbances are independent and have identical Studentt distributions. It exploits the equivalence of the Studentt distribution and an appropriate scale mixture of normals, and uses a Gibbs sampler to perform the ..."
Abstract

Cited by 128 (2 self)
 Add to MetaCart
This article takes up methods for Bayesian inference in a linear model in which the disturbances are independent and have identical Studentt distributions. It exploits the equivalence of the Studentt distribution and an appropriate scale mixture of normals, and uses a Gibbs sampler to perform the computations. The new method is applied to some wellknown macroeconomic time series. It is found that posterior odds ratios favor the independent Studentt linear model over the normal linear model, and that the posterior odds ratio in favor of difference stationarity over trend stationarity is often substantially less in the favored Studentt models.
Hierarchical Priors and Mixture Models, With Application in Regression and Density Estimation
, 1993
"... ..."
The nested chinese restaurant process and bayesian inference of topic hierarchies
, 2007
"... We present the nested Chinese restaurant process (nCRP), a stochastic process which assigns probability distributions to infinitelydeep, infinitelybranching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Spe ..."
Abstract

Cited by 123 (15 self)
 Add to MetaCart
(Show Context)
We present the nested Chinese restaurant process (nCRP), a stochastic process which assigns probability distributions to infinitelydeep, infinitelybranching trees. We show how this stochastic process can be used as a prior distribution in a Bayesian nonparametric model of document collections. Specifically, we present an application to information retrieval in which documents are modeled as paths down a random tree, and the preferential attachment dynamics of the nCRP leads to clustering of documents according to sharing of topics at multiple levels of abstraction. Given a corpus of documents, a posterior inference algorithm finds an approximation to a posterior distribution over trees, topics and allocations of words to levels of the tree. We demonstrate this algorithm on collections of scientific abstracts from several journals. This model exemplifies a recent trend in statistical machine learning—the use of Bayesian nonparametric methods to infer distributions on flexible data structures.
Regeneration in Markov Chain Samplers
, 1994
"... Markov chain sampling has received considerable attention in the recent literature, in particular in the context of Bayesian computation and maximum likelihood estimation. This paper discusses the use of Markov chain splitting, originally developed as a tool for the theoretical analysis of general s ..."
Abstract

Cited by 119 (5 self)
 Add to MetaCart
Markov chain sampling has received considerable attention in the recent literature, in particular in the context of Bayesian computation and maximum likelihood estimation. This paper discusses the use of Markov chain splitting, originally developed as a tool for the theoretical analysis of general state space Markov chains, to introduce regeneration times into Markov chain samplers. This allows the use of regenerative methods for analyzing the output of these samplers, and can also provide a useful diagnostic of the performance of the samplers. The general approach is applied to several different samplers and is illustrated in a number of examples. 1 Introduction In Markov chain Monte Carlo, a distribution ß is examined by obtaining sample paths from a Markov chain constructed to have equilibrium distribution ß. This approach was introduced by Metropolis et al. (1953) and has recently received considerable attention as a method for examining posterior distributions in Bayesian infer...