Results 1  10
of
74
Understanding GPU programming for statistical computation: Studies in massively parallel massive mixtures
 Journal of Computational and Graphical Statistics
, 2010
"... This paper describes advances in statistical computation for largescale data analysis in structured Bayesian mixture models via graphics processing unit (GPU) programming. The developments are partly motivated by computational challenges arising in fitting models of increasing heterogeneity to incr ..."
Abstract

Cited by 33 (9 self)
 Add to MetaCart
(Show Context)
This paper describes advances in statistical computation for largescale data analysis in structured Bayesian mixture models via graphics processing unit (GPU) programming. The developments are partly motivated by computational challenges arising in fitting models of increasing heterogeneity to increasingly large data sets. An example context concerns common biological studies using highthroughput technologies generating many, very large data sets and requiring increasingly highdimensional mixture models with large numbers of mixture components. We outline important strategies and processes for GPU computation in Bayesian simulation and optimization approaches, examples of the benefits of GPU implementations in terms of processing speed and scaleup in ability to analyze large data sets, and provide a detailed, tutorialstyle exposition that will benefit readers interested in developing GPUbased approaches in other statistical models. Novel, GPUoriented approaches to modifying existing algorithms software design can lead to vast speedup and, critically, enable statistical analyses that presently will not be performed due to compute time limitations in traditional computational environments. Supplemental materials are provided with all source code, example data and details that will enable readers to implement and explore the GPU approach in this mixture modelling context.
Interactive furniture layout using interior design guidelines
 ACM Trans. Graph
, 2011
"... Figure 1: Interactive furniture layout. For a given layout (left), our system suggests new layouts (middle) that respect the user’s constraints and follow interior design guidelines. The red chair has been fixed in place by the user. One of the suggestions is shown on the right. We present an intera ..."
Abstract

Cited by 26 (0 self)
 Add to MetaCart
Figure 1: Interactive furniture layout. For a given layout (left), our system suggests new layouts (middle) that respect the user’s constraints and follow interior design guidelines. The red chair has been fixed in place by the user. One of the suggestions is shown on the right. We present an interactive furniture layout system that assists users by suggesting furniture arrangements that are based on interior design guidelines. Our system incorporates the layout guidelines as terms in a density function and generates layout suggestions by rapidly sampling the density function using a hardwareaccelerated Monte Carlo sampler. Our results demonstrate that the suggestion generation functionality measurably increases the quality of furniture arrangements produced by participants with no prior training in interior design.
and big data: The consensus monte carlo algorithm, Bayes 250
, 2013
"... A useful definition of “big data ” is data that is too big to comfortably process on a single machine, either because of processor, memory, or disk bottlenecks. Graphics processing units can alleviate the processor bottleneck, but memory or disk bottlenecks can only be eliminated by splitting data a ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
(Show Context)
A useful definition of “big data ” is data that is too big to comfortably process on a single machine, either because of processor, memory, or disk bottlenecks. Graphics processing units can alleviate the processor bottleneck, but memory or disk bottlenecks can only be eliminated by splitting data across multiple machines. Communication between large numbers of machines is expensive (regardless of the amount of data being communicated), so there is a need for algorithms that perform distributed approximate Bayesian analyses with minimal communication. Consensus Monte Carlo operates by running a separate Monte Carlo algorithm on each machine, and then averaging individual Monte Carlo draws across machines. Depending on the model, the resulting draws can be nearly indistinguishable from the draws that would have been obtained by running a single machine algorithm for a very long time. Examples of consensus Monte Carlo are shown for simple models where singlemachine solutions are available, for large singlelayer hierarchical models, and for Bayesian additive regression trees (BART). 1
Efficient Bayesian Inference for Switching StateSpace Models using Particle Markov Chain Monte Carlo Methods
, 2010
"... Switching statespace models (SSSM) are a popular class of time series models that have found many applications in statistics, econometrics and advanced signal processing. Bayesian inference for these models typically relies on Markov chain Monte Carlo (MCMC) techniques. However, even sophisticated ..."
Abstract

Cited by 13 (1 self)
 Add to MetaCart
(Show Context)
Switching statespace models (SSSM) are a popular class of time series models that have found many applications in statistics, econometrics and advanced signal processing. Bayesian inference for these models typically relies on Markov chain Monte Carlo (MCMC) techniques. However, even sophisticated MCMC methods dedicated to SSSM can prove quite inefficient as they update potentially strongly correlated variables oneatatime. Particle Markov chain Monte Carlo (PMCMC) methods are a recently developed class of MCMC algorithms which use particle filters to build efficient proposal distributions in highdimensions [1]. The existing PMCMC methods of [1] are applicable to SSSM, but are restricted to employing standard particle filtering techniques. Yet, in the context of SSSM, much more efficient particle techniques have been developed [22, 23, 24]. In this paper, we extend the PMCMC framework to enable the use of these efficient particle methods within MCMC. We demonstrate the resulting generic methodology on a variety of examples including a multiple changepoints model for welllog data and a model for U.S./U.K. exchange rate data. These new PMCMC algorithms are shown to outperform experimentally stateoftheart MCMC techniques for a fixed computational complexity. Additionally they can be easily parallelized [39] which allows further substantial gains.
Estimation and prediction in spatial models with block composite likelihoods
 Journal of Computational and Graphical Statistics (To Appear
, 2013
"... A block composite likelihood is developed for estimation and prediction in large spatial datasets. The composite likelihood is constructed from the joint densities of pairs of adjacent spatial blocks. This allows large datasets to be split into many smaller datasets, each of which can be evaluated s ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
(Show Context)
A block composite likelihood is developed for estimation and prediction in large spatial datasets. The composite likelihood is constructed from the joint densities of pairs of adjacent spatial blocks. This allows large datasets to be split into many smaller datasets, each of which can be evaluated separately, and combined through a simple summation. Estimates for unknown parameters are obtained by maximizing the block composite likelihood function. In addition, a new method for optimal spatial prediction under the block composite likelihood is presented. Asymptotic variances for both parameter estimates and predictions are computed using Godambe sandwich matrices. The approach gives considerable improvements in computational efficiency, and the composite structure obviates the need to load entire datasets into memory at once, completely avoiding memory limitations imposed by massive datasets. Moreover, computing time can be reduced even further by distributing the operations using parallel computing. A simulation study shows that composite likelihood estimates and predictions, as well as their corresponding asymptotic confidence intervals, are competitive with those based on the full likelihood. The procedure is demonstrated on one dataset from the mining industry and one dataset of satellite retrievals. The realdata examples
Variance bounding and geometric ergodicity of Markov chain Monte Carlo for approximate Bayesian computation. arXiv:1210.6703 [stat.ME
, 2012
"... ar ..."
Sequential Monte Carlo on large binary sampling spaces
 Statist. Comput
, 2011
"... A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric fam ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
A Monte Carlo algorithm is said to be adaptive if it automatically calibrates its current proposal distribution using past simulations. The choice of the parametric family that defines the set of proposal distributions is critical for good performance. In this paper, we present such a parametric family for adaptive sampling on highdimensional binary spaces. A practical motivation for this problem is variable selection in a linear regression context. We want tosamplefromaBayesian posterior distribution on the model space using an appropriate version of Sequential Monte Carlo. Raw versions of Sequential Monte Carlo are easily implemented using binary vectors with independent components. For highdimensional problems, however, these simple proposals do not yield satisfactory results. The key to an efficient adaptive algorithm are binary parametric families which take correlations intoaccount, analogously tothemultivariate normaldistribution on continuous spaces. We provide a review of models for binary data and make one of them work in the context of Sequential Monte Carlo sampling. Computational studies on real life data with about a hundred covariates suggest that, on difficult instances, our Sequential Monte Carlo approach clearly outperforms standard techniques based on Markov chain exploration.
Graphical processing units and highdimensional optimization. arXiv:1003.3272v1
, 2009
"... ar ..."
(Show Context)
An adaptive interacting WangLandau algorithm for automatic density exploration
"... While statisticians are wellaccustomed to performing exploratory analysis in the modeling stage of an analysis, the notion of conducting preliminary generalpurpose exploratory analysis in the Monte Carlo stage (or more generally, the modelfitting stage) of an analysis is an area which we feel des ..."
Abstract

Cited by 8 (2 self)
 Add to MetaCart
While statisticians are wellaccustomed to performing exploratory analysis in the modeling stage of an analysis, the notion of conducting preliminary generalpurpose exploratory analysis in the Monte Carlo stage (or more generally, the modelfitting stage) of an analysis is an area which we feel deserves much further attention. Towards this aim, this paper proposes a generalpurpose algorithm for automatic density exploration. The proposed exploration algorithm combines and expands upon components from various adaptive Markov chain Monte Carlo methods, with the WangLandau algorithm at its heart. Additionally, the algorithm is run on interacting parallel chains – a feature which both decreases computational cost as well as stabilizes the algorithm, improving its ability to explore the density. Performance of this new parallel adaptive WangLandau (PAWL) algorithm is studied in several applications. Through a Bayesian variable selection example, the authors demonstrate the convergence gains obtained with interacting chains. The ability of the algorithm’s adaptive proposal to induce modejumping is illustrated through a Bayesian mixture modeling application. Lastly, through a 2D Ising model, the authors demonstrate the ability of the algorithm to overcome the high correlations encountered in spatial models. The appendices contain the full algorithmic description in pseudocode, a trimodal toy example and remarks on the convergence of the proposed algorithm. 1