Results 1  10
of
294
An Explicit Link between Gaussian Fields and Gaussian Markov random fields: the SPDE approach
 PREPRINTS IN MATHEMATICAL SCIENCES
, 2010
"... Continuously indexed Gaussian fields (GFs) is the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitive interpretation of its properties. On the computational side, GFs are hampered with the bign problem, ..."
Abstract

Cited by 115 (17 self)
 Add to MetaCart
Continuously indexed Gaussian fields (GFs) is the most important ingredient in spatial statistical modelling and geostatistics. The specification through the covariance function gives an intuitive interpretation of its properties. On the computational side, GFs are hampered with the bign problem, since the cost of factorising dense matrices is cubic in the dimension. Although the computational power today is alltimehigh, this fact seems still to be a computational bottleneck in applications. Along with GFs, there is the class of Gaussian Markov random fields (GMRFs) which are discretely indexed. The Markov property makes the involved precision matrix sparse which enables the use of numerical algorithms for sparse matrices, that for fields in R 2 only use the squareroot of the time required by general algorithms. The specification of a GMRF is through its full conditional distributions but its marginal properties are not transparent in such a parametrisation. In this paper, we show that using an approximate stochastic weak solution to (linear) stochastic partial differential equations (SPDEs), we can, for some GFs in the Matérn class, provide an explicit link, for any triangulation of R d, between GFs and GMRFs. The consequence is that we can take the best from the two worlds and do the modelling using GFs but do the computations using GMRFs. Perhaps more importantly,
Elliptical slice sampling
 JMLR: W&CP
"... Many probabilistic models introduce strong dependencies between variables using a latent multivariate Gaussian distribution or a Gaussian process. We present a new Markov chain Monte Carlo algorithm for performing inference in models with multivariate Gaussian priors. Its key properties are: 1) it h ..."
Abstract

Cited by 60 (8 self)
 Add to MetaCart
Many probabilistic models introduce strong dependencies between variables using a latent multivariate Gaussian distribution or a Gaussian process. We present a new Markov chain Monte Carlo algorithm for performing inference in models with multivariate Gaussian priors. Its key properties are: 1) it has simple, generic code applicable to many models, 2) it has no free parameters, 3) it works well for a variety of Gaussian process based models. These properties make our method ideal for use while model building, removing the need to spend time deriving and tuning updates for more complex algorithms.
Portfolio Allocation for Bayesian Optimization
"... Bayesian optimization with Gaussian processes has become an increasingly popular tool in the machine learning community. It is efficient and can be used when very little is known about the objective function, making it popular in expensive blackbox optimization scenarios. It uses Bayesian methods t ..."
Abstract

Cited by 23 (14 self)
 Add to MetaCart
(Show Context)
Bayesian optimization with Gaussian processes has become an increasingly popular tool in the machine learning community. It is efficient and can be used when very little is known about the objective function, making it popular in expensive blackbox optimization scenarios. It uses Bayesian methods to sample the objective efficiently using an acquisition function which incorporates the posterior estimate of the objective. However, there are several different parameterized acquisition functions in the literature, and it is often unclear which one to use. Instead of using a single acquisition function, we adopt a portfolio of acquisition functions governed by an online multiarmed bandit strategy. We propose several portfolio strategies, the best of which we call GPHedge, and show that this method outperforms the best individual acquisition function. We also provide a theoretical bound on the algorithm’s performance. 1
Variational inference in nonconjugate models
 Journal of Machine Learning Research
, 2013
"... Meanfield variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, meanfield methods approximately compute the posterior with a coordinateascent optimization algorithm. When the model is conditionally conjugate, the coordinate ..."
Abstract

Cited by 21 (4 self)
 Add to MetaCart
Meanfield variational methods are widely used for approximate posterior inference in many probabilistic models. In a typical application, meanfield methods approximately compute the posterior with a coordinateascent optimization algorithm. When the model is conditionally conjugate, the coordinate updates are easily derived and in closed form. However, many models of interest—like the correlated topic model and Bayesian logistic regression—are nonconjugate. In these models, meanfield methods cannot be directly applied and practitioners have had to develop variational algorithms on a casebycase basis. In this paper, we develop two generic methods for nonconjugate models, Laplace variational inference and delta method variational inference. Our methods have several advantages: they allow for easily derived variational algorithms with a wide class of nonconjugate models; they extend and unify some of the existing algorithms that have been derived for specific models; and they work well on realworld data sets. We studied our methods on the correlated topic model, Bayesian logistic regression, and hierarchical Bayesian logistic regression.
Bayesian computing with INLA: New features
 Computational Statistics & Data Analysis
, 2013
"... ar ..."
(Show Context)
Gaussian process regression with Studentt likelihood
"... In the Gaussian process regression the observation model is commonly assumed to be Gaussian, which is convenient in computational perspective. However, the drawback is that the predictive accuracy of the model can be significantly compromised if the observations are contaminated by outliers. A robus ..."
Abstract

Cited by 16 (4 self)
 Add to MetaCart
(Show Context)
In the Gaussian process regression the observation model is commonly assumed to be Gaussian, which is convenient in computational perspective. However, the drawback is that the predictive accuracy of the model can be significantly compromised if the observations are contaminated by outliers. A robust observation model, such as the Studentt distribution, reduces the influence of outlying observations and improves the predictions. The problem, however, is the analytically intractable inference. In this work, we discuss the properties of a Gaussian process regression model with the Studentt likelihood and utilize the Laplace approximation for approximate inference. We compare our approach to a variational approximation and a Markov chain Monte Carlo scheme, which utilize the commonly used scale mixture representation of the Studentt distribution. 1
and big data: The consensus monte carlo algorithm, Bayes 250
, 2013
"... A useful definition of “big data ” is data that is too big to comfortably process on a single machine, either because of processor, memory, or disk bottlenecks. Graphics processing units can alleviate the processor bottleneck, but memory or disk bottlenecks can only be eliminated by splitting data a ..."
Abstract

Cited by 16 (0 self)
 Add to MetaCart
(Show Context)
A useful definition of “big data ” is data that is too big to comfortably process on a single machine, either because of processor, memory, or disk bottlenecks. Graphics processing units can alleviate the processor bottleneck, but memory or disk bottlenecks can only be eliminated by splitting data across multiple machines. Communication between large numbers of machines is expensive (regardless of the amount of data being communicated), so there is a need for algorithms that perform distributed approximate Bayesian analyses with minimal communication. Consensus Monte Carlo operates by running a separate Monte Carlo algorithm on each machine, and then averaging individual Monte Carlo draws across machines. Depending on the model, the resulting draws can be nearly indistinguishable from the draws that would have been obtained by running a single machine algorithm for a very long time. Examples of consensus Monte Carlo are shown for simple models where singlemachine solutions are available, for large singlelayer hierarchical models, and for Bayesian additive regression trees (BART). 1
Approximate Bayesian Inference for Survival Models
, 2010
"... Bayesian analysis of timetoevent data, usually called survival analysis, has received increasing attention in the last years. In Coxtype models it allows to use information from the full likelihood instead of from a partial likelihood, so that the baseline hazard function and the model parameters ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
Bayesian analysis of timetoevent data, usually called survival analysis, has received increasing attention in the last years. In Coxtype models it allows to use information from the full likelihood instead of from a partial likelihood, so that the baseline hazard function and the model parameters can be jointly estimated. In general, Bayesian methods permit a full and exact posterior inference for any parameter or predictive quantity of interest. On the other side, Bayesian inference often relies on Markov Chain Monte Carlo (MCMC) techniques which, from the user point of view, may appear slow at delivering answers. In this paper, we show how a new inferential tool named Integrated Nested Laplace approximations (INLA) can be adapted and applied to many survival models making Bayesian analysis both fast and accurate without having to rely on MCMC based inference.
Approximate bayesian inference in spatial generalized linear mixed models
, 2006
"... In this paper we propose fast approximate methods for computing posterior marginals in spatial generalized linear mixed models. We consider the common geostatistical special case with a high dimensional latent spatial variable and observations at only a few known registration sites. Our methods of i ..."
Abstract

Cited by 14 (6 self)
 Add to MetaCart
In this paper we propose fast approximate methods for computing posterior marginals in spatial generalized linear mixed models. We consider the common geostatistical special case with a high dimensional latent spatial variable and observations at only a few known registration sites. Our methods of inference are deterministic, using no random sampling. We present two methods of approximate inference. The first is very fast to compute and via examples we find that this approximation is ’practically sufficient’. By this expression we mean that the results obtained by this approximate method do not show any bias or dispersion effects that might affect decision making. The other approximation is an improved version of the first one, and via examples we demonstrate that the inferred posterior approximations of this improved version are ’practically exact’. By this expression we mean that one would have to run Markov chain Monte Carlo simulations for longer than is typically done to detect any indications of bias or dispersion error effects in the approximate results. The two methods of approximate inference can help to expand the scope of geostatistical models, for instance in the context of model choice, model assessment, and sampling design. The