Results 1  10
of
262
Sequential Monte Carlo Samplers
, 2002
"... In this paper, we propose a general algorithm to sample sequentially from a sequence of probability distributions known up to a normalizing constant and defined on a common space. A sequence of increasingly large artificial joint distributions is built; each of these distributions admits a marginal ..."
Abstract

Cited by 303 (44 self)
 Add to MetaCart
In this paper, we propose a general algorithm to sample sequentially from a sequence of probability distributions known up to a normalizing constant and defined on a common space. A sequence of increasingly large artificial joint distributions is built; each of these distributions admits a marginal which is a distribution of interest. To sample from these distributions, we use sequential Monte Carlo methods. We show that these methods can be interpreted as interacting particle approximations of a nonlinear FeynmanKac flow in distribution space. One interpretation of the FeynmanKac flow corresponds to a nonlinear Markov kernel admitting a specified invariant distribution and is a natural nonlinear extension of the standard MetropolisHastings algorithm. Many theoretical results have already been established for such flows and their particle approximations. We demonstrate the use of these algorithms through simulation.
A tutorial on particle filtering and smoothing: fifteen years later
 OXFORD HANDBOOK OF NONLINEAR FILTERING
, 2011
"... Optimal estimation problems for nonlinear nonGaussian statespace models do not typically admit analytic solutions. Since their introduction in 1993, particle filtering methods have become a very popular class of algorithms to solve these estimation problems numerically in an online manner, i.e. r ..."
Abstract

Cited by 214 (15 self)
 Add to MetaCart
(Show Context)
Optimal estimation problems for nonlinear nonGaussian statespace models do not typically admit analytic solutions. Since their introduction in 1993, particle filtering methods have become a very popular class of algorithms to solve these estimation problems numerically in an online manner, i.e. recursively as observations become available, and are now routinely used in fields as diverse as computer vision, econometrics, robotics and navigation. The objective of this tutorial is to provide a complete, uptodate survey of this field as of 2008. Basic and advanced particle methods for filtering as well as smoothing are presented.
Probabilistic Methods for Finding People
 INTERNATIONAL JOURNAL OF COMPUTER VISION
, 2001
"... Finding people in pictures presents a particularly difficult object recognition problem. We show how to find people by finding candidate body segments, and then constructing assemblies of segments that are consistent with the constraints on the appearance of a person that result from kinematic prope ..."
Abstract

Cited by 126 (2 self)
 Add to MetaCart
Finding people in pictures presents a particularly difficult object recognition problem. We show how to find people by finding candidate body segments, and then constructing assemblies of segments that are consistent with the constraints on the appearance of a person that result from kinematic properties. Since a reasonable model of a person requires at least nine segments, it is not possible to inspect every group, due to the huge combinatorial complexity. We propose two
Evaluation methods for topic models
 In ICML
, 2009
"... A natural evaluation metric for statistical topic models is the probability of heldout documents given a trained model. While exact computation of this probability is intractable, several estimators for this probability have been used in the topic modeling literature, including the harmonic mean me ..."
Abstract

Cited by 111 (10 self)
 Add to MetaCart
(Show Context)
A natural evaluation metric for statistical topic models is the probability of heldout documents given a trained model. While exact computation of this probability is intractable, several estimators for this probability have been used in the topic modeling literature, including the harmonic mean method and empirical likelihood method. In this paper, we demonstrate experimentally that commonlyused methods are unlikely to accurately estimate the probability of heldout documents, and propose two alternative methods that are both accurate and efficient. 1.
Estimating Recombination Rates from Population Genetic Data
, 2000
"... We introduce a new method for estimating recombination rates from population genetic data. The method uses a computationallyintensive statistical procedure (importance sampling) to calculate the likelihood under a coalescentbased model. Detailed comparisons of the new algorithm with two existing m ..."
Abstract

Cited by 90 (11 self)
 Add to MetaCart
We introduce a new method for estimating recombination rates from population genetic data. The method uses a computationallyintensive statistical procedure (importance sampling) to calculate the likelihood under a coalescentbased model. Detailed comparisons of the new algorithm with two existing methods (one based on importance sampling and one based on MCMC) show it to be substantially more efficient. (The improvement over the existing importance sampling scheme is typically by four orders of magnitude.) The existing approaches not infrequently led to misleading results on the problems we investigated. We also performed a simulation study to look at the properties of the maximum likelihood estimator (mle) of the recombination rate, and its robustness to misspecification of the demographic model.
On the Quantitative Analysis of Deep Belief Networks
"... Deep Belief Networks (DBN’s) are generative models that contain many layers of hidden variables. Efficient greedy algorithms for learning and approximate inference have allowed these models to be applied successfully in many application domains. The main building block of a DBN is a bipartite undire ..."
Abstract

Cited by 84 (17 self)
 Add to MetaCart
(Show Context)
Deep Belief Networks (DBN’s) are generative models that contain many layers of hidden variables. Efficient greedy algorithms for learning and approximate inference have allowed these models to be applied successfully in many application domains. The main building block of a DBN is a bipartite undirected graphical model called a restricted Boltzmann machine (RBM). Due to the presence of the partition function, model selection, complexity control, and exact maximum likelihood learning in RBM’s are intractable. We show that Annealed Importance Sampling (AIS) can be used to efficiently estimate the partition function of an RBM, and we present a novel AIS scheme for comparing RBM’s with different architectures. We further show how an AIS estimator, along with approximate inference, can be used to estimate a lower bound on the logprobability that a DBN model with multiple hidden layers assigns to the test data. This is, to our knowledge, the first step towards obtaining quantitative results that would allow us to directly assess the performance of Deep Belief Networks as generative models of data. 1.
An Adaptive Sequential Monte Carlo Method for Approximate Bayesian Computation
, 2008
"... Approximate Bayesian computation (ABC) is a popular approach to address inference problems where the likelihood function is intractable, or expensive to calculate. To improve over Markov chain Monte Carlo (MCMC) implementations of ABC, the use of sequential Monte Carlo (SMC) methods has recently bee ..."
Abstract

Cited by 77 (13 self)
 Add to MetaCart
Approximate Bayesian computation (ABC) is a popular approach to address inference problems where the likelihood function is intractable, or expensive to calculate. To improve over Markov chain Monte Carlo (MCMC) implementations of ABC, the use of sequential Monte Carlo (SMC) methods has recently been suggested. Effective SMC algorithms that are currently available for ABC have a computational complexity that is quadratic in the number of Monte Carlo samples [4, 17, 19, 21] and require the careful choice of simulation parameters. In this article an adaptive SMC algorithm is proposed which admits a computational complexity that is linear in the number of samples and determines onthefly the simulation parameters. We demonstrate our algorithm on a toy example and a population genetics example.
Assessing approximate inference for binary Gaussian process classification
 Journal of Machine Learning Research
, 2005
"... Gaussian process priors can be used to define flexible, probabilistic classification models. Unfortunately exact Bayesian inference is analytically intractable and various approximation techniques have been proposed. In this work we review and compare Laplace’s method and Expectation Propagation for ..."
Abstract

Cited by 76 (3 self)
 Add to MetaCart
Gaussian process priors can be used to define flexible, probabilistic classification models. Unfortunately exact Bayesian inference is analytically intractable and various approximation techniques have been proposed. In this work we review and compare Laplace’s method and Expectation Propagation for approximate Bayesian inference in the binary Gaussian process classification model. We present a comprehensive comparison of the approximations, their predictive performance and marginal likelihood estimates to results obtained by MCMC sampling. We explain theoretically and corroborate empirically the advantages of Expectation Propagation compared to Laplace’s method. Keywords: Gaussian process priors, probabilistic classification, Laplace’s approximation, expectation propagation, marginal likelihood, evidence, MCMC
On the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods
 Journal of Computational and Graphical Statistics
, 2010
"... We present a casestudy on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are selfcontained parallel computational devices that can be housed in conventional desktop and la ..."
Abstract

Cited by 74 (11 self)
 Add to MetaCart
We present a casestudy on the utility of graphics cards to perform massively parallel simulation of advanced Monte Carlo methods. Graphics cards, containing multiple Graphics Processing Units (GPUs), are selfcontained parallel computational devices that can be housed in conventional desktop and laptop computers. For certain classes of Monte Carlo algorithms they offer massively parallel simulation, with the added advantage over conventional distributed multicore processors that they are cheap, easily accessible, easy to maintain, easy to code, dedicated local devices with low power consumption. On a canonical set of stochastic simulation examples including populationbased Markov chain Monte Carlo methods and Sequential Monte Carlo methods, we find speedups from 35 to 500 fold over conventional singlethreaded computer code. Our findings suggest that GPUs have the potential to facilitate the growth of statistical modelling into complex data rich domains through the availability of cheap and accessible manycore computation. We believe the speedup we observe should motivate wider
Replicated softmax: an undirected topic model
 In Advances in Neural Information Processing Systems
"... We introduce a twolayer undirected graphical model, called a “Replicated Softmax”, that can be used to model and automatically extract lowdimensional latent semantic representations from a large unstructured collection of documents. We present efficient learning and inference algorithms for this m ..."
Abstract

Cited by 67 (14 self)
 Add to MetaCart
(Show Context)
We introduce a twolayer undirected graphical model, called a “Replicated Softmax”, that can be used to model and automatically extract lowdimensional latent semantic representations from a large unstructured collection of documents. We present efficient learning and inference algorithms for this model, and show how a MonteCarlo based method, Annealed Importance Sampling, can be used to produce an accurate estimate of the logprobability the model assigns to test data. This allows us to demonstrate that the proposed model is able to generalize much better compared to Latent Dirichlet Allocation in terms of both the logprobability of heldout documents and the retrieval accuracy. 1