Results 1  10
of
70
Automatic online tuning for fast Gaussian summation
"... Many machine learning algorithms require the summation of Gaussian kernel functions, an expensive operation if implemented straightforwardly. Several methods have been proposed to reduce the computational complexity of evaluating such sums, including tree and analysis based methods. These achieve va ..."
Abstract

Cited by 35 (13 self)
 Add to MetaCart
(Show Context)
Many machine learning algorithms require the summation of Gaussian kernel functions, an expensive operation if implemented straightforwardly. Several methods have been proposed to reduce the computational complexity of evaluating such sums, including tree and analysis based methods. These achieve varying speedups depending on the bandwidth, dimension, and prescribed error, making the choice between methods difficult for machine learning tasks. We provide an algorithm that combines tree methods with the Improved Fast Gauss Transform (IFGT). As originally proposed the IFGT suffers from two problems: (1) the Taylor series expansion does not perform well for very low bandwidths, and (2) parameter selection is not trivial and can drastically affect performance and ease of use. We address the first problem by employing a tree data structure, resulting in four evaluation methods whose performance varies based on the distribution of sources and targets and input parameters such as desired accuracy and bandwidth. To solve the second problem, we present an online tuning approach that results in a black box method that automatically chooses the evaluation method and its parameters to yield the best performance for the input data, desired accuracy, and bandwidth. In addition, the new IFGT parameter selection approach allows for tighter error bounds. Our approach chooses the fastest method at negligible additional cost, and has superior performance in comparisons with previous approaches. 1
Rapid Evaluation of Multiple Density Models
 In Artificial Iintelligence and Statistics
, 2003
"... When highlyaccurate and/or assumptionfree density estimation is needed, nonparametric methods are often called upon  most notably the popular kernel density estimation (KDE) method. However, the practitioner is instantly faced with the formidable computational cost of KDE for appreciable da ..."
Abstract

Cited by 31 (4 self)
 Add to MetaCart
(Show Context)
When highlyaccurate and/or assumptionfree density estimation is needed, nonparametric methods are often called upon  most notably the popular kernel density estimation (KDE) method. However, the practitioner is instantly faced with the formidable computational cost of KDE for appreciable dataset sizes, which becomes even more prohibitive when many models with different kernel scales (bandwidths) must be evaluated  this is necessary for finding the optimal model, among other reasons. In previous work we presented an algorithm for fast KDE which addresses large dataset sizes and large dimensionalities, but assumes only a single bandwidth. In this paper we present a generalization of that algorithm allowing multiple models with different bandwidths to be computed simultaneously, in substantially less time than either running the singlebandwidth algorithm for each model independently, or running the standard exhaustive method. We show examples of computing the likelihood curve for 100,000 data and 100 models ranging across 3 orders of magnitude in scale, in minutes or seconds.
DualTree Fast Gauss Transforms
 Advances in Neural Information Processing Systems 18
, 2006
"... In previous work we presented an efficient approach to computing kernel summations which arise in many machine learning methods such as kernel density estimation. This approach, dualtree recursion with finitedifference approximation, generalized existing methods for similar problems arising in c ..."
Abstract

Cited by 28 (5 self)
 Add to MetaCart
In previous work we presented an efficient approach to computing kernel summations which arise in many machine learning methods such as kernel density estimation. This approach, dualtree recursion with finitedifference approximation, generalized existing methods for similar problems arising in computational physics in two ways appropriate for statistical problems: toward distribution sensitivity and general dimension, partly by avoiding series expansions. While this proved to be the fastest practical method for multivariate kernel density estimation at the optimal bandwidth, it is much less efficient at largerthanoptimal bandwidths. In this work, we explore the extent to which the dualtree approach can be integrated with multipolelike Hermite expansions in order to achieve reasonable efficiency across all bandwidth scales, though only for low dimensionalities. In the process, we derive and demonstrate the first truly hierarchical fast Gauss transforms, effectively combining the best tools from discrete algorithms and continuous approximation theory. 1
Rapid Deformable Object Detection using DualTree BranchandBound
"... In this work we use BranchandBound (BB) to efficiently detect objects with deformable part models. Instead of evaluating the classifier score exhaustively over image locations and scales, we use BB to focus on promising image locations. The core problem is to compute bounds that accommodate part d ..."
Abstract

Cited by 22 (4 self)
 Add to MetaCart
(Show Context)
In this work we use BranchandBound (BB) to efficiently detect objects with deformable part models. Instead of evaluating the classifier score exhaustively over image locations and scales, we use BB to focus on promising image locations. The core problem is to compute bounds that accommodate part deformations; for this we adapt the Dual Trees data structure [7] to our problem. We evaluate our approach using MixtureofDeformable Part Models [4]. We obtain exactly the same results but are 1020 times faster on average. We also develop a multipleobject detection variation of the system, where hypotheses for 20 categories are inserted in a common priority queue. For the problem of finding the strongest category in an image this results in a 100fold speedup. 1
Fast Gaussian Process Methods for Point Process Intensity Estimation
"... Point processes are difficult to analyze because they provide only a sparse and noisy observation of the intensity function driving the process. Gaussian Processes offer an attractive framework within which to infer underlying intensity functions. The result of this inference is a continuous functio ..."
Abstract

Cited by 18 (2 self)
 Add to MetaCart
(Show Context)
Point processes are difficult to analyze because they provide only a sparse and noisy observation of the intensity function driving the process. Gaussian Processes offer an attractive framework within which to infer underlying intensity functions. The result of this inference is a continuous function defined across time that is typically more amenable to analytical efforts. However, a naive implementation will become computationally infeasible in any problem of reasonable size, both in memory and run time requirements. We demonstrate problem specific methods for a class of renewal processes that eliminate the memory burden and reduce the solve time by orders of magnitude. 1.
Toward practical N² Monte Carlo: The marginal particle filter
 IN PROCEEDINGS OF THE 21ST CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE. AUAI
, 2005
"... Sequential Monte Carlo techniques are useful for state estimation in nonlinear, nonGaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time ..."
Abstract

Cited by 14 (1 self)
 Add to MetaCart
(Show Context)
Sequential Monte Carlo techniques are useful for state estimation in nonlinear, nonGaussian dynamic models. These methods allow us to approximate the joint posterior distribution using sequential importance sampling. In this framework, the dimension of the target distribution grows with each time step, thus it is necessary to introduce some resampling steps to ensure that the estimates provided by the algorithm have a reasonable variance. In many applications, we are only interested in the marginal filtering distribution which is defined on a space of fixed dimension. We present a Sequential Monte Carlo algorithm called the Marginal Particle Filter which operates directly on the marginal distribution, hence avoiding having to perform importance sampling on a space of growing dimension. Using this idea, we also derive an improved version of the auxiliary particle filter. We show theoretic and empirical results which demonstrate a reduction in variance over conventional particle filtering, and present techniques for reducing the cost of the marginal particle filter with N particles from O(N²) to O(N log N).
Lineartime algorithms for pairwise statistical problems
 In Proc. of NIPS
, 2010
"... Several key computational bottlenecks in machine learning involve pairwise distance computations, including allnearestneighbors (finding the nearest neighbor(s) for each point, e.g. in manifold learning) and kernel summations (e.g. in kernel density estimation or kernel machines). We consider the ..."
Abstract

Cited by 13 (6 self)
 Add to MetaCart
(Show Context)
Several key computational bottlenecks in machine learning involve pairwise distance computations, including allnearestneighbors (finding the nearest neighbor(s) for each point, e.g. in manifold learning) and kernel summations (e.g. in kernel density estimation or kernel machines). We consider the general, bichromatic case for these problems, in addition to the scientific problem of Nbody simulation. In this paper we show for the first timeO(
Fast Highdimensional Kernel Summations Using the Monte Carlo Multipole
"... We propose a new fast Gaussian summation algorithm for highdimensional datasets with high accuracy. First, we extend the original fast multipoletype methods to use approximation schemes with both hard and probabilistic error. Second, we utilize a new data structure called subspace tree which maps ..."
Abstract

Cited by 13 (4 self)
 Add to MetaCart
(Show Context)
We propose a new fast Gaussian summation algorithm for highdimensional datasets with high accuracy. First, we extend the original fast multipoletype methods to use approximation schemes with both hard and probabilistic error. Second, we utilize a new data structure called subspace tree which maps each data point in the node to its lower dimensional mapping as determined by any linear dimension reduction method such as PCA. This new data structure is suitable for reducing the cost of each pairwise distance computation, the most dominant cost in many kernel methods. Our algorithm guarantees probabilistic relative error on each kernel sum, and can be applied to highdimensional Gaussian summations which are ubiquitous inside many kernel methods as the key computational bottleneck. We provide empirical speedup results on low to highdimensional datasets up to 89 dimensions. 1 Fast Gaussian Kernel Summation In this paper, we propose new computational techniques for efficiently approximating the following sum for each query point qi ∈ Q: Φ(qi, R) = ∑ e −qi−rj2 /(2h 2)
A fast algorithm for learning a ranking function from large scale data sets
, 2007
"... ..."
(Show Context)
Fast nonparametric Bayesian inference on infinite trees
 In Proc. 15th International Conference on Artificial Intelligence and Statistics (AISTATS2005
, 2005
"... Given i.i.d. data from an unknown distribution, we consider the problem of predicting future items. An adaptive way to estimate the probability density is to recursively subdivide the domain to an appropriate datadependent granularity. A Bayesian would assign a dataindependent prior probability to ..."
Abstract

Cited by 10 (5 self)
 Add to MetaCart
Given i.i.d. data from an unknown distribution, we consider the problem of predicting future items. An adaptive way to estimate the probability density is to recursively subdivide the domain to an appropriate datadependent granularity. A Bayesian would assign a dataindependent prior probability to “subdivide”, which leads to a prior over infinite(ly many) trees. We derive an exact, fast, and simple inference algorithm for such a prior, for the data evidence, the predictive distribution, the effective model dimension, and other quantities. 1