Results 1  10
of
28
Automatic online tuning for fast Gaussian summation
"... Many machine learning algorithms require the summation of Gaussian kernel functions, an expensive operation if implemented straightforwardly. Several methods have been proposed to reduce the computational complexity of evaluating such sums, including tree and analysis based methods. These achieve va ..."
Abstract

Cited by 35 (13 self)
 Add to MetaCart
(Show Context)
Many machine learning algorithms require the summation of Gaussian kernel functions, an expensive operation if implemented straightforwardly. Several methods have been proposed to reduce the computational complexity of evaluating such sums, including tree and analysis based methods. These achieve varying speedups depending on the bandwidth, dimension, and prescribed error, making the choice between methods difficult for machine learning tasks. We provide an algorithm that combines tree methods with the Improved Fast Gauss Transform (IFGT). As originally proposed the IFGT suffers from two problems: (1) the Taylor series expansion does not perform well for very low bandwidths, and (2) parameter selection is not trivial and can drastically affect performance and ease of use. We address the first problem by employing a tree data structure, resulting in four evaluation methods whose performance varies based on the distribution of sources and targets and input parameters such as desired accuracy and bandwidth. To solve the second problem, we present an online tuning approach that results in a black box method that automatically chooses the evaluation method and its parameters to yield the best performance for the input data, desired accuracy, and bandwidth. In addition, the new IFGT parameter selection approach allows for tighter error bounds. Our approach chooses the fastest method at negligible additional cost, and has superior performance in comparisons with previous approaches. 1
Rapid Deformable Object Detection using DualTree BranchandBound
"... In this work we use BranchandBound (BB) to efficiently detect objects with deformable part models. Instead of evaluating the classifier score exhaustively over image locations and scales, we use BB to focus on promising image locations. The core problem is to compute bounds that accommodate part d ..."
Abstract

Cited by 22 (4 self)
 Add to MetaCart
(Show Context)
In this work we use BranchandBound (BB) to efficiently detect objects with deformable part models. Instead of evaluating the classifier score exhaustively over image locations and scales, we use BB to focus on promising image locations. The core problem is to compute bounds that accommodate part deformations; for this we adapt the Dual Trees data structure [7] to our problem. We evaluate our approach using MixtureofDeformable Part Models [4]. We obtain exactly the same results but are 1020 times faster on average. We also develop a multipleobject detection variation of the system, where hypotheses for 20 categories are inserted in a common priority queue. For the problem of finding the strongest category in an image this results in a 100fold speedup. 1
Lineartime algorithms for pairwise statistical problems
 In Proc. of NIPS
, 2010
"... Several key computational bottlenecks in machine learning involve pairwise distance computations, including allnearestneighbors (finding the nearest neighbor(s) for each point, e.g. in manifold learning) and kernel summations (e.g. in kernel density estimation or kernel machines). We consider the ..."
Abstract

Cited by 16 (7 self)
 Add to MetaCart
(Show Context)
Several key computational bottlenecks in machine learning involve pairwise distance computations, including allnearestneighbors (finding the nearest neighbor(s) for each point, e.g. in manifold learning) and kernel summations (e.g. in kernel density estimation or kernel machines). We consider the general, bichromatic case for these problems, in addition to the scientific problem of Nbody simulation. In this paper we show for the first timeO(
Fast Highdimensional Kernel Summations Using the Monte Carlo Multipole
"... We propose a new fast Gaussian summation algorithm for highdimensional datasets with high accuracy. First, we extend the original fast multipoletype methods to use approximation schemes with both hard and probabilistic error. Second, we utilize a new data structure called subspace tree which maps ..."
Abstract

Cited by 15 (5 self)
 Add to MetaCart
(Show Context)
We propose a new fast Gaussian summation algorithm for highdimensional datasets with high accuracy. First, we extend the original fast multipoletype methods to use approximation schemes with both hard and probabilistic error. Second, we utilize a new data structure called subspace tree which maps each data point in the node to its lower dimensional mapping as determined by any linear dimension reduction method such as PCA. This new data structure is suitable for reducing the cost of each pairwise distance computation, the most dominant cost in many kernel methods. Our algorithm guarantees probabilistic relative error on each kernel sum, and can be applied to highdimensional Gaussian summations which are ubiquitous inside many kernel methods as the key computational bottleneck. We provide empirical speedup results on low to highdimensional datasets up to 89 dimensions. 1 Fast Gaussian Kernel Summation In this paper, we propose new computational techniques for efficiently approximating the following sum for each query point qi ∈ Q: Φ(qi, R) = ∑ e −qi−rj2 /(2h 2)
Faster Gaussian summation: Theory and experiment
 In In Proceedings of the Twentysecond Conference on Uncertainty in Artificial Intelligence (UAI
, 2006
"... We provide faster algorithms for the problem of Gaussian summation, which occurs in many machine learning methods. We develop two new extensions an O(Dp) Taylor expansion for the Gaussian kernel with rigorous error bounds and a new error control scheme integrating any arbitrary approximation m ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
(Show Context)
We provide faster algorithms for the problem of Gaussian summation, which occurs in many machine learning methods. We develop two new extensions an O(Dp) Taylor expansion for the Gaussian kernel with rigorous error bounds and a new error control scheme integrating any arbitrary approximation method within the best discretealgorithmic framework using adaptive hierarchical data structures. We rigorously evaluate these techniques empirically in the context of optimal bandwidth selection in kernel density estimation, revealing the strengths and weaknesses of current stateoftheart
A distributed kernel summation framework for generaldimension machine learning
 In SIAM International Conference on Data Mining 2012
, 2012
"... Kernel summations are a ubiquitous key computational bottleneck in many data analysis methods. In this paper, we attempt to marry, for the first time, the best relevant techniques in parallel computing, where kernel summations are in low dimensions, with the best generaldimension algorithms from th ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
Kernel summations are a ubiquitous key computational bottleneck in many data analysis methods. In this paper, we attempt to marry, for the first time, the best relevant techniques in parallel computing, where kernel summations are in low dimensions, with the best generaldimension algorithms from the machine learning literature. We provide the first distributed implementation of kernel summation framework that can utilize: 1) various types of deterministic and probabilistic approximations that may be suitable for low and highdimensional problems with a large number of data points; 2) any multidimensional binary tree using both distributed memory and shared memory parallelism; 3) a dynamic load balancing scheme to adjust work imbalances during the computation. Our hybrid MPI/OpenMP codebase has wide applicability in providing a general framework to accelerate the computation of many popular machine learning methods. Our experiments show scalability results for kernel density estimation on a synthetic tendimensional dataset containing over one billion points and a subset of the Sloan Digital Sky Survey Data up to 6,144 cores. 1
Efficient subset selection via the kernelized Rényi distance
"... With improved sensors, the amount of data available in many vision problems has increased dramatically and allows the use of sophisticated learning algorithms to perform inference on the data. However, since these algorithms scale with data size, pruning the data is sometimes necessary. The pruning ..."
Abstract

Cited by 6 (5 self)
 Add to MetaCart
With improved sensors, the amount of data available in many vision problems has increased dramatically and allows the use of sophisticated learning algorithms to perform inference on the data. However, since these algorithms scale with data size, pruning the data is sometimes necessary. The pruning procedure must be statistically valid and a representative subset of the data must be selected without introducing selection bias. Information theoretic measures have been used for sampling the data, retaining its original information content. We propose an efficient Rényi entropy based subset selection algorithm. The algorithm is first validated and then applied to two sample applications where machine learning and data pruning are used. In the first application, Gaussian process regression is used to learn object pose. Here it is shown that the algorithm combined with the subset selection is significantly more efficient. In the second application, our subset selection approach is used to replace vector quantization in a standard object recognition algorithm, and improvements are shown. 1.
GPUML: Graphical processors for speeding up kernel machines
"... Algorithms based on kernel methods play a central role in statistical machine learning. At their core are a number of linear algebra operations on matrices of kernel functions which take as arguments the training and testing data. These range from the simple matrixvector product, to more complex ma ..."
Abstract

Cited by 6 (3 self)
 Add to MetaCart
(Show Context)
Algorithms based on kernel methods play a central role in statistical machine learning. At their core are a number of linear algebra operations on matrices of kernel functions which take as arguments the training and testing data. These range from the simple matrixvector product, to more complex matrix decompositions, and iterative formulations of these. Often the algorithms scale quadratically or cubically, both in memory and operational complexity, and as data sizes increase, kernel methods scale poorly. We use parallelized approaches on a multicore graphical processor (GPU) to partially address this lack of scalability. GPUs are used to scale three different classes of problems, a simple kernelmatrixvector product, iterative solution of linear systems of kernel function and QR and Cholesky decomposition of kernel matrices. Application of these accelerated approaches in scaling several kernel based learning approaches are shown, and in each case substantial speedups are obtained. The core software is released as an open source package, GPUML.
Recent Advances and Trends in Largescale Kernel Methods
, 2009
"... Kernel methods such as the support vector machine are one of the most successful algorithms in modern machine learning. Their advantage is that linear algorithms are extended to nonlinear scenarios in a straightforward way by the use of the kernel trick. However, naive use of kernel methods is comp ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
Kernel methods such as the support vector machine are one of the most successful algorithms in modern machine learning. Their advantage is that linear algorithms are extended to nonlinear scenarios in a straightforward way by the use of the kernel trick. However, naive use of kernel methods is computationally expensive since the computational complexity typically scales cubically with respect to the number of training samples. In this article, we review recent advances in the kernel methods, with emphasis on scalability for massive problems.
Fast Gauss bilateral filtering
 Comput. Graph. Forum
, 2010
"... In spite of high computational complexity, the bilateral filter and its modifications and extensions have recently become very popular image and shape processing tools. In this paper, we propose a fast and accurate approximation of the bilateral filter. Our approach combines a dimension elevation t ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
(Show Context)
In spite of high computational complexity, the bilateral filter and its modifications and extensions have recently become very popular image and shape processing tools. In this paper, we propose a fast and accurate approximation of the bilateral filter. Our approach combines a dimension elevation trick with a Fast Gauss Transform. First we represent the bilateral filter as a convolution in a high dimensional space. Then the convolution is efficiently approximated by using space partitioning and Gaussian function expansions. Advantages of our approach include linear computational complexity, userspecified precision, and an ability to process high dimensional and nonuniformly sampled data. We demonstrate capabilities of the approach by considering its applications to the image and volume denoising and HDR tone mapping problems.