Results 1  10
of
31
An algorithm for finding best matches in logarithmic expected time
 ACM Transactions on Mathematical Software
, 1977
"... An algorithm and data structure are presented for searching a file containing N records, each described by k real valued keys, for the m closest matches or nearest neighbors to a given query record. The computation required to organize the file is proportional to kNlogN. The expected number of recor ..."
Abstract

Cited by 759 (2 self)
 Add to MetaCart
An algorithm and data structure are presented for searching a file containing N records, each described by k real valued keys, for the m closest matches or nearest neighbors to a given query record. The computation required to organize the file is proportional to kNlogN. The expected number of records examined in each search is independent of the file size. The expected computation to perform each search is proportionalto 1ogN. Empirical evidence suggests that except for very small files, this algorithm is considerably faster than other methods.
Adaptive local linear regression with application to printer color management
 IEEE Trans. on Image Processing
"... Abstract—Local learning methods, such as local linear regression and nearest neighbor classifiers, base estimates on nearby training samples, neighbors. Usually, the number of neighbors used in estimation is fixed to be a global “optimal ” value, chosen by cross validation. This paper proposes adapt ..."
Abstract

Cited by 8 (4 self)
 Add to MetaCart
(Show Context)
Abstract—Local learning methods, such as local linear regression and nearest neighbor classifiers, base estimates on nearby training samples, neighbors. Usually, the number of neighbors used in estimation is fixed to be a global “optimal ” value, chosen by cross validation. This paper proposes adapting the number of neighbors used for estimation to the local geometry of the data, without need for cross validation. The term enclosing neighborhood is introduced to describe a set of neighbors whose convex hull contains the test point when possible. It is proven that enclosing neighborhoods yield bounded estimation variance under some assumptions. Three such enclosing neighborhood definitions are presented: natural neighbors, natural neighbors inclusive, and enclosing kNN. The effectiveness of these neighborhood definitions with local linear regression is tested for estimating lookup tables for color management. Significant improvements in
Neural Networks for Density Estimation
 Advances in Neural Information Processing Systems (NIPS 98
, 1998
"... We introduce two new techniques for density estimation. Our approach poses the problem as a supervised learning task which can be performed using Neural Networks. We introduce a stochastic method for learning the cumulative distribution and an analogous deterministic technique. We demonstrate co ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
(Show Context)
We introduce two new techniques for density estimation. Our approach poses the problem as a supervised learning task which can be performed using Neural Networks. We introduce a stochastic method for learning the cumulative distribution and an analogous deterministic technique. We demonstrate convergence of our methods both theoretically and experimentally, and provide comparisons with the Parzen estimate. Our theoretical results demonstrate better convergence properties than the Parzen estimate. 1 Introduction and Background A majority of problems in science and engineering have to be modeled in a probabilistic manner. Even if the underlying phenomena are inherently deterministic, the complexity of these phenomena often makes a probabilistic formulation the only feasible approach from the computational point of view. Although quantities such as the mean, the variance, and possibly higher order moments of a random variable have often been sufficient to characterize a particula...
Classification based on hybridization of parametric and nonparametric classifiers
"... Parametric methods of classification assume specific parametric models for competing population densities (e.g., Gaussian population densities lead to linear and quadratic discriminant analysis), and they work well when these model assumptions are valid. Violation in one or more of these parametric ..."
Abstract

Cited by 4 (0 self)
 Add to MetaCart
(Show Context)
Parametric methods of classification assume specific parametric models for competing population densities (e.g., Gaussian population densities lead to linear and quadratic discriminant analysis), and they work well when these model assumptions are valid. Violation in one or more of these parametric model assumptions often leads to a poor classifier. On the other hand, nonparametric classifiers (e.g., nearest neighbor and kernel based classifiers) are more flexible and free from parametric model assumptions. But statistical instability of these classifiers may lead to poor performance when we have small training samples. Nonparametric methods, however, do not use any parametric structure of population densities. So, even when one has some additional information about population densities, that important information is not used to modify the nonparametric classification rule. This article makes an attempt to develop some hybrid classification methods by combining the strength of parametric 1 and nonparametric approaches. We use some simulated examples and some benchmark data sets to examine the performance of these hybrid discriminant analysis tools. Asymptotic results on their misclassification rates have been derived under appropriate regularity conditions. Index Terms: Bayes risk, kernel density estimation, kernel discriminant analysis, linear discriminant analysis, misclassification rate, multiscale analysis, nearest neighbor density estimation, nearest neighbor classification, quadratic discriminant analysis. 1
Locally linear metric adaptation with application to semisupervised clustering and image retrieval
, 2005
"... ..."
Kernelbased metric adaptation with pairwise constraints
 In International Conference on Machine Learning and Cybernetics (ICMLC
, 2005
"... Abstract. Many supervised and unsupervised learning algorithms depend on the choice of an appropriate distance metric. While metric learning for supervised learning tasks has a long history, extending it to learning tasks with weaker supervisory information has only been studied very recently. In pa ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Abstract. Many supervised and unsupervised learning algorithms depend on the choice of an appropriate distance metric. While metric learning for supervised learning tasks has a long history, extending it to learning tasks with weaker supervisory information has only been studied very recently. In particular, several methods have been proposed for semisupervised metric learning based on pairwise (dis)similarity information. In this paper, we propose a kernelbased approach for nonlinear metric learning, which performs locally linear translation in the kernelinduced feature space. We formulate the metric learning problem as a kernel learning problem and solve it efficiently by kernel matrix adaptation. Experimental results based on synthetic and realworld data sets show that our approach is promising for semisupervised metric learning. 1
Selection of Distance Metrics and Feature Subsets for kNearest Neighbor Classifiers
, 1997
"... The knearest neighbor (kNN) classifier is a popular and effective method for associating a feature vector with a unique element in a known, finite set of classes. A common choice for the distance metric used in kNN classification is the quadratic distance Q(x; A; y) = (x \Gamma y) 0 A(x\Gammay), ..."
Abstract

Cited by 3 (1 self)
 Add to MetaCart
The knearest neighbor (kNN) classifier is a popular and effective method for associating a feature vector with a unique element in a known, finite set of classes. A common choice for the distance metric used in kNN classification is the quadratic distance Q(x; A; y) = (x \Gamma y) 0 A(x\Gammay), where x and y are nvectors of features, A is a symmetric n\Thetan matrix, and prime denotes transpose. For finite sets of training samples the choice of matrix A is important in optimizing classifier performance. We show that A can be approximately optimized via gradient descent on a sigmoidally smoothed estimate of the classifier's probability of error. We describe an algorithm for performing the metric selection, and compare the performance of our method with that of other methods. We demonstrate that adding noise during the descent process can reduce the effects of overfitting. We further suggest how feature subset selection can be treated as a special case of this metric selection. C...
Efficient NearestNeighbour Searches Using Weighted Euclidean Metrics
 Advances in Databases, BNCOD16 LNCS 1405
, 1998
"... Building an index tree is a common approach to speed up the k nearest neighbour search in large databases of manydimensional records. Many applications require varying distance metrics by putting a weight on different dimensions. The main problem with k nearest neighbour searches using weighted ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Building an index tree is a common approach to speed up the k nearest neighbour search in large databases of manydimensional records. Many applications require varying distance metrics by putting a weight on different dimensions. The main problem with k nearest neighbour searches using weighted euclidean metrics in a high dimensional space is whether the searches can be done efficiently We present a solution to this problem which uses the bounding rectangle of the nearestneighbour disk instead of using the disk directly. The algorithm is able to perform nearestneighbour searches using distance metrics different from the metric used to build the search tree without having to rebuild the tree. It is efficient for weighted euclidean distance and extensible to higher dimensions.
Estimation of nonlinear functionals of densities with confidence
, 2012
"... This paper introduces a class of knearest neighbor (kNN) estimators called bipartite plugin (BPI) estimators for estimating integrals of nonlinear functions of a probability density, such as Shannon entropy and Rényi entropy. The density is assumed to be smooth, have bounded support, and be unif ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
This paper introduces a class of knearest neighbor (kNN) estimators called bipartite plugin (BPI) estimators for estimating integrals of nonlinear functions of a probability density, such as Shannon entropy and Rényi entropy. The density is assumed to be smooth, have bounded support, and be uniformly bounded from below on this set. Unlike previous kNN estimators of nonlinear density functionals, the proposed estimator uses datasplitting and boundary correction to achieve lower mean square error. Specifically, we assume that T i.i.d. samples Xi ∈ R d from the density are split into two pieces of cardinality M and N respectively, with M samples used for computing a knearestneighbor density estimate and the remaining N samples used for empirical estimation of the integral of the density functional. By studying the statistical properties of kNN balls, explicit rates for the bias and variance of the BPI estimator are derived in terms of the sample size, the dimension of the samples and the underlying probability distribution. Based on these results, it is possible to specify optimal choice of tuning parameters M/T, k for maximizing the rate of decrease of the mean square error (MSE). The resultant optimized BPI estimator converges faster and achieves lower mean squared error than previous kNN entropy estimators. In addition, a central limit theorem is established for the BPI estimator that allows us to specify tight asymptotic confidence intervals.
Relaxational metric adaptation and its application to semisupervised clustering and contentbased image retrieval
 Pattern Recognition
"... The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
(Show Context)
The performance of many supervised and unsupervised learning algorithms is very sensitive to the choice of an appropriate distance metric. Previous work in metric learning and adaptation has mostly been focused on classification tasks by making use of class label information. In standard clustering tasks, however, class label information is not available. In order to adapt the metric to improve the clustering results, some background knowledge or side information is needed. One useful type of side information is in the form of pairwise similarity or dissimilarity information. Recently, some novel methods (e.g., the parametric method proposed by Xing et al.) for learning global metrics based on pairwise side information have been shown to demonstrate promising results. In this paper, we propose a nonparametric method, called relaxational metric adaptation (RMA), for the same metric adaptation problem. While RMA is local in the sense that it allows locally adaptive metrics, it is also global because even patterns not in the vicinity can have longrange effects on the metric adaptation process. Experimental results for semisupervised clustering based on both simulated and realworld data sets show that RMA outperforms Xing Preprint submitted to Elsevier Science 28 July 2005 et al.’s method under most situations. Besides applying RMA to semisupervised learning, we have also used it to improve the performance of contentbased image retrieval systems through metric adaptation. Experimental results based on two realworld image databases show that RMA significantly outperforms other methods in improving the image retrieval performance.