• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

Distance metric learning for large margin nearest neighbor classification. NIPS (2005)

by K Q Weinberger, J Blitzer, L K Saul
Add To MetaCart

Tools

Sorted by:
Results 1 - 10 of 695
Next 10 →

Information-theoretic metric learning

by Jason Davis, Brian Kulis, Suvrit Sra, Inderjit Dhillon - in NIPS 2006 Workshop on Learning to Compare Examples , 2007
"... We formulate the metric learning problem as that of minimizing the differential relative entropy between two multivariate Gaussians under constraints on the Mahalanobis distance function. Via a surprising equivalence, we show that this problem can be solved as a low-rank kernel learning problem. Spe ..."
Abstract - Cited by 359 (15 self) - Add to MetaCart
We formulate the metric learning problem as that of minimizing the differential relative entropy between two multivariate Gaussians under constraints on the Mahalanobis distance function. Via a surprising equivalence, we show that this problem can be solved as a low-rank kernel learning problem. Specifically, we minimize the Burg divergence of a low-rank kernel to an input kernel, subject to pairwise distance constraints. Our approach has several advantages over existing methods. First, we present a natural information-theoretic formulation for the problem. Second, the algorithm utilizes the methods developed by Kulis et al. [6], which do not involve any eigenvector computation; in particular, the running time of our method is faster than most existing techniques. Third, the formulation offers insights into connections between metric learning and kernel learning. 1
(Show Context)

Citation Context

...ghly problem-specific and ultimately dictates the success—or failure—of the learning algorithm. To this end, there have been several recent approaches that attempt to learn distance functions, e.g., (=-=Weinberger et al., 2005-=-; Xing et al., 2002; Globerson & Roweis, 2005; Shalev-Shwartz et al., 2004). These methods work by exploiting distance information that is intrinsically available in many learning settings. For exampl...

Is that you? Metric learning approaches for face identification

by Matthieu Guillaumin, Jakob Verbeek, Cordelia Schmid - In ICCV , 2009
"... Face identification is the problem of determining whether two face images depict the same person or not. This is difficult due to variations in scale, pose, lighting, background, expression, hairstyle, and glasses. In this paper we present two methods for learning robust distance measures: (a) a log ..."
Abstract - Cited by 159 (8 self) - Add to MetaCart
Face identification is the problem of determining whether two face images depict the same person or not. This is difficult due to variations in scale, pose, lighting, background, expression, hairstyle, and glasses. In this paper we present two methods for learning robust distance measures: (a) a logistic discriminant approach which learns the metric from a set of labelled image pairs (LDML) and (b) a nearest neighbour approach which computes the probability for two images to belong to the same class (MkNN). We evaluate our approaches on the Labeled Faces in the Wild data set, a large and very challenging data set of faces from Yahoo! News. The evaluation protocol for this data set defines a restricted setting, where a fixed set of positive and negative image pairs is given, as well as an unrestricted one, where faces are labelled by their identity. We are the first to present results for the unrestricted setting, and show that our methods benefit from this richer training data, much more so than the current state-of-the-art method. Our results of 79.3 % and 87.5 % correct for the restricted and unrestricted setting respectively, significantly improve over the current state-of-the-art result of 78.5%. Confidence scores obtained for face identification can be used for many applications e.g. clustering or recognition from a single training example. We show that our learned metrics also improve performance for these tasks. 1.
(Show Context)

Citation Context

...obability that the two faces are the same person, i.e. marginalising over who that exactly is. For this second method we also use a learned metric, albeit one that is optimised for kNN classification =-=[23]-=-. 1Metric learning has received a lot of attention, for recent work in this area see e.g. [1, 6, 9, 10, 23, 25]. Most methods learn a Mahalanobis metric based on an objective function defined by mean...

Learning globally-consistent local distance functions for shape-based image retrieval and classification

by Andrea Frome, Fei Sha, Yoram Singer, Jitendra Malik - In ICCV , 2007
"... We address the problem of visual category recognition by learning an image-to-image distance function that attempts to satisfy the following property: the distance between images from the same category should be less than the distance between images from different categories. We use patch-based feat ..."
Abstract - Cited by 149 (3 self) - Add to MetaCart
We address the problem of visual category recognition by learning an image-to-image distance function that attempts to satisfy the following property: the distance between images from the same category should be less than the distance between images from different categories. We use patch-based feature vectors common in object recognition work as a basis for our image-to-image distance functions. Our large-margin formulation for learning the distance functions is similar to formulations used in the machine learning literature on distance metric learning, however we differ in that we learn local distance functions— a different parameterized function for every image of our training set—whereas typically a single global distance function is learned. This was a novel approach first introduced in Frome, Singer, & Malik, NIPS 2006. In that work we learned the local distance functions independently, and the outputs of these functions could not be compared at test time without the use of additional heuristics or training. Here we introduce a different approach that has the advantage that it learns distance functions that are globally consistent in that they can be directly compared for purposes of retrieval and classification. The output of the learning algorithm are weights assigned to the image features, which is intuitively appealing in the computer vision setting: some features are more salient than others, and which are more salient depends on the category, or image, being considered. We train and test using the Caltech 101 object recognition benchmark. Using fifteen training images per category, we achieved a mean recognition rate of 63.2 % and
(Show Context)

Citation Context

...validation. The objective function and constraints of our largemargin formulation are the same as those in [18], which is part of a larger recent body of work in metric learning, also including [24], =-=[23]-=-, and [7]. In this line of work, the inputs x are points in some metric feature space, and the goal is to learn the matrix A which parameterizes a Mahalanobis distance of the form (x −x ′ )A(x −x ′ )....

Distance Metric Learning: A Comprehensive Survey

by Liu Yang , 2006
"... ..."
Abstract - Cited by 127 (13 self) - Add to MetaCart
Abstract not found

Dimensionality reduction of multimodal labeled data by local Fisher discriminant analysis

by Masashi Sugiyama, Sam Roweis - Journal of Machine Learning Research , 2007
"... Reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high-dimensional data analysis. Fisher discriminant analysis (FDA) is a traditional technique for supervised dimensionality reduction, but it tends to give undesired results if samples in a ..."
Abstract - Cited by 124 (12 self) - Add to MetaCart
Reducing the dimensionality of data without losing intrinsic information is an important preprocessing step in high-dimensional data analysis. Fisher discriminant analysis (FDA) is a traditional technique for supervised dimensionality reduction, but it tends to give undesired results if samples in a class are multimodal. An unsupervised dimensionality reduction method called localitypreserving projection (LPP) can work well with multimodal data due to its locality preserving property. However, since LPP does not take the label information into account, it is not necessarily useful in supervised learning scenarios. In this paper, we propose a new linear supervised dimensionality reduction method called local Fisher discriminant analysis (LFDA), which effectively combines the ideas of FDA and LPP. LFDA has an analytic form of the embedding transformation and the solution can be easily computed just by solving a generalized eigenvalue problem. We demonstrate the practical usefulness and high scalability of the LFDA method in data visualization and classification tasks through extensive simulation studies. We also show that LFDA can be extended to non-linear dimensionality reduction scenarios by applying the kernel trick.
(Show Context)

Citation Context

.... (12). Therefore, the solution T LFDA is not unique—the range of the transforis uniquely determined, but the distance metric (Goldberger et al., 2005; Globermation H ⊤ T ⊤ LFDA son and Roweis, 2006; =-=Weinberger et al., 2006-=-) in the embedding space can be arbitrary because of the arbitrariness of the matrix H. In practice, we propose determining the LFDA transformation matrix T LFDA as follows. First, we rescale the gene...

Image retrieval and classification using local distance functions

by Andrea Frome, Yoram Singer, Jitendra Malik - Advances in Neural Information Processing Systems , 2006
"... In this paper we introduce and experiment with a framework for learning local perceptual distance functions for visual recognition. We learn a distance function for each training image as a combination of elementary distances between patch-based visual features. We apply these combined local distanc ..."
Abstract - Cited by 107 (3 self) - Add to MetaCart
In this paper we introduce and experiment with a framework for learning local perceptual distance functions for visual recognition. We learn a distance function for each training image as a combination of elementary distances between patch-based visual features. We apply these combined local distance functions to the tasks of image retrieval and classification of novel images. On the Caltech 101 object recognition benchmark, we achieve 60.3 % mean recognition across classes using 15 training images per class, which is better than the best published performance by Zhang, et al. 1
(Show Context)

Citation Context

...s are computed. In this work, we learn the function in step (4) from the training data. This is similar in spirit to the recent body of metric learning work in the machine learning community [6][7][8]=-=[9]-=-[10]. While these methods have been successfully applied to recognizing digits, there are a couple drawbacks in applying these methods to the general image classification problem. First, they would re...

Visual Rank: applying Page Rank to large-scale image search

by Yushi Jing, Shumeet Baluja - IEEE Trans. Pattern Analysis and Machine Intelligence , 2008
"... Abstract—Because of the relative ease in understanding and processing text, commercial image-search systems often rely on techniques that are largely indistinguishable from text search. Recently, academic studies have demonstrated the effectiveness of employing image-based features to provide either ..."
Abstract - Cited by 96 (4 self) - Add to MetaCart
Abstract—Because of the relative ease in understanding and processing text, commercial image-search systems often rely on techniques that are largely indistinguishable from text search. Recently, academic studies have demonstrated the effectiveness of employing image-based features to provide either alternative or additional signals to use in this process. However, it remains uncertain whether such techniques will generalize to a large number of popular Web queries and whether the potential improvement to search quality warrants the additional computational cost. In this work, we cast the image-ranking problem into the task of identifying “authority ” nodes on an inferred visual similarity graph and propose VisualRank to analyze the visual link structures among images. The images found to be “authorities ” are chosen as those that answer the image-queries well. To understand the performance of such an approach in a real system, we conducted a series of large-scale experiments based on the task of retrieving images for 2,000 of the most popular products queries. Our experimental results show significant improvement, in terms of user satisfaction and relevancy, in comparison to the most recent Google Image Search results. Maintaining modest computational cost is vital to ensuring that this procedure can be used in practice; we describe the techniques required to make this system practical for large-scale deployment in commercial search engines. Index Terms—Image ranking, content-based image retrieval, eigenvector centrality, graph theory. Ç
(Show Context)

Citation Context

...ustomize image similarities through domain engineering. For example, similarity computations that capture higher order feature dependencies 1 and learning techniques can be efficiently employed [11], =-=[12]-=-, [13], [14]. Further, even nonvisual information, such as user-generated covisitation [15], [16] statistics, can be easily combined with visual features to make similarity scores more semantically re...

Fast solvers and efficient implementations for distance metric learning

by Kilian Q. Weinberger, Lawrence K. Saul - In ICML , 2008
"... In this paper we study how to improve nearest neighbor classification by learning a Mahalanobis distance metric. We build on a recently proposed framework for distance metric learning known as large margin nearest neighbor (LMNN) classification. Our paper makes three contributions. First, we describ ..."
Abstract - Cited by 85 (7 self) - Add to MetaCart
In this paper we study how to improve nearest neighbor classification by learning a Mahalanobis distance metric. We build on a recently proposed framework for distance metric learning known as large margin nearest neighbor (LMNN) classification. Our paper makes three contributions. First, we describe a highly efficient solver for the particular instance of semidefinite programming that arises in LMNN classification; our solver can handle problems with billions of large margin constraints in a few hours. Second, we show how to reduce both training and testing times using metric ball trees; the speedups from ball trees are further magnified by learning low dimensional representations of the input space. Third, we show how to learn different Mahalanobis distance metrics in different parts of the input space. For large data sets, the use of locally adaptive distance metrics leads to even lower error rates. 1.
(Show Context)

Citation Context

...ignificant improvements have been observed within several different frameworks for this problem, including neighborhood components analysis (Goldberger et al., 2005), large margin kNN classification (=-=Weinberger et al., 2006-=-), and informationtheoretic metric learning (Davis et al., 2007). These studies have established the general utility of distance metric learning for kNN classification. However, further work is requir...

Learning visual similarity measures for comparing never seen objects

by Eric Nowak - Proc. IEEE CVPR , 2007
"... In this paper we propose and evaluate an algorithm that learns a similarity measure for comparing never seen objects. The measure is learned from pairs of training images labeled “same ” or “different”. This is far less informative than the commonly used individual image labels (e.g. “car model X”), ..."
Abstract - Cited by 82 (0 self) - Add to MetaCart
In this paper we propose and evaluate an algorithm that learns a similarity measure for comparing never seen objects. The measure is learned from pairs of training images labeled “same ” or “different”. This is far less informative than the commonly used individual image labels (e.g. “car model X”), but it is cheaper to obtain. The proposed algorithm learns the characteristic differences between local descriptors sampled from pairs of “same ” and “different” images. These differences are vector quantized by an ensemble of extremely randomized binary trees, and the similarity measure is computed from the quantized differences. The extremely randomized trees are fast to learn, robust due to the redundant information they carry and they have been proved to be very good clusterers. Furthermore, the trees efficiently combine different feature types (SIFT and geometry). We evaluate our innovative similarity measure on four very different datasets and consistantly outperform the state-of-the-art competitive approaches. 1.
(Show Context)

Citation Context

... a simple distance can eventually be used in the target space. This function is generally inspired by the Mahanalobis distance, of the form d(x, y) = (x − y) t A(x − y), like in [14] or more recently =-=[21, 11, 19, 1, 10, 20]-=-. Various optimization schemes are possible to estimate A, depending on the objective function to be satisfied. The objective function plays a key role in the definition of the metric. In [21, 11] the...

Randomized clustering forests for image classification

by Frank Moosmann, Student Member, Student Member, Frederic Jurie, Ieee Computer Society - Pattern Analysis and Machine Intelligence
"... Abstract—This paper introduces three new contributions to the problems of image classification and image search. First, we propose a new image patch quantization algorithm. Other competitive approaches require a large code book and the sampling of many local regions for accurate image description, a ..."
Abstract - Cited by 82 (3 self) - Add to MetaCart
Abstract—This paper introduces three new contributions to the problems of image classification and image search. First, we propose a new image patch quantization algorithm. Other competitive approaches require a large code book and the sampling of many local regions for accurate image description, at the expense of a prohibitive processing time. We introduce Extremely Randomized Clustering Forests—ensembles of randomly created clustering trees—that are more accurate, much faster to train and test, and more robust to background clutter compared to state-of-the-art methods. Second, we propose an efficient image classification method that combines ERC-Forests and saliency maps very closely with image information sampling. For a given image, a classifier builds a saliency map online, which it uses for classification. We demonstrate speed and accuracy improvement in several state-of-the-art image classification tasks. Finally, we show that our ERC-Forests are used very successfully for learning distances between images of never-seen objects. Our algorithm learns the characteristic differences between local descriptors sampled from pairs of the “same ” or “different ” objects, quantizes these differences with ERC-Forests, and computes the similarity from this quantization. We show significant improvement over state-of-the-art competitive approaches. Index Terms—Randomized trees, image classification, object recognition, similarity measure. Ç 1
(Show Context)

Citation Context

...istance can eventually be used in the target space. This function is generally inspired by the Mahalanobis distance dðx; yÞ ðx yÞ t Aðx yÞ, as in [35] or, more recently, [58], [22], [50], [3], [21], =-=[56]-=-. Various optimization schemes are possible for estimating A, depending on the objective function to be satisfied. The objective function plays a key role in the definition of the metric. In [58], [22...

Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University