Results 1  10
of
26
STOCHASTIC TRIPLET EMBEDDING
"... This paper considers the problem of learning an embedding of data based on similarity triplets of the form “A is more similar to B than to C”. This learning setting is of relevance to scenarios in which we wish to model human judgements on the similarity of objects. We argue that in order to obtain ..."
Abstract

Cited by 12 (0 self)
 Add to MetaCart
(Show Context)
This paper considers the problem of learning an embedding of data based on similarity triplets of the form “A is more similar to B than to C”. This learning setting is of relevance to scenarios in which we wish to model human judgements on the similarity of objects. We argue that in order to obtain a truthful embedding of the underlying data, it is insufficient for the embedding to satisfy the constraints encoded by the similarity triplets. In particular, we introduce a new technique called tDistributed Stochastic Triplet Embedding (tSTE) that collapses similar points and repels dissimilar points in the embedding — even when all triplet constraints are satisfied. Our experimental evaluation on three data sets shows that as a result, tSTE is much better than existing techniques at revealing the underlying data structure. Index Terms — Partial order embedding, similarity triplets. 1.
Efficient Ranking from Pairwise Comparisons
"... The ranking of n objects based on pairwise comparisons is a core machine learning problem, arising in recommender systems, ad placement, player ranking, biological applications and others. In many practical situations the true pairwise comparisons cannot be actively measured, but a subset of all n(n ..."
Abstract

Cited by 12 (1 self)
 Add to MetaCart
(Show Context)
The ranking of n objects based on pairwise comparisons is a core machine learning problem, arising in recommender systems, ad placement, player ranking, biological applications and others. In many practical situations the true pairwise comparisons cannot be actively measured, but a subset of all n(n−1)/2 comparisons is passively and noisily observed. Optimization algorithms (e.g., the SVM) could be used to predict a ranking with fixed expected Kendall tau distance, while achieving an Ω(n) lower bound on the corresponding sample complexity. However, due to their centralized structure they are difficult to extend to online or distributed settings. In this paper we show that much simpler algorithms can match the same Ω(n) lower bound in expectation. Furthermore, if an average of O(n log(n)) binary comparisons are measured, then one algorithm recovers the true ranking in a uniform sense, while the other predicts the ranking more accurately near the top than the bottom. We discuss extensions to online and distributed ranking, with benefits over traditional alternatives. 1.
LowDimensional Embedding using Adaptively Selected Ordinal Data
"... Abstract—Lowdimensional embedding based on nonmetric data (e.g., nonmetric multidimensional scaling) is a problem that arises in many applications, especially those involving human subjects. This paper investigates the problem of learning an embedding of n objects into ddimensional Euclidean spa ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
Abstract—Lowdimensional embedding based on nonmetric data (e.g., nonmetric multidimensional scaling) is a problem that arises in many applications, especially those involving human subjects. This paper investigates the problem of learning an embedding of n objects into ddimensional Euclidean space that is consistent with pairwise comparisons of the type “object a is closer to object b than c. ” While there are O(n 3) such comparisons, experimental studies suggest that relatively few are necessary to uniquely determine the embedding up to the constraints imposed by all possible pairwise comparisons (i.e., the problem is typically overconstrained). This paper is concerned with quantifying the minimum number of pairwise comparisons necessary to uniquely determine an embedding up to all possible comparisons. The comparison constraints stipulate that, with respect to each object, the other objects are ranked relative to their proximity. We prove that at least Ω(dn log n) pairwise comparisons are needed to determine the embedding of all n objects. The lower bounds cannot be achieved by using randomly chosen pairwise comparisons. We propose an algorithm that exploits the lowdimensional geometry in order to accurately embed objects based on relatively small number of sequentially selected pairwise comparisons and demonstrate its performance with experiments. I.
Inferring Users ’ Preferences from Crowdsourced Pairwise Comparisons: A Matrix Completion Approach
"... Inferring user preferences over a set of items is an important problem that has found numerous applications. This work focuses on the scenario where the explicit feature representation of items is unavailable, a setup that is similar to collaborative filtering. In order to learn a user’s preference ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Inferring user preferences over a set of items is an important problem that has found numerous applications. This work focuses on the scenario where the explicit feature representation of items is unavailable, a setup that is similar to collaborative filtering. In order to learn a user’s preferences from his/her response to only a small number of pairwise comparisons, we propose to leverage the pairwise comparisons made by many crowd users, a problem we refer to as crowdranking. The proposed crowdranking framework is based on the theory of matrix completion, and we present efficient algorithms for solving the related optimization problem. Our theoretical analysis shows that, on average, only O(r logm) pairwise queries are needed to accurately recover the ranking list of m items for the target user, where r is the rank of the unknown rating matrix, r m. Our empirical study with two realworld benchmark datasets for collaborative filtering and one crowdranking dataset we collected via Amazon Mechanical Turk shows the promising performance of the proposed algorithm compared to the stateoftheart approaches.
Relative Ranking of Facial Attractiveness
"... Automatic evaluation of human facial attractiveness is a challenging problem that has received relatively little attention from the computer vision community. Previous work in this area have posed attractiveness as a classification problem. However, for applications that require finegrained relatio ..."
Abstract

Cited by 4 (1 self)
 Add to MetaCart
(Show Context)
Automatic evaluation of human facial attractiveness is a challenging problem that has received relatively little attention from the computer vision community. Previous work in this area have posed attractiveness as a classification problem. However, for applications that require finegrained relationships between objects, learning to rank has been shown to be superior over the direct interpretation of classifier scores as ranks [27]. In this paper, we propose and implement a personalized relative beauty ranking system. Given training data of faces sorted based on a subject’s personal taste, we learn how to rank novel faces according to that person’s taste. Using a blend of Facial Geometric Relations, HOG, GIST, L*a*b * Color Histograms, and DenseSIFT + PCA feature types, our system achieves an average accuracy of 63 % on pairwise comparisons of novel test faces. We examine the effectiveness of our method through lesion testing and find that the most effective feature types for predicting beauty preferences are HOG, GIST, and DenseSIFT + PCA features. 1.
Enhanced statistical rankings via targeted data collection
"... Given a graph where vertices represent alternatives and pairwise comparison data, yij, is given on the edges, the statistical ranking problem is to find a potential function, defined on the vertices, such that the gradient of the potential function agrees with pairwise comparisons. We study the depe ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
Given a graph where vertices represent alternatives and pairwise comparison data, yij, is given on the edges, the statistical ranking problem is to find a potential function, defined on the vertices, such that the gradient of the potential function agrees with pairwise comparisons. We study the dependence of the statistical ranking problem on the available pairwise data, i.e., pairs (i, j) for which the pairwise comparison data yij is known, and propose a framework to identify data which, when augmented with the current dataset, maximally increases the Fisher information of the ranking. Under certain assumptions, the data collection problem decouples, reducing to a problem of finding an edge set on the graph (with a fixed number of edges) such that the second eigenvalue of the graph Laplacian is maximal. This reduction of the data collection problem to a spectral graphtheoretic question is one of the primary contributions of this work. As an application, we study the Yahoo! Movie user rating dataset and demonstrate that the addition of a small number of wellchosen pairwise comparisons can significantly increase the Fisher informativeness of the ranking. 1.
Active Ranking in Practice: General Ranking Functions with Sample Complexity Bounds
"... This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In a companion paper in the regular NIPS 2011 program [1], we showed that if each object x ∈ R d is assigned a score f(x) =x − r  for some unknown r ∈ R d, then our recently p ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
This paper examines the problem of ranking a collection of objects using pairwise comparisons (rankings of two objects). In a companion paper in the regular NIPS 2011 program [1], we showed that if each object x ∈ R d is assigned a score f(x) =x − r  for some unknown r ∈ R d, then our recently proposed active ranking algorithm can recover the ranking of the scores using about d log n selectively chosen pairwise comparisons. Here we show that this same model contains all functions of the type g(x) =w T x for some unknown w ∈ R d, thus the same bound applies. We take advantage of this fact and use kernel methods to represent more general ranking functions. This extension includes popular ranking methods such as RankSVM, and we derive nontrivial query complexity bounds for active versions of such algorithms. The efficacy of the theory and method are demonstrated by applying our kernelized adaptive algorithm to two real datasets. 1 Problem statement Given a set of n objects Θ: = {θ1,..., θn}, we wish to discover how an oracle ranks these objects.
Active Collaborative Permutation Learning
"... We consider the problem of Collaborative Permutation Recovery, i.e. recovering multiple permutations over objects (e.g. preference rankings over different options) from limited pairwise comparisons. We tackle both the problem of how to recover multiple related permutations from limited observation ..."
Abstract
 Add to MetaCart
(Show Context)
We consider the problem of Collaborative Permutation Recovery, i.e. recovering multiple permutations over objects (e.g. preference rankings over different options) from limited pairwise comparisons. We tackle both the problem of how to recover multiple related permutations from limited observations, and the active learning problem of which pairwise comparison queries to ask so as to allow better recovery. There has been much work on recovering single permutations from pairwise comparisons, but we show that considering several related permutations jointly we can leverage their relatedness so as to reduce the number of comparisons needed compared to reconstructing each permutation separately. To do so, we take a collaborative filtering / matrix completion approach and use a tracenorm or maxnorm regularized matrix learning model. Our approach can also be seen as a collaborative learning version of Jamieson and Nowak’s recent work on constrained permutation recovery, where instead of basing the recovery on known features, we learn the best features de novo.
SYNCRANK: ROBUST RANKING, CONSTRAINED RANKING AND RANK AGGREGATION VIA EIGENVECTOR AND SDP SYNCHRONIZATION
, 2015
"... Abstract. We consider the classic problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. We consider the classic problem of establishing a statistical ranking of a set of n items given a set of inconsistent and incomplete pairwise comparisons between such items. Instantiations of this problem occur in numerous applications in data analysis (e.g., ranking teams in sports data), computer vision, and machine learning. We formulate the above problem of ranking with incomplete noisy information as an instance of the group synchronization problem over the group SO(2) of planar rotations, whose usefulness has been demonstrated in numerous applications in recent years in computer vision and graphics, sensor network localization and structural biology. Its least squares solution can be approximated by either a spectral or a semidefinite programming (SDP) relaxation, followed by a rounding procedure. We show extensive numerical simulations on both synthetic and realworld data sets (Premier League soccer games, a Halo 2 game tournament and NCAA College Basketball games), which show that our proposed method compares favorably to other ranking methods from the recent literature. Existing theoretical guarantees on the group synchronization problem imply lower bounds on the largest amount of noise permissible in the data while still achieving an approximate recovery of the ground truth ranking. We propose a similar synchronizationbased algorithm for the rankaggregation problem, which integrates in a globally consistent ranking many pairwise rankoffsets or partial rankings, given by different rating systems on the same set of items, an approach which yields significantly more accurate results than other aggregation methods, including RankCentrality, a recent stateoftheart algorithm. Furthermore, we discuss the problem of semisupervised ranking when there is available information on the ground truth rank of a subset of players, and propose an algorithm based on SDP
Kanji Uchino Fujitsu Laboratories of America
"... The growing set of Social Media tools such as Twitter, Facebook, Instagram, and Google Docs has the potential to enhance primary and secondary learning. To collect and evaluate suggestions for novel applications of social media to learning from online participants, we created a version of our Collec ..."
Abstract
 Add to MetaCart
(Show Context)
The growing set of Social Media tools such as Twitter, Facebook, Instagram, and Google Docs has the potential to enhance primary and secondary learning. To collect and evaluate suggestions for novel applications of social media to learning from online participants, we created a version of our Collective Discovery Engine. Over 155, educators, engineers, and social scientists responded to our emailed invitations to participate. In this paper we summarize the experiment, the data collected, and the responses. Suggestions were broadly classified into three categories: collaboration, diversity, and evaluation. We report on demographic correlations and present the suggestions that participants collectively considered most valuable (effective and/or novel). The interface is available online at: