Abstract

The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting algorithm for combining preferences called RankBoost. We also describe an efficient implementation of the algorithm for certain natural cases. We discuss two experiments we carried out to assess the performance of RankBoost. In the first experiment, we used the algorithm to combine different WWW search strategies, each of which is a query expansion for a given domain. For this task, we compare the performance of RankBoost to the individual search strategies. The second experiment is a collaborative-filtering task for making movie recommendations. Here, we present results comparing RankBoost to nearest-neighbor and regression algorithms.

Citations

6696	Statistical Learning Theory - Vapnik - 1998
2699	Introduction to Modern Information Retrieval - Salton, McGill - 1984
2528	UCI Repository of Machine Learning Databases - Blake, Keogh, et al. - 1998
1999	Bagging predictors - Breiman - 1996
1714	Schapire R: A decision-theoretic generalization of online learning and an application to boosting - Freund - 1997
1515	A theory of the learnable - Valiant - 1984
1493	Vapnik V: Support vector networks - Cortes - 1995
1326	Schapire R: Experiments with a new boosting algorithm - Freund
1180	Fundamentals of speech recognition - Rabiner - 1993
1095	Automatic text processing: The transformation, analysis, and retrieval of information by computer - Salton
936	A training algorithm for optimal margin classifiers - Boser, Guyon, et al. - 1992
897	Tibshirani R: Additive logistic regression: a statistical view of boosting. Annl Stat 2000, 28:337-374. Results of nonlinear tree harvest procedure applied to lymphoma data Node Parent Score -2Log-likelihood Size Nonlinear? 1 - Friedman, Hastie
875	Grouplens: An open architecture for collaborative filtering of netnews - Resnick, Iacovou, et al. - 1994
801	A probabilistic theory of pattern recognition - Devroye, Györfi, et al. - 1996
778	Em-pirical Analysis of Predictive Algorithms for Col-laborative Filtering - Breese, Heckerman, et al. - 1998
734	P.: Social information filtering: Algorithms for automating “word of mouth - Shardanand, Maes - 1995
687	Estimation of dependences based on empirical data - Vapnik - 1982
607	W: Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statisics - Schapire, Freund, et al. - 1998
556	The weighted majority algorithm - Littlestone, Warmuth - 1994
554	The strenght of weak learnability - Schapire - 1990
507	An Introduction to Computational Learning Theory - Kearns, Vazirani - 1994
364	An experimental comparison of three methods for constructing ensembles of decision trees: bagging, boosting and randomization - Dietterich
358	Boosting a weak learning algorithm by majority - Freund - 1995
327	Decision theoretic generalizations of the PAC model for neural net and other learning applications - Haussler - 1992
279	Cryptographic limitations on learning boolean formulae and finite automata - Kearns, Valiant - 1994
265	Learning to order things - Cohen, Schapire, et al. - 1998
251	Bagging, boosting, and C4.5 - Quinlan - 1996
229	G.: Recommending and evaluating choices in a virtual community of use - Hill, Stead, et al. - 1995
220	Discriminative Reranking for Natural Language Parsing - Collins - 2000
173	Classi cation and Regression Trees - Breiman, Friedman, et al. - 1984
156	The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network - Bartlett
144	Bias plus variance decomposition for zero-one loss functions - Kohavi, Wolpert - 1996
140	Pattern Classi and Scene Analysis - Duda, Hart - 1973
131	Error-correcting output coding corrects bias and variance - Kong, Dietterich - 1995
130	Automatic combination of multiple ranked retrieval systems - Bartell, Cottrell, et al. - 1994
128	Y.: Pranking with ranking - Crammer, Singer - 2002
117	Game theory, On-line Prediction and Boosting - Freund, Schapire - 1996
106	Adaptive game playing using multiplicative weights - Freund, Schapire - 1999
100	Learning when irrelevant attributes abound: A new linear-threshold algorithm - Littlestone - 1988
99	T.: Pruning adaptive boosting - Margineantu, Dietterich - 1997
89	Equivalence of models for polynomial learnability - HAUSSLER, KEARNS, et al. - 1991
86	A game of prediction with expert advice - Vovk - 1998
81	Boosting decision trees - Drucker, H, et al. - 1996
80	An empirical evaluation of bagging and boosting - Maclin, Opitz - 1997
78	Using output codes to boost multiclass learning problems - Schapire - 1997
76	Boosting the margin: A new explanation for the eectiveness of voting methods - Schapire, Freund, et al. - 1998
71	Boosting performance in neural networks - Drucker, Schapire, et al. - 1993
69	Boosting and other ensemble methods - Drucker, Cortes, et al. - 1301
65	An adaptive version of the boost by majority algorithm - Freund - 2000
62	Fudenberg and Jean Tirole. Game Theory - Drew - 1991