N. Duffy and D. P. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference (EuroCOLT '99), pages 18-- 33. Springer, Berlin, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....base classifiers. For example, Freund and Schapire s AdaBoost algorithm [10] and Breiman s Bagging algorithm [2] have been found to give significant performance improvements over algorithms for the corresponding base classifiers [6, 9, 16, 5] and have led to the study of many related algorithms [3, 19, 12, 17, 7, 11, 8]. Recent theoretical results suggest that the effectiveness of these algorithms is due to their tendency to produce large margin classifiers . The margin of an example is defined as the difference between the total weight assigned to the correct label and the largest weight assigned to an ....

N. Duffy and D. Helmbold. A geometric approachtoleveraging weak learners. In Computational Learning Theory: 4th European Conference, 1999. (to appear).

....problem which will now serve as a basis for deriving a boosting like algorithm. Despite the fact that this algorithm will not have the PAC boosting property [43] it works very similar to AdaBoost. However, being pedantic about not confusing the terms, we will use leveraging first introduced in [44] instead of boosting . Principally, any leveraging approach selects iteratively one hypothesis h 7 at a time and then updates the weight vector w, which can be implemented in different ways. There are essentially two alternatives: i) Ideally, one solves the optimization problem for all ....

N. Duffy and D.P. Helmbold, "A geometric approach to leveraging weak learners," in Computational Learning Theory: 4th European Conference (EuroCOLT '99), P. Fischer and H. U. Sinion, Eds., Mar. 1999, pp. 18 33, Long version to appear in TCS.

....t are upper bounded by a rather small constant. For this case it has been shown that the algorithm asymptotically converges to a combined hypothesis minimizing G. However, since the a t s need to be small, the algorithm requires many iterations to achieve this goal. Other approaches include e.g. [DH99, DH00a]. In this paper we propose a family of algorithms that are able to generate a combined hypothesis f converging to the minimum of G[ f ] if it exists) which is a functional depending on the outputs of the function f evaluated on the training set. Special cases are AdaBoost, Logistic Regression ....

....results to leveraging algorithms, we present an extension to regularized cost functions in Section 4 and finally conclude. 2 Leveraging algorithms revisited We first briefly review some of the most well known leveraging algorithms for classification and regression. For more details see e.g. [FS97, FHT98, Fri99, DH99]. We work with Algorithm 1 as a template of a generic leveraging algorithm, since these algorithms have the same algorithmical structure. Finally, we will generalize the problem and extend the notation. 2.1 AdaBoost Logistic Regression Both methods are designed for classification tasks. In ....

N. Duffy and D.P. Helmbold. A geometric approach to leveraging weak learners. In P. Fischer and H. U. Simon, editors, Computational Learning Theory: 4th European Conference (EuroCOLT '99), pages 18--33, March 1999. Long version to appear in TCS.

....these results to leveraging algorithms, we present an extension to regularized cost functions in Sec. 4 and finally conclude. 2 Leveraging algorithms revisited We first briefly review some of the most well known leveraging algorithms for classification and regression. For more details see e.g. [10, 11, 12, 8]. We work with Alg. 1 as a template for a generic leveraging algorithm, since these algorithms have the same algorithmical structure. Finally, we will generalize the problem and extend the notation. AdaBoost Logistic Regression are designed for classification tasks. In each iteration they call ....

N. Duffy and D.P. Helmbold. A geometric approach to leveraging weak learners. In P. Fischer and H. U. Simon, editors, Proc. EuroCOLT '99, pages 18--33, 1999.

....vector machines (LVM) We would like to note in passing that the resulting algorithms are not boosting algorithms in the PAC sense. For instance, the weak learnability assumption that the weak learner can always find a weak hypothesis is violated. We therefore adopt the terminology used in [2] and call the resulting classifiers leveraged vector machines. The leveraging procedure we give adopts the chunking technique from SVM. After presenting the basic leveraging algorithms we compare their performance with SVM on synthetic data. The experimental results show that the leveraged vector ....

N. Duffy and D. Helmbold. A geometric approach to leveraging weak learners. EuroCOLT '99.

....base classifiers. For example, Freund and Schapire s AdaBoost algorithm [10] and Breiman s Bagging algorithm [2] have been found to give significant performance improvements over algorithms for the corresponding base classifiers [6, 9, 16, 5] and have led to the study of many related algorithms [3, 19, 12, 17, 7, 11, 8]. Recent theoretical results suggest that the effectiveness of these algorithms is due to their tendency to produce large margin classifiers . The margin of an example is defined as the difference between the total weight assigned to the correct label and the largest weight assigned to an ....

N. Duffy and D. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference, 1999. (to appear).

....interesting to see whether some other divergences might lead to useful boosting procedures. Some updates motivated by different Bregman divergences are briefly discussed in Appendix B, but without any results on the training error of the resulting boosting procedure. Perhaps the GeoLev procedure [DH99] could be related to projections with respect to the squared Euclidean distances. Note that the relative entropy is a special divergence in that it is defined on the simplex Pm and this is the natural domain for boosting. For other divergence, an additional projection onto Pm would be needed, but ....

N. Duffy and D. P. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference (EuroCOLT '99), pages 18-- 33. Springer, Berlin, 1999.

.... For example, Freund and Schapire s AdaBoost algorithm [12] and Breiman s Bagging algorithm [3] have been found to give significant performance improvements over algorithms for the corresponding base classifiers [7, 11, 18, 6, 22, 2, 16] and have led to the study of many related algorithms [4, 21, 14, 19, 8, 13]. Recent theoretical results suggest that the effectiveness of these algorithms is due to their tendency to produce large margin classifiers. The margin of an example is defined as the difference between the total weight assigned to the correct label and the largest weight assigned to an incorrect ....

....et al. show that versions of AdaBoost modified to use regularization are more robust for noisy data. Friedman [13] describes general boosting algorithms for regression and classification using various cost functions and presents specific cases for boosting decision trees. Duffy and Helmbold [8] describe two algorithms (GeoLev and GeoArc) which attempt to produce combined classifiers with uniformly large margins on the training data. In [10] Freund presents a new boosting algorithm which uses example weights similar to those suggested by the theoretical results from [17] 2 Optimizing ....

[Article contains additional citation context not shown here]

N. Duffy and D. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference, 1999. (to appear).

....the complexity of the base hypotheses. However, if the improvement in accuracy is large, and the increase in complexity is small, then leveraging can improve generalization. Leveraging has been examined primarily in the classification setting where AdaBoost [13] and related leveraging techniques [3, 9, 5, 4, 6, 14, 20] have been found to be useful for increasing the accuracy of base classifiers. Such algorithms repeatedly call a base learning algorithm, and construct a linear combination of the hypotheses returned. These techniques have recently been viewed as performing gradient descent on a potential function ....

....have been found to be useful for increasing the accuracy of base classifiers. Such algorithms repeatedly call a base learning algorithm, and construct a linear combination of the hypotheses returned. These techniques have recently been viewed as performing gradient descent on a potential function [4, 6, 20, 14, 9, 18] and this viewpoint has enabled the derivation and analysis of new algorithms in the classification setting [14, 9, 8, 18] Recent work by Friedman has shown that this gradient descent viewpoint can also be used to construct leveraging algorithms for regression with good empirical performance ....

[Article contains additional citation context not shown here]

Nigel Duffy and David P. Helmbold. A geometric approach to leveraging weak learners. In Paul Fischer and Hans Ulrich Simon, editors, Computational Learning Theory: 4th European Conference (EuroCOLT '99), pages 18--33. Springer-Verlag, March 1999.

No context found.

N. Duffy and D. P. Helmbold. A geometric approach to leveraging weak learners. In Computational Learning Theory: 4th European Conference (EuroCOLT '99), pages 18-- 33. Springer, Berlin, 1999.

No context found.

Duy, N. and Helmbold, D. (1998). A geometric approach to leveraging weak learners. University of California, Santa Cruz, technical report (to appear in EuroColt 99, Springer Verlag).

No context found.

Duffy, N. and Helmbold, D. (1998). A geometric approach to leveraging weak learners. University of California, Santa Cruz, technical report (to appear in EuroColt 99, Springer Verlag).

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC