Osamu Watanabe. From Computational Learning Theory to Discovery Science. In Proc. of the 26th International Colloquium on Automata, Languages and Programming, ICALP'99. Lecture Notes in Computer Science 1644:134{ 148, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....of database query estimation [7] and knowledge discovery [1, 2] Furthermore, adaptivity is a very desirable property for an algorithm that is expected to be used in practical applications. See the discussion about the relevance of adaptivity in the context of learning and discovery science in [10]. As noted by the authors, a practical implementation based on the algorithmic ideas provided by them would enjoy performance on natural bounds that is considerably better than what their bounds indicate. In fact, our improvement corroborates that intuition since our modi cation uses all the ....

Osamu Watanabe. From Computational Learning Theory to Discovery Science. In Proc. of the 26th International Colloquium on Automata, Languages and Programming, ICALP'99. Lecture Notes in Computer Science 1644:134{ 148, 1999.

.... on examples are bounded in [0; 1] This suggests that it may be possible to use the new algorithm in a setting in which the boosting algorithm selects examples to present to the weak learning algorithm by filtering a stream of examples (such as a very large dataset) As pointed out by Watanabe [27] and Domingo and Watanabe [13] this is not possible with AdaBoost since its weights may become extremely large. They provide a modification of AdaBoost for this purpose in which the weights are truncated at 1. Our new algorithm may be a viable and cleaner alternative. We next describe a ....

Osamu Watanabe. From computational learning theory to discovery science. In Proceedings of the 26th International Colloquium on Automata, Languages and Programming, pages 134--148, 1999. 26

.... on examples are bounded in [0; 1] This suggests that it may be possible to use the new algorithm in a setting in which the boosting algorithm selects examples to present to the weak learning algorithm by filtering a stream of examples (such as a very large dataset) As pointed out by Watanabe [22] and Domingo and Watanabe [11] this is not possible with AdaBoost since its weights may become extremely large. They provide a modification of AdaBoost for this purpose in which the weights are truncated at 1. The new algorithm may be a viable and cleaner alternative. We next describe a ....

Osamu Watanabe. From computational learning theory to discovery science. In Proceedings of the 26th International Colloquium on Automata, Languages and Programming, pages 134--148, 1999. 12

.... for the filtering framework using the weights as defined in Freund and Schapire (1997) Unfortunately, one can show that the probability that the filter accepts one example becomes exponentially small and thus, it might take a large time to generate a new sample at each iteration (see (Domingo and Watanabe, 1999) for a formal proof of this) The reason is that, due to its weighting scheme, after some iterations, AdaBoost concentrates most of the weight under the current distribution in very few instances (assuming that the initial distribution over the training set was uniform) This problem was address ....

.... for a formal proof of this) The reason is that, due to its weighting scheme, after some iterations, AdaBoost concentrates most of the weight under the current distribution in very few instances (assuming that the initial distribution over the training set was uniform) This problem was address by Watanabe (1999) where it was proposed to use the initial value of the weights as a saturation bound so that they cannot grow uncontrolled like it happens in AdaBoost and a partial theoretical justification of its properties was provided. Domingo and Watanabe (1999) further modify the weighting scheme so a formal ....

[Article contains additional citation context not shown here]

Watanabe, O. (1999). From computational learning theory to discovery science, in Proc. 26th ICALP'99, Lecture Notes in Computer Science 1644, 134--148.

....in the, so far most successful boosting algorithm, AdaBoost due to Freund and Schapire [FS97] These problems are: 1) AdaBoost cannot be used in the boosting by filtering framework, and (2) AdaBoost does not seem to be noise resistant. The outline of our modification was first proposed in [Wat99] with only a partial proof for its justification. In this paper, we describe the modification in detail, provide a much improved analysis of its correctness and performance, and prove that our new boosting algorithm can be casted in the statistical query learning model [Kea93] and thus, it is ....

....property. For example, our experiments [DW99b] show that MadaBoost has a boosting property more or less similar to AdaBoost. We can also prove some basic boosting property of MadaBoost for a special case where (we may assume that) the advantage of every obtained hypothesis is some fixed # 0 [Wat99]. Here we prove a more general boosting property. Unfortunately, though, for our current proof, we need to modify the weighting scheme of MadaBoost even more moderate one. The di#erence from MadaBoost is the definition of # t ; here the following definition is used. # t = # # # t 1 # # t , ....

O. Watanabe, From computational learning theory to discovery science, in Proc. 26th International Colloquium on Automata, Languages and Programming, ICALP'99, Lecture Notes in Computer Science 1644, 134--148, 1999.

....in the so far most successful boosting algorithm, AdaBoost due to Freund and Schapire [FS97] These problems are: 1) AdaBoost cannot be used in the boosting by filtering framework, and (2) AdaBoost does not seem to be noise resistant. The outline of our modification was first proposed in [Wat99] with only a partial proof for its justification. In this paper, we describe the modification in detail, provide a much improved analysis of its correctness and performance, and prove that our new boosting algorithm can be casted in the statistical query learning model [Kea93] and thus, it is ....

O. Watanabe, From computational learning theory to discovery science, in Proc. 26th International Colloquium on Automata, Languages and Programming, ICALP'99, Lecture Notes in Computer Science 1644, 134--148, 1999.

....repeatedly reported to be the most e ective in the absence of noise [9, 16, 2, 1] However, AdaBoost is required to run using all the dataset and thus it is not suitable to be used with large datasets. In this paper we will use a modi cation of AdaBoost recently proposed by Domingo and Watanabe [19, 5] that is more suitable for being combined with a base learner that uses only a portion of the dataset select through sampling as discussed in Section 2. The boosting algorithm outputs a hypothesis that is the weighted majority of the hypothesis output by the base learner. Since as we argued one of ....

Watanabe, O., 1999. From Computational Learning Theory to Discovery Science. Proc. of the 26th International Colloquim on Automata, Languages and Programming, ICALP'99 Invited talk. Lecture Notes in Computer Science 1644:134-148. 10

....mends some of the problems that have been detected in the, so far most successful boosting algorithm, AdaBoost. These problems are: 1) AdaBoost cannot be used in the boosting by filtering framework and, 2) AdaBoost does not seem to be noise resistant. This modification was first proposed in [16] and the purpose of this note is to provide an improved proof of its correctness together with a proof of a key property of the algorithm, namely, that the new boosting algorithm can be casted in the statistical query learning model and thus, it is robust to classification noise in a sense that ....

Osamu Watanabe. From Computational Learning Theory to Discovery Science. Invited talk to ICALP'99. 17

....that mends some of the problems that have been detected in the, so far most successful boosting algorithm, AdaBoost. These problems are: 1) AdaBoost cannot be used in the boosting by ltering framework and, 2) AdaBoost does not seem to be noise resistant. This modi cation was rst proposed in [16] and the purpose of this note is to provide an improved proof of its correctness together with a proof of a key property of the algorithm, namely, that the new boosting algorithm can be casted in the statistical query learning model and thus, it is robust to classi cation noise in a sense that ....

Osamu Watanabe. From Computational Learning Theory to Discovery Science. Invited talk to ICALP'99.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC