Domingos, P. (1997), Why does bagging work? a Bayesian account and its implications, in D. Heckerman, H. Mannila, D. Pregibon & R. Uthurusamy, eds, `Proceedings of the third international conference on Knowledge Discovery and Data Mining', AAAI Press, pp. 155-- 158.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....one of the classifiers that was learned from a sample with a skewed distribution performs well by itself on the unskewed test set EMPIRICAL COMPARISON OF BOOSTING, BAGGING, AND VARIANTS 137 5. Boosting and Bagging both create very complex classifiers, yet they do not seem to overfit the data. Domingos (1997) claims that the multiple trees do not simply implement a Bayesian approach, but actually shift the learner s bias (machine learning bias, not statistical bias) away from the commonly used simplicity bias. Can this bias be made more explicit 6. We found that Bagging works well without pruning. ....

Domingos, P. (1997), Why does bagging work? a Bayesian account and its implications, in D. Heckerman, H. Mannila, D. Pregibon & R. Uthurusamy, eds, `Proceedings of the third international conference on Knowledge Discovery and Data Mining', AAAI Press, pp. 155-- 158.

....subsets of training data, and then collecting the models (e.g. decision trees) produced from each subset. These models may then be combined to behave as if they were a single model. We will describe the current approaches to combining multiple models in section 2. With one recent exception (Domingos, 1997, 1998) existing approaches to combining multiple models do not result in a single model of the same type as the initial set of models. The normal reasons for partitioning the training data into subsets are either to increase accuracy, or to share the computational load. Our goal is to learn ....

....nor Boosting are directly relevant to our work, because although they are methods for combining multiple models, they are applied specifically to highly overlapping subsets of data, and the final models are computed by voting. However they form a direct link with what we are trying to achieve. Domingos (1997) noted in his investigation of Bagging: We would like to find the simplest decision tree extensionally representing the same model as a bagged ensemble. In two later papers (Domingos 1997, 1998) he describes in detail his method for combining multiple models. We regard Domingos goal as being ....

[Article contains additional citation context not shown here]

P. Domingos (1997). Why Does Bagging Work? A Bayesian Account and its Implications. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, AAAI Press, pages 155-158. Newport Beach, CA.

....a new set of examples are selected, and then held until the next set of examples arrives. This overall technique is termed combining multiple models or integrating multiple learned models. We will describe the current approaches to combining multiple models in section 2. With one recent exception (Domingos, 1997, 1998) existing approaches to combining multiple models do not result in a single model of the same type as the initial set of models. In addition, the normal motivation for combining multiple models is to increase accuracy. To this end, the work done is actually increased, as each model is ....

....nor Boosting are directly relevant to our work, because although they are methods for combining multiple models, they are applied specifically to highly overlapping subsets of data, and the final models are computed by voting. However they form a direct link with what we are trying to achieve. Domingos (1997) noted in his investigation of Bagging: We would like to find the simplest decision tree extensionally representing the same model as a bagged ensemble. In two later papers (Domingos 1997, 1998) he describes in detail his method for combining multiple models. We regard Domingos goal as being ....

[Article contains additional citation context not shown here]

P. Domingos (1997). Why Does Bagging Work? A Bayesian Account and its Implications. In Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, AAAI Press, pages 155-158. Newport Beach, CA.

.... of classifiers, classifier committee 1 learning techniques have been developed with great success (Freund 1996; Freund and Schapire 1996a; 1996b; Quinlan 1996; Breiman 1996a; 1996b; Dietterich and Kong 1995; Ali 1996; Chan, Stolfo, and Wolpert 1996; Schapire, Freund, Bartlett, and Lee 1997; Domingos 1997; Bauer and Kohavi 1998) especially Boosting 2 (Freund and Schapire 1996b; Quinlan 1996; Bauer and Kohavi 1998) This type of technique generates several classifiers to form a committee by using a single base learning algorithm. At the classification stage, the committee members vote to make ....

Domingos, P. 1997. Why does Bagging Work? a Bayesian Account and its Implications.

....one of the classifiers that was learned from a sample with a skewed distribution performs well by itself on the unskewed test set EMPIRICAL COMPARISON OF BOOSTING, BAGGING, AND VARIANTS 33 5. Boosting and Bagging both create very complex classifiers, yet they do not seem to overfit the data. Domingos (1997) claims that the multiple trees do not simply implement a Bayesian approach, but actually shift the learner s bias (machine learning bias, not statistical bias) away from the commonly used simplicity bias. Can this bias be made more explicit 6. We found that Bagging works well without pruning. ....

Domingos, P. (1997), Why does bagging work? a Bayesian account and its implications, in D. Heckerman, H. Mannila, D. Pregibon & R. Uthurusamy, eds, `Proceedings of the third international conference on Knowledge Discovery and Data Mining', AAAI Press, pp. 155-- 158.

.... of classifiers, classifier committee 1 learning techniques have been developed with great success (Freund 1996; Freund and Schapire 1996a; 1996b; Quinlan 1996; Breiman 1996a; 1996b; Dietterich and Kong 1995; Ali 1996; Chan, Stolfo, and Wolpert 1996; Schapire, Freund, Bartlett, and Lee 1997; Domingos 1997; Bauer and Kohavi 1998; Zheng and Webb 1998) especially Boosting 2 (Freund and Schapire 1996b; Quinlan 1996; Bauer and Kohavi 1998) This type of technique generates several classifiers to form a committee by using a single base learning algorithm. At the classification stage, the committee ....

Domingos, P. 1997. Why does Bagging Work? a Bayesian Account and its Implications.

....a recent paper, Domingos (2000) compares Bayesian model averaging (BMA) to other model combination methods on some benchmark data sets, is surprised that BMA performs worst, and suggests that BMA may be flawed. These results are actually not surprising, especially in light of an earlier paper by Domingos (1997) where it was shown that model combination works by enriching the space of hypotheses, not by approximating a Bayesian model average. And the only flaw with BMA is the belief that it is an algorithm for model combination, when it is not. Bayesian model averaging is best thought of as a method for ....

Domingos, P. (1997). Why does bagging work? A Bayesian account and its implications.

.... one penalizing model complexity (Akaike, 1978; Schwarz, 1978; Wallace Boulton, 1968; Rissanen, 1978; Moody, 1992) This approach is only appropriate when the simpler models are truly the more accurate ones, and there is mounting evidence that this is typically not the case ( Domingos, 1998; Domingos, 1997; Schuurmans, Ungar Foster, 1997; Lawrence, Giles Tsoi, 1997; Webb, 1996; Schaffer, 1993; Murphy Pazzani, 1994) etc. Structural risk minimization (Vapnik, 1995) and PAC learning (Kearns Vazirani, 1994) are representation oriented methods that seek to bound the difference between ....

Domingos, P. (1997). Why does bagging work? A Bayesian account and its implications. Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (pp. 155-- 158). Newport Beach, CA: AAAI Press.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC