Z. Zheng and G.I. Webb. Stochastic attribute selection committees. In Proc. of the 11th Australian Joint Conf. on AI (AI'98), pages 321-332, 1998.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....this result is perplexing from an information theory perspective, it is consistent with learning theory: by removing features we simplify the learning task and thus allow the base classi ers to reach their peak performance. 10 Furthermore, IDEs also outperform random feature subset selection [2, 17] on real datasets [15] is that we have greatly simpli ed the relevance criterion: unlike other feature selection methods that consider the discriminatory ability across all classes, we only consider the relevance of the features to a single class. This typically causes each classi er in the ....

Z. Zheng and G.I. Webb. Stochastic attribute selection committees. In Proc. of the 11th Australian Joint Conf. on AI (AI'98), pages 321-332, 1998.

....(Kohavi, 1995) were carried out for each algorithm. Some of these 37 domains contain missing attribute values, but some not. To simulate the situation where unseen examples contain certain amount of missing attribute values, we randomly introduce missing attribute values into test 3 In Zheng Webb (1998a; 1998b) We use 40 domains. To reduce the computational requirements of the experiments for this study, we exclude three largest domains from the test suite. The partial results in these 3 domains that we have got are consistent with our claims in this paper. 0 10 20 30 40 50 missing value level L ( in test ....

Zheng, Z. & Webb, G.I. 1998a. Stochastic Attribute Selection Committees. Technical Report (TR C98/08), School of Computing and Mathematics, Deakin University (available at http://www3.cm.deakin.edu.au/~zijian/Papers/sasc-tr-C98-08.ps.gz).

.... Although Boosting is generally more accurate than Bagging, the performance of Boosting is more variable than that of Bagging (Quinlan 1996; Bauer and Kohavi 1998) As an alternative approach to generating different classifiers to form a committee, Sasc (Stochastic Attribute Selection Committees) (Zheng and Webb 1998) builds different classifiers by modifying the set of attributes considered at each node, while the distribution of the training set is kept unchanged. Each attribute set is selected stochastically. Experiments show that Sasc, like Boosting, can also significantly reduce the error rate 1 ....

....can also significantly reduce the error rate 1 Committees are also referred to as ensembles (Dietterich 1997) 2 Breiman (1996b) refers to Boosting as the arcing (adaptively resample and combine) method. of decision tree learning, although the two techniques use quite different mechanisms (Zheng and Webb 1998). In addition, Sasc is more stable than Boosting (Zheng and Webb 1998) There are some other classifier committee learning approaches such as generating multiple trees by manually changing learning parameters (Kwok and Carter 1990) errorcorrecting output codes (Dietterich and Bakiri 1995) and ....

[Article contains additional citation context not shown here]

Zheng, Z. and Webb, G.I. 1998. Stochastic Attribute Selection Committees. Technical Report (TR C98/08), School of Computing and Mathematics, Deakin University (available at http://www3.cm.deakin.edu.au/~zijian/Papers/sasc-tr-C98-08.ps.gz).

....and uses equal weight voting. Although Boosting is generally more accurate than Bagging, the performance of Boosting is more variable than that of Bagging [4, 11] As an alternative approach to generating different classifiers to form a committee, Sasc (Stochastic Attribute Selection Committees) [13] builds different classifiers by modifying the set of attributes considered at each node, while the distribution of the training set is kept unchanged. Each attribute set is selected stochastically. Experiments show that Sasc, like Boosting, can also significantly reduce the error rate of decision ....

....classifiers by modifying the set of attributes considered at each node, while the distribution of the training set is kept unchanged. Each attribute set is selected stochastically. Experiments show that Sasc, like Boosting, can also significantly reduce the error rate of decision tree learning [13]. In addition, Sasc is more stable than Boosting [13] Sasc [13] is a minor variant of a class of committee learning algorithm that learns a committee by randomizing the base learning process [1, 7, 8, 9] While Sasc has not been directly compared with these alternatives, comparisons of reported ....

[Article contains additional citation context not shown here]

Zheng, Z. and Webb, G.I.: Stochastic attribute selection committees. Advanced Topics in Artificial Intelligence: Proceedings of the 11th Australian Joint Conference on Artificial Intelligence. Berlin: Springer-Verlag (1998a).

....authors knowledge, no study has been carried out on the effect of missing attribute values on the accuracy performance of committee learning techniques. In this paper, we study the robustness of four recently developed committee learning techniques including Boosting [4, 7] Bagging [3] Sasc [8], and SascMB [9] relative to C4.5 for tolerating missing values in test data. The motivation for this research is as follows. These four committee learning techniques can dramatically reduce the error of C4.5. This observation was obtained from experiments in domains containing no or a small ....

....modified based on the performance of the previous classifiers. The objective is to make the generation of the next classifier concentrate on the training examples that are misclassified by the previous classifiers. Boost and Bag are our implementations of Boosting and Bagging respectively. Sasc [8] builds different classifiers by modifying the set of attributes considered at each node, while the distribution of the training set is kept unchanged. Each attribute set is selected stochastically. SascMB is a combination of Boosting, Bagging, and Sasc. It generates N subcommittees. Each ....

Zheng, Z. and Webb, G.I.: Stochastic attribute selection committees. Proceedings of the 10th Australian Joint Conference on Artificial Intelligence. Berlin: SpringerVerlag (1998).

.... and Kong 1995; Ali 1996) learning option trees (Buntine 1990; Kohavi and Kunz 1997) training a committee of neural networks by manually selecting attribute subsets (Cherkauer 1996; Tumer and Ghosh 1996) learning naive Bayesian classifier committees by randomly choosing attribute subsets (Zheng 1998), and creating committees for first order learning by adding random selection of conditions to FOIL (Ali and Pazzani 1996) Finally, different base learning algorithms can be used for learning different classifiers in committees (Wolpert 1992) A collection of recent research in this area and ....

....to form a test for the decision node from the subset. A different random subset is selected at each node of the tree. The modified version of C4.5 is called C4.5SAS (C4.5 Stochastic Attribute Selection) For a more detailed description of the C4.5Sas algorithm, see the long version of this paper (Zheng and Webb 1998). The only difference between C4.5Sas and C4.5 is that when growing a tree, at a decision node, C4.5Sas creates an attribute subset and uses the best attribute in it to form a test as described above. All other parts are identical for these two algorithms. With P = 1, C4.5Sas generates the same ....

[Article contains additional citation context not shown here]

Zheng, Z. and Webb, G.I.: Stochastic attribute selection committees. Technical Report (TR C98/08), School of Computing and Mathematics, Deakin University, Australia (1998) (available at http://www3.cm.deakin.edu.au/~zijian/Papers/ sasc-tr-C98-08.ps.gz).

.... focused on Boosting and Bagging, other classifier committee learning approaches have also been developed, including generating multiple trees by manually changing learning parameters [14] errorcorrecting output codes [15] generating decision tree committees by stochastically selecting attributes [7, 8, 16], learning option trees [17, 18] training a committee of neural networks by manually selecting attribute subsets [19, 20] learning naive Bayesian classifier committees by randomly choosing attribute subsets [21] creating Gaussian classifier committees by varying attribute sets [22] and ....

.... A collection of recent research in this area and reviews of related methods can be found in [9, 25, 8] As an alternative approach to creating classifier committees, the stochastic attribute selection committee learning method can also significantly reduce the error rates of decision tree learning [7, 8, 16]. It builds different classifiers by stochastically modifying the set of attributes considered during induction, while the distribution of the training set is kept unchanged. We have shown that Sasc, a variant of this stochastic attribute selection committee learning method, 1 and Bagging are ....

[Article contains additional citation context not shown here]

Zheng, Z. and Webb, G.I.: Stochastic attribute selection committees. Technical Report (TR C98/08), School of Computing and Mathematics, Deakin University, Australia (1998) (available at http://www3.cm.deakin.edu.au/~zijian/Papers/ sasc-tr-C98-08.ps.gz).

.... 1 learning techniques have been developed with great success (Freund 1996; Freund and Schapire 1996a; 1996b; Quinlan 1996; Breiman 1996a; 1996b; Dietterich and Kong 1995; Ali 1996; Chan, Stolfo, and Wolpert 1996; Schapire, Freund, Bartlett, and Lee 1997; Domingos 1997; Bauer and Kohavi 1998; Zheng and Webb 1998), especially Boosting 2 (Freund and Schapire 1996b; Quinlan 1996; Bauer and Kohavi 1998) This type of technique generates several classifiers to form a committee by using a single base learning algorithm. At the classification stage, the committee members vote to make the final decision. Given ....

....method. function of the performance of a classifier as the weight for voting, while the former uses equal weight voting. In contrast to Bagging and Boosting, Sasc (Stochastic Attribute Selection Committees) adopts an alternative approach to generating different classifiers to form a committee (Zheng and Webb 1998). It builds different classifiers by modifying the set of attributes considered at each node, while the distribution of the training set is kept unchanged. The selection of an attribute set is carried out stochastically. Experiments show that as Boost, Sasc can also significantly reduce the error ....

[Article contains additional citation context not shown here]

Zheng, Z. and Webb, G.I. 1998. Stochastic Attribute Selection Committees. Technical Report (TR C98/08), School of Computing and Mathematics, Deakin University, Australia (available at http://www3.cm.deakin.edu.au/~zijian/Papers/ sasc-tr-C98-08.ps.gz).

....uses 2 Breiman [3] refers to Boosting as the arcing (adaptively resample and combine) method. equal weight voting. In contrast to Bagging and Boosting, SASC (Stochastic Attribute Selection Committees) adopts an alternative approach to generating different classifiers to form a committee [21]. It builds different classifiers by modifying the set of attributes considered at each node during tree induction, while the distribution of the training set is kept unchanged. The selection of an attribute set is carried out stochastically. Experiments show that as BOOST, SASC can also ....

....while the distribution of the training set is kept unchanged. The selection of an attribute set is carried out stochastically. Experiments show that as BOOST, SASC can also significantly reduce the error rate of decision tree learning, although the two techniques use quite different mechanisms [21]. In the light of this finding, we propose, in this paper, a novel approach to further improving the accuracy of decision tree learning. The new approach is called SASCB (Stochastic Attribute Selection Committees with Boosting) a combination of the Boosting and SASC techniques. Since SASC and ....

[Article contains additional citation context not shown here]

Z. Zheng and G. Webb. Stochastic attribute selection committees. In Proceedings of the Eleventh Australian Joint Conference on Artificial Intelligence. Berlin: Springer-Verlag, 1998.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC