Abstract:
Many existing constructive decision tree learning algorithms such as Fringe and Citre construct conjunctions or disjunctions directly from paths of decision trees. This paper investigates a novel attribute construction method for decision tree learning. It creates conjunctions from production rules that are transformed from decision trees. Irrelevant or unimportant conditions are eliminated when paths are transformed into production rules. Therefore, this new method is likely to construct new attributes with relevant conditions. Three constructive induction algorithms based on this basic idea are described and are empirically evaluated by comparing with C4.5 and a Fringe-like algorithm in a set of artificial and natural domains. The experimental results reveal that constructing conjunctions using production rules can significantly improve the performance of decision tree learning in the majority of the domains tested in terms of both higher prediction accuracy and lower theory complexity. These results suggest an advantage of the attribute construction method that uses production rules over the method of constructing new attributes directly from paths in noisy domains.
Citations
|
2438
|
Classification and Regression Trees
– Breiman, Friedman, et al.
- 1984
|
|
2138
|
UCI Repository of Machine Learning Databases
– Merz, Murphy
- 1996
|
|
509
|
C4.5: Programs for
– Quinlan
- 1993
|
|
280
|
A universal prior for integers and estimation by minimum description length
– Rissanen
- 1983
|
|
265
|
Inferring decision trees using the minimum description length principle
– Quinlan, Rivest
- 1989
|
|
191
|
Boolean feature discovery in empirical learning
– Pagallo, Haussler
- 1990
|
|
125
|
Generating production rules from decision trees
– Quinlan
- 1987
|
|
103
|
HypothesisDriven Constructive Induction in AQ17-HCI: A Method and Experiments
– Wnek, Michalski
- 1994
|
|
82
|
Constructive Induction on Decision Trees
– Matheus, Rendell
- 1989
|
|
81
|
Coding decision trees
– Wallace, Patrick
- 1993
|
|
54
|
Id2-of-3: Constructive induction of m-of-n concepts for discriminators in decision trees
– Murphy, Pazzani
- 1991
|
|
53
|
Lookahead feature construction for learning hard concepts
– Ragavan, Rendell
- 1993
|
|
38
|
Data-driven constructive induction
– Bloedorn, Michalski
- 1998
|
|
37
|
Feature Construction: An Analytic Framework and an Application to Decision Trees
– Matheus
- 1989
|
|
31
|
Multivariate versus univariate decision trees
– Brodley, Utgoff
- 1992
|
|
29
|
Pattern Recognition as Knowledge-Guided Computer Induction
– Michalski
- 1978
|
|
25
|
Adaptive Decision Tree Algorithms for Learning From Examples
– Pagallo
- 1990
|
|
16
|
A scheme for feature construction and a comparison of empirical methods
– YANG, RENDELL, et al.
- 1991
|
|
6
|
Constructing New Attributes for Decision Tree Learning
– Zheng
- 1996
|
|
1
|
2000 37 Constructing Conjunctive Attributes using Production Rules BLOEDORN
– E, MICHALSKI
|
|
1
|
Dr Z Zheng is a research fellow at School of Computing and Mathematics
– NOTES
- 1996
|
