Abstract:
This paper discusses a dynamic path-based mathod for constructing conjunctions as new attributes for decision tree learning. It searches for conditions (attributevalue pairs) from paths to form new attributes. CAT, a constructive decision tree learning algorithm, which adopts this dynamic path-based method is described. It employs the hypothesis-driven strategy for constructing new attributes, and uses conjunction and negation (implicitly) as its constructive operators. Compared with other hypothesis-driven constructive decision tree learning algorithms such as algorithms of the Fringe family, the new idea of CAT is that it carries out systematic search with pruning over each path of a tree to select conditions for generating a conjunction. Therefore, in CAT, conditions for constructing new attributes are decided dynamically during search. Empirically investigation in a set of artificial and real-world domains shows that CAT can improve the performance of selective decision tree learning in terms of both higher prediction accuracy and lower theory complexity. In addition, it shows some performance advantages over the constructive decision tree learning algorithms that use a fixed path-based method and a fixed rule-based method to construct conjunctions as new attributes. 1
Citations
|
655
|
UCI Repository of Machine Learning Databases [machine-readable data repository
– Murphy, Aha
- 1992
|
|
509
|
C4.5: Programs for
– Quinlan
- 1993
|
|
265
|
Inferring decision trees using the minimum description length principle
– Quinlan, Rivest
- 1989
|
|
191
|
Boolean feature discovery in empirical learning
– Pagallo, Haussler
- 1990
|
|
162
|
The MONK’s problems – a performance comparison of different learning algorithms
– Thrun
- 1991
|
|
84
|
Search through Systematic Set Enumeration
– Rymon
- 1992
|
|
82
|
Constructive Induction on Decision Trees
– Matheus, Rendell
- 1989
|
|
67
|
Oversearching and layered search in empirical learning
– Quinlan, Cameron-Jones
- 1995
|
|
61
|
Efficiently Inducing Determinations: A Complete and Systematic Search Algorithm that Uses Optimal Pruning
– Schlimmer
- 1993
|
|
59
|
OPUS: An efficient admissible algorithm for unordered search
– Webb
- 1995
|
|
54
|
Id2-of-3: Constructive induction of m-of-n concepts for discriminators in decision trees
– Murphy, Pazzani
- 1991
|
|
53
|
Lookahead feature construction for learning hard concepts
– Ragavan, Rendell
- 1993
|
|
37
|
Feature Construction: An Analytic Framework and an Application to Decision Trees
– Matheus
- 1989
|
|
31
|
Multivariate versus univariate decision trees
– Brodley, Utgoff
- 1992
|
|
27
|
An SE-tree based characterization of the induction problem
– Rymon
- 1993
|
|
25
|
Adaptive Decision Tree Algorithms for Learning From Examples
– Pagallo
- 1990
|
|
23
|
Systematic search for categorical attribute-value data-driven machine learning
– Webb
- 1993
|
|
16
|
A scheme for feature construction and a comparison of empirical methods
– YANG, RENDELL, et al.
- 1991
|
|
14
|
Constructing nominal x-of-n attributes
– Zheng
- 1995
|
|
9
|
Constructing conjunctive tests for decision trees
– Zheng
- 1992
|
|
6
|
Constructing New Attributes for Decision Tree Learning
– Zheng
- 1996
|
|
4
|
ID2-of-3: Constructive Induction of Mof -N Concepts for Discriminators in Decision Trees
– Murphy, Pazzani
- 1991
|
|
2
|
Inclusive pruning: a new class of pruning rule for unordered search and its application to classification learning
– Webb
- 1996
|
|
1
|
An SE-tree based characterisation of the induction problem
– Rymon
- 1993
|
|
