In this paper, a hybrid learning approach named HDT is proposed. HDT simulates human reasoning by using symbolic learning to do qualitative analysis and using neural learning to do subsequent quantitative analysis. It generates the trunk of a binary hybrid decision tree according to the binary information gain ratio criterion in an instance space defined by only original unordered attributes. If unordered attributes cannot further distinguish training examples falling into a leaf node whose diversity is beyond the diversity-threshold, then the node is marked as a dummy node. After all those dummy nodes are marked, a specific feedforward neural network named FANNC that is trained in an instance space defined by only original ordered attributes is exploited to accomplish the learning task. Moreover, this paper distinguishes three kinds of incremental learning tasks. Two incremental learning procedures designed for example-incremental learning with different storage requirements are provided, which enables HDT to deal gracefully with data sets where new data are frequently appended. Also a hypothesis-driven constructive induction mechanism is provided, which enables HDT to generate compact concept descriptions.
|
3215
|
C4.5: Programs for Machine Learning
– Quinlan
- 1993
|
|
2489
|
Induction of Decision Trees
– Quinlan
- 1986
|
|
2438
|
Classification and Regression Trees
– Breiman, Friedman, et al.
- 1984
|
|
2138
|
UCI Repository of Machine Learning Databases
– Merz, Murphy
- 1996
|
|
625
|
A theory and methodology of inductive learning
– Michalski
- 1983
|
|
593
|
Hierarchical Mixtures of Experts and the EM algorithm
– Jordan, Jacobs
- 1993
|
|
551
|
The cascade-correlation learning architecture
– Fahlman, Lebiere
- 1991
|
|
301
|
Supervised and unsupervised discretization of continuous features
– Dougherty, Kohavi, et al.
- 1995
|
|
255
|
Adaptive pattern classification and universal recoding: Part I. Parallel development and codings of neural feature detectors
– Grossberg
- 1976
|
|
191
|
A system for induction of oblique decision trees
– Murthy, Kasif, et al.
- 1994
|
|
191
|
Boolean feature discovery in empirical learning
– Pagallo, Haussler
- 1990
|
|
157
|
Learning representations by back-propagating errors. Nature
– Rumelhart, Hinton, et al.
- 1986
|
|
151
|
An Empirical Comparison of Pruning Methods for Decision Tree
– Mingers
- 1987
|
|
131
|
Incremental induction of decision trees
– Utgoff
- 1989
|
|
113
|
Learning machines
– Nilsson
- 1965
|
|
102
|
Dynamic node creation in backpropagation networks
– Ash
- 1989
|
|
86
|
Decision tree induction based on efficient tree restructuring
– Utgoff, Berkman, et al.
- 1997
|
|
83
|
Error-based and entropy-based discretization of continuous features
– Kohavi, Sahami
- 1996
|
|
82
|
Constructive Induction on Decision Trees
– Matheus, Rendell
- 1989
|
|
74
|
A case study of incremental concept induction
– immer, Fisher
- 1986
|
|
63
|
Extracting tree-structured representations of trained networks
– Craven, Shavlik
- 1996
|
|
53
|
Perceptron trees - a case study in hybrid concept representation
– Utgoff
- 1988
|
|
51
|
Entropy nets: From decision trees to neural networks
– Sethi
- 1990
|
|
41
|
SuperSAB: Fast adaptive back propagation with good scaling properties. Neural Networks
– Tollenaere
- 1990
|
|
38
|
Data-driven constructive induction
– Bloedorn, Michalski
- 1998
|
|
37
|
Extensions to the CART algorithm
– Crawford
- 1989
|
|
37
|
Classification trees with neural network feature extraction
– Guo, Gelfand
- 1992
|
|
35
|
Logical versus Analogical or Symbolic versus Connectionist or Neat versus Scruffy
– Minsky
- 1991
|
|
31
|
Linear machine decision trees
– Utgoff, Brodley
- 1991
|
|
30
|
Decision graphs - an extension of decision trees
– Oliver
- 1993
|
|
27
|
A performance comparison of trained multilayer perceptrons and trained classification trees
– Atlas, Cole, et al.
- 1990
|
|
26
|
An incremental method for finding multivariate splits for decision trees
– Utgoff
- 1990
|
|
22
|
A tree-structured adaptive network for function approximation in high-dimensional spaces
– Sanger
- 1991
|
|
21
|
Probabilistic linear tree
– Gama
- 1997
|
|
17
|
Small nets and Short Paths: Optimising Neural Computation
– Frean
- 1990
|
|
17
|
Developing hybrid symbolic/connectionist models
– Hendler
- 1991
|
|
16
|
Decision trees can initialize radial-basis function networks
– Kubat
- 1998
|
|
16
|
Advanced methods in neural computing
– Wasserman
- 1993
|
|
14
|
Online ensemble learning: An empirical study
– Fern, Givan
- 2000
|
|
12
|
ANN-DT: An algorithm for extraction of decision trees from artificial neural networks
– Schmitz, Aldrich, et al.
- 1999
|
|
11
|
Efficient incremental induction of decision trees
– Kalles, Morris
- 1996
|
|
6
|
Structurally adaptive selforganizing neural trees
– Li, Fang, et al.
- 1992
|
|
6
|
Optimal pruning of neural tree networks for improved generalization
– Sankar, Mammone
- 1991
|
|
5
|
CNeT: Competitive neural trees for pattern classification
– Behnke, Karayiannis
- 1996
|
|
5
|
Neural units recruitment algorithm for generation of decision trees
– Deffuant
- 1990
|
|
5
|
The multiscale classifier
– Lovell, Bradley
- 1996
|
|
5
|
Integrating Neural and Symbolic Processes: The Cognitive Dimension
– Sun, L
- 1992
|
|
4
|
Structure-driven induction of decision tree classifiers through neural learning
– Sethi, Yoo
- 1997
|
|
4
|
Globally optimal fuzzy decision trees for classification and regression
– Suarez
- 1999
|
|
2
|
Learning fuzzy decision trees. Neural Networks
– Apolloni, Zamponi, et al.
- 1998
|
