We present a new machine learning method that, given a set of training examples, induces a definition of the target concept in terms of a hierarchy of intermediate concepts and their definitions. This effectively decomposes the problem into smaller, less complex problems. The method is inspired by the Boolean function decomposition approach to the design of switching circuits. To cope with high time complexity of finding an optimal decomposition, we propose a suboptimal heuristic algorithm. The method, implemented in program HINT (Hierarchy INduction Tool), is experimentally evaluated using a set of artificial and real-world learning problems. In particular, the evaluation addresses the generalization property of decomposition and its capability to discover meaningful hierarchies. The experiments show that HINT performs well in both respects.
|
5825
|
Introduction to Algorithms
– Cormen, Leiserson, et al.
- 2001
|
|
3215
|
C4.5: Programs for Machine Learning
– Quinlan
- 1993
|
|
2489
|
Induction of Decision Trees
– Quinlan
- 1986
|
|
655
|
UCI Repository of Machine Learning Databases [machine-readable data repository
– Murphy, Aha
- 1992
|
|
625
|
A theory and methodology of inductive learning
– Michalski
- 1983
|
|
469
|
Some studies in machine learning using the game of checkers
– Samuel
- 1959
|
|
128
|
The Decomposition of Switching Function
– Ashenhurst
- 1957
|
|
89
|
Identifying hierarchical structure in sequences: A linear-time algorithm
– Nevill-Manning, Witten
- 1997
|
|
89
|
On Comparing Classifiers: Pitfalls to Avoid and a Recommended Approach. Data Mining and Knowledge Discovery 1:3
– Salzberg
- 1997
|
|
68
|
KARDIO: A Study in Deep and Qualitative Knowledge for Expert Systems
– Bratko, I, et al.
- 1989
|
|
61
|
Upper Bound for the Chromatic Number of a Graph and Its Applications to Timetabling Problems
– Welsh, Powell
- 1967
|
|
53
|
Lookahead feature construction for learning hard concepts
– Ragavan, Rendell
- 1993
|
|
49
|
Bdd based decomposition of logic for functions with applications to FPGA synthesis
– Lai, Pedram, et al.
- 1993
|
|
47
|
Bottom-up induction of oblivious, read-once decision graphs
– Kohavi
- 1994
|
|
36
|
Understanding the nature of learning: Issues and research directions
– MICHALSKI
- 1986
|
|
27
|
Structured induction in expert systems. Turing Institute Press in association with Addison-Wesley
– Shapiro
- 1987
|
|
22
|
A new approach to the decomposition of incompletely specified functions based on graph-coloring and local transformations and its application to FPGA mapping
– Wan, Perkowski
- 1992
|
|
21
|
Inductive Acquisition of Expert Knowledge
– Muggleton
- 1990
|
|
20
|
DEX: An expert system shell for decision support. http://www-ai.ijs.si /MarkoBohanec/dex.html
– Bohanec
- 2002
|
|
19
|
The role of abstractions in learning qualitative models
– Mozetic
- 1987
|
|
17
|
Decomposition of multiple-valued functions
– Luba
- 1995
|
|
17
|
Tree-structured bias
– Russell
- 1988
|
|
17
|
Machine learning by function decomposition
– Zupan, Bohanec, et al.
- 1997
|
|
16
|
Learning nonoverlapping perceptron networks from examples and membership queries
– Hancock, Golea, et al.
- 1994
|
|
15
|
Problem decomposition and the learning of skills
– Michie
- 1995
|
|
13
|
Signature table systems and learning
– Biermann, Fairfield, et al.
- 1982
|
|
12
|
Knowledge acquisition and explanation for multiattribute decision making
– Bohanec, Rajkovic
- 1988
|
|
12
|
HELLERSTEIN: Learning boolean read-once formulas over generalized bases
– BSHOUTY, HANCOCK, et al.
- 1995
|
|
12
|
A New Approach to the Design of Switching Functions
– Curtis
- 1962
|
|
12
|
Pattern theoretic feature extraction and constructive induction
– Ross, Noviskey, et al.
- 1994
|
|
11
|
Feature transformation by function decomposition
– Zupan, Bohanec, et al.
- 1998
|
|
10
|
Pattern theoretic knowledge discovery
– Goldman
- 1994
|
|
10
|
OBDD-based function decomposition: algorithms and implementation
– Lai, Pan, et al.
- 1996
|
|
9
|
A management decision support system for allocating housing loans
– Bohanec, Cestnik, et al.
- 1996
|
|
9
|
Functional decomposition of mvl functions using multi-valued decision diagrams
– Files, Drechsler, et al.
- 1997
|
|
8
|
Overcoming the myopia of inductive learning algorithms with ReliefF
– Kononenko, Simec, et al.
- 1997
|
|
8
|
A general approach to boolean function decomposition and its application in FPGA-based synthesis
– Luba, Selvaraj
- 1995
|
|
8
|
Machine Learning Based on Function Decomposition
– Zupan
- 1997
|
|
7
|
Structuring knowledge by asking questions
– Muggleton
- 1987
|
|
7
|
An application for admission in public school systems
– Olave, Rajkovic, et al.
- 1989
|
|
6
|
An expert system for decision making
– Bohanec, Bratko, et al.
- 1983
|
|
6
|
Constructing intermediate concepts by decomposition of real functions
– Demsar, Zupan, et al.
- 1997
|
|
6
|
Learning of qualitative models
– Mozetic
- 1987
|
|
6
|
Controlling constructive induction in CiPF
– Pfahringer
- 1994
|
|
5
|
Multiattribute decisionmaking using a fuzzy heuristic approach
– Efstathiou, Rajkovic
- 1979
|
|
4
|
Automatic design of finite state machines with electronically programmable devices
– Perkowski, Uong
- 1987
|
|
4
|
Decision support by knowledge explanation
– Rajkovic, Bohanec
- 1991
|
|
3
|
Estimating attributes
– Kononenko
- 1994
|
|
3
|
Learning from examples and membership queries with structured determinations. Machine Learning
– Tadepalli, Russell
- 1998
|
|
3
|
A dataset decomposition approach to data mining and machine discovery
– Zupan, Bohanec, et al.
- 1997
|
|
