On-line learning in domains where the target concept depends on some hidden context poses serious problems. Context shifts can induce changes in the target concepts, producing what is known as concept drift. We describe a family of learning algorithms that flexibly react to concept drift and can take advantage of situations where contexts reappear. The general approach underlying all these algorithms consists of (1) keeping only a window of currently trusted examples and hypotheses; (2) storing concept descriptions and re-using them if a previous context re-appears; and (3) controlling both of these functions by a heuristic that constantly monitors the system's behavior. The paper reports on experiments that test the systems ' performance under various levels noise and different extent and speed of concept drift. Key words. Incremental concept learning, on-line learning, context dependence, concept drift, forgetting
|
1395
|
A theory of the learnable
– Valiant
- 1984
|
|
809
|
Analyses of instance-based learning algorithms
– Albert, Aha
- 1991
|
|
631
|
A theory and methodology of inductive learning
– Michalski
- 1983
|
|
545
|
Learnability and the VapnikChervonenkis dimension
– Blumer, Ehrenfeucht, et al.
- 1989
|
|
480
|
Explanation-based generalization: a unifying view
– Mitchell, Keller, et al.
- 1986
|
|
284
|
Quantitative results concerning the utility of explanationbased learning
– Minton
- 1990
|
|
247
|
A weighted nearest neighbor algorithm for learning with symbolic features
– Cost, Salzberg
- 1993
|
|
245
|
Rough Sets
– PAWLAK
- 1991
|
|
155
|
A nearest hyperrectangle learning method
– Salzberg
- 1991
|
|
139
|
Competitive learning from interactive activation to adaptive resonance
– Grossberg
- 1987
|
|
116
|
The ART of adaptive pattern recognition by self-organizing neural networks
– Carpenter, Grossberg
- 1988
|
|
88
|
Experiments with incremental concept formation
– Lebowitz
- 1987
|
|
63
|
Neurocomputing
– Hecht-Nielsen
- 1989
|
|
60
|
Remembering
– Bartlett
- 1932
|
|
56
|
Tracking Drifting Concepts by Minimizing Disagreements
– Helmbold, Long
- 1994
|
|
55
|
Incremental learning from noisy data
– Schlimmer, Granger
- 1986
|
|
38
|
Average-case analysis of a nearest neighbor algorithm
– Langley, Iba
- 1993
|
|
32
|
Some tests of the decay theory of immediate memory
– Brown
- 1958
|
|
25
|
Learning timevarying concepts
– Kuh, Petsche, et al.
- 1991
|
|
24
|
Associative reinforcement learning
– Kaelbling
- 1994
|
|
24
|
Fast, Robust Adaptive Control by Learning only Forward Models
– Moore
- 1992
|
|
23
|
Robust classification with context-sensitive features
– Turney
- 1993
|
|
22
|
Density-Adaptive Learning and Forgetting
– Salganicoff
- 1993
|
|
18
|
Tracking drifting concepts using random examples
– Helmbold, Long
- 1991
|
|
18
|
Floating Approximation in Time-Varying Knowledge Bases
– Kubat
- 1989
|
|
17
|
On-line Learning with an Oblivious Environment and the Power of Randomization
– Maass
- 1991
|
|
16
|
Some Chunks are Expensive
– Tambe, Newell
- 1988
|
|
12
|
COBBIT - A Control Procedure for COBWEB in the Presence of Concept Drift
– Kilander, Jansson
- 1993
|
|
11
|
Learning Flexible Concepts from Streams of Examples: FLORA2
– Widmer, Kubat
- 1992
|
|
9
|
A Machine Learning Based Approach to Load Balancing in Computer Networks
– Kubat
- 1992
|
|
9
|
Without surreptitious rehearsal, information in short-term memory decays
– Reitman
- 1974
|
|
8
|
The Role of Forgetting
– Markovitch, Scott
- 1988
|
|
7
|
Hill-Climbing Theories of Learning
– Langley, Gennari, et al.
- 1987
|
|
7
|
Combining Robustness and Flexibility in Learning Drifting Concepts
– Widmer
- 1994
|
|
6
|
Incrementally Learning Time-varying Half-planes
– Kuh, Petsche, et al.
- 1992
|
|
6
|
The Mental Representation of the Meaning of Words
– Johnson-Laird
- 1987
|
|
3
|
Conceptual Inductive Learning: The Case of Unreliable Teachers
– Kubat
- 1991
|
|
3
|
Detecting Small Amounts of Information in Memory: Savings for Nonrecognized Items
– Nelson
- 1978
|
|
2
|
Explicit Forgetting Algorithms for Memory-Based Learning
– Salganicoff
- 1993
|
|
1
|
Apple-Tasting and Nearly One-Sided Learning
– Helmbold, Littlestone, et al.
- 1992
|
|
1
|
FAVORIT: Concept Formation with Ageing
– Krizakova, Kubat
- 1992
|
|
1
|
Flexible Concept Learning in Real-Time Systems. Journal of Intelligent and Robotic Systems 8, pp.155--171
– Kubat
- 1993
|
|
1
|
Density-AdaptiveLearning and Forgetting
– Salganicoff
- 1993
|
|
1
|
Reprinted in: Veith of
– Gedachtnis
- 1985
|
|
1
|
Struktur und Aktivierung des Gedachtnisses. Das Vernetzungsmodell: Grundlagen und Elemente einer ubergreifenden Theorie
– Klimesch
- 1988
|
|
