Abstract

This paper describes two methods for hierarchically organizing temporal behaviors. The first is more intuitive: grouping together common sequences of events into single units so that they may be treated as individual behaviors. This system immediately encounters problems, however, because the units are binary, meaning the behaviors must execute completely or not at all, and this hinders the construction of good training algorithms. The system also runs into difficulty when more than one unit is (or should be) active at the same time. The second system is a hierarchy of transition values. This hierarchy dynamically modifies the values that specify the degree to which one unit should follow another. These values are continuous, allowing the use of gradient descent during learning. Furthermore, many units are active at the same time as part of the system's normal functionings. 1 Introduction The importance of hierarchy in adaptive systems that perform temporal tasks has been noted often...

Citations

1060	Learning to predict by the methods of temporal differences - Sutton - 1988
448	Adaptive switching circuits - Widrow, Hoff - 1960
427	Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming - Sutton - 1990
197	The induction of dynamical recognizers - Pollack - 1991
185	Learning and sequential decision making - Barto, Sutton - 1990
158	Made-Up Minds: A Constructivist Approach to Artificial Intelligence - Drescher - 1991
157	Neuronlike elements that can solve difficult learning control problems - Barto, Sutton, et al. - 1983
151	Transfer of learning by composing solutions of elemental sequential tasks - Singh - 1992
132	Finite state automata and simple recurrent networks - Cleeremans, Servan-Schreiber, et al.
105	Realtime learning and control using asynchronous dynamic programmming - Baro, Bradtke, et al. - 1991
82	Induction of finite-state languages using second-order recurrent networks - Watrous, Kuhn - 1992
67	The Recurrent Cascade-Correlation Architecture - Fahlman - 1991
54	Induction of multiscale temporal structure - Mozer - 1992
37	Extracting and learning an unknown grammar with recurrent neural networks - Giles, Miller, et al. - 1992
30	Hierarchical Credit Allocation in a Classifier System - Wilson - 1987
26	Hierarchical organisation: a candidate principle for ethology - Dawkins - 1976
26	Learning sequential tasks by incrementally adding higher orders - Ring - 1993
24	Learning unambiguous reduced sequence descriptions - Schmidhuber - 1992
23	Incremental development of complex behaviors through automatic construction of sensory-motor hierarchies - Ring - 1991
16	Scaling reinforcement learning techniques via modularity - Wixson - 1991
13	Mechanisms of planning and problem solving in the brain - Albus - 1979
11	Connectionist Modeling and Control of Finite State Environments - Bachrach - 1992
10	Cognitive action theory as a control architecture - Roitblat