Abstract

Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.

Citations

545	Some Studies in Machine Learning using the Game of Checkers - Samuel - 2000
427	Integrated architectures for learning, planning, and reacting based on approximating integrated architectures for learning, planning, and reacting based on approximating dynamic programming - Sutton - 1990
232	A New Approach to Manipulator control: the Cerebellar Model Articulation Controller - Albus - 1975
219	Temporal Credit Assignment in Reinforcement Learning - Sutton - 1984
185	Learning and sequential decision making - Barto, Sutton - 1990
117	input generalization in delayed reinforcement learning: an algorithm and performance comparisons - Chapman, Kaelbling
105	Realtime learning and control using asynchronous dynamic programmming - Baro, Bradtke, et al. - 1991
94	Efficient Memory-based Learning for Robot Control. Unpublished - Moore - 1990
78	Efficient algorithms with neural network behavior - Omohundro - 1987
58	Active Exploration in Dynamic Environments - Thrun, Moeller - 1992
49	Learning and problem solving with multilayer connectionist systems - Anderson - 1986
43	Consistency of HDP applied to a simple reinforcement learning problem - Werbos - 1990
34	The learning of world models by connectionist networks - Sutton, Pinette - 1985
16	Navigating through temporal difference - Dayan - 1987
10	Reinforcing connectionism: Learning the statistical way - Dayan - 1991
9	The convergence of TD(A) for general A - Dayan - 1992
4	Learningfrom delayed rewards - Watkins - 1989
2	Learning to predict by the methods of temporal difference - Thesis - 1988