Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding

DMCA

Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding (1996)

Cached

Download Links

Other Repositories/Bibliography

CiteULike
DBLP

by Richard S. Sutton

Venue:	Advances in Neural Information Processing Systems 8
Citations:	431 - 20 self

BibTeX

@INPROCEEDINGS{Sutton96generalizationin,
    author = {Richard S. Sutton},
    title = {Generalization in Reinforcement Learning: Successful Examples Using Sparse Coarse Coding},
    booktitle = {Advances in Neural Information Processing Systems 8},
    year = {1996},
    pages = {1038--1044},
    publisher = {MIT Press}
}

OpenURL

Abstract

On large problems, reinforcement learning systems must use parameterized function approximators such as neural networks in order to generalize between similar situations and actions. In these cases there are no strong theoretical results on the accuracy of convergence, and computational results have been mixed. In particular, Boyan and Moore reported at last year's meeting a series of negative results in attempting to apply dynamic programming together with function approximation to simple control problems with continuous state spaces. In this paper, we present positive results for all the control tasks they attempted, and for one that is significantly larger. The most important differences are that we used sparse-coarse-coded function approximators (CMACs) whereas they used mostly global function approximators, and that we learned online whereas they learned offline. Boyan and Moore and others have suggested that the problems they encountered could be solved by using actual outcomes (...

Keyphrases

successful example sparse coarse coding reinforcement learning large problem function approximation sparse-coarse-coded function approximators parameterized function approximators computational result neural network negative result strong theoretical result similar situation positive result last year global function approximators actual outcome control problem continuous state space important difference

DMCA