Learning from scarce experience

Learning from scarce experience (2002)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Leonid Peshkin , Christian R. Shelton

Venue:	Proceedings of the Nineteenth International Conference on Machine Learning
Citations:	18 - 0 self

BibTeX

@INPROCEEDINGS{Peshkin02learningfrom,
    author = {Leonid Peshkin and Christian R. Shelton},
    title = {Learning from scarce experience},
    booktitle = {Proceedings of the Nineteenth International Conference on Machine Learning},
    year = {2002},
    pages = {498--505}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

Searching the space of policies directly for the optimal policy has been one popular method for solving partially observable reinforcement learning problems. Typically, with each change of the target policy, its value is estimated from the results of following that very policy. This requires a large number of interactions with the environment as different polices are considered. We present a family of algorithms based on likelihood ratio estimation that use data gathered when executing one policy (or collection of policies) to estimate the value of a different policy. The algorithms combine estimation and optimization stages. The former utilizes experience to build a non-parametric representation of an optimized function. The latter performs optimization on this estimate. We show positive empirical results and provide the sample complexity bound. 1.

Citations

6694	Statistical Learning Theory - Vapnik - 1998
2827	Reinforcement Learning: An Introduction - Sutton, Barto - 1998
313	Simulation and the Monte Carlo Method - Rubinstein - 1981
262	Policy gradient methods for reinforcement learning with function approximation - Sutton, Mcallester, et al. - 2000
116	Gradient Descent for General Reinforcement Learning - Baird, Moore - 1999
114	Learning to cooperate via policy search - Peshkin, Kim, et al. - 2000
99	Universal Prediction - Merhav, Feder - 1998
97	Approximate planning in large POMDPs via reusable trajectories - Kearns, Mansour, et al. - 2000
88	Importance Sampling for Stochastic Simulations - Glynn, Iglehart - 1989
67	Learning finitestate controllers for partially observable environments - Meuleau, Peshkin, et al. - 1999
61	Reinforcement learning in pomdp’s via direct gradient ascent, in - Baxter, Bartlett - 2000
49	Likelihood ratio gradient estimation for stochastic systems - Glynn - 1990
45	Kaelbling. Learning policies with external memory - Peshkin, Meuleau, et al. - 1999
32	Eligibility traces for off-policy policy evaluation - Precup, Sutton, et al. - 2000
24	Importance Sampling for Reinforcement Learning with Multiple Objectives - Shelton - 2001
23	Simulation-based methods for Markov Decision Processes - Marbach - 1998
21	Exploration in gradient-based reinforcement learning - Meuleau, Peshkin, et al. - 2001
16	Optimization of stochastic systems - Glynn - 1986
15	2001, Policy Improvement for POMDPs Using Normalized Importance Sampling - Shelton
11	Kernel-based reinforcement learning in average-cost problems - Ormoneit, Glynn
11	Bounds on sample size for policy evaluation in markov environments - Peshkin, Mukherjee - 2001
8	Importance sampling for markov chains: asymptotics for thevariance - Glynn - 1994
7	Function optimization using connectionist reinforcement learning algorithms - Williams, Peng - 1991
6	Surveys in Combinatorics, chapter On the method of bounded differences - McDiarmid - 1989
5	Universal sequential coding of single measures - Shtarkov - 1987
2	personal communication - Szepesvari - 1999
1	Architectures for policy search. Doctoral dissertation - Peshkin - 2001