Learning Policies with External Memory

Learning Policies with External Memory (2001)

Cached

Download Links

[arxiv.org]
[www.cs.brown.edu]
[wilma.cs.brown.edu]

Other Repositories/Bibliography

DBLP

by Leonid Peshkin , Nicolas Meuleau , Leslie Pack Kaelbling

Citations:

45 - 7 self

BibTeX

@MISC{Peshkin01learningpolicies,
    author = {Leonid Peshkin and Nicolas Meuleau and Leslie Pack Kaelbling},
    title = {Learning Policies with External Memory},
    year = {2001}
}

Years of Citing Articles

Bookmark

OpenURL

Abstract

In order for an agent to perform well in partially observable domains, it is usually necessary for actions to depend on the history of observations. In this paper, we explore a stigmergic approach, in which the agent’s actions include the ability to set and clear bits in an external memory, and the external memory is included as part of the input to the agent. In this case, we need to learn a reactive policy in a highly non-Markovian domain. We explore two algorithms: sarsa(λ), which has had empirical success in partially observable domains, and vaps, a new algorithm due to Baird and Moore, with convergence guarantees in partially observable domains. We compare the performance of these two algorithms on benchmark problems.

Citations

3116	A tutorial on hidden Markov models and selected applications in speech recognition - Rabiner - 1989
2827	Reinforcement Learning: An Introduction - Sutton, Barto - 1998
1135	Learning from delayed rewards - Watkins - 1989
629	Planning and acting in partially observable stochastic domains - Kaelbling, Littman, et al. - 1998
498	Markov Decision Processes - Puterman - 1994
417	Markov games as a framework for multi-agent reinforcement learning - Littman - 1994
243	Reinforcement Learning with Selective Perception and Hidden State - McCallum - 1996
181	Classifier systems and genetic algorithms - Booker, Goldberg, et al. - 1989
155	From local actions to global tasks: Stigmergy and collective robotics - Beckers, Holland, et al. - 1994
137	Reinforcement learning algorithm for partially observable Markov decision problems - Jaakkola, Singh, et al. - 1994
116	Gradient Descent for General Reinforcement Learning - Baird, Moore - 1999
115	Learning without state-estimation in partially observable markovian decision processes - Singh, Jaakkola, et al. - 1994
88	Memoryless policies: Theoretical limitations and practical results. In From Animals to Animats 3 - Littman - 1994
87	Solving POMDPs by search in policy space - Hansen - 1998
67	Learning finitestate controllers for partially observable environments - Meuleau, Peshkin, et al. - 1999
63	Continual Learning in Reinforcement Environments - Ring - 1994
49	Using eligibility traces to find the best memoryless policy in Partially Observable Markov Decision Processes - Loch, Singh - 1998
49	Solving POMDPs by searching the space of finite policies - Meuleau, Kim, et al. - 1999
31	Planning and control in stochastic domains with imperfect information. phd dissertation - Hauskrecht - 1997
29	Finite-Memory Control of Partially Observable Systems - Hansen - 1998
24	Model-free reinforcement learning for non-Markovian decision problems - Singh, Jaakkola, et al. - 1994
22	A class of gradient-estimating algorithms for reinforcement learning in neural networks - Williams - 1987
21	Csaba Szepesvari. Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms - Singh, Jaakkola, et al.
12	Learning controllers for partially observable environments - Meuleau, Peshkin, et al. - 1999
9	Classi systems and genetic algorithms - Booker, Goldberg - 1989
6	Solving POMDPs by searching the space of policies - Meuleau, Kim, et al. - 1999
5	Using eligibility traces to the best memoryless policy in partially observable Markov decision processes - Loch, Singh - 1998
2	Searching for finite-state POMDP controllers - Kaelbling, Kim, et al. - 1999
2	Reinforcement Learning for Embedded Agents facing Complex Tasks - Martin - 1998
2	Wiering and Juergen Schmidhuber. HQlearning - Marco - 1997
2	Reinforcement Learning for Embedded Agents facing Complex Tasks - Martn - 1998
1	Searching for POMDP controllers - Kaelbling, Kim, et al. - 1999