Greedy linear value-approximation for factored markov decision processes (2002) [18 citations — 7 self]
Popular Tags
No tags have been applied to this document.
BibTeX | Add To MetaCart
@INPROCEEDINGS{Patrascu02greedylinear,author = {Relu Patrascu and Pascal Poupart},
title = {Greedy linear value-approximation for factored markov decision processes},
booktitle = {In Proceedings of the 18th National Conference on Artificial Intelligence},
year = {2002},
pages = {285--291}
}
Years of Citing Articles
Abstract:
Significant recent work has focused on using linear representations to approximate value functions for factored Markov decision processes (MDPs). Current research has adopted linear programming as an effective means to calculate approximations for a given set of basis functions, tackling very large MDPs as a result. However, a number of issues remain unresolved: How accurate are the approximations produced by linear programs? How hard is it to produce better approximations? and Where do the basis functions come from? To address these questions, we first investigate the complexity of minimizing the Bellman error of a linear value function approximation—showing that this is an inherently hard problem. Nevertheless, we provide a branch and bound method for calculating Bellman error and performing approximate policy iteration for general factored MDPs. These methods are more accurate than linear programming, but more expensive. We then consider linear programming itself and investigate methods for automatically constructing sets of basis functions that allow this approach to produce good approximations. The techniques we develop are guaranteed to reduce Äerror, but can also empirically reduce Bellman error. 1
