M. Kearns and D. Koller. Efficient Reinforcement Learning in Factored MDPs. Proceedings of the International Joint Conference on Artificial Intelligence, 16:740--747, 1999.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... Singh s E algorithm [ Kearns and Singh, 1998 ] was the first provably optimal polynomial time algorithm for learning in Markov decision processes (MDPs) E was extended later to handle single controller stochastic games (SCSGs) Brafman and Tennenholtz, 2000 ] as well as structured MDPs [ Kearns and Koller, 1999 ] In E the agent learns by updating a model of its environment using statistics it collects. This learning process continues as long as it can be done relatively efficiently. Once this is no longer the case, the agent uses its learned model to compute an optimal policy and follows it. The ....

M. Kearns and D. Koller. Efficient reinforcement learning in factored mdps. In Proc. 16th International Joint Conference on Artificial Intelligence (IJCAI), pages 740--747, 1999.

.... function approximators it allows practically efficient planning algorithms based on linear programming to be easily implemented [13, 22] Both lines of research exploration exploitation and compact representations have recently been brought together in the algorithm of Kearns and Koller [14]. This algorithm combines the theoretical exploration exploitation guarantees of E with the ability to scale up to large state spaces afforded by factored MDP representations. However, there is a shortcoming with the result: it relies on an oracle planning algorithm that must guarantee ....

....of states, among other quantities. Unfortunately, the number of states is usually too large to be handled explicitly (i.e. exponential) and hence these methods are not usually practical. Here again one could exploit structure in a factored MDP to make these algorithms feasible. Kearns and Koller [14] proposed Factored E version the E algorithm for factored MDPs. In Section 5, we present this algorithm and, in Section 6, we propose Factored Rmax , a similar extension for the Rmax algorithm. 3. Factored MDPs Factored MDPs allow one to exploit problem structure to represent ....

[Article contains additional citation context not shown here]

M. Kearns and D. Koller. Efficient reinforcement learning in factored MDPs. In Proc. IJCAI, 1999.

....and then progress slowed. It is only recently that provably correct and efficient algorithms for exploration in small nondeterministic state spaces became known (such as the E algorithm[4] and its generalizations[5] This approach has been generalized to factored MDPs under certain assumptions [3], but there remain many unresolved questions regarding efficient exploration in large MDPs, including whether model based approaches are required . In general, it is intuitively clear that any general exploration algorithm has a polynomial dependence on the size of the state (see [7] for a ....

....algorithm has a polynomial dependence on the size of the state (see [7] for a more formal statement) Hence, to obtain near optimal algorithms with sub linear dependence on the size of the state space further assumptions and restrictions on the MDP must be made. The factored E algorithm [3] considers one restriction where the MDP are represented in terms of a factored graph (ie a dynamic Bayes net) Here, the number of steps the agent must act in the MDP in order to obtain a T step near optimal policy is polynomial in the representation size of the factored graph. In this work, we ....

[Article contains additional citation context not shown here]

M. Kearns and D. Koller. "Efficient Reinforcement Learning in Factored MDPs". Proceedings of IJCAI, 1999.

No context found.

M. Kearns and D. Koller. Efficient Reinforcement Learning in Factored MDPs. Proceedings of the International Joint Conference on Artificial Intelligence, 16:740--747, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC