S.P. Meyn. Stability, Performance Evaluation, and Optimization To appear as a chapter in MARKOV DECISION PROCESSES: Models, Methods, Directions, and Open Problems, 2000. 15  Home/Search   Document Not in Database   Summary   Related Articles

..... 14 1 Introduction These lecture notes are a self contained treatment of the solution of equilibria equations and ergodicity for irreducible Markov chains . For a bit more depth, and in particular the treatment of general state spaces, see [12] (available at black.csl.uiuc.edu emeyn) or see [11] The Markov chains that we consider evolve on a countable state space, denoted X. The chain itself is denoted fX(t) t 2 ZZ g, with transition law denoted P : PfX(t 1) x j X(t) yg = P (y; x) x; y 2 X: 2 Equilibria equations There ....

.... to ergodic theorems such as, 1 N N 1 X t=0 f(X(t) N 1 E[f(X(t) t 1: 3) The existence of f leads to ner results: 1) The solution to Poisson s equation is central to optimal control where f is a one step cost function, and f is called the relative value function [5, 6, 1, 16, 12]. 2) Approximate solutions to Poisson s equation lead to direct performance bounds (estimates of ) 8, 15, 7, 3, 4] 3) The solution to Poisson s equation allows us to construct the useful martingale: M(t) t 1 X i=0 f(X(i) f(X(t) t 3 This leads to the central limit ....

S.P. Meyn. Stability, Performance Evaluation, and Optimization To appear as a chapter in MARKOV DECISION PROCESSES: Models, Methods, Directions, and Open Problems, 2000. 15

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC