S. P. Meyn and R. Tweedie, Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes, Adv Appl Probab 25 (1993), 518--548.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....twice continuously di#erentiable if V # cV b1l C (4) for some constants b, c 0, and some compact non empty set C, where V (x) b i (x) #V (x) #x i 1 i,j a i,j (x) V (x) #x i #x j (5) is the mean velocity of V (X(t) at X(t) x. Proof The proof follows directly from [13] and using a similar argument to the proof of Theorem 2.1 in [19] 3 Langevin Tempered (LT) Di#usions 3.1 Motivation We now motivate the choice of a(x) # 2d u (x)I as a di#usion matrix. The di#usion X can be thought of as a time change of a tempered di#usion Z, defined by dZ t = log # ....

....ergodic with V = Bx, x) 1. Proof If we choose the test function V = # r , 0 r 1 2d, then from the definition of V (x) 5) we have that 2L V (x) # u 2d (x)V (x) r r(1 2d) # log # u (x) r# log # u (x) 4) follows now directly from (15) so that by Theorem 6. 1 of [13] the di#usion is exponentially ergodic. If we choose the test function V (x) Bx, x) 1, then (4) follows now directly from (16) The proof follows now directly from Theorem 2.1. 7 Example 1 The Multidimensional Exponential Class Pm : We consider the exponential family Pm introduced and ....

[Article contains additional citation context not shown here]

S.P. Meyn and R.L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

....(1993a) with their V : f and c; d as above) to conclude that P (lim m = 1jV (0) v) 1 for all v 0. Hence we may assume without loss of generality that limm = limTm j 1 and that V (t) is bounded on bounded intervals. Let A denote the extended generator of W . By ( and Theorem 2. 1(iii) in Meyn and Tweedie (1993a) we have for all c 0 E[V (t)jW (0) e ct (V (0) d=c) 2.17) Similarly as above we obtain E[V (t)jW (0) V (0) t Gamma E Z t 0 1fV (s) v 0 gV (s) d(W (s) r(V (s) ds fi fi V (0) GammaE Z t 0 1fV (s) v 0 gV (s) d(W (s) r(V (s) ds fi fi V (0) By ( d(W (s) 0 for V (s) v 0 and ....

S. P. Meyn and R.L. Tweedie, Stability of Markovian Processes III: Foster-Lyapunov criteria for continuous-time processes, Adv. Appl. Prob. 25 (1993a) 518--548.

....t 1, of A t to its stationary distribution or of moments of A t to its stationary moments. By strong statements we mean limit theorems together with rates of convergence. We show that, with the correct choice of the Lyapunov function, we may do so very easily, once we have the recent tools of Meyn and Tweedie (1993) and Tuominen and Tweedie (1994) available. Throughout, we assume that F is absolutely continuous with density f and let Z be a random variable with distribution F . We also assume 0 Gamma1 : EZ 1. The first assumption is not entirely essential from the point of view of renewal theory: ....

....g corresponding to an f via A (two g s may differ only on a set of potential zero see Revuz and Yor,1991, p.263) and thus we may consider the relation A as a function; we write g = Af as well. The extended generator of the age process is readily computed in the next section. Actually, as in Meyn and Tweedie (1993), we will assume that for each f in the domain of A we have E x R t 0 jAf(X s )jds 1, for all x; t. Regarding convergence concepts, we adopt the notion of p convergence: suppose that fX t g is Harris ergodic with stationary probability measure and let p : S R be some measurable function; ....

[Article contains additional citation context not shown here]

Meyn, S.P. and Tweedie, R.L. (1993) Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes. Adv. Appl. Prob. 25, 518--548.

....(with their V : f and c; d as above) to conclude that P (lim m = 1jV (0) v) 1 for 9 all v 0. Hence, without loss of generality, we assume that lim m = lim Tm j 1 and that V (t) is bounded on bounded intervals. Let A denote the extended generator of W . By (2.22) and Theorem 2. 1(iii) in Meyn and Tweedie (1993a) we have for all c 0 E[V (t)jW (0) e ct (V (0) d=c) 2.23) Similarly as above we obtain E[V (t)jW (0) V (0) t Gamma E Z t 0 1fV (s) v 0 gV (s) d(W (s) r(V (s) dsjV (0) GammaE Z t 0 1fV (s) v 0 gV (s) d(W (s) r(V (s) dsjV (0) By (2.18) d(W (s) 0 for V (s) v 0 and combining ....

S. P. Meyn and R.L. Tweedie, Stability of Markovian Processes III: Foster-Lyapunov criteria for continuous-time processes, Adv. Appl. Prob. 25 (1993a) 518--548.

....and for the time being we shall assume the existence of these states (though in Section 5 we will show how this hypothesis can be eliminated) Notice that in our context it is possible for more than one maximal state to exist. However, since the set of maximal states is an atom in the sense of [22], it is straightforward to produce an SRS which merges all chains started at maximal states in a single iteration. Let X (x;T ) n be the value at time n of a chain started in x at time GammaT . The perfect simulation algorithm works as follows. Choose a time T 1 0: Generate a stream of ....

S.P. Meyn and R.L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Advances in Applied Probability, 25:518--548, 1993.

No context found.

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Ann. Appl. Probab., 25:518--548, 1993.

No context found.

S. P. Meyn and R. L. Tweedie, "Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes," Adv. Appl. Probab., vol. 25, pp. 518--548, 1993.

No context found.

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 1993 (to appear).

No context found.

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518-548, 1993.

....from the geometric ergodicity of the embedded skeletons or, under appropriate continuity conditions (in t) on the semigroup P t , of a form of simultaneous geometric ergodicity of the resolvent chains. These simultaneity and continuity conditions are shown to be redundant in Section 6. In [17] it was shown that, as in (b) a drift condition on the generator is sufficient to guarantee exponential ergodicity for the process. This generalized results known for countable spaces [27] and for diffusion processes [8] to quite general models, and as shown in (for example) 17, 24] the ....

....in Section 6. In [17] it was shown that, as in (b) a drift condition on the generator is sufficient to guarantee exponential ergodicity for the process. This generalized results known for countable spaces [27] and for diffusion processes [8] to quite general models, and as shown in (for example) [17, 24], the conditions on the generator then provide practical criteria for evaluating the exponential convergence of specific models. Here we show that such a drift for the extended generator is also necessary for exponential ergodicity. The conclusions in this paper thus strengthen the known results ....

[Article contains additional citation context not shown here]

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

.... time t and 0, if (C) 0 and there exists a probability measure ( Delta) on X satisfying the minorisation condition P t (x; Delta) Delta) x 2 C : 14) In this context we utilise drift conditions which are based on the generator of P (for background and discussion see, e.g. [10]) We let A be the weak generator of our Markov process, and let D be the one sided extended domain consisting of all functions W : X IR which satisfy the one sided Dynkin s formula E x (W (X t ) E x Z t 0 AW (X s ) ds W (x) 15) This formula holds with equality if W is in the ....

.... satisfy the one sided Dynkin s formula E x (W (X t ) E x Z t 0 AW (X s ) ds W (x) 15) This formula holds with equality if W is in the domain of the strong generator; furthermore, it holds at least with inequality if W is in the domains of certain stopped versions of the process [10]. We now say that P satisfies a drift condition if there exists a small set C and a function V 2 D, such that for positive constants d; ffi and , V 1; sup x2C V (x) d; AV (x) GammaffiV (x) 1 C (x) 16) 3 Continuous time regeneration bounds 8 In order to construct a geometric trials ....

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

....1995 Abstract In this paper we define a class of continuous time threshold ARMA (CTARMA) processes uniquely in terms of the weak solution of a certain stochastic differential equation, and investigate stability properties of these processes. We apply criteria for stability of weak solutions (see [13, 16, 17]) to CTARMA processes and thus obtain criteria for transience, Harris recurrence, positive Harris recurrence and geometric ergodicity for these processes. In order to do this it is shown that CTARMA processes satisfy suitable continuity conditions, and so can be analyzed as irreducible ....

....to CTARMA processes and thus obtain criteria for transience, Harris recurrence, positive Harris recurrence and geometric ergodicity for these processes. In order to do this it is shown that CTARMA processes satisfy suitable continuity conditions, and so can be analyzed as irreducible T processes [13]. KEYWORDS: Continuous time SETARMA models, Exponential Ergodicity, Irreducible Markov processes, Non linear Time Series, Recurrence, Stochastic Differential Equations, Stationary Distributions, Transience. AMS Subject Classifications: 60J60, 60J35, 60J70 1 Introduction In recent years there has ....

[Article contains additional citation context not shown here]

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

....(x) exp(ffiV (x) For sufficiently small ffi , we can perform the Taylor series approximation used in Theorem 16.3.1 of Meyn and Tweedie [39] to show that for some fl 0, b 1, and a compact set C, e AV (x) Gammafl V (x) b 1l C (x) The limit (10) then follows from Theorem 6. 1 of [41]. ut Fluid limit models and stability Chen and Mandelbaum in [5] show generally that complex network models can be approximated by a fluid limit model. This gives rise to a new approach to stability, which is far more general than the direct stochastic Lyapunov approach described above. By ....

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

No context found.

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518-548, 1993.

....Generalized Jackson networks are treated in Borovkov [15] Sigman [16] Meyn and Down [17] and Baccelli and Foss [18] Re entrant lines are considered in Kumar et al. 19, 20, 2, 21] In recent years, significant progress has been made in two lines of research. The work of Meyn and Tweedie [22, 23, 24, 25] gives a framework for the analysis of continuous time, general state space Markov processes, which is in particular applicable to jump processes such as queueing and storage models. The work of Harrison [26] Chen and Mandelbaum [27, 28] and Harrison and Nguyen [29] has focused on the 1 ....

.... Delta ) The following result shows that the f total variation norm distance between the transient and steady state distributions converges to zero with only minor extra conditions beyond what was assumed in Theorem 5.5. The proof follows from the drift property Proposition 6.1 and Theorem 5. 3 of [23]. Theorem 6.2 Suppose that Assumptions (A1) A3) hold, and that the fluid model is stable. Then we have kP t (x; Delta ) Gamma ( Delta )k fp 0; t 1; x 2 X: In particular, for each initial condition, lim t 1 E x [jQ(t)j p ] E [jQ(0)j p ] 1 ut 6 Convergence 27 6.3 Rates ....

S. P. Meyn and R. L. Tweedie, "Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes," Adv. Appl. Probab., vol. 25, pp. 518--548, 1993.

....of the embedded skeletons or, under appropriate continuity conditions (in t) on the semigroup P t , of a form of simultaneous 2 Discrete time analogues 2 geometric ergodicity of the resolvent chains. These simultaneity and continuity conditions are shown to be redundant in Section 6. In [17] it was shown that, as in (b) a drift condition on the generator is sufficient to guarantee exponential ergodicity for the process. This generalized results known for countable spaces [27] and for diffusion processes [8] to quite general models, and as shown in (for example) 17, 24] the ....

....in Section 6. In [17] it was shown that, as in (b) a drift condition on the generator is sufficient to guarantee exponential ergodicity for the process. This generalized results known for countable spaces [27] and for diffusion processes [8] to quite general models, and as shown in (for example) [17, 24], the conditions on the generator then provide practical criteria for evaluating the exponential convergence of specific models. Here we show that such a drift for the extended generator is also necessary for exponential ergodicity. The conclusions in this paper thus strengthen the known results ....

[Article contains additional citation context not shown here]

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

....methods are widespread in operations research and systems modeling, as is evident in, for example, 1, 34] It is natural to seek similar results for continuous time models. The obvious analogue of the drift operator Delta in (1) is the infinitesimal generator or one of its generalizations. In [24, 13], conditions are found on the extended generator of the process (as defined in Section 2 below) which enable classification of models as stable, essentially emulating (1) while in [33] conditions similar to (2) are formulated in continuous time to obtain 2 criteria for transience. This ....

....while in [33] conditions similar to (2) are formulated in continuous time to obtain 2 criteria for transience. This approach provides appropriate tools for establishing stability or instability for both diffusion type models and for jump deterministic models such as occur in operations research [24]. This paper surveys such powerful stochastic Lyapunov function methods for general state space Markov processes. Under the hypothesis of irreducibility, which is satisfied for a broad class of operations research and diffusion models [23] we obtain criteria for transience; recurrence; positive ....

[Article contains additional citation context not shown here]

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

.... and positive recurrence, in the sense that if in discrete time L(x; C) j 1 when C is petite, then the chain is Harris whilst if further E x [ C ] is bounded on C then the chain is positive Harris: these results are given in much more detail in [8] and their continuous time analogues are in [9, 7, 10, 2]. It is thus vital to identify these sets in a topological setting, and under Condition T we have exactly what we want. Theorem 5.1 If every compact set is petite then Phi is a T model; and conversely, if Phi is a irreducible T model then every compact set is petite. 6 The existence of a ....

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab., 1993 (to appear).

....Generalized Jackson networks are treated in Borovkov [15] Sigman [16] Meyn and Down [17] and Baccelli and Foss [18] Re entrant lines are considered in Kumar et al. 19, 20, 2, 21] In recent years, significant progress has been made in two lines of research. The work of Meyn and Tweedie [22, 23, 24, 25] gives a framework for the analysis of continuous time, general state space Markov processes, which is in particular applicable to jump processes such as queueing and storage models. The work of Harrison [26] Chen and Mandelbaum [27, 28] and Harrison and Nguyen [29] has focused on the dynamics ....

.... Delta ) The following result shows that the f total variation norm distance between the transient and steady state distributions converges to zero with only minor extra conditions beyond what was assumed in Theorem 5.5. The proof follows from the drift property Proposition 6.1 and Theorem 5. 3 of [23]. Theorem 6.2 Suppose that Assumptions (A1) A3) hold, and that the fluid model is stable. Then we have kP t (x; Delta ) Gamma ( Delta )k fp 0; t 1; x 2 X: In particular, for each initial condition, lim t 1 E x [jQ(t)j p ] E [jQ(0)j p ] 1 ut 6.3 Rates of convergence We ....

S. P. Meyn and R. L. Tweedie, "Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes," Adv. Appl. Probab., vol. 25, pp. 518--548, 1993.

....the form X n = f(X n Gamma1 ; X n Gammak ) W n ; where W n is again an i.i.d. set of noise variables but f is a non linear function. These cannot easily be analysed by traditional time series methods but the statespace representation is again Markovian. For details of such models see Meyn and Tweedie (1993a) and Tong (1990). iii) Perhaps the most spectacular recent growth in the use of Markov chains is as a tool in simulation theory. Gibbs sampling, the Metropolis Hastings algorithm, and other extensions to more general Markov chain Monte Carlo methods of simulation have had great impact on a number of areas (Gilks ....

Meyn, S. P. and Tweedie, R. L. (1993c). Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548.

....(X(t) satisfies the stochastic differential equation dV (X(t) L V (X(t) dt V (X(t) oe(X(t) dW (t) 27) where L V (x) V (x) Gammaa(x)x c(x) 1 2 oe 2 (x) V (x) is the mean velocity of V (X(t) at X(t) x. It is easily seen that L V is the extended generator as defined in [14]. Therefore we conclude that a CTAR(1) process is stable if there is a compact set C towards which the process drifts, in the sense that for some function f 0 and constant d 1 L V (x) Gammaf (x) d1l C (x) and conversely, if there is outward drift in the sense that for some bounded ....

....initial condition X 0 = x: Then X is Harris recurrent if lim jxj Gamma 1 [a(x)x 2 Gamma 2c(x)x] 0 (28) and transient otherwise. Proof From the form of the generator L V in (27) the proof that condition (28) is sufficient for Harris recurrence follows from Theorem 3.2 and Theorem 3. 3 of [14] with V (x) x 2 . The proof that condition (28) is necessary for recurrence follows from Theorem 3.2, Theorem 3.2 of [17] and Theorem 3.3 of [17] with V (x) exp( Gammax 2 ) ut In Figure 1 and Figure 2 we give two sample paths of CTAR(1) processes which have the same coefficients a(x) ....

[Article contains additional citation context not shown here]

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

....PIA does yield an optimal policy over the class of all nonlinear feedback control laws. 6 Continuous time processes Because the general theory of Harris processes in continuous time is now rather complete, the previous results carry over to this setting with few changes. The reader is referred to [32, 13, 34] for details on how to extend the theory of [33] to continuous time processes. We sketch here what is necessary for the understanding of the PIA in continuous time. The text [39] also describes methods for lifting continuous time optimality criteria from existing discrete time theory. This ....

S. P. Meyn and R. L. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous time processes. Adv. Appl. Probab., 25:518--548, 1993.

No context found.

S. P. Meyn and R. Tweedie, Stability of Markovian processes III: Foster-Lyapunov criteria for continuous-time processes, Adv Appl Probab 25 (1993), 518--548.

No context found.

S. P. Meyn and R. Tweedie. Stability of Markovian processes III: FosterLyapunov criteria for continuous-time processes. Adv. Appl. Probab., 25:518--548, 1993.

No context found.

Meyn, S. P. and Tweedie, R. L. Stability of Markovian processes III: Foster-Lyapunov criteria for continuous time processes. Adv. Appl. Probab. 25, 518--548 (1993).

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC