#### DMCA

## Automated Verification Techniques for Probabilistic Systems

Citations: | 39 - 16 self |

### Citations

5092 |
Probability and measure
- Billingsley
- 1986
(Show Context)
Citation Context ...rising some definitions and notations relating to probability distributions and measures. We assume that the reader has some familiarity with basic probability theory. Good introductory texts include =-=[19,42]-=-. Definition 1 (Probability distribution). A (discrete) probability distribution over a countable set S is a function µ : S → [0, 1] satisfying ∑ s∈S µ(s)=1. We use [s0↦→x0, . . . , sn↦→xn] to denote ... |

3826 |
Dynamic Programming
- Bellman
- 1957
(Show Context)
Citation Context ...at are closely related to MDPs and clarifying certain differences in terminology used elsewhere. Our definition of MDPs in this tutorial essentially coincides with the classical definitions (see e.g. =-=[14,57,76]-=-), although there are notational differences. Also commonly used in probabilistic verification is the model of (simple) probabilistic automata (PAs), due to Segala [80,81]. These permit multiple distr... |

2912 |
An introduction to probability theory and its applications
- Feller
- 1968
(Show Context)
Citation Context ...rising some definitions and notations relating to probability distributions and measures. We assume that the reader has some familiarity with basic probability theory. Good introductory texts include =-=[19,42]-=-. Definition 1 (Probability distribution). A (discrete) probability distribution over a countable set S is a function µ : S → [0, 1] satisfying ∑ s∈S µ(s)=1. We use [s0↦→x0, . . . , sn↦→xn] to denote ... |

1849 |
Markov Decision Processes: Discrete Stochastic Dynamic Programming
- Puterman
- 1994
(Show Context)
Citation Context ...er [7], which provide a wealth of in-depth material on a variety of topics relating to MDPs, along with detailed pointers to other relevant literature. Finally, although not focusing on verification, =-=[76]-=- is an excellent general reference on MDPs. Outline. The tutorial is structured as follows. We begin, in Section 2, with background material on probability theory and discrete-time Markov chains. In S... |

1623 |
The temporal logic of programs
- Pnueli
- 1977
(Show Context)
Citation Context ...echniques of A ,(s0,q0)(reach(S×{q2}))=0.2. Therefore, using Proposition 1, we have Pr min M1,s0 (ΦA)=1−0.2=0.8, yielding s0 |= P�0.8[ΦA]. 7.2 LTL and ω-regular Properties LTL (linear temporal logic) =-=[75]-=- is a widely used temporal logic that is particularly well suited for expressing long-run properties of systems. As discussed above, in this presentation, we use LTL to define properties of MDPs in te... |

1089 |
Finite Markov Chains
- Kemeny, Snell
- 1960
(Show Context)
Citation Context ...ability that certain paths are taken. We proceed by constructing, for each state s ∈ S, a probability space over the set of infinite paths IPathD,s. This is outlined below and for further details see =-=[64]-=-. The basis of the construction is the probability of individual finite paths induced by the transition probability matrix P. More precisely, the probability of the path ρ=s0 . . . sn is given by P(ρ)... |

1009 | Design and synthesis of synchronization skeletons using branching-time temporal logic
- Clarke, Emerson
- 1981
(Show Context)
Citation Context ...ions 4 and 5 can be used to perform model checking of these properties. 6.1 The Logic PCTL PCTL (Probabilistic Computational Tree Logic) [54,18] is a probabilistic extension of the temporal logic CTL =-=[33]-=-. PCTL is used to express properties of both DTMCs [54] and MDPs [18]. Here, we focus on MDPs. In the last part of this section, we will extend PCTL to reward-based properties and, in Section 7, we wi... |

747 |
The Theory and Practice of Concurrency
- Roscoe
- 1997
(Show Context)
Citation Context ...(s2). This form of parallel composition [80,81], which allows multi-way synchronisation over the same action by several components, is in a similar style to the scheme used in the process algebra CSP =-=[79]-=- and is also used in PRISM [56]. By default, we assume that M1 and M2 synchronise over all common actions. However, this can easily be generalised to incorporate more flexible definitions of synchroni... |

737 |
Introduction to the Numerical Solution of Markov Chains
- Stewart
- 1994
(Show Context)
Citation Context ...e-time Markov chains (DTMCs). We provide just a brief overview of DTMCs, as required for the remainder of this tutorial. For more in-depth coverage of the topic, we recommend the textbooks by Stewart =-=[83]-=- and Kulkarni [65]. For a tutorial on probabilistic model checking for Markov chains, see for example [67]. Definition 3 (Discrete-time Markov chain). A discrete-time Markov chain (DTMC) is a tuple D ... |

728 | Dynamic Programming and Markov Processes - Howard - 1960 |

359 | B.: A logic for reasoning about time and reliability
- Hansson, Jonsson
- 1994
(Show Context)
Citation Context ... properties. We then show how the techniques introduced in Sections 4 and 5 can be used to perform model checking of these properties. 6.1 The Logic PCTL PCTL (Probabilistic Computational Tree Logic) =-=[54,18]-=- is a probabilistic extension of the temporal logic CTL [33]. PCTL is used to express properties of both DTMCs [54] and MDPs [18]. Here, we focus on MDPs. In the last part of this section, we will ext... |

358 | Probabilistic simulations for probabilistic processes - Segala, Lynch - 1995 |

338 | Modelling and verification of randomized distributed real time systems
- Segala
- 1995
(Show Context)
Citation Context ...ter 10 of [11] covers some of the MDP model checking techniques presented here, but in greater depth, with additional focus on the underlying theory and proofs. We also recommend the theses by Segala =-=[80]-=-, de Alfaro [1] and Baier [7], which provide a wealth of in-depth material on a variety of topics relating to MDPs, along with detailed pointers to other relevant literature. Finally, although not foc... |

311 | P.: Reasoning about infinite computations
- Vardi, Wolper
- 1994
(Show Context)
Citation Context ...ccepts precisely the words satisfying ψ, i.e. for any infinite word w: w |= ψ ⇐⇒ w ∈ L(Aψ) The construction of the DRA Aψ from the formula ψ is beyond the scope of this tutorial; for details see e.g. =-=[85,36,11]-=-. In fact, the set of properties that can be captured by a DRA is a strict superset of those expressible as LTL formulas, known as ω-regular properties. The same class of properties can also be captur... |

310 |
Automatic verification of probabilistic concurrent finite-state programs
- Vardi
- 1985
(Show Context)
Citation Context ...dels, which distinguish between states that offer a probabilistic choice and those that offer a nondeterministic choice. Examples include the model of Hansson [53] and the concurrent Markov chains of =-=[34,84]-=-. We do not attempt a complete survey of MDP-like models here. See [48] for a classification scheme of such models, [82] for a thorough comparison and [80,1,7] for further references and discussion. 4... |

284 | Model checking of probabilistic and nondeterministic systems
- Bianco, Alfaro
- 1995
(Show Context)
Citation Context ...r than numerical, computation. Algorithms 1–4 give a formal description of how to compute the above sets; examples of executing these algorithms are presented in Section 4.2. For further details, see =-=[18,1]-=-.12 Forejt, Kwiatkowska, Norman, Parker Input: MDP M = (S, s, αM, δM, L), target set T ⊆ S Output: the set S 0 min = {s ∈ S | Pr min s (reach(T ))=0} 1 R := T ; 2 do 3 R ′ := R; 4 R := R ′ ∪ { s ∈ S ... |

278 |
Modeling and Analysis of Stochastic Systems
- Kulkarni
- 1995
(Show Context)
Citation Context ...ns (DTMCs). We provide just a brief overview of DTMCs, as required for the remainder of this tutorial. For more in-depth coverage of the topic, we recommend the textbooks by Stewart [83] and Kulkarni =-=[65]-=-. For a tutorial on probabilistic model checking for Markov chains, see for example [67]. Definition 3 (Discrete-time Markov chain). A discrete-time Markov chain (DTMC) is a tuple D = (S, s, P, L) whe... |

273 | 2006): PRISM: A tool for automatic verification of probabilistic systems
- Hinton, Kwiatkowska, et al.
(Show Context)
Citation Context ...checking for discrete- and continuous-time Markov chains, rather than Markov decision processes. There is also an accompanying website [91], providing models for the PRISM probabilistic model checker =-=[56]-=- that correspond to the various running examples used throughout and to the case studies in Section 10. There are many other good references relating to the material covered in this tutorial and we pr... |

220 |
Probabilistic automata
- ˇSíma, Rabin, et al.
- 1963
(Show Context)
Citation Context ...geably, for example, the PRISM model checker [56] supports both PAs and MDPs, but refers to them simply as MDPs. Confusingly, there is an alternative model called probabilistic automata, due to Rabin =-=[78]-=-, which is also well known. From a syntactic point of view, these are essentially the same as MDPs, but are typically used in a language-theoretic setting, rather than for modelling and verification. ... |

192 | Reactive, generative, and stratified models of probabilistic processes
- Glabbeek, Smolka, et al.
- 1990
(Show Context)
Citation Context ...nd those that offer a nondeterministic choice. Examples include the model of Hansson [53] and the concurrent Markov chains of [34,84]. We do not attempt a complete survey of MDP-like models here. See =-=[48]-=- for a classification scheme of such models, [82] for a thorough comparison and [80,1,7] for further references and discussion. 4 Probabilistic Reachability In the remainder of this tutorial, we will ... |

186 |
Dynamic Programming and Optimal
- Bertsekas
- 2007
(Show Context)
Citation Context ...otherwise. When solving these equations, we in fact find the probability Pr min s (reach(T )) for all states s of the MDP, rather than just a specific state of interest. From the results presented in =-=[17,16]-=- (since the problem of finding minimum reachability probabilities is a special case of the stochastic shortest path problem), the equations above have a unique solution. Furthermore, it is actually su... |

169 |
Formal Verification of Probabilistic Systems
- Alfaro
- 1997
(Show Context)
Citation Context ...overs some of the MDP model checking techniques presented here, but in greater depth, with additional focus on the underlying theory and proofs. We also recommend the theses by Segala [80], de Alfaro =-=[1]-=- and Baier [7], which provide a wealth of in-depth material on a variety of topics relating to MDPs, along with detailed pointers to other relevant literature. Finally, although not focusing on verifi... |

165 |
Time and Probability in Formal Design of Distributed Systems, volume1ofReal-Time Safety Critical Systems
- Hansson
- 1994
(Show Context)
Citation Context ...other approach is to use “alternating” models, which distinguish between states that offer a probabilistic choice and those that offer a nondeterministic choice. Examples include the model of Hansson =-=[53]-=- and the concurrent Markov chains of [34,84]. We do not attempt a complete survey of MDP-like models here. See [48] for a classification scheme of such models, [82] for a thorough comparison and [80,1... |

134 | Fast randomized consensus using shared memory
- Aspnes, Herlihy
- 1990
(Show Context)
Citation Context ...n some initial value supplied by their environment, eventually terminate agreeing on the same value. In this case study, we consider the randomised distributed consensus algorithm of Aspnes & Herlihy =-=[5]-=- and use it to demonstrate the applicability of the compositional verification approach for safety properties introduced in Section 9. The algorithm of Apnes & Herlihy allows N processes in a distribu... |

131 | Model checking for a probabilistic branching time logic with fairness - Baier, Kwiatkowska - 1998 |

128 | An analysis of stochastic shortest path problems
- BERTSEKAS, TSITSIKLIS
- 1991
(Show Context)
Citation Context ...otherwise. When solving these equations, we in fact find the probability Pr min s (reach(T )) for all states s of the MDP, rather than just a specific state of interest. From the results presented in =-=[17,16]-=- (since the problem of finding minimum reachability probabilities is a special case of the stochastic shortest path problem), the equations above have a unique solution. Furthermore, it is actually su... |

124 | Policy optimization for dynamic power management
- Benini, Bogliolo, et al.
- 1999
(Show Context)
Citation Context ... based on observations of the system and a DPM policy. The particular system we consider here is the IBM TravelStar VP [58], a commercially available hard disk drive. Our model is based on the one in =-=[15]-=-. The hard disk, i.e. the SP, can operate in five different states as shown in Table 2(a), which also provides the power dissipation in each of these states. It is only in state active that the device... |

105 |
Token management schemes and random walks yield selfstabilizing mutual exclusion
- Israeli, Jalfon
- 1990
(Show Context)
Citation Context ...nsforms a system from an unstable state to a stable state in a finite number of steps and without any outside intervention. Here, we consider Israeli and Jalfon’s randomised self-stabilising protocol =-=[59]-=-. The protocol of Israeli and Jalfon is designed for a network which is an oriented ring of identical processes P1, . . . , PN with bidirectional communication. It operates asynchronously with an arbi... |

103 | Stochastic model checking
- Kwiatkowska, Norman, et al.
- 2007
(Show Context)
Citation Context ...s probabilistic model checking algorithms and their implementation, and gives an illustration of the application of these techniques to some large case studies. The tutorial is intended to complement =-=[67]-=-, which focuses on probabilistic model checking for discrete- and continuous-time Markov chains, rather than Markov decision processes. There is also an accompanying website [91], providing models for... |

101 | It usually works: The temporal logic of stochastic systems
- Aziz, Singhal, et al.
(Show Context)
Citation Context ...s2 = xℓ−1 s2 x ℓ s3 = max{xℓ−1 s3 , 5 + xℓ−1 s0 }Automated Verification Techniques for Probabilistic Systems 23 for 1�ℓ�4. The following are the values x i = (x i s0 , xi s1 , xi s2 , xi s3 ): x 1 = =-=(2, 6, 0, 5)-=- x 2 = (8, 8.5, 0, 7) x 3 = (10.5, 9.5, 0, 13) x 4 = (11.5, 12.5, 0, 15.5) So, e.g., Emax s0 (C �4 r ) = 11.5. The computed optimal adversary σmax is in fact memoryless and satisfies σmax (s1) = [risk... |

91 |
Katoen JP. Principles of Model Checking
- Baier
- 2008
(Show Context)
Citation Context ...roughout and to the case studies in Section 10. There are many other good references relating to the material covered in this tutorial and we provide pointers throughout. In particular, Chapter 10 of =-=[11]-=- covers some of the MDP model checking techniques presented here, but in greater depth, with additional focus on the underlying theory and proofs. We also recommend the theses by Segala [80], de Alfar... |

85 |
Markov decision processes and regular events”,IEEETransactionsonAutomaticControl
- COURCOUBETIS, YANNAKAKIS
- 1998
(Show Context)
Citation Context ...ing the minimum and maximum reachability probabilities is to construct and solve a linear programming (LP) problem. In the case of minimum probabilities Pr min s (reach(T )), it has been demonstrated =-=[17,35,1]-=- that the following linear program: maximise ∑ xs = 1 xs = 0 s∈S xs � ∑ s ′ ∈S δM(s, a)(s′ ) · xs ′ xs subject to the constraints: for all s ∈ S 1 min for all s ∈ S 0 min for all s ̸∈ S1 min ∪ S 0 min... |

78 |
Improved automata generation for linear temporal logic
- Daniele, Giunchiglia, et al.
(Show Context)
Citation Context ...ccepts precisely the words satisfying ψ, i.e. for any infinite word w: w |= ψ ⇐⇒ w ∈ L(Aψ) The construction of the DRA Aψ from the formula ψ is beyond the scope of this tutorial; for details see e.g. =-=[85,36,11]-=-. In fact, the set of properties that can be captured by a DRA is a strict superset of those expressible as LTL formulas, known as ω-regular properties. The same class of properties can also be captur... |

72 | I.: The ins and outs of the probabilistic model checker MRMC
- Katoen, Hahn, et al.
- 2009
(Show Context)
Citation Context ...d PASS [50] both provide verification of MDPs using abstraction and refinement. There are also various other probabilistic model checkers for discrete- and continuous-time Markov chains, notably MRMC =-=[61]-=-. For a more extensive list, see [92]. In the following sections, we present three large probabilistic model checking case studies, based on the use of Markov decision processes. A selection of furthe... |

67 | On algorithmic verification methods for probabilistic systems
- Baier
- 1998
(Show Context)
Citation Context ...the MDP model checking techniques presented here, but in greater depth, with additional focus on the underlying theory and proofs. We also recommend the theses by Segala [80], de Alfaro [1] and Baier =-=[7]-=-, which provide a wealth of in-depth material on a variety of topics relating to MDPs, along with detailed pointers to other relevant literature. Finally, although not focusing on verification, [76] i... |

57 |
Liquor: A tool for qualitative and quantitative linear time analysis of reactive systems
- Ciesinski, Baier
- 2006
(Show Context)
Citation Context ...hniques described in this tutorial. It also supports discrete- and continuous-time Markov chains, and probabilistic timed automata. Two other tools for probabilistic model checking of MDPs are LiQuor =-=[30]-=-, which has an expressive modelling language extending Promela with probabilities, and ProbDiVinE [13], which focuses on parallel and distributed implementations of LTL model checking for MDPs. RAPTUR... |

51 | Efficient computation of time-bounded reachability probabilities in uniform continuous-time Markov decision processes, Theoretical Computer Science 345
- Baier, Hermanns, et al.
(Show Context)
Citation Context ... (see e.g. [70] for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. =-=[10,23,73,77]-=-); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the deve... |

51 | Recursive Markov decision processes and recursive stochastic games
- Etessami, Yannakakis
- 2005
(Show Context)
Citation Context ...abilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. =-=[41,22]-=-). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frameworks [37,55,26,63], some of which have been appl... |

44 | Symmetry reduction for probabilistic model checking
- Kwiatkowska, Norman, et al.
- 2009
(Show Context)
Citation Context ...actice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction =-=[66,39]-=-, algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on ... |

41 |
K.G.: Reduction and Refinement Strategies for Probabilistic Analysis
- D’Argenio, Jeannet, et al.
- 2002
(Show Context)
Citation Context ... recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frameworks =-=[37,55,26,63]-=-, some of which have been applied in practice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order... |

41 | Assume-guarantee verification for probabilistic systems
- Kwiatkowska, Norman, et al.
- 2010
(Show Context)
Citation Context ... . Consequently: (i) the probability of events in M is preserved in M⊗Aerr P ; and (ii) each path of M⊗Aerr P that corresponds to a path of M that violates ΦP contains a state in S×F . Proposition 1 (=-=[69]-=-). If M=(S, s, αM, δM, LM) is an MDP, s ∈ S, ΦP is a safety property such that αP ⊆ αM, and Aerr P is an error automaton for ΦP with accepting states F , then: Pr min M,s(ΦP ) = 1 − Pr max M⊗A err P ,... |

40 | Multi-objective model checking of Markov decision processes
- Etessami, Kwiatkowska, et al.
(Show Context)
Citation Context ...he algorithm runs in time polynomial in |M| and doubly exponential in |ψ|. 8 Multi-objective Probabilistic Model Checking In this section, we consider multi-objective verification techniques for MDPs =-=[40]-=-. These permit the analysis of trade-offs between several linear-time objectives, for example “the probability of reaching a good state is at least 0.98 and, with probability at most 0.3, it will be r... |

35 | On probabilistic computation tree logic
- Ciesinski, Größer
- 2004
(Show Context)
Citation Context ...imilar. In this presentation, we opt for action-based properties since these are required for the compositional probabilistic model checking techniques that we discuss in Section 9. See, for example, =-=[35,11,32]-=- for details of the state-based approach. 7.1 Probabilistic Safety Properties To define probabilistic safety properties, we first recall the definitions of deterministic finite automata and regular sa... |

35 | L.: Probabilistic CEGAR
- Hermanns, Wachter, et al.
- 2008
(Show Context)
Citation Context ... recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frameworks =-=[37,55,26,63]-=-, some of which have been applied in practice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order... |

34 | Probabilistic automata: system types, parallel composition and comparison
- Sokolova, Vink
- 2004
(Show Context)
Citation Context ...amples include the model of Hansson [53] and the concurrent Markov chains of [34,84]. We do not attempt a complete survey of MDP-like models here. See [48] for a classification scheme of such models, =-=[82]-=- for a thorough comparison and [80,1,7] for further references and discussion. 4 Probabilistic Reachability In the remainder of this tutorial, we will introduce a variety of properties of MDPs and des... |

32 | Decision algorithms for probabilistic bisimulations
- Segala, Lynch
- 2002
(Show Context)
Citation Context ...robabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relations =-=[25,86]-=- and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic countere... |

32 |
Run-time efficient probabilistic model checking
- Filieri, Ghezzi, et al.
- 2011
(Show Context)
Citation Context ...ication under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model checking =-=[24,44]-=-. Acknowledgments The authors are supported in part by EU FP7 project CONNECT, ERC Advanced Grant VERIWARE and EPSRC grant EP/F001096/1. Vojtěch Forejt is also supported by a Royal Society Newton Fell... |

31 | Controller synthesis for probabilistic systems
- Baier, Größer, et al.
- 2004
(Show Context)
Citation Context ...ry σ by stipulating Adv={σ} in the semantics presented in Definition 13. The problem is to determine whether there exists a σ under which the given formula is true. This problem has also been studied =-=[9,21,20]-=-, and—perhaps surprisingly— it is fundamentally different from the problem in which Adv is the set of all adversaries. In particular, answering the question whether there is a satisfying adversary is ... |

28 | Rapture: A tool for verifying Markov decision processes
- Jeannet, dArgenio, et al.
(Show Context)
Citation Context ...which has an expressive modelling language extending Promela with probabilities, and ProbDiVinE [13], which focuses on parallel and distributed implementations of LTL model checking for MDPs. RAPTURE =-=[60]-=- and PASS [50] both provide verification of MDPs using abstraction and refinement. There are also various other probabilistic model checkers for discrete- and continuous-time Markov chains, notably MR... |

28 | Using probabilistic model checking for dynamic power management
- Norman, Parker, et al.
- 2003
(Show Context)
Citation Context ...2sec 6sec idlelp 5ms - - 2.2sec 6sec stby 2.2sec - - - 6sec sleep 6sec - - - - Table 2. Dynamic power management: properties of the states of the hard drive For further details of the PRISM model see =-=[74,91]-=-. In Figure 19 we present model checking results for computation of the minimum and maximum values for the following properties, when the maximum queue size is 2 and there is no constraint on the batt... |

26 | R.: Automated verification of a randomized distributed consensus protocol using Cadence
- Kwiatkowska, Norman, et al.
- 2001
(Show Context)
Citation Context ...ety property, by computing an upper bound on the maximum probability of performing a bad prefix (see Proposition 1). The probabilistic automata model of the algorithm is based on the one presented in =-=[71]-=-. It comprises the parallel composition of: N PAs, each representing one process, and R PAs, one for the shared coin protocol of each round. The compositional verification consists of the following st... |

25 | Significant diagnostic counterexamples in probabilistic model checking
- Andrés, D’Argenio, et al.
- 2009
(Show Context)
Citation Context ... probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation =-=[4,3]-=-, verification under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model ch... |

24 | Quantitative multiobjective verification for probabilistic systems
- Forejt, Kwiatkowska, et al.
- 2011
(Show Context)
Citation Context ... t2 q0 q2 b2 b2 1 a{2} b1 1 1 0.5 s{2} Fig. 15. The MDP M ′′ , built from M ′ = M⊗A1⊗A2, for Example 17 lems in this chapter, we can consider quantitative approaches to multi-objective model checking =-=[69,45]-=-. We can define numerical queries, which may be more useful than existential ones in practice. These optimise one objective, subject to constraints on several others. Formally we have the following de... |

24 | Probabilistic reachability for parametric markov models
- Hahn, Hermanns, et al.
- 2009
(Show Context)
Citation Context ...n MDPs and related models. These include: probabilistic counterexample generation [4,3], verification under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking =-=[51]-=-, synthesis of parameters [52] and models [28], and run-time probabilistic model checking [24,44]. Acknowledgments The authors are supported in part by EU FP7 project CONNECT, ERC Advanced Grant VERIW... |

22 | Measuring and synthesizing systems in probabilistic environments
- Chatterjee, Henzinger, et al.
- 2010
(Show Context)
Citation Context ...abilistic counterexample generation [4,3], verification under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters [52] and models =-=[28]-=-, and run-time probabilistic model checking [24,44]. Acknowledgments The authors are supported in part by EU FP7 project CONNECT, ERC Advanced Grant VERIWARE and EPSRC grant EP/F001096/1. Vojtěch Fore... |

22 | Reconciling nondeterministic and probabilistic choices
- Cheung
- 2006
(Show Context)
Citation Context ...eresting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation [4,3], verification under fairness [8] and under restricted classes of adversaries =-=[47,29]-=-, parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model checking [24,44]. Acknowledgments The authors are supported in part by EU FP7 project C... |

22 | D.: Abstraction refinement for probabilistic software
- Kattenbelt, Kwiatkowska, et al.
- 2009
(Show Context)
Citation Context ...ple is the development of abstraction and refinement frameworks [37,55,26,63], some of which have been applied in practice to verification of probabilistic timed automata [68], probabilistic software =-=[62]-=- and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relations [25,86] and compositional ... |

21 | D.: A game-based abstraction-refinement framework for Markov decision processes
- Kattenbelt, Kwiatkowska, et al.
- 2010
(Show Context)
Citation Context ... recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frameworks =-=[37,55,26,63]-=-, some of which have been applied in practice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order... |

19 | D.: Stochastic games for verification of probabilistic timed automata
- Kwiatkowska, Norman, et al.
- 2009
(Show Context)
Citation Context ...improve scalability. One example is the development of abstraction and refinement frameworks [37,55,26,63], some of which have been applied in practice to verification of probabilistic timed automata =-=[68]-=-, probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relatio... |

19 |
M.R.: Model checking interactive markov chains
- Zhang, Neuhäußer
- 2010
(Show Context)
Citation Context ...ues for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]); interactive Markov chains (see e.g. =-=[87]-=-); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frame... |

18 | Compositional Verification of Probabilistic Systems Using Learning
- Feng, Kwiatkowska, et al.
- 2010
(Show Context)
Citation Context ...g directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques =-=[69,43,38]-=-. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation [4,3], verification under fairness [8] and under ... |

18 |
Safety verification for probabilistic hybrid systems. European Journal of Control, 18(6):572–587, 2012. Index abstraction, 21 complete, 92, 105 simulation, 110 sound, 92, 105 verification by, 94 barrier certificate, 37 continuous (dynamical) system, 28 co
- Zhang, She, et al.
(Show Context)
Citation Context ... for the following extensions of MDPs: probabilistic timed automata (see e.g. [70] for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. =-=[88,46]-=-); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approa... |

17 | Counterexamples for model checking of Markov decision processes
- Aljazzar, Leue
- 2007
(Show Context)
Citation Context ... probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation =-=[4,3]-=-, verification under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model ch... |

16 | Dynamic QoS management and optimisation in service-based systems
- Calinescu, Grunske, et al.
- 2011
(Show Context)
Citation Context ...ication under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model checking =-=[24,44]-=-. Acknowledgments The authors are supported in part by EU FP7 project CONNECT, ERC Advanced Grant VERIWARE and EPSRC grant EP/F001096/1. Vojtěch Forejt is also supported by a Royal Society Newton Fell... |

15 |
Verifying temporal properties of finite state probabilistic programs
- Courcoubetis, Yannakakis
- 1988
(Show Context)
Citation Context ...dels, which distinguish between states that offer a probabilistic choice and those that offer a nondeterministic choice. Examples include the model of Hansson [53] and the concurrent Markov chains of =-=[34,84]-=-. We do not attempt a complete survey of MDP-like models here. See [48] for a classification scheme of such models, [82] for a thorough comparison and [80,1,7] for further references and discussion. 4... |

15 |
Approximate parameter synthesis for probabilistic time-bounded reachability
- Han, Katoen, et al.
- 2008
(Show Context)
Citation Context ...se include: probabilistic counterexample generation [4,3], verification under fairness [8] and under restricted classes of adversaries [47,29], parametric model checking [51], synthesis of parameters =-=[52]-=- and models [28], and run-time probabilistic model checking [24,44]. Acknowledgments The authors are supported in part by EU FP7 project CONNECT, ERC Advanced Grant VERIWARE and EPSRC grant EP/F001096... |

14 | From Fairness to Chance
- Alfaro
- 1999
(Show Context)
Citation Context ...mit Adv from the subscript. An alternative would be to consider the class of fair adversaries. We do not discuss the issue of fairness when model checking MDPs in this tutorial; for details, see e.g. =-=[2,8,12]-=-. The “Next” Operator. If ψ = X φ, then it follows that: Pr min ∑ s (X φ) = mina∈A(s) s ′ ∈Sat(φ) δM(s, a)(s ′ ) Pr max ∑ s (X φ) = maxa∈A(s) s ′ ∈Sat(φ) δM(s, a)(s ′ ) cAutomated Verification Techni... |

14 |
Stochastic games with branching-time winning objectives
- Brázdil, Brožek, et al.
- 2006
(Show Context)
Citation Context ...ry σ by stipulating Adv={σ} in the semantics presented in Definition 13. The problem is to determine whether there exists a σ under which the given formula is true. This problem has also been studied =-=[9,21,20]-=-, and—perhaps surprisingly— it is fundamentally different from the problem in which Adv is the set of all adversaries. In particular, answering the question whether there is a satisfying adversary is ... |

14 |
Continuous-time stochastic games with time-bounded reachability
- Brázdil, Forejt, et al.
(Show Context)
Citation Context ... (see e.g. [70] for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. =-=[10,23,73,77]-=-); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the deve... |

13 |
Reduction techniques for model checking Markov decision processes
- Ciesinski, Baier, et al.
- 2008
(Show Context)
Citation Context ...hich have been applied in practice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction =-=[49,31]-=-, symmetry reduction [66,39], algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting ... |

13 | Symmetry reduction for probabilistic model checking using generic representatives
- Donaldson, Miller
- 2006
(Show Context)
Citation Context ...actice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction =-=[66,39]-=-, algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on ... |

13 |
Engineering constraint solvers for automatic analysis of probabilistic hybrid automata
- FRÄNZLE, TEIGE, et al.
- 2010
(Show Context)
Citation Context ... for the following extensions of MDPs: probabilistic timed automata (see e.g. [70] for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. =-=[88,46]-=-); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approa... |

13 | On automated verification of probabilistic programs
- Legay, Murawski, et al.
- 2008
(Show Context)
Citation Context ...well known. From a syntactic point of view, these are essentially the same as MDPs, but are typically used in a language-theoretic setting, rather than for modelling and verification. An exception is =-=[72]-=-, which uses Rabin’s probabilistic automata to build a game-theoretic framework for verifying probabilistic programs. Another approach is to use “alternating” models, which distinguish between states ... |

12 |
Quantitative analysis under fairness constraints
- Baier, Größer, et al.
- 2009
(Show Context)
Citation Context ...mit Adv from the subscript. An alternative would be to consider the class of fair adversaries. We do not discuss the issue of fairness when model checking MDPs in this tutorial; for details, see e.g. =-=[2,8,12]-=-. The “Next” Operator. If ψ = X φ, then it follows that: Pr min ∑ s (X φ) = mina∈A(s) s ′ ∈Sat(φ) δM(s, a)(s ′ ) Pr max ∑ s (X φ) = maxa∈A(s) s ′ ∈Sat(φ) δM(s, a)(s ′ ) cAutomated Verification Techni... |

12 | ProbDiVinE-MC: Multi-core LTL model checker for probabilistic systems
- Barnat, Brim, et al.
- 2008
(Show Context)
Citation Context ...probabilistic timed automata. Two other tools for probabilistic model checking of MDPs are LiQuor [30], which has an expressive modelling language extending Promela with probabilities, and ProbDiVinE =-=[13]-=-, which focuses on parallel and distributed implementations of LTL model checking for MDPs. RAPTURE [60] and PASS [50] both provide verification of MDPs using abstraction and refinement. There are als... |

9 | Qualitative reachability in stochastic BPA games
- Brázdil, Brozek, et al.
(Show Context)
Citation Context ...abilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. =-=[41,22]-=-). Scalability. A variety of approaches are being considered to improve scalability. One example is the development of abstraction and refinement frameworks [37,55,26,63], some of which have been appl... |

9 |
A.: Probabilistic contracts: A compositional reasoning methodology for the design of stochastic systems
- Delahaye, Caillaud, et al.
- 1998
(Show Context)
Citation Context ...g directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques =-=[69,43,38]-=-. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation [4,3], verification under fairness [8] and under ... |

9 |
Pass: Abstraction refinement for infinite probabilistic models
- Hahn, Hermanns, et al.
- 2010
(Show Context)
Citation Context ...xpressive modelling language extending Promela with probabilities, and ProbDiVinE [13], which focuses on parallel and distributed implementations of LTL model checking for MDPs. RAPTURE [60] and PASS =-=[50]-=- both provide verification of MDPs using abstraction and refinement. There are also various other probabilistic model checkers for discrete- and continuous-time Markov chains, notably MRMC [61]. For a... |

9 |
Modeling and Verification of Real-Time Systems
- Kwiatkowska, Norman, et al.
- 2008
(Show Context)
Citation Context ...cability of probabilistic verification are being improved. Models. Recent advances have been made regarding model checking for the following extensions of MDPs: probabilistic timed automata (see e.g. =-=[70]-=- for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. [10,23,73,77]);... |

7 |
On the automatic verification of distributed probabilistic automata with partial information
- Giro
- 2010
(Show Context)
Citation Context ...eresting topics are being studied on MDPs and related models. These include: probabilistic counterexample generation [4,3], verification under fairness [8] and under restricted classes of adversaries =-=[47,29]-=-, parametric model checking [51], synthesis of parameters [52] and models [28], and run-time probabilistic model checking [24,44]. Acknowledgments The authors are supported in part by EU FP7 project C... |

6 |
A.: Controller synthesis and verification for Markov decision processes with qualitative branching time objectives
- Brázdil, Forejt, et al.
- 2008
(Show Context)
Citation Context ...ry σ by stipulating Adv={σ} in the semantics presented in Definition 13. The problem is to determine whether there exists a σ under which the given formula is true. This problem has also been studied =-=[9,21,20]-=-, and—perhaps surprisingly— it is fundamentally different from the problem in which Adv is the set of all adversaries. In particular, answering the question whether there is a satisfying adversary is ... |

6 |
C.: Partial order reduction for Markov decision processes: a survey
- Groesser, Baier
- 2006
(Show Context)
Citation Context ...hich have been applied in practice to verification of probabilistic timed automata [68], probabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction =-=[49,31]-=-, symmetry reduction [66,39], algorithms for simulation and bisimulation relations [25,86] and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting ... |

5 |
Time-bounded reachability probabilities in continuous-time Markov decision processes
- Neuhäußer, Zhang
- 2010
(Show Context)
Citation Context ... (see e.g. [70] for aAutomated Verification Techniques for Probabilistic Systems 55 survey); probabilistic hybrid systems (see e.g. [88,46]); continuous-time MDPs and continuous-time games (see e.g. =-=[10,23,73,77]-=-); interactive Markov chains (see e.g. [87]); and recursive MDPs and games (see e.g. [41,22]). Scalability. A variety of approaches are being considered to improve scalability. One example is the deve... |

5 | Deciding simulations on probabilistic automata
- Zhang, Hermanns
- 2007
(Show Context)
Citation Context ...robabilistic software [62] and PRISM models [50]. Other promising directions include: partial order reduction [49,31], symmetry reduction [66,39], algorithms for simulation and bisimulation relations =-=[25,86]-=- and compositional probabilistic verification techniques [69,43,38]. Other directions. Many other interesting topics are being studied on MDPs and related models. These include: probabilistic countere... |

4 | M.: A counterexample guided abstraction-refinement framework for markov decision processes
- Chadha, Viswanathan
- 2010
(Show Context)
Citation Context |

4 | T.: Value iteration
- Chatterjee, Henzinger
- 2008
(Show Context)
Citation Context ... resulting values are within ε of the true answer. In theory, it is possible to make certain guarantees on the precision obtained, based on the denominators of the (rational) transition probabilities =-=[27]-=-. However, it is unclear whether these are practically applicable. An illustration of how value iteration can be implemented is given in Algorithm 5. In practice, there is no need to store all vectors... |

3 |
Optimal Time-Abstract Schedulers for CTMDPs and Markov Games
- Rabe, Schewe
- 2010
(Show Context)
Citation Context |