Results 1 - 10
of
61
Recursive Markov decision processes and recursive stochastic games
- In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract
-
Cited by 52 (11 self)
- Add to MetaCart
(Show Context)
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic context-free grammars and multi-type branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
Process Algebras for Quantitative Analysis
, 2005
"... In the 1980s process algebras became widely accepted formalisms for describing and analysing concurrency. Extensions of the formalisms, incorporating some aspects of systems which had previously been abstracted, were developed for a number of different purposes. In the area of performance analysis m ..."
Abstract
-
Cited by 47 (6 self)
- Add to MetaCart
In the 1980s process algebras became widely accepted formalisms for describing and analysing concurrency. Extensions of the formalisms, incorporating some aspects of systems which had previously been abstracted, were developed for a number of different purposes. In the area of performance analysis models must quantify both timing and probability. Addressing this domain led to the formulation of stochastic process algebras. In this paper we give a brief overview of stochastic process algebras and the problems which motivated them, before focussing on their relationship with the underlying mathematical stochastic process. This is presented in the context of the PEPA formalism.
Algorithmic verification of recursive probabilistic state machines
- In Proc. 11th TACAS
, 2005
"... Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multi-type branching (stochastic) processes. In thi ..."
Abstract
-
Cited by 43 (7 self)
- Add to MetaCart
Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multi-type branching (stochastic) processes. In this paper, we study the problem of model checking an RMC against a given ω-regular specification. Namely, given an RMC A and a Büchi automaton B, we wish to know the probability that an execution of A is accepted by B. We establish a number of strong upper bounds, as well as lower bounds, both for qualitative problems (is the probability = 1, or = 0?), and for quantitative problems (is the probability ≥ p?, or, approximate the probability to within a desired precision). Among these, we show that qualitative model checking for general RMCs can be decided in PSPACE in |A | and EXPTIME in |B|, and when A is either a single-exit RMC or when the total number of entries and exits in A is bounded, it can be decided in polynomial time in |A|. We then show that quantitative model checking can also be done in PSPACE in |A|, and in EXPSPACE in |B|. When B is deterministic, all our complexities in |B | come down by one exponential. For lower bounds, we show that the qualitative model checking problem, even for a fixed RMC, is already EXPTIME-complete. On the other hand, even for simple reachability analysis, we showed in [EY04] that our PSPACE upper bounds in A can not be improved upon without a breakthrough on a well-known open problem in the complexity of numerical computation. 1
On the decidability of temporal properties of probabilistic pushdown automata
- IN PROC. OF STACS’05
, 2005
"... We consider qualitative and quantitative model-checking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative model-checking problem for ω-regular properties and pPDA is in 2-EXPSPACE and 3-EXPTIME, respectively. We also pro ..."
Abstract
-
Cited by 42 (11 self)
- Add to MetaCart
We consider qualitative and quantitative model-checking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative model-checking problem for ω-regular properties and pPDA is in 2-EXPSPACE and 3-EXPTIME, respectively. We also prove that model-checking the qualitative fragment of the logic PECTL ∗ for pPDA is in 2-EXPSPACE, and model-checking the qualitative fragment of PCTL for pPDA is in EXPSPACE. Furthermore, model-checking the qualitative fragment of PCTL is shown to be EXPTIME-hard even for stateless pPDA. Finally, we show that PCTL model-checking is undecidable for pPDA, and PCTL + model-checking is undecidable even for stateless pPDA.
Quantitative Verification: Models, Techniques and Tools
, 2007
"... Automated verification is a technique for establishing if certain properties, usually expressed in temporal logic, hold for a system model. The model can be defined using a high-level formalism or extracted directly from software using methods such as abstract interpretation. The verification procee ..."
Abstract
-
Cited by 36 (16 self)
- Add to MetaCart
(Show Context)
Automated verification is a technique for establishing if certain properties, usually expressed in temporal logic, hold for a system model. The model can be defined using a high-level formalism or extracted directly from software using methods such as abstract interpretation. The verification proceeds through exhaustive exploration of the state-transition graph of the model and is therefore more powerful than testing. Quantitative verification is an analogous technique for establishing quantitative properties of a system model, such as the probability of battery power dropping below minimum, the expected time for message delivery and the expected number of messages lost before protocol termination. Models analysed through this method are typically variants of Markov chains, annotated with costs and rewards that describe resources and their usage during execution. Properties are expressed in temporal logic extended with probabilistic and reward operators. Quantitative verification involves a combination of a traversal of the state-transition graph of the model and numerical computation. This paper gives a brief overview of current research in quantitative verification, concentrating on the potential of the method and outlining future challenges. The modelling approach is described and the usefulness of the methodology illustrated with an example of a real-world protocol standard – Bluetooth device discovery – that has been analysed using the PRISM model checker (www.prismmodelchecker.org).
Model checking discounted temporal properties
- In TACAS04, LNCS 2988
, 2004
"... Temporal logic is two-valued: formulas are interpreted as either true or false. When applied to the analysis of stochastic systems, or systems with imprecise for-mal models, temporal logic is therefore fragile: even small changes in the model can lead to opposite truth values for a specication. We p ..."
Abstract
-
Cited by 33 (8 self)
- Add to MetaCart
(Show Context)
Temporal logic is two-valued: formulas are interpreted as either true or false. When applied to the analysis of stochastic systems, or systems with imprecise for-mal models, temporal logic is therefore fragile: even small changes in the model can lead to opposite truth values for a specication. We present a generalization of the branching-time logic Ctl which achieves robustness with respect to model pertur-bations by giving a quantitative interpretation to predicates and logical operators, and by discounting the importance of events according to how late they occur. In every state, the value of a formula is a real number in the interval [0,1], where 1 corresponds to truth and 0 to falsehood. The boolean operators and and or are replaced by min and max, the path quantiers 9 and 8 determine sup and inf over all paths from a given state, and the temporal operators 3 and 2 specify sup and inf over a given path; a new operator averages all values along a path. Furthermore, all path operators are discounted by a parameter that can be chosen to give more weight to states that are closer to the beginning of the path.
Analysis and Prediction of the Long-Run Behavior of Probabilistic Sequential Programs with Recursion (Extended Abstract)
"... We introduce a family of long-run average properties of Markov chains that are useful for purposes of performance and reliability analysis, and show that these properties can effectively be checked for a subclass of infinite-state Markov chains generated by probabilistic programs with recursive proc ..."
Abstract
-
Cited by 18 (9 self)
- Add to MetaCart
We introduce a family of long-run average properties of Markov chains that are useful for purposes of performance and reliability analysis, and show that these properties can effectively be checked for a subclass of infinite-state Markov chains generated by probabilistic programs with recursive procedures. We also show how to predict these properties by analyzing finite prefixes of runs, and present an efficient prediction algorithm for the mentioned subclass of Markov chains.
Verifying Probabilistic Procedural Programs
, 2004
"... Monolithic nite-state probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri - cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmi ..."
Abstract
-
Cited by 14 (3 self)
- Add to MetaCart
Monolithic nite-state probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri - cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmic veri cation of probabilistic procedural programs ([BKS,EKM04,EY04]). Probabilistic procedural programs can more naturally be modeled by recursive Markov chains ([EY04]), or equivalently, probabilistic pushdown automata ([EKM04]). A very rich theory emerges for these models. While our recent work solves a number of veri cation problems for these models, many intriguing questions remain open.
Bounded Model Checking for GSMP Models of Stochastic Real-time Systems
- In Proc. of HSCC’06, LNCS 3927
, 2006
"... Model checking is a popular algorithmic verification technique for checking temporal requirements of mathematical models of systems. In this paper, we consider the problem of verifying bounded reachability properties of stochastic real-time systems modeled as generalized semi-Markov processes (GS ..."
Abstract
-
Cited by 14 (2 self)
- Add to MetaCart
(Show Context)
Model checking is a popular algorithmic verification technique for checking temporal requirements of mathematical models of systems. In this paper, we consider the problem of verifying bounded reachability properties of stochastic real-time systems modeled as generalized semi-Markov processes (GSMP).
Safe On-The-Fly Steady-State Detection for Time-Bounded Reachability
, 2005
"... The time-bounded reachability problem for continuous-time Markov chains (CTMCs) amounts to determine the probability to reach a (set of) goal state(s) within a given time span, such that prior to reaching the goal certain states are avoided. Efficient algorithms for time-bounded reachability are at ..."
Abstract
-
Cited by 11 (4 self)
- Add to MetaCart
The time-bounded reachability problem for continuous-time Markov chains (CTMCs) amounts to determine the probability to reach a (set of) goal state(s) within a given time span, such that prior to reaching the goal certain states are avoided. Efficient algorithms for time-bounded reachability are at the heart of probabilistic model checkers such as PRISM and ETMCC. For large time spans, on-the-fly steady-state detection is commonly applied. To ob-tain correct results (up to a given accuracy), it is essential to avoid detecting premature stationarity. This technical report gives a detailed account of criteria for steady-state detection in the setting of time-bounded reachability. This is done for forward and backward reachability algorithms. As a spin-off of this study, new results for on-the-fly steady-state detection during CTMC transient analysis are reported. Based on these results, a precise procedure for steady-state detection for time-bounded reachability is obtained. Experiments show the impact of these results in probabilistic model checking.
