Results 1  10
of
63
Recursive Markov decision processes and recursive stochastic games
 In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
(Show Context)
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic contextfree grammars and multitype branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
Process Algebras for Quantitative Analysis
, 2005
"... In the 1980s process algebras became widely accepted formalisms for describing and analysing concurrency. Extensions of the formalisms, incorporating some aspects of systems which had previously been abstracted, were developed for a number of different purposes. In the area of performance analysis m ..."
Abstract

Cited by 49 (6 self)
 Add to MetaCart
In the 1980s process algebras became widely accepted formalisms for describing and analysing concurrency. Extensions of the formalisms, incorporating some aspects of systems which had previously been abstracted, were developed for a number of different purposes. In the area of performance analysis models must quantify both timing and probability. Addressing this domain led to the formulation of stochastic process algebras. In this paper we give a brief overview of stochastic process algebras and the problems which motivated them, before focussing on their relationship with the underlying mathematical stochastic process. This is presented in the context of the PEPA formalism.
On the decidability of temporal properties of probabilistic pushdown automata
 IN PROC. OF STACS’05
, 2005
"... We consider qualitative and quantitative modelchecking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative modelchecking problem for ωregular properties and pPDA is in 2EXPSPACE and 3EXPTIME, respectively. We also pro ..."
Abstract

Cited by 43 (12 self)
 Add to MetaCart
We consider qualitative and quantitative modelchecking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative modelchecking problem for ωregular properties and pPDA is in 2EXPSPACE and 3EXPTIME, respectively. We also prove that modelchecking the qualitative fragment of the logic PECTL ∗ for pPDA is in 2EXPSPACE, and modelchecking the qualitative fragment of PCTL for pPDA is in EXPSPACE. Furthermore, modelchecking the qualitative fragment of PCTL is shown to be EXPTIMEhard even for stateless pPDA. Finally, we show that PCTL modelchecking is undecidable for pPDA, and PCTL + modelchecking is undecidable even for stateless pPDA.
Algorithmic verification of recursive probabilistic state machines
 In Proc. 11th TACAS
, 2005
"... Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In thi ..."
Abstract

Cited by 43 (7 self)
 Add to MetaCart
Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In this paper, we study the problem of model checking an RMC against a given ωregular specification. Namely, given an RMC A and a Büchi automaton B, we wish to know the probability that an execution of A is accepted by B. We establish a number of strong upper bounds, as well as lower bounds, both for qualitative problems (is the probability = 1, or = 0?), and for quantitative problems (is the probability ≥ p?, or, approximate the probability to within a desired precision). Among these, we show that qualitative model checking for general RMCs can be decided in PSPACE in A  and EXPTIME in B, and when A is either a singleexit RMC or when the total number of entries and exits in A is bounded, it can be decided in polynomial time in A. We then show that quantitative model checking can also be done in PSPACE in A, and in EXPSPACE in B. When B is deterministic, all our complexities in B  come down by one exponential. For lower bounds, we show that the qualitative model checking problem, even for a fixed RMC, is already EXPTIMEcomplete. On the other hand, even for simple reachability analysis, we showed in [EY04] that our PSPACE upper bounds in A can not be improved upon without a breakthrough on a wellknown open problem in the complexity of numerical computation. 1
Quantitative Verification: Models, Techniques and Tools
, 2007
"... Automated verification is a technique for establishing if certain properties, usually expressed in temporal logic, hold for a system model. The model can be defined using a highlevel formalism or extracted directly from software using methods such as abstract interpretation. The verification procee ..."
Abstract

Cited by 35 (15 self)
 Add to MetaCart
(Show Context)
Automated verification is a technique for establishing if certain properties, usually expressed in temporal logic, hold for a system model. The model can be defined using a highlevel formalism or extracted directly from software using methods such as abstract interpretation. The verification proceeds through exhaustive exploration of the statetransition graph of the model and is therefore more powerful than testing. Quantitative verification is an analogous technique for establishing quantitative properties of a system model, such as the probability of battery power dropping below minimum, the expected time for message delivery and the expected number of messages lost before protocol termination. Models analysed through this method are typically variants of Markov chains, annotated with costs and rewards that describe resources and their usage during execution. Properties are expressed in temporal logic extended with probabilistic and reward operators. Quantitative verification involves a combination of a traversal of the statetransition graph of the model and numerical computation. This paper gives a brief overview of current research in quantitative verification, concentrating on the potential of the method and outlining future challenges. The modelling approach is described and the usefulness of the methodology illustrated with an example of a realworld protocol standard – Bluetooth device discovery – that has been analysed using the PRISM model checker (www.prismmodelchecker.org).
Model checking discounted temporal properties
 In TACAS04, LNCS 2988
, 2004
"... Temporal logic is twovalued: formulas are interpreted as either true or false. When applied to the analysis of stochastic systems, or systems with imprecise formal models, temporal logic is therefore fragile: even small changes in the model can lead to opposite truth values for a specication. We p ..."
Abstract

Cited by 32 (8 self)
 Add to MetaCart
(Show Context)
Temporal logic is twovalued: formulas are interpreted as either true or false. When applied to the analysis of stochastic systems, or systems with imprecise formal models, temporal logic is therefore fragile: even small changes in the model can lead to opposite truth values for a specication. We present a generalization of the branchingtime logic Ctl which achieves robustness with respect to model perturbations by giving a quantitative interpretation to predicates and logical operators, and by discounting the importance of events according to how late they occur. In every state, the value of a formula is a real number in the interval [0,1], where 1 corresponds to truth and 0 to falsehood. The boolean operators and and or are replaced by min and max, the path quantiers 9 and 8 determine sup and inf over all paths from a given state, and the temporal operators 3 and 2 specify sup and inf over a given path; a new operator averages all values along a path. Furthermore, all path operators are discounted by a parameter that can be chosen to give more weight to states that are closer to the beginning of the path.
Analysis and Prediction of the LongRun Behavior of Probabilistic Sequential Programs with Recursion (Extended Abstract)
"... We introduce a family of longrun average properties of Markov chains that are useful for purposes of performance and reliability analysis, and show that these properties can effectively be checked for a subclass of infinitestate Markov chains generated by probabilistic programs with recursive proc ..."
Abstract

Cited by 18 (9 self)
 Add to MetaCart
We introduce a family of longrun average properties of Markov chains that are useful for purposes of performance and reliability analysis, and show that these properties can effectively be checked for a subclass of infinitestate Markov chains generated by probabilistic programs with recursive procedures. We also show how to predict these properties by analyzing finite prefixes of runs, and present an efficient prediction algorithm for the mentioned subclass of Markov chains.
Bounded Model Checking for GSMP Models of Stochastic Realtime Systems
 In Proc. of HSCC’06, LNCS 3927
, 2006
"... Model checking is a popular algorithmic verification technique for checking temporal requirements of mathematical models of systems. In this paper, we consider the problem of verifying bounded reachability properties of stochastic realtime systems modeled as generalized semiMarkov processes (GS ..."
Abstract

Cited by 14 (2 self)
 Add to MetaCart
(Show Context)
Model checking is a popular algorithmic verification technique for checking temporal requirements of mathematical models of systems. In this paper, we consider the problem of verifying bounded reachability properties of stochastic realtime systems modeled as generalized semiMarkov processes (GSMP).
Safe OnTheFly SteadyState Detection for TimeBounded Reachability
, 2005
"... The timebounded reachability problem for continuoustime Markov chains (CTMCs) amounts to determine the probability to reach a (set of) goal state(s) within a given time span, such that prior to reaching the goal certain states are avoided. Efficient algorithms for timebounded reachability are at ..."
Abstract

Cited by 12 (4 self)
 Add to MetaCart
The timebounded reachability problem for continuoustime Markov chains (CTMCs) amounts to determine the probability to reach a (set of) goal state(s) within a given time span, such that prior to reaching the goal certain states are avoided. Efficient algorithms for timebounded reachability are at the heart of probabilistic model checkers such as PRISM and ETMCC. For large time spans, onthefly steadystate detection is commonly applied. To obtain correct results (up to a given accuracy), it is essential to avoid detecting premature stationarity. This technical report gives a detailed account of criteria for steadystate detection in the setting of timebounded reachability. This is done for forward and backward reachability algorithms. As a spinoff of this study, new results for onthefly steadystate detection during CTMC transient analysis are reported. Based on these results, a precise procedure for steadystate detection for timebounded reachability is obtained. Experiments show the impact of these results in probabilistic model checking.
Verifying Probabilistic Procedural Programs
, 2004
"... Monolithic nitestate probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri  cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmi ..."
Abstract

Cited by 12 (3 self)
 Add to MetaCart
Monolithic nitestate probabilistic programs have been abstractly modeled by nite Markov chains, and the algorithmic veri  cation problems for them have been investigated very extensively. In this paper we survey recent work conducted by the authors together with colleagues on the algorithmic veri cation of probabilistic procedural programs ([BKS,EKM04,EY04]). Probabilistic procedural programs can more naturally be modeled by recursive Markov chains ([EY04]), or equivalently, probabilistic pushdown automata ([EKM04]). A very rich theory emerges for these models. While our recent work solves a number of veri cation problems for these models, many intriguing questions remain open.