Results 1  10
of
12
Recursive Markov decision processes and recursive stochastic games
 In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
(Show Context)
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic contextfree grammars and multitype branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
Algorithmic verification of recursive probabilistic state machines
 In Proc. 11th TACAS
, 2005
"... Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In thi ..."
Abstract

Cited by 43 (7 self)
 Add to MetaCart
(Show Context)
Abstract. Recursive Markov Chains (RMCs) ([EY04]) are a natural abstract model of procedural probabilistic programs and related systems involving recursion and probability. They succinctly define a class of denumerable Markov chains that generalize multitype branching (stochastic) processes. In this paper, we study the problem of model checking an RMC against a given ωregular specification. Namely, given an RMC A and a Büchi automaton B, we wish to know the probability that an execution of A is accepted by B. We establish a number of strong upper bounds, as well as lower bounds, both for qualitative problems (is the probability = 1, or = 0?), and for quantitative problems (is the probability ≥ p?, or, approximate the probability to within a desired precision). Among these, we show that qualitative model checking for general RMCs can be decided in PSPACE in A  and EXPTIME in B, and when A is either a singleexit RMC or when the total number of entries and exits in A is bounded, it can be decided in polynomial time in A. We then show that quantitative model checking can also be done in PSPACE in A, and in EXPSPACE in B. When B is deterministic, all our complexities in B  come down by one exponential. For lower bounds, we show that the qualitative model checking problem, even for a fixed RMC, is already EXPTIMEcomplete. On the other hand, even for simple reachability analysis, we showed in [EY04] that our PSPACE upper bounds in A can not be improved upon without a breakthrough on a wellknown open problem in the complexity of numerical computation. 1
Sliding window abstraction for infinite Markov chains
 In Proc. CAV, volume 5643 of LNCS
, 2009
"... Abstract. We present an onthefly abstraction technique for infinitestate continuoustime Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic ..."
Abstract

Cited by 22 (8 self)
 Add to MetaCart
Abstract. We present an onthefly abstraction technique for infinitestate continuoustime Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic modeling of biological systems. We approximate the transient probability distributions at various time instances by solving a sequence of dynamically constructed abstract models, each depending on the previous one. Each abstract model is a finite Markov chain that represents the behavior of the original, infinite chain during a specific time interval. Our approach provides complete information about probability distributions, not just about individual parameters like the mean. The error of each abstraction can be computed, and the precision of the abstraction refined when desired. We implemented the algorithm and demonstrate its usefulness and efficiency on several case studies from systems biology. 1
On Automated Verification of Probabilistic Programs
, 2007
"... We introduce a simple procedural probabilistic programming language which is suitable for coding a wide variety of randomised algorithms and protocols. This language is interpreted over nite datatypes and has a decidable equivalence problem. We have implemented an automated equivalence checker, whic ..."
Abstract

Cited by 14 (7 self)
 Add to MetaCart
(Show Context)
We introduce a simple procedural probabilistic programming language which is suitable for coding a wide variety of randomised algorithms and protocols. This language is interpreted over nite datatypes and has a decidable equivalence problem. We have implemented an automated equivalence checker, which we call apex, for this language, based on game semantics. We illustrate our approach with three nontrivial case studies: (i) Herman's selfstabilisation algorithm; (ii) an analysis of the average shape of binary search trees obtained by certain sequences of random insertions and deletions; and (iii) the problem of anonymity in the Dining Cryptographers protocol. In particular, we record an exponential speedup in the latter over stateoftheart competing approaches.
Decidability results for wellstructured transition systems with auxiliary storage
 In CONCUR, vol. 4703 of LNCS
, 2007
"... Abstract. We consider the problem of verifying the safety of wellstructured transition systems (WSTS) with auxiliary storage. WSTSs with storage are automata that have (possibly) infinitely many control states along with an auxiliary store, but which have a wellquasiordering on the set of control ..."
Abstract

Cited by 6 (1 self)
 Add to MetaCart
(Show Context)
Abstract. We consider the problem of verifying the safety of wellstructured transition systems (WSTS) with auxiliary storage. WSTSs with storage are automata that have (possibly) infinitely many control states along with an auxiliary store, but which have a wellquasiordering on the set of control states. The set of reachable configurations of the automaton may themselves not be wellquasiordered because of the presence of the extra store. We consider the coverability problem for such systems, which asks if it is possible to reach a control state (with some store value) that covers some given control state. Our main result shows that if control state reachability is decidable for automata with some store and finitely many control states then the coverability problem can be decided for WSTSs (with infinitely many control states) and the same store, provided the ordering on the control states has some special property. The special property we require is defined in terms of the existence of a ranking function compatible with the transition relation. We then show that there are several classes of infinite state systems that can be viewed as WSTSs with an auxiliary storage. These observations can then be used to both reestablish old decidability results, as well as discover new ones. 1
DECISIVE MARKOV CHAINS
"... ABSTRACT. We consider qualitative and quantitative verification problems for infinitestate Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Marko ..."
Abstract

Cited by 5 (2 self)
 Add to MetaCart
ABSTRACT. We consider qualitative and quantitative verification problems for infinitestate Markov chains. We call a Markov chain decisive w.r.t. a given set of target states F if it almost certainly eventually reaches either F or a state from which F can no longer be reached. While all finite Markov chains are trivially decisive (for every set F), this also holds for many classes of infinite Markov chains. Infinite Markov chains which contain a finite attractor are decisive w.r.t. every set F. In particular, all Markov chains induced by probabilistic lossy channel systems (PLCS) contain a finite attractor and are thus decisive. Furthermore, all globally coarse Markov chains are decisive. The class of globally coarse Markov chains includes, e.g., those induced by probabilistic vector addition systems (PVASS) with upwardclosed sets F, and all Markov chains induced by probabilistic noisy Turing machines (PNTM) (a generalization of the noisy Turing machines (NTM) of Asarin and Collins). We consider both safety and liveness problems for decisive Markov chains. Safety: What is the probability that a given set of states F is eventually reached. Liveness: What is the probability that a given set of states is reached infinitely often. There are three variants of these questions. (1) The qualitative problem, i.e., deciding if the probability is one (or zero); (2) the approximate quantitative
Eager Markov chains
 In Proc. ATVA ’06, 4Ø�Int. Symp. on Automated Technology for Verification and Analysis
, 2006
"... Abstract. We consider infinitestate discrete Markov chains which are eager: the probability of avoiding a defined set of final states for more thanÒsteps is bounded by some exponentially decreasing function�(Ò). We prove that eager Markov chains include those induced by Probabilistic Lossy Channel ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
(Show Context)
Abstract. We consider infinitestate discrete Markov chains which are eager: the probability of avoiding a defined set of final states for more thanÒsteps is bounded by some exponentially decreasing function�(Ò). We prove that eager Markov chains include those induced by Probabilistic Lossy Channel Systems, Probabilistic Vector Addition Systems with States, and Noisy Turing Machines, and that the bounding function�(Ò) can be effectively constructed for them. Furthermore, we study the problem of computing the expected reward (or cost) of runs until reaching the final states, where rewards are assigned to individual runs by computable reward functions. For eager Markov chains, an effective path exploration scheme, based on forward reachability analysis, can be used to approximate the expected reward upto an arbitrarily small error. 1
A Note on the AttractorProperty of InfiniteState Markov Chains
, 2005
"... In the past 5 years, a series of verification algorithms has been proposed for infinite Markov chains that have a finite attractor, i.e., a set that will be visited infinitely often almost surely starting from any state. In this paper, we establish a sufficient criterion for the existence of an attr ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
In the past 5 years, a series of verification algorithms has been proposed for infinite Markov chains that have a finite attractor, i.e., a set that will be visited infinitely often almost surely starting from any state. In this paper, we establish a sufficient criterion for the existence of an attractor. We show that if the states of a Markov chain can be given levels (positive integers) such that the expected next level for states at some level n > 0 if less than n for some positive D, then the states at level 0 constitute an attractor for the chain. As an application, we obtain a direct proof that some probabilistic channel systems combining message losses with duplication and insertion errors have a finite attractor.
On Probabilistic Parallel Programs with Process Creation and Synchronisation ⋆
, 1012
"... Abstract. We initiate the study of probabilistic parallel programs with dynamic process creation and synchronisation. To this end, we introduce probabilistic splitjoin systems (pSJSs), a model for parallel programs, generalising both probabilistic pushdown systems (a model for sequential probabilis ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract. We initiate the study of probabilistic parallel programs with dynamic process creation and synchronisation. To this end, we introduce probabilistic splitjoin systems (pSJSs), a model for parallel programs, generalising both probabilistic pushdown systems (a model for sequential probabilistic procedural programs which is equivalent to recursive Markov chains) and stochastic branching processes (a classical mathematical model with applications in various areas such as biology, physics, and language processing). Our pSJS model allows for a possibly recursive spawning of parallel processes; the spawned processes can synchronise and return values. We study the basic performance measures of pSJSs, especially the distribution and expectation of space, work and time. Our results extend and improve previously known results on the subsumed models. We also show how to do performance analysis in practice, and present two case studies illustrating the modelling power of pSJSs. 1
On the Memory Consumption of Probabilistic Pushdown Automata
 LIPICS LEIBNIZ INTERNATIONAL PROCEEDINGS IN INFORMATICS
, 2009
"... ..."