Results 1  10
of
43
Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations
 IN STACS
, 2005
"... We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer ..."
Abstract

Cited by 95 (13 self)
 Add to MetaCart
(Show Context)
We define Recursive Markov Chains (RMCs), a class of finitely presented denumerable Markov chains, and we study algorithms for their analysis. Informally, an RMC consists of a collection of finitestate Markov chains with the ability to invoke each other in a potentially recursive manner. RMCs offer a natural abstract model for probabilistic programs with procedures. They generalize, in a precise sense, a number of well studied stochastic models, including Stochastic ContextFree Grammars (SCFG) and MultiType Branching Processes (MTBP). We focus on algorithms for reachability and termination analysis for RMCs: what is the probability that an RMC started from a given state reaches another target state, or that it terminates? These probabilities are in general irrational, and they arise as (least) fixed point solutions to certain (monotone) systems of nonlinear equations associated with RMCs. We address both the qualitative problem of determining whether the probabilities are 0, 1 or inbetween, and
Recursive Markov decision processes and recursive stochastic games
 In Proc. of 32nd Int. Coll. on Automata, Languages, and Programming (ICALP’05
, 2005
"... Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural ..."
Abstract

Cited by 52 (11 self)
 Add to MetaCart
Abstract. We introduce Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs), and study the decidability and complexity of algorithms for their analysis and verification. These models extend Recursive Markov Chains (RMCs), introduced in [EY05a,EY05b] as a natural model for verification of probabilistic procedural programs and related systems involving both recursion and probabilistic behavior. RMCs define a class of denumerable Markov chains with a rich theory generalizing that of stochastic contextfree grammars and multitype branching processes, and they are also intimately related to probabilistic pushdown systems. RMDPs & RSSGs extend RMCs with one controller or two adversarial players, respectively. Such extensions are useful for modeling nondeterministic and concurrent behavior, as well as modeling a system’s interactions with an environment. We provide a number of upper and lower bounds for deciding, given an RMDP (or RSSG) A and probability p, whether player 1 has a strategy to force termination at a desired exit with probability at least p. We also address “qualitative ” termination questions, where p = 1, and model checking questions. 1
On the decidability of temporal properties of probabilistic pushdown automata
 IN PROC. OF STACS’05
, 2005
"... We consider qualitative and quantitative modelchecking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative modelchecking problem for ωregular properties and pPDA is in 2EXPSPACE and 3EXPTIME, respectively. We also pro ..."
Abstract

Cited by 43 (12 self)
 Add to MetaCart
We consider qualitative and quantitative modelchecking problems for probabilistic pushdown automata (pPDA) and various temporal logics. We prove that the qualitative and quantitative modelchecking problem for ωregular properties and pPDA is in 2EXPSPACE and 3EXPTIME, respectively. We also prove that modelchecking the qualitative fragment of the logic PECTL ∗ for pPDA is in 2EXPSPACE, and modelchecking the qualitative fragment of PCTL for pPDA is in EXPSPACE. Furthermore, modelchecking the qualitative fragment of PCTL is shown to be EXPTIMEhard even for stateless pPDA. Finally, we show that PCTL modelchecking is undecidable for pPDA, and PCTL + modelchecking is undecidable even for stateless pPDA.
Quantitative analysis of probabilistic pushdown automata: . . .
, 2005
"... Probabilistic pushdown automata (pPDA) have been identified as a natural model for probabilistic programs with rcursive procedure calls. Previous works considered the decidability and complexity of the modelchecking problem for pPDA and various probabilistic temporal logics. In this paper we concen ..."
Abstract

Cited by 41 (14 self)
 Add to MetaCart
Probabilistic pushdown automata (pPDA) have been identified as a natural model for probabilistic programs with rcursive procedure calls. Previous works considered the decidability and complexity of the modelchecking problem for pPDA and various probabilistic temporal logics. In this paper we concentrate on computing the expected values and variances of various random variables defined over runs of a given probabilistic pushdown automaton. In particular, we show how to compute the expected accumulated reward and the expected gain for certain classes of reward functions. Using these results, we show how to analyze various quantitative properties of pPDA that are not expressible in conventional probabilistic temporal logics.
Probabilistic CEGAR
 University of Saarland
, 2007
"... Abstract. Counterexampleguided abstraction refinement (CEGAR) has been en vogue for the automatic verification of very large systems in the past years. When trying to apply CEGAR to the verification of probabilistic systems, various foundational questions arise. This paper explores them in the cont ..."
Abstract

Cited by 36 (3 self)
 Add to MetaCart
Abstract. Counterexampleguided abstraction refinement (CEGAR) has been en vogue for the automatic verification of very large systems in the past years. When trying to apply CEGAR to the verification of probabilistic systems, various foundational questions arise. This paper explores them in the context of predicate abstraction. 1
Recursive concurrent stochastic games
 In Proc. of 33rd Int. Coll. on Automata, Languages, and Programming (ICALP’06
, 2006
"... Abstract. We study Recursive Concurrent Stochastic Games (RCSGs), extending our recent analysis of recursive simple stochastic games [16, 17] to a concurrent setting where the two players choose moves simultaneously and independently at each state. For multiexit games, our earlier work already show ..."
Abstract

Cited by 30 (4 self)
 Add to MetaCart
Abstract. We study Recursive Concurrent Stochastic Games (RCSGs), extending our recent analysis of recursive simple stochastic games [16, 17] to a concurrent setting where the two players choose moves simultaneously and independently at each state. For multiexit games, our earlier work already showed undecidability for basic questions like termination, thus we focus on the important case of singleexit RCSGs (1RCSGs). We first characterize the value of a 1RCSG termination game as the least fixed point solution of a system of nonlinear minimax functional equations, and use it to show PSPACE decidability for the quantitative termination problem. We then give a strategy improvement technique, which we use to show that player 1 (maximizer) has ǫoptimal randomized Stackless & Memoryless (rSM) strategies for all ǫ> 0, while player 2 (minimizer) has optimal rSM strategies. Thus, such games are rSMdetermined. These results mirror and generalize in a strong sense the randomized memoryless determinacy results for finite stochastic games, and extend the classic HoffmanKarp [22] strategy improvement approach from the finite to an infinite state setting. The proofs in our infinitestate setting are very different however, relying on subtle analytic properties of certain power series that arise from studying 1RCSGs. We show that our upper bounds, even for qualitative (probability 1) termination, can not be improved, even to NP, without a major breakthrough, by giving two reductions: first a Ptime reduction from the longstanding squareroot sum problem to the quantitative termination decision problem for finite concurrent stochastic games, and then a Ptime reduction from the latter problem to the qualitative termination problem for 1RCSGs. 1.
On the convergence of Newton’s method for monotone systems of polynomial equations
 In Proceedings of STOC
, 2007
"... kiefersn, luttenml, esparza o ..."
Quasibirthdeath processes, TreeLike QBDs, probabilistic 1counter automata, and pushdown systems
, 2008
"... We begin by observing that (discretetime) QuasiBirthDeath Processes (QBDs) are equivalent, in a precise sense, to (discretetime) probabilistic 1Counter Automata (p1CAs), and both TreeLike QBDs (TLQBDs) and TreeStructured QBDs (TSQBDs) are equivalent to both probabilistic Pushdown Systems ..."
Abstract

Cited by 23 (8 self)
 Add to MetaCart
(Show Context)
We begin by observing that (discretetime) QuasiBirthDeath Processes (QBDs) are equivalent, in a precise sense, to (discretetime) probabilistic 1Counter Automata (p1CAs), and both TreeLike QBDs (TLQBDs) and TreeStructured QBDs (TSQBDs) are equivalent to both probabilistic Pushdown Systems
Efficient qualitative analysis of classes of recursive markov decision processes and simple stochastic games
 In Proc. STACS’06
, 2006
"... Abstract. Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs) are natural models for recursive systems involving both probabilistic and nonprobabilistic actions. As shown recently [10], fundamental problems about such models, e.g., termination, are undecidable ..."
Abstract

Cited by 22 (9 self)
 Add to MetaCart
Abstract. Recursive Markov Decision Processes (RMDPs) and Recursive Simple Stochastic Games (RSSGs) are natural models for recursive systems involving both probabilistic and nonprobabilistic actions. As shown recently [10], fundamental problems about such models, e.g., termination, are undecidable in general, but decidable for the important class of 1exit RMDPs and RSSGs. These capture controlled and game versions of multitype Branching Processes, an important and wellstudied class of stochastic processes. In this paper we provide efficient algorithms for the qualitative termination problem for these models: does the process terminate almost surely when the players use their optimal strategies? Polynomial time algorithms are given for both maximizing and minimizing 1exit RMDPs (the two cases are not symmetric). For 1exit RSSGs the problem is in NP∩coNP, and furthermore, it is at least as hard as other wellknown NP∩coNP problems on games, e.g., Condon’s quantitative termination problem for finite SSGs ([3]). For the class of linearlyrecursive 1exit RSSGs, we show that the problem can be solved in polynomial time.
Sliding window abstraction for infinite Markov chains
 In Proc. CAV, volume 5643 of LNCS
, 2009
"... Abstract. We present an onthefly abstraction technique for infinitestate continuoustime Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic ..."
Abstract

Cited by 22 (8 self)
 Add to MetaCart
Abstract. We present an onthefly abstraction technique for infinitestate continuoustime Markov chains. We consider Markov chains that are specified by a finite set of transition classes. Such models naturally represent biochemical reactions and therefore play an important role in the stochastic modeling of biological systems. We approximate the transient probability distributions at various time instances by solving a sequence of dynamically constructed abstract models, each depending on the previous one. Each abstract model is a finite Markov chain that represents the behavior of the original, infinite chain during a specific time interval. Our approach provides complete information about probability distributions, not just about individual parameters like the mean. The error of each abstraction can be computed, and the precision of the abstraction refined when desired. We implemented the algorithm and demonstrate its usefulness and efficiency on several case studies from systems biology. 1