#### DMCA

## Game-based abstraction for Markov decision processes (2006)

### Cached

### Download Links

Citations: | 53 - 15 self |

### Citations

5092 | Probability and measure - Billingsley - 1986 |

3193 | Model checking
- Clarke, Grumberg, et al.
- 1999
(Show Context)
Citation Context ...s paper, we consider abstraction techniques, which have been established as one of the most effective ways of reducing the state-space explosion problem for non-probabilistic model checking (see e.g. =-=[CGP99]-=-). The basic idea of such methods is to construct an abstract model, typically much smaller than the original (concrete) model, in which details not relevant to a particular property of interest have ... |

1089 |
Finite Markov Chains
- Kemeny, Snell
- 1960
(Show Context)
Citation Context ...or which last(π) = last(π ′ ) we have A(π) = A(π ′ ). The behaviour under a given adversary A is purely probabilistic and we can define a probability measure Prob A s over the set of paths Path A (s) =-=[KSK66]-=-. Below, we introduce two quantitative measures for MDPs which together form the basis for probabilistic model checking of MDPs [dA97, BK98]. 2.1.1 Probabilistic Reachability The first measure is prob... |

835 | Counterexample-guided abstraction refinement
- Clarke, Grumberg, et al.
- 2000
(Show Context)
Citation Context .... For properties not satisfied in the abstract model, this is not the case, but information obtained during the verification process, such as a counterexample, maybe be used to refine the abstraction =-=[6]-=-. In the probabilistic setting, it is typically necessary to consider quantitative properties, in which case the actual probability of some behaviour being observed must be determined, e.g. “the proba... |

359 | B.: A logic for reasoning about time and reliability
- Hansson, Jonsson
- 1994
(Show Context)
Citation Context ...c transitions labelled with intervals, rather than exact values. Conservative model checking of such models is achieved through a three-valued semantics of probabilistic computation tree logic (PCTL) =-=[HJ94]-=-. Huth also proves the ‘optimality’ of the abstraction technique: for any finite set of until-freeformulae, there always exists an abstraction in which satisfaction of each formula agrees with the con... |

338 | Modelling and verification of randomized distributed real time systems
- Segala
- 1995
(Show Context)
Citation Context ...through the use of partial order techniques [CGC04, DN04]. D’Argenio et al. [DJJL01] introduce an approach for verifying quantitative reachability properties of MDPs based on probabilistic simulation =-=[Seg95]-=-. Properties are analysed on abstractions obtained through successive refinements, starting from an initial coarse partition derived from the property under study. This approach only produces a lower ... |

273 | 2006): PRISM: A tool for automatic verification of probabilistic systems
- Hinton, Kwiatkowska, et al.
(Show Context)
Citation Context ... We implemented a prototype Java implementation of the MDP abstraction process described in Section 3 and then applied it to a range of concrete models of the Zeroconf protocol constructed with PRISM =-=[18, 29]-=-. The sizes of the resulting models (number of states and transitions) can be seen in Table 1 (recall that N denotes the number of hosts with configured IP address and M denotes the number of availabl... |

206 | The complexity of stochastic games
- Condon
- 1992
(Show Context)
Citation Context ...inction between the nondeterminism from the original MDP and the nondeterminism introduced during the abstraction process. To achieve this, we model abstractions as simple stochastic two-player games =-=[Con92]-=-, where the two players correspond to the two different forms of nondeterminism. We can then analyse these models using techniques developed for such games [Con93, dAHK98, CdAH04]. Our analysis of the... |

201 | Probabilistic Symbolic Model Checking with PRISM: A Hybrid Approach
- Kwiatkowska, Norman, et al.
- 2002
(Show Context)
Citation Context ...ces the possibility of applying our technique to infinite state MDPs. We anticipate that, due to the similarity of the numerical methods (Lemma 5 and Lemma 9), the symbolic methods developed in PRISM =-=[KNP04]-=- can be extended to solving simple stochastic games constructed in the abstraction process. We also intend to look at ways of automatically or semi-automatically generating partitions based on the pre... |

194 | Bisimulation for labelled Markov processes
- Desharnais, Edalat, et al.
- 2002
(Show Context)
Citation Context ...belled with intervals and the logic PCTL has a three-valued interpretation. It is shown that model checking in this setting has the same complexity as that for standard Markov chains against PCTL. In =-=[DGJP03]-=- a method for approximating continuous state (and hence infinite state) Markov processes by a family of finite state Markov chains is presented. It is shown that, for simple quantitative modal logic, ... |

169 |
Formal Verification of Probabilistic Systems
- Alfaro
- 1997
(Show Context)
Citation Context ...ine a probability measure Prob A s over the set of paths Path A (s) [22]. Below, we introduce two quantitative measures for MDPs which together form the basis for probabilistic model checking of MDPs =-=[12, 1]-=-. Probabilistic Reachability The first measure is probabilistic reachability, namely the minimum and maximum probability of reaching, from some state s, a set F ⊆ S of target states. For a given adver... |

131 | Model checking for a probabilistic branching time logic with fairness
- Baier, Kwiatkowska
- 1998
(Show Context)
Citation Context ...l [0, ε1] and the maximum probability is in the interval [1−ε2, 1]. In this case, a single pair of bounds could at best establish that both the minimum and maximum probability lie within the interval =-=[0, 1]-=-, effectively yielding no information. Related Work. Below, we summarise work on abstraction methods for quantitative analysis of Markov decision processes and Markov chains. General issues relating t... |

128 | An analysis of stochastic shortest path problems
- BERTSEKAS, TSITSIKLIS
- 1991
(Show Context)
Citation Context ...) and e max s (F ) = sup e A∈Adv A s (F ) . Computing values for expected reachability (and probability) reduces to the stochastic shortest path problem for Markov decision processes; see for example =-=[2, 13]-=-. A key result in this respect is that optimality with respect to probabilistic and expected reachability can always be achieved with simple adversaries (see Definition 2). A consequence of this is th... |

116 |
C.: Abstraction, Refinement and Proof for Probabilistic Systems
- McIver, Morgan
- 2005
(Show Context)
Citation Context ...Markov decision processes with infinite state spaces. Finally, McIver and Morgan have developed a framework for the refinement and abstraction of probabilistic programs using expectation transformers =-=[MM04]-=-. The proof techniques developed in this work have been implemented in the HOL theorem-proving environment [HMM05]. Outline of the Paper. In the next section we present background material required fo... |

83 | On algorithms for simple stochastic games
- Condon
- 1993
(Show Context)
Citation Context ...ns as simple stochastic two-player games [8], where the two players correspond to the two different forms of nondeterminism. We can then analyse these models using techniques developed for suchgames =-=[9, 14, 4]-=-. Our analysis of these abstract models results in a separate lower and an upper bound for both the minimum and maximum probabilities (or expected reward) of reaching a set of states. This approach is... |

74 | Performance analysis of probabilistic timed automata using digital clocks
- Kwiatkowska, Norman, et al.
- 2006
(Show Context)
Citation Context ...during transmission. Hence, it is possible that the new device will end up using an IP address that is already in use. 4.1 The Model We use a slightly simplified version of the model of Zeroconf from =-=[KNPS03]-=- (see Appendix B). We model the situation where a new device joins a network of N existing hosts, in which there are a total of M IP addresses available. We assume that the communication medium betwee... |

72 | Reachability analysis of probabilistic systems by successive refinements
- D’Argenio, Jeannet, et al.
- 2001
(Show Context)
Citation Context ..., for example [SG03]. Another approach to improving the efficiency of model checking for large Markov decision processes is through the use of partial order techniques [CGC04, DN04]. D’Argenio et al. =-=[DJJL01]-=- introduce an approach for verifying quantitative reachability properties of MDPs based on probabilistic simulation [Seg95]. Properties are analysed on abstractions obtained through successive refinem... |

67 | Concurrent reachability games
- Alfaro, Henzinger, et al.
- 1998
(Show Context)
Citation Context ...ns as simple stochastic two-player games [8], where the two players correspond to the two different forms of nondeterminism. We can then analyse these models using techniques developed for suchgames =-=[9, 14, 4]-=-. Our analysis of these abstract models results in a separate lower and an upper bound for both the minimum and maximum probabilities (or expected reward) of reaching a set of states. This approach is... |

61 |
E.: Dynamic configuration of IPv4 link-local addresses (draft August 2002). Zeroconf Working Group of the Internet Engineering Task Force (www.zeroconf.org
- Cheshire, Adoba, et al.
(Show Context)
Citation Context ...ine that the minimum and maximum reachability probabilities both lay in the interval [0.7, 1]. 4 Case Study We now demonstrate the applicability of our approach to a case study: the Zeroconf protocol =-=[CAG]-=- for dynamic self-configuration of local IP addresses within a local network. Zeroconf provides a distributed, ‘plug and play’ approach to IP address configuration, managed by the individual devices o... |

51 | Computing minimum and maximum reachability times in probabilistic systems
- Alfaro
- 1999
(Show Context)
Citation Context ...) and e max s (F ) = sup e A∈Adv A s (F ) . Computing values for expected reachability (and probability) reduces to the stochastic shortest path problem for Markov decision processes; see for example =-=[2, 13]-=-. A key result in this respect is that optimality with respect to probabilistic and expected reachability can always be achieved with simple adversaries (see Definition 2). A consequence of this is th... |

34 | V.: Don’t know in probabilistic systems
- Fecher, Leucker, et al.
- 2006
(Show Context)
Citation Context ...imality’ of the abstraction technique: for any finite set of until-freeformulae, there always exists an abstraction in which satisfaction of each formula agrees with the concrete model. Fecher et al. =-=[FLW06]-=- also consider an abstraction technique for Markov chains where probabilistic transitions are labelled with intervals and the logic PCTL has a three-valued interpretation. It is shown that model check... |

30 | F.: Partial order reduction for probabilistic systems
- Baier, Grosser, et al.
- 2004
(Show Context)
Citation Context ...of nonprobabilistic model checking, for example [32]. Another approach to improving the efficiency of model checking for large Markov decision processes is through the use of partial order techniques =-=[3, 11]-=-. D’Argenio et al. [10] introduce an approach for verifying quantitative reachability properties of MDPs based on probabilistic simulation [30]. Properties are analysed on abstractions obtained throug... |

28 |
P.: Partial order reduction on concurrent probabilistic programs
- D’Argenio, Niebert
- 2004
(Show Context)
Citation Context ...of nonprobabilistic model checking, for example [32]. Another approach to improving the efficiency of model checking for large Markov decision processes is through the use of partial order techniques =-=[3, 11]-=-. D’Argenio et al. [10] introduce an approach for verifying quantitative reachability properties of MDPs based on probabilistic simulation [30]. Properties are analysed on abstractions obtained throug... |

27 | Abstract interpretation of programs as Markov decision processes
- Monniaux
- 2005
(Show Context)
Citation Context ...v chains is presented. It is shown that, for simple quantitative modal logic, if the continuous Markov process satisfies a formula, then one of the approximations also satisfies the formula. Monniaux =-=[Mon05]-=- also considers infinite state systems, demonstrating that the framework of abstract interpretation can be applied to Markov decision processes with infinite state spaces. Finally, McIver and Morgan h... |

26 | A game-based framework for CTL counterexamples and 3-valued abstraction-refinemnet, in
- Shoham, Grumberg
(Show Context)
Citation Context .... Progress has been made in the area of qualitative probabilistic verification, see for example [ZPK02], and games have also been applied in the field of non-probabilistic model checking, for example =-=[SG03]-=-. Another approach to improving the efficiency of model checking for large Markov decision processes is through the use of partial order techniques [CGC04, DN04]. D’Argenio et al. [DJJL01] introduce a... |

26 | Model-checking Markov chains in the presence of uncertainties
- Sen, Viswanathan, et al.
- 2006
(Show Context)
Citation Context ...hat model checking in this setting has the same complexity as that for standard Markov chains against PCTL. An alternative approach using Markov chains with intervals of probabilities can be found in =-=[31]-=-. In [15] a method for approximating continuous state (and hence infinite state) Markov processes by a family of finite state Markov chains is presented. It is shown that, for simple quantitative moda... |

23 |
On finite-state approximants for probabilistic computation tree logic
- Huth
- 2005
(Show Context)
Citation Context ...probability and hence appears more suited to analysing Markov chains (models with discrete probabilities and no nondeterminism) since the minimum and maximum probabilities coincide in this case. Huth =-=[Hut05]-=- considers an abstraction approach for infinite state Markov chains where the abstract models (finite state approximations) contain probabilistic transitions labelled with intervals, rather than exact... |

18 | T.: Trading memory for randomness
- Chatterjee, Alfaro, et al.
- 2004
(Show Context)
Citation Context ...ns as simple stochastic two-player games [8], where the two players correspond to the two different forms of nondeterminism. We can then analyse these models using techniques developed for suchgames =-=[9, 14, 4]-=-. Our analysis of these abstract models results in a separate lower and an upper bound for both the minimum and maximum probabilities (or expected reward) of reaching a set of states. This approach is... |

13 | Analysing randomized distributed algorithms, in Validation of Stochastic Systems - A Guide to Current Research
- Norman
- 2004
(Show Context)
Citation Context ...k on abstraction methods for quantitative analysis of Markov decision processes and Markov chains. General issues relating to abstraction in the field of probabilistic model checking are discussed in =-=[20, 28]-=-. Progress has been made in the area of qualitative probabilistic verification, see for example [33], and games have also been applied in the field of nonprobabilistic model checking, for example [32]... |

11 |
An abstraction framework for mixed nondeterministic and probabilistic systems
- Huth
- 2004
(Show Context)
Citation Context ...k on abstraction methods for quantitative analysis of Markov decision processes and Markov chains. General issues relating to abstraction in the field of probabilistic model checking are discussed in =-=[20, 28]-=-. Progress has been made in the area of qualitative probabilistic verification, see for example [33], and games have also been applied in the field of nonprobabilistic model checking, for example [32]... |

5 | Automatic verification of probabilistic free choice
- Zuck, Pnueli, et al.
- 2002
(Show Context)
Citation Context ...al issues relating to abstraction in the field of probabilistic model checking are discussed in [20, 28]. Progress has been made in the area of qualitative probabilistic verification, see for example =-=[33]-=-, and games have also been applied in the field of nonprobabilistic model checking, for example [32]. Another approach to improving the efficiency of model checking for large Markov decision processes... |

4 |
C.: Probabilistic guarded commands mechanized
- Hurd, McIver, et al.
- 2005
(Show Context)
Citation Context ...he refinement and abstraction of probabilistic programs using expectation transformers [MM04]. The proof techniques developed in this work have been implemented in the HOL theorem-proving environment =-=[HMM05]-=-. Outline of the Paper. In the next section we present background material required for the remainder of the paper. In particular, we summarise results relating to Markov decision processes and to sim... |