Results 1  10
of
11
Robust Control of Uncertain Markov Decision Processes with Temporal Logic Specifications
"... Abstract—We present a method for designing robust controllers for dynamical systems with linear temporal logic specifications. We abstract the original system by a finite Markov Decision Process (MDP) that has transition probabilities in a specified uncertainty set. A robust control policy for the ..."
Abstract

Cited by 5 (1 self)
 Add to MetaCart
(Show Context)
Abstract—We present a method for designing robust controllers for dynamical systems with linear temporal logic specifications. We abstract the original system by a finite Markov Decision Process (MDP) that has transition probabilities in a specified uncertainty set. A robust control policy for the MDP is generated that maximizes the worstcase probability of satisfying the specification over all transition probabilities in the uncertainty set. To do this, we use a procedure from probabilistic model checking to combine the system model with an automaton representing the specification. This new MDP is then transformed into an equivalent form that satisfies assumptions for stochastic shortest path dynamic programming. A robust version of dynamic programming allows us to solve for a suboptimal robust control policy with time complexity O(log1/) times that for the nonrobust case. We then implement this control policy on the original dynamical system. I.
Approximation Metrics based on Probabilistic Bisimulations for General StateSpace Markov Processes: a Survey
 HAS 2011
, 2011
"... This article provides a survey of approximation metrics for stochastic processes. We deal with Markovian processes in discrete time evolving on general state spaces, namely on domains with infinite cardinality and endowed with proper measurability and metric structures. The focus of this work is to ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
This article provides a survey of approximation metrics for stochastic processes. We deal with Markovian processes in discrete time evolving on general state spaces, namely on domains with infinite cardinality and endowed with proper measurability and metric structures. The focus of this work is to discuss approximation metrics between two such processes, based on the notion of probabilistic bisimulation: in particular we investigate metrics characterized by an approximate variant of this notion. We suggests that metrics between two processes can be introduced essentially in two distinct ways: the first employs the probabilistic conditional kernels underlying the two stochastic processes under study, and leverages notions derived from algebra, logic, or category theory; whereas the second looks at distances between trajectories of the two processes, and is based on the dynamical properties of the two processes (either their syntax, via the notion of bisimulation function; or their semantics, via sampling techniques). The survey moreover covers the problem of constructing formal approximations of stochastic processes according to the introduced metrics.
Symbolic control of stochastic systems via approximately bisimilar finite abstractions. arXiv
"... Abstract. Symbolic approaches to the control design over complex systems employ the construction of finitestate models that are related to the original control systems, then use techniques from finitestate synthesis to compute controllers satisfying specifications given in a temporal logic, and fi ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
(Show Context)
Abstract. Symbolic approaches to the control design over complex systems employ the construction of finitestate models that are related to the original control systems, then use techniques from finitestate synthesis to compute controllers satisfying specifications given in a temporal logic, and finally translate the synthesized schemes back as controllers for the concrete complex systems. Such approaches have been successfully developed and implemented for the synthesis of controllers over nonprobabilistic control systems. In this paper, we extend the technique to probabilistic control systems modeled by controlled stochastic differential equations. We show that for every stochastic control system satisfying a probabilistic variant of incremental inputtostate stability, and for every given precision ε> 0, a finitestate transition system can be constructed, which is εapproximately bisimilar (in the sense of moments) to the original stochastic control system. Moreover, we provide results relating stochastic control systems to their corresponding finitestate transition systems in terms of probabilistic bisimulation relations known in the literature. We demonstrate the effectiveness of the construction by synthesizing controllers for stochastic control systems over rich specifications expressed in linear temporal logic. The discussed technique enables a new, automated, correctbyconstruction controller synthesis approach for stochastic control systems, which are common mathematical models employed in many safety critical systems subject to structured uncertainty and are thus relevant for cyberphysical applications. 1. Introduction, Literature Background
Approximate Markovian Abstractions for Linear Stochastic Systems
"... Abstract — In this paper, we present a method to generate a finite Markovian abstraction for a discrete time linear stochastic system evolving in a full dimensional polytope. Our approach involves an adaptation of an existing approximate abstraction procedure combined with a bisimulationlike refine ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — In this paper, we present a method to generate a finite Markovian abstraction for a discrete time linear stochastic system evolving in a full dimensional polytope. Our approach involves an adaptation of an existing approximate abstraction procedure combined with a bisimulationlike refinement algorithm. It proceeds by approximating the transition probabilities from one region to another by calculating the probability from a single representative point in the first region. We derive the exact bound of the approximation error and an explicit expression for its growth over time. To achieve a desired error value, we employ an adaptive refinement algorithm that takes advantage of the dynamics of the system. We demonstrate the performance of our method through simulations. I.
Verification
"... aachen.de This paper deals with the notion of approximate probabilistic bisimulation (APB) relation for discretetime labeled Markov Chains (LMC). In order to provide a quantified upper bound on a metric over probabilistic realizations for LMC, we exploit the structure and properties of the APB an ..."
Abstract
 Add to MetaCart
(Show Context)
aachen.de This paper deals with the notion of approximate probabilistic bisimulation (APB) relation for discretetime labeled Markov Chains (LMC). In order to provide a quantified upper bound on a metric over probabilistic realizations for LMC, we exploit the structure and properties of the APB and leverage the mathematical framework of Markov setChains. Based on this bound, the article proves that the existence of an APB implies the preservation of robust PCTL formulae, which are formulae that allow being properly relaxed or strengthened, according to the underlying APB. This leads to a notion of robustness for probabilistic model checking.
SYMBOLIC MODELS FOR STOCHASTIC SWITCHED SYSTEMS: A DISCRETIZATION AND A DISCRETIZATIONFREE APPROACH
"... Abstract. Stochastic switched systems are a relevant class of stochastic hybrid systems with probabilistic evolution over a continuous domain and controldependent discrete dynamics over a finite set of modes. In the past few years several different techniques have been developed to assist in the st ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract. Stochastic switched systems are a relevant class of stochastic hybrid systems with probabilistic evolution over a continuous domain and controldependent discrete dynamics over a finite set of modes. In the past few years several different techniques have been developed to assist in the stability analysis of stochastic switched systems. However, more complex and challenging objectives related to the verification of and the controller synthesis for logic specifications have not been formally investigated for this class of systems as of yet. With logic specifications we mean properties expressed as formulae in linear temporal logic or as automata on infinite strings. This paper addresses these complex objectives by constructively deriving approximately equivalent (bisimilar) symbolic models of stochastic switched systems. More precisely, this paper provides two different symbolic abstraction techniques: one requires state space discretization, but the other one does not require any space discretization which can be potentially more efficient than the first one when dealing with higher dimensional stochastic switched systems. Both techniques provide finite symbolic models that are approximately bisimilar to stochastic switched systems under some stability assumptions on the concrete model. This allows formally synthesizing controllers (switching signals) that are valid for the concrete system over the finite symbolic model, by means of mature automatatheoretic techniques in the literature. The effectiveness of the results are illustrated by synthesizing switching signals enforcing logic specifications for two case studies including temperature control of a sixroom building. 1.
1Traffic Network Control from Temporal Logic Specifications
"... We propose a framework for generating a signal control policy for a traffic network of signalized intersections to accomplish control objectives expressible using linear temporal logic. By applying techniques from model checking and formal methods, we obtain a correctbyconstruction controller that ..."
Abstract
 Add to MetaCart
We propose a framework for generating a signal control policy for a traffic network of signalized intersections to accomplish control objectives expressible using linear temporal logic. By applying techniques from model checking and formal methods, we obtain a correctbyconstruction controller that is guaranteed to satisfy complex specifications. To apply these tools, we identify and exploit structural properties particular to traffic networks that allow for efficient computation of a finite state abstraction. In particular, traffic networks exhibit a componentwise monotonicity property which allows reach set computations that scale linearly with the dimension of the continuous state space. I.
Temporal Logic Control for Stochastic Linear Systems using Abstraction Refinement of Probabilistic Games
"... We consider the problem of computing the set of initial states of a dynamical system such that there exists a control strategy to ensure that the trajectories satisfy a temporal logic specification with probability 1 (almostsurely). We focus on discretetime, stochastic linear dynamics and specifi ..."
Abstract
 Add to MetaCart
(Show Context)
We consider the problem of computing the set of initial states of a dynamical system such that there exists a control strategy to ensure that the trajectories satisfy a temporal logic specification with probability 1 (almostsurely). We focus on discretetime, stochastic linear dynamics and specifications given as formulas of the Generalized Reactivity(1) fragment of Linear Temporal Logic over linear predicates in the states of the system. We propose a solution based on iterative abstractionrefinement, and turnbased 2player probabilistic games. While the theoretical guarantee of our algorithm after any finite number of iterations is only a partial solution, we show that if our algorithm terminates, then the result is the set of satisfying initial states. Moreover, for any (partial) solution our algorithm synthesizes witness control strategies to ensure almostsure satisfaction of the temporal logic specification. We demonstrate our approach on an illustrative case study.
$%675$&7 68%(&77(506 180%(5 2) 3$*(6
"... HIIHUVRQ'DYLV+LJKZD\6XLWH$UOLQJWRQ9$5HVSRQGHQWVVKRXOGEHDZDUHWKDWQRWZLWKVWDQGLQJDQ\RWKHUSURYLVLRQRIODZQRSHUVRQVKDOOEH VXEMHFWWRDQ\SHQDOW\IRUIDLOLQJWRFRPSO\ZLWKDFROOHFWLRQRILQIRUPDWLRQLILWGRHVQRWGLVSOD\DFXUUHQWO\YDOLG20%FRQWUROQXPEHU 3/($6('21275(7851<285)250727+($%29($''5(66 ..."
Abstract
 Add to MetaCart
(Show Context)
HIIHUVRQ'DYLV+LJKZD\6XLWH$UOLQJWRQ9$5HVSRQGHQWVVKRXOGEHDZDUHWKDWQRWZLWKVWDQGLQJDQ\RWKHUSURYLVLRQRIODZQRSHUVRQVKDOOEH VXEMHFWWRDQ\SHQDOW\IRUIDLOLQJWRFRPSO\ZLWKDFROOHFWLRQRILQIRUPDWLRQLILWGRHVQRWGLVSOD\DFXUUHQWO\YDOLG20%FRQWUROQXPEHU 3/($6('21275(7851<285)250727+($%29($''5(66 '$7(6&29(5(')URP7R E*5$17180%(5 F352*5$0(/(0(17180%(5 G352(&7180%(5 H7$6.180%(5 I:25.81,7180%(5 6321625021,725 65(3257 180%(56 6(&85,7<&/$66,),&$7,212) E7(/(3+21(180%(5,QFOXGHDUHDFRGH