Results 1  10
of
182
The Complexity of Decentralized Control of Markov Decision Processes
 Mathematics of Operations Research
, 2000
"... We consider decentralized control of Markov decision processes and give complexity bounds on the worstcase running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partiallyobservable case that allow for decentralized control are described. ..."
Abstract

Cited by 409 (46 self)
 Add to MetaCart
We consider decentralized control of Markov decision processes and give complexity bounds on the worstcase running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partiallyobservable case that allow for decentralized control are described. For even two agents, the finitehorizon problems corresponding to both of these models are hard for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov decision processes. In contrast to the problems involving centralized control, the problems we consider provably do not admit polynomialtime algorithms. Furthermore, assuming EXP NEXP, the problems require superexponential time to solve in the worst case.
A characterization of convex problems in decentralized control
 IEEE Transactions on Automatic Control
"... Abstract—We consider the problem of constructing optimal decentralized controllers. We formulate this problem as one of minimizing the closedloop norm of a feedback system subject to constraints on the controller structure. We define the notion of quadratic invariance of a constraint set with respe ..."
Abstract

Cited by 131 (24 self)
 Add to MetaCart
(Show Context)
Abstract—We consider the problem of constructing optimal decentralized controllers. We formulate this problem as one of minimizing the closedloop norm of a feedback system subject to constraints on the controller structure. We define the notion of quadratic invariance of a constraint set with respect to a system, and show that if the constraint set has this property, then the constrained minimumnorm problem may be solved via convex programming. We also show that quadratic invariance is necessary and sufficient for the constraint set to be preserved under feedback. These results are developed in a very general framework, and are shown to hold in both continuous and discrete time, for both stable and unstable systems, and for any norm. This notion unifies many previous results identifying specific tractable decentralized control problems, and delineates the largest known class of convex problems in decentralized control. As an example, we show that optimal stabilizing controllers may be efficiently computed in the case where distributed controllers can communicate faster than their dynamics propagate. We also show that symmetric synthesis is included in this classification, and provide a test for sparsity constraints to be quadratically invariant, and thus amenable to convex synthesis. Index Terms—Convex optimization, decentralized control, delayed control, extended linear spaces, networked control. I.
The scenario approach to robust control design
 IEEE TRANS. AUTOM. CONTROL
, 2006
"... This paper proposes a new probabilistic solution framework for robust control analysis and synthesis problems that can be expressed in the form of minimization of a linear objective subject to convex constraints parameterized by uncertainty terms. This includes the wide class of NPhard control prob ..."
Abstract

Cited by 121 (11 self)
 Add to MetaCart
This paper proposes a new probabilistic solution framework for robust control analysis and synthesis problems that can be expressed in the form of minimization of a linear objective subject to convex constraints parameterized by uncertainty terms. This includes the wide class of NPhard control problems representable by means of parameterdependent linear matrix inequalities (LMIs). It is shown in this paper that by appropriate sampling of the constraints one obtains a standard convex optimization problem (the scenario problem) whose solution is approximately feasible for the original (usually infinite) set of constraints, i.e., the measure of the set of original constraints that are violated by the scenario solution rapidly decreases to zero as the number of samples is increased. We provide an explicit and efficient bound on the number of samples required to attain apriori specified levels of probabilistic guarantee of robustness. A rich family of control problems which are in general hard to solve in a deterministically robust sense is therefore amenable to polynomialtime solution, if robustness is intended in the proposed riskadjusted sense.
Stability criteria for switched and hybrid systems
 SIAM Review
, 2007
"... The study of the stability properties of switched and hybrid systems gives rise to a number of interesting and challenging mathematical problems. The objective of this paper is to outline some of these problems, to review progress made in solving these problems in a number of diverse communities, an ..."
Abstract

Cited by 112 (8 self)
 Add to MetaCart
(Show Context)
The study of the stability properties of switched and hybrid systems gives rise to a number of interesting and challenging mathematical problems. The objective of this paper is to outline some of these problems, to review progress made in solving these problems in a number of diverse communities, and to review some problems that remain open. An important contribution of our work is to bring together material from several areas of research and to present results in a unified manner. We begin our review by relating the stability problem for switched linear systems and a class of linear differential inclusions. Closely related to the concept of stability are the notions of exponential growth rates and converse Lyapunov theorems, both of which are discussed in detail. In particular, results on common quadratic Lyapunov functions and piecewise linear Lyapunov functions are presented, as they represent constructive methods for proving stability, and also represent problems in which significant progress has been made. We also comment on the inherent difficulty of determining stability of switched systems in general which is exemplified by NPhardness and undecidability results. We then proceed by considering the stability of switched systems in which there are constraints on the switching rules, through both dwell time requirements and state dependent switching laws. Also in this case the theory of Lyapunov functions and the existence of converse theorems is reviewed. We briefly comment on the classical Lur’e problem and on the theory of stability radii, both of which contain many of the features of switched systems and are rich sources of practical results on the topic. Finally we present a list of questions and open problems which provide motivation for continued research in this area.
On the Undecidability of Probabilistic Planning and InfiniteHorizon Partially Observable Markov Decision Problems
, 1999
"... We investigate the computability of problems in probabilistic planning and partially observable infinitehorizon Markov decision processes. The undecidability of the stringexistence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in pr ..."
Abstract

Cited by 103 (0 self)
 Add to MetaCart
(Show Context)
We investigate the computability of problems in probabilistic planning and partially observable infinitehorizon Markov decision processes. The undecidability of the stringexistence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exists a plan with success probability exceeding a desirable threshold. Analogous policyexistence problems for partially observable infinitehorizon Markov decision processes under discounted and undiscounted total reward models, averagereward models, and stateavoidance models are all shown to be undecidable. The results apply to corresponding approximation problems as well. 1 Introduction We show that problems in probabilistic planning (Kushmerick, Hanks, & Weld 1995; Boutilier, Dean, & Hanks 1999) and infinitehorizon partially observable Markov decision processes (POMDPs) (L...
On the Undecidability of Probabilistic Planning and Related Stochastic Optimization Problems
 Artificial Intelligence
, 2003
"... Automated planning, the problem of how an agent achieves a goal given a repertoire of actions, is one of the foundational and most widely studied problems in the AI literature. The original formulation of the problem makes strong assumptions regarding the agent's knowledge and control over the ..."
Abstract

Cited by 74 (0 self)
 Add to MetaCart
Automated planning, the problem of how an agent achieves a goal given a repertoire of actions, is one of the foundational and most widely studied problems in the AI literature. The original formulation of the problem makes strong assumptions regarding the agent's knowledge and control over the world, namely that its information is complete and correct, and that the results of its actions are deterministic and known.
The Boundedness of All Products of a Pair of Matrices is Undecidable
, 2000
"... We show that the boundedness of the set of all products of a given pair Sigma of rational matrices is undecidable. Furthermore, we show that the joint (or generalized) spectral radius #(#) is not computable because testing whether #(#)61 is an undecidable problem. As a consequence, the robust stabil ..."
Abstract

Cited by 68 (13 self)
 Add to MetaCart
We show that the boundedness of the set of all products of a given pair Sigma of rational matrices is undecidable. Furthermore, we show that the joint (or generalized) spectral radius #(#) is not computable because testing whether #(#)61 is an undecidable problem. As a consequence, the robust stability of linear systems under timevarying perturbations is undecidable, and the same is true for the stability of a simple class of hybrid systems. We also discuss some connections with the socalled "finiteness conjecture". Our results are based on a simple reduction from the emptiness problem for probabilistic finite automata, which is known to be undecidable.
Complexity of Stability and Controllability of Elementary Hybrid Systems
, 1997
"... this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NPhard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into tw ..."
Abstract

Cited by 57 (9 self)
 Add to MetaCart
this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NPhard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into two halfspaces, and the dynamics in eachhalfspace correspond to a differentlinear system
Termination of Linear Programs
 In CAV’2004: Computer Aided Verification, volume 3114 of LNCS
, 2004
"... We show that termination of a class of linear loop programs is decidable. Linear loop programs are discretetime linear systems with a loop condition governing termination, that is, a while loop with linear assignments. We relate the termination of such a simple loop, on all initial values, to t ..."
Abstract

Cited by 55 (0 self)
 Add to MetaCart
(Show Context)
We show that termination of a class of linear loop programs is decidable. Linear loop programs are discretetime linear systems with a loop condition governing termination, that is, a while loop with linear assignments. We relate the termination of such a simple loop, on all initial values, to the eigenvectors corresponding to only the positive real eigenvalues of the matrix defining the loop assignments. This characterization of termination is reminiscent of the famous stability theorems in control theory that characterize stability in terms of eigenvalues.
Undecidable Problems of Decentralized Observation and Control
, 2001
"... We introduce a new notion of decentralized observability for discreteevent systems, which we call joint observability. We prove that checking joint observability of a regular language w.r.t. one observer is decidable, whereas for two (or more) observers the problem becomes undecidable. Based on thi ..."
Abstract

Cited by 52 (4 self)
 Add to MetaCart
(Show Context)
We introduce a new notion of decentralized observability for discreteevent systems, which we call joint observability. We prove that checking joint observability of a regular language w.r.t. one observer is decidable, whereas for two (or more) observers the problem becomes undecidable. Based on this result, we show that a related decentralized control problem is also undecidable. We finally provide an extensive study relating our work to existing work in the literature.