Results 1 - 10
of
187
The Complexity of Decentralized Control of Markov Decision Processes
- Mathematics of Operations Research
, 2000
"... We consider decentralized control of Markov decision processes and give complexity bounds on the worst-case running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partially-observable case that allow for decentralized control are described. ..."
Abstract
-
Cited by 411 (46 self)
- Add to MetaCart
We consider decentralized control of Markov decision processes and give complexity bounds on the worst-case running time for algorithms that find optimal solutions. Generalizations of both the fullyobservable case and the partially-observable case that allow for decentralized control are described. For even two agents, the finite-horizon problems corresponding to both of these models are hard for nondeterministic exponential time. These complexity results illustrate a fundamental difference between centralized and decentralized control of Markov decision processes. In contrast to the problems involving centralized control, the problems we consider provably do not admit polynomial-time algorithms. Furthermore, assuming EXP NEXP, the problems require super-exponential time to solve in the worst case.
A characterization of convex problems in decentralized control
- IEEE Transactions on Automatic Control
"... Abstract—We consider the problem of constructing optimal decentralized controllers. We formulate this problem as one of minimizing the closed-loop norm of a feedback system subject to constraints on the controller structure. We define the notion of quadratic invariance of a constraint set with respe ..."
Abstract
-
Cited by 133 (24 self)
- Add to MetaCart
(Show Context)
Abstract—We consider the problem of constructing optimal decentralized controllers. We formulate this problem as one of minimizing the closed-loop norm of a feedback system subject to constraints on the controller structure. We define the notion of quadratic invariance of a constraint set with respect to a system, and show that if the constraint set has this property, then the constrained minimum-norm problem may be solved via convex programming. We also show that quadratic invariance is necessary and sufficient for the constraint set to be preserved under feedback. These results are developed in a very general framework, and are shown to hold in both continuous and discrete time, for both stable and unstable systems, and for any norm. This notion unifies many previous results identifying specific tractable decentralized control problems, and delineates the largest known class of convex problems in decentralized control. As an example, we show that optimal stabilizing controllers may be efficiently computed in the case where distributed controllers can commu-nicate faster than their dynamics propagate. We also show that symmetric synthesis is included in this classification, and provide a test for sparsity constraints to be quadratically invariant, and thus amenable to convex synthesis. Index Terms—Convex optimization, decentralized control, de-layed control, extended linear spaces, networked control. I.
The scenario approach to robust control design
- IEEE TRANS. AUTOM. CONTROL
, 2006
"... This paper proposes a new probabilistic solution framework for robust control analysis and synthesis problems that can be expressed in the form of minimization of a linear objective subject to convex constraints parameterized by uncertainty terms. This includes the wide class of NP-hard control prob ..."
Abstract
-
Cited by 121 (11 self)
- Add to MetaCart
This paper proposes a new probabilistic solution framework for robust control analysis and synthesis problems that can be expressed in the form of minimization of a linear objective subject to convex constraints parameterized by uncertainty terms. This includes the wide class of NP-hard control problems representable by means of parameter-dependent linear matrix inequalities (LMIs). It is shown in this paper that by appropriate sampling of the constraints one obtains a standard convex optimization problem (the scenario problem) whose solution is approximately feasible for the original (usually infinite) set of constraints, i.e., the measure of the set of original constraints that are violated by the scenario solution rapidly decreases to zero as the number of samples is increased. We provide an explicit and efficient bound on the number of samples required to attain a-priori specified levels of probabilistic guarantee of robustness. A rich family of control problems which are in general hard to solve in a deterministically robust sense is therefore amenable to polynomial-time solution, if robustness is intended in the proposed risk-adjusted sense.
Stability criteria for switched and hybrid systems
- SIAM Review
, 2007
"... The study of the stability properties of switched and hybrid systems gives rise to a number of interesting and challenging mathematical problems. The objective of this paper is to outline some of these problems, to review progress made in solving these problems in a number of diverse communities, an ..."
Abstract
-
Cited by 114 (8 self)
- Add to MetaCart
(Show Context)
The study of the stability properties of switched and hybrid systems gives rise to a number of interesting and challenging mathematical problems. The objective of this paper is to outline some of these problems, to review progress made in solving these problems in a number of diverse communities, and to review some problems that remain open. An important contribution of our work is to bring together material from several areas of research and to present results in a unified manner. We begin our review by relating the stability problem for switched linear systems and a class of linear differential inclusions. Closely related to the concept of stability are the notions of exponential growth rates and converse Lyapunov theorems, both of which are discussed in detail. In particular, results on common quadratic Lyapunov functions and piecewise linear Lyapunov functions are presented, as they represent constructive methods for proving stability, and also represent problems in which significant progress has been made. We also comment on the inherent difficulty of determining stability of switched systems in general which is exemplified by NP-hardness and undecidability results. We then proceed by considering the stability of switched systems in which there are constraints on the switching rules, through both dwell time requirements and state dependent switching laws. Also in this case the theory of Lyapunov functions and the existence of converse theorems is reviewed. We briefly comment on the classical Lur’e problem and on the theory of stability radii, both of which contain many of the features of switched systems and are rich sources of practical results on the topic. Finally we present a list of questions and open problems which provide motivation for continued research in this area.
On the undecidability of probabilistic planning and infinite-horizon partially observable Markov decision process problems
- Artificial Intelligence
"... Abstract We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence i ..."
Abstract
-
Cited by 103 (0 self)
- Add to MetaCart
(Show Context)
Abstract We investigate the computability of problems in probabilistic planning and partially observable infinite-horizon Markov decision processes. The undecidability of the string-existence problem for probabilistic finite automata is adapted to show that the following problem of plan existence in probabilistic planning is undecidable: given a probabilistic planning problem, determine whether there exists a plan with success probability exceeding a desirable threshold. Analogous policy-existence problems for partially observable infinite-horizon Markov decision processes under discounted and undiscounted total reward models, average-reward models, and state-avoidance models are all shown to be undecidable. The results apply to corresponding approximation problems as well.
On the Undecidability of Probabilistic Planning and Related Stochastic Optimization Problems
- Artificial Intelligence
, 2003
"... Automated planning, the problem of how an agent achieves a goal given a repertoire of actions, is one of the foundational and most widely studied problems in the AI literature. The original formulation of the problem makes strong assumptions regarding the agent's knowledge and control over the ..."
Abstract
-
Cited by 74 (0 self)
- Add to MetaCart
Automated planning, the problem of how an agent achieves a goal given a repertoire of actions, is one of the foundational and most widely studied problems in the AI literature. The original formulation of the problem makes strong assumptions regarding the agent's knowledge and control over the world, namely that its information is complete and correct, and that the results of its actions are deterministic and known.
The Boundedness of All Products of a Pair of Matrices is Undecidable
, 2000
"... We show that the boundedness of the set of all products of a given pair Sigma of rational matrices is undecidable. Furthermore, we show that the joint (or generalized) spectral radius #(#) is not computable because testing whether #(#)61 is an undecidable problem. As a consequence, the robust stabil ..."
Abstract
-
Cited by 68 (13 self)
- Add to MetaCart
We show that the boundedness of the set of all products of a given pair Sigma of rational matrices is undecidable. Furthermore, we show that the joint (or generalized) spectral radius #(#) is not computable because testing whether #(#)61 is an undecidable problem. As a consequence, the robust stability of linear systems under time-varying perturbations is undecidable, and the same is true for the stability of a simple class of hybrid systems. We also discuss some connections with the so-called "finiteness conjecture". Our results are based on a simple reduction from the emptiness problem for probabilistic finite automata, which is known to be undecidable.
Complexity of Stability and Controllability of Elementary Hybrid Systems
, 1997
"... this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NP-hard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into tw ..."
Abstract
-
Cited by 57 (9 self)
- Add to MetaCart
this paper, weconsider simple classes of nonlinear systems and provethatbasic questions related to their stabilityandcontrollabilityare either undecidable or computationally intractable (NP-hard). As a special case, weconsider a class of hybrid systems in which the state space is partitioned into two halfspaces, and the dynamics in eachhalfspace correspond to a differentlinear system
Termination of Linear Programs
- In CAV’2004: Computer Aided Verification, volume 3114 of LNCS
, 2004
"... We show that termination of a class of linear loop programs is decidable. Linear loop programs are discrete-time linear systems with a loop condition governing termination, that is, a while loop with linear assignments. We relate the termination of such a simple loop, on all initial values, to t ..."
Abstract
-
Cited by 54 (0 self)
- Add to MetaCart
(Show Context)
We show that termination of a class of linear loop programs is decidable. Linear loop programs are discrete-time linear systems with a loop condition governing termination, that is, a while loop with linear assignments. We relate the termination of such a simple loop, on all initial values, to the eigenvectors corresponding to only the positive real eigenvalues of the matrix defining the loop assignments. This characterization of termination is reminiscent of the famous stability theorems in control theory that characterize stability in terms of eigenvalues.
Undecidable Problems of Decentralized Observation and Control
, 2001
"... We introduce a new notion of decentralized observability for discrete-event systems, which we call joint observability. We prove that checking joint observability of a regular language w.r.t. one observer is decidable, whereas for two (or more) observers the problem becomes undecidable. Based on thi ..."
Abstract
-
Cited by 52 (4 self)
- Add to MetaCart
(Show Context)
We introduce a new notion of decentralized observability for discrete-event systems, which we call joint observability. We prove that checking joint observability of a regular language w.r.t. one observer is decidable, whereas for two (or more) observers the problem becomes undecidable. Based on this result, we show that a related decentralized control problem is also undecidable. We finally provide an extensive study relating our work to existing work in the literature.
