Results 1  10
of
13
Optimal and approximate Qvalue functions for decentralized POMDPs
 J. Artificial Intelligence Research
"... Decisiontheoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In singleagent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Qvalue functions: an optimal Qvalue functi ..."
Abstract

Cited by 62 (27 self)
 Add to MetaCart
(Show Context)
Decisiontheoretic planning is a popular approach to sequential decision making problems, because it treats uncertainty in sensing and acting in a principled way. In singleagent frameworks like MDPs and POMDPs, planning can be carried out by resorting to Qvalue functions: an optimal Qvalue function Q ∗ is computed in a recursive manner by dynamic programming, and then an optimal policy is extracted from Q ∗. In this paper we study whether similar Qvalue functions can be defined for decentralized POMDP models (DecPOMDPs), and how policies can be extracted from such value functions. We define two forms of the optimal Qvalue function for DecPOMDPs: one that gives a normative description as the Qvalue function of an optimal pure joint policy and another one that is sequentially rational and thus gives a recipe for computation. This computation, however, is infeasible for all but the smallest problems. Therefore, we analyze various approximate Qvalue functions that allow for efficient computation. We describe how they relate, and we prove that they all provide an upper bound to the optimal Qvalue function Q ∗. Finally, unifying some previous approaches for solving DecPOMDPs, we describe a family of algorithms for extracting policies from such Qvalue functions, and perform an experimental evaluation on existing test problems, including a new firefighting benchmark problem. 1.
Anytime coordination using separable bilinear programs
 In AAAI
, 2007
"... Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime perfor ..."
Abstract

Cited by 21 (11 self)
 Add to MetaCart
(Show Context)
Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime performance, but an error bound on the results has not been established. We reformulate the algorithm and derive both online and offline error bounds for approximate solutions. Moreover, we propose an effective way to automatically reduce the complexity of the interaction. Our experiments show that this is a promising approach to solve a broad class of decentralized decision problems. The general formulation used by the algorithm makes it both easy to implement and widely applicable to a variety of other AI problems.
A bilinear programming approach for multiagent planning
 Journal of Artificial Intelligence Research
"... Abstract Multiagent planning and coordination problems are common and known to be computationally hard. We show that a wide range of twoagent problems can be formulated as bilinear programs. We present a successive approximation algorithm that significantly outperforms the coverage set algorithm, ..."
Abstract

Cited by 16 (2 self)
 Add to MetaCart
(Show Context)
Abstract Multiagent planning and coordination problems are common and known to be computationally hard. We show that a wide range of twoagent problems can be formulated as bilinear programs. We present a successive approximation algorithm that significantly outperforms the coverage set algorithm, which is the stateoftheart method for this class of multiagent problems. Because the algorithm is formulated for bilinear programs, it is more general and simpler to implement. The new algorithm can be terminated at any time andunlike the coverage set algorithmit facilitates the derivation of a useful online performance bound. It is also much more efficient, on average reducing the computation time of the optimal solution by about four orders of magnitude. Finally, we introduce an automatic dimensionality reduction method that improves the effectiveness of the algorithm, extending its applicability to new domains and providing a new way to analyze a subclass of bilinear programs.
The CrossEntropy Method for Policy Search in Decentralized POMDPs
, 2008
"... Decentralized POMDPs (DecPOMDPs) are becoming increasingly popular as models for multiagent planning under uncertainty, but solving a DecPOMDP exactly is known to be an intractable combinatorial optimization problem. In this paper we apply the CrossEntropy (CE) method, a recently introduced metho ..."
Abstract

Cited by 11 (9 self)
 Add to MetaCart
Decentralized POMDPs (DecPOMDPs) are becoming increasingly popular as models for multiagent planning under uncertainty, but solving a DecPOMDP exactly is known to be an intractable combinatorial optimization problem. In this paper we apply the CrossEntropy (CE) method, a recently introduced method for combinatorial optimization, to DecPOMDPs, resulting in a randomized (samplingbased) algorithm for approximately solving DecPOMDPs. This algorithm operates by sampling pure policies from an appropriately parametrized stochastic policy, and then evaluates these policies either exactly or approximately in order to define the next stochastic policy to sample from, and so on until convergence. Experimental results demonstrate that the CE method can search huge spaces efficiently, supporting our claim that combinatorial optimization methods can bring leverage to the approximate solution of DecPOMDPs. Povzetek: Prispevek opisuje novo metodo multiagentnega načrtovanja. 1
Abstracting Influences for Efficient Multiagent Coordination Under Uncertainty
"... ii ..."
(Show Context)
Interaction structure and dimensionality in decentralized problem solving
 In Conference on Artificial Intelligence (AAAI
, 2008
"... Decentralized Markov Decision Processes are a powerful general model of decentralized, cooperative multiagent problem solving. The high complexity of the general problem leads to a focus on restricted models. While the worstcase hardness of such reduced problems is often better, less is known about ..."
Abstract

Cited by 3 (2 self)
 Add to MetaCart
Decentralized Markov Decision Processes are a powerful general model of decentralized, cooperative multiagent problem solving. The high complexity of the general problem leads to a focus on restricted models. While the worstcase hardness of such reduced problems is often better, less is known about the actual expected difficulty of given instances. We show tight connections between the structure of agent interactions and the essential dimensionality of various problems. Bounds can be placed on the difficulty of solving problems, based upon restrictions on the type and number of interactions between agents. These bounds arise from a bilinear programming formulation of the problem; from such a formulation, a more compact reduced form can be automatically generated, and the original problem can be rewritten to take advantage of the reduction. These results are of theoretical and practical importance, improving our understanding of multiagent problem domains, and paving the way for methods that reduce the complexity of such problems by limiting the degree of interaction between agents.
Decision Making in omplex Multiagent Contexts: A Tale of Two Frameworks
 AI MAGAZINE
, 2012
"... ..."
Anytime Coordination Using Separable Bilinear Programs
"... Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime perfor ..."
Abstract
 Add to MetaCart
(Show Context)
Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime performance, but an error bound on the results has not been established. We reformulate the algorithm and derive both online and offline error bounds for approximate solutions. Moreover, we propose an effective way to automatically reduce the complexity of the interaction. Our experiments show that this is a promising approach to solve a broad class of decentralized decision problems. The general formulation used by the algorithm makes it both easy to implement and widely applicable to a variety of other AI problems.
Anytime Coordination Using Separable Bilinear Programs
"... Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime perfor ..."
Abstract
 Add to MetaCart
(Show Context)
Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime performance, but an error bound on the results has not been established. We reformulate the algorithm and derive both online and offline error bounds for approximate solutions. Moreover, we propose an effective way to automatically reduce the complexity of the interaction. Our experiments show that this is a promising approach to solve a broad class of decentralized decision problems. The general formulation used by the algorithm makes it both easy to implement and widely applicable to a variety of other AI problems.
A Successive Approximation Algorithm for Coordination Problems ∗
"... Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime perfor ..."
Abstract
 Add to MetaCart
(Show Context)
Developing scalable coordination algorithms for multiagent systems is a hard computational challenge. One useful approach, demonstrated by the Coverage Set Algorithm (CSA), exploits structured interaction to produce significant computational gains. Empirically, CSA exhibits very good anytime performance, but an error bound on the results has not been established. We reformulate the algorithm and derive an online error bound for approximate solutions. Moreover, we propose an effective way to automatically reduce the complexity of the interaction. Our experiments show that this is a promising approach to solve a broad class of decentralized decision problems. The general formulation used by the algorithm makes it both easy to implement and widely applicable to a variety of other AI problems. 1