Results 1  10
of
65
Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling
 IEEE TRANSACTIONS ON AUTOMATIC CONTROL
, 2010
"... The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multiagen ..."
Abstract

Cited by 97 (12 self)
 Add to MetaCart
The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multiagent coordination, estimation in sensor networks, and largescale machine learning. We develop and analyze distributed algorithms based on dual subgradient averaging, and we provide sharp bounds on their convergence rates as a function of the network size and topology. Our analysis allows us to clearly separate the convergence of the optimization algorithm itself and the effects of communication dependent on the network structure. We show that the number of iterations required by our algorithm scales inversely in the spectral gap of the network and confirm this prediction’s sharpness both by theoretical lower bounds and simulations for various networks. Our approach includes the cases of deterministic optimization and communication as well as problems with stochastic optimization and/or communication.
Broadcast gossip algorithms for consensus
 IEEE TRANS. SIGNAL PROCESS
, 2009
"... Motivated by applications to wireless sensor, peertopeer, and ad hoc networks, we study distributed broadcasting algorithms for exchanging information and computing in an arbitrarily connected network of nodes. Specifically, we study a broadcastingbased gossiping algorithm to compute the (possib ..."
Abstract

Cited by 93 (7 self)
 Add to MetaCart
(Show Context)
Motivated by applications to wireless sensor, peertopeer, and ad hoc networks, we study distributed broadcasting algorithms for exchanging information and computing in an arbitrarily connected network of nodes. Specifically, we study a broadcastingbased gossiping algorithm to compute the (possibly weighted) average of the initial measurements of the nodes at every node in the network. We show that the broadcast gossip algorithm converges almost surely to a consensus. We prove that the random consensus value is, in expectation, the average of initial node measurements and that it can be made arbitrarily close to this value in mean squared error sense, under a balanced connectivity model and by trading off convergence speed with accuracy of the computation. We provide theoretical and numerical results on the mean square error performance, on the convergence rate and study the effect of the “mixing parameter ” on the convergence rate of the broadcast gossip algorithm. The results indicate that the mean squared error strictly decreases through iterations until the consensus is achieved. Finally, we assess and compare the communication cost of the broadcast gossip algorithm to achieve a given distance to consensus through theoretical and numerical results.
On ergodicity, infinite flow and consensus in random models
, 2010
"... We consider the ergodicity and consensus problem for a discretetime linear dynamic model driven by random stochastic matrices, which is equivalent to studying these concepts for the product of such matrices. Our focus is on the model where the random matrices have independent but timevariant dist ..."
Abstract

Cited by 31 (14 self)
 Add to MetaCart
We consider the ergodicity and consensus problem for a discretetime linear dynamic model driven by random stochastic matrices, which is equivalent to studying these concepts for the product of such matrices. Our focus is on the model where the random matrices have independent but timevariant distribution. We introduce a new phenomenon, the infinite flow, and we study its fundamental properties and relations with the ergodicity and consensus. The central result is the infinite flow theorem establishing the equivalence between the infinite flow and the ergodicity for a class of independent random models, where the matrices in the model have a common steady state in expectation and a feedback property. For such models, this result demonstrates that the expected infinite flow is both necessary and sufficient for the ergodicity. The result is providing a deterministic characterization of the ergodicity, which can be used for studying the consensus and average consensus over random graphs.
Asynchronous gossip algorithms for stochastic optimization
 In Proceedings of the 48th IEEE Conference on Decision and Control
, 2009
"... Abstract — We consider a distributed multiagent network system where the goal is to minimize an objective function that can be written as the sum of component functions, each of which is known partially (with stochastic errors) to a specific network agent. We propose an asynchronous algorithm that ..."
Abstract

Cited by 23 (3 self)
 Add to MetaCart
(Show Context)
Abstract — We consider a distributed multiagent network system where the goal is to minimize an objective function that can be written as the sum of component functions, each of which is known partially (with stochastic errors) to a specific network agent. We propose an asynchronous algorithm that is motivated by random gossip schemes where each agent has a local Poisson clock. At each tick of its local clock, the agent averages its estimate with a randomly chosen neighbor and adjusts the average using the gradient of its local function that is computed with stochastic errors. We investigate the convergence properties of the algorithm for two different classes of functions. First, we consider differentiable, but not necessarily convex functions, and prove that the gradients converge to zero with probability 1. Then, we consider convex, but not necessarily differentiable functions, and show that the iterates converge to an optimal solution almost surely. I.
Optimization and Analysis of Distributed Averaging with Short Node Memory
"... Distributed averaging describes a class of network algorithms for the decentralized computation of aggregate statistics. Initially, each node has a scalar data value, and the goal is to compute the average of these values at every node (the socalled average consensus problem). Nodes iteratively exc ..."
Abstract

Cited by 23 (6 self)
 Add to MetaCart
(Show Context)
Distributed averaging describes a class of network algorithms for the decentralized computation of aggregate statistics. Initially, each node has a scalar data value, and the goal is to compute the average of these values at every node (the socalled average consensus problem). Nodes iteratively exchange information with their neighbors and perform local updates until the value at every node converges to the initial network average. Much previous work has focused on algorithms where each node maintains and updates a single value; every time an update is performed, the previous value is forgotten. Convergence to the average consensus is achieved asymptotically. The convergence rate is fundamentally limited by network connectivity, and it can be prohibitively slow on topologies such as grids and random geometric graphs, even if the update rules are optimized. In this paper, we provide the first theoretical demonstration that adding a local prediction component to the update rule can significantly improve the convergence rate of distributed averaging algorithms. We focus on the case where the local predictor is a linear combination of the node’s current and previous values (i.e., two memory taps), and our update rule computes a combination of the predictor and the usual weighted linear combination of values received from neighbouring nodes. We derive the optimal mixing parameter for combining the predictor with the neighbors ’ values, and conduct a theoretical analysis of the improvement in convergence rate that can be achieved using this acceleration methodology. For a chain topology on N nodes, this leads to a factor of N improvement over standard consensus, and for a twodimensional grid, our approach achieves a factor of √ N improvement, in terms of the number of iterations required to reach a prescribed level of accuracy.
Polynomial Filtering for Fast Convergence in Distributed Consensus
, 2008
"... In the past few years, the problem of distributed consensus has received a lot of attention, particularly in the framework of ad hoc sensor networks. Most methods proposed in the literature address the consensus averaging problem by distributed linear iterative algorithms, with asymptotic convergenc ..."
Abstract

Cited by 21 (1 self)
 Add to MetaCart
(Show Context)
In the past few years, the problem of distributed consensus has received a lot of attention, particularly in the framework of ad hoc sensor networks. Most methods proposed in the literature address the consensus averaging problem by distributed linear iterative algorithms, with asymptotic convergence of the consensus solution. The convergence rate of such distributed algorithms typically depends on the network topology and the weights given to the edges between neighboring sensors, as described by the network matrix. In this paper, we propose to accelerate the convergence rate for given network matrices by the use of polynomial filtering algorithms. The main idea of the proposed methodology is to apply a polynomial filter on the network matrix that will shape its spectrum in order to increase the convergence rate. Such an algorithm is equivalent to periodic updates in each of the sensors by aggregating a few of its previous estimates. We formulate the computation of the coefficients of the optimal polynomial as a semidefinite program that can be efficiently and globally solved for both static and dynamic network topologies. We finally provide simulation results that demonstrate the effectiveness of the proposed solutions in accelerating the convergence of distributed consensus averaging problems.
The impact of mobility on gossip algorithms
 IN PROC. 28TH CONF. COMPUTER COMMUNICATIONS (INFOCOM), RIO DE JANEIRO
, 2009
"... We analyze how node mobility can influence the convergence time of averaging gossip algorithms on networks. Our main result is that even a small number of fully mobile nodes can yield a significant decrease in convergence time. We develop a method for deriving lower bounds on the convergence time b ..."
Abstract

Cited by 17 (2 self)
 Add to MetaCart
We analyze how node mobility can influence the convergence time of averaging gossip algorithms on networks. Our main result is that even a small number of fully mobile nodes can yield a significant decrease in convergence time. We develop a method for deriving lower bounds on the convergence time by merging nodes according to their mobility pattern. We use this method to show that if the agents have onedimensional mobility in the same direction the convergence time is improved by at most a constant. We also obtain upper bounds on the convergence time using the Poincaré inequality and show that simple models of mobility can dramatically accelerate gossip as long as the mobility paths significantly overlap. We use simulations to show that our bounds are still valid for more general mobility models that seem analytically intractable, and further illustrate that different mobility patterns can have significantly different effects on the convergence of distributed algorithms.
Anytime reliable transmission of realvalued information through digital noisy channels, invited paper
 in Proc. of FortySixth Allerton Conf
"... Abstract. The problem of reliably transmitting a realvalued random vector through a digital noisy channel is relevant for the design of distributed estimation and control techniques over networked systems. One important example consists in the remote state estimation under communication constrain ..."
Abstract

Cited by 15 (2 self)
 Add to MetaCart
(Show Context)
Abstract. The problem of reliably transmitting a realvalued random vector through a digital noisy channel is relevant for the design of distributed estimation and control techniques over networked systems. One important example consists in the remote state estimation under communication constraints. In this case, an anytime transmission scheme consists of an encoder –which maps the real vector into a sequence of channel inputs – and a decoder –which sequentially updates its estimate of the vector as more and more channel outputs are observed. The encoder performs both source and channel coding of the data. Assuming that no channel feedback is available at the transmitter, this paper studies the rates of convergence to zero of the mean squared error. Two coding strategies are analyzed: the first one has exponential convergence rate but it is expensive in terms of its encoder/decoder computational complexity, while the second one has a convenient computational complexity, but subexponential convergence rate. General bounds are obtained describing the convergence properties of these classes of methods. Key words. State estimation with communication constraints, anytime transmission, realtime communication, unequal error protection, digital fountain codes. AMS subject classifications. 93E10, 94A34, 94B70. 1. Introduction. Reliable
Distributed consensus over network with noisy links
 in Proceedings of the 12th International Conference on Information Fusion, 2009
"... Abstract – We consider a distributed consensus problem where a set of agents want to agree on a common value through local computations and communications. We assume that agents communicate over a network with timevarying topology and noisy communication links. We are interested in the case when t ..."
Abstract

Cited by 12 (4 self)
 Add to MetaCart
(Show Context)
Abstract – We consider a distributed consensus problem where a set of agents want to agree on a common value through local computations and communications. We assume that agents communicate over a network with timevarying topology and noisy communication links. We are interested in the case when the link noise is independent in time, and it has zero mean and bounded variance. We present and study an iterative algorithm with a diminishing stepsize. We show that the algorithm converges in expectation and almost surely to a “random ” consensus, and we characterize the statistics of the consensus. In particular, we give the expected value of the consensus and provide an upper bound on its variance.