Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling (2010)
Cached
Download Links
| Venue: | IEEE TRANSACTIONS ON AUTOMATIC CONTROL |
| Citations: | 97 - 12 self |
BibTeX
@MISC{Duchi10dualaveraging,
author = {John C. Duchi and Alekh Agarwal and Martin J. Wainwright},
title = { Dual Averaging for Distributed Optimization: Convergence Analysis and Network Scaling},
year = {2010}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
The goal of decentralized optimization over a network is to optimize a global objective formed by a sum of local (possibly nonsmooth) convex functions using only local computation and communication. It arises in various application domains, including distributed tracking and localization, multiagent co-ordination, estimation in sensor networks, and largescale machine learning. We develop and analyze distributed algorithms based on dual subgradient averaging, and we provide sharp bounds on their convergence rates as a function of the network size and topology. Our analysis allows us to clearly separate the convergence of the optimization algorithm itself and the effects of communication dependent on the network structure. We show that the number of iterations required by our algorithm scales inversely in the spectral gap of the network and confirm this prediction’s sharpness both by theoretical lower bounds and simulations for various networks. Our approach includes the cases of deterministic optimization and communication as well as problems with stochastic optimization and/or communication.
Keyphrases
network scaling dual averaging distributed optimization convergence analysis decentralized optimization network size dual subgradient averaging convergence rate sensor network largescale machine learning prediction sharpness local computation spectral gap deterministic optimization various network network structure multiagent co-ordination various application domain convex function communication dependent global objective stochastic optimization sharp bound
