Abstract—The Lasso is a popular technique for joint estimation and continuous variable selection, especially well-suited for sparse and possibly under-determined linear regression problems. This paper develops algorithms to estimate the regression coefficients via Lasso when the training data are distributed across different agents, and their communication to a central processing unit is prohibited for e.g., communication cost or privacy reasons. A motivating application is explored in the context of wireless communications, whereby sensing cognitive radios collaborate to estimate the radio-frequency power spectrum density. Attaining different tradeoffs between complexity and convergence speed, three novel algorithms are obtained after reformulating the Lasso into a separable form, which is iteratively minimized using the alternating-direction method of multipliers so as to gain the desired degree of parallelization. Interestingly, the per agent estimate updates are given by simple soft-thresholding operations, and inter-agent communication overhead remains at affordable level. Without exchanging elements from the different training sets, the local estimates consent to the global Lasso solution, i.e., the fit that would be obtained if the entire data set were centrally available. Numerical experiments with both simulated and real data demonstrate the merits of the proposed distributed schemes, corroborating their convergence and global optimality. The ideas in this paper can be easily extended for the purpose of fitting related models in a distributed fashion, including the adaptive Lasso, elastic net, fused Lasso and nonnegative garrote. Index Terms—Distributed linear regression, Lasso, parallel op-timization, sparse estimation. I.

Abstract not found

Abstract not found

Abstract not found

Gossip algorithms have recently received significant attention, mainly because they constitute simple and robust message-passing schemes for distributed information processing over networks. However for many topologies that are realistic for wireless ad-hoc and sensor networks (like grids and random geometric graphs), the standard nearest-neighbor gossip converges as slowly as flooding (O(n 2) messages). A recently proposed algorithm called geographic gossip improves gossip efficiency by a √ n factor, by exploiting geographic information to enable multi-hop long distance communications. In this paper we prove that a variation of geographic gossip that averages along routed paths, improves efficiency by an additional √ n factor and is order optimal (O(n) messages) for grids and random geometric graphs. We develop a general technique (travel agency method) based on Markov chain mixing time inequalities, which can give bounds on the performance of randomized message-passing algorithms operating over various graph topologies.

Abstract — In this paper we extend and analyze the dis-tributed dual averaging algorithm [1] to handle communication delays and general stochastic consensus protocols. Assuming each network link experiences some fixed bounded delay, we show that distributed dual averaging converges and the error decays at a rate O(T−0.5) where T is the number of iterations. This bound is an improvement over [1] by a logarithmic factor in T for networks of fixed size. Finally, we extend the algorithm to the case of using general non-averaging consensus protocols. We prove that the bias introduced in the optimization can be removed by a simple correction that depends on the stationary distribution of the consensus matrix. I.

Abstract not found

We analyze how node mobility can influence the convergence time of averaging gossip algorithms on networks. Our main result is that even a small number of fully mobile nodes can yield a significant decrease in convergence time. We develop a method for deriving lower bounds on the convergence time by merging nodes according to their mobility pattern. We use this method to show that if the agents have one-dimensional mobility in the same direction the convergence time is improved by at most a constant. We also obtain upper bounds on the convergence time using the Poincaré inequality and show that simple models of mobility can dramatically accelerate gossip as long as the mobility paths significantly overlap. We use simulations to show that our bounds are still valid for more general mobility models that seem analytically intractable, and further illustrate that different mobility patterns can have significantly different effects on the convergence of distributed algorithms.

Abstract not found

Abstract not found