Many important machine learning problems are modeled and solved via semidefinite programs; examples include metric learning, nonlinear embedding, and certain clustering problems. Often, off-the-shelf software is invoked for the associated optimization, which can be inappropriate due to excessive computational and storage requirements. In this paper, we introduce the use of convex perturbations for solving semidefinite programs (SDPs), and for a specific perturbation we derive an algorithm that has several advantages over existing techniques: a) it is simple, requiring only a few lines of MATLAB, b) it is a first-order method, and thereby scalable, and c) it can easily exploit the structure of a given SDP (e.g., when the constraint matrices are low-rank, a situation common to several machine learning SDPs). A pleasant byproduct of our method is a fast, kernelized version of the large-margin nearest neighbor metric learning algorithm (Weinberger et al., 2005). We demonstrate that our algorithm is effective in finding fast approximations to large-scale SDPs arising in some machine learning applications. 1

Abstract — The maximum lifetime routing problem in wireless sensor networks has received increasing attention in recent years. One way is to formulate it as a linear programming problem by maximizing the time at which the first node runs out of energy subject to the flow conservation constraints. The solutions in this problem correspond to the rates allocated to each link. In this paper, we first show that, under certain conditions, the solutions of this problem are not unique for some network topologies. Given the feasible solutions set, one can further define a secondary optimization problem by minimizing the end-to-end packet transfer delay or power consumption. Rather than solving two sequential optimization problems, in this paper, we propose the use of a regularization method which can jointly maximize the network lifetime and minimize another objective (e.g., packet delay). We describe the fully distributed implementation and provide performance comparisons with other algorithms. I.

This paper develops a general framework for solving a variety of convex cone problems that frequently arise in signal processing, machine learning, statistics, and other fields. The approach works as follows: first, determine a conic formulation of the problem; second, determine its dual; third, apply smoothing; and fourth, solve using an optimal first-order method. A merit of this approach is its flexibility: for example, all compressed sensing problems can be solved via this approach. These include models with objective functionals such as the total-variation norm, ‖W x‖1 where W is arbitrary, or a combination thereof. In addition, the paper also introduces a number of technical contributions such as a novel continuation scheme, a novel approach for controlling the step size, and some new results showing that the smooth and unsmoothed problems are sometimes formally equivalent. Combined with our framework, these lead to novel, stable and computationally efficient algorithms. For instance, our general implementation is competitive with state-of-the-art methods for solving intensively studied problems such as the LASSO. Further, numerical experiments show that one can solve the Dantzig selector problem, for which no efficient large-scale solvers exist, in a few hundred iterations. Finally, the paper is accompanied with a software release. This software is not a single, monolithic solver; rather, it is a suite of programs and routines designed to serve as building blocks for constructing complete algorithms. Keywords. Optimal first-order methods, Nesterov’s accelerated descent algorithms, proximal algorithms, conic duality, smoothing by conjugation, the Dantzig selector, the LASSO, nuclearnorm minimization.

Sparse overcomplete representations have attracted much interest recently for their applications to signal processing. In a recent work, Donoho, Elad, and Temlyakov [12] showed that, assuming sufficient sparsity of the ideal underlying signal and approximate orthogonality of the overcomplete dictionary, the sparsest representation can be found, at least approximately if not exactly, by either an orthogonal greedy algorithm or  ¢ ¡ by-norm minimization subject to a noise tolerance constraint. In this paper, we sharpen the approximation bounds under more relaxed conditions. We also derive analogous results for a stepwise projection algorithm.

Several important machine learning problems can be modeled and solved via semidefinite programs. Often, researchers invoke off-the-shelf software for the associated optimization, which can be inappropriate for many applications due to computational and storage requirements. In this paper, we introduce the use of convex perturbations for semidefinite programs (SDPs). Using a particular perturbation function, we arrive at an algorithm for SDPs that has several advantages over existing techniques: a) it is simple, requiring only a few lines of MATLAB, b) it is a first-order method which makes it scalable, c) it can easily exploit the structure of a particular SDP to gain efficiency (e.g., when the constraint matrices are low-rank). We demonstrate on several machine learning applications that the proposed algorithm is effective in finding fast approximations to large-scale SDPs. 1

In wireless sensor networks (WSNs), the field information (e.g., temperature, humidity, airflow) is acquired via several battery-equipped wireless sensors and is relayed towards a sink node. As the size of the WSNs increases, it becomes inefficient (in terms of power consumption) to gather all information in a single sink. To tackle this problem, one can increase the number of sinks. The set of sensor nodes sending data to sink k is called commodity k. In this paper, we formulate the lexicographically optimal commodity lifetime routing problem. A stepwise centralized algorithm, called the lexicographically optimal commodity lifetime (LOCL) algorithm, is proposed which can obtain the optimal routing solution and lead to lexicographical fairness among commodity lifetimes. We then show that under certain assumptions, the lexicographical optimality among commodity lifetimes can be achieved by providing lexicographical optimality among node lifetimes. This motivates us to propose our second algorithm, called the lexicographically optimal node lifetime (LONL) algorithm, which suitable for practical implementation. Simulation results show that our proposed LOCL and LONL algorithms increase the normalized commodity and node lifetimes compared to the maximum lifetime with multiple sinks (MLMS) [1] and lexicographical max-min fair (LMM) [2] routing algorithms.

In this paper we study a class of uncertain linear estimation problems in which the data are affected by random uncertainty. In this setting, we consider two estimation criteria, one based on minimization of the expected ℓ1 or ℓ2 norm residual and one based on minimization of the level within which the ℓ1 or ℓ2 norm residual is guaranteed to lie with an a-priori fixed probability (residual at risk). The random uncertainty affecting the data is characterized by means of its first two statistical moments, and the above criteria are intended in a worst-case probabilistic sense, that is worst-case expectations and probabilities over all possible distribution having the specified moments are considered. The ensuing estimation problems can be solved efficiently via convex programming, yielding exact solutions in the ℓ2 norm case and upper-bounds on the optimal solutions in the ℓ1 case.