Results 11  20
of
149
Distance metric learning with eigenvalue optimization
 Journal of Machine Learning Research (Special Topics on Kernel and Metric Learning
, 2012
"... The main theme of this paper is to develop a novel eigenvalue optimization framework for learning a Mahalanobis metric. Within this context, we introduce a novel metric learning approach called DMLeig which is shown to be equivalent to a wellknown eigenvalue optimization problem called minimizing ..."
Abstract

Cited by 46 (2 self)
 Add to MetaCart
The main theme of this paper is to develop a novel eigenvalue optimization framework for learning a Mahalanobis metric. Within this context, we introduce a novel metric learning approach called DMLeig which is shown to be equivalent to a wellknown eigenvalue optimization problem called minimizing the maximal eigenvalue of a symmetric matrix (Overton, 1988; Lewis and Overton, 1996). Moreover, we formulate LMNN (Weinberger et al., 2005), one of the stateoftheart metric learning methods, as a similar eigenvalue optimization problem. This novel framework not only provides new insights into metric learning but also opens new avenues to the design of efficient metric learning algorithms. Indeed, firstorder algorithms are developed for DMLeig and LMNN which only need the computation of the largest eigenvector of a matrix per iteration. Their convergence characteristics are rigorously established. Various experiments on benchmark data sets show the competitive performance of our new approaches. In addition, we report an encouraging result on a difficult and challenging face verification data set called Labeled Faces in the Wild (LFW).
ANGULAR SYNCHRONIZATION BY EIGENVECTORS AND SEMIDEFINITE PROGRAMMING: ANALYSIS AND APPLICATION TO CLASS AVERAGING IN CRYOELECTRON MICROSCOPY
, 2009
"... The angular synchronization problem is to obtain an accurate estimation (up to a constant additive phase) for a set of unknown angles θ1,..., θn from m noisy measurements of their offsets θi − θj mod 2π. Of particular interest is angle recovery in the presence of many outlier measurements that are ..."
Abstract

Cited by 45 (18 self)
 Add to MetaCart
(Show Context)
The angular synchronization problem is to obtain an accurate estimation (up to a constant additive phase) for a set of unknown angles θ1,..., θn from m noisy measurements of their offsets θi − θj mod 2π. Of particular interest is angle recovery in the presence of many outlier measurements that are uniformly distributed in [0,2π) and carry no information on the true offsets. We introduce an efficient recovery algorithm for the unknown angles from the top eigenvector of a specially designed Hermitian matrix. The eigenvector method is extremely stable and succeeds even when the number of outliers is exceedingly large. For example, we successfully estimate n = 400 angles from a full set of m = `400 ´ offset measurements of which 90 % are outliers in less than a second 2 on a commercial laptop. We use random matrix theory to prove that the eigenvector method q gives
Practical LargeScale Optimization for MaxNorm Regularization
"... The maxnorm was proposed as a convex matrix regularizer in [1] and was shown to be empirically superior to the tracenorm for collaborative filtering problems. Although the maxnorm can be computed in polynomial time, there are currently no practical algorithms for solving largescale optimization ..."
Abstract

Cited by 44 (13 self)
 Add to MetaCart
(Show Context)
The maxnorm was proposed as a convex matrix regularizer in [1] and was shown to be empirically superior to the tracenorm for collaborative filtering problems. Although the maxnorm can be computed in polynomial time, there are currently no practical algorithms for solving largescale optimization problems that incorporate the maxnorm. The present work uses a factorization technique of Burer and Monteiro [2] to devise scalable firstorder algorithms for convex programs involving the maxnorm. These algorithms are applied to solve huge collaborative filtering, graph cut, and clustering problems. Empirically, the new methods outperform mature techniques from all three areas. 1
Regularization methods for semidefinite programming
 SIAM JOURNAL ON OPTIMIZATION
, 2009
"... We introduce a new class of algorithms for solving linear semidefinite programming (SDP) problems. Our approach is based on classical tools from convex optimization such as quadratic regularization and augmented Lagrangian techniques. We study the theoretical properties and we show that practical im ..."
Abstract

Cited by 44 (7 self)
 Add to MetaCart
We introduce a new class of algorithms for solving linear semidefinite programming (SDP) problems. Our approach is based on classical tools from convex optimization such as quadratic regularization and augmented Lagrangian techniques. We study the theoretical properties and we show that practical implementations behave very well on some instances of SDP having a large number of constraints. We also show that the “boundary point method” from [PRW06] is an instance of this class.
Ensemble Pruning Via Semidefinite Programming
 Journal of Machine Learning Research
, 2006
"... An ensemble is a group of learning models that jointly solve a problem. However, the ensembles generated by existing techniques are sometimes unnecessarily large, which can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. The purpose of ensemble pruning is ..."
Abstract

Cited by 42 (2 self)
 Add to MetaCart
An ensemble is a group of learning models that jointly solve a problem. However, the ensembles generated by existing techniques are sometimes unnecessarily large, which can lead to extra memory usage, computational costs, and occasional decreases in effectiveness. The purpose of ensemble pruning is to search for a good subset of ensemble members that performs as well as, or better than, the original ensemble. This subset selection problem is a combinatorial optimization problem and thus finding the exact optimal solution is computationally prohibitive. Various heuristic methods have been developed to obtain an approximate solution. However, most of the existing heuristics use simple greedy search as the optimization method, which lacks either theoretical or empirical quality guarantees. In this paper, the ensemble subset selection problem is formulated as a quadratic integer programming problem. By applying semidefinite programming (SDP) as a solution technique, we are able to get better approximate solutions. Computational experiments show that this SDPbased pruning algorithm outperforms other heuristics in the literature. Its application in a classifiersharing study also demonstrates the effectiveness of the method.
An implementable proximal point algorithmic framework for nuclear norm minimization
, 2010
"... The nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints. Such a problem often arises from the convex relaxation of a rank minimization problem with noisy data, and arises in many fields of engineering and science. In ..."
Abstract

Cited by 40 (5 self)
 Add to MetaCart
(Show Context)
The nuclear norm minimization problem is to find a matrix with the minimum nuclear norm subject to linear and second order cone constraints. Such a problem often arises from the convex relaxation of a rank minimization problem with noisy data, and arises in many fields of engineering and science. In this paper, we study inexact proximal point algorithms in the primal, dual and primaldual forms for solving the nuclear norm minimization with linear equality and second order cone constraints. We design efficient implementations of these algorithms and present comprehensive convergence results. In particular, we investigate the performance of our proposed algorithms in which the inner subproblems are approximately solved by the gradient projection method or the accelerated proximal gradient method. Our numerical results for solving randomly generated matrix completion problems and real matrix completion problems show that our algorithms perform favorably in comparison to several recently proposed stateoftheart algorithms. Interestingly, our proposed algorithms are connected with other algorithms that have been studied in the literature. Key words. Nuclear norm minimization, proximal point method, rank minimization, gradient projection method, accelerated proximal gradient method.
Lowrank matrix completion by riemannian optimization
 ANCHPMATHICSE, Mathematics Section, École Polytechnique Fédérale de
"... The matrix completion problem consists of finding or approximating a lowrank matrix based on a few samples of this matrix. We propose a novel algorithm for matrix completion that minimizes the least square distance on the sampling set over the Riemannian manifold of fixedrank matrices. The algorit ..."
Abstract

Cited by 40 (4 self)
 Add to MetaCart
(Show Context)
The matrix completion problem consists of finding or approximating a lowrank matrix based on a few samples of this matrix. We propose a novel algorithm for matrix completion that minimizes the least square distance on the sampling set over the Riemannian manifold of fixedrank matrices. The algorithm is an adaptation of classical nonlinear conjugate gradients, developed within the framework of retractionbased optimization on manifolds. We describe all the necessary objects from differential geometry necessary to perform optimization over this lowrank matrix manifold, seen as a submanifold embedded in the space of matrices. In particular, we describe how metric projection can be used as retraction and how vector transport lets us obtain the conjugate search directions. Additionally, we derive secondorder models that can be used in Newton’s method based on approximating the exponential map on this manifold to second order. Finally, we prove convergence of a regularized version of our algorithm under the assumption that the restricted isometry property holds for incoherent matrices throughout the iterations. The numerical experiments indicate that our approach scales very well for largescale problems and compares favorable with the stateoftheart, while outperforming most existing solvers. 1
ThreeDimensional Structure Determination from Common Lines in CryoEM by Eigenvectors and Semidefinite Programming
, 2011
"... The cryoelectron microscopy reconstruction problem is to find the threedimensional (3D) structure of a macromolecule given noisy samples of its twodimensional projection images at unknown random directions. Present algorithms for finding an initial 3D structure model are based on the “angular r ..."
Abstract

Cited by 33 (17 self)
 Add to MetaCart
(Show Context)
The cryoelectron microscopy reconstruction problem is to find the threedimensional (3D) structure of a macromolecule given noisy samples of its twodimensional projection images at unknown random directions. Present algorithms for finding an initial 3D structure model are based on the “angular reconstitution ” method in which a coordinate system is established from three projections, and the orientation of the particle giving rise to each image is deduced from common lines among the images. However, a reliable detection of common lines is difficult due to the low signaltonoise ratio of the images. In this paper we describe two algorithms for finding the unknown imaging directions of all projections by minimizing global selfconsistency errors. In the first algorithm, the minimizer is obtained by computing the three largest eigenvectors of a specially designed symmetric matrix derived from the common lines, while the second algorithm is based on semidefinite programming (SDP). Compared with existing algorithms, the advantages of our algorithms are fivefold: first, they accurately estimate all orientations at very low commonline detection rates; second, they are extremely fast, as they involve only the computation of a few top eigenvectors or a sparse SDP; third, they are nonsequential and use the information in all common lines at once; fourth, they are amenable to a rigorous mathematical analysis using spectral analysis and random matrix theory; and finally, the algorithms are optimal in the sense that they reach the information theoretic Shannon bound up to a constant for an idealized probabilistic model.
Semidefinite Programming Relaxations and Algebraic Optimization in Control
 EUROPEAN JOURNAL OF CONTROL (2003)9:307321
, 2003
"... We present an overview of the essential elements of semidefinite programming as a computational tool for the analysis of systems and control problems. We make particular emphasis on general duality properties as providing suboptimality or infeasibility certificates. Our focus is on the exciting deve ..."
Abstract

Cited by 33 (5 self)
 Add to MetaCart
(Show Context)
We present an overview of the essential elements of semidefinite programming as a computational tool for the analysis of systems and control problems. We make particular emphasis on general duality properties as providing suboptimality or infeasibility certificates. Our focus is on the exciting developments which have occured in the last few years, including robust optimization, combinatorial optimization, and algebraic methods such as sumofsquares. These developments are illustrated with examples of applications to control systems.