Results 1  10
of
11
SNOPT: An SQP Algorithm For LargeScale Constrained Optimization
, 2002
"... Sequential quadratic programming (SQP) methods have proved highly effective for solving constrained optimization problems with smooth nonlinear functions in the objective and constraints. Here we consider problems with general inequality constraints (linear and nonlinear). We assume that first deriv ..."
Abstract

Cited by 597 (24 self)
 Add to MetaCart
(Show Context)
Sequential quadratic programming (SQP) methods have proved highly effective for solving constrained optimization problems with smooth nonlinear functions in the objective and constraints. Here we consider problems with general inequality constraints (linear and nonlinear). We assume that first derivatives are available, and that the constraint gradients are sparse. We discuss
SecondOrder Cone Programming
 MATHEMATICAL PROGRAMMING
, 2001
"... In this paper we survey the second order cone programming problem (SOCP). First we present several applications of the problem in various areas of engineering and robust optimization problems. We also give examples of optimization problems that can be cast as SOCPs. Next we review an algebraic struc ..."
Abstract

Cited by 247 (11 self)
 Add to MetaCart
In this paper we survey the second order cone programming problem (SOCP). First we present several applications of the problem in various areas of engineering and robust optimization problems. We also give examples of optimization problems that can be cast as SOCPs. Next we review an algebraic structure that is connected to SOCP. This algebra is a special case of a Euclidean Jordan algebra. After presenting duality theory, complementary slackness conditions, and definitions and algebraic characterizations of primal and dual nondegeneracy and strict complementarity we review the logarithmic barrier function for the SOCP problem and survey the pathfollowing interior point algorithms for it. Next we examine numerically stable methods for solving the interior point methods and study ways that sparsity in the input data can be exploited. Finally we give some current and future research direction in SOCP.
First and SecondOrder Methods for Learning: between Steepest Descent and Newton's Method
 Neural Computation
, 1992
"... Online first order backpropagation is sufficiently fast and effective for many largescale classification problems but for very high precision mappings, batch processing may be the method of choice. This paper reviews first and secondorder optimization methods for learning in feedforward neura ..."
Abstract

Cited by 177 (7 self)
 Add to MetaCart
Online first order backpropagation is sufficiently fast and effective for many largescale classification problems but for very high precision mappings, batch processing may be the method of choice. This paper reviews first and secondorder optimization methods for learning in feedforward neural networks. The viewpoint is that of optimization: many methods can be cast in the language of optimization techniques, allowing the transfer to neural nets of detailed results about computational complexity and safety procedures to ensure convergence and to avoid numerical problems. The review is not intended to deliver detailed prescriptions for the most appropriate methods in specific applications, but to illustrate the main characteristics of the different methods and their mutual relations.
A Computationally Efficient Feasible Sequential Quadratic Programming Algorithm
 SIAM Journal on Optimization
, 2001
"... . A sequential quadratic programming (SQP) algorithm generating feasible iterates is described and analyzed. What distinguishes this algorithm from previous feasible SQP algorithms proposed by various authors is a reduction in the amount of computation required to generate a new iterate while the pr ..."
Abstract

Cited by 56 (0 self)
 Add to MetaCart
. A sequential quadratic programming (SQP) algorithm generating feasible iterates is described and analyzed. What distinguishes this algorithm from previous feasible SQP algorithms proposed by various authors is a reduction in the amount of computation required to generate a new iterate while the proposed scheme still enjoys the same global and fast local convergence properties. A preliminary implementation has been tested and some promising numerical results are reported. Key words. sequential quadratic programming, SQP, feasible iterates, feasible SQP, FSQP AMS subject classifications. 49M37, 65K05, 65K10, 90C30, 90C53 PII. S1052623498344562 1.
Low rank updates for the Cholesky decomposition (Tech. Rep
, 2007
"... Usage of the ShermanMorrisonWoodbury formula to update linear systems after low rank modifications of the system matrix is widespread in machine learning. However, it is well known that this formula can lead to serious instabilities in the presence of roundoff error. If the system matrix is symme ..."
Abstract

Cited by 23 (3 self)
 Add to MetaCart
(Show Context)
Usage of the ShermanMorrisonWoodbury formula to update linear systems after low rank modifications of the system matrix is widespread in machine learning. However, it is well known that this formula can lead to serious instabilities in the presence of roundoff error. If the system matrix is symmetric positive definite, it is almost always possible to use a representation based on the Cholesky decomposition which renders the same results (in exact arithmetic) at the same or less operational cost, but typically is much more numerically stable. In this note, we show how the Cholesky decomposition can be updated to incorporate low rank additions or downdated for low rank subtractions. We also discuss a special case of an indefinite update of rank two. The methods discussed here are wellknown in the numerical mathematics literature, and code for most of them can be found in the LINPACK suite. Note: Matlab MEX implementations for most of the techniques described here are available for download at
QuasiNewton methods on Grassmannians and multilinear approximations of tensors
, 2009
"... Abstract. In this paper we proposed quasiNewton and limited memory quasiNewton methods for objective functions defined on Grassmann manifolds or a product of Grassmann manifolds. Specifically we defined bfgs and lbfgs updates in local and global coordinates on Grassmann manifolds or a product of ..."
Abstract

Cited by 18 (4 self)
 Add to MetaCart
(Show Context)
Abstract. In this paper we proposed quasiNewton and limited memory quasiNewton methods for objective functions defined on Grassmann manifolds or a product of Grassmann manifolds. Specifically we defined bfgs and lbfgs updates in local and global coordinates on Grassmann manifolds or a product of these. We proved that, when local coordinates are used, our bfgs updates on Grassmann manifolds share the same optimality property as the usual bfgs updates on Euclidean spaces. When applied to the best multilinear rank approximation problem for general and symmetric tensors, our approach yields fast, robust, and accurate algorithms that exploit the special Grassmannian structure of the respective problems, and which work on tensors of large dimensions and arbitrarily high order. Extensive numerical experiments are included to substantiate our claims. Key words. Grassmann manifold, Grassmannian, product of Grassmannians, Grassmann quasiNewton, Grassmann bfgs, Grassmann lbfgs, multilinear rank, symmetric multilinear rank, tensor, symmetric tensor, approximations
ReducedHessian QuasiNewton Methods For Unconstrained Optimization
 SIAM J. OPTIM
, 1999
"... QuasiNewton methods are reliable and efficient on a wide range of problems, but they can require many iterations if the problem is illconditioned or if a poor initial estimate of the Hessian is used. In this paper, we discuss methods designed to be more efficient in these situations. All the metho ..."
Abstract

Cited by 11 (2 self)
 Add to MetaCart
(Show Context)
QuasiNewton methods are reliable and efficient on a wide range of problems, but they can require many iterations if the problem is illconditioned or if a poor initial estimate of the Hessian is used. In this paper, we discuss methods designed to be more efficient in these situations. All the methods to be considered exploit the fact that quasiNewton methods accumulate approximate secondderivative information in a sequence of expanding subspaces. Associated with each of these subspaces is a certain reduced approximate Hessian that provides a complete but compact representation of the second derivative information approximated up to that point. Algorithms that compute an explicit reduced Hessian approximation have two important advantages over conventional quasiNewton methods. First, the amount of computation for each iteration is signicantly less during the early stages. This advantage is increased by forcing the iterates to linger on a manifold whose dimension can be significantly sma...
MATRIX FACTORIZATIONS IN OPTIMIZATION OF NONLINEAR FUNCTIONS SUBJECT TO LINEAR CONSTRAINTS
, 1975
"... Several ways of implementing methods for solving nonlinear optimization problems involving linear inequality and equality constraints using numerically stable matrix factorizations are described. The methods considered all follow an active constraint set approach and include quadratic programming, v ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Several ways of implementing methods for solving nonlinear optimization problems involving linear inequality and equality constraints using numerically stable matrix factorizations are described. The methods considered all follow an active constraint set approach and include quadratic programming, variable metric, and modified Newton methods.
Statistical Learning in Multiple Instance Problems
"... 1As a matter of fact, for some of these methods, it is actually claimed that they use the standard MI assumption stated above. i to which they can best be applied. The empirical results presented in this thesis show that they are competitive on standard benchmark datasets. Finally, we explore some p ..."
Abstract
 Add to MetaCart
1As a matter of fact, for some of these methods, it is actually claimed that they use the standard MI assumption stated above. i to which they can best be applied. The empirical results presented in this thesis show that they are competitive on standard benchmark datasets. Finally, we explore some practical applications of MI learning, both existing and new ones. This thesis makes three contributions: a new framework for MI learning, new MI methods based on this framework and experimental results for new applications of MI learning. ii