BYRD R. H., LU P., NOCEDAL J., ZHU C.: A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing 16, 6 (1994), 1190--1208.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....vectors l and u are xed, and the inequalities are taken componentwise. Since the algorithm does not require second derivatives, the method can be applied when the Hessian is not available or not practical to compute. The method in this paper is similar to the ones in Byrd, Lu, Nocedal, and Zhu[7] in that we create a quadratic model function using gradient information in such a way that the storage required in linear in n. The main di erence, however, is that while they use gradients to construct the model function and then minimize the model over a sequence of subspaces, we use projected ....

....will have little e ect upon the step direction. 5 Numerical Experiments We implemented the algorithm in C language and used the PETSc [2, 3, 4] package for linear algebra support. Using a set a benchmark problems we compared its performance to the limited memory variable metric method L BFGS B[7]. The calculations were performed using a Pentium II processor with 512 KB cache and a clock speed of 400MHz and the Linux operating system. The L BFGS B solver also uses a limited memory BFGS matrix to compute step directions. However, it does not use projected gradients to update the matrix. ....

R. H. Byrd, P. Lu, J. Nocedal, and C. Y. Zhu, A limited memory algorithm for bound constrained optimization, SIAM Journal on Scienti c Computing, 16 (1995), pp. 1190-1208.

....Method The implementation of the semismooth Newton method uses BFGS approximations of the Hessian matrix. The resulting semismooth Newton systems have a similar structure as those arising in the step computation of the successful Limited Memory BFGS method L BFGS B by Byrd, Lu, Nocedal and Zhu [25, 148]. Hence, in our implementation we decided to follow the design of L BFGS B (the computations for this chapter were done before we developed our trust region theory in section 6) 9.4.1 Quasi Newton BFGS Approximations In this section, we focus on the use of BFGS approximations in semismooth ....

....of the discrete objective function is approximated by Lim ited Memory BFGS matrices. Hereby, we choose Bo such that it represents a finite difference approximation of the inner product on U. 2. The globalization is similar as in the well accepted L BFGS B method of Byrd, Lu, Nocedal and Zhu [25, 148]: i. At the current point u C B , the objective function j is approximated by a quadratic model qh k ii. Starting from u, a generalized Cauchy point u2 c C B is computed by an Armijo type linesearch for q along the projected gradient path tj ) t O. iii. The semismooth Newton method is ....

R. H. Byrd, P. Lu, J. Nocedal, and C. Y. Zhu, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Cornput., 16 (1995), pp. 1190-1208.

....is discussed, and a derivation of the resulting cost functional is sketched in Section 2. A formula for the gradient of the cost functional is also given in this section, following the analysis in [17] A fast limited memory quasi Newton optimization method allowing bound constraints [1] for the minimization of our particular cost functional is the topic of Section 3. The method is a nonlinear minimization technique which combines low cost with rapid convergence. Only the cost functional and its gradient are computed at each iteration, allowing an efficient approximation to the ....

....simplifies according to the choice of L in (9) 3 Limited Memory Optimization To minimize the reduced cost functional J [OE 1 ; OE T ] given in (13) we apply a limited memory quasiNewton optimization method allowing bound constraints on the phase. The basic algorithm is given in [1] and has been incorporated into an optimization package at the Argonne National Laboratory in a highly efficient code. Quasi Newton methods yield approximations to a (local) minimizer u to the phase screens OE t in (13) of the form u i 1 = u i s i ; i = 0; 1; where s i = d i updates ....

[Article contains additional citation context not shown here]

R.H. Byrd, P. Lu, J. Nocedal and C. Zhu, "A limited memory algorithm for bound constrained optimization, " SIAM J. Scientific Computing, Vol. 21, pp. 1190-1208, 1996.

....of freedom of the control. First the state equation ( Navier Stokes ) is solved and then this solution is used as input to the adjoint equation that is solved next and gives the gradient of the objective function. Optimization is performed with a limited memory quasi Newton method described in Byrd et al. 1994). The resulting control will be optimal for the specific perturbation and time domain studied. In the linear case, optimal (H 2 ) controllers and estimators are developed for the 3D OrrSommerfeld Squire equations at a large array of wavenumber pairs fk x ; k z g, using a technique closely ....

Byrd, R. H. , Peihuang, L. , Nocedal, J. ,and Zhu, C., 1994 "A limited memory algorithm for bound constrained optimization" Technical Report NAM-08 Northwestern University.

....version of the identity matrix. One then updates m times, however without storing the updated matrices explicitly but instead storing the m pairs fl and ffi. Most importantly m is typically very small, say five. The scaling of the initial matrix is also important. Other recent references include Byrd et al. 1993) and Zou et al. 1993) However, it is unclear as to whether the relative success of naive preconditioners, limited memory with small m and naive scaling of the identity matrix are mostly a consequence of the not very extensive testing that has been carried out to date. In particular, most ....

....(1992c) and Conn et al. 1993c) These describe tests using all the LANCELOT options on about one thousand problem instances. The basic conclusions are that LANCELOT appears to be very robust and the symmetric rank one update is the best quasi Newton update in that trust region context (see also Byrd et al. 1993a, who based upon their convergence analysis, recommend updating even when steps are rejected) From the point of view of general comparisons, there is not a great deal of large scale experience 16 in the published literature. Eldersveld et al. 1993) looked at very sparse problems that have ....

R. H. Byrd, P. Lu, and J. Nocedal. A limited memory algorithm for bound constrained optimization. Technical Report NAM-08, Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, Illinois, 1993.

....divided into M = 64 bins, the camera taking S = 64 stops, and divide the region of interest into 64 64 pixels. Then 5N MS for a single head camera, and even 5N 3MS when a triple head camera is used. And in fact, our implementation, based on the limited memory BFGS approach of Nocedal et al. [21, 5] shows that (NL) is a dicult problem subject to instabilities in particular when the eigenvalues tend to coalesce, 1i 2i . Reconstructing a typical slice above may take between 1 and 2 hours CPU, which is too slow for convenient clinical applications. In order to circumvent the numerical ....

Byrd RH, Lu P, Nocedal J, Zhu C, A limited memory algorithm for bound constrained optimization, SIAM J. on Scientic Computing 16, 1996, 1190 - 1208.

....We write the unconstrained optimization problem as min f(x) 1.1) xC n where f is a twice continuously differentiable function whose gradient will be denoted by g. The motivation for this work arose during the development of a limited memory code (LBFGS B) for bound constrained optimization [5], 14] We observed that for some problems this code was unable to reduce the gradient norm Ilg(x)IIoc as much as we desired, but that LANCELOT [7] had no difficulties in doing so. Initially we reported this as a failure of the limited memory code to achieve high accuracy in the solution, but a ....

....in general not directly dependent on the condition number 7, since we can vary 7 why leaving 6 unchanged. Assumption 2 has been made throughout this section to simplify the exposition. We note, however, that (3. 6) can be relaxed without altering the results stated here, as discussed by Forsythe [8, 5]. On the other hand, 3.7) is assumed for convenience and without loss of generality. Figure 2: worst case c 2 = 1 Intervals [ 5] of possible values of c 2, as a function of 5 G [0, 1) 10 4 Maximum Oscillation in the Gradient Norm The following result provides an upper bound on the ....

[Article contains additional citation context not shown here]

R. H. Byrd, P. Lu, J. Nocedal and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5):1190-1208, 1995.

....unconstrained problems. The user must supply the gradient g, but knowledge about the Hessian matrix of f is not required. For this reason the algorithm can be useful for solving large problems in which the Hessian matrix is difficult to compute or is dense. The algorithm is described in detail in [8], and proceeds roughly as follows. At each iteration a limited memory BFGS approximation to the Hessian is updated. This limited memory matrix is used to define a quadratic model of the objective function f . A search direction is then computed using a two stage approach: first, the gradient ....

....are active at the solution, it is appropriate to stop the iteration when the norm of the gradient g is sufficiently small. The corresponding quantity for the case when some bounds are active is the norm of the projected gradient, which we denote by kproj gk, and which is defined, for example, in [8]. Both the output of L BFGS B and its documentation, make reference to the projected gradient. L BFGS B is an extension of the limited memory algorithm (L BFGS) for unconstrained optimization described in [16] and implemented as Harwell routine VA15 [12] The main improvement is the ability of ....

[Article contains additional citation context not shown here]

R. H. Byrd, P. Lu, J. Nocedal and C. Zhu. "A limited memory algorithm for bound constrained optimization" Tech. Report, EECS Department, Northwestern University, 1993, to appear in SIAM Journal on Scientific Computing.

....arising in BFGS updating. There are, in addition, compact representations for the symmetric rank one (SR1) updating formula, which is particularly appealing in the constrained setting because it is not restricted by the positive definiteness requirement. The recently developed code L BFGS B [12] [65] uses a gradient projection approach together with compact limited memory BFGS matrices to solve the bound constrained optimization problem min f(x) subject to l x u: Table 3 illustrates the performance of L BFGS B on bound constrained problems from the CUTE collection. Once more we use ....

R. H. Byrd, P. Lu, J. Nocedal and C. Zhu (1995). A limited memory algorithm for bound constrained optimization, SIAM Journal on Scientific Computing, 16, 5, pp. 1190--1208.

No context found.

R.H. Byrd, P. Lu, J. Nocedal, and C. Y. Zhu, A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16 (1995), pp. 1190--1208.

....write the unconstrained optimization problem as min x2lR n f(x) 1.1) where f is a twice continuously differentiable function whose gradient will be denoted by g. The motivation for this work arose during the development of a limited memory code (LBFGS B) for bound constrained optimization [5], 14] We observed that for some problems this code was unable to reduce the gradient norm kg(x)k 1 as much as we desired, but that LANCELOT [7] had no difficulties in doing so. Initially we reported this as a failure of the limited memory code to achieve high accuracy in the solution, but a ....

R. H. Byrd, P. Lu, J. Nocedal and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing, 16(5):1190--1208, 1995.

....trial values generated in the line search procedure; see for example [13] After x k 1 has been computed, a new Hessian approximation H k 1 is generated based on the differences x k 1 Gamma x k and g k 1 Gamma g k . Some of the nonlinear optimization codes in NEOS, such as TRON [7] and L BFGS [2], already have such internal client server design. In these codes a driver computes the function and gradient values at the current iterate, and then calls the optimization solver, which returns a better estimate of the solution. iNEOS has been designed to exploit this structure in an interactive ....

....f(x) subject to l x u; 3) where l and u are n vectors of bounds. The description of the interactive environment given above applies to (3) provided the vectors l and u are transmitted by the client during the first invocation of the server. The optimization is performed by means of L BFGS B [2, 17], a limited memory quasi Newton method. Solvers for general nonlinear optimization problems (with equality and inequality constraints) will be added in the future. They will require the transmission of second derivatives from the client to the server. 2 Implementation Several technologies can be ....

[Article contains additional citation context not shown here]

R.H. Byrd, P. Lu, J. Nocedal and C. Zhu (1995). "A limited memory algorithm for bound constrained optimization", SIAM Journal on Scientific Computing, 16, 5, pp. 1190-1208.

No context found.

R.H. Byrd, P. Lu, J. Nocedal and C. Zhu (1995). A limited memory algorithm for bound constrained optimization, SIAM Journal on Scientific Computing, 16, 5, pp. 1190--1208.

No context found.

BYRD R. H., LU P., NOCEDAL J., ZHU C.: A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific Computing 16, 6 (1994), 1190--1208.

No context found.

R. H. Byrd, P. Lu, and J. Nocedal. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific and Statistical Computing, 16 no. 5:1190--1208, 1995.

No context found.

Richard H. Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization. Technical Report NAM-08, Northwestern University, Evanston, IL, USA, May 1994.

No context found.

Byrd, R. H., Lu, P., Nocedal, J. and Zhu, C. (1995), "A limited memory algorithm for bound constrained optimization", SIAM J. Scientific Computing, 16, 1190-1208.

No context found.

Byrd, R., Lu, P., Nocedal, J.: A limited memory algorithm for bound constrained optimization. SIAM J Sci Stat Comp 16 (1995) 1190--1208

No context found.

R. H. Byrd, P. Lu, and J. Nocedal. A limited memory algorithm for bound constrained optimization. SIAM Journal on Scientific and Statistical Computing, 16(5):1190--1208, 1995.

No context found.

Byrd R.H., Lu P., Nocedal J. and Zhu C. (1995), A limited memory algorithm for bound constrained optimization, SIAM J.Scientific Computing, 16, 1190--1208

No context found.

Byrd, R. H., Lu, P., Nocedal, J., and Zhu, C. (1995). A limited memory algorithm for bound constrained optimization. SIAM Journal on Scienti c Computing, 16:1190-1208.

No context found.

R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu, A limited memory algorithm for bound constrained optimization, Technical report NAM-08, Northwestern University, Department of Electrical Engineering and Computer Science, 1994.

No context found.

R. H. Byrd, P. Lu, J. Nocedal, and C. Zhu. A limited memory algorithm for bound constrained optimization. SIAM J. Scientific Computing, 16(5):1190--1208, 1995.

No context found.

Richard H. byrd & Peihuang Lu, A limited memory algorithm for bound constrained optimization, Technical report NAM-08, Northwestern University.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC