Conn, Andrew R., Gould, Nicholas I. M., and Toint, Philippe L., Testing a Class of Methods for Solving Minimization Problems with Simple Bounds on the Variables, Math. Comp. 50 182 (1988), 399--430.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16] 17] 3] to determine the active set, because recent studies [7] 5] indicate that it possess good theoretical properties, and because it also appears to be efficient on many large problems [8], 20] However some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the Hessian of the objective function. 2. Outline of the algorithm. At the beginning of each iteration, the current iterate x k , the ....

....and stop the iteration when a boundary is encountered or when the residual is small enough. Note that the accuracy of the solution controls the rate of convergence of the algorithm, once the correct active set is identified, and should therefore be chosen with care. We follow Conn, Gould and Toint [8] and stop the conjugate gradient iteration when the residual r of (5.13) satisfies krk min(0:1; q k: We also stop the iteration at a bound when a conjugate gradient step is about to violate a bound, thus guaranteeing that (5.6) is satisfied. The conjugate gradient method is appropriate ....

A. R. Conn, N. I. M. Gould, and PH. L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation. Vol. 50, No 182 (1988): 399-430.

....At each iteration a limited memory BFGS approximation to the Hessian is updated. This limited memory matrix is used to define a quadratic model of the objective function f . A search direction is then computed using a two stage approach: first, the gradient projection method [15] 3] 18] [9] is used to identify a set of active variables, i.e. variables that will be held at their bounds; then the quadratic model is approximately minimized with respect to the free variables. The search direction is defined to be the vector leading from the current iterate to this approximate minimizer. ....

A. R. Conn, N. I. M. Gould, and PH. L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation. Vol. 50, No 182 (1988), pp. 399-430.

....of one iteration of the algorithm can be kepttobeofordern. We used the gradient projection approach[16] 17] 3] to determine the active set, because recent studies [7] 5] indicate that it possess good theoretical properties, and because it also appears to be efficientonmany large problems [8], 20] However some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the Hessian of the objective function. 2. Outline of the algorithm. At the beginning of each iteration, the current iterate x k , the ....

....and stop the iteration when a boundary is encountered or when the residual is small enough. Note that the accuracy of the solution controls the rate of convergence of the algorithm, once the correct active set is identified, and should therefore be chosen with care. We follow Conn, Gould and Toint[8] and stop the conjugate gradient iteration when the residual r of( satisfies krk :1# k: We also stop the iteration at a bound when a conjugate gradient step is about to violate a bound, thus guaranteeing that (at is satisfied. The conjugate gradient method is appropriate here ....

A. R. Conn, N. I. M. Gould, and PH. L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation. Vol. 50, No 182( 1988): 399-430.

....1 otherwise. The same strategy is used to update A (see 6 for more details) We let Ao = O. llloll where the II 112 is used for the subspace method and II IIc for the Steihaug method ( 6] We used twenty different unconstrained nonlinear test problems. All but four are test problems described in [12], but with all the bound constraints removed. The problems EROSENBROCK and EPOWELL are taken from [13] The last two problems, molecule problems MOLE1 and MOLE3, are described in [9, 10] For all problems, the number of variables r is 260. The minimization algorithm terminates when Ilgll2 10 6. ....

....and r2 = v 10 = 10 6. We also impose an upper bound of 600 on the number of iterations. We first report the results of the STIR method using the modified Cholesky factorization. Table 5 lists the number of iterations required for some standard testing problems (for details of these problems see [12]) For all the results in this paper, the number of iterations is the same as the number of objective function evaluations. The problem sizes vary from 100 to 10,000. The results in Table 5 indicate that, for these testing problems at least, the number of iterations increases only slightly, if ....

[Article contains additional citation context not shown here]

Andrew R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50(182):399430, 1988.

....at Sandia National Labs in Livermore, California. CPlant is a cluster of DEC Alpha Miata 433 MHz Processors. For our tests, we used 50 processors dedicated to our sole use. 5.1. Standard Test Problems. We compare APPS and PPS with 8, 16, 24, and 32 processors on six four dimensional test problems [20, 5], shown in Table 5.1. 1 2 3 4 5 6 broyden2a broyden2b chebyquad epowell toint trig vardim Table 5.1 Six standard test problems. Since the function evaluations are extremely fast, we added extra busy work, in the form of solving a 100 101 nonnegative least squares problem) in order to ....

....; e n g. The remaining p 2n directions are generated randomly (with a di erent seed for every run) and normalized to unit length. This construction ensures that D is a positive spanning set. We initialize = 1:0 and tol = 0:001. We start each of these six problems from the standard starting point [20, 5]. Method Process Function Function Init Idle Total ID Evals Breaks Time Time Time APPS 0 237 66 0.17 0.00 24.72 1 266 70 0.02 0.12 22.36 2 302 89 0.02 0.12 24.32 3 274 77 0.02 0.15 22.31 4 270 62 0.02 0.04 24.56 5 282 81 0.02 0.04 24.58 6 273 59 0.02 0.04 24.59 7 276 61 0.02 0.03 24.55 ....

A. R. Conn, N. I. M. Gould, and P. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Mathematics of Computation, 50 (1988), pp. 399-430.

....at Sandia National Labs in Livermore, California. CPlant is a cluster of DEC Alpha Miata 433 MHz Processors. For our tests, we used 50 processors dedicated to our sole use. 5.1. Standard Test Problems. We compare APPS and PPS with 8, 16, 24, and 32 processors on six four dimensional test problems [20, 5], shown in Table 5.1. 1 2 3 4 5 6 broyden2a broyden2b chebyquad epowell toint trig vardim Table 5.1 Six standard test problems. Since the function evaluations are extremely fast, we added extra busy work, in the ASYNCHRONOUS PARALLEL PATTERN SEARCH 15 form of solving a 100 101 nonnegative ....

....; e n g. The remaining p 2n directions are generated randomly (with a di erent seed for every run) and normalized to unit length. This construction ensures that D is a positive spanning set. We initialize = 1:0 and tol = 0:001. We start each of these six problems from the standard starting point [20, 5]. Method Process Function Function Init Idle Total ID Evals Breaks Time Time Time APPS 0 237 66 0.17 0.00 24.72 1 266 70 0.02 0.12 22.36 2 302 89 0.02 0.12 24.32 3 274 77 0.02 0.15 22.31 4 270 62 0.02 0.04 24.56 5 282 81 0.02 0.04 24.58 6 273 59 0.02 0.04 24.59 7 276 61 0.02 0.03 24.55 ....

A. R. Conn, N. I. M. Gould, and P. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Research Report CS{86-45, Faculty of Mathematics, University of Waterloo, Waterloo, Canada, 1986.

....described in the literature. For large scale problems, we use a strategy based on trust regions derived from the inexact Newton approach. See [6] 7 Essentially, this strategy is the one given in [18] for bound constrained minimization and it is also related to the box constrained algorithms of [5]. The strategy is described by the following algorithm. Algorithm 4.1. Assume that Delta min 0; ff 2 (0; 1) are given independently of the iteration k. Define k (x) kF (x k ) J(x k ) x Gamma x k )k 2 , Delta k Delta min . Step 1. Compute an approximate minimizer x of k (x) ....

Conn, A.R., Gould, N.I.M. and Toint, Ph.L. (1989). Testing a class of methods for solving minimization problems with simple bounds on the variables, Mathematics of Computation 50, 399-430.

....and conclusions. We have implemented the TRIP algorithms using MATLAB 4.2a in a Sun (Sparc) workstation. We have used ffi 0 = 1, p = 1, oe k = oe = 0:99995 for all k, ffl 1 = 10 Gamma4 , and ffl = 10 Gamma5 . We have tested the algorithms in a set of problems given in Conn, Gould and Toint [2]. This set of problems is divided in two groups, labeled by U and C (see Table 1) In problems U, the solution lies in the interior of B and therefore these problems correspond to the situation described in Section 2.1. In the cases where the initial point given in [2] is not strictly feasible, we ....

....in Conn, Gould and Toint [2] This set of problems is divided in two groups, labeled by U and C (see Table 1) In problems U, the solution lies in the interior of B and therefore these problems correspond to the situation described in Section 2.1. In the cases where the initial point given in [2] is not strictly feasible, we scale it back into the interior of B according to the rules used in [1] The scheme 2.3 (see Section 3) used to update the trust radius is the following: ffl If r k 10 Gamma4 , reject s k and set ffi k 1 = 0:5kS Gamma1 k s k k. ffl If 10 Gamma4 r k 0:1, ....

[Article contains additional citation context not shown here]

A. R. Conn, N. I. M. Gould, and P. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp., 50 (1988), pp. 399--340.

....a cluster of DEC Alpha Miata 433 MHz Processors. For our tests, we used 50 nodes dedicated to our sole use. 5. 1 Standard Test Problems We compare APPS and PPS with 8, 16, 24,and 32 processors on six four dimensional test problems: broyden2a, broyden2b, chebyquad, epowell, toint trig, and vardim [18, 5]. Since the function evaluations are extremely fast, we added extra busy work in order to slow them down to better simulate the types of objective functions we are interested in. 8 The parameters for APPS and PPS were set as follows. Let n = 4 be the problem dimension, and let p be the number ....

A. R. Conn, N. I. M. Gould, and P. L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Research Report CS{86-45, Faculty of Mathematics, University of Waterloo, Waterloo, Canada, 1986.

....prove the same q superlinear convergence result for the new method. Finally, we discuss the issues and di#culties involved in extending this approach to trust regions methods using updates in the Broyden class, such as the BFGS, SR1, and DFP. 1. Introduction Several recent computational studies (Conn, Gould, and Toint [1988, 1992] Khalfan, Byrd, and Schnabel [1993] have shown that trust region quasi Newton methods using the SR1, PSB, and BFGS updates are e#ective methods for solving the unconstrained optimization problem minimize f(x) x # R n . In addition, the analyses in Powell [1975] and Byrd, Khalfan, ....

A. R. Conn, N. I. M. Gould and Ph. Toint, (1988), Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp.

....f(x) is assumed to be twice continuously differentiable, l and u are given bound vectors in n , and n is the number of variables, which is assumed to be large. Many algorithms have been proposed for solving small to medium sized problems of the form (1.1) 1. 2) for example see [4] and [5]) There are also some algorithms which are available for large scale problems, such as the Lancelot algorithm of Conn, Gould and Toint [6] Recently, The authors were partially supported by the Stat key project Scientific and Engineering Computing and Chinese NNSF grant 19525101 a ....

....12 23 4.61 9 13 2.95 TP17 5000 71 91 21.98 41 49 13.93 40 48 11.71 43 57 19.73 TP20 5000 12 68 4.40 8 12 2.01 7 11 2.08 7 11 1.66 TP21 5000 6 7 3.66 5 7 3.48 3 5 2.35 3 5 2. 26 In order to investigate the behavior of the SLMQN algorithm for very large problems, we choose 10 test problems from [5], where number of variables is enlarged to n = 10000. The termination condition is that the infinity norm of the projected gradient is reduced below 10 Gamma4 , and m is chosen as 2. Numerical results are shown in Table 5. For TP6, TP7, TP10, TP11, TP20 and TP21, CG is better than other three ....

A.R. Conn, N.I.M. Gould and Ph.L. Toint, Testing a class of methods for solving minimization problems with simple bounds on the variables, Math. Comp. 50 (1988), 399-430.

....At each iteration a limited memory BFGS approximation to the Hessian is updated. This limited memory matrix is used to define a quadratic model of the objective function f . A search direction is then computed using a two stage approach: first, the gradient projection method [15] 3] 18] [9] is used to identify a set of active variables, i.e. variables that will be held at their bounds; then the quadratic model is approximately minimized with respect to the free variables. The search direction is defined to be the vector leading from the current iterate to this approximate minimizer. ....

A. R. Conn, N. I. M. Gould, and PH. L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation. Vol. 50, No 182 (1988), pp. 399-430.

....et al. 1992b) and is based upon incorporating the equality constraints via a Lagrangian barrier function whilst handling upper and lower bounds directly. The sequential, approximate minimization of the Lagrangian barrier function is performed in a trust region framework such as that proposed by Conn et al. 1988a) 1 Our aim in this paper is to consider how these two different algorithms mesh together. In particular, we aim to show that ultimately very little work is performed in the iterative sequential minimization algorithm for every iteration of the outer Lagrangian barrier algorithm. This is ....

....(k 1) fl 2) k 1) 0( k 1) ff ; j (k 1) j0 ( k 1) ff j : 3:10) Increase k by one and go to Step 1. End of Algorithm 3.1 Figure 1: Outer iteration algorithm 5 is possible to satisfy the convergence test (3. 5) after a single iteration of the algorithm given in Conn et al. 1988a) The specific inner iteration algorithm we shall consider is given in Figure 2. There are a number of possible ways of choosing fl (k;j) 0 and fl (k;j) 3 in Step 4. The simplest is merely to pick fl (k;j) 0 = fl 0 and fl (k;j) 3 = fl 3 ; other alternatives are discussed by Conn et al. to ....

[Article contains additional citation context not shown here]

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

....boxes the projection is trivial. Such a projected gradient approach was proposed by McCormick (1969) and independently in Bertsekas (1982) and Levitin and Polyak (1966) More recently it has been exploited extensively in the context of large scale optimization by many authors, see for example Conn et al. 1988b) Dembo and Tulowitski (1983) Mor e and Toraldo (1989) and Mor e and Toraldo (1991) As in the unconstrained case, global convergence can be guaranteed, provided one does at least as well as the generalized Cauchy point. One obtains better convergence, and ultimately a satisfactory asymptotic ....

....global convergence can be guaranteed, provided one does at least as well as the generalized Cauchy point. One obtains better convergence, and ultimately a satisfactory asymptotic convergence rate, by further reducing the model function. This is the trust region basis for the kernel algorithm SBMIN (Conn et al. 1988a) of LANCELOT (Conn et al. 1992b) It can be summarized as follows: ffl Find the generalized Cauchy point based upon a local (quadratic) model. ffl Fix activities to those at the generalized Cauchy point. ffl (Approximately) solve the resulting reduced problem whilst maintaining account of ....

[Article contains additional citation context not shown here]

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

.... the number of active constraints (that is the number of variables exactly at one of their bounds) can change very rapidly from one iteration to the next (see [6] 9] 20] 21] 25] 27] The first of these algorithms has shown to be quite efficient on general nonlinear and quadratic problems [7]. Our purpose in the present paper is to specialize the class of algorithms described in [6] 7] 20] and [25] to the particular case of (1) 2) Section 2 will introduce a variant of the algorithm proposed by Lotstedt in [18] and two new methods based on the nonlinear techniques cited above. ....

.... can change very rapidly from one iteration to the next (see [6] 9] 20] 21] 25] 27] The first of these algorithms has shown to be quite efficient on general nonlinear and quadratic problems [7] Our purpose in the present paper is to specialize the class of algorithms described in [6] [7], 20] and [25] to the particular case of (1) 2) Section 2 will introduce a variant of the algorithm proposed by Lotstedt in [18] and two new methods based on the nonlinear techniques cited above. Section 3 is devoted to the presentation of some comparative numerical results involving these ....

[Article contains additional citation context not shown here]

A.R. Conn, N.I.M. Gould and Ph.L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation, vol. 50(182), pp. 399--430, 1988.

....P (x; r x 9(x; k) s (k) 0 (3:14) is required at optimality for such a problem, 3.5) can be viewed as an inexact stopping rule for iterative algorithms for solving it. We merely mention here that the projected gradient methods of Calamai and Mor e (1987) Burke and Mor e (1988) Conn et al. 1988a) Conn et al. 1988b) and Burke et al. 1990) and the interior point method of Nash and Sofer (1991) are all appropriate, but that methods which take special account of the nature of (1.4) may yet be prefered. 3.4 Further discussion We should also comment on the rather peculiar test (3.8) in ....

....(k) s (k) 0 (3:14) is required at optimality for such a problem, 3.5) can be viewed as an inexact stopping rule for iterative algorithms for solving it. We merely mention here that the projected gradient methods of Calamai and Mor e (1987) Burke and Mor e (1988) Conn et al. 1988a) Conn et al. 1988b) and Burke et al. 1990) and the interior point method of Nash and Sofer (1991) are all appropriate, but that methods which take special account of the nature of (1.4) may yet be prefered. 3.4 Further discussion We should also comment on the rather peculiar test (3.8) in Algorithm 3.1. In our ....

[Article contains additional citation context not shown here]

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

....history as tools for the solution of nonlinear, nonconvex, optimization problems. They have been studied and applied to unconstrained problems (see [6] 16] 24] 28] 29] 30] 31] 33] 34] 38] and to problems involving various classes of constraints: simple bounds ( 5] 10] [11], 27] 32] convex constraints ( 1] 2] 9] 41] and also nonconvex ones ( 4] 7] 15] 35] 42] This long lasting interest is probably justified by the attractive combination of a solid convergence theory, a noted algorithmic robustness, the existence of numerically efficient ....

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

....(1992a) and is based upon incorporating the equality constraints via a Lagrangian barrier function whilst handling upper and lower bounds directly. The sequential, approximate minimization of the Lagrangian 1 2 barrier function is performed in a trust region framework such as that proposed by Conn et al. 1988a) Our aim in this paper is to consider how these two different algorithms mesh together. In particular, we aim to show that ultimately very little work is performed in the iterative sequential minimization algorithm for every iteration of the outer Lagrangian barrier algorithm. This is contrary ....

....per outer iteration. More specifically, under certain assumptions, we first show that (3.8) is eventually satisfied at each outer iteration. We then show that, under additional assumptions, it is possible to satisfy the convergence test (3. 5) after a single iteration of the algorithm given in Conn et al. 1988a) The specific inner iteration algorithm we shall consider is given in Figure 2. There are a number of possible ways of choosing fl (k;j) 0 and fl (k;j) 3 in Step 4. The simplest is merely to pick fl (k;j) 0 = fl 0 and fl (k;j) 3 = fl 3 ; other alternatives are discussed by Conn et al. ....

[Article contains additional citation context not shown here]

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

No context found.

A.R. Conn, N.I.M. Gould and Ph.L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation, vol. 50(182), pp. 399--430, 1988.

No context found.

Conn, Andrew R., Gould, Nicholas I. M., and Toint, Philippe L., Testing a Class of Methods for Solving Minimization Problems with Simple Bounds on the Variables, Math. Comp. 50 182 (1988), 399--430.

No context found.

A. R. Conn, N. I. M. Gould, and Ph. L. Toint. Testing a class of methods for solving minimization problems with simple bounds on the variables. Mathematics of Computation, 50:399--430, 1988.

No context found.

Conn, A. R., Gould, N. I. M. and T oint, P. L. #1988b#. Testing a class of methods for solving minimization problems with simple bounds on the variables, Mathematics of Computation 50: 399#430.

No context found.

A. R. Conn, N.I.M. Gould, and PH. L. Toint (1988b), "Testing a class of methods for solving minimization problems with simple bounds on the variables", Mathematics of Computation 50, 399--430.

No context found.

A.R. Conn, N.I.M. Gould, and Ph.L. Toint, "Testing a class of methods for solving minimization problems with simple bounds on the variables," Mathematics of Computation 50/182 (1988) 399--430.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC