P. H. Calamai and J. J. More, Projected gradient methods for linearly constrained problems, Math. Programming 39 (1987) pp. 93-116.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....(x) 0 for all x 2 R . b) For all x; y 2 R , k P (y) Gamma P (x) k k y Gamma x k. c) If T (x) is the tangent cone of Omega at x, then minf (g(x) v) j v 2 T (x) k v k 1 g = Gamma k P (g(x) k : The proofs of (a) and (b) may be found in [8] and the proof of (c) may be found in [5] (there is a difference in sign between P (g(x) here and the quantity r Omega f(x) as it is defined in the latter reference) We first prove (6) If x 2 Omega Gamma then P (x) x and so from (b) k q(x) k = k P (x Gamma g(x) Gamma P (x) k k (x Gamma g(x) Gamma x k = k g(x) k ; which ....

....will require a sufficiently rich set of directions in the pattern so that at any face of the feasible region, one will have directions both normal and tangent to the constraints. This we will pursue elsewhere. One issue we have not discussed is that of identifying active constraints, as in [4, 5]. One would wish to show that if the sequence fx k g converges to a nondegenerate 17 stationary point x , then in a finite number of iterations the iterates x k land on the constraints active at x and remain thereafter on those constraints. There are three difficulties in proving such a result ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....the compact representations of limited memory matrices described by Byrd, Nocedal and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16] 17] 3] to determine the active set, because recent studies [7], 5] indicate that it possess good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the Hessian ....

....This inequality implies that g d k 0 if B k is positive definite and d k is not zero. In this paper we do not present any convergence analysis or study the possibility of zigzagging. However, given the use of gradient projection in the step computation we believe analyses similar to those in [7] and [9] should be possible, and that zigzagging should only be a problem in the degenerate case. The Hessian approximations B k used in our algorithm are limited memory BFGS matrices (Nocedal [21] and Byrd, Nocedal and Schnabel [6] Even though these matrices do not take advantage of the ....

P. H. Calamai, and J. J. Mor'e, "Projected gradient methods for linearly constrained problems " Mathematical Programming 39 (1987): 93-116

....use of the compact representations of limited memory matrices described by Byrd, Nocedal and Schnabel [6] the computational cost of one iteration of the algorithm can be kepttobeofordern. We used the gradient projection approach[16] 17] 3] to determine the active set, because recent studies [7], 5] indicate that it possess good theoretical properties, and because it also appears to be efficientonmany large problems [8] 20] However some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the Hessian ....

....k : This inequality implies that g k d k 0ifB k is positive definite and d k is not zero. In this paper we do not presentanyconvergence analysis or study the possibility of zigzagging. However, given the use of gradient projection in the step computation webelieve analyses similar to those in [7] and [9] should be possible, and that zigzagging should only be a problem in the degenerate case. The Hessian approximations B k used in our algorithm are limited memory BFGS matrices (a cedal [21] and Byrd, Nocedal and Schnabel [6] Even though these matrices do not take advantage of the ....

P. H. Calamai, and J. J. Mor'e, "Projected gradient methods for linearly constrained problems " Mathematical Programming 39(" 93-116

....t u i and consider the projection of x 2 R onto the feasible region of problem (2) Q(x) p i (x i )e i ; where e i is the ith standard coordinate vector. Note that x is a stationary point of (2) if and only if q(x) Q(x g(x) x = 0, where g(x) is the gradient of f at x (e.g. see [15, 16]) In the bound constrained theory, the quantity q(x) plays the role of the gradient g(x) providing a continuous measure of how close x is to a constrained stationary point. If g(x) 0 such that the constraints are not active, then clearly q(x) Q(x g(x) x = Q(x) x = 0. Otherwise, q(x) is a ....

P. H. Calamai and J. More, \Projected gradient methods for linearly constrained problems," Mathematical Programming, vol. 39, pp. 93-116, 1987.

....several useful properties of the projected gradient path. Lemma 6.21. t . Then for all (0, 1] and all s 1 holds II( 11u II)11u 11( 11u, 6.26) g, s 711( 11 P( II( IIu P( 6.27) Proo e first inequality in (6.26) is well known, see, e.g. 136, Lem. 2] The second inequaliW is proved in [27]. For (6.27) we use that (Pr(v) v, Pr(v) v 2 0 V , v U, 6.28) since w Pr(v) minimizes IIw vll on . we set va(t) ua tga and derive (9, P( II(z)11 ( z) P( z) P( z) I1)11 , where we have used (6.28) in the last step. From X P (a) a (1) and (6.26) ....

P. H. Calamai and J. J. Mor6, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), pp. 93-116.

....of Evolutionary Pattern Search and consider the projection of x # R n onto the feasible region of problem (2) Q(x) n # i=1 p i (x i )e i , where e i is the ith standard basis vector. Note that x is a stationary point of (2) if and only if q(x) Q(x g(x) x = 0 (e.g. see Calamai and Mor e (1987) and Conn et al. 1988) In the bound constrained theory, the quantity q(x) plays the role of the gradient g(x) providing a continuous measure of how close x is to a constrained stationary point. If g(x) 0 such that the constraints are not active, then clearly q(x) Q(x g(x) x = ....

Calamai, P. H. and Mor e, J. J. (1987). Projected gradient methods for linearly constrained problems.

....each iteration while condition (3.23) is needed to guarantee that every step taken is non negligible. Mor e shows that it is always possible to pick such a value of t C(k;j) using a backtracking linesearch, starting on or near to the trust region boundary. Similar methods have been proposed by Calamai and Mor e (1987), Burke and Mor e (1988) Toint (1988) and Burke et al. 1990) 2. Secondly, we pick p (k;j) so that x (k;j) p (k;j) lies within (2.6) kp (k;j) k t fi 2 Delta (k;j) and m (k;j) x (k;j) Gamma m (k;j) x (k;j) p (k;j) fi 3 [m (k;j) x (k;j) Gamma m (k;j) ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....in our analysis. Here the constraint set X can be any nonempty closed convex set. Lemma 2.5 [34] The projection operator #X ( satisfies (i) for any x # X, # X (z) z] T [# X (z) x] # 0 for all z # R n ; ii) ##X (y) #X (z)# # #y z# for all y, z # R n . Lemma 2. 6 [16, 5] Given x # R n and d # R n , the function # defined by #(#) ##X (x #d) x# #, # 0 is antitone (nonincreasing) Lemma 2.6 actually implies that if x # X is stationary point of (1.14) then dG (#) #X [x #dG ] x = 0 # # # 0. 3 Properties of Search Directions In ....

P.H. Calamai and J.J. More, "Projected gradient methods for linearly constrained problems", Mathematical Programming, 39 (1987), 93--116.

....the algorithm searches for a point x C k by following a partial projected gradient direction. We called rf(x k ) a partial gradient direction since it contains only components of rf(x k ) which are in the working set of x k . For a standard projected gradient method (e.g. Calamai and Mor e [4], and Bertsekas [2] x C k P (x k rf(x k ) where P is the projection into and x C k is referred as the Cauchy point. The sucient decrease of function values at Cauchy points is guaranteed by conditions (9) and (10) and is used for the convergence proof. Then by requiring f(x k 1 ) f(x ....

....Proof Based on the analysis on special structure of SVM formulations in Section II, in this section we will provide the convergence proof. As the only di erence between P k and P is the restriction on the domain variable x such that x i = x k ) i ; i = 2 B k , following from Calamai and Mor e [4], we have Lemma III.1: Let P k be the projection into k : a) If z 2 k , then (P k (x) x) T (z P k (x) 0; 8x 2 R n k : 18) b) P k is a monotone operator, that is (P k (y) P k (x) T (y x) 0; 8x; y 2 R n k : 19) If P k (y) 6= P k (x) then strict inequality holds. c) Given x 2 ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. More. Projected gradient methods for linearly constrained problems. Math. Programming, 39:93-116, 1987.

....for instance, if the objective is believed to be untrustworthy in its accuracy, or if f(x) is not available as a numerical value and only comparison of objective values is possible. 7.2. Identifying active constraints. Another practical issue is that of identifying active constraints, as in [2, 3, 4]. A desirable feature of an algorithm for linearly constrained minimization is the identification of active constraints in a finite number of iterations, that is, if the sequence x k converges to a stationary point x # , then in a finite number of iterations the iterates x k land on the ....

....on those constraints. As discussed in [8] for the case of bound constraints, there are several impediments to proving such results for pattern search algorithms and showing that ultimately the iterates will land on the active constraints and remain there. For algorithms such as those considered in [2, 3, 4], this is not a problem because the explicit use of the gradient impels the iterates to do so in the neighborhood of a constrained stationary point. However, pattern search methods do not have this information, and at this point it is not clear how to avoid the possibility that these algorithms ....

P. H. Calamai and J. J. Mor e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....by Goldstein [16] and Levitin and Polyak [19] We also apply the new ingredients to the feasible continuous projected path that will be properly defined in Section 2. The convergence properties of the projected gradient method for different choices of stepsize have been extensively studied. See [2, 3, 7, 11, 16, 19, 22, 30], and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Mor e [7] For a complete survey see Dunn [12] In Section 2 of this paper we define the spectral projected gradient algorithms and 2 we prove ....

....of the projected gradient method for different choices of stepsize have been extensively studied. See [2, 3, 7, 11, 16, 19, 22, 30] and other authors. For an interesting review of the different convergence results that have been obtained under different assumptions, see Calamai and Mor e [7]. For a complete survey see Dunn [12] In Section 2 of this paper we define the spectral projected gradient algorithms and 2 we prove global convergence results. In Section 3 we present numerical experiments. This set of experiments show that, in fact, the spectral choice of the steplength ....

P. H. Calamai and J. J. Mor'e [1987], Projected gradient methods for linearly constrained problems, Mathematical Programming 39, pp. 93--116.

....(1.4) within (1.17) Indeed, as the condition P (x; r x 9(x; k) s (k) 0 (3:14) is required at optimality for such a problem, 3.5) can be viewed as an inexact stopping rule for iterative algorithms for solving it. We merely mention here that the projected gradient methods of Calamai and Mor e (1987), Burke and Mor e (1988) Conn et al. 1988a) Conn et al. 1988b) and Burke et al. 1990) and the interior point method of Nash and Sofer (1991) are all appropriate, but that methods which take special account of the nature of (1.4) may yet be prefered. 3.4 Further discussion We should also ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....3. The Step 3 of the algorithm searches for a point x C k by following a partial projected gradient direction. We called rf(x k ) a partial gradient direction since it contains only components of rf(x k ) which are in the working set of x k . For a standard projected gradient method (e.g. Calamai and Mor e [ 1987 ] x C k j P (x k Gamma ffrf(x k ) where P is the projection into Omega Gamma and x C k is referred as the Cauchy point. The sufficient decrease of function values at Cauchy points is guaranteed by conditions (2.5) and (2.6) and is used for the convergence proof. Then by requiring f(x k 1 ....

....the fact that OP (x k ) OP (x k ) and Lemma 2.2, we have Lemma 2.4 OP (x k ) 0 if and only if x k is a stationary point. 3 The Convergence Proof As the only difference between P k and P is the restriction on the domain variable x such that x i = x k ) i ; i = 2 B k , following from Calamai and Mor e [ 1987 ] we have Lemma 3.1 Let P k be the projection into Omega k : a) If z 2 Omega k , then (P k (x) Gamma x) T (z Gamma P k (x) 0; 8x 2 R n k : 3.1) b) P k is a monotone operator, that is (P k (y) Gamma P k (x) T (y Gamma x) 0; 8x; y 2 R n k : 3.2) If P k (y) 6= P k (x) ....

[Article contains additional citation context not shown here]

P. H. Calamaiand J. J. Mor'e. Projected gradient methods for linearly constrained problems. Math. Programming, 39:93--116, 1987.

....and successful history as tools for the solution of nonlinear, nonconvex, optimization problems. They have been studied and applied to unconstrained problems (see [6] 16] 24] 28] 29] 30] 31] 33] 34] 38] and to problems involving various classes of constraints: simple bounds ([5], 10] 11] 27] 32] convex constraints ( 1] 2] 9] 41] and also nonconvex ones ( 4] 7] 15] 35] 42] This long lasting interest is probably justified by the attractive combination of a solid convergence theory, a noted algorithmic robustness, the existence of numerically ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....each iteration while condition (3.23) is needed to guarantee that every step taken is non negligible. Mor e shows that it is always possible to pick such a value of t C(k;j) using a backtracking linesearch, starting on or near to the trust region boundary. Similar methods have been proposed by Calamai and Mor e (1987), Burke and Mor e (1988) Toint (1988) and Burke et al. 1990) 2. Secondly, we pick p (k;j) so that x (k;j) p (k;j) lies within (2.6) kp (k;j) k t fi 2 1 (k;j) and m (k;j) x (k;j) 0m (k;j) x (k;j) p (k;j) fi 3 [m (k;j) x (k;j) 0m (k;j) x (k;j) p ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....and local convergence properties of Algorithm 2.1. If, however, the test (4) or its modification just described) is not satisfied either because the search direction d k is a bad direction or because we were not able to compute d k in Step (S. 3) we switch to a projected gradient step, cf. [10, 3, 2] for further details. Note that the projections onto [l; u] can be carried out trivially in our situation. However, the projected gradient step is there not only as a safeguard. It is also used in order to change the active set A k . In fact, this set can change rapidly when using a projected ....

P.H. Calamai and J.J. Mor' e: Projected gradient methods for linearly constrained problems. Mathematical Programming 39, 1987, pp. 93--116.

....P (x) 0 for all x 2 R n . b) For all x; y 2 R n , k P (y) Gamma P (x) k k y Gamma x k. c) If T (x) is the tangent cone of Omega at x, then min f (g(x) v) j v 2 T (x) k v k 1 g = Gamma k P (g(x) k : The proofs of (a) and (b) may be found in [8] and the proof of (c) may be found in [5] (there is a difference in sign between P (g(x) here and the quantity r Omega f(x) as it is defined in the latter reference) We first prove (6) If x 2 Omega Gamma then P (x) x and so from (b) k q(x) k = k P (x Gamma g(x) Gamma P (x) k k (x Gamma g(x) Gamma x k = k g(x) k ; ....

....will require a sufficiently rich set of directions in the pattern so that at any face of the feasible region, one will have directions both normal and tangent to the constraints. This we will pursue elsewhere. One issue we have not discussed is that of identifying active constraints, as in [4, 5]. One would wish to show that if the sequence fx k g converges to a nondegenerate stationary point x , then in a finite number of iterations the iterates x k land on the constraints active at x and remain thereafter on those constraints. There are three difficulties in proving such a result for ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....for instance, if the objective is believed to be untrustworthy in its accuracy, or if f(x) is not available as a numerical value and only comparison of objective values is possible. 7.2. Identifying active constraints. Another practical issue is that of identifying active constraints, as in [2, 3, 4]. A desirable feature of an algorithm for linearly constrained minimization is the identification of active constraints in a finite number of iterations, that is, if the sequence x k converges to a stationary point x # , then in a finite number of iterations the iterates x k land on the ....

....on those constraints. As discussed in [8] for the case of bound constraints, there are several impediments to proving such results for pattern search algorithms and showing that ultimately the iterates will land on the active constraints and remain there. For algorithms such as those considered in [2, 3, 4], this is not a problem because the explicit use of the gradient impels the iterates to do so in the neighborhood of a constrained stationary point. However, pattern search methods do not have this information, and at this point it is not clear how to avoid the possibility that these algorithms ....

P. H. Calamai and J. J. Mor e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....x, the quantity q(x) P (x g(x) x. In the bound constrained theory the quantity q(x) plays the role of g(x) in the unconstrained theory, giving us a continuous measure of how close we are to constrained stationarity, as in the theory for methods based explicitly on derivatives (e.g. [5, 6, 8]) The following proposition summarizes two results concerning q that we will shortly need, particularly the fact that x is a constrained stationary point for (1.1) if and only if q(x) 0. While stated for the particular domain## the proposition holds for any closed convex domain. The results are ....

....imposes few additional requirements and as we have seen in 6, the classical pattern search methods for unconstrained minimization or straightforward variants thereof carry over to the bound constrained case. One issue we have not discussed is that of identifying active constraints, as in [4, 5]. One would wish to show that if the sequence x k converges to a nondegenerate stationary point x # , then in a finite number of iterations the iterates x k land on the constraints active at x # and remain thereafter on those constraints. There are three di#culties in proving such a ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. Mor e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....variation of the gradient projection method. This type of rule has thereafter been studied by Gafni and Bertsekas [11] and by Dunn (see [10] for instance) Bertsekas [3] also analyzed projected Newton methods using Armijo type rules for the problem minff(x)jx 0g. More recently, Calamai and Mor e [4] generalized the Armijo type rule proposed by Bertsekas [1] and considered a gradient projection method that selects the step size a k using conditions on the model s decrease and the steplength that are in the spirit of Goldstein s rules (see [2] for instance) Trust region methods for nonlinear ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....the compact representations of limited memory matrices described by Byrd, Nocedal and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16] 17] 3] to determine the active set, because recent studies [7], 5] indicate that it possess good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the Hessian ....

....inequality implies that g T k d k 0 if B k is positive definite and d k is not zero. In this paper we do not present any convergence analysis or study the possibility of zigzagging. However, given the use of gradient projection in the step computation we believe analyses similar to those in [7] and [9] should be possible, and that zigzagging should only be a problem in the degenerate case. The Hessian approximations B k used in our algorithm are limited memory BFGS matrices (Nocedal [21] and Byrd, Nocedal and Schnabel [6] Even though these matrices do not take advantage of the ....

P. H. Calamai, and J. J. Mor'e, "Projected gradient methods for linearly constrained problems " Mathematical Programming 39 (1987): 93-116

....z k 1 (ff) B (z k Gamma ffrf(z k ) satisfies f(z k ) Gamma f(z k 1 (ff) oerf(z k ) z k Gamma z k 1 (ff) 8) The scalars ff max 0, and oe 2 (0; 0:5) are fixed. Conditions under which this method identifies the active set after a finite number of iterations are given in [10, 11]. Since there is potential for a large change in the current active set as a result of a single step CRASH TECHNIQUES 5 in this method, these methods are used in large scale codes to determine an active set quickly, after which a Newton method is applied on the identified face [13] Note that ....

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....function g fffi . We rst restate a result from Yamashita et al. 19] Lemma 2.1 Let 0 . Then 0 2 kx 0 y fi (x)k 2 g fffi (x) 0 2 kx 0 y ff (x)k 2 for all x 2 IR n . The following result follows immediately from the de nition of y and Lemma 2. 2 in Calamai and Mor e [1] (see also [9, 15] SOLVING BOX CONSTRAINED VARIATIONAL INEQUALITIES 4 Lemma 2.2 Let 0 ff . Then ffkx y ff (x)k kx y (x)k for all x 2 IR n . We are now able to state and prove the main result of this section. We note that this result has essentially been established by Peng and ....

P.H. Calamai and J.J. More, \Projected gradient methods for linearly constrained problems ", Mathematical Programming 39 93-116 (1987).

....the compact representations of limited memory matrices described by Byrd, Nocedal, and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16] 18] 3] to determine the active set, because recent studies [7], 5] indicate that it possesses good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However, some of the main components of our algorithm could be useful in other frameworks, as long as limited memory matrices are used to approximate the ....

P. H. Calamai, and J. J. Mor'e, "Projected gradient methods for linearly constrained problems " Mathematical Programming 39 (1987): 93-116

....determine a Gauss Newton point is that one must be sure that the limit point of the algorithm is stationary for in each piece of smoothness (orthant) containing that limit point. The second key idea, motivated by the characterizations above, is to apply variants of the projected gradient method [2] simultaneously to a single cell and n rays, to reduce . This means that the work performed by the projected gradient algorithm at each step of the Gauss Newton method is comparable to performing just two projected gradient steps. iii) Our algorithm depends heavily on the projected gradient ....

....Repulsion (NSR) of the Projected Gradient Method Let Omega be a nonempty closed convex set in IR n and OE : Omega IR be C 1 . We are thinking of Omega being an orthant and OE = j Omega . We paraphrase the description of the projected gradient (PG) algorithm given by Calamai and Mor e [2] for the problem min x2 Omega OE(x) 6) For any ff 0, the first order necessary condition for x to be a local minimizer of this problem is that Omega (x Gamma ffrOE(x) x: When x k 2 Omega is nonstationary, a step length ff k 0 is chosen by searching the path x k (ff) def = ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. Mor'e, Projected gradient methods for linearly constrained problems, Mathematical Programming 39 (1987) 93--116.

....reputation on the basis of their remarkable numerical reliability in conjunction with a sound and complete convergence theory. They have been intensively studied and applied to unconstrained problems (see for instance [11] 14] and [15] and also to problems including bound constraints (see [4], 7] 12] convex constraints (see [2] 6] 18] and non convex ones (see [3] 5] and [19] for instance) At each iteration of a trust region method, the nonlinear objective function is replaced by a simple model centered on the current iterate. This model is built using first and ....

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93--116.

....trf(x k ) x k Gamma tP (rf(x k ) for 0 t fi 1 , where fi 1 is the first breakpoint of the piecewise linear path, if P (rf(x k ) 6= 0. 3.2 Convergence analysis The transfer of the convergence properties from theorem 2.1 to the bound constrained case follows from theorem 5.4. in [CM87]. Calamai and Mor e refer to an algorithm combining the projected gradient steps with another method satisfying x k 1 2 Omega B , f(x k 1 ) f(x k ) and A(x k 1 ) ae A(x k ) Theorem 5.4. in [CM87] shows that a bounded sequence generated by this algorithm stays on the correct active ....

....properties from theorem 2.1 to the bound constrained case follows from theorem 5.4. in [CM87] Calamai and Mor e refer to an algorithm combining the projected gradient steps with another method satisfying x k 1 2 Omega B , f(x k 1 ) f(x k ) and A(x k 1 ) ae A(x k ) Theorem 5.4. in [CM87] shows that a bounded sequence generated by this algorithm stays on the correct active set after a finite number of steps, whenever the strict complementary condition (60) holds in each stationary point. Theorem 3.1 Let f 2 C 3 (D) D open, convex and nonempty, x 0 2 D and L f (f(x 0 ) ....

[Article contains additional citation context not shown here]

P.H. Calamai and J.J. Mor'e. Projected gradient methods for linearly constrained problems. Math. Programming, 39:93--116, 1987.

....a result from Yamashita et al. 19] Lemma 2.1 Let 0 ff fi. Then fi Gamma ff 2 kx Gamma y fi (x)k 2 g fffi (x) fi Gamma ff 2 kx Gamma y ff (x)k 2 for all x 2 IR n . The following result follows immediately from the definition of y fl and Lemma 2. 2 in Calamai and Mor e [1] (see also [9, 15] Lemma 2.2 Let 0 ff fi. Then ffkx Gamma y ff (x)k fikx Gamma y fi (x)k for all x 2 IR n . We are now able to state and prove the main result of this section. We note that this result has essentially been established by Peng and Fukushima [15] recently. However, ....

P.H. Calamai and J.J. Mor'e, "Projected gradient methods for linearly constrained problems ", Mathematical Programming 39 93--116 (1987).

....history as tools for the solution of nonlinear, nonconvex, optimization problems. They have been studied and applied to unconstrained problems (see [7] 17] 25] 28] 29] 30] 31] 34] 35] 38] and to problems involving various classes of constraints, including simple bounds ([6], 10] 11] 27] 32] convex constraints ( 2] 3] 14] 41] and nonconvex ones ( 5] 8] 16] 36] 44] This long lasting interest is probably justified by the attractive combination of a solid convergence theory, a noted algorithmic robustness, the existence of numerically ....

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

....appropriate. Indeed, since min ##f(x) v# : v # T (x) #v# # 1 = ### f(x)#, 2.7) it might be more appropriate to call ## f(x) the projected steepest descent direction. The optimality property (2. 7) follows from the properties of the projection on convex cones; Calamai and More [8] provide a direct proof of (2.7) LARGE BOUND CONSTRAINED OPTIMIZATION 1105 The projected gradient should not be confused with the reduced gradient. When# is the bound constrained set (1.2) the reduced gradient is the vector with components # i f(x) if l i x i u i , while for the ....

....need only an expression for ### f(x)#. The projected gradient ## f can be used to characterize stationary points because if# is a convex set, then x ## is a stationary point of problem (1.1) if and only if ## f(x) 0. In general, ## f is discontinuous, but as proved by Calamai and More [8], if f : R n # R is continuously di#erentiable on ## then the mapping x ## ### f(x)# is lower semicontinuous on ## This property implies that if x k is a sequence in# that converges to x # , and if ## f(x k ) converges to zero, then x # is a stationary point of problem (1.1) In ....

P. H. Calamai and J. J. Mor e, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), pp. 93--116.

....Indeed, since min fhrf(x) vi : v 2 T (x) kvk 1g = Gammakr Omega f (x)k; 2.7) it might be more appropriate to call r Omega f(x) the projected steepest descent direction. The optimality property (2. 7) follows from the properties of the projection on convex cones; Calamai and Mor e [8] provide a direct proof of (2.7) The projected gradient should not be confused with the reduced gradient. When Omega is the bound constrained set (1.2) the reduced gradient is the vector with components i f(x) if l i x i u i , while for the projected gradient Gamma[r Omega f(x) i = ....

....f(x)k. The projected gradient r Omega f can be used to characterize stationary points because if Omega is a convex set, then x 2 Omega is a stationary point of problem (1.1) if and only if r Omega f(x) 0. In general, r Omega f is discontinuous, but as proved by Calamai and Mor e [8], if f : R n R is continuously differentiable on Omega Gamma then the mapping x 7 kr Omega f (x)k is lower semicontinuous on Omega Gamma This property implies that if fx k g is a sequence in Omega that converges to x , and if fr Omega f(x k )g converges to zero, then x is a ....

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), pp. 93--116.

.... f(z ) argmin fjjd rf(z )jj : d 2 T l (z ) l 2 Lg: If, for l 2 L, we let d 0 l (ff l ; fi l ; fl l ) 8 : C T ff l W T fi l when n l 0 C T ff l S T fl l when A = or n l = 0; where A, C, W l , S, ff l , fi l , fl l and n l are defined as before, then (see [7]) this projected gradient can be computed using r Upsilon f(z ) Gammarf (z ) d 0 (ff ; fi ; fl ) where, for l 2 L and ff c l defined as before, ff l ; fi l ; fl l ) is a solution to the linear least squares problem minfjjrf(z ) d 0 l (ff l ; fi l ; ....

P. Calamai and J. Mor'e, Projected gradient methods for linearly constrained problems, Mathematical Programming 39 (1987) 93--116.

....Theorem 4. 1 in terms of a specific representation of Omega Gamma Note, in particular, that if Omega is the polyhedral set defined by the set of linear constraints Omega = fx 2 IR n : hc j ; xi ffi j ; 1 j mg ; 4:3) for some vectors c j 2 IR n and scalars ffi j , then Calamai and Mor e [6] proved that the projected gradient that appears in (4.2) can be expressed in the familiar form P T (x) Gammarf (x) Gammarf (x) X j2A(x) j c j ; 4:4) where the set of active constraints is defined by A(x) j fj : hc j ; xi = ffi j g; and j for j 2 A(x) solves the bound constrained ....

....i 0g ae A(x k ) k k 0 : 4:7) Proof. Theorem 4.4 shows that x 2 E[ Gammarf (x ) fi 2 A(x ) i 0g ae A(x) x 2 Omega : Thus, the result is a direct consequence of Theorem 4.2. Xi This result has immediate applications to several algorithms. For example, Calamai and Mor e [6] show that the gradient projection method generates sequences that satisfy (4.6) Hence, Theorem 4.5 implies that if the sequence fx k g generated by the gradient projection method converges to x , then (4.7) holds. Closely related convergence results for the gradient projection method are due ....

[Article contains additional citation context not shown here]

P. H. Calamai and J. J. Mor' e, Projected gradient methods for linearly constrained problems, Math. Programming, 39 (1987), pp. 93--116.

No context found.

P. H. Calamai and J. J. More, Projected gradient methods for linearly constrained problems, Math. Programming 39 (1987) pp. 93-116.

No context found.

P. H. Calamai and J. J. Mor e, Projected gradient methods for linearly constrained problems, Mathematical Programming, 39 (1987), pp. 93-116.

No context found.

P. H. Calamai and J. J. Mor'e. Projected gradient methods for linearly constrained problems. Mathematical Programming, 39:93--116, 1987.

No context found.

P. H. Calamai and J. J. More , Projected gradient methods for linearly constrained prob-

No context found.

P.H. Calamai and J.J. Mor'e (1987), Projected gradient methods for linearly constrained problems, Mathematical Programming, 39, pp. 93--116.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC