A. A. Goldstein. Convex programming in Hilbert space. Bull. Amer. Math. Soc. 70 (1964): 709--710.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....method, as will be discussed later on. We find that by making use of the compact representations of limited memory matrices described by Byrd, Nocedal and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16], 17] 3] to determine the active set, because recent studies [7] 5] indicate that it possess good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However some of the main components of our algorithm could be useful in other frameworks, as ....

A. A. Goldstein, "Convex programming in Hilbert space", Bull. Amer. Math. Soc. 70 (1964): 709-710.

....k Gamma for some 0) These results extend the early work of [25] 31] and [39, 40] where certain form of strong convexity is assumed. Our analysis is fairly general as it is applicable to a wide class of iterative descent algorithms, including a gradient projection algorithm of Goldstein [11] and Levitin and Polyak [2] a certain matrix splitting algorithm ( 33, 36] coordinate descent methods (see [1, 5, 7, 22, 35, 42] the extragradient method of Korpelevich [16] the proximal minimization algorithm of Martinet [34] The rest of the paper is organized as follows. Section 2 ....

....k 1 k; for some 1 0: 3.11) and f(x k 1 ) Gamma f(x k ) Gamma 2 kx k Gamma x k 1 k 2 for some 2 0; 3.12) The condition (3.12) ensures sufficient descent at each iteration. The above scheme (3. 10) is a broad class that includes (a) a gradient projection algorithm of Goldstein [11] and Levitin and Polyak [2] b) a certain matrix splitting algorithm ( 33, 36] coordinate descent methods (see [1, 5, 7, 22, 35, 42] c) the extragradient method of Korpelevich [16] d) the proximal minimization algorithm of Martinet [34] The reference [28] provides a detailed justification ....

A.A. Goldstein, "Convex programming in Hilbert space," Bull. Am. Math. Soc., 70 (1964), 709--710.

....some neural networks for solving linear, quadratic and nonlinear convex programming problems, which are based on KKT system for optimization problems. Their models correspond to some variants of the projective gradient method for the complementarity problem and the variational inequality problem [7, 23, 8, 9, 10, 11]. It should be noted that for Lagrangian conditions with the nonnegative Lagrange multiplier, if the Lagrange multiplier is penalized or transformed into unrestricted case, the resulting neural network model belongs to the first class. On the other hand, if the nonnegative Lagrange multiplier is ....

A. A. Goldstein, "Convex programming in Hilbert space", Bulletin of American Mathematical Society, vol. 70, pp. 709-710, 1964.

....is that both f 1 ; f p and their gradients may be expensive to evaluate (as in the case of neural network learning) so neither second order methods nor line search can be practically applied. A simple gradient scheme for the preceding problem is the gradient projection method of Goldstein [Gol64] and Levitin and Polyak [LeP65] which updates x according to x : x 0 ff p X j=1 rf j (x) where ff is a positive stepsize and [1] denotes the orthogonal projection operator onto X . It is well known that if X is either a box (i.e. the Cartesian product of closed intervals) or an ....

A. A. Goldstein, Convex programming in Hilbert space, Bull. Am. Math. Soc., 70 (1964), 709--710.

....can be represented by the formula x new : x Gamma jrf(x) e(x) 3.1.3) where j is a positive scalar, and mapping e : n n is the defining feature of each particular algorithm (see Section 3. 3) This is a rather general framework that includes a gradient projection algorithm [15, 27]; proximal minimization algorithm [44, 55] and the extragradient algorithm [25, 43] among others. We note, in the passing, that for characteristic mappings e( Delta) of feasible descent methods, e(x i ) 0 as i 1 by algorithm construction [31] In this chapter, we are concerned with the ....

....point x 2 X s such that kx Gamma xk ffl; where is as specified in (3.1.7) 3.3 Applications In this Section, we briefly discuss applications of our analysis to a number of well known algorithms. 3.3. 1 Gradient Projection Algorithm We first consider the gradient projection algorithm [15, 27]. In the presence of perturbations, it takes the following form x i 1 = x i Gamma j i rf(x i ) ffi(x i ) Obviously, this method is a special case of Algorithm 3.2.1 corresponding to e(x) 0 8x 2 X: 77 Consequently, we can take c 1 = 0, and c 2 = 0 in (3.2.1) 3.2.3) ....

A.A. Goldstein. Convex programming in Hilbert space. Bull. Am. Math. Soc., 70:709--710, 1964.

....) Gamma1 rf(x t ) where P Q t X [ Delta] denotes the projection onto X with respect to the norm kzk Q t : p z T Q t z. Among the algorithms in this class we find the gradient projection algorithm (Q t : I n ) and Newton s method (Q t : r 2 f(x t ) fl t : 1) See [19, 27, 50, 5]. A first order Taylor expansion of f is obtained from choosing t (y) 0; this results in T t (y) rf(x t ) T y (if u = 0 is still assumed) which is the subproblem objective in the Frank Wolfe algorithm ( 17, 5] We next provide the first example of the very useful fact that ....

Goldstein, A. A.: `Convex programming in Hilbert space', Bulletin of the American Mathematical Society 70 (1964), 709--710.

....to introduce both belong to the class of algorithms that use information from the projected gradient to guess the set of bounds that are active at the solution. Such algorithms have been analyzed in the context of general nonlinear optimization by many authors : originally introduced by Goldstein [12], Levitin and Polyak [17] this class of algorithms has enjoyed continued development (see [1] and [10] for example) More recently, the concept of guessing the set of active bounds using a projected gradient step has been incorporated in efficient trust region algorithms by Conn, Gould and Toint ....

A.A. Goldstein, "Convex programming in Hilbert space", Bull. Amer. Math. Soc., vol. 70, pp. 709--710, 1964.

....method, as will be discussed later on. We find that by making use of the compact representations of limited memory matrices described by Byrd, Nocedal and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16], 17] 3] to determine the active set, because recent studies [7] 5] indicate that it possess good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However some of the main components of our algorithm could be useful in other frameworks, as ....

A. A. Goldstein, "Convex programming in Hilbert space", Bull. Amer. Math. Soc. 70 (1964): 709-710.

....U: 1.1) If P : H U denotes the projection onto the feasible set, then the gradient projection method iterates are given by u = P(u c Gamma ff c rOE(u c ) 1.2) where ff 0 is determined by a step size rule. In Hilbert space, this algorithm has been formulated and investigated by Goldstein [10] and Levitin and Polyak [14] The books [4] and [3] discussed the convergence properties of gradient projection methods. In [7] a thorough convergence analysis of the gradient projection method was presented which yields various convergence rates of the algorithm under various assumptions. Since ....

A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), pp. 709--710.

....method, as will be discussed later on. We find that by making use of the compact representations of limited memory matrices described by Byrd, Nocedal, and Schnabel [6] the computational cost of one iteration of the algorithm can be kept to be of order n. We used the gradient projection approach [16], 18] 3] to determine the active set, because recent studies [7] 5] indicate that it possesses good theoretical properties, and because it also appears to be efficient on many large problems [8] 20] However, some of the main components of our algorithm could be useful in other frameworks, ....

A. A. Goldstein, "Convex programming in Hilbert space," Bull. Amer. Math. Soc. 70 (1964): 709--710.

....j1 Gamma flM jg ; 4.40) where m and M are the smallest and largest eigenvalue of r 2 f(x ) respectively. c) Let further u j ffi X , where X is a nonempty, closed and convex set in n . Then the algorithm is equivalent to the Goldstein Levitin Polyak gradient projection algorithm ([43, 58]) Moreover, the convergence criterion (4.27) and the linear convergence ratio (4.40) coincide with those of this algorithm. d) Assume further that X = n . Then the algorithm is equivalent to the steepest descent method with fixed step length fl ( 84] with the corresponding convergence ....

....minimized over x 2 X. If the matrix B k is positive definite on X, y k is given by y k = P Bk X (x k Gamma fl k B Gamma1 k rf(x k ) This is the subproblem of the class of (deflected) gradient projection methods, including, as special cases, the gradient projection method (B k j I) [43, 58]) and Newton s method (B k = r 2 f(x k ) fl k = 1, for all k) 58, 90, 31] The framework contains a class of regularization methods. Let k (x) f(x) 1= 2fl k )D k (x) where fl k 0 and where the function D k is convex and in C 1 on X. Furthermore, if we assume that rD k (x ....

[Article contains additional citation context not shown here]

A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), pp. 709--710.

....can be represented by the formula x new : x Gamma jrf(x) e(x; j) 1. 3) where j is a positive scalar, and mapping e : n 1 n is the defining feature of each particular algorithm (see Section 3) This is a rather general framework that includes a gradient projection algorithm [6, 8]; proximal minimization algorithm [22, 28] the extragradient algorithm [7, 21] and the incremental gradient algorithms [31] among others. We note, in the passing, that (in the noise free case) the characteristic mappings e( Delta; Delta) of classical feasible descent methods satisfy the ....

.... Gamma xk d 1 ffl; where is as specified in (1.7) and d 1 is given in Theorem 2.1. 3 Applications In this Section, we briefly discuss applications of our analysis to a number of well known algorithms. 3. 1 Gradient Projection Algorithm We first consider the gradient projection algorithm [6, 8]. In the presence of perturbations, it takes the following form x i 1 = x i Gamma j i rf(x i ) ffi(x i ; j i ) Obviously, this method is a special case of Algorithm 2.1 corresponding to e(x; j) 0 8x 2 X: Consequently, we can take c 1 = 0, and c 2 = 0 in (2.1) 2.3) Provided ....

A.A. Goldstein. Convex programming in Hilbert space. Bull. Am. Math. Soc., 70:709-- 710, 1964.

....j1 Gamma flMjg ; 5.1) where m and M are the smallest and largest eigenvalue of r 2 f(x ) respectively. b) Let further u j ffi X , where X is a nonempty, closed and convex set in n . Then, the algorithm is equivalent to the Goldstein Levitin Polyak gradient projection algorithm ([Gol64, LeP66]) Moreover, the convergence criterion (3.20) and the linear convergence ratio (5.1) coincide with those of this algorithm. c) Let further X : n . Then, the algorithm is equivalent to the steepest descent method with fixed step length fl ( Pol63] with the corresponding convergence ....

A. A. Goldstein, Convex programming in Hilbert space, Bull. Amer. Math. Soc., 70 (1964), pp. 709--710.

....given by a particular matrix norm, see [69] Approximate Newton methods are defined through approximations B (l) of the Hessian matrix, for instance as block diagonal matrices, making the corresponding subproblems separable with respect to the commodities, when applied to traffic equilibrium. In [40, 56] the matrix is chosen as the identity matrix, while in [8] it is chosen positive diagonal. These methods are often referred to as (scaled) gradient projection algorithms. Such algorithms are seen to be instances of the partial linearization methods, by replacing r 2 T i x (l) j with B ....

Goldstein, A.A. (1964), Convex programming in Hilbert space. Bulletin of the American Mathematical Society 70, pp. 709-710.

No context found.

A. A. Goldstein. Convex programming in Hilbert space. Bull. Amer. Math. Soc. 70 (1964): 709--710.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC