P. Tseng and D. Bertsekas [1993] On the convergence of the exponential multiplier method for convex programming, Mathematical Programming 17, pp. 670-690.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....for example [8, 21, 22] Further, for simplicity of notation, we consider the case when i , i are independent of i; the extension onto the general case is quite straightforward. Augmented Lagrangian approach Methods of multipliers, involving nonquadratic augmented Lagrangians [10, 6, 15, 3, 19, 20, 4, 7, 11] successfully compete with the interior point and other methods in non linear and semidefinite programming. They are especially ecient when a very high accuracy of solution is required. This success is partially explained by the fact, that due to iterative update of multipliers, the penalty ....

Tseng, P. and Bertsekas, D.P. (1993). \On the Convergence of the Exponential Multiplier Method for Convex Programming", Mathematical Programming 60, 1-19.

....a max type function [2, 9] This leads to a new type of augmented Lagrangian for sum max problem, which does not require arti cial variables. We show, that the powerful tool for analysis of augmented Lagrangian algorithms, based on their correspondence to the proximal point methods in dual space [18, 13, 14, 4], can be extended to our case. The practical eciency of the algorithm is supported by computational results for large scale problems, arising in structural optimization. 2 Smoothing of Max function Let us introduce a parameterized smooth approximation of the maximum function r(t) max( t; t) ....

P. Tseng and D. Bertsekas, On the Convergence of the Exponential Multiplier Method for Convex Programming, Math. Programming 60 (1993) 1-19.

....and elliptic. Hence, the stable properties of EM in the rst iterations can be preserved while superlinear acceleration can be insured by appropriate relaxation. 5.2. Entropic methods. Entropic proximal methods have recently received much attention; for instance, see [9] 14] 29] 34] [36], 1] 20] and [35] In this section, we propose a new analysis of entropic type methods in the spirit of Section 3. In particular, the convergence of entropic type methods is established under very unrestrictive assumptions on the divergence function. Moreover, neither convexity nor ....

Tseng, Paul and Bertsekas,Dimitri P. On the convergence of the exponential multiplier method for convex programming. Math. Program. 60A, No.1, 1-19 (1993).

....the function (2) can be derived from the dual problem of an entropy optimization problem [11] we call function (2) an entropic smoothing approximation function. Independently, Li [21] discovered a few properties of this function and named it as the aggregate function. Related work can be found in [1, 3, 14, 25, 26, 27, 37]. In particular, Peng and Lin [25] also use this function in developing their non interior continuation method for solving VLCP. Also note that since a lower bound of the value of g(x) is singled out in the representation (3) it can be used in computation to avoid the potential overflow problem ....

P. Tseng and D.P. Bertsekas, On the convergence of the exponential multiplier method for convex programming, Math. Program., 60(1993): 1--19.

.... include the classical quadratic augmented Lagrangian method [16] x k 2 Arg min x2 m ( f(x) 1 2c k m X i=1 maxf0; p k Gamma1 i c k g i (x)g 2 ) 2) p k i = maxf0; p k Gamma1 i c k g i (x k )g i = 1; m (3) and the exponential method of multipliers (see for example [23]) x k 2 Arg min x2 m ( f(x) 1 c k m X i=1 p k Gamma1 i e c k g i (x) 4) p k i = p k Gamma1 i e c k g i (x k ) i = 1; m: 5) Here, fc k g is a sequence of positive scalars bounded away from zero, and the p k 2 m are Lagrange multiplier estimates for the ....

P. Tseng and D. P. Bertsekas. On the convergence of the exponential multiplier method for convex programming. Mathematical Programming 60 (1993) 1-19.

....and elliptic. Hence, the stable properties of EM in the rst iterations can be preserved while superlinear acceleration can be insured by appropriate relaxation. 5.2. Entropic methods. Entropic proximal methods have recently received much attention; for instance, see [8] 13] 27] 31] [33], 1] 18] and [32] In this section, we propose a new analysis of entropic type methods in the spirit of Section 3. In particular, the convergence of entropic type methods is established under very unrestrictive assumptions on the divergence function. Moreover, neither convexity nor ....

Tseng, Paul and Bertsekas,Dimitri P. On the convergence of the exponential multiplier method for convex programming. Math. Program. 60A, No.1, 1-19 (1993).

....[1] structural design, robust optimization and many others (see for example [4] can be expressed in terms of SDP. Various extensions to semide nite programming can be found in [1, 8, 16, 22, 24] and references therein. Methods of multipliers, involving nonquadratic augmented Lagrangians [25, 29, 6, 10, 7] successfully compete with the interior point methods in non linear programming. Especially ecient they are when a very high accuracy of solution is required. This success is partially explained by the fact, that due to iterative update of multipliers, the penalty parameter does not need to become ....

.... PBM algorithm, based on its connection to an appropriate (non quadratic) proximal point algorithm, applied to the dual problem (5) A similar dual interpretation is well known in nonlinear programming for the quadratic augmented Lagrangian method [26] and for nonquadratic augmented Lagrangians [28, 29, 7]. We show that the sequence of the dual function values, corresponding to the sequence of PBM multipliers, is monotonically nondecreasing and is bounded from above by the optimal primal value. 5.1 Conjugate penalty function The conjugate function of p k is given by Legendre transformation (see ....

P. Tseng and D. Bertsekas, On the Convergence of the Exponential Multiplier Method for Convex Programming, Math. Programming 60 (1993) 1-19.

....[1] structural design, robust optimization and many others (see for example [4] can be expressed in terms of SDP. Various extensions to semidefinite programming can be found in [1, 8, 15, 19, 21] and references therein. Methods of multipliers, involving nonquadratic augmented Lagrangians [6, 7, 10, 22, 26, 29] successfully compete with the interior point methods in non linear programming. Especially efficient they are when a very high accuracy of solution is required. This success is partially explained by the fact, that due to iterative update of multipliers, the penalty parameter does not need to ....

.... PBM algorithm, based on its connection to an appropriate (non quadratic) proximal point algorithm, applied to the dual problem (5) A similar dual interpretation is well known in nonlinear programming for the quadratic augmented Lagrangian method [23] and for nonquadratic augmented Lagrangians [25, 26, 7, 29]. We show that the sequence of the dual function values, corresponding to the sequence of PBM multipliers, is monotonically nondecreasing and is bounded from above by the optimal primal value. 5.1 Conjugate penalty function The conjugate function of p k is given by Legendre transformation (see ....

P. Tseng and D. Bertsekas, On the Convergence of the Exponential Multiplier Method for Convex Programming, Math. Programming 60 (1993) 1-19.

.... d OE (x; y) instead of a Bregman distance D (x; y) in (2) see (60) and (61) in Section 5) For the entropic barrier, which can be seen as either the Bregman distance or the OE divergence induced by the functions of Examples 1(a) and 2(b) in Section 3 respectively, it was proved in [22] that all cluster points of the sequence f s k g are dual optimal solutions. The case of the shifted logarithmic barrier (i.e. the OE divergence induced by the function of Example 2(b) in Section 3) was considered in [9] where it was proved that some cluster points of f s k g are dual ....

P. Tseng and D. Bertsekas, On the convergence of the exponential multiplier method for convex programming, Mathematical Programming, 60 (1993), pp. 1--19.

....Austria. y Systems Research Institute, Newelska 6, 01 447 Warsaw, Poland (kiwiel ibspan.waw.pl) Even if all f i are twice differentiable on C, the Lagrangian L (2) c k ( Delta; y k ) is differentiable only once. This may create difficulties for methods used to find x k in (3a) [Ber82, GoT89, KoB76, Man75, TsB93]. Other twice differentiable Lagrangians are either quite complicated [Ber82, GoT74, GoT89, KoB76] or nonconcave with respect to y [Man75] or difficult to analyze [TsB93] In the next section we exhibit a simple twice differentiable Lagrangian. It is derived from the recent work of [Eck93, ....

....) is differentiable only once. This may create difficulties for methods used to find x k in (3a) Ber82, GoT89, KoB76, Man75, TsB93] Other twice differentiable Lagrangians are either quite complicated [Ber82, GoT74, GoT89, KoB76] or nonconcave with respect to y [Man75] or difficult to analyze [TsB93]. In the next section we exhibit a simple twice differentiable Lagrangian. It is derived from the recent work of [Eck93, Teb92] on Bregman related Lagrangians. 2 The cubic Lagrangian Consider using the cubic augmented Lagrangian L (3) c (x; y) f 0 (x) 1 3c m X i=1 f[sign(y i )jy i j ....

P. Tseng and D. P. Bertsekas, On the convergence of the exponential multiplier method for convex programming, Math. Programming 60 (1993) 1--19.

....to variational inequalities on polyhedra were recently analyzed in [1] The application of method (1. 2) to the dual functional of a convex program gives rise to several interesting nonquadratic augmented Lagrangian methods [9] 26] These include, for example, algorithms given in [2] 21] and [27]. These methods have an important practical advantage over the classical augmented # Received by the editors September 20, 1995; accepted for publication (in revised form) June 19, 1996. This author was partially supported by National Science Foundation grant DMS 9401871 and by Israeli Ministry ....

<F3.515e+05> P. Tseng and D.<F3.784e+05> Bertsekas,<F3.411e+05> On the convergence of the exponential multiplier method for convex<F3.784e+05> programming, Math. Programming, 60 (1993), pp. 1--19.

.... multiplier method in [12] and [13] A more general scheme is studied in [9] Implementations reporting good numerical results are given in [3] and [7] For the exponential method of multipliers new convergence results, including rate of convergence (for the linear programming case) are obtained in [18]. Here we introduce a class of methods called Penalty Barrier Multiplier (PBM) methods, which are based on nonquadratic augmented lagrangians. A member in the PBM class is specified by a penalty barrier function and a penalty updating function , responsible for updating the penalty parameters ....

....to be very efficient and capable of solving large scale problems to a high degree of accuracy (see x7, x8) The requirement on the penalty updating function is that it is a sublinear function of the multipliers. This requirement was inspired by a suggestion in the paper by Tseng and Bertsekas [18]. We point out that an augmented lagrangian resulting from such a choice of is a nonlinear function of the multipliers. In x4 we show that the PBM method is associated with a proximal point algorithm, which simultaneously solves the dual convex programming problem. The distance like function ....

[Article contains additional citation context not shown here]

Tseng, P. and Bertsekas, D.P. (1993). "On the Convergence of the Exponential Multiplier Method for Convex Programming", Mathematical Programming 60, 1--19.

.... were obtained for a Modified Barrier Function (MBF) method in [32] and an implementation reporting good numerical results is given in [14] and [15] For the Exponential Method of Multipliers new convergence results, including rate of convergence (for the linear programming case) are obtained in [41]. More general convergence results are reported in [39] and [34] 1.1 Penalty Barrier Multiplier (PBM) Method for Constrained Nonlinear Optimization Our research originated in 1991 from the attempt to develop an efficient algorithm for Truss Topology Design (see Section 2.6.1) Among methods ....

....to develop a new convergence analysis scheme. To provide convergence of the method we introduce sublinear dependence of the penalty parameters on Lagrange multipliers (in this case each constraint has its own penalty parameter) Similar linear dependence of the penalty parameters was suggested in [41] to provide local quadratic convergence of Exponential Method of Multipliers for linear program. Here using such an approach, we prove global convergence of the method for a wide family of penalty functions, including shifted logarithmic barrier, exponential and quadratic logarithmic penalty ....

[Article contains additional citation context not shown here]

Tseng, P. and Bertsekas, D.P. (1993). "On the Convergence of the Exponential Multiplier Method for Convex Programming", Mathematical Programming 60, 1--19.

.... or replace it with a piecewise linear local approximation, as done in the convex optimization methods in [29, 28] On the theoretical level, an open issue is whether the convergence properties are preserved if the quadratic penalty is replaced by other penalty functions, such as Bregman s [45, 11], or the class of strongly convex functions employed in the auxiliary problem method [7] ....

P. Tseng and D.P. Bertsekas. On the convergence of the exponential multiplier method for convex programming. Mathematical Programming, 60:1--19, 1993.

No context found.

Tseng, P., and Bertsekas, D. P., On the Convergence of the Exponential Multiplier Method for Convex Programming, Laboratory for Information and Decision Systems, Massachusetts Institute of Technology Report P-1995, Cambridge, MA, 1990; to appear in Mathematical Programming. 28

No context found.

P. Tseng and D. Bertsekas [1993] On the convergence of the exponential multiplier method for convex programming, Mathematical Programming 17, pp. 670-690.

No context found.

P. Tseng and D.P. Bertsekas. On the convergence of the exponential multiplier methods for convex programming. Mathematical Programming 60 (1993) 1--19.

No context found.

P. Tseng and D.P. Bertsekas, On the convergence of the exponential multiplier method for convex programming, Math. Programming, 60 (1993), pp. 1--19.

No context found.

Tseng, P. and Bertsekas, D.P. (1993). "On the Convergence of the Exponential Multiplier Method for Convex Programming", Mathematical Programming 60, 1--19. 24

No context found.

P. Tseng and D. Bertsekas, "On the convergence of the exponential multiplier method for convex programming," Mathematical Programming, 60, 1993, 1--19.

No context found.

P. Tseng and D. Bertsekas, \On the convergence of the exponential multiplier method for convex programming," Mathematical Programming, 60, 1993, 1-19.

No context found.

Tseng, P. and Bertsekas, D.P. (1993). On the convergence of the exponential multiplier method for convex programming. Mathematical Programming, 60, 1--19.

No context found.

Tseng, P. and Bertsekas, D.P. (1993). "On the Convergence of the Exponential Multiplier Method for Convex Programming", Mathematical Programming 60, 1--19.

No context found.

Tseng, P. and Bertsekas, D.P., On the convergence of the exponential multiplier method for convex programming. Mathematical Programming 60, pp. 1--19, 1993.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC