Kiwiel, K.C. (1997) Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim., 35:1142--1168.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... was further studied in [30] and has since been known as a Bregman distance (see [33] for an historical account) In R , various investigations have focused on the use of Bregman distances in projection, proximal point, and xed point algorithms, see [7, 31, 32, 33, 46, 47, 83] see also [58, 59] where extensions of (1.4) to nondi erentiable functions were studied) Extensions to Hilbert [18, 20, 61] and Banach [1, 8, 21, 23, 24, 25, 26, 27, 55, 56, 75] spaces have also been considered more recently. In the present paper, we adopt the following de nition for Bregman distances. ....

K. C. Kiwiel, Proximal minimization methods with generalized Bregman functions, SIAM J. Control Optim., 35 (1997), pp. 1142-1168.

....elliptic. Hence, the stable properties of EM in the rst iterations can be preserved while superlinear acceleration can be insured by appropriate relaxation. 5.2. Entropic methods. Entropic proximal methods have recently received much attention; for instance, see [9] 14] 29] 34] 36] 1] [20], and [35] In this section, we propose a new analysis of entropic type methods in the spirit of Section 3. In particular, the convergence of entropic type methods is established under very unrestrictive assumptions on the divergence function. Moreover, neither convexity nor di erentiability are ....

Kiwiel, K. C. Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optimization 35, No.4, 1142-1168 (1997). [ISSN 0363-0129].

....cone of vectors normal to the set B at x. It is well known that, under mild regularity conditions, 1) is the special case of (2) for which T = f , the subgradient mapping of f . The last decade has seen considerable progress in the theory of proximal point methods based on generalized distances [11, 13, 19, 5, 21, 31, 14, 2, 3, 17]. Such methods use a scalarvalued regularization function to derive better behaved versions of problems (1) and (2) In this article, we consider separable regularization terms of the form D(x; y) n X i=1 d i (x i ; y i ) where d 1 ; d n are scalar functions conforming to very ....

....fi 2 (0; 1) with with a i = 0, b i = 1. Finally, we note that for finite a i we do not yet assume that h i (x i ) must approach a finite limit as x i a i , nor similarly for x i b i 1. Such an assumption is quite common in the theory of Bregman distances [11, 13, 9, 29] but, similarly to [21], it is not needed for the results of Section 3 below. We will use it, however, in the variational inequality analysis of Section 4. RRR 35 99 Page 11 2.2.2 divergences The divergence regularizations have been studied in the context of proximal methods, for example, in [19] and more ....

[Article contains additional citation context not shown here]

K. C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization, 35:1142-1168, 1997.

....a rather strong condition. Finally, our dual viewpoint extends the one that has been developed for proximaltype algorithms, which is limited to the case where (0.1) itself is a lagrangian dual of (cf. 9.2) A. Frangioni 35 Differentiability of D x ,t , but not strict convexity, is dropped in [Ki98], where B functions are introduced; there, the compactness requirement is also different (there is no need for local compactness, as the solution of (1.12) is assumed given) An implementable version of the proximal method using B functions, the bundle Bregman proximal method, is then proposed ....

K. KIWIEL "Proximal Minimization Methods with Generalized Bregman Distances" SIAM J. on Control and Opt. 35, p. 1142-1168, 1998

....mapping on n and fc k g is a positive sequence bounded away from zero. It has been shown that fx k g converges to a zero of T whenever the set of zeros is nonempty [22] Recently, generalized PPAs with nonlinear kernels such as Bregman function or divergence have been studied extensively [1, 2, 7, 8, 11, 12, 18, 24, 25]. Moreover, when applied to problems with separable structure, the PPA yields decomposition algorithms, among which the Douglas Rachford splitting method and its variants are well known because of their desirable convergence properties [5, 6, 9, 10, 13, 14, 15, 16, 17, 20, 26] In this paper, we ....

.... the authors [19] and the other is the entropic proximal decomposition method (EPDM) proposed by Auslender and Teboulle [3] Similar to the PCPMM, the NPCPMM is an algorithm for solving convex programming problems with separable structure, and is based on the nonlinear PPA using Bregman functions [7, 8, 11, 12, 18, 24]. On the other hand, the EPDM is designed to solve the VIP (1) with SX = n , SZ = m , and is based on the logarithmic quadratic proximal method [1, 2] These two algorithms have an advantage over the original PCPMM in that, because the nonlinear kernels serve as an interior penalty ....

K.C. Kiwiel, Proximal minimization methods with generalized Bregman functions, SIAM Journal on Control and Optimization, Vol. 35, pp. 1142-1168, 1997.

....generating D Phi ( Delta; Delta) Of course, 8) may be only a conceptual method, since no explicit formula for Q 0 may exist. Now, while a wealth of different criteria are known for approximately computing the minimum in (8) for various forms of the distance kernel D, as for example in [1, 8, 9, 15, 18, 19, 22], all have proven awkward or impossible to translate into tractable conditions for the minimization in the equivalent algorithm (6) 7) The most successful approaches up to now in this context [1, 9, 16] require ffl k optimality of x k , that is, OE k (x k ) inf x2 n fOE k (x)g ffl k ....

.... minimum in (8) for various forms of the distance kernel D, as for example in [1, 8, 9, 15, 18, 19, 22] all have proven awkward or impossible to translate into tractable conditions for the minimization in the equivalent algorithm (6) 7) The most successful approaches up to now in this context [1, 9, 16], require ffl k optimality of x k , that is, OE k (x k ) inf x2 n fOE k (x)g ffl k : 9) In practice, inf x2 nfOE k (x)g is generally unknown, and such a condition may be difficult or impossible to verify without making additional, stringent assumptions on f , g, and or D. Other ....

K. C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization 35 (1997) 1142-1168.

....when f is given by means of a black box . Finally, our dual viewpoint extends the one that has been developed for proximal type algorithms, which is limited to the case where (0.1) itself is a lagrangian dual of (cf. 9. 2) Differentiability of D x ,t (but not strict convexity) is dropped in [Ki98] also, where B functions (generalized D functions) are introduced; there, the compactness requirement is also different (there is no need for local compactness, as the solution of (1.12) is assumed given) An implementable version of the proximal method using B functions is then proposed in ....

K. KIWIEL "Proximal Minimization Methods with Generalized Bregman Distances" SIAM J. on Control and Opt. 35, p. 1142-1168, 1998

....point in C, i.e. to a solution of the convex feasibility problem. Further results can be found in [1] 5] 8] 16] 17] 18] 20] 23] 43] Bregman distances are increasingly employed in other elds; see, for instance, 10] 11] 12] 13] 14] 15] 21] 28] 29] 38] 41] [42], 48] 2 If we set f : 1 2 k k 2 , then rf = I and the Bregman projection is actually the ordinary orthogonal projection and the method of cyclic Bregman projections becomes the famous method of cyclic (orthogonal) projections. See also [6] The method of cyclic projections di ers ....

K.C. KIWIEL. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization, 35(4):1142-1168, 1997.

....elliptic. Hence, the stable properties of EM in the rst iterations can be preserved while superlinear acceleration can be insured by appropriate relaxation. 5.2. Entropic methods. Entropic proximal methods have recently received much attention; for instance, see [8] 13] 27] 31] 33] 1] [18], and [32] In this section, we propose a new analysis of entropic type methods in the spirit of Section 3. In particular, the convergence of entropic type methods is established under very unrestrictive assumptions on the divergence function. Moreover, neither convexity nor di erentiability are ....

Kiwiel, K. C. Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optimization 35, No.4, 1142-1168 (1997). [ISSN 0363-0129].

....(see, for example, 34, 8] 1 X k=0 jje k jj 1: Note that even though the proximal subproblems are better conditioned than the original problem, structurally they are as dicult to solve. This observation motivates the development of the nonlinear or generalized proximal point method [16, 13, 11, 19, 23, 22, 20, 6]. In the generalized proximal point method, x k 1 is obtained solving the generalized proximal point subproblem 0 2 c k T (x) rf(x) rf(x k ) The function f is the Bregman function [2] namely it is strictly convex, differentiable in the interior of C and its gradient is divergent on the ....

....k=0 jje k jj 1 and P 1 k=0 D e k ; x k E exists and is nite , 3) then the generated sequence converges to a solution (provided it exists) under basically the same assumptions that are needed for the convergence of the exact method. Other inexact generalized proximal algorithms are [7, 23, 41]. However, the approach of [14] is the simplest and the easiest to use in practical computation (see the discussion in [14] Still, the error criterion given by (3) is not totally satisfactory. Obviously, there exist many error sequences fe k g that satisfy the rst relation in (3) and it is ....

K.C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization, 35:1142-1168, 1997.

....by Martinet [27, 28] and generalized by Rockafellar [44, 45] x k 1 = I ff k T ) Gamma1 (x k ) k = 0; 1; where ff k 0. This method and its dual version in the context of convex programming, the method of multipliers of Hesteness and Powell, have been extensively studied (see [1, 15, 18, 21] and references therein) and are known to yield as special cases decomposition methods such as the method of partial inverse [48, 52] the Douglas Rachford splitting method and the alternating direction method of multipliers [8, 9, 10, 22] In the case of T = A B, where A and B are maximal ....

....properties as, but is more complicated than, the modified F B method. The analysis in [58] is for the case H = n , although extension to a Hilbert space setting seems possible. Lastly, there recently have been much study of proximal point methods using a non quadratic proximal term (see [18] and references therein) and it would be interesting to extend the modified F B method to this setting. ....

K.C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim., 35:1142--1168, 1997.

....(see, for example, 34, 8] 1 X k=0 jje k jj 1: Note that even though the proximal subproblems are better conditioned than the original problem, structurally they are as difficult to solve. This observation motivates the development of the nonlinear or generalized proximal point method [16, 13, 11, 19, 23, 22, 20, 6]. In the generalized proximal point method, x k 1 is obtained solving the generalized proximal point subproblem 0 2 c k T (x) rf(x) Gamma rf(x k ) The function f is the Bregman function [2] namely it is strictly convex, differentiable in the interior of C and its gradient is divergent ....

....k=0 jje k jj 1 and P 1 k=0 D e k ; x k E exists and is finite , 3) then the generated sequence converges to a solution (provided it exists) under basically the same assumptions that are needed for the convergence of the exact method. Other inexact generalized proximal algorithms are [7, 23, 41]. However, the approach of [14] is the simplest and the easiest to use in practical computation (see the discussion in [14] Still, the error criterion given by (3) is not totally satisfactory. Obviously, there exist many error sequences fe k g that satisfy the first relation in (3) and it is ....

K.C. Kiwiel. Proximal minimization methods with generalized Bregman functions. SIAM Journal on Control and Optimization, 35:1142--1168, 1997.

....of positive scalars. If T is a maximal monotone operator, then (I c k T ) Gamma1 is a point to point mapping and the sequence fx k g converges to a root of T . Recently, generalized PPAs using nonlinear functions such as Bregman function or divergence have been studied extensively [1, 2, 7, 8, 10, 11, 15, 22, 23]. The nonlinear PPA (NPPA) using Bregman function [10, 11] generates a sequence fx k g with the following iterative scheme: x k 1 (rh c k T ) Gamma1 (rh(x k ) where h is a Bregman function whose definition will be given in Section 2. In particular, when applied to convex programming ....

....generates a next point u from a current point u 2 dom h by the iterative scheme: u = arg min u2 n ae h(u) 1 c D OE (u; u) oe ; where OE : S is a Bregman function with dom h S, and c is a positive scalar. An approximate version of the PMAD has been proposed [15], in which the above exact minimization is replaced with finding a pair (u ; fl ) satisfying the following conditions for some ffl 0: fl 2 ffl h(u ) fl 1 c 1 D OE (u ; u) 3 0; 6) where ffl h is the ffl subdifferential of h defined by ffl h(u) fp 2 n j ....

K.C. Kiwiel, Proximal minimization methods with generalized Bregman functions, SIAM Journal on Control and Optimization, Vol. 35, pp. 1142-1168, 1997.

....an inexact proximal like algorithm based on (1.2) and an ergodic type convergence result for maximal monotone operators. Similar extensions and convergence results for proximal like methods based on Bregman functions have been given by Kabbadj [12] and more recently in strengthened form by Kiwiel [11]. However, it should be noted that the analysis of proximal like methods based on Bregman distances does not carry over to method (1.2) except for the case #(t) t log t t 1, for which the two distances coincide; see, e.g. 26] This is due mainly to the fact that the nice ....

....in the spirit of the existing results for the classical quadratic proximal algorithm; see Guler [8] and Lemaire [15] For proximallike methods based on Bregman functions, the estimate (4.9) has been derived by Chen and Teboulle [6] and results analogous to Theorem 4. 1 have been given by Kiwiel [11]. Remark 4.2. As pointed out by one referee, the condition # 1 k # k # k # 0 in Theorem 4.3(ii) could be replaced by the simpler condition # # k #. First, we note that with the sole condition # k # 0 we have lim inf n# # f(x n ) inf f(x) x # R p . Indeed, from Lemma ....

[Article contains additional citation context not shown here]

<F3.515e+05> K. C.<F3.784e+05> Kiwiel,<F3.411e+05> Proximal minimization methods with generalized Bregman<F3.784e+05> functions, SIAM J. Control Optim., 35 (1997), pp. 1142--1168.

.... is a consequence of some particular properties of g(x) kxk 2 (like the triangular inequality) which do not hold for other Bregman functions appropriate for the constrained case (C 6= R n ) In the context of constrained problems, a related inexact proximal point method has been considered in [11] for the optimization case, i.e. with T = f , so that VIP(T; C) becomes min f(x) s.t. x 2 C. Its iteration is of the form 0 2 ( k f H k ) x k 1 ) Since, as mentioned before, we have in general T 6= f for T = f , our method, when applied to optimization problems, does not reduce ....

Kiwiel, K. Proximal minimization methods with generalized Bregman functions (to be published)

....function, and, as in much subsequent work, h is assumed to be a Bregman function with zone S, defined similarly to B1 B7 below. Further analysis and applications appeared in [26] and the case of a general maximal monotone operator T was addressed in [12] Subsequent papers, including for example [5, 8, 9, 15, 16, 18], have dealt with many other variations, special cases, and related algorithms. Exactly computing solutions x k 1 to (4) is fraught with the same practical difficulties as apply to (3) This paper addresses the introduction into (4) of a non zero error sequence fe k g, analogous to that ....

.... difficulties as apply to (3) This paper addresses the introduction into (4) of a non zero error sequence fe k g, analogous to that permitted in the classical proximal point algorithm (2) To date, a number of different researchers have considered approximate versions of (4) One example is [18], which allows very general auxiliary functions h, but restricts T to be the subgradient mapping of a closed proper convex function. Here, the approximation protocol is not exactly as in (2) but uses the theory of ffl subgradients. 27] gives related ffl subgradient based results in the context ....

[Article contains additional citation context not shown here]

K.C. Kiwiel, Proximal minimization methods with generalized Bregman functions, SIAM Journal on Control and Optimization 35 (1997) 1142-1168.

No context found.

Kiwiel, K.C. (1997) Proximal minimization methods with generalized Bregman functions. SIAM J. Control Optim., 35:1142--1168.

No context found.

K. C. Kiwiel [1997], Proximal minimization methods with generalized Bregman functions, SIAM Journal on Control and Optimization 35, pp. 1142-1168.

No context found.

K.C.Kiwiel, Proximal minimization methods with generalized Bregman functions, SIAM J. Control Optim. 35 (1997), 1142--1168.

No context found.

K.C. Kiwiel, "Proximal minimization methods with generalized Bregman functions," SIAM Journal on Control and Optimization, 35, 1997, 1142--1168.

No context found.

K.C. Kiwiel, \Proximal minimization methods with generalized Bregman functions," SIAM Journal on Control and Optimization, 35, 1997, 1142-1168.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC