J. E. Dennis, Jr. and J. J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), no. 126, 549--560.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....# # F #BP# #BP # # # 2 2 # #B# 2 2 # # (#E#F #B # 2 )EP s )B P s (Z PZ )EP s PZ )B P s for some positive constant C 5 . C. Appendix: Classical Results Theorem C. 1 (Dennis More [9]) Let F : C be a continuously di#erentiable function on the open convex set C and x C such that F (x ) 0 and F # (x ) is nonsingular. Let nn be a sequence of nonsingular matrices. Suppose that for some x 0 C the sequence defined by 1 k F (x k ) k = 0, 1, ....

....matrices. Suppose that for some x 0 C the sequence defined by 1 k F (x k ) k = 0, 1, remains in C, with x k for all k, and converges to x . Then converges superlinearly to x if and only if #[D ) x k 1 x k = 0. C.3) Proof. See, e.g. Theorem 3.1. in [9]. Theorem C.2 (Broyden, Dennis More [4] Let F : IR be di#erentiable in the open, convex set C, and assume that for some p 0 and K we have that F # satisfies # (x) K#x for all x in a neighborhood of x where F (x ) 0 and F # (x) is nonsingular. Let D be the current ....

J. E. Dennis Jr. and J. J. More. A characterization of superlinear convergence and its application to quasi--Newton methods. Math. Comp., 28:549--560, 1974.

....)k Lks k k kB k J(x k )k(1 ks k kkv k k jv Lks k k kv k k 2jv k s k j (1 1= kB k J(x k )k (L L= 2 ) ks k k: Therefore, 19) holds. Among the rank one quasi Newton formulae that satisfy the bounded deterioration condition (19) we can cite Broyden s good and bad methods [2, 5, 20], the ColumnUpdating method [10, 17] and the Inverse Column Updating method [14, 21] QuasiNewton methods that are not of rank one type can be found in [1, 23] Theorem 2. Assume that 1. F is uniformly continuous in IR 2. fx k g is generated by Algorithm 1; 7 3. K = fk 1 ; k 2 ; g ....

J. E. Dennis Jr. and J. J. More, A characterization of superlinear convergence and its applications to quasi-Newton methods, SIAM Review 19 (1977), pp. 46-89.

....that 0 as n e. As noted in Theorem 2.1, given that the sequence of inexact Newton iterates converges, this additional assumption on the Z n S implies that the convergence is at least superlinear. The main result of this subsection is a modification of a theorem obtained by Dennis and Mor [10], and shows that the global strategy based on the above c and conditions will permit full inexact Newton steps when close to a minimizer of f, 1 provided that c and . Theorem 3.13 Let F : R hr R hr be twice continuously differentiable in an open convex set D C R N, and for f FTF ....

J. E. Dennis and J. J. Mor, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28, pp. 549-560.

....(dfp) Soon after, another algorithm was proposed by Fletcher and Colin M. Reeves [98] These methods gained full recognition after the methods proposed by Charles G. Broyden in 1965 [39] and the condition obtained by John E. Dennis and Jorge J. Mor e in 1974 for their superlinear convergence [77]. In these methods, the approximation of the Jacobian (or of its inverse) is updated at each iteration by a rank one or a rank two modi cation. See [42] for some historical account. There exist many methods of this type. They are very much in favor in optimization; see, for example, 96] Their ....

J.E. Dennis, J.J. More, A characterization of superlinear convergence and its applications to quasi{Newton methods, Math. Comput., 28 (1974) 549-560.

....implicitly assume that J is symmetric and positive definite. Another update, avoiding this assumption, was proposed by Huang [42] A result similar to that of Theorem 5, but with conditions on the matrices H n instead of the matrices C n , can be obtained by means of the Dennis Mor e condition [29, 30]. This condition says that a superlinear convergence is obtained for a quasi Newton method if lim n 1 kE n (x n 1 Gamma x n )k kx n 1 Gamma x n k = 0 12 where E n = J Gamma H n . A less general formulation of this condition is given in [45, Theorem 7.1.1, p.115 116] It is also possible ....

J.E. Dennis, J.J. Mor'e, A characterization of superlinear convergence and its applications to quasi--Newton methods, Math. Comput., 28 (1974) 549--560.

....convergence speed is analyzed in sections 3 and 4. In particular, it is shown that, for fixed 0, the primal dual pairs (x, #) converge q superlinearly toward a solution of (1. 2) The tools used to prove convergence are essentially those of A BFGS INTERIOR POINT ALGORITHM 203 the BFGS theory [6, 13, 40]. In section 5, the overall algorithm is presented and it is shown that the sequence of outer iterates is globally convergent, in the sense that it is bounded and that its accumulation points are primal dual solutions of problem (1.1) If, in addition, strict complementarity holds, the whole ....

....V, which was added to # to get the merit function , has the right curvature around z , so that the unit step size in both x and # is accepted by the line search. In the following proposition, we establish a necessary and su#cient condition of q superlinear convergence of the Dennis and More [13] type. The analysis assumes that the unit step size is taken and that the updated matrix M k is su#ciently good asymptotically in a manner given by the estimate (4.5) which is slightly di#erent from (4.1) Proposition 4.2. Suppose that Assumption 2.1 holds and that f and c are twice ....

J.E. Dennis and J.J. Mor e, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549--560.

....there exists L 0 such that kJ(x) J(y)k Lkx yk (12) for all x; y 2 . Then, fx k g converges superlinearly to a solution of F (x) 0. Proof. By Theorem 2, the sequence converges to a solution x . Since B k J(x ) we also have that lim k 1 kx k d k x k kx k x k = 0: 13) See [6]. So, we only need to prove that, for k large enough, the condition (3) is ful lled with k = 1. By (13) and the nonsingularity of J(x ) we have that lim k 1 kF (x k d k )k kF (x k )k = 0: 14) Therefore, for k large enough, kF (x k d k )k kF (x k )k 1 2 kF (x k )k: But kF (x ....

.... kd k k. So, for k large enough we have: kF (x k d k )k kF (x k )k kd k k 2 (1 k )kF (x k )k kd k k 2 : Therefore, the proof is complete. 2 Among the rank one quasi Newton formulae that satisfy the bounded deterioration condition (9) we can cite Broyden s good and bad methods [3, 6, 19], the Column Updating method [17] and the Inverse Column Updating method [18] which, according to a numerical study of Luk san and Vl cek [15] is the one that behaves better in large scale problems. In the following section, we compare those methods when they are used as intermediate ....

J. E. Dennis Jr. and J. J. More, A characterization of superlinear convergence and its applications to quasi-Newton methods, SIAM Review 19 (1977), pp. 46-89.

....some norm jj:jj, and some L, p 0. The general quasi Newton method for the solution of D( 0 is given by (n 1) n) Gamma ff Gamma1 n D( n) 3. 6) where ff n is a nonsingular matrix approximating the Jacobian matrix J( The convergence result of such an algorithm can be found in [DM74]. We choose six algorithms for the present comparison: a modified Newton method, Broyden s method, Schubert s method, Schubert Kim method, Schubert Powell method and an adaptive ff method. We are interested to apply these methods to the linear system D( j J Gamma b = 0 where J is an s Theta s ....

Dennis J. J. and Mor'e J. (1974) A characterization of superlinear convergence and its application to quasi-newton methods. Math Comp 28: 549--560.

....k 1 = GammaH k rN P ff( Omega k ) where H k is updated by the inverse BFGS rule without damping. We skip the discussion of the structure and properties of the various kinds of QuasiNewton update rules found in the literature. For further update rules and their theory of convergence we refer to [11, 12, 17, 19]. 12 For the sake of completeness, we present an algorithmical scheme in figure 2.1. We formulate it with respect to the penalty functional. In particular, a second order approximation is proposed for performing the line search update if a descent fails. Clearly, it is also applicable for the ....

J.E. Dennis and J.J. Mor'e. A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods. Math. Comp., 28:549--560, 1974.

....j n 0 as n 1. As noted in Theorem 2.1, given that the sequence of inexact Newton iterates converges, this additional assumption on the j n s implies that the convergence is at least superlinear. The main result of this subsection is a modification of a theorem obtained by Dennis and Mor e [10], and shows that the global strategy based on the above ff Gamma and fi Gamma conditions will permit full inexact Newton steps when close to a minimizer of f , provided that ff 1 2 and fi 1 2 . Theorem 3.13 Let F : R N R N be twice continuously differentiable in an open convex set ....

J. E. Dennis and J. J. Mor'e, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28, pp. 549-560.

....(4.7) If k is determined by Step 3, then (4.6) follows from (4.9) and (3.4) with oe = oe 2 , and hence (4.7) holds. Let us denote j k = p n(ffl k Gamma ffl k 1 ) j k 0. Since j k satisfies (3.3) we have 1 P k=1 j k 1. Therefore, 4.8) follows from (4.7) and, by Lemma 3. 3 in [12], fk Phi k (x k )kg converges. 2 Lemma 4.2 Let fx k g be generated by Algorithm 2. Then the following four statements are equivalent. i) lim inf k 1 k Phi(x k )k = 0: ii) lim inf k 1 k Phi k (x k )k = 0: iii) lim k 1 k Phi(x k )k = 0: iv) lim k 1 k Phi k (x k )k = 0: ....

J.E. Dennis, Jr. and J.J. Mor'e, "A characterization of superlinear convergence and its applications to quasi-Newton methods", Math. Comp., vol. 28, pp. 549-560, 1974.

.... cg; where X is the solution set of (1.1) Concrete examples of local error bounds can be found in [8] When m = n and F 0 (x ) is nonsingular at a solution x , x is a locally unique solution of (1.1) and kF (x)k provides a local error bound for (1. 1) in a neighborhood of x [2]. Moreover, kF (x)k may provide a local error bound even if F 0 (x) is singular at a solution [9] Therefore, the condition that kF (x)k provides a local error bound in a neighborhood of x is milder than the condition that F 0 (x ) is nonsingular. In the LMM considered in [9] it is ....

J. E. Dennis, Jr. and J. J. More, A characterization of superlinear convergence and its application to Quasi-Newton methods, Mathematics of Computation 28 (1974) 549-560.

....i;i = i ) 1 Gammaff = min if i 2 A 0 if i 2 I (4:11) Assumptions AS5 and AS7 are often known as strict complementary slackness conditions. We observe that AS8 is closely related to the necessary and sufficient conditions for superlinear convergence of the inner iterates given by Dennis and Mor e (1974). We also observe that AS9 is entirely equivalent to requiring that the matrix B [J 1 ;J 1 ] A(x ) T [A ;J 1 ] D [A ;A ] A(x ) A ;J 1 ] 4:12) is positive definite (see, for instance, Gould, 1986) The uniqueness of the limit point in AS9 can also be relaxed by requiring ....

J. E. Dennis and J. J. Mor'e. A characterization of superlinear convergence and its application to quasi-Newton methods. Mathematics of Computation, 28(126):549-- 560, 1974.

....without any additional assumption on f . In the following sections, when we consider the local analysis of specific quasi Newton formulae, we require rf to be locally Lipschitzian, we assume also that it admits directional derivatives at x. In Section 3, we adapt to our case the criterion of [6] for superlinear convergence. Then we give superlinear convergence results for a wide class of quasi Newton methods, including PSB and DFP, assuming that f has at x a Hessian, in a strong sense. Under the same assumptions, we concentrate in Section 4 on both global and superlinear convergence ....

....(x 1 ; M 1 ) is good enough and that a bounded deterioration property holds for fM n g as is done in [14] for standard quasi Newton algorithms. We characterize the superlinear convergence in Section 3. 3, giving the prox version of the well known characterization for superlinear convergence of [6]. Finally, under stronger smoothness assumptions, we obtain local and superlinear convergence results for a wide class of quasi Newton formulae, including the prox versions of the PSB and DFP algorithms. For this we extend the approach of Grzeg orski [12] to variational quasi Newton methods with ....

[Article contains additional citation context not shown here]

J.E. Dennis and J.J. Mor'e, "A characterization of superlinear convergence and its application to quasi-Newton methods", Mathematics of Computation, 28(1974) 549-560.

....k, the unit stepsize is accepted by the algorithm. That is, x k 1 = x k d k . We remark that the direction d k produced by system EQ(x k , H k ; A k ) can be regarded as a quasi Newton direction for the equality constrained problem min f(x) s.t. g A k(x) 0. 46) Dennis and More [4] gave a necessary and su#cient condition on the superlinear convergence of quasi Newton methods for unconstrained optimization. Boggs, Tolle, and Wang [2] extended the classical result of Dennis and More to the equality constrained case. However, since the strict complementarity condition does not ....

J.E. Dennis and J.J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comput., 28(1974) 549-560. 21

....[29] the following result can be obtained. Theorem 2. In addition to the hypotheses of Theorem 1, suppose that lim k 1 k[B k Gamma B ] x k 1 Gamma x k )k kx k 1 Gamma x k k = 0: 31) Then, 26) holds. Theorem 2 corresponds, in the case r = 0, to the well known DennisMor e condition [24], which characterizes the superlinear convergence of sequences generated by (7) Now, by (30) Theorem 2 implies the following more practical result. 9 Theorem 3. Assume the hypotheses of Theorem 1, and lim k 1 k[B k 1 Gamma B ] x k 1 Gamma x k )k kx k 1 Gamma x k k = 0: 32) Then, 26) ....

Dennis Jr. , J. E. ; Mor'e, J. J. [1974]: A characterization of superlinear convergence and its application to quasi - Newton methods, Mathematics of Computation 28, pp. 549-560.

....the local version of the ane scaling interior point algorithm for nonlinear programming. The analysis will follow the approach given by Yamashita and Yabe [29] for primal dual interior point algorithms, which in turn relies on the theory developed by Broyden et al. [4] and Dennis and Mor e [11] for quasi Newton methods. However, the technical results needed for the analysis are obtained di erently from [29] and they will be the subject of a careful study in Section 3. The results for linear, superlinear, and quadratic convergence are stated in Section 4. 2 The ane scaling interior point ....

....of k. The proof is complete since we know, from fact 2 (in Section 2.1) that the matrix rF ;k (w ) is nonsingular. 2 9 4 Local convergence The results in this section rely on the classical theory of quasi Newton methods (see the papers by Broyden, Dennis, and Mor e [4] and Dennis and Mor e [11]) and correspond to the results that Yamashita and Yabe [29] obtained for the local version of the primal dual interior point algorithm. The proofs are similar and are omitted. The rst result is the q linear convergence of the anescaling interior point algorithm. We require the approximation H k ....

J. E. Dennis and J. J. More. A characterization of superlinear convergence and its application to quasi{Newton methods. Math. Comp., 28:549-560, 1974.

....Sherman Morrison formula. More examples will be given in Section 6. 3 Superlinear convergence In this section we prove a sufficient condition for superlinear convergence of a sequence generated by Algorithm 2.1. For the choice k = 1 our result reduces to the classical Dennis Mor e condition (Dennis and Mor e [1974]) Throughout this section we suppose that the following local assumptions are satisfied. Local assumptions. Let Omega be an convex subset of IR n . We assume that F 2 C 1( Omega Gamma and that there exist x 2 Omega ; L; p 0, such that F (x ) 0, J(x ) is nonsingular and 4 kJ(x) ....

....ks k k ; s k ks k k Gamma 1 fi fi fi fi fi k[I Gamma B Gamma1 k J(x k ) s k =ks k k k: 3.10) So, by (3.7) and (3.10) the numerator of (3.9) tends to 1. This completes the proof of (3.5) for the choice (2.4) The superlinear convergence of fx k g follows from Corollary 2. 3 of Dennis and Mor e [1974]. 2 Remark. Observe that boundedness of kB Gamma1 k k was necessary to prove (3.8) with the choice (2.4) but is not necessary for the choice (2.3) Corollary 3.2. Assume the hypotheses of Theorem 3.1, except (3.4) Instead of this, assume that lim k 1 k(B k 1 Gamma J(x )s k k ks k k = ....

J. E. Dennis and J. J. Mor' e A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974) 549-560.

....quadratically to x . Furthermore, under the same assumptions any sequence fy k g whichconverges to x superlinearly is closely related to Newton s method by the fact that the relative difference between y k 1 ; y k and the Newton correction ;F (y k ) 1 F (y k ) will tend to zero [3]. The quadratic convergence of Newton s method is attractive. However, the method depends on a good initial approximation and it requires in general n 2 n function evaluations per iteration besides the solution of an n Theta n linear system. Quasi Newton methods were developed to save ....

J.E. Dennis Jr and J. Mor'e, A characterization of superlinear convergence and its applications to quasi Newton methods, Math. Comp., 28, 548--560, (1974).

....) i;i = 3 i ) 10ff = min if i 2 A 3 0 if i 2 I 3 (4:11) Assumptions AS5 and AS7 are often known as strict complementary slackness conditions. We observe that AS8 is closely related to the necessary and sufficient conditions for superlinear convergence of the inner iterates given by Dennis and Mor e (1974). We also observe that AS9 is entirely equivalent to requiring that the matrix B 3 [J 1 ;J 1 ] A(x 3 ) T [A 3 ;J 1 ] D 3 [A 3 ;A 3 ] A(x 3 ) A 3 ;J 1 ] 4:12) is positive definite (see, for instance, Gould, 1986) The uniqueness of the limit point in AS9 can also be relaxed by ....

J. E. Dennis and J. J. Mor'e. A characterization of superlinear convergence and its application to quasi-Newton methods. Mathematics of Computation, 28(126):549--560, 1974. 24

No context found.

J. E. Dennis, Jr. and J. J. More, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp. 28 (1974), no. 126, 549--560.

No context found.

J. E. Dennis, Jr. and J. J. Mor e, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549--560.

No context found.

J. E. DENNIS AND J. J. MOR ' E, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549--560.

No context found.

J. E. Dennis, Jr. and J. J. Mor'e. A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods. Mathematics of Computation, 28:549--560, 1974.

No context found.

J. E. DENNIS AND J. J. MORl, A characterization of superlinear convergence and its application to quasi-Newton methods, Math. Comp., 28 (1974), pp. 549-560.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC