J. Dennis and J. More. Quasi-newton methods, motivation and theory. SIAM Review, 19:46---89, 1977.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....backward error matrix to be Hermitian has little e ect on its norm. Note that for a single eigenpair (x; with x of unit 2 norm and r = I A)x being the residual, E opt is given by E opt = rx xr (r x)xx which is a result well known in the elds of nonlinear equations and optimization [10], 11, p. 171] and numerical linear algebra [6] 19] In this case, 2 ( x Ax) In practice, if X k has columns that are close to being orthonormal, one can replace them by the unitary factor of either its QR factorization or its polar decomposition. 13 A = Ig is the class of ....

J. E. Dennis, Jr. and Jorge J. Mor e, Quasi-Newton methods, motivation and theory, SIAM Review, 19 (1977), pp. 46-89.

....nitedimensional case. Namely, if in the case of nite dimensions one applies the Frobenious norm, which takes into account the values of all elements of matrix P , then (4) is a unique solution to the optimization problem (3) Using the standard technique from the theory of quasiNewton methods [8, 9, 10, 11, 14], we can prove the following result. Theorem 1 Let F : U Y be continuously di erentiable in S(u d ; r) fu 2 U j jju u d jj rg, where u d is a solution of the operator equation (1) and r 0. Let q 2 (0; 1) and the following conditions hold. u d )jj ; u 1 ) F (u 2 )jj jju 1 ....

J.E. Dennis, Jr., and J.J. More, \Quasi-Newton methods, motivation and theory", SIAM Review, Vol. 19, pp.46-89, 1977.

.... superlinear convergence variable metric method 1. Introduction and Background Quasi Newton methods for nonlinear optimization problems have been studied extensively since the late 60s. While there are a number of updates and convergence analyses for the unconstrained case (see, e.g. [10] and [4] the constrained Both authors: School of OR IE, Cornell University, Ithaca, NY 14853 3801, e mail: wagner miketodd orie.cornell.edu Copyright (C) by Springer Verlag. Mathematical Programming 87 (2000) 317 350. case has only been discussed more recently, e.g. in [6] 22] 7] ....

....for all iterations. Hence (C.4) holds for our update (2.9) and we get linear convergence as a consequence. To bound the convergence rate more we need to do (even) more work, and although the proof of superlinear convergence is almost identical to the proof of the second part of Theorem 5. 2 in [10], we give it here for the sake of completeness. We set # k = and use (6.10) and the bounded deterioration result together with the fact that for all a, b IR with a b 0, a b a (2a) 1 b # k 1 k # 1 # k # k # 2 # k , where # k = max #x k 1 and # ....

J. E. Dennis Jr. and J. J. More. Quasi--Newton methods, motivation and theory. SIAM Review, 19(1):46--89, 1977.

....class of methods for unconstrained optimization, quasi Newton methods, define the search direction by solving B k d = where B k is a quasi Newton matrix. The linear system determines the next iterate, therefore play the essential role for the convergence rate of the method. It is well known([3]) that the superlinear convergence of quasi Newton methods is equavalent to lim k## #(B f(x ) d k #d = 0. 1.14) For constrained optimization problems, the search directions or the trial steps are computed by solving some subproblems. These subproblems are some kinds of ....

J.E. Dennis and J.J. More, Quasi-Newton method, motivation and theory, SIAM Review 19(1977) 46-89.

....implicitly assume that J is symmetric and positive definite. Another update, avoiding this assumption, was proposed by Huang [42] A result similar to that of Theorem 5, but with conditions on the matrices H n instead of the matrices C n , can be obtained by means of the Dennis Mor e condition [29, 30]. This condition says that a superlinear convergence is obtained for a quasi Newton method if lim n 1 kE n (x n 1 Gamma x n )k kx n 1 Gamma x n k = 0 12 where E n = J Gamma H n . A less general formulation of this condition is given in [45, Theorem 7.1.1, p.115 116] It is also possible ....

J.E. Dennis, J.J. Mor'e, Quasi--Newton methods, methods, motivation and theory, SIAM Rev., 19 (1977) 46--89.

....Jacobians or divided difference approximations) convergence of the derivatives can be obtained easily. As an immediate consequence of Assumptions 1 and 2, we note that by standard arguments kP k k c 0 (1 ffi ) and kP Gamma1 k k c 0 = 1 Gamma ffi ) 3) In the case of secant methods [13], the condition (2) is usually imposed for k = 0 and deduced for k 1 to guarantee local convergence. If one assumes a certain kind of uniform linear independence for the sequence of the search directions, it can be shown [19] that ffi = 0. This is a sufficient, but by no means necessary, ....

....behave eventually like equation solvers. In the general nonlinear equations case, the progress towards the solution is usually gauged in terms of some norm of the residual vector F . Often a monotonic decrease of such a merit function is enforced by a suitable line search or trust region strategy [13]. In the optimization case, one may use the objective function f itself, which we have utilized for a line search consisting of a single parabolic interpolation. This simple strategy works here because the objective function is convex and very smooth. Since the unique solution x = 0 is ....

J. E. Dennis and J. J. Mor'e. Quasi-Newton methods, motivation and theory. SIAM Review, 19:46--89, 1977.

.... in the literature are: an abstract sequence, as in [10] Mn j I, as in [8] Mn = n I, with heuristic rules for computing n ; see [9] 15] RR n2128 4 Claude Lemar echal , Claudia Sagastiz abal An essential feature of our present development consists of a quasi Newton update of Mn [4] using the so called Moreau Yosida regularization ( 12] 19] we also pay some attention to the updates used by Shor [17] The next section is devoted to the Moreau Yosida regularization; we recall a few results, slightly generalized in the sense that we admit semi positive definite matrices Mn ....

J.E. Dennis and J.J. Mor'e. Quasi-Newton methods, motivation and theory. SIAM Review, 19:46--89, 1977.

....of E k 1 as an approximation to E is bounded in such a way that the error jE k 1 Gamma E j is less than the error jE k Gamma E j plus a term which is proportional to the error kx k Gamma x k. This is a typical bounded deterioration principle , as introduced in [11] See, 8 also, [25, 26, 29, 91] and many other papers. By bounded deterioration, the parameters E k cannot escape from a neighborhood of E for which it can be guaranteed that local convergence holds. Therefore, Assumptions 1 to 5 are sufficient to prove the following theorem. Theorem 1. Suppose that Assumptions 1 to 5 hold ....

....its incorporation to ordinary practice of problem solvers in Physics, Chemistry, Engineering and Industry. Other times, promising algorithms are completely forgotten, both in research and applications. The situation of the area surveyed in this paper is perhaps intermediate. The classical paper [25] is cited in most works concerning quasi Newton methods for nonlinear systems. While this survey was being written it had been cited 361 times in indexed scientific journals. The last 100 citations go from 1992 to the present days. 42 of these citations come from nonmathematical journals. It must ....

[Article contains additional citation context not shown here]

Dennis Jr. , J. E. ; Mor'e, J. J. [1977]: Quasi-Newton methods, motivation and theory, SIAM Review 19, pp. 46-89.

.... superlinear convergence variable metric method 1. Introduction and Background Quasi Newton methods for nonlinear optimization problems have been studied extensively since the late 60s. While there are a number of updates and convergence analyses for the unconstrained case (see, e.g. [10] and [4] the constrained case has only been discussed more recently, e.g. in [6] 22] 7] 19] and [14] and in Peter Fenyes Ph.D. thesis [11] The motivation for this study came mainly from a paper by Todd [26] where he derives di#erent well known updates for unconstrained problems in the ....

....for all iterations. Hence (C.4) holds for our update (2.9) and we get linear convergence as a consequence. To bound the convergence rate more we need to do (even) more work, and although the proof of superlinear convergence is almost identical to the proof of the second part of Theorem 5. 2 in [10], we give it here for the sake of completeness. We set # k = #B k B # #F and use (5.10) and the bounded deterioration result together with the fact that for all a, b # IR with a b 0, a 2 b 2 ) 1 2 # a (2a) 1 b 2 to get # k 1 # # 1 1 2# 2 k # 2 k # 1 ....

J. E. Dennis Jr. and J. J. More. Quasi--Newton methods, motivation and theory. SIAM Review, 19(1):46--89, 1977.

....or divided difference approximations) derivative convergence of the derivatives can be obtained easily. As an immediate consequence of Assumptions 1 and 2, we note that by standard arguments kP k k c 0 (1 ffi ) and kP Gamma1 k k c 0 = 1 Gamma ffi ) 3) In the case of secant methods [5], the condition (2) is usually imposed for k = 0 and deduced for k 1 to guarantee local convergence. If one assumes a certain kind of uniform linear independence for the sequence of the search directions, it can be shown [6] that ffi = 0. This is a sufficient, but by no means necessary, ....

??? J. Dennis and Jorge More. Motivation and Theory

....if the size is large. Quasi Newton methods mimic Newton s method by generating less expensive approximations B k to the matrix B k . These approximations are generated by updating the approximation for B k Gamma1 , and some have come to be interpreted as matrix approximation problems [28]. The most popular quasiNewton variant is that proposed by Broyden, Fletcher, Goldfarb, and Shanno (BFGS) which is defined by the update formula B k 1 = B k Gamma B k s k s T k B k s T k B k s k y k y T k y T k s k where y k is the change in gradient and s k is the change in x. An ....

J. E. Dennis, Jr. and Jorge J. Mor`e. Quasi-Newton methods, motivation and theory. SIAM Review, 19:46--89, 1977.

....can be used to solve (5) However, if r 2 f(x k ) is not positive definite, 5) may lead toward a saddle point or even a maximizer. Moreover, computation of r 2 f(x k ) may be very large and time consuming. Quasi Newton methods aim to eliminate these disadvantages (see [Gilbert1989a] and [Dennis1977a]) Some update formulae based upon the secant method use the sequence of data f(x i ; rf(x i ) i = 1 : kg to compute a positive definite approximation H k of r 2 f(x k ) or directly W k = H k Gamma1 . More precisely, we set s = s k = x k 1 Gamma x k and y = y k = rf(x k 1 ) Gamma ....

J. E. Dennis, Jr. and J. J. Mor' e, Quasi-Newton methods, motivation and theory, SIAM Review, 19 (1977), pp. 46--89.

....Algorithms with Inexact Line Searches on Convex Functions. Ariyawansa [2] has presented a class of collinear scaling algorithms for unconstrained minimization. A certain family of algorithms contained in this class may be considered as an extension of quasi Newton methods with the Broyden family [11] of approximants of the objective function Hessian. Byrd, Nocedal and Yuan [7] have shown that all members except the DFP [11] method of the Broyden convex family of quasiNewton methods with Armijo [1] and Goldstein [12] line search termination criteria are globally and q superlinearly convergent ....

....for unconstrained minimization. A certain family of algorithms contained in this class may be considered as an extension of quasi Newton methods with the Broyden family [11] of approximants of the objective function Hessian. Byrd, Nocedal and Yuan [7] have shown that all members except the DFP [11] method of the Broyden convex family of quasiNewton methods with Armijo [1] and Goldstein [12] line search termination criteria are globally and q superlinearly convergent on uniformly convex functions. Extension of this result to the above class of collinear scaling algorithms of Ariyawansa [2] ....

[Article contains additional citation context not shown here]

Dennis Jr, J. E. and Mor'e, J. J.: Quasi-Newton methods, motivation and theory. SIAM Rev. 19, 46--89 (1977).

....point satisfies Ax = b) we establish the superlinear convergence rate of the iteration in the inner loop of Algorithm 1. We first show that the full Newton step may be taken when x j is close to the optima x by showing that the following lemma is applicable. Lemma 8 (Dennis and Mor e [4]) Let f : R n R be twice continuously differentiable in an open set D and consider iteration x j 1 = x j ff j p j , where j = 0; 1; Delta Delta Delta, rf(x j ) T p j 0, and ff j is chosen to satisfy the Goldstein Armijo conditions. If fx j g converges to a point x in D at which ....

J. E. Dennis and Jorge J. Mor'e. Quasi-newton methods, motivation and theory. SIAM Review, 19(1):46--89, 1977.

....y T k s k Gamma H k s k s T k H k s T k H k s k ; where s k = x k 1 Gamma x k and y k = rf(x k 1 ) Gamma rf(x k ) If H k is positive definite and y T k s k 0, then H k 1 is also positive definite. The fundamental material about secant updates can be found in the classical references [38], 39] 31 The Normal Decomposition A popular step decomposition, which amounts to special choices for s q k and W k , is the normal decomposition: s k = s n k s t k = s n k Z k s t k ; 3.6) s n k is the minimum norm solution of the linearized constraints, and the columns ....

J. E. Dennis and J. J. Mor' e, Quasi--Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46--89.

....with its well defined convergence properties. We consider matrix algorithms to solve (1.2) as an inner iteration and the embedding Newton iteration as the outer iteration in a nested iteration scheme. Much work has been done to find efficient ways for employing Newton s method globally ( 1] [4], 10] 13] and the nested iteration scheme has been thoroughly analysed for inexact Newton methods ( 3] 6] However, little is said about the relation between inner and outer iterations in the proximity of the solution. As a practical method, Jacobian updates are sometimes dropped at the ....

J.E. Dennis, J.J. Mor'e, Quasi-Newton Methods, Motivation and Theory, SIAM Review, Vol. 19, No. 1 (1977), pp. 46-89.

....on a generalized logistic function is trained to make predictions. The analytic expressions (see Appendix) which we have derived for the gradients of the feedforward network have been used in a Successive Quadratic Programming based training algorithm. This algorithm employs the BFGS approach (see [3]) to updating the Hessian matrix. The BFGS method is a successive approximation of the true Hessian matrix. Watrous (1987) 4] has considered the BFGS method for the enhancement of the backpropagation training algorithm. This approach contrasts with ours in that while the backpropagation algorithm ....

Dennis, J. E. and More, J. J. Quasi-Newton Methods, Motivation and Theory. SIAM Rev., 19(1):46--89, 1977.

....unconstrained optimization problems. Because of favorable numerical experience and fast theoretical convergence, it has become a method of choice for engineers and mathematicians who are interested in solving optimization problems. Local convergence theory of BFGS method has been well established [3, 4]. The study on global convergence of BFGS method has also made good progress. In particular, for convex minimization problems, it has been shown that the iterates generated by BFGS are globally convergent, if the exact line search or some special inexact line search is used [1, 2, 5, 8, 13, 14, ....

J.E. Dennis and J.J. Mor'e, Quasi-Newton methods, motivation and theory, SIAM Review, 19 (1977) 46-89.

....i s i k 2 s T i B i s i (L r) 2 ffl (k 1) Since B k 1 is positive de nite, tr B k 1 0. The last inequality implies (3.4) 2 Lemma 3.3 There is a constant c 1 0 such that for su ciently large k, k Y i=0 i c k 1 : 3:5) Proof By means of the formula (see e.g. Lemma 7. 6 of [5]) det(I u 1 u T 2 u 3 u T 4 ) 1 u T 1 u 2 ) 1 u T 3 u 4 ) u T 1 u 4 ) u T 2 u 3 ) taking determinant in (2.13) we get det B k 1 = detfB k (I s k s T k B k s T k B k s k B 01 k y k y T k y T k s k )g = det B k det(I s k (B k s k ) T s T k B k s k B 01 ....

J.E. Dennis and J.J. More, Quasi-Newton methods, motivation and theory, SIAM Rev. 19 (1977) 46-89.

....is the exact convergence rate of the steepest descent method. By imposing further conditions on the functions k , however, so that the obtained search direction approaches the Newton direction in the limit, giving quasi Newton methods, stronger convergence rate results may be achieved (e.g. [28]) For these methods, the convergence rate analysis made above is too conservative, since fB k g is not related to r 2 f(x ) 7 Extensions and further research In this paper we have introduced the principle of cost approximation as a means to characterize iterative descent algorithms for ....

J. E. Dennis and J. J. Mor' e, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46--89.

....smaller than any of the quotients kEk F = kAk F . 2 We wish now to establish a lower bound j inf A A ( x) for the backward error. We base our analysis on the backward error for a symmetric system subject to symmetric perturbations: such a formula has been obtained by Dennis and Mor e [5] in the context of secant methods for unconstrained optimization (symmetric secant update of Powell) 6] The following theorem interprets their results in terms of backward error analysis. Theorem 3.3 Let A 2 IR n Thetan be a symmetric matrix, x and b be two vectors of IR n . Let r = b ....

J. E. Dennis and J. J. Mor'e. Quasi-newton methods, motivation and theory. SIAM Rev., 19:46--89, 1977.

....interpolatory condition L k (x k Gamma1 ) F (x k Gamma1 ) k = 1; 2; This condition is equivalent to the secant equation B k 1 (x k 1 Gamma x k ) F (x k 1 ) Gamma F (x k ) k = 0; 1; 2; For n 1, there exist infinite many matrices B that satisfy the secant equation. See Dennis and Mor e [1977]. In general, secant methods do not need computation of derivatives and the resolution of the system B k s k = GammaF (x k ) is not expensive, due to suitable updating procedures. The best known class of secant methods is the least change secant update (LCSU) family (Dennis and Schnabel [1979, ....

Dennis, J, E. and Mor'e, J. J. [1977]: Quasi-Newton methods, motivation and theory, SIAM Review 19, pp. 46-89.

....x; y 2 R n there is a nonsingular matrix B such that x = By. Thus the iterates generated by the DDP algorithm can be regarded as B k d k ddp = Gammag k for some nonsingular matrix. Since the DDP algorithm is quadratically convergent, the theorem follows by Theorem 3. 1 of Dennis and Mor e [7]. 2 We note that (1) shows that d k ddp approaches d k n in both length and direction. Both Procedure 1 and Procedure 2 have the same computational complexity (we ignore the lower order terms) 3] N Delta (2n 3 y 7 2 n 2 y n x 2n y n 2 x 1 3 n 3 x ) 2) though in Procedure 2 ....

....such that d k = Gamma(B k ) Gamma1 g k . We will show that (3) is equivalent to k[B k Gamma H ] x k 1 Gamma x k )k 3 kx k 1 Gamma x k k 2 ; 5) for all sufficiently large k and some 3 , which in turn is obviously equivalent to (4) by Lemma 3. 2 of Dennis and Mor e [7]. Assume first that (5) holds. Since [B k Gamma H ] x k 1 Gamma x k ) g k 1 Gamma g k Gamma H (x k 1 Gamma x k ) Gamma g k 1 ; 6) the continuity of H at x and (4) imply that kg k 1 k flkx k 1 Gamma x k k 2 ; 7) for large k, where fl is a constant. ....

J. E. Dennis and J. J. Mor'e. Quasi-Newton methods, motivation and theory. SIAM Rev., 19:46--89, 1977.

....differences followed by a summation of m n Theta n matrices is often very high. It is therefore well motivated to use quasi Newton methods for such problems. Quasi Newton methods have shown to be very efficient in nonlinear optimization and they play an important role in many implementations [5, 9]. Although, with the use quasi Newton methods instead of the Newton method the local convergence rate is decreased from q quadratic to as best q superlinear, its total computational efficiency is often superior to the analytic Newton method. The objective of this paper is to study and propose such ....

J. E. Dennis and J. J. Mor'e. Quasi-Newton methods, motivation and theory. SIAM Review, 19:46--89, 1977. 238 Paper V

....affine transformation and Gamma k X j=1 ln ffi k Gamma k X j=1 ln k c for all k, where c is some constant. the theorem thus follows from Theorem 2.7, Theorem 3.2 of Byrd and Nocedal [2] Corollary 2.3 of Dennis and Mor e [4] and the remarks below Theorem 8. 9 of Dennis and Mor e [5]. 2 We note that for the BFGS method one can prove [3] tr(B k 1 ) tr(B k ) Gamma kB k s k k 2 s T k B k s k y T k y k y T k s k ; 23) det(B k 1 ) det(B k ) Delta ( s T k y k s T k B k s k ) 24) Lemma 2.4 and (24) show that the modified BFGS method corrects small ....

J. E. Dennis and J. J. Mor'e. Quasi-Newton methods, motivation and theory. SIAM Rev., 19:46--89, 1977.

....in H 0 is an effect of the scaling dependence of the EN method (which is also considered in [51] We develop scaling invariant versions of the EN method in Section 2.4. 2. 2 The Family of Broyden Methods An important class of methods based on rank k updates is the class of quasi Newton methods [10]. The purpose of quasi Newton methods is to determine a zero of a function F or minimize a function G using approximations of the Jacobian of F , or the Hessian of G, or their inverses. There are several effective quasi Newton methods, including the Davidon FletcherPowell method and the BFGS ....

....The purpose of quasi Newton methods is to determine a zero of a function F or minimize a function G using approximations of the Jacobian of F , or the Hessian of G, or their inverses. There are several effective quasi Newton methods, including the Davidon FletcherPowell method and the BFGS method [10]. These methods assume symmetric (and often positive definite) matrices, and we do not consider them here, since our goal is to solve nonsymmetric linear systems. Instead, we focus on variants of Broyden s method, a quasi Newton method that is suitable for solving nonsymmetric linear systems ....

J. Dennis, Jr., J. More, Quasi-Newton methods, motivation and theory, SIAM Review 1(1977), 46-89.

....discretized elliptic boundary value problems also take the form of (1.1) with symmetric Jacobian (see Chapter 1 in [15] DFP method is a well known quasi Newton method for solving (1. 1) Like many other quasi Newton methods, DFP method possesses local superlinear convergence property (see e.g. [5]) Moreover, when used to solve convex unconstrained optimization problems, it converges globally and superlinearly if exact line search is used (see [16] A remarkable result due to Dixon [6] shows that when exact line search is used, Broyden s class of quasi Newton methods including DFP method ....

J.E. Dennis and J.J. More, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977) 46-89.

....interpolates the current and preceding iterands and associated gradients. That Pm should be symmetric is due to the symmetry of the true Hessian. As is well known, the positive definiteness property yields good local convergence results. Excellent references in the literature on this field are [5, 8]. Fitting a quadratic model to the current and preceding iterands clearly utilizes some information about f(x) obtained during the course of the algorithm. A natural extension of this idea is to first define the real n by m matrices: Fm : x m Gamma xm Gamma1 ) x 1 Gamma x 0 ) ....

J.E. Dennis Jr. and J.J. More, Quasi-Newton Methods, Motivation and Theory, SIAM. Review, 19:46--89 (1977).

....Matrix algorithms to solve (2) can be considered as the inner iteration and the Newton step (1) as the outer iteration of a nested iteration scheme. The outer iteration viewpoint has been helpful to establish variants of Newton s method with favourable global convergence properties ( 1] [4], 11] 12] and the numerical effect on the outer iteration depending on the tolerances of the inner iteration has been thoroughly analysed for inexact Newton Methods ( 3] 7] However, little has been said about the influence of the outer iteration on the properties of the inner iteration ....

J.E. Dennis, J.J. Mor'e, Quasi-Newton Methods, Motivation and Theory, SIAM Review, Vol. 19, No. 1 (1977), pp. 46-89.

.... is trapped (e.g. GLL86] The ability of an algorithm to attain long steps is, however, often associated with a high convergence rate; for Newton type methods, the attainment of unit steps after at most a finite number of iterations characterizes their superlinear convergence rate (e.g. [DeM74, DeM77]) Further, if a line search can be avoided then it will result in substantial time savings when the objective function is difficult to evaluate, as in applications to problems in control theory. It therefore seems natural to construct nonmonotone techniques, that is, methods based on rules which ....

J. E. Dennis and J. J. Mor' e, Quasi-Newton methods, motivation and theory, SIAM Review, 19 (1977), pp. 46--89.

....there has been significant progress in the theoretical study on quasi Newton methods, especially in the local convergence analysis. Early work on local convergence analysis for quasi Newton methods can be found in [2] and [3] A systematic and comprehensive review is given by Dennis and Mor e [4]. On the contrary, the study on global convergence of quasi Newton methods is scarce. The main difficulty seems to lie in the lack of effective line searches. Line searches that require the calculation of derivatives are inappropriate for quasi Newton methods. Therefore, to develop a global ....

J.E. Dennis, Jr. and J.J. Mor' e, Quasi-Newton methods, Motivation and the theory, SIAM Rev., 19 (1977), pp. 46-89.

....the unconstrained flow control problem and worked with a nonlinear conjugate gradient method for its solution. In the following, we want to solve the same problem, but with the control subject to pointwise bound constraints. We then apply our inexact semismooth Newton methods and use BFGS updates [41, 42] to approximate the Hessian of the reduced objective function. Therefore, in the following we restrict the control by pointwise bound constraints (with the realistic interpretation that we are only allowed to inject or draw off fluid with a certain maximum speed) and arrive at the following flow ....

J. E. Dennis, Jr. and J. J. Mor6, Quasi-Newton methods, motivation and theory, SIAM Rev., 19 (1977), pp. 46-89.

....in Step 4 of Algorithm 1, B k 1 is updated by the Broyden like formula B k 1 = B k k (y k Gamma B k s k )s T k ks k k 2 ; 3. 4) where s k = z k 1 Gamma z k and y k = H k (z k 1 ) Gamma H k (z k ) The parameter k is chosen to satisfy j k Gamma 1j and B k 1 is nonsingular [9]. We call the algorithm with (3.4) Broyden like method. To prove global convergence of Broyden like method, we need the following assumption. Assumption A (i) The level set Omega defined by (2.9) is contained in a bounded convex set D. ii) For each ffl 0, function H ffl is continuously ....

J.E. Dennis, Jr. and J.J. Mor'e, Quasi-Newton methods, Motivation and the theory, SIAM Rev., 19 (1977), 46--89.

....Since this matrix does not contain necessarily derivative information at x k , the quasi Newton approach is appealing for solving nonsmooth systems. The local convergence theory of quasi Newton methods for continuously differentiable nonlinear systems is well established. See [10] 1] 7] [8], 11] 16] 17] In the last few years, some authors addressed the problem of developing a suitable theory for particular classes of nonsmooth problems ( 2] 19] 15] and others showed that, in practice, quasi Newton algorithms are effective for solving many nondifferentiable systems [14] ....

....from singularity (see [12] In this case, the BFGS updating seems to be adequate. In the following lemma we prove that the distance between P xz (E) and E cannot be much larger than the distance between E and E . Results of this type are known as Bounded Deterioration Principles (see [1] [8], 10] 16] 17] Lemma 3.1 Let F; V; E and E satisfy Assumptions 2 and 3. Then, there exist positive constants c 3 ; c 4 such that, for all x; z 2 Omega and E 2 E, kP xz (E) Gamma E k [1 c 4 oe(x; z) q ]kE Gamma E k c 3 oe(x; z) p : 3.13) Proof. See the proof of Lemma 3.1 in ....

Dennis Jr., J. E. and Mor'e, J. J., Quasi-Newton methods, motivation and theory, SIAM Review 19 (1970) 46-89.

No context found.

J. Dennis and J. More. Quasi-newton methods, motivation and theory. SIAM Review, 19:46---89, 1977.

No context found.

J. E. Dennis, Jr. and J. J. Mor e, Quasi-Newton methods, motivation and theory, Siam Review, 19 (1977), pp. 46--89.

No context found.

J.E. Dennis, Jr., and J.J. More, \Quasi-Newton methods, motivation and theory", SIAM Review, Vol. 19, pp.46-89, 1977.

No context found.

J.E. Dennis and J.J. More, Quasi{Newton methods, motivation and theory, SIAM Review,vol. 19, pp.46-89, 1977.

No context found.

J.E. Dennis and J.J. More, Quasi-Newton methods, motivation and theory, SIAM Review, 19 (1977) 46-89.

No context found.

Dennis Jr., J.E.; Mor'e, J.J. [1977]: Quasi-Newton methods, motivation and theory, SIAM Review 19, pp. 46-89.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC