Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput., 17:16-32, 1996.  Home/Search   Document Details and Download   Summary   Related Articles   Check

.... using iterative methods (the inner iterations) Although an iterative method can reduce the complexity, it may oversolve the approximate Jacobian equation in the sense that the last tens or hundreds inner iterations before convergence may not improve the convergence of the outer Newton iterations [2]. The inexact Newton like method is a method that stops the inner iterations before convergence. By choosing suitable stopping criteria, we can reduce the total cost of the whole inner outer iterations. In essence, one does not need to solve the approximate Jacobian equation exactly in order that ....

.... Jacobian equation (6) is solved exactly, see [4] However, for general Newton like methods, it is known that one can solve the approximated Jacobian equation inexactly (e.g. using an iterative method with a suitably chosen stopping criterion) and still retain fast convergence, see for instance [2]. The main aim of this paper is to derive a computable stopping criterion and establish the convergence rate. For general nonlinear equation f (c) 0, the stopping criterion of inexact Newton methods is usually given in terms of f (c) see for instance [2, 3, 5] By (2) this will involve ....

[Article contains additional citation context not shown here]

S. Eisenstat and H. Walker, Choosing the Forcing Terms in an Inexact Newton Method, SIAM J. Sci. Comput., 17 (1996), 16--32.

....shown that if #N # # then linearly; if #N # # then # # # superlinearly; and if #N # ###### then we recover the quadratic convergence rate of a Newton method. The forcing term is usually given by #N # ## ### # # #### # (17) For other choices of #N and details on inexact Newton methods, see [10] and the references therein. The extension of inexact methods to optimization is immediate, especially for unconstrained optimization. In [17] a global analysis is provided for a trust region RSQP based algorithm. Close to a KKT point the theory for Newton s method applies and one can use the ....

....satisfy the line search conditions, we then switch to QN RSQP. If QN RSQP fails too, then we reduce the continuation parameter Re and return to the outer loop. The linear solves in Steps 8, 16 and 17 are performed inexactly (that is, by early termination of iterative solvers) In Step 8 we follow [10] in choosing the forcing term. In Steps 16 and 17 the forcing term is based on the formulas developed in Section 4.3. In Step 6 we use the adjoint variables to update the reduced gradient. This is equivalent to # z # # d # # s # s ,if# is computed by solving exactly # # s # #.When# is taken ....

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM Journal on Scientific Computing, 17(1):16--32, 1996.

.... a Krylov Schwarz domain decomposition algorithm for the finite element solution of the nonlinear full potential equation of aerodynamics, extending our model studies of linear convection diffusion problems in [5] and of linear aerodynamic design optimization problems in [33] Newton Krylov methods [2, 3, 14, 15, 39] are potentially well suited and increasingly popular for the implicit solution of nonlinear problems whenever it is expensive to compute or store a true Jacobian. We employ a combined algorithm, called Newton Krylov Schwarz, and focus on the interplay of the three nested components of the ....

....convergence, the three components have to be tuned simultaneously. We discuss these components in turn. 4. 1 The matrix free Newton method In this subsection, we briefly discuss the well known matrix free inexact finite difference Newton algorithm, and the Eisenstat Walker forcing functions [15]. Starting from an initial guess Phi 0 , which is sufficiently close to the solution, a solution of the nonlinear system (9) is sought by using an inexact Newton method: For some j k 2 [0; 1) find s k that satisfies kF ( Phi k ) J ( Phi k )s k k j k (14) and set Phi k 1 = Phi k k s k ; ....

[Article contains additional citation context not shown here]

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16-32.

.... one may employ iterative methods to solve both systems (the inner iterations) Although iterative methods can reduce the complexity, it may oversolve the systems in the sense that the last few inner iterations before convergence may not improve the convergence of the outer Newton iteration, see [2]. The inexact Newton like method is a method that stops the inner iteration before convergence. By choosing suitable stopping criteria, we can minimize the total cost of the whole inner outer iteration. rchan math.cuhk.edu.hk) Department of Mathematics, Chinese University of Hong Kong, Shatin, ....

....to reduce the cost of the inner iterations. It is interesting to know how accurately one has to solve these systems in order to retain the superlinear convergence rate of the whole algorithm, and this is the main thesis of this paper. For a general nonlinear equation g(x) 0, it was shown in [2] that the Jacobian equation J(x ) x ) Gammag(x ) 8) need not be solved exactly. In fact, if (8) is to be solved by an iterative method, then the last few iterations before convergence are usually insignificant as far as the convergence of 3 the (outer) Newton iteration is concerned. ....

[Article contains additional citation context not shown here]

S. Eisenstat and H. Walker, Choosing the Forcing Terms in an Inexact Newton Method, SIAM J. Sci. Comput., 17 (1996), pp. 16--32.

.... a Krylov Schwarz domain decomposition algorithm for the finite element solution of the nonlinear full potential equation of aerodynamics, extending our model studies of linear convection diffusion problems in [5] and of linear aerodynamic design optimization problems in [34] Newton Krylov methods [2, 3, 15, 16, 40] are potentially well suited and increasingly popular for the implicit solution of nonlinear problems whenever it is expensive to compute or store a true Jacobian. We employ a combined algorithm, called Newton KrylovSchwarz, and focus on the interplay of the three nested components of the ....

....convergence, the three components have to be tuned simultaneously. We discuss these components in turn. 4.1. The matrix free Newton method. In this subsection, we briefly discuss the well known matrix free inexact finite difference Newton algorithm, and the Eisenstat Walker forcing functions [16]. Starting from an initial guess Phi 0 , which is sufficiently close to the solution, a solution of the nonlinear system (7) is sought by using an inexact Newton method: For some jk 2 [0; 1) find sk that satisfies kF ( Phi k ) J ( Phi k)skk jk (10) 5 and set Phi k 1 = Phi k ksk ; where ....

[Article contains additional citation context not shown here]

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16-32.

....special structure such as Toeplitz or Hankel matrices. Finally, we point out that in many applications A is sparse and should therefore no longer be transformed to a condensed form. Instead, one can use iterative solvers to find a solution z i to (6.10) for each right hand side x i . It turns out [10] that a high relative accuracy of (6.10) is only needed in the last few steps of the iteration and hence that a lot of flexibility can be built into the iterative procedure. Our recommendation is thus to use this algorithm on a block tridiagonal form of A when A is dense and not to reduce A at ....

S. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16--32.

....# ##### p # ### # p # # #x ##### : 3) 3 SOLUTION METHOD An inexact Newton Krylov method is our nonlinear solver. Our use of this method is described by Woodward in (Woodward 1998) and includes the use of dynamic selection of linear system tolerances with a method of Eisenstat and Walker (Eisenstat Walker 1996) and the line search globalization strategy given by Dennis and Schnabel (Dennis Schnabel 1983) We also use the restarted version of the GMRES Krylov iterative solver developed by Saad and Schultz (Saad Schultz 1986) to solve the linear Jacobian systems. The main advantage of using a Krylov ....

Eisenstat, S. C. &Walker, H. F. (1996). Choosing the forcing terms in an inexact Newton method. SIAM J. Sci.

.... F # (u (k) p (k) # # # k #F (u (k) # (1.2) Step 2: Compute the new approximate solution u (k 1) u (k) # (k) p (k) 1. 3) Here # k # [0, 1) is a scalar that determines how accurately the Jacobian system needs to be solved using, for example, Krylov subspace methods [2, 3, 12, 13]. # (k) is another scalar that determines how far one should go in the selected inexact Newton direction [8] We comment on our strategy for selecting # (k) in section 4. IN has two well known features, namely, a) if the initial guess is close enough to the desired solution then the ....

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16-32.

....special structure such as Toeplitz or Hankel matrices. Finally, we point out that in many applications A is sparse and should therefore not be transformed anymore to a condensed form. Instead, one can use iterative solvers to nd a solution z i to (6.10) for each right hand side x i . It turns out [10] that a high relative accuracy of (6.10) is only needed in the last few steps of the iteration and hence that a lot of exibility can be built into the iterative procedure. Our recommendation is thus to use this algorithm on a block tridiagonal form of A when A is dense and not to reduce A at all ....

S. Eisenstat and H. Walker, Choosing the forcing terms in an inexact newton method, SIAM J. Scientic Comp., 17 (1996), pp. 16-32.

.... p (k) such that kF (u (k) F 0 (u (k) p (k) k k kF (u (k) k; 5) and then the new approximate solution u (k 1) u (k) k) p (k) Here k is a scalar that determines how accurately the Jacobian system needs to be solved 2 using, for example, Krylov subspace methods [3, 4, 11, 12]. k) is another scalar that determines how far one should go in the selected inexact Newton direction [8] IN has two well known properties. First, if the initial guess is close enough to the desired solution then the convergence is very fast. Second, such a good initial guess is generally ....

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16-32.

....solution, and thus the linear approximation made by Newton s method can be very inaccurate. Solving the linear system of equations very accurately does not, in general, equally reduce the residual of the non linear system of equations which causes a waste of CPU time. This is known as oversolving [24]. Strategies for choosing a sequence of inner tolerances that lead to an efficient rate of convergence of the inexact Newton method without oversolving have been developed by several authors, e.g. 24] For our applications, we found that using high inner tolerances (i.e. not solving the linear ....

....the non linear system of equations which causes a waste of CPU time. This is known as oversolving [24] Strategies for choosing a sequence of inner tolerances that lead to an efficient rate of convergence of the inexact Newton method without oversolving have been developed by several authors, e.g. [24]. For our applications, we found that using high inner tolerances (i.e. not solving the linear system accurately) gives slightly better efficiency. We reduce the inner residual by a factor of two for the first 10 outeriterations. This guarantees no oversolving for most problems. We then reduce ....

Eisenstat, S.C., and Walker, H.F., "Choosing the Forcing Terms in an Inexact Newton Method," SIAM J. Sci. Comput., vol. 17, pp. 16--32, January 1996.

....solution is far from the converged solution, and thus the linear approximation of F can be very inaccurate. The use of a strict tolerance (i.e. a small value of j k ) during these early iterations is thus not beneficial to the rate of convergence and wastes CPU time. This is known as oversolving [29]. Strategies for choosing a sequence of j k s leading to an efficient rate of convergence of the inexact Newton method without oversolving have been developed by several authors [29] We found that choosing j k = fl kF (Q k )k kF (Q k Gamma1 )k ff (4) with fl 2 [0; 1] ff 2 (1; 2] and ....

....iterations is thus not beneficial to the rate of convergence and wastes CPU time. This is known as oversolving [29] Strategies for choosing a sequence of j k s leading to an efficient rate of convergence of the inexact Newton method without oversolving have been developed by several authors [29]. We found that choosing j k = fl kF (Q k )k kF (Q k Gamma1 )k ff (4) with fl 2 [0; 1] ff 2 (1; 2] and j 0 2 [0; 1) is an effective strategy. However, for our applications, we have found the following approach to give slightly better efficiency. We use j k = 0:5 for the first 10 ....

Eisenstat, S.C., and Walker, H.F., "Choosing the Forcing Terms in an Inexact Newton Method," SIAM J. Sci. Comput., vol. 17, pp. 16--32, January 1996.

....special structure suchasToeplitz or Hankel matrices. Finally,we point out that in many applications A is sparse and should therefore not be transformed anymore to a condensed form. Instead, one can use iterative solvers to find a solution z i to (26) for each right hand side x i . It turns out [EW96] that a high relative accuracy of (26) is only needed in the last few steps of the iteration and hence that a lot of flexibility can be built into the iterative procedure. Our recommendation is thus to use this algorithm on ablock tridiagonal form of A when A is dense and not to reduce A at all ....

S. Eisenstat and H. Walker, Choosing the forcing terms in an inexact newton method, SIAM J. Scientific Comp. 17 (1996), 16--32.

.... Krylov Schwarz domain decomposition algorithm for the finite element solution of the nonlinear full potential equation of aerodynamics, extending our model studies of linear convection di#usion problems in [5] and of linear aerodynamic design optimization problems in [34] Newton Krylov methods [2, 3, 15, 16, 40] are potentially well suited and increasingly popular for the implicit solution of nonlinear problems whenever it is expensive to compute or store a true Jacobian. We employ a combined algorithm, called Newton Krylov Schwarz (NKS) and focus on the interplay of the three nested components of the ....

....convergence, the three components have to be tuned simultaneously. We discuss these components in turn. 4.1. The matrix free Newton method. In this subsection, we briefly discuss the well known matrix free inexact finite di#erence Newton algorithm, and the Eisenstat Walker forcing functions [16]. Starting from an initial guess # 0 , which is su#ciently close to the solution, a solution of the nonlinear system (7) is sought by using an inexact Newton method: for some # k # [0, 1) find s k that satisfies #F(# k ) J(# k )s k # # # k (10) and set # k 1 = # k # k s k , where # k ....

[Article contains additional citation context not shown here]

S. C. EISENSTAT AND H. F. WALKER, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16--32.

....rather than decrease, backtracking will not accept the update. Instead, the algorithm looks in the same direction as the solution update from the Newton iteration. Performing residual calculations along the solution path given by this direction, it finds the solution that minimizes the residual [9, 10]. Backtracking has been shown in some cases to help converge to a steady state when Newton s method without backtracking failed. The default flag disables backtracking for transient runs but enables backtracking for all other solution types (pseudo, steady, and continuation) Default = default. ....

....Absolute Error Tolerance = float] These two flags set the tolerances that are used in calculating the convergence criterion for the update vector in the nonlinear solver. This criterion is , 3. 1) Flag Value Choice for eta k in Inexact Newton s Method 0 1 Eisenstat and Walker, Method 1 [9, 10] 2 Eisenstat and Walker, Method 2a 3 Eisenstat and Walker, Method 2b 4 Linear Solver Normalized Residual Tolerance ( Exact Newton ) Table 3.2. This table summarizes the choices for the Inexact Newton forcing term. The variable eta k is the required drop in the linear residual for a successful ....

S.C. Eisenstat and H.F. Walker. "Choosing the forcing terms in an inexact Newton method," SIAM J. Sci. Comput., 17 (1996) 16-32.

....the Newton corrections, which can in turn permit 2 approximation of the Jacobian matrix) over a sequence of Newton iterations, while still converging quadratically. This theory was revisited to provide inexpensive, constructive formulae for the sequence of inexact tolerances by Eisenstat Walker [25]. Smooke [72] and Schreiber Keller [69] devised Newton chord methods with models for cost effective frequency of Jacobian reevaluation. The use of various approximate Newton methods in CFD emerged independently in various regimes. Vanka [82] implemented Newton solvers in primitive variable ....

....asymptotic convergence of the true Newton method. Inexact Newton methods require a strategy for terminating the inner linear iterations, in effect choosing j , in jjf(u Gamma1 ) f 0 (u Gamma1 ) u Gamma u Gamma1 )jj j jjf(u Gamma1 )jj : 2. 7) One of the Eisenstat Walker [25] criteria is j = fi fi jjf(u Gamma1 )jj Gamma jjf(u Gamma1 ) f 0 (u Gamma1 ) u Gamma u Gamma1 )jj fi fi jjf(u l Gamma1 )jj : 2.8) 6 Ajmani et al. 1] adopt: j = log ) Gamma1 . Venkatakrishnan Mavriplis [86] adaptively choose j so that work and ....

[Article contains additional citation context not shown here]

S. C. Eisenstat and H. F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Scientific Computing, 17:16--32, 1996.

No context found.

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput., 17:16-32, 1996.

No context found.

S. C. Eisenstat and H. F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput., 17:16-32, 1996.

No context found.

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput., 17:16--32, 1996.

No context found.

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comput., 17:16--32, 1996.

No context found.

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16--32.

No context found.

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact newton method. SIAM Journal on Scientific Computing, 17(1):16--32, January 1996.

No context found.

S. C. Eisenstat and H. F. Walker, Choosing the forcing terms in an inexact Newton method, SIAM J. Sci. Comput., 17 (1996), pp. 16--32.

No context found.

Stanley C. Eisenstat and Homer F. Walker. Choosing the forcing terms in an inexact Newton method. SIAM J. Sci. Comp., 17(1):16-32, January 1996.

No context found.

Eisenstat, S.C., and Walker, H.F., Choosing the forcing terms in an Inexact Newton method, SIAM. J. Sci Comput. 17-1, 16-32, 1996.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC