P.C. HANSEN and D.P. O'LEARY. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput., 14(6):1487--1503, 1993.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....an effective criterion for selecting its optimal value. For the Tikhonov method, in the absence of a priori information on the variance of the noise on the data, the Generalized Cross Validation (GCV) is the most used criterion. For the TSVD and TGSVD, the Picard condition and the L curve rule [17] are the most popular tests for choosing the best value of the regularizing parameter k. The L curve rule consists in plotting in log log scale the values of the norms kHy k k versus the residuals kAy k Gamma bk for k = 1; p. The resulting curve is typically L shaped. The chosen value of ....

....parameter k. The L curve rule consists in plotting in log log scale the values of the norms kHy k k versus the residuals kAy k Gamma bk for k = 1; p. The resulting curve is typically L shaped. The chosen value of the parameter k is the one which characterizes the corner of the L [17]. Since, in our numerical experiments, the L curve has been, on the average, more effective than the Picard condition, in this paper all comments and results concerning the optimal choice of k, will refer to the L curve. With respect to the extrapolation methods, the L curve proved to be quite ....

P.C. Hansen, D.P. O'Leary, The use of the L--curve in the regularization of discrete ill-- posed problems, SIAM J. Sci. Stat. Comput., 14 (1993) 1487-1503.

....for example. For a more detailed description of the methods, we refer the reader to [37] 31] and [15] and the references therein. Early surveys of regularization methods appeared in [14] 5] and [76] A common framework for the study of numerical regularization methods is proposed in [37] and [38], and more recently in [58] 19 3.2.1 Methods for Rank Deficient Problems As we mentioned before, the coefficient matrix of a rank deficient problem has a singular value spectrum such that the smallest singular values are clearly separated from the rest. This feature makes it possible to ....

....in Tikhonov regularization. 10 6 10 5 10 4 10 3 10 2 10 1 10 0 10 4 10 2 10 0 10 2 10 4 10 6 10 8 log ( A x b ) Figure 3.5 The L curve for Tikhonov Regularization. The use of this curve to estimate the regularization parameter has been studied in [35] and [38]. The idea is to interpolate the curve in order to estimate the corner . The L curve criterion performs better than the GCV criterion when the noise in the data is correlated and comparably well for white noise. The advantage of the L curve criterion over the GCV in the presence of correlated ....

[Article contains additional citation context not shown here]

P.C. Hansen and D.P. O'Leary. The use of the L--curve in the regularization of discrete ill--posed problems. SIAM J. Sci. Comput., 14(6):1487--1503, 1993.

....the survey of regularization methods by Hanke and Hansen [52] as well as the recent book by Hansen [58] Several methods have been proposed. The discrepancy principle places an upper bound on the residual jjg Gamma Kf jj 2 and has been studied extensively; for a recent presentation, see [59]. While the discrepancy principle is simple to implement, it does have its disadvantages. One is that a good upper bound may not be known. Another is that it often oversmooths the solution. That is, it chooses a Tikhonov parameter too large, drops too many singular values in the tsvd, or ....

....it does have its disadvantages. One is that a good upper bound may not be known. Another is that it often oversmooths the solution. That is, it chooses a Tikhonov parameter too large, drops too many singular values in the tsvd, or causes iterative regularization methods to stop too early [59]. Another approach for choosing regularization parameters is the discrete Picard condition. As the svd analysis shows, the noise in g will dominate the coefficients g T u i which correspond to the small singular values. The discrete Picard condition states that in order to compute a reasonable ....

[Article contains additional citation context not shown here]

P. C. Hansen and D. P. O'Leary. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput., 14:1487--1503, 1993.

....Figure B.1: Schematic L curve Now we address the problem of computing Deltas = arg min[k J Deltas Gamma b k 2 2 2 k Deltas 2 k 2 2 ] B.4) where b = Psi Gamma Phi(s k ) 34 The regularization parameter is chosen by means of the L curve (see figure B. 1) as described in [20, 19, 21]. This curve is a plot of the norm of the solution k Deltas k 2 versus the norm of the residual k J Deltas Gamma b k 2 , where is a positive parameter. Let us consider the singular value decomposition of the Jacobian matrix J = U GammaV T ; where U 2 I R NM ThetaN M ; V 2 I R ....

Hansen, P. C., O'Leary, D. P., The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput. 14, 1993, pp. 1487-1503.

....for this method in the next release. Figure 4.4: The TSVD menu from the menu bar. 2. Lavrent yev (Fig. 4.5) starts the Lavrent yev regularization method. The related regularization parameter can be computed by using one of the following approaches: L Curve: it uses the L curve method [6, 8] which is quite accurate, but it needs long time for the execution; GCV: it uses the generalized cross validation method; Customize: you introduce a regularization parameter by typing a custom value. The value must be a positive number: the default is 0:1. 3. CG type starts one of the four ....

D. P. O'Leary, P. C. Hansen, The use of the L{curve in the regularization of discrete ill{posed problems, SIAM J. Sci. Comp. 14 (1993), pp. 1487-1503.

.... of the noise in the right hand side y, the Discrepancy Principle leads to essentially optimal values of [3] On the other hand if only a rough estimate of the noise norm is known, the Generalized Cross Validation (GCV) method [4] or more heuristic approaches such as the L curve criterion [5] [6] can be used. The regularization matrix Q is relative to the smoothness of the solution and usually it is the identity matrix or the discretization of a linear differential operator. If Q = I, the method is given in standard form: min k Kx Gamma y k 2 k x k 2 (11) where the solution x can be ....

P.C. Hansen, D.P. O'Leary, The use of the L--curve in the regularization of discrete ill--posed problems, SIAM J. Sci. Comput. vol.14, no.6, pp 1487--1503 (1993).

.... rule is to use the discrepancy principle which sets the residual norm equal to some upper bound for the norm of the noise term kfflk 2 [5] Other techniques, such as a version of the Generalized Cross Validation which takes into account the particular nature of the PSF [20] or the L curve method [16], can be included in the solution process. A common choice of the matrix C consists in a discrete differential operator as smoothing condition [15] 21] When Euclidean norm is used, problem (3)can be written as: min f 1 2 f t Af Gamma f t H t g (4) where A = H t H flC t C is a ....

D. P. O'Leary P.C.Hansen. The use of the L--curve in the regularization of discrete ill--posed problems. SIAM J. Sci. Comp., 14:1487--1503, 1993.

....its index lies between 2 and N Gamma 2. A corner at k = 1; N Gamma 1 is a clear sign that only one branch of the L curve has been found, and further points should be computed to see whether the corner moves or becomes proper. 3 An envelope guided conjugate gradient algorithm Hansen O Leary [8] suggest a scheme for determining the point of maximum curvature of the Tikhonov L curve, by calculating suitable points along the L curve, using a bracketing scheme to find the point of maximum curvature. Clearly, their algorithm assumes that the calculation of the optimum x of the penalty ....

P.C. Hansen and D.P. O'Leary, The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems, SIAM J. Sci. Computing 14 (1993), 1487-1506.

.... ffi ) of Geman McClure [1] which all have an additional parameter ffi . The merit of some of these functions and the need for algorithms to determine good values for ffi and are described by Lalush Tsui [10] II. The envelope guided conjugate gradient algorithm The L curve approach of [5] for determining a good value of to be used in (3) is simple to use for small problems and is more robust than the standard generalized crossvalidation approach of [16] when the noise is correlated with the signal of the data. Because it entails minimizing (3) for many values of , it is too time ....

P.C. Hansen and D.P. O'Leary, The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems, SIAM J. Sci. Computing 14 (1993), 1487-1506.

....least squares case is treated in [74] and [15] for example. For a more detailed description of the methods, we refer the reader to [34] Early surveys of regularization methods appeared in [4] and [73] A common framework for the study of numerical regularization methods is proposed in [34] and [35]. 3.2.1 Methods for Rank Deficient Problems As we mentioned before, the coefficient matrix A of a rank deficient problem has a well determined gap in the singular value spectrum, which makes it possible to determine the numerical rank of the matrix. The numerical rank r (A) of the matrix ....

.... of the curve. Figure 3.5 shows the L curve (and its corner) for problem heat from [33] In this example, the curve is based on the values of the regularization parameter in Tikhonov regularization. The use of this curve to estimate the regularization parameter has been studied in [32] and [35]. The idea is to interpolate the curve in order to estimate the corner . The L curve method performs better than the GCV method when the noise in the data is correlated and comparably well for white noise. The advantage of the L curve criterion over the GCV in the presence of correlated noise ....

[Article contains additional citation context not shown here]

P.C. Hansen and D.P. O'Leary. The use of the L--curve in the regularization of discrete ill--posed problems. SIAM J. Sci. Comput., 14(6):1487--1503, 1993.

....with a few comments on how to choose in (23) Of course, this is a delicate issue of the algorithm, and we have to admit that we do not have a final answer for this problem. One approach that has been used successfully in [7] for circulant preconditioners, is based on the L curve method (cf. [9]) A direct application of that approach to the Toeplitz preconditioner of this paper would proceed as follows. Given the known right hand side vector y, for each eigenvalue k , k = 0; 1; 2n Gamma 1, denote by Q (k) the preconditioner obtained with = k in (23) and compute jjx ....

....Index of corner: k = 5082 Corresponding : 0.0154 Figure 4: L curves for various approaches for choosing . The corners that determine the value of are indicated by small circles. The corners, indicated by small circles, were determined using the algorithms suggested by Hansen and O Leary [9]. These corners correspond to = 3218 = 0:0202; 6731 = 0:0099; and = 5082 = 0:0154; 29) respectively. Gamma 0 Gamma 0 10 Gamma8 10 Gamma6 10 Gamma4 10 Gamma2 1 Figure 5: Values of f n with contour lines for the three thresholds . To illustrate these thresholds, ....

P. C. Hansen and D. P. O'Leary, The use of the L-curve in regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487--1503.

.... least squares case has been studied in [54] and [11] For a more detailed description of the methods, we refer the reader to [26] Early surveys of regularization methods appeared in [3] and [53] A common framework for the study of numerical regularization methods is proposed in [26] and [27]. 5.1 Methods for Rank deficient Problems We recall from section 1 that the coefficient matrix A of a rank deficient problem has a well determined gap in the singular value spectrum, which makes possible to determine the numerical rank of the matrix. The numerical ffl rank r ffl (A) of the ....

.... corner of the curve. Figure 5 shows the L curve (and its corner) for problem heat from [25] In this example, the curve is based on the values of the regularization parameter in Tikhonov regularization. The use of this curve to estimate the regularization parameter has been studied in [22] and [27]. The idea is to interpolate the curve in order to estimate the corner . The L curve method performs better than the GCV method when the noise in the data is correlated and comparably well for white noise. The advantage of the L curve criterion over the GCV in the presence of correlated noise ....

[Article contains additional citation context not shown here]

P.C. Hansen and D.P. O'Leary. The use of the L--curve in the regularization of discrete ill--posed problems. SIAM J. Sci. Comp., 14(6):1487-- 1503, 1993.

....( t( x) y( x) a( Figure 1: The a curve. 1. Moreover, just as for the L curve, we may, from points fx i g, define an approximating curve. For ill posed linear equations the L curve in the log log scale ( j( log(t(x( log(y(x( has usually a distinct corner. Hansen and O Leary [6] have developed an algorithm for identifying that corner, defined as the point where the curve ( j( has its greatest curvature. It is quite possible to use this approach in the nonlinear case (even if the analysis is different) However, our purpose with the L and a curve is to get global ....

P. C. Hansen and P. O'Leary. The use of the L-curve in the regularization of discrete ill-posed problems. Technical Report UNI*C, UMIACS-TR-91-142, Danish Computing Center for Research and Education, Technical Univ. of Denmark, DK-2800 Lyngby, Denmark, 1991.

....its index lies between 2 and N Gamma 2. A corner at k = 1; N Gamma 1 is a clear sign that only one branch of the L curve has been found, and further points should be computed to see whether the corner moves or becomes proper. 4 An envelope guided conjugate gradient algorithm Hansen O Leary [8] suggest a scheme for determining the point of maximum curvature of the Tikhonov L curve, by calculating suitable points along the L curve, using a bracketing scheme to find the point of maximum curvature. Clearly, their algorithm assumes that the calculation of the optimum x of the penalty ....

P.C. Hansen and D.P. O'Leary, The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems, SIAM J. Sci. Computing 14 (1993), 1487-1506.

....its index lies between 2 and N Gamma 2. A corner at k = 1; N Gamma 1 is a clear sign that only one branch of the L curve has been found, and further points should be computed to see whether the corner moves or becomes proper. 3 An envelope guided conjugate gradient algorithm Hansen O Leary [6] suggest a scheme for determining the point of maximum curvature of the Tikhonov L curve, by calculating suitable points along the L curve, using a bracketing scheme to find the point of maximum curvature. Clearly, their algorithm assumes that the calculation of the optimum x of the penalty ....

P.C. Hansen and D.P. O'Leary, The Use of the L-Curve in the Regularization of Discrete Ill-Posed Problems, SIAM J. Sci. Computing 14 (1993), 1487-1506.

....of the regularization parameter, when the norm of the error e is not explicitly known, is based on the curve L (log #x #, #Ax b#) 0 . 6) This curve is usually referred to as the L curve, because for a large class of problems it is shaped like the letter L. Hansen and O Leary [10, 13] propose to choose the value of the parameter that corresponds to the point at the vertex of the L, where the vertex is defined to be the point on the L curve with curvature # of largest magnitude. We denote this value of the regularization parameter by L . A heuristic motivation for ....

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput. 14 (1993), pp. 1487--1503.

....of the regularization parameter when the norm of the error e is not explicitly known is based on the curve L : f(log kAx bk; log kx k) 0g: 6) This curve is usually referred to as the L curve, because for a large class of problems it is shaped like the letter L. Hansen and O Leary [10, 13] propose to choose the value of the parameter that corresponds to the point at the vertex of the L, where the vertex is de ne to be the point on the L curve of largest curvature . We denote this value of the regularization parameter by L . A heuristic motivation for choosing the value ....

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput. 14 (1993), pp. 1487-1503.

....can arise in practical applications. 1. Introduction Of vital importance in the implementation of regularization methods for ill posed problems is an appropriate choice of the regularization parameter. Recently a parameter selection technique known as the L curve method has gained attention (see [7, 8, 9]) We will consider the implementation of this method for ill posed linear operator equations Af = z (1.1) in a Hilbert space setting. An example of such an operator is the Fredholm first kind integral, Af ] x) Z Omega a(x; y)f(y) dy; x 2 Omega ; 1.2) which is a compact operator on L ....

....The next section deals with operator approximations and their spectral properties. In Section 3, the stochastic error model (1.5) and its ramifications are discussed in detail. In section 4, tools presented in Sections 2 and 3 are applied to analyze the L curve method. Following Hansen and O Leary [9], we characterize the corner of the L curve as the point of maximum curvature. With this characterization and under a few well motivated assumptions, we rigorously prove that the L curve method fails to converge. Finally, Section 5 contains a numerical example to illustrate the analysis and to ....

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., vol. 14 (1993), pp. 1487-1503.

....4.6.1 in [21] The logarithmic scale, on the other hand, emphasizes the difference between the L curves for an exact right hand side b and for pure noise e, and it also emphasizes the two different parts of the L curve for a noisy right hand side b = b e. These issues are discussed in detail in [22]. 5 The curvature of the L curve As we shall see in the next two sections, the curvature of the L curve plays an important role in the understanding and use of the L curve. In this section we shall therefore derive a convenient expression for this curvature. Let j = kx k 2 2 ; ae = kA x ....

....so called L curve criterion for choosing the regularization parameter is one of the few current methods that involve both the residual norm kA x Gamma bk 2 and the solution norm kx k 2 . In order to provide a strict mathematical definition of the corner of the L curve, Hansen and O Leary [22] suggested using the point on the L curve (b ae=2; b j=2) with maximum curvature given by Eq. 18) It is easy to use a one dimensional minimization procedure to compute the maximum of . Various issues in locating the corner of L curves associated with other methods than Tikhonov regularization ....

[Article contains additional citation context not shown here]

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487--1503.

....4.6.1 in [16] The logarithmic scale, on the other hand, emphasizes the difference between the L curves for an exact right hand side b and for pure noise e, and it also emphasizes the two different parts of the L curve for a noisy right hand side b = b e. These issues are discussed in detail in [17]. 5 When the L Curve is Concave In the previous section we made some arguments that an exact righthand side b that satisfies the Discrete Picard Condition, or a righthand side e corresponding to pure white noise, leads to an L curve that is concave when plotted in log log scale. In this section ....

....so called L curve criterion for choosing the regularization parameter is one of the few current methods that involve both the residual norm kA x Gamma bk 2 and the solution norm kx k 2 . In order to provide a strict mathematical definition of the corner of the L curve, Hansen and O Leary [17] suggested to use the point on the L curve (b ae; b j) with maximum curvature. Recall that b ae and b j are functions of , and let b ae 0 , b j 0 , b ae 00 , and b j 00 denote the first and second derivatives of b ae and b j with respect to . Then the curvature of the L curve (b ae; b ....

[Article contains additional citation context not shown here]

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487--1503.

....the norm of the solution much, while for values smaller than this, the norm of the solution increases rapidly without much decrease in residual. In practice, only a few points on the L curve are computed and the corner is located by approximate methods, estimating the point of maximum curvature [19]. Like GCV, this method of determining a regularization parameter does not depend on specific knowledge about the noise vector. 3.4. Disadvantages of these parameter choice algorithms. The appropriate choice of regularization parameter especially for projection algorithms is a difficult ....

....compute kr k k 2 with only a few scalar calculations. Paige and Saunders give a similar method for computing kx k k 2 [29] but, with GMRES, the cost for computing kx k k 2 is O(k 2 ) In using this method or GCV, one must go a few iterations beyond the optimal k in order to verify the optimum [19]. 4.2. Regularization by projection plus TSVD. If projection alone does not regularize, then we can compute the TSVD regularized solution to the projected problem (19) We need the SVD of the (k 1) Theta k matrix B k . This requires O(k 3 ) operations, but can also be computed from the SVD ....

P. C. Hansen and D. P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM J. Sci. Comput., 14 (1993), pp. 1487--1503.

No context found.

P.C. HANSEN and D.P. O'LEARY. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput., 14(6):1487--1503, 1993.

No context found.

P.C. Hansen and D.P. O'Leary, The use of the L-curve in the regularization of discrete ill-posed problems, SIAM Journal on Scientific Computing 14 (1993), pp. 1487--1503.

No context found.

P. C. Hansen and D. P. O'Leary. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput., 14(6):1487-- 1503, November 1993.

No context found.

P. C. Hansen and D. P. O'Leary. The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comp., 14:1487--1503, 1993.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC