G. H. GOLUB and U. von MATT. Generalized cross-validation for large-scale problems. Journal of Computational and Graphical Statistics, 6(1):1--34, March 1997.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....value of fi; see Golub, Heath and Wahba [27, 80] The GCV method requires the computation of quantities that are expensive to evaluate when the system is large. Therefore, this method has primarily been used for small problems that can be solved by direct methods. Recently, Golub and von Matt [29] proposed modifications of the GCV method that allow application to large scale problems. Another technique for determining a suitable value of the regularization parameter when the norm of the error (0.13) is not available is known as the L curve method. This method is based on the observation ....

....[28, Chapter 10] shows that m steps of the standard conjugate gradient algorithm (Algorithm 2.1) yields the same approximate solution in the absence of round off errors. Several applications of the Lanczos process to the solution of ill posed problems can be found in the literature; see, e.g. [6, 8, 21, 29, 30, 64]. For large problems, the main drawback of this approach is the storage requirement for the matrix Qk in (0.35) We remark that it may be attractive to apply the Lanczos process even when the matrix Qk cannot be stored in fast computer memory; one either can store Qk on secondary memory, or ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, J. Comput. Graph. Stat. 6 (1997), pp. 1--34.

....this paper k Delta k denotes the Euclidean vector norm or the associated induced matrix norm. Functions f discussed in the literature include f(t) 1=t, f(t) 1= t ) and f(t) exp(fft) where and ff are given constants. In a sequence of papers Golub and collaborators, see, e.g. [1, 2, 3, 4, 5, 7], have exploited the connection between matrix functionals of the form (1) Stieltjes integrals, Gauss type quadrature rules and the Lanczos process to derive powerful inexpensive algorithms for the computation of upper and lower bounds for F (A) For definiteness, introduce the spectral ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, J. Comput. Graph. Stat. 6 (1997), pp. 1--34.

....of U i, there holds VVk lZ Z, ad hence, VAVk Z. Inserting this into (15) we arrive at and therefore it follows from (13) that A UU = Since E is a nonsingular matrix this shows that A (l) V ) Ml, as was to be shown. 5 Numerical results Recent work on hybrid methods ( 3, 14] see also [8, 13, 7]) has mostly been focused on the choice of appropriate regularization parameters, i.e. the stopping index k for the Lanczos process and the truncation index l for the embedded TSVD regularization. While this question is certainly the most important one for practical purposes, one should ....

G. H. Golub and U. von Matt. Generalized cross-validation for large- scale problems. J. Cornput. Graph. Statist., 6:1-34, 1997.

.... rules can be successfully used to estimate the entries of an arbitrary matrix function f(A) where f is a sufficiently smooth function (hopefully C 1 ) and A is an n Theta n matrix which is symmetric positive definite and admissible for f (i.e. f(A) is defined) Golub and Urs von Matt [16] estimated the GCV function using quadrature rules as well. Philippe and Sidje [29] and Sidje [33] have shown how the bi orthogonal Lanczos algorithm associated with the uniform rational Chebyshev approximation to the matrix exponential possibly enables the computation of the scalar valued ....

G. H. Golub and Urs von Matt. Generalized cross-validation for large scale problems. Journal of Computational and Graphical Statistics, 6(1):1--34, 1997.

....that it is possible to solve the regularized problem in only O(N 2 log N) operations, the aforementioned open problem of choosing an appropriate regularization parameter remains a crucial issue to deal with. Some attempts to solve this problem can be found in the literature (cf. e.g. [1, 6, 8, 15, 18]) but more research is still necessary. 4 Regularization by Iteration As an alternative we advocate the possibility of regularizing by iteration. Historically, this technique originated with the Landweber iteration, which can be viewed as a variant of the steepest descent method applied to the ....

G. H. Golub and U. von Matt, Generalized Cross-Validation for large-scale problems, J. Comput. Graph. Statist., 6 (1997), pp. 1-34.

....a sequence of linear systems of the form (A oe (i) I)x (i) b with the same right hand side. In this case, the Krylov subspace generated by a Lanczos method is the same for any system and it can be used very efficiently along with recurrences to compute the quantities required by GCV [11, 21]. Here we assume that the different right hand sides b (i) are computed sequentially. This situation arises for instance when a new right hand side depends upon previous solutions. A first idea is to use previous systems to derive an initial guess for the current system. In [10] the current ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, Tech. Rep. TR-96-28, ETH, Zurich, Sept. 1996.

....a sequence of linear systems of the form (A oe (i) I)x (i) b with the same right hand side. In this case, the Krylov subspace generated by a Lanczos method is the same for any system and it can be used very eOEciently along with recurrences to compute the quantities required by GCV [11, 21]. Here we assume that the dioeerent right hand sides b (i) are computed sequentially. This situation arises for instance when a new right hand side depends upon previous solutions. A rst idea is to use previous systems to derive an initial guess for the current system. In [10] the current ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, Tech. Rep. TR-96-28, ETH, Z#rich, Sept. 1996.

.... Gamma A( Letting f ( be f with the data vector y replaced by , then the estimate of 1 n tr(I Gamma A( is 1 n 0 [ Gamma f ( Excellent results with n around 600 were reported in [42] for example, and have been also reported by other authors, see, for example [13]. Wahba and Luo Smoothing Spline ANOVA for Large Data Sets 6 3 Backfitting Hastie and Tibshirani [26] Section 5.2.3, discuss the backfitting (a. k. a. GaussSeidel) algorithm in the context of the general setup of SS ANOVA problems as was described by [6] Further discussion of the backfitting ....

G. Golub and Urs VonMatt. Generalized cross validation for large scale problems. Technical Report xx, Stanford University, Stanford, CA, 1995.

....should be used for all values of . This results in an estimate for V ( which is a smooth function of and appears to have the same shape as V ( computed exactly. Excellent results with n around 600 were reported in [42] for example, and have been also reported by other authors, see, for example [13]. Since the (converged) backfitting algorithm produces f j P p fi=0 f fi it can be used to compute the randomized trace estimate of tr(I Gamma A( for selected values of . Now, let z fl = y Gamma P fi 6=fl f fi . Note that at convergence, when (14) is satisfied, ky Gamma f ....

G. Golub and Urs VonMatt. Generalized cross validation for large scale problems. Technical Report xx, Stanford University, Stanford, CA, 1995. Wahba and Luo / Smoothing Spline ANOVA for Large Data Sets 19

No context found.

G. Golub and U. von Matt. Generalized Cross-Validation for Large Scale Problems. Journal of Computational and Graphical Statistics, 6, pp. 1-34, 1997.

No context found.

G. H. Golub and U. von Matt. Generalized cross-validation for large scale problems. In S. Van Hu#el, editor, Recent Advances in Total Least Squares Techniques and Errors-in-Variables Modeling, pages 139--148. SIAM, 1997.

No context found.

G. Golub and U. von Matt. Generalized cross-validation for large-scale problems. Journal of Computational and Graphical Statistics, 6:1--34, 1997.

....7 7 : 5.31) It can be shown that the nodes i of Gauss quadrature rule are the eigenvalues of T k , which are also the zeros of the polynomial p k . The weights i are the square of the rst component of the normalized eigenvectors of T k . The Gauss quadrature approximation is given by (see [34]) I G [f ] kvk 1 f(T k I)u 1 ; 5.32) where u = 1 0 0 i T is a k vector with one in the rst entry and zeros elsewhere. For the Gauss Radau rule, we need to adjust the last entry k of T k so that the adjusted tridiagonal matrix T k has an eigenvalue at the prescribed node ....

G. Golub and U. von Matt. Generalized cross-validation for large-scale problems. Journal of Computational and Graphical Statistics, 6:1-34, 1997.

No context found.

G. Golub and U. von Matt. Generalized cross-validation for large-scale problems. Journal of Computational and Graphical Statistics,6:1--34,1997.

....finding ff reduces to finding the positive root of H(ff) Delta = d T (C ffI) Gamma2 d Gamma j 2 ff 2 Theta b T b Gamma d T (C ffI) Gamma1 d Gamma ffd T (C ffI) Gamma2 d : 3. 23) In this form, one can consider techniques similar to those suggested in [5] to find ff efficiently. A NEW METHOD FOR PARAMETER ESTIMATION WITH UNCERTAIN DATA 17 4. Restricted Perturbations. We have so far considered the case in which all the columns of the A matrix are subject to perturbations. It may happen in practice, however, that only selected columns are ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, Technical report SCCM-96-06, (1996). Computer Science Dept., Stanford University.

.... to finding the positive root of H(ff) Delta = d T (C ff Delta I) Gamma2 d Gamma j 2 ff 2 h b T Delta B Gamma d T (C ff Delta I) Gamma1 d Gamma ff Delta d T (C ff Delta I) Gamma2 d i : In this form, one can develop techniques similar to those suggested in [6] to find ff efficiently. 3 Another New Formulation: TLS with Bounded Uncertainties We now introduce another optimization problem that turns out to involve the minimization of a cost function in an indefinite metric, in a way that is similar to more recent works on robust estimation and ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, in this volume, S. Van Huffel, ed., SIAM, Philadelphia, PA, 1997.

....T ffI) Gamma1 u (6) is an unbiased estimator of t(ff) trace Gamma (AA T ffI) Gamma1 Delta : Therefore we will only consider the minimization of the stochastic GCV function OE GCV (ff) q b T (AA T ffI) Gamma2 b u T (AA T ffI) Gamma1 u from now on. In [15] it is shown that the minimizers of OE GCV and OE GCV are equally well suited for the Tikhonov regularization in (2) 2.5 The L Curve Criterion The L curve criterion is based on a plot of the solution norm kx ff k 2 versus the residual norm kb Gamma Ax ff k 2 in a log log scale [18,20] The ....

Gene H. Golub and Urs von Matt, Generalized Cross-Validation for Large Scale Problems, Journal of Computational and Graphical Statistics, to appear, ftp://ftp.cscs.ch/pub/CSCS/techreports/1996/TR-96-28.ps.gz.

.... to estimate the bounds for the quantities of the trace of the inverse tr(A Gamma1 ) and the determinant det(A) of a matrix A, such as in the study of fractals [14, 18] lattice Quantum Chromodynamics (QCD) 15, 3] crystals [11, 12] the generalized cross validation and its applications (see [7] and references therein) In this paper, we focus on deriving lower and upper bounds for the quantities tr(A Gamma1 ) The author was supported in part by by an NSF grant ASC 9313958 and in part by an DOE grant DE FG03 94ER25219. y The work of this author was in part supported by NSF under ....

G. Golub and U. Von Matt. Generalized cross-validation for large scale problems. Report SCCM-96-05, Stanford University, Stanford, 1996.

....vector norm. Golub and collaborators have, in a sequence of papers, described how approximations of F (A) can be evaluated inexpensively by exploiting the connection between matrix functionals of the form (1. 1) Stieltjes integrals, Gauss type quadrature rules and the Lanczos algorithm; see, e.g. [1, 2, 3, 5] and references therein. For definiteness, introduce the vector [ 1 ; 2 ; n ] u T S and, using (1.2) write the functional (1.1) as F (A) u T Sf ( S T u = n X j=1 f( j ) 2 j : 1.3) The right hand side of (1.3) is a Stieltjes integral If : Z 1 Gamma1 f(t)d (t) ....

....are described in Section 2. A numerical example is presented in Section 3. 2. An error bound. We first discuss the computation of Gm f , and then show how a bound for the error Em f can be determined with very little additional work. The computation of Gauss quadrature rules is discussed in [1, 2, 3, 5]. Our review of these results allows us to introduce notation necessary to discuss the evaluation of the error bound. Gauss quadrature rules with respect to the measure d (t) can conveniently be determined by the Lanczos algorithm. Application of m steps of the Lanczos algorithm to the matrix A ....

G. H. Golub and U. von Matt, Generalized cross-validation for large scale problems, J. Comput. Graph. Stat. 6 (1997), pp. 1--34.

No context found.

G. H. GOLUB and U. von MATT. Generalized cross-validation for large-scale problems. Journal of Computational and Graphical Statistics, 6(1):1--34, March 1997.

No context found.

G. H. Golub and U. von Matt. Generalized cross-validation for large scale problems. Journal of Computational and Graphical Statistics, 6(1):1--34, 1997.

No context found.

G.H. Golub and U. von Matt, "Generalized cross-validation for large-scale problems," revised version, Stanford University Technical Report. Medical Imaging: Image Processing, K.M. Hanson, ed., SPIE Proc. 3661, pp. 562-573, 1999.

No context found.

Golub, G. & vonMatt, U. (1997), `Generalized cross-validation for large-scale problems', J. Comput. Graph. Statist. 6, 1--34.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC