Peter Craven and Grace Wahba, Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation, Numer. Math., 1978/79, no. 4, 377-- 403.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....subset selection and the ic method, classifying the faces dataset using the rectangle method. Unfortunately, using this equation requires access to the inverse matrix G, which is often computationally intractable and numerically unstable. An alternative approach, introduced by Craven and Wahba [31], is known as the generalized approximate cross validation, or GACV for short. Instead of actually using the entries of the inverse matrix G directly, an approximation to the leave one out value is made using the trace of G: f S i (x i ) tr(G) This approach, while being only an ....

P. Craven and G. Wahba, Smoothing noisy data with spline functions, Numerical Mathematics 31 (1966) 377--390.

....of overcoming this di#culty, is being studied for systems linear in the state variables. The smoothing spline #(t) is defined by: Here the value of # can be chosen to provide a compromise between data fit and smoothness. An alternative stochastic formulation is given by Wahba [9]: E y(t) y 1 , y 2 , # # # d# dt . 40) Here # = 1 # . The consistency result available is #(t) # E y(t) provided # is chosen appropriately. For our purposes a key step is the generalisation to more general di#erential operators (g splines) M k #) As # gets ....

Craven, P., Wahba, G.: Smoothing noisy data with spline functions. Numer. Math. 31 (1979) 377--403

....in additive models. This parameter indicates how rough the estimated function should remain. The optimal smoothing parameter is balancing the bias to the variance of the estimated curve, or more exactly, minimizing the mean squared error. It can be estimated by generalized cross validation [7]. Having once introduced additive models, it is quite intuitive to adjust the procedure to a specific problem. If, for example, some functions are already known and given by parametric models, a semi parametric method can be set up simply by replacing the smoothers of these functions with their ....

....GCV(#) N 1 i=1 S ii (#) 1 tr S(#) or, again eliminating # , GCV(#) N 1 (1 1 tr S(#) 7) In these expressions S(#) is the hat matrix for smoothing splines and tr S(#) its trace. Generalized cross validation was introduced by Craven and Wahba [7]. Since the trace of a matrix is equivalent to the sum of all eigenvalues, GCV can be calculated very e#ciently. If the design points are equally distributed on the interval [a, b] tr S(#) can be approximated by: tr S(#) 2 #=3 h 1 C 4 # (# 1.5) i , where C = N # 1 ....

Craven P. and Wahba G. Smoothing noisy data with spline functions. Num. Math., 31:377--403, 1979.

....subset selection and the ic method, classifying the faces dataset using the rectangle method. Unfortunately, using this equation requires access to the inverse matrix G,whichis often computationally intractable and numerically unstable. An alternative approach, introduced by Craven and Wahba [31], is known as the generalized approximate cross validation, or GACV for short. Instead of actually using the entries of the inverse matrix G directly, an approximation to the leave one out valueismadeusingthetrace of G: f S i (x i ) tr(G) This approach, while being only an ....

P. Craven and G. Wahba, Smoothing noisy data with spline functions, Numerical Mathematics 31 (1966) 377--390.

....maximum order of variable interactions (products of variables) allowed in the functions B k , as well as the maximum value of M allowed in the forward stage, are parameters that need to be tuned experimentally. Backward model selection uses the generalized cross validation criterion introduced in [23]. The original MARS algorithm ts only scalar valued functions and is therefore not well suited to discrimination tasks with more than two classes. A recent proposal called Flexible Discriminant Analysis (FDA) 59] with its publicly available S Plus implementation in the StatLib program library ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerical Mathematics, 31:317403, 1979.

....the noise level is known or a good estimate of it is available. GCV, Approximate GPP, GPE and Their Relationships. We can use an approximation similar to the one used with the standard CV error function (Breiman (1995) of a linear model that result in the Generalized Cross Validation (GCV) (Craven and Wahba (1979)) for the GP model also. More speci cally, let p = 1 N P N i=1 p ii and use the approximation p ii p; 8 i. With this approximation, let us denote G( GE ( H( and 2 as G( GE ( H( and 2 respectively and refer G( and GE ( as the approximate ....

Craven, P. and Wahba, G. (1979) Smoothing noisy data with spline functions.

....y) 2 with a square root normalization (z) z 1=2 , as discussed in Section 2. To evaluate the ecacy of TRI in this problem we compared its performance to a number of standard model selection strategies, including: structural risk minimization, SRM [CMV97, Vap96] RIC [FG94] SMS [Shi81] GCV [CW79], BIC [Sch78] AIC [Aka74] CP [Mal73] and FPE [Aka70] We also compared it to 10 fold cross validation, CVT (a standard hold out method [Efr79, WK91, Koh95] We conducted a simple series of experiments by xing a domain distribution P X on X = IR and then xing various target functions f : IR ....

....To test the basic e ectiveness of our approach, we repeated the experiments of Section 3.2. The rst class of methods we compared against were the same model selection methods considered before: 10 fold cross validation CVT, structural risk minimization SRM [CMV97] RIC [FG94] SMS [Shi81] GCV [CW79], BIC [Sch78] AIC [Aka74] CP [Mal73] FPE [Aka70] and the metric based model selection strategy, ADJ, introduced in Section 3.3. However, since none of the statistical methods, RIC, SMS, GCV, BIC, AIC, CP, FPE, performed competitively in our experiments, we report results only for GCV which ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerische Mathematik, 31:377-403, 1979.

.... R( fK ) To this end, K is obtained by minimizing an adjusted training risk RGCV ( f K) # n n p # q 1 n n # i=1 y i fK (x i ) q , 11) where p =1 (K 1) d 2) This adjustment, called generalized cross validation, was originally introduced for L 2 loss and linear smoothers (Craven Wahba 1979). Its application here is justified primarily on empirical grounds. In simulation studies, with R approximated using a large test set, R( f K ) was typically close to minK R( fK ) A straightforward search is employed to minimize (11) The GCV risk is evaluated for successive values of K, ....

P. Craven & G. Wahba (1979). Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerical Mathematics, 31, 317--403.

.... The form of the basis functions in the MARS model is found by starting the algorithm with only B 1 (the constant basis function) in the model and then by stepwise addition of the basis functions which most reduce the chosen lack of t criterion (usually the generalized cross validation measure (Craven and Wabha, 1979)) The candidate bases which can be added are found by splitting the bases that are currently in the model; this prevents the candidate search space from becoming unmanageably large even though it is restrictive. After the model has been grown to have many basis functions stepwise deletion takes ....

Craven, P. and Wabha, G. (1979) Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of cross-validation. Numerische Mathematik, 31, 317-403.

....p. In the simulation, I used two methods to estimate an optimal value of p on the basis of statistical considerations of the data only: 1) the p value that gives a mean square error (MSE) closest to an error variance known a priori and (2) a generalized cross validation (GCV) criterion (Craven and Wahba, 1979; Woltring, 1985, 1986a) The GCV and MSE alternatives were referred to by Woltring (1986b) as mode 2 and mode 3 respectively. I used: error = 2[0.5(1 h) h] 2 (13) as the error variance for MSE, where h is 1 3, the frequency of incorrectly locating the true pixel (see above) The value ....

CRAVEN, P. AND WAHBA, G. (1979). Smoothing noisy data with spline functions. Estimating the degree of smoothing by the method of generalized cross-validation. Numer. Math. 31, 377--403.

.... For a smoothing spline there is a surprisingly simple relationship: the cross validation residual is given by e (i) y i Gamma b f i (x i ) y i Gamma b f (x i ) 1 Gamma A ii ( e i ( 1 Gamma A ii ( 7) The relationship (7) was shown in Craven and Wahba (1979)[2], and in the same paper an approximation to (6) called the generalized cross validation function, was proposed: GCV ( 1 n P n i=1 (y i Gamma b f (x i ) 2 (1 Gamma trA( n) 2 (8) Under reasonable conditions GCV ( is a consistent estimate of EASE( oe 2 . Thus one would expect ....

Peter Craven and Grace Wahba. Smoothing noisy data with spline functions. Numerische Mathematik, 31:377--403, 1979.

.... criteria range from the sample variance of the residuals oe 2 (or, equivalently, R 2 ) to Akaike s Information Criterion (AIC) 2] Akaike s Finite Prediction Error (FPE) 1] Schwarz s Bayesian Information Criterion (BIC) 19] Craven and Wahba s Generalized Cross Validation (GCV) [6], and those of Hannan and Quinn (HQ) 12] Rice [18] and Shibata [22] A model with a lower value of each criterion statistic is judged to be preferable. For a survey of such approaches see Engle and Brown [8] A Monte Carlo simulation was carried out to illustrate the benefits of h block ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerical Mathematics, 13:377--403, 1979.

.... combination c k d d(h k ; P YjX ) As mentioned, there are many variants of this general approach, including the minimum description length principle [Ris86] Bayesian maximum a posteriori selection, structural risk minimization [Vap82, Vap96, Vap98] generalized cross validation [CW79], and a host of other statistical criteria [Aka70, Mal73, Aka74, Sch78, Shi81, FG94, Mal95] These strategies di er in the speci c complexity values they assign and the particular tradeo function 8 . ....

....y) 2 with a square root normalization (z) z 1=2 , as discussed in Section 2. To evaluate the ecacy of TRI in this problem we compared its performance to a number of standard model selection strategies, including: structural risk minimization, SRM [Vap96, CMV96] RIC [FG94] SMS [Shi81] GCV [CW79], BIC [Sch78] AIC [Aka74] CP [Mal73] and FPE [Aka70] We also compared it to 10 fold cross validation, CVT (a standard hold out method [Efr79, WK91, Koh95] We conducted a simple series of experiments by xing a domain distribution P X on X = IR and then xing various target functions f : IR ....

[Article contains additional citation context not shown here]

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerische Mathematik, 31:377-403, 1979.

....error distribution. As a comparison, in addition to reporting the employment density quantile splines, we report cubic spline estimation of the employment density function below. To allow more flexibility, we select the smoothing parameter both by cross validation, and by Craven and Wahba s [7] generalized cross validation for the cubic splines. 8 8 Both methods for choosing the knots and smoothing parameter are more general, and allow more flexibility, than is the equally spaced knots method (Anderson [2] Anderson [3] Zheng [25] 11 3 Data We estimate employment density ....

P. Craven and G. Wahba, Smoothing noisy data with spline functions, Numerische Mathematik, 31, 377--403 (1979).

....of degree 0, 1, 2, etc. and attempt to select the best one using various forms of complexity penalization and hold out testing. The methods we compared were: 10 fold cross validation, CVT (Efron, 1979) structural risk minimization, SRM (Vapnik, 1996; Cherkassky et al. 1996) GCV (Craven Wahba, 1979); AIC (Akaike, 1974) BIC (Schwarz, 1978) FPE (Shibata, 1981) CP (Mallows, 1973) RIC (Foster George, 1994) and the metric based model selection strategy, ADJ, introduced in (Schuurmans, 1997) However, since none of the statistical methods, GCV, BIC, FPE, CP and RIC, performed competitively ....

Craven, P., & Wahba, G. (1979).Smoothing noisy data with spline functions. Numer. Math., 31, 377-403.

....of degree 0,1,2, etc. and attempt to select the best one using various forms of complexity penalization and hold out testing. The methods we compared were: 10 fold cross validation CVT (Efron, 1979) structural risk minimization SRM (Vapnik, 1996; Cherkassky, Mulier, Vapnik, 1996) GCV (Craven Wahba, 1979), AIC (Akaike, 1974) BIC (Schwarz, 1978) FPE (Shibata, 1981) CP (Mallows, 1973) RIC (Foster George, 1994) and the metric based model selection strategy ADJ introduced in (Schuurmans, 1997) However, none of the statistical methods GCV, BIC, FPE, CP and RIC performed competetively in our ....

Craven, P., & Wahba, G. (1979). Smoothing noisy data with spline functions. Numer. Math., 31, 377-403.

....estimate of the noise variance oe 2 . However, even with an accurate estimate of oe 2 , this method is known to generally produce systematic oversmoothing [11] Recently, generalized cross validation (GCV) has become a very popular method for the estimation of the best from the data. See [17, 10, 1, 13] for theoretical results justifying MINIMUM RECONSTRUCTION ERROR CRITERION 3 GCV or its fast randomized version [9] and see [10, 6, 8] for general algorithms to compute this criterion. GCV consists of choosing the value of which minimizes a function called the GCV function, denoted here GCV( ....

....(3) that GCV may have some difficulties in determining the optimal parameter because high oscillations in x( can be smoothed out in Bx( see Section 1.4, and Figure 7 in Section 4. 4 for another example) A second method to choose the value of is to use a statistical estimator of MSPE (see [1, 10, 13, 9]) known as Mallows CL criterion. By construction, this method may also have the same difficulties. Rice [14] compares the use of the mean square estimation (or reconstruction) error (MSEE) criterion, also called the domain risk, MSEE( IE(1=njjx( Gamma x 0 jj 2 ) 4) to the use of ....

P. Craven, G. Wahba, Smoothing noisy data with spline functions. Num. Math., 31 (1979), pp. 377-403.

....the cross validation approach is equivalent to the penalizing functions concept and shares the same asymptotic properties. Note that (30) is a function of the i th diagonal element of the smoother matrix. More precisely, cross validation is equivalent with generalized cross validation (Craven and Wahba, 1979) in this case. Hardle, Hall and Marron (1988) show asymptotic optimality of the selected bandwidth, the rate of convergence is slow though. An improved bandwidth selection is discussed in Hardle, Hall and Marron (1992) We want to remark that (29) and (30) also imply that the computation of CV (H) ....

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions, Numer.

....choosing infinity (or just a very large value) produces a least squares straight line fit to the data. The curves shown in Figure 1 are optimal smoothing spline fits obtained from the algorithm CUBGCV [6] which chooses the smoothing constant to minimize the generalized cross validation statistic [7]. For a given value of the smoothing constant, that statistic is computed as the sum of squared errors obtained by successively dropping one data point from the set, fitting the corresponding smoothing spline to the remaining points and calculating the error in predicting the point that was ....

P. Craven and G. Wahba, "Smoothing noisy data with spline functions," Numerische Mathematik, 31 (1979) pp. 377403.

.... NN ) cross validation, see e.g. STONE, 1974) SRSS(j) y P j y ) T ( I T j ) 1 ( y P j y ) type (B) standardized residual sum of squares, see e.g. HOCKING, 1976) GCV(j) y P y N k( j) j 2 2 1 1 [ type (A) or (B) generalized cross validation, see CRAVEN; WAHBA, 1979) RMS(j) y P y N k( j) j 2 1 1 [ type (A) or (B) residual mean square, see e.g. HOCKING, 1976) RSS(j) y P j y 2 [ type (A) or (B) residual sum of squares) For surveys on model choice criteria, including interpretation and properties, see in particular HOCKING ....

CRAVEN, P.; WAHBA, G. (1979): Smoothing noisy data with spline functions.

....linear function of its parameters, or in general by using one of the approximate cross validation criteria in [26] It is shown that the approximate criteria are asymptotically equivalent to FPE, as l 1. Another approximation to cross validation is the Generalized Cross Validation (GCV) criterion [4] JGCV (M) 1 (1 Gamma p(M) l) 2 JASR (M) which is easily seen to be asymptotically equivalent to FPE, and also assumes linear parameterization of the predictor. Any one of these criteria can be applied with the structure identification algorithm we will present in the next section. 3 System ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerical Math., 31:317--403, 1979.

....bandwith, or binwidth, depending on the different estimates) shall be estimated from data using a procedure usually referred to as the generalized cross validation (GCV) test. GCV procedures were studied for kernel (see, for instance, Rice, 1984) Hardle and Marron, 1985) spline ( Li, 1986) (Craven and Wahba, 1979)) and projection estimates (c.f. Polyak and Tsybakov, 1990) Li, 1987) Let us consider, for instance, the procedure for the projection estimates 10 . To make the model order explicit in formula (56) we shall write b f m;N instead of b f N . Set S 2 m;N = N Gamma1 P N i=1 kY i Gamma ....

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions. Numer. Math., 31:337--403.

....Then the ordinary cross validation for our model would be OCV (r; ff) 1 n 1 n 2 n1 n2 X k=1 (y k Gamma K k f [k] r;ff ) 2 : OCV is a measurement of overall predictability of data points by the estimate f r;ff . It is then not hard to see that the leaving out one lemma (Lemma 3. 1 of Craven and Wahba (1979), also see Wahba (1990) is true here, i.e. we have: Lemma 2.2.1 If we let h r;ff [k; z] be the solution to (1.4) with the kth data point y r k being replaced by z, then h r;ff [k; K r k f [k] r;ff ] f [k] r;ff . 2 Now, consider the following identity: y k Gamma K k f [k] r;ff = y r ....

....K 2 fk 2 oe (4.1) We believe that under some conditions, the GCV r estimate should have the following property that as n 1, I p (GCV r) T p (r gcv ; ff gcv ) inf r;ff T p (r; ff) 1 in probability, where I p (GCV r) is the relative inefficiency with respect to the GCV r. Please see Craven and Wahba (1979), Li (1985) Li (1986) and Li (1987) for more detailed discussions on one source data case. In Gao (1993) there are some discussions on two source data case. When the stochastic model is true, it will not be surprising to see that the GML r estimate performs better than the GCV r, see Stein ....

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions, Numer. Math.

....order of variable interactions (products of variables) allowed in the functions B k , as well the maximum value of M allowed in the forward stage, are parameters that need to be tuned experimentally. Backward model selection uses the generalized cross validation criterion (GCV) introduced in [13]. GCV(M) 1=n) P n i=1 (y i Gamma r(x i ) 2 [1 Gamma C(M) n] 2 : Here the numerator is the lack of fit based on the training data and the denominator imposes a penalty for increasing the model complexity, C(M ) This complexity function is taken to be C(M) M Delta (ff=2 1) where M ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerical Mathematics, 31:317--403, 1979.

....ridge regression, univariate and multivariate smoothing spline regression, partial spline models, penalized GLIM (generalized linear models) penalized likelihood estimation, penalized log density, and log hazard estimation are mentioned. Further references in the area of smoothing splines include [4, 5, 20, 22, 25, 28, 36, 37, 41, 42]. The application of generalized cross validation to global scale numerical weather prediction models is described in [43] Further applications can be found in image processing [2, 29, 35] astronomy [34] and chemistry [23] The reader is also referred to [8] and [18, p. 460] where several other ....

P. Craven and G. Wahba, Smoothing Noisy Data with Spline Functions, Numerische Mathematik, 31 (1979), pp. 377--403.

....regularization parameter fl that minimizes FPER: fl(Z N ) arg min fl0 FPER(fl; Z N ) which can be found using a simple line search algorithm. Alternative approaches includes the use of a separate data set, data re sampling techniques like cross validation or generalized cross validation [2] to estimate the MSE. 5 Simulation Example This simulation example is based on the results reported in [3] where model based control of a batch fermentation reactor is studied. The simulated true system model describes the fermentation of glucose to gluconic acid by the micro organism ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized crossvalidation. Numerical Math., 31:317--403, 1979.

....estimators of J MSE , which are also computed from the samples in D train only, are the Final Prediction Error (FPE) criterion [Akaike 69] given by J FPE = 1 N N X n=1 (y n Gamma Theta y n ) 2 , 1 Gamma F=N 1 F=N (2. 5) and the Generalized Cross Validation (GCV) criterion [Craven and Wabha 79] given by JGCV = 1 N N X n=1 (y n Gamma Theta y n ) 2 = 1 Gamma F=N) 2 (2.6) where F is the eoeective number of independent parameters (degrees of freedom) in the model, f . Both these criteria are particularly useful in iterative algorithms since they penalize models with ....

P. Craven and G. Wabha. Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik, vol. 31, pp. 317403, 1979.

....space E p to E n , Wahba (1978) describes the GCV estimate as coming from rotating the E n coordinate system to a new design matrix X, such that X is circulant (each column of X is equal to the previous column rotated downwards by one element) and doing ordinary CV in the new system. Craven and Wahba (1979) introduce GCV for choosing the smoothing parameter in spline models. GCV has been used in many applications in atmospheric models (O Sullivan and Wahba 1985) and in larger scale models, such as numerical weather prediction (Wahba, Johnson, Gao, and Gong 1995) Let GCV denote the value of which ....

....1 X j=1 x 2 ij = k 3 1 and the eigenvalues of XX 0 , v , v = 1; n satisfy v nv Gammam , where m 1. This means that k 3 = 1 X v=1 v Gammam . Then tr(H) n 0 , tr(H) 2 ntr(H 2 ) 0 if n 1=m 1 . Result 2. 1 may now be applied to show I r # 1 as n 1 2 Craven and Wahba (1979) give similar results for smoothing splines. Wahba (1977) gives a slight simplification of the results in Golub et al. 1979) If n p, then as n 1, E[PMSE( EGCV ) E[PMSE( EPMSE ) 1 O p n 22 If n, p 1, P fi 2 v 1, v = O(nv Gammam ) where m 1, v = 1; n, ....

Craven, P. and G. Wahba (1979). Smoothing noisy data with spline functions.

....differently. The model order (or bandwith, or binwidth, depending on the different estimates) shall be estimated from data using a procedure usually referred to as the Generalized Cross Validation (GCV) test. GCV procedures were studied for kernel (see, for instance, 69] 40] spline ( 49] [12]) and projection estimates (c.f. 66] 50] Let us consider, for instance, the procedure for the projection estimates 3 . To make the model order explicit in formula (2.13) we shall write b fm;N instead of b f N . Set S 2 m;N = N Gamma1 P N i=1 kY i Gamma b fm;N (X i )k 2 . As for ....

P. Craven and G. Wahba, Smoothing noisy data with spline functions, Numer. Math., 31 (1979), pp. 337--403.

.... cross validation criteria in this setting are respectively CV (p) n Gamma1 n X i=1 (Y i Gamma g n (X i ) 2 (1 Gamma s ii ) 2 and GCV (p) n Gamma1 n X i=1 (Y i Gamma g n (X i ) 2 (n Gamma1 tr(I Gamma A p ) 2 ; where s ii is the ith diagonal element of A p (see [13] pp.386 387 and [14] pp.223 224) For smoothing splines, key references include [13] 14] 15] 16] and [17] In order to formulate the information criteria, we need the degree of freedom of the smoothing operator A p and the estimator of the variance of the random error. Note that E kY ....

.... n X i=1 (Y i Gamma g n (X i ) 2 (1 Gamma s ii ) 2 and GCV (p) n Gamma1 n X i=1 (Y i Gamma g n (X i ) 2 (n Gamma1 tr(I Gamma A p ) 2 ; where s ii is the ith diagonal element of A p (see [13] pp.386 387 and [14] pp.223 224) For smoothing splines, key references include [13], 14] 15] 16] and [17] In order to formulate the information criteria, we need the degree of freedom of the smoothing operator A p and the estimator of the variance of the random error. Note that E kY Gamma g n k 2 = n Gamma tr(2A p Gamma A p A 0 p ) oe 2 0 kg Gamma E(A p Y)k ....

[Article contains additional citation context not shown here]

Craven, P. and Wahba, G. (1979), Smoothing noisy data with spline functions, Numer. Math., 31, 377-409.

....related. Both too complex or too simple a model can yield poor results, the objective being to find just the right size model. Independent test cases or resampling by cross validation are effective for estimating future performance. In the absence of these estimates, approximations, such as GCV (Craven Wahba, 1979; Friedman, 1991) as described in equation 6, have been used in the statistics literature to estimate performance 2 . Both measures of training error and model complexity are used in the estimates. C(M) is a measure of model complexity expressed in terms of parameters estimated (such as the ....

Craven, P., & Wahba, G. (1979). Smoothing noisy data with spline functions. estimating the correct degree of smoothing by the method of generalized cross-validation. Numer.

....n; s i ; s i ) Gamma 1 n Gamma 1 and the last term, being independent of can be dropped. The sum P n i=1 G n; s i ; s i ) is in fact the trace of the matrix occurring in spline smoothing problems and have appeared in the generalized cross validation criterion introduced for such problems (Craven and Wahha, 1979). An approximation for it have been proposed in Silverman (1984b) A simpler approximation can be obtained from Lemmas 2 and 3. It consists in replacing G n; s; s) by (0)f(s) 3=4 3=4 and the integration with respect to dFn (s) by that with respect to f n; s)ds. This yields 1 n n X i=1 ....

Craven, P. and Wahba, G (1979) Smoothing noisy data with spline functions. Numerische Mathematik 31, 377--403.

....model is important. The form of the MARS function is found by starting the algorithm with only B 1 (the constant basis function) in the model and then by stepwise addition of the basis functions which most reduce the chosen lack of fit criterion [usually the generalized cross validation measure (Craven and Wabha, 1979)] The candidate bases which can be added are found by splitting the bases that are currently in the model; this prevents the candidate search space from becoming unmanageably large even though it is restrictive. After the model has been grown to have many basis functions stepwise deletion takes ....

Craven, P. and Wabha, G. (1979). Smoothing noisy data with spline functions.

....it the least is removed at each step. Finally the resulting model can be made to have a continuous first derivative by rounding at the split points as mentioned above. The lack of fit measure used by Friedman (1991) is the generalised cross validation criterion which was originally proposed by Craven and Wahba (1979). The aim of this paper is to provide a Bayesian algorithm which mimics the MARS procedure. This is done by considering the number of basis functions, along with their type (see Section 2.1) their coefficients and their form (the positions of the split points and the sign indicators) random. We ....

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions.

.... j ( z Gamma h(t i ) k 2 1 M M X i=1 i6=j k Gamma Gamma1=2 i (p i Gamma h(t i ) k 2 D X i=1 k g k (h) 49) It follows that fj; j) t j )g = j) This result is a generalization of the so called leaving outone lemma, which was first proved for the case D = 1 in [4]. Using the leaving out one lemma and the equation (32) one can see that (j) t j ) fj; j) t j )g(t j ) M L X i=1 i6=j H (ji) p i H (jj) j) t j ) b; 50) where b is independent of p. Employing the equations (32) and (50) one obtains p j Gamma (t j ) p j Gamma ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numer. Math., 31:377--403, 1979.

....2 [a; b] fg : g 0 abs. continous and R (g 00 ) 2 1g. Define g as the estimate of the curve g so that: g = arg min g2W 2 2 [a;b] A (g) It is well known that g is necessarily a natural cubic spline with knots at t i (see, for example, Silverman and Green (1994) Wahba (1981) and Craven and Wahba (1979)) Note that the roughness penalty R b a (g 00 (t) 2 dt has the property of reducing the problem of choosing g from an infinite dimensional class of functions to a finite class of functions since g can be written as linear combination of basis functions. Although this fact might lead ....

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions, Numerische Mathematik 31: 377--403.

.... The form of the basis functions in the MARS model is found by starting the algorithm with only B 1 (the constant basis function) in the model and then by stepwise addition of the basis functions which most reduce the chosen lack of fit criterion (usually the generalized cross validation measure (Craven and Wabha, 1979)) The candidate bases which can be added are found by splitting the bases that are currently in the model; this prevents the candidate search space from becoming unmanageably large even though it is restrictive. After the model has been grown to have many basis functions stepwise deletion takes ....

Craven, P. and Wabha, G. (1979) Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of cross-validation. Numerische Mathematik, 31, 317-403.

....threshold proportional to the noise level. But in many practical cases the actual amount of noise is not known. Instead of estimating the noise level, we try to find a good threshold directly, only using the input data. Weyrich and Warhola [18] and Nason [15] applied the idea of Cross Validation [7, 17, 1], and obtained excellent results. This Cross Validation is a function of the threshold value only based on the input data. Its minimum is a good approximation for the optimal threshold. Wahba [17] uses the same idea to find an optimal smoothing parameter for a spline fitting procedure. ....

....estimation: ffi = q 2 log(N) oe: 9) This formula and other, more complicated estimators require knowledge of the noise variance oe 2 , which may not be readily available in practical applications. Weyrich and Warhola [18] therefore suggest to adapt Wahba s Generalized Cross Validation (GCV ) [17, 1] for automatic spline smoothing. Applied to our wavelet procedure, this GCV should be a function of the threshold value, using only known data and having approximately the same minimum as the residual function R. 3 The mean square error function R(ffi) Since we are looking for an approximation of ....

[Article contains additional citation context not shown here]

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerische Mathematik, 31:377--403, 1979.

....with absolute value below the threshold. Nason [13] and Weyrich and Warhola [22] describe cross validation procedures. Bayesian procedures [1, 17, 20] start from a statistical model for noisy and noise free coefficients. This paper discusses the possibilities of a generalized cross validation [21, 5] method. Weyrich and Warhola introduce it in a wavelet threshold procedure and obtain nice results. Asymptotical optimality properties [9] explain these results. For the greater part, the results of this paper summerize findings of previous papers [9, 8] Section 2 introduces the method of wavelet ....

....gets over smoothed. The expectation ER(ffi) is called Risk function. However, the noise free data are unknown, and so the optimal threshold cannot be computed exactly. 3 The principle of Generalized Cross Validation To estimate the risk, we apply the Generalized Cross Validation (GCV) function [21, 5]. This is a formalized generalization of a leaving out one operation: to validate the performance of a smoothing parameter at a given data point y i , we leave out this point and apply the smoothing procedure on the remaining data. We interpolate this intermediate result in the missing point and ....

P. Craven and G. Wahba. Smoothing noisy data with spline functions. Numerische Mathematik, 31:377--403, 1979.

....problem is solved in principle by calculating g(x) Popular approaches to this class of smoothing problems are essentially non parametric. Given data D = fy i ; x i ; i = 1; ng, g(x) can be estimated by using one of several techniques, including lowess (Cleveland, 1979) spline smoothing (Craven Wahba, 1979; Silverman, 1985) and kernel smoothing (Gasser M uller, 1984) The common assumption among these methods is that x 1 ; x n are the design points of an experiment at which the observations y 1 ; y n are obtained, and that there is an unknown regression function g(x) satisfying ....

CRAVEN, P. & WAHBA, G. (1979). Smoothing noisy data with spline functions.

.... (e.g. the additive combination c i cerr(h i ) There are many variants of this basic approach, including the minimum description length principle (Rissanen 1986) Bayesian maximum a posteriori selection, structural risk minimization (Vapnik 1982; 1996) generalized cross validation (Craven Wahba 1979), and even regularization (Moody 1992) These strategies differ in the specific complexity values they assign and the particular tradeoff function they optimize, but the basic idea is still the same. The other most common strategy is hold out testing. Here one asks: for the given set of training ....

....main points it will suffice to consider two strategies that embody distinct penalization policies. To describe these strategies, let r = i=t be the number of complexity levels being considered per training example. 3 The first penalization strategy we consider is Generalized Cross Validation GCV (Craven Wahba 1979). Following (Moody Utans 1992) we can write the adjusted error estimate of this strategy as c err GCV (h i ) c err(h i ) 2r Gamma r 2 (1 Gamma r) 2 c err(h i ) The other penalization strategy we consider is Vapnik s Structural Risk Minimization procedure SRM (Vapnik ....

Craven, P., and Wahba, G. 1979. Smoothing noisy data with spline functions. Numer. Math. 31:377--403.

No context found.

Peter Craven and Grace Wahba, Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation, Numer. Math., 1978/79, no. 4, 377-- 403.

No context found.

Craven P., Wahba G, Smoothing noisy data with spline functions. Number. Math 31 (1979) 377--403

No context found.

Craven P., Wahba G.: Smoothing noisy data with spline functions. Number. Math 31 (1979) 377-403

No context found.

Craven, P., Wahba, G. (1979). Smoothing noisy data with spline functions. Numer. Math., 31, 377-390.

No context found.

, 351--358. Craven, P. & Wahba, G. (1979), `Smoothing noisy data with spline functions', Numerical Mathematics

No context found.

Craven, P. and Wahba, G. (1979). Smoothing noisy data with spline functions. Numerische Mathematik, 31, 377---403.

No context found.

Craven, P. and G. Wahba (1979). "Smoothing noisy data with spline functions," Numerische Mathematik, 31, 377-403.

No context found.

P. Craven and G. Wahba, "Smoothing noisy data with spline functions," Numer. Math. 48, pp. 671--698, 1986.

No context found.

Craven, P., and G. Wahba (1979) Smoothing Noisy Data with Spline Functions, Numerical Mathematics, 31, 377403.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC