J.H. Friedman and B.W. Silverman. Flexible parsimonious smoothing and additive modeling. Technometrics, vol. 31, pp. 3--39, 1989.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....like to estimate f in a nonparametric fashion. It is anticipated that f may possess some abrupt changing structures but is otherwise smooth. 2. 2 Regression Splines In the statistics literature an increasingly popular nonparametric curve fitting method is regression spline smoothing; e.g. see [8], 10] 13] 11] and [16] Regression splines are a special kind of piecewise polynomials. Those breakpoints that divide a regression spline into pieces is known as knots. Usually these divided pieces are low order polynomials having the 5 same order, and they are forced to join smoothly at ....

....of data points is large, finding the best estimate defined by the above AIC BIC criterion would involve solving a hard, large scale minimization problem. Common techniques for dealing with these types of problems include knot addition, knot deletion, knot movement or combinations of them; e.g. [8], 13] and [16] However, these techniques do not provision the inclusion of break points in our model. In this article we suggest using genetic algorithms, which are also known as evolutionary algorithms [7] for solving our needs. It has been demonstrated that, when correctly applied, genetic ....

J. H. Friedman and B. W. Silverman. Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics, 31:3--21, 1989.

....For example, see Wahba (1990, chapters 1 and 2) and Hastie and Tibshirani (1990, chapters 1 and 2) A second approach to avoid overfitting is to use a data driven method to select significant basis terms. This becomes a variable selection problem and has a number of solutions# for example, see Friedman and Silverman (1989), Friedman (1991) and Stone, Hansen, Kooperberg and Truong (1997) The current paper uses the Bayesian solution proposed by Smith and Kohn (1996, 1997a) and developed further by Dennison, Mallick and Smith (1998) and Holmes and Mallick (1998) The paper is organized as follows. Section 2 ....

Friedman, J.H. and Silverman, B.W. (1989) `Flexible parsimonious smoothing and additive modeling,' Technometrics. 31, 3-39.

....for example, Eubank (1988) or Green and Silverman (1994) for an introduction to these algorithms. Regression splines can be fit by ordinary least squares once the knots have been selected, but knot selection requires sophisticated algorithms that can be computationally intensive; see, for example, Friedman and Silverman s (1989) Turbo, Friedman s (1991) MARS algorithm, and Smith and Kohn s (1996) Bayesian knot selector based on Gibbs sampling. In this paper, we combine features of smoothing splines and regression splines. Our models often have far fewer parameters than a smoothing spline, but unlike MARS and other ....

Friedman, J.H., andS ilverman, B.W. (1989), "Flexible parsimonious smoothing and additive modeling (with discussion)," Technometric, 31, 3--39.

....AIC; Density estimation; Extended linear models; Finite elements; Free knot splines; GCV; Linear splines; Multivariate splines; Regression. 1. Introduction Polynomial splines are at the heart of many popular techniques for nonparametric function estimation. For regression problems, TURBO (Friedman and Silverman, 1989), multivariate adaptive regression splines or MARS (Friedman, 1991) and Pi (Breiman, 1991) have all met with considerable success. In the context of density estimation, the Logspline procedure of Kooperberg and Stone (1991, 1992) exhibits excellent spatial adaptation, capturing the full height of ....

Friedman, J. H. and Silverman, B. W. (1989) Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics, 31, 3--39.

....model both using all nine explanatory variables, as well as a subset of only four that were chosen via a stepwise algorithm (the four are TEMP, IBH, DPG, and VIS) The comparison of the R 2 s is shown in Table 2. Hastie and Tibshirani (1984) t a Generalized Additive Model (GAM) to the data. Friedman and Silverman (1989) t the data using TURBO. In the discussion of the previous paper, Hawkins (1989) ts the data with linear regression after using Box Tidwell style transformations on the variables. For comparison, I include the result of my model with three nodes and ve explanatory variables. Table 2 shows that ....

Friedman, J. H. and Silverman, B. W. (1989). \Flexible Parsimonious Smoothing and Additive Modelling (with discussion)." Technometrics , 31, 3-39.

.... differ from classical smoothing splines (Eubank 1988; Wahba 1990) and other penalized regression splines (O Sullivan 1986; Eilers and Marx 1996) in that the amount of smoothing is locally adaptive (knots move to areas where the function is less smooth) In contrast to adaptive regression splines (Friedman and Silverman 1989; Kooperberg, Stone, and Truong 1995; Stone, Hansen, Kooperberg, and Truong 1997) and hybrid smoothing splines (Luo and Wahba 1997) the choice of knot locations in a free knot spline is a parameter estimation, rather than a predictor subset selection, PENALIZED ESTIMATION OF FREE KNOT SPLINES ....

Friedman, J. H., and Silverman, B. W. (1989), "Flexible Parsimonious Smoothing and Additive Modeling," Technometrics, 31, 3--39.

....parameter estimate. It should be noted that in general smoothing methods work because they exploit higher order smoothness in f . One simple modification to control under smoothing is to increase the cost, C, associated with each effective parameter in the curve (see Friedman Silverman (1989)[5]) This idea leads to GCV ( C) 1 n P n i=1 (y i Gamma b f (x i ) 2 (1 Gamma CtrA( n) 2 : 9) This modification produces a pole at the effective number of parameters equal to n=C and since tr(A( is a monotonic function of the result is a lower (although data dependent) limit on ....

.... fi 3;11 (x) f 3 (x) 1 3 fi 20;5 (x) 1 3 fi 12;12 (x) 1 3 fi 7;30 (x) f 4 (x) fi 50;50 (x) where fi p;q (x) is a beta density function with parameters p and q and x 2 [0; 1] The Motorcycle function is taken from a model of motorcycle impact data in Friedman Silverman (1989)[5], given here by f 5 (x) I x 0 sinf2 (1 Gamma x) 2 g, where x 2 [ Gamma0:2; 1] and I x 0 is 1 for positive x and zero elsewhere. This function violates the cubic smoothing spline assumptions since at x = 0 the second derivative is infinite. It is included in order to represent a more ....

Jerome H. Friedman and Bernard W. Silverman. Flexible parsimonious smoothing and additive modeling. Technometrics, 31:3--21, 1989.

....continuous derivatives everywhere. The pth derivative of m takes a jump of size b k at the kth knot, k . When fitting model (2) to noisy data, one must prevent overfitting which can cause near interpolation of the data. Methods for obtaining a smooth spline estimate include knot selection, e.g. Friedman and Silverman (1989), Friedman (1991) and Stone, Hansen, Kooperberg, and Truong (1997) and smoothing splines (Wahba, 1990; Eubank, 1988) With the first set of methods, the knots are selected from a set of candidate knots by a technique similar to stepwise regression and then, given the selected knots, the ....

Friedman, J.H., & Silverman, B.W. (1989), "Flexible parsimonious smoothing and additive modeling (with discussion)," Technometric, 31, 3--39.

....Section 7.2) pointed out that nding the right number and location of knots by visual inspection of the data is impossible in most cases. Therefore data driven methods for adaptive knot placement are needed for (in some sense) nearly optimal estimators f j . Frequentist approaches (see e.g. Friedman and Silverman, 1989, or Stone, Hansen, Kooperberg and Truong, 1997) use rst forward steps to add knots which are optimal with respect to some chosen criterion (for example Rao statistics) and afterwards delete knots in backward steps using another criterion (for example the AIC criterion) The results of these ....

Friedman, J. H. and Silverman, B. W. (1989). Flexible Parsimonious Smoothing and Additive Modeling (with discussion), Technometrics 31: 3-39.

....sacri ce interpretability in the pursuit of accuracy [e.g. Cheng and Titterington (1994) A popular alternative, that is both interpretable and exible, is provided by the use of local linear and piecewise linear models. See for example, Cleveland, Grosse and Shyu (1992) Fan and Gijbels (1996) Friedman and Silverman (1989) and the articles in Murray Smith and Johansen (1997) Worthy of special mention is the paper of Breiman (1993) who considered a piecewise linear model that is closest to the one we discuss here, which Breiman called hinging hyperplanes . The use of piecewise and local linear models has strong ....

Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modelling (with discussion), Technometrics 31: 3-39.

....are usually sufficient when the function g 0 does not exhibit dramatic changes in its derivatives. Non uniform knots are desirable when the function has very different local behaviors in different regions. we adopt a stepwise strategy for knot placement and deletion, in a way similar to that of Friedman and Silverman (1989). The stepwise selection procedure works as follows. We consider a subset of locations defined by the distinct values realized by the data set. Knots will be selected from this collection in order to follow the change in the curve while containing costs. Suppose that the first k knots fT i 1 ; T i ....

Friedman, J. H. and Silverman, B. W. (1989), Flexible parsimonious smoothing and additive modeling (with discussion), Technometrics, 31, 3-21.

....to get a good approximation of h. Once the spline knots (the parameters of the spline space) are given, the estimate can be obtained easily based on the least square principle. The spline knots can be determined with a forward backward algorithm which minimizes a model selection criterion (see [7], 8] and [9]for details) As we have mentioned above, to obtain a nonparametric regression estimate one has to determine the parameters for the estimate of h, for example, the bandwidth for kernels, the penalty parameter for smoothing splines, the spline knots for regression splines, etc. The ....

....[0; 1] determined by ft ji g,where max k (t jk Gamma t jk Gamma1 ) min k (t jk Gamma t jk Gamma1 ) ff 0 uniformly in j and n for some constant ff 0 . The details can be seen in [13] The spline knot t ji is placed on the i=k jn th quantile of the components of the observed regressors as in [7], where a knot placement and deletion procedure was proposed. Let B i 1 ; Delta Delta Delta;i J = B i 1 ; Delta Delta Delta;i J (x 1 ; Delta Delta Delta ; x J ) Q J j=1 j;i j (x j ) and (x) x 1 ; Delta Delta Delta, x J ) B 1; Delta Delta Delta;1 , Delta Delta Delta, B p1 ....

Friedman J H, Silverman B W. Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics, 1989, 31, 3-21.

.... best function estimate is defined as its minimizer and (ii) a knot deletion algorithm which attempts to locate this minimizer. Various non MDL based regression spline smoothing procedures have been proposed in the literature. They are chiefly based on cross validation or Bayesian approaches: Friedman Silverman (1989), Smith Kohn (1996) Luo Wahba (1997) and Denison, Mallick Smith (1998) Notice that most of these procedures fix the order of the spline a priori. 2 Nonparametric Regression as Model Selection Suppose that n pairs of measurements fx i ; y i g n i=1 , y i = f(x i ) ffl i , ffl i iid ....

Friedman, J. H. & Silverman, B. W. (1989), `Flexible parsimonious smoothing and additive modeling (with discussion)', Technometrics 31, 3--21.

....(u) p = u p I(u 0) and 1 Delta Delta Delta K are fixed knots. When fitting model (2) to noisy data, care is needed to prevent overfitting which causes a rough fit tending interpolate the data. The traditional methods of obtaining a smooth spline estimate are knot selection, e.g. Friedman Silverman (1989), Friedman (1991) and Stone, Hansen, Kooperberg, Truong (1997) and smoothing splines (Wahba, 1990; Eubank, 1988) With the first set of methods, the knots are selected from a set of candidate knots by a technique similar to stepwise regression and then, given the selected knots, the ....

Friedman, J.H., & Silverman, B.W. (1989), "Flexible parsimonious smoothing and additive modeling (with discussion)," Technometric, 31, 3--39.

....paper using polynomial splines for additive linear regression and well as additive logistic regression is Stone and Koo (1986a) in which knots were placed at nonadaptive (predetermined) quantiles. Stepwise knot selection, forward and backward, was used in the additive regression program TURBO by Friedman and Silverman (1989). A somewhat different approach to additive regression involving stepwise knot selection was developed by Breiman (1993) In the applications of cubic splines in these papers, linear constraints were placed on the tails of the splines mainly to control the variance of the corresponding estimates. ....

Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 31 3--39.

....(u) p : u p I(u 0) and 1 Delta Delta Delta K are fixed knots. When fitting model (2) to noisy data, care is needed to prevent overfitting which causes a rough fit tending interpolate the data. The traditional methods of obtaining a smooth spline estimate are knot selection, e.g. Friedman and Silverman (1989), Friedman (1991) and Stone, Hansen, Kooperberg, and Truong (1997) and smoothing splines (Wahba, 1990; Eubank, 1988) With the first set of methods, the knots are selected from a set of candidate knots by a technique similar to stepwise regression and then, given the selected knots, the ....

Friedman, J.H., and Silverman, B.W. (1989), "Flexible parsimonious smoothing and additive modeling (with discussion)," Technometric, 31, 3--39.

....finding the right number and location of knots by visual inspection of the data is impossible in most cases (see Eubank, 1988, Section 7.2) we need data driven methods for knot placement to get (in some sense) nearly optimal estimators f . For normal response y, such data driven methods exist. Friedman and Silverman (1989) present an adaptible knot placement algorithm with forward and backward steps. In the forward steps they add knots which are optimal with respect to the average squared residual criterion, while in the backward steps they delete knots yielding the model being optimal for the generalized ....

....for estimating the resulting generalized linear model (4) So, in contrast to Denison et al. 1998) where the estimation of the basis coefficients given the knots is done by ordinary least squares methods for normal response, we use a fully Bayesian approach in nonnormal cases. And contrary to Friedman and Silverman (1989) and Stone et al. 1997) where the result is one somehow optimal knot placement, the RJMCMC method can neither find one optimal number k of knots nor an optimal placement of these k knots. But in each iteration of the RJMCMC algorithm both the number of knots and the knot placement may vary. ....

Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion), Technometrics 31(1): 3--39.

.... points arise with increasing frequency in a range of applications, including dimension reduction methods such as projection pursuit and ACE (e.g. Friedman and Stuetzle 1981, Breiman and Friedman 1985, Huber 1985) flexible multivariate models for high dimensional data (e.g. Friedman 1988, 1991; Friedman and Silverman 1989), and generalized additive models (e.g. Hastie and Tibshirani 1986) In one dimension, the virtues of local linear smoothing are well known (e.g. Cleveland and Devlin 1988, Fan 1993, Hastie and Loader 1993) Multivariate generalisations of theoretical properties have been discussed by Ruppert and ....

Friedman, J.H. and Silverman, B.W. (1989), "Flexible Parsimonious Smoothing and Additive Modeling", Technometrics, 31, 3--39.

.... points arise with increasing frequency in a range of applications, including dimension reduction methods such as projection pursuit and ACE (e.g. Friedman and Stuetzle 1981, Breiman and Friedman 1985, Huber 1985) flexible multivariate models for high dimensional data (e.g. Friedman 1988, 1991; Friedman and Silverman 1989), and generalized additive models (e.g. Hastie and Tibshirani 1986, Cleveland and Devlin 1988) In one dimension, the virtues of local linear smoothing are well known (e.g. Fan 1993, Hastie and Loader 1993) Multivariate generalizations of theoretical properties have been discussed by Ruppert and ....

Friedman, J.H. and Silverman, B.W. (1989), "Flexible Parsimonious Smoothing and Additive Modeling", Technometrics, 31, 3--39.

....freedom of model (5) or an adaptively chosen cubic spline basis function should be between 1 and 2. Note that R 1 is only continuous, not differentiable, therefore the standard asymptotic theory does not apply. The simulation study done by Hinkley for a similar simple change point model, used in Friedman and Silverman (1989) for the purpose of deciding how many extra degrees of freedom should be given to an adaptively chosen basis function, indicates that the model sum of squares then is approximately distributed as 2 3 . This is supported also by Owen (1991) s theoretical argument. Another way to investigate the ....

Friedman, Jerome H. and Silverman, Bernard W. (1989). Flexible Parsimonious Smoothing and Additive Modeling. Technometrics, Vol. 31, No. 1, 3-39.

....see de Boor (1977, 1978) Cox (1981) or Dierckx (1993) The choice of knots has been a subject of much research: too many knots lead to overfitting of the data, too few knots lead to underfitting. Some authors have proposed automatic schemes for optimizing the number and the positions of the knots (Friedman and Silverman 1989; Kooperberg and Stone 1991) This is a difficult numerical problem and, to our knowledge, no attractive all purpose scheme exists. A different track was chosen by O Sullivan (1986, 1988) He proposed to use a relatively large number of knots. To prevent overfitting, a penalty on the second ....

Friedman, J. and Silverman, B.W. (1989) Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics. 31, 3--39.

....data indicates that a very small number of knots (1 k 6 ) are typically needed for data sets of modest size. The knot selection process described above applies to both constrained and unconstrained curve fitting. Stepwise knot selection methods have been used by a number of authors including Friedman and Silverman (1989), Kooperberg and Stone (1992) Shi and Li (1995) and He and Shi (1996) We start from a set of uniform knots (in percentile ranks) mainly for keeping the amount of computation as small as possible. A stepwise model building method by adding one knot at a time can also be used, but it is generally ....

Friedman, J. H. & Silverman, B. W. (1989), Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 31, 3-21.

....are many workers actively experimenting with representing empirical data as superposition of flexiblyspecified elements; the current effort might also be considered as related to this. The closest connection seems to come when we model smooth functions as piecewise polynomials with variable knots [38, 39, 55]. A stationary wavelet dictionary, based on the right choice of wavelet filters, consists of biorthogonal spline functions [11] A sparse decomposition in this dictionary, therefore, involves a piecewise polynomial CHAPTER 8. DISCUSSION 106 with a nearly arbitrary knot pattern. Here BP in a ....

J.H. Friedman and B.W. Silverman. Flexible parsimonious smoothing and additive modeling. Technometrics, vol. 31, pp. 3--39, 1989.

.... and Statistics, Peking University, Beijing 100871) ABSTRACT Based on Kullback Leibler information we propose a data driven selector, called GAIC (c) for choosing parameters of regression splines in nonparametric regression via a stepwise forward backward knot placement and deletion strategy [1] . This criterion unifies the commonly used information criteria and includes the Akaike information criterion (AIC) 2] and the corrected Akaike information criterion (AICC) 3] as special cases. To show the performance of GAIC (c) for c = 1=2, 3=4, 7=8, and 15=16, we compare it with ....

....noisy data Y 1 ; Delta Delta Delta ; Y n , where the Y i s are generated from the following model Y i = g(X i ) e i 1 i n; 1:1) the X i s are fixed design points and the e i s are independent, identically distributed (i.i.d. random variables of mean zero. Several authors, including [1], 4] 5] and [6] proposed to recover g(x) by regression spline methods. That is, for a spline basis 1 (x) Delta Delta Delta ; N (x) we obtain the estimator, g n (x) b = P n k=1 k (x) 0 b Nk , of g(x) by minimizing n X i=1 Y i Gamma N X k=1 k (X i ) k 2 0 ....

[Article contains additional citation context not shown here]

Friedman, J. H. and Silverman, B. W. (1989), Flexible parsimonious smoothing and additive modeling (with discussion), Technometrics, 31, 3-21.

....M procedure is good for the semiparametric regression model and (b) which of the two kinds of estimators, Huber s M estimators and the LS estimators, is better for finite samples when the error distribution is normal, symmetric contaminated normal, and Cauchy. In a way similar to that used in [17], 22] and [24] we implement the proposed method by adopting an automatically stepwise forward backward knot selection strategy. 3.1. Simulation Conditions We use Huber s function with c = 1:5 and set d = 3, fi 0 = 1; 2; 3) and g 0 (t) 2 Cos(3t ) fX i = x i1 ; x i2 ; x i3 ) ....

Friedman, J. H. and Silverman, B. W., Flexible parsimonious smoothing and additive modeling (with discussion), Technometrics, 31, 3-21, (1989).

....(6) the nonparametric regression problem can be expressed as the linear regression model y = Xfi e. The most important question associated with fitting such a regression spline is the choice of both the number and the location of the knots x 1 ; xm ; 17 see, for example, Friedman and Silverman (1989). If the knots are badly located, details of the curve can be missed, while if too many knots are included the fitted spline based on these knots will have high local variance. One way to solve this problem is to introduce a large number of knots from which a significant subset can be selected. ....

Friedman J. H. and Silverman, B.W. (1989), `Flexible parsimonious smoothing and additive modeling', Technometrics, 31, 3-39.

....parameters in a more ad hoc fashion with the estimates usually based on the values of the independent variables, but not the dependent variable. At present kernel based smoothing with the smoothing parameter(s) estimated by direct plugin seems confined to the univariate case. We also mention Friedman and Silverman (1989) who use regression splines for nonparametric regression and select the knots by a cross validation procedure. This is computationally very intensive, making it difficult to traverse all possible knot combinations when seeking optimal knot placement. More generally, it seems difficult to make the ....

....knots are badly located, details of the curve can be missed, while if too many knots are included the fitted spline based on these knots will have high local variance. One way solve the problem is to introduce a large number of potential knots from which a significant subset can be selected, e.g. Friedman and Silverman (1989, pp. 9 11) The problem then becomes one of variable selection where each knot corresponds to a column of a design matrix from which a significant subset is to be determined. Although the number of knots introduced, m, will typically be large so that r will be large, the number of significant ....

Friedman J. H. and B.W. Silverman , 1989, Flexible parsimonious smoothing and additive modeling. Technometrics 31, 3-39.

No context found.

J.H. Friedman and B.W. Silverman. Flexible parsimonious smoothing and additive modeling. Technometrics, vol. 31, pp. 3--39, 1989.

No context found.

Friedman, J.H. and Silverman, B., #1987# #Flexible parsimonious smoothing and additive modeling ",Stanford Technical Report, Sept. 1987.

No context found.

J. H. Friedman and B. W. Silverman. Flexible parsimonious smoothing and additive modeling. Technometrics, 31:3--39, 1989. 12

No context found.

Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 31 3--39.

No context found.

Friedman, J. H. and Silverman, B. W. (1989). Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 31 3--39.

No context found.

Friedman, J.H., and Silverman, B. W. (1989), Flexible parsimonious smoothing and additive modeling. Technometrics, 31, 3-39.

No context found.

Friedman, J. H. and Silverman, B. W. Flexible parsimonious smoothing and additive modeling (with discussion). TECHNOMETRICS. 1989.

No context found.

Friedman, J. H. and Silverman, B. W. (1989) Flexible parsimonious smoothing and additive modelling (with discussion). Technometrics, 31, 3--39.

No context found.

FRIEDMAN, J.H. & SILVERMAN, B.W. (1989). Flexible Parsimonious Smoothing and Additive Modeling. (with discussion). Technometrics 31, 3--39.

No context found.

Friedman, J.H. and Silverman, B.W. (1989) Flexible parsimonious smoothing and additive modeling. Technometrics 31, 3-21.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC