Penalized splines, or P-splines, are regression splines fit by least-squares with a roughness penaly. P-splines have much in common with smoothing splines, but the type of penalty used with a P-spline is somewhat more general than for a smoothing spline. Also, the number and location of the knots of a P-spline is not fixed as with a smoothing spline. Generally, the knots of a P-spline are at fixed quantiles of the independent variable and the only tuning parameter to choose is the number of knots. In this article, the effects of the number of knots on the performance of P-splines are studied. Two algorithms are proposed for the automatic selection of the number of knots. The myoptic algorithm stops when no improvement in the generalized cross validation statistic (GCV) is noticed with the last increase in the number of knots. The full search examines all candidates in a fixed sequence of possible numbers of knots and chooses the candidate that minimizes GCV. The myoptic algo...

this paper is a recent article on model-based geostatistics by Diggle, Tawn and Moyeed (1998) where pure kriging (i.e. no covariates) is the focus. Our paper inherits some of its aspects: model-based and with mixed model connections. In particular the comment by Bowman (1998) in the ensuing discussion suggested that additive modelling would be a worthwhile extension. This paper essentially follows this suggestion. However, this paper is not the first to combine the notions of geostatistics and additive modelling. References known to us are Kelsall and Diggle (1998), Durban Reguera (1998) and Durban, Hackett, Currie and Newton (2000). Nevertheless, we believe that our approach has a number of attractive features (see (1)-(4) above), not all shared by these references. Section 2 describes the motivating application and data in detail. Section 3 shows how one can express additive models as a mixed model, while Section 4 does the same for kriging and merges the two into the geoadditive model. Issues concerning the amount of smoothing are discussed in Section 5 and inferential aspects are treated in Section 6. Our analysis of the Upper Cape Cod reproductive data is presented in Section 7. Section 8 discusses extension to the generalised context.We close the paper with some disussion in Section 9. 2 Description of the application and data

We propose a new method for model selection and model fitting in nonparametric regression models, in the framework of smoothing spline ANOVA. The "COSSO" is a method of regularization with the penalty functional being the sum of component norms, instead of the squared norm employed in the traditional smoothing spline method. The COSSO provides a unified framework for several recent proposals for model selection in linear models and smoothing spline ANOVA models. Theoretical properties, such as the existence and the rate of convergence of the COSSO estimator, are studied. In the special case of a tensor product design with periodic functions, a detailed analysis reveals that the COSSO applies a novel soft thresholding type operation to the function components and selects the correct model structure with probability tending to one. We give

In the presence of covariate measurement error, estimating a regression function nonparametrically is extremely dicult, the problem being related to deconvolution. Various frequentist approaches exist for this problem, but to date there has been no Bayesian treatment. In this paper we describe Bayesian approaches to modeling a exible regression function when the predictor variable is measured with error. The regression function is modeled with smoothing splines and regression P{splines. Two methods are described for exploration of the posterior. The rst is called iterative conditional modes (ICM) and is only partially Bayesian. ICM uses a componentwise maximization routine to nd the mode of the posterior. It also serves to create starting values for the second method, which is fully Bayesian and uses Markov chain Monte Carlo techniques to generate observations from the joint posterior distribution. Using the MCMC approach has the advantage that interval estimates that directly model and adjust for the measurement error are easily calculated. We provide simulations with several nonlinear regression functions and provide an illustrative example. Our simulations indicate that the frequentist mean squared error properties of the fully Bayesian method are better than those of ICM and also of previously proposed frequentist methods, at least in the examples we have studied. KEY WORDS: Bayesian methods; Eciency; Errors in variables; Functional method; Generalized linear models; Kernel regression; Measurement error; Nonparametric regression; P{splines; Regression Splines; SIMEX; Smoothing Splines; Structural modeling. Short title. Nonparametric Regression with Measurement Error Author Aliations Scott M. Berry (E-mail: scott@berryconsultants.com) is Statistical Scientist,...

Abstract Penalized splines have gained much popularity as a flexible tool for smoothing and semi-parametric models. Two approaches have been advocated: 1) use a B-spline basis, equally-spaced knots and difference penalties

This paper presents a nonparametric penalized likelihood approach for variable selection and model building, called likelihood basis pursuit (LBP). In the setting of a tensor product reproducing kernel Hilbert space, we decompose the log likelihood into the sum of different functional components such as main effects and interactions, with each component represented by appropriate basis functions. The basis functions are chosen to be compatible with variable selection and model building in the context of a smoothing spline ANOVA model. Basis pursuit is applied to obtain the optimal decomposition in terms of having the smallest l 1 norm on the coefficients. We use the functional L 1 norm to measure the importance of each component and determine the "threshold" value by a sequential Monte Carlo bootstrap test algorithm. As a generalized LASSO-type method, LBP produces shrinkage estimates for the coefficients, which greatly facilitates the variable selection process, and provides highly interpretable multivariate functional estimates at the same time. To choose the regularization parameters appearing in the LBP models, generalized approximate cross validation (GACV) is derived as a tuning criterion. To make GACV widely applicable to large data sets, its randomized version is proposed as well. A technique "slice modeling" is used to solve the optimization problem and makes the computation more efficient. LBP has great potential for a wide range of research and application areas such as medical studies, and in this paper we apply it to two large on-going epidemiological studies: the Wisconsin Epidemiological Study of Diabetic Retinopathy (WESDR) and the Beaver Dam Eye Study (BDES).

Penalised–spline–based additive models allow a simple mixed model representation where the variance components control departures from linear models. The smoothing parameter is the ratio between the random-coefficient and error variances and tests for linear regression reduce to tests for zero random-coefficient variances. We propose exact likelihood and restricted likelihood ratio tests for testing polynomial regression versus a general alternative modelled by penalised splines. Their spectral decompositions are used as the basis of fast simulation algorithms. We derive the asymptotic local power properties of the tests under weak conditions. In particular we characterise the local alternatives that are detected with asymptotic probability one. Confidence intervals for the smoothing parameter are obtained by inverting the tests for a fixed smoothing parameter versus a general alternative. We discuss F and R tests and show that ignoring the variability in the smoothing parameter estimator can have a dramatic effect on their null distributions. The powers of several known tests are investigated and a small set of tests with good power properties is identified. The restricted likelihood ratio test is among the best in terms of power.

This article presents penalized spline models that incorporate factor by curve interactions into additive models. A mixed model formulation for penalized splines allows for straightforward model fitting, smoothing parameter selection, and hypothesis testing. We illustrate the proposed model by applying it to pollen ragweed data in which seasonal trends vary by year.

Penalized spline-based additive models allow a simple mixed model representation where the variance components control departures from linear models. The smoothing parameter is the ratio between the random-coefficient and error variances and tests for linear regression reduce to tests for zero random-coefficient variances. We propose exact likelihood and restricted likelihood ratio tests, (R)LRTs, for testing polynomial regression versus a general alternative modeled by penalized splines. Their spectral decompositions are used as the basis of fast simulation algorithms. We derive the asymptotic local power properties of (R)LRTs under weak conditions. In particular we characterize the local alternatives that are detected with asymptotic probability 1. Confidence intervals for the smoothing parameter are obtained by inverting the (R)LRT for a fixed smoothing parameter versus a general alternative. We discuss F and R tests and show that ignoring the variability in the smoothing parameter estimator can have a dramatic effect on their null distributions. The power of several known tests is investigated and a small set of tests with good power properties is identified. Some key words: Linear mixed models, penalized splines, smoothing, zero variance components.

Numerous nonparametric regression methods exist which yield consistent estimators of function curves. Often one is also interested in constructing confidence intervals for the unknown function. Pointwise confidence intervals are available using globally crossvalidated smoothing spline (GCV) estimation. When the function estimate is based on a single global smoothing parameter the resulting confidence intervals may hold their desired confidence level 1 \Gamma ff on average but because bias in nonparametric estimation is not uniform, they do not hold the desired level uniformly at all design points. To deal with this problem, a new smoothing spline estimator is developed which uses a local cross-validation (LCV) criterion to determine a separate smoothing parameter for each design point. The local smoothing parameters are then used to compute the point estimators of the regression curve and the corresponding pointwise confidence intervals. Incorporation of local information th...