xical, but they are defined below by taking very literally the notion that likelihood is the probability of observing the actual data values at hand. 1 Nonparametric Maximum Likelihood The empirical distribution function is well known as a nonparametric maximum likelihood estimate (NPMLE). If X 1 ; : : : ; X n are i.i.d. in R d from some distribution F 0 , the empirical distribution function is F n = 1 n n X i=1 ffi X i where ffi X is a distribution taking the value X with probability one. The probability that F n attaches to a set A, denoted F n (A), is 1=n times the number of sample observations belonging to A. In particular,<F

One-step efficient GMM estimation has been developed in the recent papers of Back and Brown (1990), Imbens (1993) and Qin and Lawless (1994). These papers emphasized methods that correspond to using Owen's (1988) method of empirical likelihood to reweight the data so that the reweighted sample obeys all the moment restrictions at the parameter estimates. In this paper we consider an alternative KLIC motivated weighting and show how it and similar discrete reweightings define a class of unconstrained optimization problems which includes GMM as a special case. Such KLIC-- motivated reweightings introduce M auxiliary `tilting' parameters, where M is the number of moments; parameter and overidentification hypotheses can be recast in terms of these tilting parameters. Such tests, when appropriately conditioned on the estimates of the original parameters, are often startlingly more effective than their conventional counterparts. This is apparently due to the local ancillarity of the original parameters for the tilting parameters. 1.

Microeconometrics researchers have increasingly realized the essential need to account for any within-group dependence in estimating standard errors of regression parameter estimates. The typical preferred solution is to calculate cluster-robust or sandwich standard errors that permit quite general heteroskedasticity and within-cluster error correlation, but presume that the number of clusters is large. In applications with few (5-30) clusters, standard asymptotic tests can overreject considerably. We investigate more accurate inference using cluster bootstrap-t procedures that provide asymptotic refinement. These procedures are evaluated using Monte Carlos, including the much-cited differences-in-differences example of Bertrand, Mullainathan and Duflo (2004). In situations where standard methods lead to rejection rates in excess of ten percent (or more) for tests of nominal size 0.05, our methods can reduce this to five percent. In principle a pairs cluster bootstrap should work well, but in practice a wild cluster bootstrap performs better.

. A class of weighted-bootstrap techniques, called biasedbootstrap methods, is proposed. It is motivated by the need to adjust more conventional, uniform-bootstrap methods in a surgical way, so as to alter some of their features while leaving others unchanged. Depending on the nature of the adjustment, the biased bootstrap can be used to reduce bias, or reduce variance, or render some characteristic equal to a predetermined quantity. More specifically, applications of bootstrap methods include hypothesis testing, variance stabilisation, both density estimation and nonparametric regression under constraints, `robustification ' of general statistical procedures, sensitivity analysis, generalised method of moments, shrinkage, and many more. 1991 Mathematics Subject Classification: Primary 62G09, Secondary 62G05 Keywords and Phrases: Bias reduction, empirical likelihood, hypothesis testing, local-linear smoothing, nonparametric curve estimation, variance stabilisation, weighted bootstrap 1...

Importance sampling in Monte Carlo simulation is the process of estimating a distribution using observations from a different distribution. Estimates are computed using weights that are roughly proportional to the likelihood ratio between the two distributions. Importance sampling has been very successful as a variance reduction technique in rare event applications. It can also be applied in many other applications, as a variance reduction technique, as a means of solving a problem that is otherwise intractable, or for analyzing the performance of an estimate or a physical process under multiple input distributions using a single set of observations, as in response surface estimation or in the analysis of robust estimates. The classical importance sampling estimate is well-suited for variance reduction in rare event applications. It fails in many other applications.

We construct bootstrap confidence intervals for smoothing spline and smoothing spline ANOVA estimates based on Gaussian data, and penalized likelihood smoothing spline estimates based on data from exponential families. Several variations of bootstrap confidence intervals are considered and compared. We find that the commonly used bootstrap percentile intervals are inferior to the T intervals and to intervals based on bootstrap estimation of mean squared errors. The best variations of the bootstrap confidence intervals behave similar to the well known Bayesian confidence intervals. These bootstrap confidence intervals have an average coverage probability across the function being estimated, as opposed to a pointwise property. Keywords: BAYESIAN CONFIDENCE INTERVALS, BOOTSTRAP CONFIDENCE INTERVALS, PENALIZED LOG LIKELIHOOD ESTIMATES, SMOOTHING SPLINES, SMOOTHING SPLINE ANOVA'S. 1 Introduction Smoothing splines and smoothing spline ANOVAs (SS ANOVAs) have been used successfully in a bro...

A Monte Carlo method for computing the bias and standard deviation of estimates of the parameters of a psychometricfunction such as the WeibulllQuick is described. The method, based on Efron’s parametric bootstrap, can also be used to estimate confidence intervals for these parameters. The method’s ability to predict bias, standard deviation, and confidence intervals is evaluated in two ways. First, its predictions are compared tothe outcomes of Monte Carlo simulations ofpsychophysical experiments. Second, its predicted confidenceintervals were compared with the actual variability of human observers in a psychophysical task. Computer programs implementing the method are available from the author. The performance of an observer in a detection or discrimination task is typically summarized by fitting a psychometric function to the data. Examples of fitting methods include probit analysis (Finney, 1971) and maximum-likelihood fits using the Weibull/Quick psychometric function (Quick, 1974; Watson, 1979; Weibull, 1951). These methods retain an estimate of threshold and

For conventional single-event, nonlinear, least-squares hypocentral estimates, I show that the total error is expressible as a linear combination of three terms: (1) measurement error; (2) modeling errors caused by inadequacy of the travel-time tables; and (3) a nonlinear term. Errors in calculating travel-time partial derivatives are shown to have no effect, provided a stable solution can be found. This is in contrast to linear problems where errors in calculating matrix elements can distort the solution drastically. The error appraisal technique developed here examines each of the three error terms independently. The first can be analyzed by standard confidence ellipsoids with critical values based on measurement error statistics. The second can cause conventional error ellipsoid calculations that derive a critical value from an estimate based on rms residuals, to give misleading results. I introduce an alternative extremal bound procedure for appraising such errors. Travel-time modeling errors are bounded as the product of ray arc length and an estimate of the nominal scale of slowness errors along the ray path. These are used to derive an upper bound on systematic errors in each hypocentral coordinate based on a novel bounding criteria. Finally, I show that, for errors of a reasonable scale, the nonlinear error term can be estimated adequately using a second-order approximation. Given an upper bound on the total location error, bounds on the travel-time error induced by nonlinearity can be calculated from the spectral norm of the Hessian for each measured arrival time. The systematic errors in each hypocentral coordinate due to nonlinearity can then be bounded using the same criteria used for constructing modeling error bounds. This overall procedure is complete because it allows one to independently appraise the relative importance of all sources of hypocentral errors. It is practical because the required computational effort is small.

This paper compares different procedures to compute confidence intervals for parameters and quantiles of the Weibull, lognormal, and similar log-location-scale distributions from Type I censored data that typically arise from life test experiments. The procedures can be classified into three groups. The first group contains procedures based on the commonlyused normal approximation for the distribution of studentized (possibly after a transformation) maximum likelihood estimators. The second group contains procedures based on the likelihood ratio statistic and its modifications. The procedures in the third group use a parametric bootstrap approach, including the use of bootstrap-type simulation, to calibrate the procedures in the first two groups. The procedures in all three groups are justified on the basis of large-sample asymptotic theory. We use Monte Carlo simulation to investigate the finite sample properties of these procedures. Details are reported for the Weibull distribution. ...

. We suggest a general method for tackling problems of density estimation under constraints. It is in effect a particular form of the weighted bootstrap, in which resampling weights are chosen so as to minimise distance from the empirical or uniform bootstrap distribution subject to the constraints being satisfied. A number of constraints are treated as examples. They include conditions on moments, quantiles and entropy, the latter as a device for imposing qualitative conditions such as those of unimodality or "interestingness." For example, without altering the data or the amount of smoothing we may construct a density estimator that enjoys the same mean, median and quartiles as the data. Different measures of distance give rise to slightly different results. KEYWORDS. Biased bootstrap, Cressie-Read distance, curve estimation, empirical likelihood, entropy, kernel methods, mode, smoothing, weighted bootstrap. SHORT TITLE. Constrained density estimation. AMS SUBJECT CLASSIFICATION. ...