Results 1  10
of
20
Confidence intervals and hypothesis testing for highdimensional regression. arXiv: 1306.3171
"... Fitting highdimensional statistical models often requires the use of nonlinear parameter estimation procedures. As a consequence, it is generally impossible to obtain an exact characterization of the probability distribution of the parameter estimates. This in turn implies that it is extremely cha ..."
Abstract

Cited by 31 (3 self)
 Add to MetaCart
Fitting highdimensional statistical models often requires the use of nonlinear parameter estimation procedures. As a consequence, it is generally impossible to obtain an exact characterization of the probability distribution of the parameter estimates. This in turn implies that it is extremely challenging to quantify the uncertainty associated with a certain parameter estimate. Concretely, no commonly accepted procedure exists for computing classical measures of uncertainty and statistical significance as confidence intervals or pvalues. We consider here a broad class regression problems, and propose an efficient algorithm for constructing confidence intervals and pvalues. The resulting confidence intervals have nearly optimal size. When testing for the null hypothesis that a certain parameter is vanishing, our method has nearly optimal power. Our approach is based on constructing a ‘debiased ’ version of regularized Mestimators. The new construction improves over recent work in the field in that it does not assume a special structure on the design matrix. Furthermore, proofs are remarkably simple. We test our method on a diabetes prediction problem. 1
Highdimensional Inference: Confidence intervals, pvalues and Rsoftware hdi. arXiv:1408.4026v1
, 2014
"... Abstract. We present a (selective) review of recent frequentist highdimensional inference methods for constructing pvalues and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which complements the viewpoint from statistical methodology ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
Abstract. We present a (selective) review of recent frequentist highdimensional inference methods for constructing pvalues and confidence intervals in linear and generalized linear models. We include a broad, comparative empirical study which complements the viewpoint from statistical methodology and theory. Furthermore, we introduce and illustrate the Rpackage hdi which easily allows the use of different methods and supports reproducibility.
Selecting the number of principal components: estimation of the true rank of a noisy matrix
"... Principal component analysis (PCA) is a wellknown tool in multivariate statistics. One big challenge in using the method is the choice of the number of components. In this paper, we propose an exact distributionbased method for this purpose: our approach is related to the adaptive regression frame ..."
Abstract

Cited by 2 (1 self)
 Add to MetaCart
Principal component analysis (PCA) is a wellknown tool in multivariate statistics. One big challenge in using the method is the choice of the number of components. In this paper, we propose an exact distributionbased method for this purpose: our approach is related to the adaptive regression framework of Taylor et al. (2013). Assuming Gaussian noise, we use the conditional distribution of the eigenvalues of a Wishart matrix as our test statistic, and derive exact hypothesis tests and confidence intervals for the true singular values. In simulation studies we find that our proposed method compares well to the proposal of Kritchman & Nadler (2008), which uses the asymptotic distribution of singular values based on the TracyWidom laws.
The Cluster Elastic Net for HighDimensional Regression With Unknown Variable Grouping
, 2013
"... In the highdimensional regression setting, the elastic net produces a parsimonious model by shrinking all coefficients towards the origin. However, in certain settings, this behavior might not be desirable: if some features are highly correlated with each other and associated with the response, the ..."
Abstract

Cited by 1 (0 self)
 Add to MetaCart
(Show Context)
In the highdimensional regression setting, the elastic net produces a parsimonious model by shrinking all coefficients towards the origin. However, in certain settings, this behavior might not be desirable: if some features are highly correlated with each other and associated with the response, then we might wish to perform less shrinkage on the coefficients corresponding to that subset of features. We propose the cluster elastic net, which selectively shrinks the coefficients for such variables towards each other, rather than towards the origin. Instead of assuming that the clusters are known a priori, the cluster elastic net infers clusters of features from the data, on the basis of correlation among the variables as well as association with the response. These clusters are then used in order to more accurately perform regression. We demonstrate the theoretical advantages of our proposed approach, and explore its performance in a simulation study, and in an application to HIV drug resistance data. Supplementary Materials are available online.
New York University
, 2015
"... Crossvalidation and hypothesis testing in neuroimaging: an irenic comment on the exchange between Friston and Lindquist et al. ..."
Abstract
 Add to MetaCart
(Show Context)
Crossvalidation and hypothesis testing in neuroimaging: an irenic comment on the exchange between Friston and Lindquist et al.
Statistical learning and selective inference
"... We describe the problem of "selective inference." This addresses the following challenge: Having mined a set of data to find potential associations, how do we properly assess the strength of these associations? The fact that we have "cherrypicked"searched for the strongest ass ..."
Abstract
 Add to MetaCart
We describe the problem of "selective inference." This addresses the following challenge: Having mined a set of data to find potential associations, how do we properly assess the strength of these associations? The fact that we have "cherrypicked"searched for the strongest associationsmeans that we must set a higher bar for declaring significant the associations that we see. This challenge becomes more important in the era of big data and complex statistical modeling. The cherry tree (dataset) can be very large and the tools for cherry picking (statistical learning methods) are now very sophisticated. We describe some recent new developments in selective inference and illustrate their use in forward stepwise regression, the lasso, and principal components analysis. inference  P values  lasso
EigenPrism: Inference for HighDimensional SignaltoNoise Ratios
"... Abstract Consider the following three important problems in statistical inference, namely, constructing confidence intervals for (1) the error of a highdimensional (p > n) regression estimator, (2) the linear regression noise level, and (3) the genetic signaltonoise ratio of a continuousvalu ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Consider the following three important problems in statistical inference, namely, constructing confidence intervals for (1) the error of a highdimensional (p > n) regression estimator, (2) the linear regression noise level, and (3) the genetic signaltonoise ratio of a continuousvalued trait (related to the heritability). All three problems turn out to be closely related to the littlestudied problem of performing inference on the 2 norm of the coefficient vector in highdimensional linear regression. We derive a novel procedure for this, which is asymptotically correct and produces valid confidence intervals in finite samples as well. The procedure, called EigenPrism, is computationally fast and makes no assumptions on coefficient sparsity or knowledge of the noise level. We investigate the width of the EigenPrism confidence intervals, including a comparison with a Bayesian setting in which our interval is just 5% wider than the Bayes credible interval. We are then able to unify the three aforementioned problems by showing that the EigenPrism procedure with only minor modifications is able to make important contributions to all three. We also investigate the robustness of coverage and find that the method applies in practice and in finite samples much more widely than in the regime covered by the theory. Finally, we apply EigenPrism to a genetic dataset to estimate the genetic signaltonoise ratio for a number of continuous phenotypes.
Inference for Highdimensional Exponential Family Graphical Models
"... Abstract Probabilistic graphical models have been widely used to model complex systems and aid scientific discoveries. Most existing work on highdimensional estimation of exponential family graphical models, including Gaussian and Ising models, is focused on consistent model selection. However, the ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Probabilistic graphical models have been widely used to model complex systems and aid scientific discoveries. Most existing work on highdimensional estimation of exponential family graphical models, including Gaussian and Ising models, is focused on consistent model selection. However, these results do not characterize uncertainty in the estimated structure and are of limited value to scientists who worry whether their findings will be reproducible and if the estimated edges are present in the model due to random chance. In this paper, we propose a novel estimator for edge parameters in an exponential family graphical models. We prove that the estimator is √ nconsistent and asymptotically Normal. This result allows us to construct confidence intervals for edge parameters, as well as, hypothesis tests. We establish our results under conditions that are typically assumed in the literature for consistent estimation. However, we do not require that the estimator consistently recovers the graph structure. In particular, we prove that the asymptotic distribution of the estimator is robust to model selection mistakes and uniformly valid for a large number of datagenerating processes. We illustrate validity of our estimator through extensive simulation studies.
LETTER Communicated by Ilya M. Nemenman Bayesian Feature Selection with Strongly Regularizing Priors Maps to the Ising Model
"... Identifying small subsets of features that are relevant for prediction and classification tasks is a central problem inmachine learning and statistics. The feature selection task is especially important, and computationally difficult, for modern data sets where the number of features can be compara ..."
Abstract
 Add to MetaCart
(Show Context)
Identifying small subsets of features that are relevant for prediction and classification tasks is a central problem inmachine learning and statistics. The feature selection task is especially important, and computationally difficult, for modern data sets where the number of features can be comparable to or even exceed the number of samples. Here, we show that feature selection with Bayesian inference takes a universal form and reduces to calculating the magnetizations of an Ising model under some mild conditions. Our results exploit the observation that the evidence takes a universal form for strongly regularizing priors—priors that have a large effect on the posterior probability even in the infinite data limit. We derive explicit expressions for feature selection for generalized linear models, a large class of statistical techniques that includes linear and logistic regression. We illustrate the power of our approach by analyzing feature selection in a logistic regressionbased classifier trained to distinguish between the letters B and D in the notMNIST data set. 1