Results 11  20
of
98
Penalized Likelihood Methods for Estimation of sparse high dimensional directed acyclic graphs
, 2010
"... Directed acyclic graphs are commonly used to represent causal relationships among random variables in graphical models. Applications of these models arise in the study of physical, as well as biological systems, where directed edges between nodes represent the influence of components of the system o ..."
Abstract

Cited by 20 (8 self)
 Add to MetaCart
Directed acyclic graphs are commonly used to represent causal relationships among random variables in graphical models. Applications of these models arise in the study of physical, as well as biological systems, where directed edges between nodes represent the influence of components of the system on each other. Estimation of directed graphs from observational data is computationally NPhard. In addition, directed graphs with the same structure may be indistinguishable based on observations alone. When the nodes exhibit a natural ordering, the problem of estimating directed graphs reduces to the problem of estimating the structure of the network. In this paper, we propose an efficient penalized likelihood method for estimation of the adjacency matrix of directed acyclic graphs, when variables inherit a natural ordering. We study variable selection consistency of both the lasso, as well as the adaptive lasso penalties in high dimensional sparse settings, and propose an errorbased choice for selecting the tuning parameter. We show that although the lasso is only variable selection consistent under stringent conditions, the adaptive lasso can consistently estimate the true graph under the usual regularity assumptions. Simulation studies indicate that the correct ordering of the variables becomes less critical in estimation of high dimensional sparse networks.
Supplement to “Needles and Straw in a Haystack: Posterior concentration for possibly sparse sequences.” DOI:10.1214/12AOS1029SUPP
, 2012
"... ar ..."
Discussion of “Onestep sparse estimates in nonconcave penalized likelihood models” (auths
, 2007
"... Hui Zou and Runze Li ought to be congratulated for their nice and interesting work which presents a variety of ideas and insights in statistical methodology, computing and asymptotics. We agree with them that one or even multistep (orstage) procedures are currently among the best for analyzing co ..."
Abstract

Cited by 14 (3 self)
 Add to MetaCart
(Show Context)
Hui Zou and Runze Li ought to be congratulated for their nice and interesting work which presents a variety of ideas and insights in statistical methodology, computing and asymptotics. We agree with them that one or even multistep (orstage) procedures are currently among the best for analyzing complex datasets. The focus of our discussion is mainly on highdimensional problems where p ≫ n: we will illustrate, empirically and by describing some theory, that many of the ideas from the current paper are very useful for the p ≫ n setting as well. 1. Nonconvex objective function and multistep convex optimization. The paper demonstrates a nice, and in a sense surprising, connection between difficult nonconvex optimization and computationally efficient Lassotype methodology which involves one (or multi) step convex optimization. The SCADpenalty function [5] has been often criticized from a computational point of view as it corresponds to a nonconvex objective function which is difficult to minimize; mainly in situations with many covariates, optimizing SCADpenalized likelihood becomes an awkward task. The usual way to optimize a SCADpenalized likelihood is to use a local quadratic approximation. Zou and Li show here what happens if one uses a local linear approximation instead. In 2001, when Fan and Li [5] proposed the SCADpenalty, it was probably easier to work with a quadratic approximation. Nowadays, and because of the contribution of the current paper, a local linear approximation seems as easy to use, thanks to the homotopy method [12] and the LARS algorithm [4]. While the latter is suited for linear models, more sophisticated algorithms have been proposed for generalized linear models; cf. [6, 8, 13]. In addition, and importantly, the local linear approximation yields sparse model fits where quite a few or even many of the coefficients in a linear or
Sharp Support Recovery from Noisy Random Measurements by ℓ1 minimization
, 2011
"... In this paper, we investigate the theoretical guarantees of penalized ℓ1minimization (also called Basis Pursuit Denoising or Lasso) in terms of sparsity pattern recovery (support and sign consistency) from noisy measurements with nonnecessarily random noise, when the sensing operator belongs to th ..."
Abstract

Cited by 11 (5 self)
 Add to MetaCart
In this paper, we investigate the theoretical guarantees of penalized ℓ1minimization (also called Basis Pursuit Denoising or Lasso) in terms of sparsity pattern recovery (support and sign consistency) from noisy measurements with nonnecessarily random noise, when the sensing operator belongs to the Gaussian ensemble (i.e. random design matrix with i.i.d. Gaussian entries). More precisely, we derive sharp nonasymptotic bounds on the sparsity level and (minimal) signaltonoise ratio that ensure support identification for most signals and most Gaussian sensing matrices by solving the Lasso with an appropriately chosen regularization parameter. Our first purpose is to establish conditions allowing exact sparsity pattern recovery when the signal is strictly sparse. Then, these conditions are extended to cover the compressible or nearly sparse case. In these two results, the role of the minimal signaltonoise ratio is crucial. Our third main result gets rid of this assumption in the strictly sparse case, but this time, the Lasso allows only partial recovery of the support. We also provide in this case a sharp ℓ2consistency result on the coefficient vector. The results of the present work have several distinctive features compared to previous ones. One of them is that the leading constants involved in all the bounds are sharp and explicit. This is illustrated by some numerical experiments where it is indeed shown that the sharp sparsity level threshold identified by our theoretical results below which sparsistency of the Lasso solution is guaranteed meets the one empirically observed.
Highdimensional regression with unknown variance
 SUBMITTED TO THE STATISTICAL SCIENCE
, 2012
"... We review recent results for highdimensional sparse linear regression in the practical case of unknown variance. Different sparsity settings are covered, including coordinatesparsity, groupsparsity and variationsparsity. The emphasis is put on nonasymptotic analyses and feasible procedures. In ..."
Abstract

Cited by 10 (1 self)
 Add to MetaCart
We review recent results for highdimensional sparse linear regression in the practical case of unknown variance. Different sparsity settings are covered, including coordinatesparsity, groupsparsity and variationsparsity. The emphasis is put on nonasymptotic analyses and feasible procedures. In addition, a small numerical study compares the practical performance of three schemes for tuning the Lasso estimator and some references are collected for some more general models, including multivariate regression and nonparametric regression.
ModelConsistent Sparse Estimation through the Bootstrap
, 2009
"... We consider the leastsquare linear regression problem with regularization by the ℓ 1norm, a problem usually referred to as the Lasso. In this paper, we first present a detailed asymptotic analysis of model consistency of the Lasso in lowdimensional settings. For various decays of the regularizati ..."
Abstract

Cited by 9 (0 self)
 Add to MetaCart
(Show Context)
We consider the leastsquare linear regression problem with regularization by the ℓ 1norm, a problem usually referred to as the Lasso. In this paper, we first present a detailed asymptotic analysis of model consistency of the Lasso in lowdimensional settings. For various decays of the regularization parameter, we compute asymptotic equivalents of the probability of correct model selection. For a specific rate decay, we show that the Lasso selects all the variables that should enter the model with probability tending to one exponentially fast, while it selects all other variables with strictly positive probability. We show that this property implies that if we run the Lasso for several bootstrapped replications of a given sample, then intersecting the supports of the Lasso bootstrap estimates leads to consistent model selection. This novel variable selection procedure, referred to as the Bolasso, is extended to highdimensional settings by a provably consistent twostep procedure.
Estimation and Selection via Absolute Penalized Convex Minimization And Its Multistage Adaptive Applications
"... The ℓ1penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of highdimensional statistical problems. In this article, we consider a ..."
Abstract

Cited by 7 (3 self)
 Add to MetaCart
(Show Context)
The ℓ1penalized method, or the Lasso, has emerged as an important tool for the analysis of large data sets. Many important results have been obtained for the Lasso in linear regression which have led to a deeper understanding of highdimensional statistical problems. In this article, we consider a class of weighted ℓ1penalized estimators for convex loss functions of a general form, including the generalized linear models. We study the estimation, prediction, selection and sparsity properties of the weighted ℓ1penalized estimator in sparse, highdimensional settings where the number of predictors p can be much larger than the sample size n. Adaptive Lasso is considered as a special case. A multistage method is developed to approximate concave regularized estimation by applying an adaptive Lasso recursively. We provide prediction and estimation oracle inequalities for single and multistage estimators, a general selection consistency theorem, and an upper bound for the dimension of the Lasso estimator. Important models including the linear regression, logistic regression and loglinear models are used throughout to illustrate the applications of the general results.
Smoothing ℓ1penalized estimators for highdimensional timecourse data
 Electronic Journal of Statistics
, 2007
"... Abstract: When a series of (related) linear models has to be estimated it is often appropriate to combine the different datasets to construct more efficient estimators. We use ℓ1penalized estimators like the Lasso or the Adaptive Lasso which can simultaneously do parameter estimation and model sel ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
Abstract: When a series of (related) linear models has to be estimated it is often appropriate to combine the different datasets to construct more efficient estimators. We use ℓ1penalized estimators like the Lasso or the Adaptive Lasso which can simultaneously do parameter estimation and model selection. We show that for a timecourse of highdimensional linear models the convergence rates of the Lasso and of the Adaptive Lasso can be improved by combining the different timepoints in a suitable way. Moreover, the Adaptive Lasso still enjoys oracle properties and consistent variable selection. The finite sample properties of the proposed methods are illustrated on simulated data and on a real problem of motif finding in DNA sequences.