#### DMCA

## SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR (2007)

### Cached

### Download Links

- [www.stat.berkeley.edu]
- [www.cmi.univ-mrs.fr]
- [eprints.pascal-network.org]
- [www.latp.univ-mrs.fr]
- [www.proba.jussieu.fr]
- [www.cmi.univ-mrs.fr]
- [www.latp.univ-mrs.fr]
- [www.cmi.univ-mrs.fr]
- [www.latp.univ-mrs.fr]
- [www.stat.berkeley.edu]
- [www.stat.berkeley.edu]
- [stat-www.berkeley.edu]
- [stat-www.berkeley.edu]
- [pluto.mscc.huji.ac.il]
- [pluto.mscc.huji.ac.il]
- [www.stat.berkeley.edu]
- [www.stat.berkeley.edu]
- [www.stat.berkeley.edu]
- [stat-www.berkeley.edu]
- [pluto.huji.ac.il]
- [pluto.mscc.huji.ac.il]
- [pluto.huji.ac.il]
- [arxiv.org]
- [arxiv.org]

Venue: | SUBMITTED TO THE ANNALS OF STATISTICS |

Citations: | 471 - 11 self |

### Citations

4203 | Regression shrinkage and selection via the lasso - Tibshirani - 1996 |

1325 | R.: Least angle regression
- Efron, Hastie, et al.
- 2004
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

875 | The Dantzig selector: Statistical estimation when p is much larger than n
- Candès, Tao
- 2007
(Show Context)
Citation Context ...estimator of parameters in highdimensional linear regression when the number of variables can be much larger than the sample size [8, 9, 11, 17, 18, 20–22, 26] and[27]. Quite recently, Candes and Tao =-=[7]-=- have proposed a new estimate for such linear models, the Dantzig selector, for which they establish optimal ℓ2 rate properties under a sparsity scenario; that is, when the number of nonzero component... |

735 | High-dimensional graphs and variable selection with the - Meinshausen, Bühlmann - 2006 |

698 | Regression shrinkage and selection via the - Tibshirani - 1996 |

460 | Stable recovery of sparse overcomplete representations in the presence of noise
- Donoho, Elad, et al.
(Show Context)
Citation Context ...c| 0 and, if (4.1) holds, |δ T J0XT ∣ XδJ c|/n ≤|δ 0 J c|1 max ∣ /n 0 j∈J c 0 ≤ θs,1|δ J c 0 |1|δJ0 |2 √ ≤ c0θs,1 s|δJ0 |22 . ∣ δ T J0 XT x(j) Another type of assumption related to “mutual coherence” =-=[8]-=- is discussed in connection to Lasso in [4, 5]. We state it in two different forms, which are given below.1714 P. J. BICKEL, Y. RITOV AND A. B. TSYBAKOV ASSUMPTION 4. where c0 > 0 is a constant. (4.2... |

324 | Pathwise coordinate optimization
- Friedman, Hastie, et al.
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

276 | The group lasso for logistic regression
- Meier, Geer, et al.
- 2008
(Show Context)
Citation Context ... estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to [9, 10, 20, 21, 24] and=-=[16]-=- for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the convex funct... |

262 | Asymptotics for Lasso-type estimators. - Knight, Fu - 2000 |

244 | A new approach to variable selection in least squares problems.
- Osborne, Presnell, et al.
- 2000
(Show Context)
Citation Context ...the corresponding Lasso estimator M∑ (2.2) ̂fL(x) = f̂βL (x) = ̂βj,Lfj (z). The criterion in (2.1) isconvexinβ, so that standard convex optimization procedures can be used to compute ̂βL. We refer to =-=[9, 10, 20, 21, 24]-=- and[16] for detailed discussion of these optimization problems and fast algorithms. A necessary and sufficient condition of the minimizer in (2.1) is that 0 belongs to the subdifferential of the conv... |

209 | The Lasso and its dual.
- OSBORNE, PRESNELL, et al.
- 2000
(Show Context)
Citation Context |

171 | Sparsity oracle inequalities for the lasso. - Bunea, Tsybakov, et al. - 2007 |

145 |
On model selection consistency of
- Zhao, Yu
- 2006
(Show Context)
Citation Context ...heℓ1 penalized least squares (Lasso) estimator of parameters in highdimensional linear regression when the number of variables can be much larger than the sample size [8, 9, 11, 17, 18, 20–22, 26] and=-=[27]-=-. Quite recently, Candes and Tao [7] have proposed a new estimate for such linear models, the Dantzig selector, for which they establish optimal ℓ2 rate properties under a sparsity scenario; that is, ... |

144 | Aggregation for Gaussian regression.
- Bunea, Tsybakov, et al.
- 2007
(Show Context)
Citation Context ... study these two estimators in parallel. For the fixed design Gaussian regression model, we recover, as particular cases, sparsity oracle inequalities for the Lasso, as in Bunea, Tsybakov and Wegkamp =-=[4]-=-, and ℓ2 bounds for the coefficients of Dantzig selector, as in Candes and Tao [7]. This is obtained as a consequence of our more general results, which are the following: • In the nonparametric regre... |

128 | High-dimensional generalized linear models and the lasso - Geer - 2008 |

119 | Persistency in high dimensional linear predictorselection and the virtue of over-parametrization, - Greenshtein, Ritov - 2004 |

60 | The Dantzig selector and sparsity oracle inequalities.
- Koltchinskii
- 2009
(Show Context)
Citation Context ...r nonparametric regression with random design are suggested and studied under prediction loss in [14] and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in =-=[15]-=-. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for the coefficients of Lasso that is quite different from the bound on the same loss for the Dantzig sele... |

48 |
High-dimensional generalized linear models and the
- Geer
- 2008
(Show Context)
Citation Context ...ersions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss in [14] and=-=[25]-=-. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for the coe... |

45 |
Sparsity in penalized empirical risk minimization.
- Koltchinskii
- 2009
(Show Context)
Citation Context ...dified versions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss in =-=[14]-=- and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu [18] establish a bound on the ℓ2 loss for... |

41 | Functional aggregation for nonparametric estimation. - Juditsky, Nemirovski - 2000 |

25 |
Aggregation and sparsity via ℓ1-penalized least squares. Learning theory
- Bunea, Tsybakov, et al.
- 2006
(Show Context)
Citation Context ...for all 1 <p≤ 2, we have | ̂βL − β ∗ | p { √ s (7.10) p ≤ 16 1 + 3 m } 2(p−1) s ( Aσ κ 2 (s, m, 3) √ ) p log M . n Inequalities of the form similar to (7.7)and(7.8) can be deduced from the results of =-=[3]-=- under more restrictive conditions on the Gram matrix (the mutual coherence assumption, cf. Assumption 5 of Section 4). Assumptions RE(s, 1) and RE(s, 3), respectively, can be dropped in Theorems 7.1 ... |

17 | Sparsity in penalized empirical risk minimization. Annales de l’Institut Henri Poincaré Probabilitiés et Statistiques - Koltchinskii |

16 | Topics in non-parametric statistics,” in Ecole d’Ete de Probabilites de Saint-Flour - Nemirovski - 1998 |

13 | Aggregation for regression learning - Bunea, Tsybakov, et al. - 2004 |

13 |
On algorithms for solving least squares problems under an L 1 penalty or an L 1 constraint
- Turlach
- 2005
(Show Context)
Citation Context |

13 | Model-selection consistency of the LASSO in highdimensional linear regression. - Zhang, Huang - 2008 |

9 | Sparse density estimation with ℓ1 penalties
- Bunea, Tsybakov, et al.
- 2007
(Show Context)
Citation Context ...tion in classical nonparametric regression settings, as well as for the problem of aggregation of estimators. An analog of Lasso for density estimation with similar properties (SPADES) is proposed in =-=[6]-=-. Modified versions of Lasso estimators (nonquadratic terms and/or penalties slightly different from ℓ1) for nonparametric regression with random design are suggested and studied under prediction loss... |

8 |
type recovery of sparse representations for high- dimensional data
- Meinshausen, Yu
(Show Context)
Citation Context ...and studied under prediction loss in [14] and[25]. Sparsity oracle inequalities for the Dantzig selector with random design are obtained in [15]. In linear fixed design regression, Meinshausen and Yu =-=[18]-=- establish a bound on the ℓ2 loss for the coefficients of Lasso that is quite different from the bound on the same loss for the Dantzig selector proven in [7]. The main message of this paper is that, ... |

7 | Discussion of the Dantzig selector: statistical estimation when p is much larger than n, by e. j. candès and t
- Bickel
- 2007
(Show Context)
Citation Context ... in Appendix A. There exist other sufficient conditions for Assumptions RE(s, c0) and RE(s, m, c0) to hold. We mention here three of them implying Assumption RE(s, c0). The first one is the following =-=[1]-=-. ASSUMPTION 3. For an integers such that 1 ≤ s ≤ M, wehave √ φmin(s) > 2c0θs,1 s, where c0 > 0 is a constant. To argue that Assumption 3 implies RE(s, c0), it suffices to remark that 1 n |Xδ|2 1 2 ≥ ... |

4 | On model selection consistency of Lasso, The Journal of Machine Learning Research 7 (2006), 2541–2563. Mohsen Bayati is an assistant professor of operations and information technology at Stanford university Graduate School of Business. Mohsen received his - Zhao, Yu |

2 |
Comments on “Regularization in statistics”, by
- Tsybakov
- 2006
(Show Context)
Citation Context ...ion losses of the Dantzig and Lasso estimators is of the same order as the distances between them and their oracle approximations. A general discussion of sparsity oracle inequalities can be found in =-=[23]-=-. Such inequalities have been recently obtained for the Lasso type estimators in a number of settings [2–6, 14]and[25]. In particular, the regression model with fixed design that we study here is cons... |

1 | Discussion of “Regularization in Statistics”, by P.Bickel and B.Li - Tsybakov - 2006 |