• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations

Fused sparsity and robust estimation for linear models with unknown variance (2012)

by Yin Chen, Arnak S. Dalalyan
Venue:In NIPS
Add To MetaCart

Tools

Sorted by:
Results 1 - 8 of 8

Robust Subspace Clustering

by Mahdi Soltanolkotabi, Ehsan Elhamifar, Emmanuel J. Candes , 2013
"... Subspace clustering refers to the task of finding a multi-subspace representation that best fits a collection of points taken from a high-dimensional space. This paper introduces an algorithm inspired by sparse subspace clustering (SSC) [17] to cluster noisy data, and develops some novel theory demo ..."
Abstract - Cited by 22 (1 self) - Add to MetaCart
Subspace clustering refers to the task of finding a multi-subspace representation that best fits a collection of points taken from a high-dimensional space. This paper introduces an algorithm inspired by sparse subspace clustering (SSC) [17] to cluster noisy data, and develops some novel theory demonstrating its correctness. In particular, the theory uses ideas from geometric functional analysis to show that the algorithm can accurately recover the underlying subspaces under minimal requirements on their orientation, and on the number of samples per subspace. Synthetic as well as real data experiments complement our theoretical study, illustrating our approach and demonstrating its effectiveness.
(Show Context)

Citation Context

...ation scheme to simultaneous optimize the regularization parameter and the regression coefficients. In recent years there has been much progress on this issue in the sparse regression literature, see =-=[2,13,23,28,51]-=- and references therein. It is an open research direction to see whether any of these approaches can be applied to automatically learn the regularization parameter when both the response vector and co...

Intersecting singularities for multi-structured estimation

by Emile Richard, Francis Bach
"... We address the problem of designing a convex nonsmooth regularizer encouraging multiple structural effects simultaneously. Focusing on the inference of sparse and low-rank matrices we suggest a new complexity index and a convex penalty approximating it. The new penalty term can be written as the tra ..."
Abstract - Cited by 6 (1 self) - Add to MetaCart
We address the problem of designing a convex nonsmooth regularizer encouraging multiple structural effects simultaneously. Focusing on the inference of sparse and low-rank matrices we suggest a new complexity index and a convex penalty approximating it. The new penalty term can be written as the trace norm of a linear function of the matrix. By analyzing theoretical properties of this family of regularizers we come up with oracle inequalities and compressed sensing results ensuring the quality of our regularized estimator. We also provide algorithms and supporting numerical experiments. 1.

Learning heteroscedastic models by convex programming under group sparsity

by Arnak S. Dalalyan, Mohamed Hebiri, Katia Meziani, Université Paris Dauphine, Joseph Salmon - Proc. of the International conference on Machine Learning , 2013
"... Popular sparse estimation methods based on ℓ1-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks—such as t ..."
Abstract - Cited by 2 (0 self) - Add to MetaCart
Popular sparse estimation methods based on ℓ1-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutes a major obstacle in applying these methods in several frameworks—such as time series, random fields, inverse problems—for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a highdimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for very large scale problems by solving a secondorder cone program (SOCP). We present theoretical analysis and numerical results assessing the performance of the proposed procedure. 1.
(Show Context)

Citation Context

...-relaxations: the square-root Lasso (Antoniadis, 2010; Belloni et al., 2011; Sun and Zhang, 2012; Gautier and Tsybakov, 2011), the scaled Lasso (Städler et al., 2010) and the scaled Dantzig selector (=-=Dalalyan and Chen, 2012-=-). These methods are tailored to the context of a fixed noise level across observations (homoscedasticity), which reduces their attractiveness for applications in the aforementioned fields. In the pre...

Clustering Consistent Sparse Subspace Clustering

by Yining Wang, Yu-xiang Wang, Aarti Singh , 2015
"... Subspace clustering is the problem of clustering data points into a union of low-dimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image pro-cessing and has been drawing avid attention in machine learning and statistics recently. I ..."
Abstract - Cited by 1 (1 self) - Add to MetaCart
Subspace clustering is the problem of clustering data points into a union of low-dimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image pro-cessing and has been drawing avid attention in machine learning and statistics recently. In particular, a line
(Show Context)

Citation Context

...) and is more robust in practice. Remark 4 There has been extensive study of using restricted eigenvalue assumptions in the analysis of Lasso-type problems (Bickel et al., 2009; Lounici et al., 2011; =-=Chen & Dalayan, 2012-=-; Raskutti et al., 2010). However, in our problem the assumption is used in a very different manner. In particular, we used the restricted eigenvalue assumption to prove one key lemma (Lemma 2) that l...

Graph Connectivity in Noisy Sparse Subspace Clustering

by Yining Wang , Yu-Xiang Wang , Aarti Singh
"... Abstract Subspace clustering is the problem of clustering data points into a union of lowdimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image processing and machine learning. A line of recent work ..."
Abstract - Add to MetaCart
Abstract Subspace clustering is the problem of clustering data points into a union of lowdimensional linear/affine subspaces. It is the mathematical abstraction of many important problems in computer vision, image processing and machine learning. A line of recent work
(Show Context)

Citation Context

...r the adversarial noise model. Remark 3 Some components of Algorithm 2 can be revised to make the method more robust in practical applications. For example, instead of randomly picking d points and computing their range, one could apply robust PCA on all points in the connected component, which is more robust to potential outliers. In addition, the single linkage clustering step could be replaced by k-means clustering, which is more robust to false connections in practice. Remark 4 There has been extensive study of using restricted eigenvalue assumptions in the analysis of Lasso-type problems [1, 13, 2, 18]. However, in our problem the assumption is used in a very different manner. In particular, we used the restricted eigenvalue assumption to prove one key lemma (Lemma C.2) that lower bounds the support size of the optimal solution to a Lasso problem. Such results might be of independent interest as a nice contribution to the analysis of Lasso in general. 3.3 Discussion on Assumption 3.1 Assumption 3.1 requires a spectral gap for every subset of data points in each subspace. This seems a very strong assumption that restricts the maximum tolerable noise magnitude to be very small. In this sectio...

A Service of zbw A lava attack on the recovery of sums of dense and sparse signals A LAVA ATTACK ON THE RECOVERY OF SUMS OF DENSE AND SPARSE SIGNALS

by Victor Chernozhukov , Christian Hansen , Yuan Liao , Victor Chernozhukov , Christian Hansen , Yuan Liao
"... We consider a generalization of these two basic models, termed here a "sparse+dense" model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estima ..."
Abstract - Add to MetaCart
We consider a generalization of these two basic models, termed here a "sparse+dense" model, in which the signal is given by the sum of a sparse signal and a dense signal. Such a structure poses problems for traditional sparse estimators, such as the lasso, and for traditional dense estimation methods, such as ridge estimation. We propose a new penalization-based method, called lava, which is computationally efficient. With suitable choices of penalty parameters, the proposed method strictly dominates both lasso and ridge. We derive analytic expressions for the finite-sample risk function of the lava estimator in the Gaussian sequence model. We also provide a deviation bound for the prediction risk in the Gaussian regression model with fixed design. In both cases, we provide Stein's unbiased estimator for lava's prediction risk. A simulation example compares the performance of lava to lasso, ridge, and elastic net in a regression example using feasible, data-dependent penalty parameters and illustrates lava's improved performance relative to these benchmarks.
(Show Context)

Citation Context

... is designed specifically to achieve good prediction and estimation performance in “sparse+dense” models. In such models, the high-dimensional parameter is represented as the sum of a sparse vector with a few large non-zero entries and a dense vector with many small entries. This structure renders traditional sparse or dense estimation methods, such as lasso or ridge, suboptimal for prediction and other estimation purposes. The proposed approach thus complements other approaches to structured sparsity problems such as those considered in fused sparsity estimation (Tibshirani et al. (2005) and Chen and Dalalyan (2012)) and structured matrix decomposition problems (Candes et al. (2011), Chandrasekaran et al. (2011), Fan et al. (2013), and Klopp et al. (2014)). There are a number of interesting research directions that remain to be considered. An immediate extension of the present results would be to consider semi-pivotal estimators akin to the root-lasso/scaled-lasso of Belloni et al. (2011) and Sun and Zhang (2012). For instance, we can define θroot-lava := β + δ, (β, δ) := arg min β,δ,σ { 1 2nσ2 ‖Y −X(β + δ)‖22 + (1− a)σ 2 + λ2‖β‖22 + λ1‖δ‖1 } . Thanks to the characterization of Theorem 3.1, the met...

page_id=43" Learning Heteroscedastic Models by Convex Programming under Group Sparsity

by Arnak S. Dalalyan, Mohamed Hebiri, Université Paris Est, Katia Meziani, Université Paris Dauphine, Joseph Salmon, Ltci Telecom Paristech , 2013
"... Popular sparse estimation methods based on ℓ1-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutesamajorobstacleinapplyingthese methods in severalframeworks—suchas time seri ..."
Abstract - Add to MetaCart
Popular sparse estimation methods based on ℓ1-relaxation, such as the Lasso and the Dantzig selector, require the knowledge of the variance of the noise in order to properly tune the regularization parameter. This constitutesamajorobstacleinapplyingthese methods in severalframeworks—suchas time series, random fields, inverse problems—for which the noise is rarely homoscedastic and its level is hard to know in advance. In this paper, we propose a new approach to the joint estimation of the conditional mean and the conditional variance in a highdimensional (auto-) regression setting. An attractive feature of the proposed estimator is that it is efficiently computable even for verylargescaleproblemsbysolvingasecondordercone program(SOCP). We present theoreticalanalysisand numericalresults assessing the performance of the proposed procedure. 1.
(Show Context)

Citation Context

...1-relaxations: the squareroot Lasso (Antoniadis, 2010; Belloni et al., 2011; Sun and Zhang, 2012; Gautier and Tsybakov, 2011), the scaled Lasso (Städler et al., 2010) and the scaled Dantzig selector (=-=Dalalyan and Chen, 2012-=-). These methods are tailored to the context of a fixed noise level across observations (homoscedasticity), which reduces their attractiveness for applications in the aforementioned fields. In the pre...

Sharp Threshold for Multivariate Multi-Response Lin- ear Regression via Block Regularized Lasso

by Weiguang Wang, Yingbin Liang, Eric P. Xing
"... ar ..."
Abstract - Add to MetaCart
Abstract not found
Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University