Results 1 
8 of
8
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
, 2007
"... A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combin ..."
Abstract

Cited by 423 (37 self)
 Add to MetaCart
A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena; in particular, the existence of easilyverifiable conditions under which optimallysparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several wellknown signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable. There is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems energizes research on such signal and image processing problems – to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical
PACBayesian bounds for sparse regression estimation with exponential weights
 Electronic Journal of Statistics
"... Abstract. We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering highdimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
(Show Context)
Abstract. We consider the sparse regression model where the number of parameters p is larger than the sample size n. The difficulty when considering highdimensional problems is to propose estimators achieving a good compromise between statistical and computational performances. The BIC estimator for instance performs well from the statistical point of view [11] but can only be computed for values of p of at most a few tens. The Lasso estimator is solution of a convex minimization problem, hence computable for large value of p. However stringent conditions on the design are required to establish fast rates of convergence for this estimator. Dalalyan and Tsybakov [19] propose a method achieving a good compromise between the statistical and computational aspects of the problem. Their estimator can be computed for reasonably large p and satisfies nice statistical properties under weak assumptions on the design. However, [19] proposes sparsity oracle inequalities in expectation for the empirical excess risk only. In this paper, we propose an aggregation procedure similar to that of [19] but with improved statistical performances. Our main theoretical result is a sparsity oracle inequality in probability for the true excess risk for a version of exponential weight estimator. We also propose a MCMC method to compute our estimator for reasonably large values of p.
ELASTICNET REGULARIZATION IN LEARNING THEORY
, 2008
"... Abstract. Within the framework of statistical learning theory we analyze in detail the socalled elasticnet regularization scheme proposed by Zou and Hastie [45] for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its ..."
Abstract

Cited by 17 (6 self)
 Add to MetaCart
(Show Context)
Abstract. Within the framework of statistical learning theory we analyze in detail the socalled elasticnet regularization scheme proposed by Zou and Hastie [45] for the selection of groups of correlated variables. To investigate on the statistical properties of this scheme and in particular on its consistency properties, we set up a suitable mathematical framework. Our setting is randomdesign regression where we allow the response variable to be vectorvalued and we consider prediction functions which are linear combination of elements (features) in an infinitedimensional dictionary. Under the assumption that the regression function admits a sparse representation on the dictionary, we prove that there exists a particular “elasticnet representation ” of the regression function such that, if the number of data increases, the elasticnet estimator is consistent not only for prediction but also for variable/feature selection. Our results include finitesample bounds and an adaptive scheme to select the regularization parameter. Moreover, using convex analysis tools, we derive an iterative thresholding algorithm for computing the elasticnet solution which is different from the optimization procedure originally proposed in [45]. 1.
Iterative feature selection in least square regression estimation
 Ann. Inst. Henri Poincaré Probab. Stat
"... ha l0 ..."
A Taste of Compressed Sensing ∗
, 2007
"... The usual paradigm for signal processing is to model a signal as a bandlimited function and capture the signal by means of its time samples. The ShannonNyquist theory says that the sampling rate needs to be at least twice the bandwidth. For broadbanded signals, such high sampling rates may be impos ..."
Abstract
 Add to MetaCart
(Show Context)
The usual paradigm for signal processing is to model a signal as a bandlimited function and capture the signal by means of its time samples. The ShannonNyquist theory says that the sampling rate needs to be at least twice the bandwidth. For broadbanded signals, such high sampling rates may be impossible to implement in circuitry. Compressed Sensing is a new area of signal processing whose aim is to circumvent this dilemma by sampling signals closer to their information rate instead of their bandwidth. Rather than model the signal as bandlimited, Compressed Sensing, assumes the signal can be represented or approximated by a few suitably chosen terms from a basis expansion of the signal. It also enlarges the concept of sample to include the application of any linear functional applied to the signal. In this paper, we shall give a brief introduction to compressed sensing that centers on the effectiveness and implementation of random sampling.
1Information Theory and Mixing LeastSquares Regressions
"... Abstract — For Gaussian regression, we develop and analyse methods for combining estimators from various models. For squarederror loss, an unbiased estimator of the risk of the mixture of general estimators is developed. Special attention is given to the case that the component estimators are least ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract — For Gaussian regression, we develop and analyse methods for combining estimators from various models. For squarederror loss, an unbiased estimator of the risk of the mixture of general estimators is developed. Special attention is given to the case that the component estimators are leastsquares projections into arbitrary linear subspaces, such as those spanned by subsets of explanatory variables in a given design. We relate the unbiased estimate of the risk of the mixture estimator to estimates of the risks achieved by the components. This results in simple and accurate bounds on the risk and its estimate, in the form of sharp and exact oracle inequalities. That is, without advance knowledge of which model is best, the resulting performance is comparable to or perhaps even superior to what is achieved by the best of the individual models. Furthermore, in the case that the unknown parameter has a sparse representation, our mixture estimator adapts to the underlying sparsity. Simulations show that the performance of these mixture estimators is better than that of a related modelselection estimator which picks a model with the highest weight. Also, the connection between our mixtures with Bayes procedures is discussed. Index Terms — combining leastsquares regressions, model adaptation, model selection target, oracle inequalities, unbiased risk estimate, Bayes mixtures, complexity, resolvability, sparsity I.
LASSO, Iterative Feature Selection and the Correlation Selector: Oracle inequalities and numerical performances
, 2008
"... ..."
(Show Context)
c ○ Association des Publications de l’Institut Henri Poincaré, 2008
, 2005
"... www.imstat.org/aihp ..."