Results 1  10
of
26
Compressed sensing
, 2004
"... We study the notion of Compressed Sensing (CS) as put forward in [14] and related work [20, 3, 4]. The basic idea behind CS is that a signal or image, unknown but supposed to be compressible by a known transform, (eg. wavelet or Fourier), can be subjected to fewer measurements than the nominal numbe ..."
Abstract

Cited by 3625 (22 self)
 Add to MetaCart
We study the notion of Compressed Sensing (CS) as put forward in [14] and related work [20, 3, 4]. The basic idea behind CS is that a signal or image, unknown but supposed to be compressible by a known transform, (eg. wavelet or Fourier), can be subjected to fewer measurements than the nominal number of pixels, and yet be accurately reconstructed. The samples are nonadaptive and measure ‘random’ linear combinations of the transform coefficients. Approximate reconstruction is obtained by solving for the transform coefficients consistent with measured data and having the smallest possible `1 norm. We perform a series of numerical experiments which validate in general terms the basic idea proposed in [14, 3, 5], in the favorable case where the transform coefficients are sparse in the strong sense that the vast majority are zero. We then consider a range of lessfavorable cases, in which the object has all coefficients nonzero, but the coefficients obey an `p bound, for some p ∈ (0, 1]. These experiments show that the basic inequalities behind the CS method seem to involve reasonable constants. We next consider synthetic examples modelling problems in spectroscopy and image pro
For Most Large Underdetermined Systems of Linear Equations the Minimal ℓ1norm Solution is also the Sparsest Solution
 Comm. Pure Appl. Math
, 2004
"... We consider linear equations y = Φα where y is a given vector in R n, Φ is a given n by m matrix with n < m ≤ An, and we wish to solve for α ∈ R m. We suppose that the columns of Φ are normalized to unit ℓ 2 norm 1 and we place uniform measure on such Φ. We prove the existence of ρ = ρ(A) so that ..."
Abstract

Cited by 568 (10 self)
 Add to MetaCart
We consider linear equations y = Φα where y is a given vector in R n, Φ is a given n by m matrix with n < m ≤ An, and we wish to solve for α ∈ R m. We suppose that the columns of Φ are normalized to unit ℓ 2 norm 1 and we place uniform measure on such Φ. We prove the existence of ρ = ρ(A) so that for large n, and for all Φ’s except a negligible fraction, the following property holds: For every y having a representation y = Φα0 by a coefficient vector α0 ∈ R m with fewer than ρ · n nonzeros, the solution α1 of the ℓ 1 minimization problem min �x�1 subject to Φα = y is unique and equal to α0. In contrast, heuristic attempts to sparsely solve such systems – greedy algorithms and thresholding – perform poorly in this challenging setting. The techniques include the use of random proportional embeddings and almostspherical sections in Banach space theory, and deviation bounds for the eigenvalues of random Wishart matrices.
Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit
, 2006
"... Finding the sparsest solution to underdetermined systems of linear equations y = Φx is NPhard in general. We show here that for systems with ‘typical’/‘random ’ Φ, a good approximation to the sparsest solution is obtained by applying a fixed number of standard operations from linear algebra. Our pr ..."
Abstract

Cited by 274 (22 self)
 Add to MetaCart
(Show Context)
Finding the sparsest solution to underdetermined systems of linear equations y = Φx is NPhard in general. We show here that for systems with ‘typical’/‘random ’ Φ, a good approximation to the sparsest solution is obtained by applying a fixed number of standard operations from linear algebra. Our proposal, Stagewise Orthogonal Matching Pursuit (StOMP), successively transforms the signal into a negligible residual. Starting with initial residual r0 = y, at the sth stage it forms the ‘matched filter ’ Φ T rs−1, identifies all coordinates with amplitudes exceeding a speciallychosen threshold, solves a leastsquares problem using the selected coordinates, and subtracts the leastsquares fit, producing a new residual. After a fixed number of stages (e.g. 10), it stops. In contrast to Orthogonal Matching Pursuit (OMP), many coefficients can enter the model at each stage in StOMP while only one enters per stage in OMP; and StOMP takes a fixed number of stages (e.g. 10), while OMP can take many (e.g. n). StOMP runs much faster than competing proposals for sparse solutions, such as ℓ1 minimization and OMP, and so is attractive for solving largescale problems. We use phase diagrams to compare algorithm performance. The problem of recovering a ksparse vector x0 from (y, Φ) where Φ is random n × N and y = Φx0 is represented by a point (n/N, k/n)
For most large underdetermined systems of equations, the minimal l1norm nearsolution approximates the sparsest nearsolution
 Comm. Pure Appl. Math
, 2004
"... We consider inexact linear equations y ≈ Φα where y is a given vector in R n, Φ is a given n by m matrix, and we wish to find an α0,ɛ which is sparse and gives an approximate solution, obeying �y − Φα0,ɛ�2 ≤ ɛ. In general this requires combinatorial optimization and so is considered intractable. On ..."
Abstract

Cited by 122 (1 self)
 Add to MetaCart
(Show Context)
We consider inexact linear equations y ≈ Φα where y is a given vector in R n, Φ is a given n by m matrix, and we wish to find an α0,ɛ which is sparse and gives an approximate solution, obeying �y − Φα0,ɛ�2 ≤ ɛ. In general this requires combinatorial optimization and so is considered intractable. On the other hand, the ℓ 1 minimization problem min �α�1 subject to �y − Φα�2 ≤ ɛ, is convex, and is considered tractable. We show that for most Φ the solution ˆα1,ɛ = ˆα1,ɛ(y, Φ) of this problem is quite generally a good approximation for ˆα0,ɛ. We suppose that the columns of Φ are normalized to unit ℓ 2 norm 1 and we place uniform measure on such Φ. We study the underdetermined case where m ∼ An, A> 1 and prove the existence of ρ = ρ(A) and C> 0 so that for large n, and for all Φ’s except a negligible fraction, the following approximate sparse solution property of Φ holds: For every y having an approximation �y − Φα0�2 ≤ ɛ by a coefficient vector α0 ∈ R m with fewer than ρ · n nonzeros, we have �ˆα1,ɛ − α0�2 ≤ C · ɛ. This has two implications. First: for most Φ, whenever the combinatorial optimization result α0,ɛ would be very sparse, ˆα1,ɛ is a good approximation to α0,ɛ. Second: suppose we are given noisy data obeying y = Φα0 + z where the unknown α0 is known to be sparse and the noise �z�2 ≤ ɛ. For most Φ, noisetolerant ℓ 1minimization will stably recover α0 from y in the presence of noise z. We study also the barelydetermined case m = n and reach parallel conclusions by slightly different arguments. The techniques include the use of almostspherical sections in Banach space theory and concentration of measure for eigenvalues of random matrices.
Fast solution of ℓ1norm minimization problems when the solution may be sparse
, 2006
"... The minimum ℓ1norm solution to an underdetermined system of linear equations y = Ax, is often, remarkably, also the sparsest solution to that system. This sparsityseeking property is of interest in signal processing and information transmission. However, generalpurpose optimizers are much too slo ..."
Abstract

Cited by 54 (1 self)
 Add to MetaCart
(Show Context)
The minimum ℓ1norm solution to an underdetermined system of linear equations y = Ax, is often, remarkably, also the sparsest solution to that system. This sparsityseeking property is of interest in signal processing and information transmission. However, generalpurpose optimizers are much too slow for ℓ1 minimization in many largescale applications. The Homotopy method was originally proposed by Osborne et al. for solving noisy overdetermined ℓ1penalized least squares problems. We here apply it to solve the noiseless underdetermined ℓ1minimization problem min ‖x‖1 subject to y = Ax. We show that Homotopy runs much more rapidly than generalpurpose LP solvers when sufficient sparsity is present. Indeed, the method often has the following kstep solution property: if the underlying solution has only k nonzeros, the Homotopy method reaches that solution in only k iterative steps. When this property holds and k is small compared to the problem size, this means that ℓ1 minimization problems with ksparse solutions can be solved in a fraction of the cost of solving one fullsized linear system. We demonstrate this kstep solution property for two kinds of problem suites. First,
A fast reconstruction algorithm for deterministic compressive sensing using second order ReedMuller codes
 Conference on Information Sciences and Systems (CISS), Princeton, ISBN: 9781424422463, pp: 11  15
, 2008
"... Abstract—This paper proposes a deterministic compressed sensing matrix that comes by design with a very fast reconstruction algorithm, in the sense that its complexity depends only on the number of measurements n and not on the signal dimension N. The matrix construction is based on the second order ..."
Abstract

Cited by 31 (5 self)
 Add to MetaCart
(Show Context)
Abstract—This paper proposes a deterministic compressed sensing matrix that comes by design with a very fast reconstruction algorithm, in the sense that its complexity depends only on the number of measurements n and not on the signal dimension N. The matrix construction is based on the second order ReedMuller codes and associated functions. This matrix does not have RIP uniformly with respect to all ksparse vectors, but it acts as a near isometry on ksparse vectors with very high probability. I.
Compressed Sensing with Cross Validation
 IEEE Transactions on Information Theory
, 2009
"... Compressed Sensing decoding algorithms can efficiently recover an N dimensional realvalued vector x to within a factor of its best kterm approximation by taking m = 2k log N/k measurements y = Φx. If the sparsity or approximate sparsity level of x were known, then this theoretical guarantee would i ..."
Abstract

Cited by 25 (3 self)
 Add to MetaCart
(Show Context)
Compressed Sensing decoding algorithms can efficiently recover an N dimensional realvalued vector x to within a factor of its best kterm approximation by taking m = 2k log N/k measurements y = Φx. If the sparsity or approximate sparsity level of x were known, then this theoretical guarantee would imply quality assurance of the resulting compressed sensing estimate. However, because the underlying sparsity of the signal x is unknown, the quality of a compressed sensing estimate ˆx using m measurements is not assured. Nevertheless, we demonstrate that sharp bounds on the error x − ˆx  lN can be achieved with almost no ef2 fort. More precisely, we assume that a maximum number of measurements m is preimposed; we reserve 4 log p of the original m measurements and compute a sequence of possible estimates ( ) p ˆxj j=1 to x from the m − 4 log p remaining measurements; the errors x − ˆxj  lN for 2 j = 1,..., p can then be bounded with high probability. As a consequence, numerical upper and lower bounds on the error between x and the best kterm approximation to x can be estimated for p values of k with almost no cost. Our observation has applications outside of compressed sensing as well.
"Preconditioning" for feature selection and regression in highdimensional problems
 ANN. STATIST
, 2008
"... We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a “preconditioned” response variable. The primary method used for this initial regression is su ..."
Abstract

Cited by 19 (4 self)
 Add to MetaCart
(Show Context)
We consider regression problems where the number of predictors greatly exceeds the number of observations. We propose a method for variable selection that first estimates the regression function, yielding a “preconditioned” response variable. The primary method used for this initial regression is supervised principal components. Then we apply a standard procedure such as forward stepwise selection or the LASSO to the preconditioned response variable. In a number of simulated and real data examples, this twostep procedure outperforms forward stepwise selection or the usual LASSO (applied directly to the raw outcome). We also show that under a certain Gaussian latent variable model, application of the LASSO to the preconditioned response variable is consistent as the number of predictors and observations increases. Moreover, when the observational noise is rather large, the suggested procedure can give a more accurate estimate than LASSO. We illustrate our method on some real problems, including survival analysis with microarray data.
Sparse solution of underdetermined linear systems: algorithms and applications
, 2007
"... in my opinion, it ..."
(Show Context)
L0normbased sparse representation through alternate projections
 in ICIP, 2006
"... We present a simple and robust method for finding sparse representations in overcomplete transforms, based on minimization of the L0norm. Our method is better than current solutions based on minimization of the L1norm in terms of energy compaction. These results strongly question the equivalence o ..."
Abstract

Cited by 7 (0 self)
 Add to MetaCart
(Show Context)
We present a simple and robust method for finding sparse representations in overcomplete transforms, based on minimization of the L0norm. Our method is better than current solutions based on minimization of the L1norm in terms of energy compaction. These results strongly question the equivalence of minimizing both norms in real conditions. We also show application to inpainting (interpolation of lost pixels). Index Terms — Image representation, restoration. 1.