Citation Context

...e l0-minimization. Three most renown recovery algorithms based on convex relaxation proposed in the literature are: the Basis Pursuit (BP) [7], the Dantzig selector (DS) [15], and the LASSO estimator =-=[53]-=- (or Basis Pursuit Denoising [7]): (BP) : min f̃∈Rn ‖f̃‖1 subject to ‖Af̃ − y‖2 ≤ ε, (DS) : min f̃∈Rn ‖f̃‖1 subject to ‖A∗(Af̃ − y)‖∞ ≤ λnσ, (LASSO) : min f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µnσ‖f̃‖1, here ‖ · ...

Citation Context

... 1 Introduction 1.1 Standard compressed sensing Compressed sensing predicts that sparse signals can be reconstructed from what was previously believed to be incomplete information. The seminal papers =-=[11, 12, 19]-=- have triggered a large research activity in mathematics, engineering and computer science with a lot of potential applications. ∗This work is supported by NSF of China under grant numbers 11171299 an...

Citation Context

...minimization which can be viewed as a convex relaxation of the l0-minimization. Three most renown recovery algorithms based on convex relaxation proposed in the literature are: the Basis Pursuit (BP) =-=[7]-=-, the Dantzig selector (DS) [15], and the LASSO estimator [53] (or Basis Pursuit Denoising [7]): (BP) : min f̃∈Rn ‖f̃‖1 subject to ‖Af̃ − y‖2 ≤ ε, (DS) : min f̃∈Rn ‖f̃‖1 subject to ‖A∗(Af̃ − y)‖∞ ≤ λn...

Citation Context

... 1 Introduction 1.1 Standard compressed sensing Compressed sensing predicts that sparse signals can be reconstructed from what was previously believed to be incomplete information. The seminal papers =-=[11, 12, 19]-=- have triggered a large research activity in mathematics, engineering and computer science with a lot of potential applications. ∗This work is supported by NSF of China under grant numbers 11171299 an...

Citation Context

...vector in the feasible set of possible solutions, which leads to an l0-minimization problem. However, solving the l0-minimization directly is NP-hard in general and thus is computationally infeasible =-=[43, 44]-=-. It is then natural to consider the method of l1-minimization which can be viewed as a convex relaxation of the l0-minimization. Three most renown recovery algorithms based on convex relaxation propo...

Citation Context

...SSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that for an m× n matrix A and s ≤ n, the RIP constant δs =-=[11, 14, 20]-=- is defined as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. So far, all good constructions of matrices with the RIP use randomness. It is w...

Citation Context

...rs with small or zero errors provided that the measurement matrix A satisfies a restricted isometry property (RIP) condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖z‖2 is small =-=[13, 12, 6, 16, 29, 41]-=-. Similar results were obtained for the DS and the LASSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that...

Citation Context

... 1 Introduction 1.1 Standard compressed sensing Compressed sensing predicts that sparse signals can be reconstructed from what was previously believed to be incomplete information. The seminal papers =-=[11, 12, 19]-=- have triggered a large research activity in mathematics, engineering and computer science with a lot of potential applications. ∗This work is supported by NSF of China under grant numbers 11171299 an...

Citation Context

...d as a convex relaxation of the l0-minimization. Three most renown recovery algorithms based on convex relaxation proposed in the literature are: the Basis Pursuit (BP) [7], the Dantzig selector (DS) =-=[15]-=-, and the LASSO estimator [53] (or Basis Pursuit Denoising [7]): (BP) : min f̃∈Rn ‖f̃‖1 subject to ‖Af̃ − y‖2 ≤ ε, (DS) : min f̃∈Rn ‖f̃‖1 subject to ‖A∗(Af̃ − y)‖∞ ≤ λnσ, (LASSO) : min f̃∈Rn 1 2 ‖(Af̃...

Citation Context

...compressed sensing based on pursuit algorithms in the literature, including Orthogonal Matching Pursuit (OMP) [48, 23], Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit =-=[46]-=-, Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of these pursuit methods. 1.2 l1-synthesis For signals which are sparse in the standard ...

Citation Context

...rs with small or zero errors provided that the measurement matrix A satisfies a restricted isometry property (RIP) condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖z‖2 is small =-=[13, 12, 6, 16, 29, 41]-=-. Similar results were obtained for the DS and the LASSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that...

Citation Context

... therein for more details on sparse noise. There are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the literature, including Orthogonal Matching Pursuit (OMP) =-=[48, 23]-=-, Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of...

Citation Context

...d as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. So far, all good constructions of matrices with the RIP use randomness. It is well known =-=[14, 3, 42, 50]-=- that many types of random measurement matrices such as Gaussian matrices or SubGaussian matrices have the RIP constant δs ≤ δ with overwhelming probability provided that m ≥ Cδ−2s log(n/s). Up to the...

Citation Context

...SSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that for an m× n matrix A and s ≤ n, the RIP constant δs =-=[11, 14, 20]-=- is defined as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. So far, all good constructions of matrices with the RIP use randomness. It is w...

Citation Context

...vector in the feasible set of possible solutions, which leads to an l0-minimization problem. However, solving the l0-minimization directly is NP-hard in general and thus is computationally infeasible =-=[43, 44]-=-. It is then natural to consider the method of l1-minimization which can be viewed as a convex relaxation of the l0-minimization. Three most renown recovery algorithms based on convex relaxation propo...

Citation Context

... [13, 12, 6, 16, 29, 41]. Similar results were obtained for the DS and the LASSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small =-=[15, 5, 16]-=-. Recall that for an m× n matrix A and s ≤ n, the RIP constant δs [11, 14, 20] is defined as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. S...

Citation Context

...is but in terms of an overcomplete dictionary, which means that our signal f ∈ Rn is now expressed as f = Dx where D ∈ Rn×d (d ≥ n) is a redundant dictionary and x is (approximately) sparse, see e.g. =-=[7, 4, 8]-=- and the reference therein. Examples include signal modeling in array signal processing (oversampled array steering matrix), reflected radar and sonar signals (Gabor frames), and images with curves (C...

Citation Context

...t algorithms in the literature, including Orthogonal Matching Pursuit (OMP) [48, 23], Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding =-=[2]-=-, Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of these pursuit methods. 1.2 l1-synthesis For signals which are sparse in the standard coordinate basis or sparse in ter...

Citation Context

...terature, including Orthogonal Matching Pursuit (OMP) [48, 23], Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit =-=[22]-=- and many other variants. Refer to [55] for an overview of these pursuit methods. 1.2 l1-synthesis For signals which are sparse in the standard coordinate basis or sparse in terms of some other orthon...

Citation Context

...s on sparse noise. There are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the literature, including Orthogonal Matching Pursuit (OMP) [48, 23], Stagewise OMP =-=[24]-=-, Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of these pursuit metho...

Citation Context

...d as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. So far, all good constructions of matrices with the RIP use randomness. It is well known =-=[14, 3, 42, 50]-=- that many types of random measurement matrices such as Gaussian matrices or SubGaussian matrices have the RIP constant δs ≤ δ with overwhelming probability provided that m ≥ Cδ−2s log(n/s). Up to the...

Citation Context

...ume that the noise vector z ∼ N(0, σ2I), i.e., z is i.i.d. Gaussian noise, which is of particular interest in signal processing and in statistics. The case 2 of Gaussian noise was first considered in =-=[33]-=-, which examined the performance of l0-minimization with noisy measurements. Since the Gaussian noise is essentially bounded (e.g. [15, 17]), all stably recovery results mentioned above for bounded er...

Citation Context

...rs with small or zero errors provided that the measurement matrix A satisfies a restricted isometry property (RIP) condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖z‖2 is small =-=[13, 12, 6, 16, 29, 41]-=-. Similar results were obtained for the DS and the LASSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that...

Citation Context

... therein for more details on sparse noise. There are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the literature, including Orthogonal Matching Pursuit (OMP) =-=[48, 23]-=-, Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of...

Citation Context

...re are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the literature, including Orthogonal Matching Pursuit (OMP) [48, 23], Stagewise OMP [24], Regularized OMP =-=[47]-=-, Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to [55] for an overview of these pursuit methods. 1.2 l1-synthesis F...

Citation Context

...ondition imposed by the standard compressed sensing assumptions. However, if D is a coherent frame, AD does not generally satisfy the standard RIP [49, 8]. Also, the mutual incoherence property (MIP) =-=[21]-=- may not apply, as it is very hard for AD to satisfy the MIP as well when D is highly correlated. 3 1.3 l1-analysis An alternative to l1-synthesis is l1-analysis, which finds the estimator f̂ directly...

Citation Context

... Pursuit (OMP) [48, 23], Stagewise OMP [24], Regularized OMP [47], Compressive Sampling Matching Pursuit [46], Iterative Hard Thresholding [2], Subspace Pursuit [22] and many other variants. Refer to =-=[55]-=- for an overview of these pursuit methods. 1.2 l1-synthesis For signals which are sparse in the standard coordinate basis or sparse in terms of some other orthonormal basis, the techniques above hold....

Citation Context

...is but in terms of an overcomplete dictionary, which means that our signal f ∈ Rn is now expressed as f = Dx where D ∈ Rn×d (d ≥ n) is a redundant dictionary and x is (approximately) sparse, see e.g. =-=[7, 4, 8]-=- and the reference therein. Examples include signal modeling in array signal processing (oversampled array steering matrix), reflected radar and sonar signals (Gabor frames), and images with curves (C...

Citation Context

...signal modeling in array signal processing (oversampled array steering matrix), reflected radar and sonar signals (Gabor frames), and images with curves (Curvelet frames), etc. The l1-synthesis (e.g. =-=[7, 49, 25]-=-) consists in finding the sparsest possible coefficient x̂ by solving an l1-minimization problem (BP or LASSO) with the decoding matrix AD instead of A, and then reconstruct the signal by a synthesis ...

Citation Context

...d as the smallest number δ such that for all s sparse vectors x̃ ∈ Rn, (1− δ)‖x̃‖22 ≤ ‖Ax̃‖22 ≤ (1 + δ)‖x̃‖22. So far, all good constructions of matrices with the RIP use randomness. It is well known =-=[14, 3, 42, 50]-=- that many types of random measurement matrices such as Gaussian matrices or SubGaussian matrices have the RIP constant δs ≤ δ with overwhelming probability provided that m ≥ Cδ−2s log(n/s). Up to the...

Citation Context

...ssian matrices or SubGaussian matrices have the RIP constant δs ≤ δ with overwhelming probability provided that m ≥ Cδ−2s log(n/s). Up to the constant, the lower bounds for Gelfand widths of l1-balls =-=[31, 30]-=- show that this dependence on n and s is optimal. The fast multiply partial random Fourier matrix has the RIP constant δs ≤ δ with very high probability provided thatm ≥ Cδ−2s(log n)4 [14, 50, 34]. In...

Citation Context

... directly by solving an l1minimization problem. There are two most renown analysis recovery algorithms proposed in the literature: the analysis Basis Pursuit (ABP) [8] and the analysis LASSO (ALASSO) =-=[25, 54]-=-1: (ABP) : f̂ = argmin f̃∈Rn ‖D∗f̃‖1 subject to ‖Af̃ − y‖2 ≤ ε, (1.2) (ALASSO) : f̂AL = argmin f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µ‖D∗f̃‖1. (1.3) Here µ is a tuning parameter, and ε is a measure of the noise l...

Citation Context

...ies (1.6). Recall that an isotropic ψ2 vector a is one that satisfies for all v, E|〈a, v〉| = ‖v‖22 and inf{t : E exp(〈a, v〉2/t2) ≤ 2} ≤ α‖v‖2, for some constant α [42]. Very recently, Ward and Kramer =-=[35]-=- showed that randomizing the column signs of any matrix that satisfies the standard RIP results in a matrix which satisfies the Johnson-Lindenstrauss lemma. Therefore, nearly all random matrix constru...

Citation Context

...A∗z‖∞ ≤ σ √ 2(1 + α)(1 + δ1) log d ) = 1− P ( ‖D∗A∗z‖∞ > σ √ 2(1 + α)(1 + δ1) log d ) ≥ 1− 1 dα √ (1 + α)π log d . 5.2 Proof of Theorem 2.1 Proof of Theorem 2.1. The proof makes use of the ideas from =-=[8, 15, 6, 10]-=-. Let f and f̂ADS be as in the theorem, and let T0 = T denote the set of the s largest coefficients of D ∗f in magnitude. Set h = f̂ADS − f and observe that by the triangle inequality ‖D∗A∗Ah‖∞ ≤ ‖D∗A...

Citation Context

...rgmin f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µ‖D∗f̃‖1. (1.3) Here µ is a tuning parameter, and ε is a measure of the noise level. Several works exist in the literature that are related to the analysis model (e.g. =-=[25, 51, 8, 1, 39, 45]-=-). It has been shown that l1-analysis and l1-synthesis approaches are exactly equivalent when D is orthogonal otherwise there is a remarkable difference between the two despite their apparent similari...

Citation Context

...1-balls [31, 30] show that this dependence on n and s is optimal. The fast multiply partial random Fourier matrix has the RIP constant δs ≤ δ with very high probability provided thatm ≥ Cδ−2s(log n)4 =-=[14, 50, 34]-=-. In many common settings it is natural to assume that the noise vector z ∼ N(0, σ2I), i.e., z is i.i.d. Gaussian noise, which is of particular interest in signal processing and in statistics. The cas...

Citation Context

...for the standard compressed sensing, we derive similar result as in [15, Theorem 1.3]. (d) We have not tried to optimize the D-RIP condition. We expect that with a more complicated proof as in [6] or =-=[29, 16, 26, 41]-=-, one can still improve this condition. The error bound (2.1) is within a log-like factor of the minimax risk over the class of vectors which are at most s sparse in terms of D: Theorem 2.5. Let D be ...

Citation Context

...of the effectiveness of the analysis approach can be found in [25] for signal denoising and in [51] for signal and image restoration. Numerical algorithms have been proposed to solve the ALASSO, e.g. =-=[32, 9, 40]-=-. More recently, Candès et al. [8] showed that the ABP recovers a signal f̂ with an error bound ‖f̂ − f‖2 ≤ C0 ‖D∗f − (D∗f)[s]‖1√ s + C1ε, (1.4) provided that A satisfies a restricted isometry proper...

Citation Context

...rgmin f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µ‖D∗f̃‖1. (1.3) Here µ is a tuning parameter, and ε is a measure of the noise level. Several works exist in the literature that are related to the analysis model (e.g. =-=[25, 51, 8, 1, 39, 45]-=-). It has been shown that l1-analysis and l1-synthesis approaches are exactly equivalent when D is orthogonal otherwise there is a remarkable difference between the two despite their apparent similari...

Citation Context

...rs with small or zero errors provided that the measurement matrix A satisfies a restricted isometry property (RIP) condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖z‖2 is small =-=[13, 12, 6, 16, 29, 41]-=-. Similar results were obtained for the DS and the LASSO provided that A satisfies a RIP condition δcs ≤ δ for some constants c, δ > 0 and that the error bound ‖A∗z‖∞ is small [15, 5, 16]. Recall that...

Citation Context

...o sparse [37]. This can occur in practice due to shot noise, malfunctioning hardware, transmission errors, or narrowband interference. Several recovery techniques have been developed for sparse noise =-=[37, 52, 36]-=-. We refer the readers to [37, 52, 36] and the reference therein for more details on sparse noise. There are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the ...

Citation Context

...ssian matrices or SubGaussian matrices have the RIP constant δs ≤ δ with overwhelming probability provided that m ≥ Cδ−2s log(n/s). Up to the constant, the lower bounds for Gelfand widths of l1-balls =-=[31, 30]-=- show that this dependence on n and s is optimal. The fast multiply partial random Fourier matrix has the RIP constant δs ≤ δ with very high probability provided thatm ≥ Cδ−2s(log n)4 [14, 50, 34]. In...

Citation Context

...signal modeling in array signal processing (oversampled array steering matrix), reflected radar and sonar signals (Gabor frames), and images with curves (Curvelet frames), etc. The l1-synthesis (e.g. =-=[7, 49, 25]-=-) consists in finding the sparsest possible coefficient x̂ by solving an l1-minimization problem (BP or LASSO) with the decoding matrix AD instead of A, and then reconstruct the signal by a synthesis ...

Citation Context

...of the effectiveness of the analysis approach can be found in [25] for signal denoising and in [51] for signal and image restoration. Numerical algorithms have been proposed to solve the ALASSO, e.g. =-=[32, 9, 40]-=-. More recently, Candès et al. [8] showed that the ABP recovers a signal f̂ with an error bound ‖f̂ − f‖2 ≤ C0 ‖D∗f − (D∗f)[s]‖1√ s + C1ε, (1.4) provided that A satisfies a restricted isometry proper...

Citation Context

...statistics. The case 2 of Gaussian noise was first considered in [33], which examined the performance of l0-minimization with noisy measurements. Since the Gaussian noise is essentially bounded (e.g. =-=[15, 17]-=-), all stably recovery results mentioned above for bounded error related to the BP, the DS and the LASSO can be extended directly to the Gaussian noise case. While the BP and the DS (or the Lasso) pro...

Citation Context

...for the standard compressed sensing, we derive similar result as in [15, Theorem 1.3]. (d) We have not tried to optimize the D-RIP condition. We expect that with a more complicated proof as in [6] or =-=[29, 16, 26, 41]-=-, one can still improve this condition. The error bound (2.1) is within a log-like factor of the minimax risk over the class of vectors which are at most s sparse in terms of D: Theorem 2.5. Let D be ...

Citation Context

...rgmin f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µ‖D∗f̃‖1. (1.3) Here µ is a tuning parameter, and ε is a measure of the noise level. Several works exist in the literature that are related to the analysis model (e.g. =-=[25, 51, 8, 1, 39, 45]-=-). It has been shown that l1-analysis and l1-synthesis approaches are exactly equivalent when D is orthogonal otherwise there is a remarkable difference between the two despite their apparent similari...

Citation Context

...rgmin f̃∈Rn 1 2 ‖(Af̃ − y)‖22 + µ‖D∗f̃‖1. (1.3) Here µ is a tuning parameter, and ε is a measure of the noise level. Several works exist in the literature that are related to the analysis model (e.g. =-=[25, 51, 8, 1, 39, 45]-=-). It has been shown that l1-analysis and l1-synthesis approaches are exactly equivalent when D is orthogonal otherwise there is a remarkable difference between the two despite their apparent similari...

Citation Context

...tioned recovery algorithms provide guarantees only for noise that is bounded or bounded with high probability. However, these algorithms perform suboptimally when the measurement noise is also sparse =-=[37]-=-. This can occur in practice due to shot noise, malfunctioning hardware, transmission errors, or narrowband interference. Several recovery techniques have been developed for sparse noise [37, 52, 36]....

Citation Context

...t this work to the setting of real valued signals f ∈ Rn. For perspective, it is known that compressed sensing results ([6]) such as for the BP are also valid for complex valued signals f ∈ Cd, e.g., =-=[27]-=-. Note also that we have restricted to the tight frame case and that a signal being sparse in a non-tight frame is also interesting. 6 1.5 Notation The following notation is used throughout this paper...

Citation Context

...ing with general frames. Aldroubi et al. [1] showed that the ABP is robust to measurement noise, and stable with respect to perturbations of the measurement matrix A and the general frames D. Foucart =-=[28]-=- studied the ABP algorithm under the setting that the measurement matrices are Weibull random matrices. Recall that the D-RIP of a measurement matrix A, which first appeared in [8] and is a natural ex...

Citation Context

...o sparse [37]. This can occur in practice due to shot noise, malfunctioning hardware, transmission errors, or narrowband interference. Several recovery techniques have been developed for sparse noise =-=[37, 52, 36]-=-. We refer the readers to [37, 52, 36] and the reference therein for more details on sparse noise. There are many other algorithmic approaches to compressed sensing based on pursuit algorithms in the ...

Citation Context

... (1.4) provided that A satisfies a restricted isometry property adapted to D (D-RIP) condition with δ2s < 0.08, where D is a tight frame for R n. Later, the D-RIP condition is improved to δ2s < 0.493 =-=[38]-=-. Note that we denote x[s] to be the vector consisting of the s largest coefficients of x ∈ Rd in magnitude, i.e. x[s] is the best s sparse approximation to the vector x. Following [8], Liu et al. [39...