#### DMCA

## From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images (2007)

### Cached

### Download Links

Citations: | 426 - 36 self |

### Citations

7737 |
Matrix analysis
- Horn, Johnson
- 1985
(Show Context)
Citation Context ...| ≤µ : 1 ≤ k, j ≤ m, k ̸= j} . Consider an arbitrary minor from G of size p × p, built by choosing a subgroup of p columns from Ã and computing their sub-Gram matrix. From the Gershgorin disk theorem =-=[91]-=-, if this minor is diagonally dominant—i.e., if ∑ j̸=i |Gi,j| < |Gi,i| for every i—then this submatrix of G is positive definite, and so those p columns from Ã are linearly independent. The condition ... |

3611 | Compressed sensing
- Donoho
- 2006
(Show Context)
Citation Context ...tries, rather than y, which has n. Attempt recovery by solving min ‖x‖0 subject to ‖c − PAx‖2 ≤ ɛ x to obtain the sparse representation and then synthesizing an approximate reconstruction using Axɛ 0 =-=[18, 15, 17, 58, 42]-=-. • Morphological Component Analysis (MCA). Suppose that the observed signal is a superposition of two different subsignals y1, y2 (i.e., y = y1 + y2), where y1 is sparsely generated using model M1 an... |

2718 | Atomic decomposition by basis pursuit
- Chen, Donoho, et al.
- 1999
(Show Context)
Citation Context ...l local minimum will actually be a good approximation to a global minimum of (P0). Another strategy is to replace the ℓ0 norm by the ℓ1 norm, which is, in a natural sense, its best convex approximant =-=[24, 25, 142]-=-; many optimization tools are available “off the shelf” for solving (P1). Turning from (P0) to its regularizations (Pp) with 0 <p≤ 1, care must be taken with respect to normalization of the columns in... |

2621 | Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information - Candes, Romberg, et al. - 2006 |

2132 |
Vector Quantization and Signal Compression
- Gersho, Gray
- 1992
(Show Context)
Citation Context ...tation coefficients to be binary (1 or 0), the above-posed problem reduces to a clustering task. Furthermore, in such a case the above training algorithms simplify to the well-known K-means algorithm =-=[81]-=-. While each iteration of K-means computes means over K different subsets, the K-SVD algorithm performs the SVD over each of K different submatrices, hence the name K-SVD (K is assumed to be the numbe... |

1686 |
Learning the parts of objects by non-negative matrix factorization
- Lee, Seung
- 1999
(Show Context)
Citation Context ...he matrix Y as AX, where A and X have the indicated shapes and X has sparse columns. The matrix factorization viewpoint connects this problem with related problems of nonnegative matrix factorization =-=[104, 55]-=- and sparse nonnegative matrix factorization [92, 1]. Clearly, there is no general practical algorithm for solving problem (52) or (53), for the same reasons that there is no general practical algorit... |

1509 | Near-Optimal signal recovery from random projections: Universal encoding strategies - Candes, Tao - 2006 |

1398 | Decoding by linear programming
- Candes, Tao
- 2005
(Show Context)
Citation Context ...k ≈ r(δ)n for an unspecified function r > 0. Candès and Tao considered random Gaussian matrices and were able to show that k ≤ r(m/n)n was sufficient for equivalence for a certain explicit function r =-=[18]-=-. These qualitative results opened the way to asking for the precise quantitative behavior, i.e., for ρW above. • Tropp, Gilbert, and coworkers [158] studied running OMP over the problem suite consist... |

1389 | Stable signal recovery from incomplete and inaccurate measurements
- Candès, Romberg, et al.
- 2006
(Show Context)
Citation Context ...ilarity of the coherence-based results for the two methods. A great deal of algorithmic progress was made while this paper was in review and revision. We mention only two examples. Candès and Romberg =-=[14]-=- have developed a fast approximate ℓ1 solver using projections onto convex sets. Stephen Boyd and coworkers [100] have found a way to speed up standard interior-point methods so that, when the solutio... |

1277 | De-Noising by Soft-Thresholding
- Donoho
- 1995
(Show Context)
Citation Context ...n is key to widely used techniques of transformbased image compression. Transform sparsity is also a driving factor for other important signal and image processing problems, including image denoising =-=[50, 51, 27, 43, 53, 52, 144, 124, 96]-=- and image deblurring [76, 75, 74, 41]. Repeatedly, it has been shown that a better representation technique—one that leads to more sparsity—can be the basis for a practically better solution to such ... |

1267 | Ideal spatial adaptation by wavelet shrinkage
- Donoho, Johnstone
- 1994
(Show Context)
Citation Context ...n is key to widely used techniques of transformbased image compression. Transform sparsity is also a driving factor for other important signal and image processing problems, including image denoising =-=[50, 51, 27, 43, 53, 52, 144, 124, 96]-=- and image deblurring [76, 75, 74, 41]. Repeatedly, it has been shown that a better representation technique—one that leads to more sparsity—can be the basis for a practically better solution to such ... |

930 | K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation - Aharon, Elad, et al. - 2006 |

745 | An iterative thresholding algorithm for linear inverse problems with a sparsity constraint - Daubechies, Defrise, et al. - 2004 |

629 | Optimally sparse representation in general (nonorthogonal) dictionaries via ` minimization
- Donoho, Elad
- 2003
(Show Context)
Citation Context ...=J1(x) =‖x‖1. This problem is somehow intermediate between (P2) and (P0). It is a convex optimization problem, and among convex problems it is in some sense the one closest to (P0). We will see below =-=[49, 93, 46]-=- that for matrices A with incoherent columns, whenever (P0) has a sufficiently sparse solution, that solution is unique and is equal to the solution of (P1). Since (P1) is convex, the solution can thu... |

598 |
Image denoising via sparse and redundant representations over learned dictionaries
- Elad, Aharon
- 2006
(Show Context)
Citation Context ...f results from JPEG, JPEG2000, PCA, and sparse coding with K-SVD dictionary training. The values below each result show the PSNR. 6.2.2. Methodology and Algorithms. The denoising methods described in =-=[63, 64]-=- take a different approach: by training a dictionary on the image content directly. One option is to use a standard library of clean images, e.g., the Corel library of 60,000 images, and develop a sta... |

581 | Uncertainty principles and ideal atomic decomposition
- Donoho, Huo
(Show Context)
Citation Context ...rinciple was true; while the uncertainty principle guarantees that the number of nonzeros in the combined time-frequency analysis must exceed √ n; in fact, the typical number is closer to n. Also, in =-=[49]-=- simulations very much like those reported above in section 3.3.1 were presented to show that the equivalence between ℓ1 and ℓ0 representations is typical at surprisingly weak levels of sparsity; in f... |

567 | For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution
- Donoho
- 2006
(Show Context)
Citation Context ... 0% success. As the problem size increases, the transition from typicality of success to typicality of failure becomes increasingly sharp—in the large-n limit, perfectly sharp. A rigorous result from =-=[44, 56, 57]-=- explains the meaning of the curve in panel (a). Theorem 11. Fix a (δ, ρ) pair.At problem size n, set mn = ⌊n/δ⌋ and kn = ⌊nρ⌋.Draw a problem instance y = Ax at random with A an n × mn matrix from the... |

537 | Sparse MRI: The application of compressed sensing for rapid MR imaging
- Lustig, Donoho, et al.
- 2007
(Show Context)
Citation Context ...ures 5 and 6 two worked out large-scale applications. Figure 5 presents compressed sensing of dynamic MRI—real-time acquisition of heart motion—by Michael Lustig and coworkers at the Stanford MRI lab =-=[112, 111]-=-. They obtain a successful reconstruction of moving imagery of the beating heart from raw pseudorandom samples of the k-t space, with a factor of 7 undersampling, i.e., they solve a system of equation... |

513 | The contourlet transform : an efficient directional multiresolution image representation.”
- Do, Vetterli
- 2005
(Show Context)
Citation Context ... A to perform well on the signals we have in mind? One line of work considered choosing preconstructed dictionaries, such as undecimated wavelets [149], steerable wavelets [145, 37, 136], contourlets =-=[38, 39, 40, 70, 71]-=-, curvelets [146, 12], and others [22, 123]. These are generally suitable for stylized “cartoon-like” image content, assumed to be piecewise smooth and with smooth boundaries. Some of these papers pro... |

498 | Non-negative Matrix Factorization with Sparseness Constraints,
- Hoyer
- 2004
(Show Context)
Citation Context ...apes and X has sparse columns. The matrix factorization viewpoint connects this problem with related problems of nonnegative matrix factorization [104, 55] and sparse nonnegative matrix factorization =-=[92, 1]-=-. Clearly, there is no general practical algorithm for solving problem (52) or (53), for the same reasons that there is no general practical algorithm for solving (P0), only more so! However, just as ... |

459 | Stable recovery of sparse overcomplete representations in the presence of noise
- Donoho, Elad, et al.
- 2006
(Show Context)
Citation Context ...esponds to the OMP algorithm as described in Exhibit 1, and the other for BP (i.e., solving (P1) in place of (P0)). 2.3.1. The GA Solves (P0) in Sufficiently Sparse Cases. Theorem 6 (equivalence: OGA =-=[156, 48]-=-). For a system of linear equations Ax = b (A ∈ Rn×m full-rank with n<m), if a solution x exists obeying (9) ‖x‖0 < 1 ( 1+ 2 1 ) , µ(A) an OGA run with threshold parameter ɛ0 =0is guaranteed to find i... |

429 | New tight frames of curvelets and optimal representations of objects with piecewise c-2 singularities
- Candès, Donoho
(Show Context)
Citation Context ...ery powerful sparsity constraint. The weak ℓp norm is a popular measure of sparsity in the mathematical analysis community; models of cartoon images have sparse representations as measured in weak ℓp =-=[26, 13]-=-. Almost equivalent are the usual ℓ p norms, defined by ( ∑ ‖x‖p = |xi| p i ) 1/p These will seem more familiar objects than the weak ℓp norms, in the range 1 ≤ p ≤∞; however, for measuring sparsity, ... |

372 |
Image compression through wavelet transform coding
- DeVore, Jawerth, et al.
- 1992
(Show Context)
Citation Context ...ld in such cases seek a better prior. Careful empirical modeling of wavelet coefficients of images with edges has shown that, in many cases, the prior model p(y) ∝ exp(−λ‖Ty‖1) can indeed be improved =-=[35, 144, 10]-=-. The Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.SPARSE MODELING OF SIGNALS AND IMAGES 63 general form p(y) ∝ exp(−λ‖Ty‖r r) with 0 <r<1 has been studied, and values... |

366 | Sparse signal reconstruction from limited data using focuss: a re-weighted minimum norm algorithm
- Gorodnitsky, Rao
- 1997
(Show Context)
Citation Context ...k, as it calls for a combinatorial search over all possible subsets of columns from A. The importance of this property of matrices for the study of the uniqueness of sparse solutions was unraveled in =-=[84]-=-. Interestingly, this property previously appeared in the literature of psychometrics (termed Kruskal rank), used in the context of studying uniqueness of tensor decomposition [102, 110]. The spark is... |

362 | Improved Adaptive Wavelet Threshold for Image Denoising”.
- Zhang, Yu, et al.
- 2009
(Show Context)
Citation Context ...resent edges than do Fourier and wavelet methods. By shrinkage of transform coefficients followed by reconstruction, some reduction in image noise is observed, while edges are approximately preserved =-=[103, 19, 20, 21, 146, 136, 70, 71, 88, 89]-=-. Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.SPARSE MODELING OF SIGNALS AND IMAGES 73 Original JPEG (26.59dB) JPEG-2000 (27.81dB) PCA (29.27dB) K-SVD (33.26dB) Origi... |

353 | Learning overcomplete representations - Lewicki, Senowski - 2000 |

352 | An EM algorithm for wavelet-based image restoration
- Figueiredo, Nowak
(Show Context)
Citation Context ...ompression. Transform sparsity is also a driving factor for other important signal and image processing problems, including image denoising [50, 51, 27, 43, 53, 52, 144, 124, 96] and image deblurring =-=[76, 75, 74, 41]-=-. Repeatedly, it has been shown that a better representation technique—one that leads to more sparsity—can be the basis for a practically better solution to such problems. For example, it has been fou... |

349 | Good quantum error-correcting codes exist.
- Calderbank, Shor
- 1996
(Show Context)
Citation Context ...nformation theory, constructing error-correcting codes using a collection of orthogonal bases with minimal coherence, obtaining similar bounds on the mutual coherence for amalgams of orthogonal bases =-=[11]-=-. Mutual coherence, relatively easy to compute, allows us to lower bound the spark, which is often hard to compute. Lemma 4 (see [46]). For any matrix A ∈ R n×m , the following relationship holds: (7)... |

320 | Minimax estimation via wavelet shrinkage
- Donoho, Johnstone
- 1998
(Show Context)
Citation Context ...n is key to widely used techniques of transformbased image compression. Transform sparsity is also a driving factor for other important signal and image processing problems, including image denoising =-=[50, 51, 27, 43, 53, 52, 144, 124, 96]-=- and image deblurring [76, 75, 74, 41]. Repeatedly, it has been shown that a better representation technique—one that leads to more sparsity—can be the basis for a practically better solution to such ... |

311 |
Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics
- Kruskal
- 1977
(Show Context)
Citation Context ...ns was unraveled in [84]. Interestingly, this property previously appeared in the literature of psychometrics (termed Kruskal rank), used in the context of studying uniqueness of tensor decomposition =-=[102, 110]-=-. The spark is also related to established notions in matroid theory; formally, it is precisely the girth of the linear matroid defined by A, i.e., the length of the shortest cycle in that matroid [16... |

306 | Translation-invariant de-noising
- Coifman, Donoho
- 1995
(Show Context)
Citation Context |

295 | Wavelet shrinkage: asymptopia? - Donoho, Johnstone, et al. - 1995 |

274 | Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit,”
- Donoho, Tsaig, et al.
- 2006
(Show Context)
Citation Context ... that are concatenations of unitary matrices, and iteratively updates the solution one part at a time, using shrinkage [143]. Handling of the general case is somewhat more involved, as can be seen in =-=[76, 75, 32, 62, 5, 67, 59]-=-. Exhibit 3 spells out the details. The algorithm is particularly useful in large-scale problems, where A is defined not by an explicitly given matrix, but instead by an operator which we know how to ... |

269 | Uncertainty principles and signal recovery - Donoho, Stark - 1989 |

268 | Orthogonal least squares methods and their application to non-linear system identification,” Int - Chen, Billings, et al. - 1989 |

252 | On sparse representations in arbitrary redundant bases
- Fuchs
(Show Context)
Citation Context ...of the problem in the general case. However, if the equation actually has a “sufficiently sparse” solution, the success of these algorithms in addressing the original objective (P0) can be guaranteed =-=[46, 86, 79, 156, 133, 134, 135]-=-. We present here two such results, one that corresponds to the OMP algorithm as described in Exhibit 1, and the other for BP (i.e., solving (P1) in place of (P0)). 2.3.1. The GA Solves (P0) in Suffic... |

246 | A generalized uncertainty principle and sparse representation in pairs of bases
- Elad, Bruckstein
- 2002
(Show Context)
Citation Context ...le β is the frequency-domain representation. For certain pairs of bases ΨΦ, an interesting phenomenon occurs: either α can be sparse, or β can be sparse, but not both! In fact, we have the inequality =-=[54, 49, 65]-=- (32) (Uncertainty Principle 1) : ||α||0 + ||β||0 ≥ 2/µ(A). So if the mutual coherence of two bases is small, then α and β cannot both be very sparse. For example, if, as above, Ψ is the identity and ... |

238 | Image compression via joint statistical characterization in the wavelet domain.
- Buccigrossi, Simoncelli
- 1999
(Show Context)
Citation Context ...ld in such cases seek a better prior. Careful empirical modeling of wavelet coefficients of images with edges has shown that, in many cases, the prior model p(y) ∝ exp(−λ‖Ty‖1) can indeed be improved =-=[35, 144, 10]-=-. The Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.SPARSE MODELING OF SIGNALS AND IMAGES 63 general form p(y) ∝ exp(−λ‖Ty‖r r) with 0 <r<1 has been studied, and values... |

224 | Spatially Adaptive Wavelet Thresholding with Context Modeling for Image denoising”, - Chang, Yu, et al. - 2000 |

215 |
Simultaneous cartoon and texture image inpainting using morphological component analysis
- Elad, Starck, et al.
- 2005
(Show Context)
Citation Context ...2, 148, 147, 7, 8]. An appealing image processing application that relies on MCA is inpainting, where missing pixels in an image are filled in, based on a sparse representation of the existing pixels =-=[68]-=-. MCA is necessary because the piecewise smooth (cartoon) and texture contents of the image must be separated as part of this recovery process. See [68] for more details. A wide range of other applica... |

200 | When does non-negative matrix factorization give a correct decomposition into parts
- Donoho, Stodden
- 2003
(Show Context)
Citation Context ...he matrix Y as AX, where A and X have the indicated shapes and X has sparse columns. The matrix factorization viewpoint connects this problem with related problems of nonnegative matrix factorization =-=[104, 55]-=- and sparse nonnegative matrix factorization [92, 1]. Clearly, there is no general practical algorithm for solving problem (52) or (53), for the same reasons that there is no general practical algorit... |

194 | Sparse nonnegative solution of underdetermined linear equations by linear programming.
- Donoho, Tanner
- 2005
(Show Context)
Citation Context ... 0% success. As the problem size increases, the transition from typicality of success to typicality of failure becomes increasingly sharp—in the large-n limit, perfectly sharp. A rigorous result from =-=[44, 56, 57]-=- explains the meaning of the curve in panel (a). Theorem 11. Fix a (δ, ρ) pair.At problem size n, set mn = ⌊n/δ⌋ and kn = ⌊nρ⌋.Draw a problem instance y = Ax at random with A an n × mn matrix from the... |

190 | Adaptive greedy approximations - Davis, Mallat, et al. - 1997 |

183 |
Dictionary learning algorithms for sparse representation
- Kreutz-Delgado, Murray, et al.
- 2003
(Show Context)
Citation Context ... rough empirical match between the properties of a learned dictionary and some known properties of the population of simple cells. Later work extended their methodology and algorithm in various forms =-=[107, 108, 69, 101, 106, 3, 2]-=-. Here we describe a related training mechanism based on [69, 101, 3, 2]. Assume that ɛ—the model deviation—is known, and that our aim is the estimation of A. Consider the following optimization probl... |

181 | Quantitative robust uncertainty principles and optimally sparse decompositions - Candes, Romberg - 2006 |

160 | Least angle regression. Annals of Statistics - Efron, Hastie, et al. - 2004 |

159 | Basis pursuit. - Chen, Donoho - 1994 |

140 | Probabilistic framework for the adaptation and comparison of image codes.
- Lewicki, Olshausen
- 1999
(Show Context)
Citation Context ... rough empirical match between the properties of a learned dictionary and some known properties of the population of simple cells. Later work extended their methodology and algorithm in various forms =-=[107, 108, 69, 101, 106, 3, 2]-=-. Here we describe a related training mechanism based on [69, 101, 3, 2]. Assume that ɛ—the model deviation—is known, and that our aim is the estimation of A. Consider the following optimization probl... |

139 |
Some remarks on greedy algorithms,”
- DeVore, Temlyakov
- 1996
(Show Context)
Citation Context ...ires O(nm k0 k0 2 ) flops. Thus, this single-term-at-a-time strategy can be much more efficient than exhaustive search—if it works! The strategy can fail badly, i.e., there are explicit examples (see =-=[154, 155, 36]-=-) where a simple k-term representation is possible, but this approach yields an n-term (i.e., dense) representation. In general, all that can be said is that among single-term-at-a-time strategies, th... |

134 | Recovery of exact sparse representations in the presence of noise,”
- Fuchs
- 2005
(Show Context)
Citation Context ...nce vector of finite energy ‖z‖ 2 2 = δ 2 . Roughly speaking (P δ 0 ) aims to find x0, i.e., to do roughly the same thing as (P0) would do on noiseless data b = Ax0. Several papers study this problem =-=[163, 48, 47, 157, 80]-=-, and we briefly discuss some of what is now known. Results are in some ways parallel to those in the noiseless case, although the notions of uniqueness and equivalence no longer apply—they are replac... |

122 | Noise reduction using an undecimated discrete wavelet transform,” IEEE of the practical signal. „a… Original results. „b… Results after „d… Results after using the proposed strategy.ts lts.Signal
- Lang, Guo, et al.
- 1996
(Show Context)
Citation Context ...resent edges than do Fourier and wavelet methods. By shrinkage of transform coefficients followed by reconstruction, some reduction in image noise is observed, while edges are approximately preserved =-=[103, 19, 20, 21, 146, 136, 70, 71, 88, 89]-=-. Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.SPARSE MODELING OF SIGNALS AND IMAGES 73 Original JPEG (26.59dB) JPEG-2000 (27.81dB) PCA (29.27dB) K-SVD (33.26dB) Origi... |

118 | Ideal denoising in an orthonormal basis chosen from a library of bases.
- Donoho, Johnstone
- 1994
(Show Context)
Citation Context |

115 | simple shrinkage is still relevant for redundant representations ?”
- Elad
- 2006
(Show Context)
Citation Context ...avy task. An emerging alternative to approximately solving (P1) has been recently introduced in several independent papers, leading to similar algorithms that may be called iterated shrinkage methods =-=[32, 76, 75, 5, 62, 67]-=-. These iteratively use multiplication by A and its adjoint, and a simple 1D operation that sets to zero small entries—a shrinkage operation. These methods can compete with the greedy methods in simpl... |

105 | The Finite Ridgelet Transform for Image Representation,
- Do, Vetterli
- 2003
(Show Context)
Citation Context ... A to perform well on the signals we have in mind? One line of work considered choosing preconstructed dictionaries, such as undecimated wavelets [149], steerable wavelets [145, 37, 136], contourlets =-=[38, 39, 40, 70, 71]-=-, curvelets [146, 12], and others [22, 123]. These are generally suitable for stylized “cartoon-like” image content, assumed to be piecewise smooth and with smooth boundaries. Some of these papers pro... |

100 | Approximation of functions over redundant dictionaries using coherence,” in
- Gilbert, Muthukrishnan, et al.
- 2003
(Show Context)
Citation Context ...ns why this type of algorithm has earned the name “greedy algorithm” in approximation theory. Many variants on this algorithm are available, offering improvements in accuracy or in complexity or both =-=[118, 34, 33, 23, 130, 30, 159, 82]-=-. This family of GAs is well known and extensively used, and, in fact, these algorithms have been reinvented in various fields. In the setting of statistical modeling, greedy stepwise least squares is... |

95 |
Adaptive time-frequency decompositions’,
- DAVIS, MALLAT, et al.
- 1994
(Show Context)
Citation Context ...ns why this type of algorithm has earned the name “greedy algorithm” in approximation theory. Many variants on this algorithm are available, offering improvements in accuracy or in complexity or both =-=[118, 34, 33, 23, 130, 30, 159, 82]-=-. This family of GAs is well known and extensively used, and, in fact, these algorithms have been reinvented in various fields. In the setting of statistical modeling, greedy stepwise least squares is... |

92 | Neighborliness of randomly-projected simplices in high dimensions
- Donoho, Tanner
- 2005
(Show Context)
Citation Context ...ssful recovery by BP and OMP as ρ increased from .3 to.6. Such phenomena have been observed for a variety of sparsity-seeking algorithms. A typical example is given in Figure 4. Panel (a), taken from =-=[57]-=-, depicts the unit square of interesting δ − ρ behavior; the shaded attribute displays simulation results for the probability that the solutions to ℓ1 and ℓ0 are equivalent. Just as in Figure 2, there... |

81 | Recovering edges in ill-posed inverse problems optimality of curvelet frames
- Candès, Donoho
(Show Context)
Citation Context ... coworkers [148, 147], where the image Barbara is decomposed into piecewise smooth (cartoon) and texture, using MCA as described above. They used a dictionary combining two representations: curvelets =-=[146, 12, 13]-=- for representing the cartoon part, and local overlapped DCT for the texture. The second row in this figure, taken from [68], presents inpainting results, where missing values (the text) are recovered... |

81 | On sparse representation in pairs of bases
- Feuer, Nemirovski
- 2003
(Show Context)
Citation Context ...of Pursuit Algorithms. Given the uniqueness of a sufficiently sparse solution of [Ψ Φ]x = b, it becomes natural to ask how specific algorithms perform. A result, similar to Theorem 7, was obtained in =-=[65, 73]-=-, showing that (36) √ 2 − 0.5 ‖x‖0 < µ(A) ensures that BP finds the proper (sparsest) solution. This was the first result of its kind; only later was the more general A case addressed. This result is ... |

80 | On the exponential convergence of matching pursuits in quasi-incoherent dictionaries,”
- Gribonval, V
- 2006
(Show Context)
Citation Context ...34, 33] or OMP [23, 130]. Approximation theorists refer to these algorithms as GAs and consider several variants of them—the pure (PGA), the orthogonal (OGA), the relaxed (RGA), and the weak GA (WGA) =-=[154, 155, 36, 4, 87]-=-. 2.2.2. Convex Relaxation Techniques. A second way to render (P0) more tractable is to regularize the (highly discontinuous) ℓ0 norm, replacing it by a continuous or even smooth approximation. Exampl... |

78 |
Signal processing and compression with wavelet packets. In: Meyer Y and Roques S (eds) Progress in wavelet analysis and applications. Paris: Editions Frontieres,
- RR, Meyer, et al.
- 1990
(Show Context)
Citation Context ..., one can use a tunable selection, in which a basis or frame is generated under the control of particular parameter (discrete or continuous): wavelet packets (parameter is time-frequency subdivision) =-=[28, 29, 121]-=- or bandelettes (parameter is spatial partition) [105, 117]. A third option is to build a training database of signal instances similar to those anticipated in the application, and build an empiricall... |

71 | Image Denoising Via Learned Dictionaries and Sparse representation.,”
- Elad, Aharon
- 2006
(Show Context)
Citation Context ...f results from JPEG, JPEG2000, PCA, and sparse coding with K-SVD dictionary training. The values below each result show the PSNR. 6.2.2. Methodology and Algorithms. The denoising methods described in =-=[63, 64]-=- take a different approach: by training a dictionary on the image content directly. One option is to use a standard library of clean images, e.g., the Corel library of 60,000 images, and develop a sta... |

70 |
A bound optimization approach to wavelet-based image deconvolution
- Figueiredo, Nowak
(Show Context)
Citation Context ...ompression. Transform sparsity is also a driving factor for other important signal and image processing problems, including image denoising [50, 51, 27, 43, 53, 52, 144, 124, 96] and image deblurring =-=[76, 75, 74, 41]-=-. Repeatedly, it has been shown that a better representation technique—one that leads to more sparsity—can be the basis for a practically better solution to such problems. For example, it has been fou... |

67 | Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavytailed priors,”
- Bioucas-Dias
- 2006
(Show Context)
Citation Context ...avy task. An emerging alternative to approximately solving (P1) has been recently introduced in several independent papers, leading to similar algorithms that may be called iterated shrinkage methods =-=[32, 76, 75, 5, 62, 67]-=-. These iteratively use multiplication by A and its adjoint, and a simple 1D operation that sets to zero small entries—a shrinkage operation. These methods can compete with the greedy methods in simpl... |

64 | Framing pyramids
- Do, Vetterli
- 2003
(Show Context)
Citation Context ... A to perform well on the signals we have in mind? One line of work considered choosing preconstructed dictionaries, such as undecimated wavelets [149], steerable wavelets [145, 37, 136], contourlets =-=[38, 39, 40, 70, 71]-=-, curvelets [146, 12], and others [22, 123]. These are generally suitable for stylized “cartoon-like” image content, assumed to be piecewise smooth and with smooth boundaries. Some of these papers pro... |

64 | Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising: Part 2 - adaptive algorithms,”
- Guleryuz
- 2004
(Show Context)
Citation Context ...nt in image compression and image denoising. Due to space imitations, we are unable to discuss many other interesting examples, including problems in array processing [114, 115], inpainting in images =-=[68, 88, 89, 72]-=-, image decomposition to cartoon and texture [122, 148, 147], and others [99, 7, 113, 137]. Also, the applications presented here rely on an adapted dictionary using the K-SVD algorithm, but successfu... |

63 |
Sparse decompositions in unions of bases
- Gribonval, Nielsen
- 2003
(Show Context)
Citation Context ...of the problem in the general case. However, if the equation actually has a “sufficiently sparse” solution, the success of these algorithms in addressing the original objective (P0) can be guaranteed =-=[46, 86, 79, 156, 133, 134, 135]-=-. We present here two such results, one that corresponds to the OMP algorithm as described in Exhibit 1, and the other for BP (i.e., solving (P1) in place of (P0)). 2.3.1. The GA Solves (P0) in Suffic... |

55 | Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden markov models,
- Do, Vetterli
- 2002
(Show Context)
Citation Context ... A. How can we wisely choose A to perform well on the signals we have in mind? One line of work considered choosing preconstructed dictionaries, such as undecimated wavelets [149], steerable wavelets =-=[145, 37, 136]-=-, contourlets [38, 39, 40, 70, 71], curvelets [146, 12], and others [22, 123]. These are generally suitable for stylized “cartoon-like” image content, assumed to be piecewise smooth and with smooth bo... |

54 |
Noise Reduction by Wavelet Thresholding,
- Jansen
- 2001
(Show Context)
Citation Context |

54 |
The widths of certain finite-dimensional sets and classes of smooth functions
- Kashin
- 1977
(Show Context)
Citation Context ...ces; this principle has been used in [44, 18] in studying the equivalence of (P1) and (P0) and in [158] in studying OMP. Other fundamental ideas include Kashin’s results on n-widths of the octahedron =-=[98]-=-, Milman’s quotient of a subspace theorem, and Szarek’s volume bounds [132], all reflecting the miraculous properties of ℓ1 norms when restricted to random subspaces, which lie at the heart of the (P0... |

54 |
Analysis of sparse representation and blind source separation. Neural Computation
- Li, Cichocki, et al.
- 2004
(Show Context)
Citation Context ...Such source separation problems are fundamental in the processing of acoustic signals, for example, in the separation of speech from impulsive noise by independent component analysis (ICA) algorithms =-=[94, 164, 109]-=-. Turning to the signal model presented here, if we could solve min x1,x2 ‖x1‖0 + ‖x2‖0 subject to ‖y − A1x1 − A2x2‖ 2 2 ≤ ɛ 2 1 + ɛ 2 2, the resulting solution (x ɛ 1, x ɛ 2) would generate a plausib... |

52 | On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them
- Aharon, Elad, et al.
(Show Context)
Citation Context ... rough empirical match between the properties of a learned dictionary and some known properties of the population of simple cells. Later work extended their methodology and algorithm in various forms =-=[107, 108, 69, 101, 106, 3, 2]-=-. Here we describe a related training mechanism based on [69, 101, 3, 2]. Assume that ɛ—the model deviation—is known, and that our aim is the estimation of A. Consider the following optimization probl... |

46 | K-svd and its non-negative variant for dictionary design. volume 5914,
- Aharon, Elad, et al.
- 2005
(Show Context)
Citation Context ...apes and X has sparse columns. The matrix factorization viewpoint connects this problem with related problems of nonnegative matrix factorization [104, 55] and sparse nonnegative matrix factorization =-=[92, 1]-=-. Clearly, there is no general practical algorithm for solving problem (52) or (53), for the same reasons that there is no general practical algorithm for solving (P0), only more so! However, just as ... |

44 | Denoising by sparse approximation: Error bounds based on rate-distortion theory - Fletcher, Rangan, et al. - 2006 |

44 |
A method for large-scale ℓ1-regularized least squares problems with applications in signal processing and statistics
- Kim, Koh, et al.
- 2007
(Show Context)
Citation Context ...this paper was in review and revision. We mention only two examples. Candès and Romberg [14] have developed a fast approximate ℓ1 solver using projections onto convex sets. Stephen Boyd and coworkers =-=[100]-=- have found a way to speed up standard interior-point methods so that, when the solution is sparse, they run quickly. 3.2.4. Performance of Pursuit Algorithms. Can pursuit methods approximately solve ... |

43 | On the optimality of the backward greedy algorithm for the subset selection problem
- Couvreur, Bresler
- 2000
(Show Context)
Citation Context ...ns why this type of algorithm has earned the name “greedy algorithm” in approximation theory. Many variants on this algorithm are available, offering improvements in accuracy or in complexity or both =-=[118, 34, 33, 23, 130, 30, 159, 82]-=-. This family of GAs is well known and extensively used, and, in fact, these algorithms have been reinvented in various fields. In the setting of statistical modeling, greedy stepwise least squares is... |

43 |
Multi-frame compression: Theory and design
- Engan, Aase, et al.
- 2000
(Show Context)
Citation Context |

39 |
Fitting Equations to Data: Computer Analysis of Multifactor Data. 2nd ed
- Daniel, Wood, et al.
- 1980
(Show Context)
Citation Context ...s have been reinvented in various fields. In the setting of statistical modeling, greedy stepwise least squares is called forward stepwise regression and has been widely used since at least the 1960s =-=[31, 90]-=-. When used in the signal processing setting this goes by the name of Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.SPARSE MODELING OF SIGNALS AND IMAGES 43 matching pu... |

39 | k-t SPARSE: high frame rate dynamic MRI exploiting spatio-temporal sparsity,”
- Lustig, Santos, et al.
- 2006
(Show Context)
Citation Context ...ures 5 and 6 two worked out large-scale applications. Figure 5 presents compressed sensing of dynamic MRI—real-time acquisition of heart motion—by Michael Lustig and coworkers at the Stanford MRI lab =-=[112, 111]-=-. They obtain a successful reconstruction of moving imagery of the beating heart from raw pseudorandom samples of the k-t space, with a factor of 7 undersampling, i.e., they solve a system of equation... |

38 | Image denoising with shrinkage and redundant representation,” in CVPR,
- Elad, Matalon, et al.
- 2006
(Show Context)
Citation Context ...tions presented here rely on an adapted dictionary using the K-SVD algorithm, but successful applications can be demonstrated using other dictionaries, such as curvelets, contourlets, and others; see =-=[146, 148, 68, 66, 67]-=- for some examples. 6.1. Compression of Facial Images. 6.1.1. The Application. Image compression is fundamental to today’s use of digital imagery; we exploit it on a daily basis, in our digital camera... |

37 | Yu,Wavelet approximation in the space C - DeVore, Petrushev, et al. - 1992 |

37 | Learning unions of orthonormal bases with thresholded singular value decomposition - Lesage, Gribonval, et al. - 2004 |

36 | Sparse representation-based image deconvolution by iterative thresholding
- Fadili, Starck
- 2006
(Show Context)
Citation Context ...nt in image compression and image denoising. Due to space imitations, we are unable to discuss many other interesting examples, including problems in array processing [114, 115], inpainting in images =-=[68, 88, 89, 72]-=-, image decomposition to cartoon and texture [122, 148, 147], and others [99, 7, 113, 137]. Also, the applications presented here rely on an adapted dictionary using the K-SVD algorithm, but successfu... |

34 | A simple test to check the optimality of a sparse signal approximation, - Gribonval, Ventura, et al. - 2006 |

32 | Cramér-Rao lower bounds for low-rank decomposition of multidimensional arrays
- Liu, Sidiropoulos
- 2001
(Show Context)
Citation Context ...ns was unraveled in [84]. Interestingly, this property previously appeared in the literature of psychometrics (termed Kruskal rank), used in the context of studying uniqueness of tensor decomposition =-=[102, 110]-=-. The spark is also related to established notions in matroid theory; formally, it is precisely the girth of the linear matroid defined by A, i.e., the length of the shortest cycle in that matroid [16... |

30 | Morphological diversity and source separation,”
- Bobin, Moudden, et al.
- 2006
(Show Context)
Citation Context ...ld generate a plausible solution ˆy1 = Ax ɛ 1, ˆy2 = Ax ɛ 2 to the separation problem. In fact, there have been several successful trials of this idea, first in acoustic and later in image processing =-=[122, 148, 147, 7, 8]-=-. An appealing image processing application that relies on MCA is inpainting, where missing pixels in an image are filled in, based on a sparse representation of the existing pixels [68]. MCA is neces... |

29 | Sparse image representation via combined transforms
- Huo
- 1999
(Show Context)
Citation Context ...=J1(x) =‖x‖1. This problem is somehow intermediate between (P2) and (P0). It is a convex optimization problem, and among convex problems it is in some sense the one closest to (P0). We will see below =-=[49, 93, 46]-=- that for matrices A with incoherent columns, whenever (P0) has a sufficiently sparse solution, that solution is unique and is equal to the solution of (P1). Since (P1) is convex, the solution can thu... |

24 |
Cross-modal localization via sparsity
- Kidron, Schechner, et al.
- 2007
(Show Context)
Citation Context ...scuss many other interesting examples, including problems in array processing [114, 115], inpainting in images [68, 88, 89, 72], image decomposition to cartoon and texture [122, 148, 147], and others =-=[99, 7, 113, 137]-=-. Also, the applications presented here rely on an adapted dictionary using the K-SVD algorithm, but successful applications can be demonstrated using other dictionaries, such as curvelets, contourlet... |

17 | Sparse representations are most likely to be the sparsest possible,”
- Elad
- 2006
(Show Context)
Citation Context ...10]. The spark is also related to established notions in matroid theory; formally, it is precisely the girth of the linear matroid defined by A, i.e., the length of the shortest cycle in that matroid =-=[162, 6, 61]-=-. Finally, if we consider the same definition where the arithmetic underlying the matrix product is performed not over the fields of real or complex numbers but instead over the ring of integers mod q... |

17 | Translation-invariant contourlet transform and its application to image denoising, submitted to the
- Eslami, Radha
- 2004
(Show Context)
Citation Context |

16 | Wavelet thresholding for multiple noisy image copies - Chang, Yu, et al. - 2000 |

12 | The contourlet transform for image de-noising using cycle spinning,
- Eslami, Radha
- 2003
(Show Context)
Citation Context |

11 | Adapted waveform analysis as a tool for modeling, feature extraction
- Coifman, Wickerhauser
- 1994
(Show Context)
Citation Context ..., one can use a tunable selection, in which a basis or frame is generated under the control of particular parameter (discrete or continuous): wavelet packets (parameter is time-frequency subdivision) =-=[28, 29, 121]-=- or bandelettes (parameter is spatial partition) [105, 117]. A third option is to build a training database of signal instances similar to those anticipated in the application, and build an empiricall... |

8 | Adaptive approximation and learning by greedy algorithms - Barron, Cohen, et al. |

8 |
Surflets: A sparse representation for multidimensional functions containing smooth discontinuities
- Chandrasekaran, Wakin, et al.
(Show Context)
Citation Context ... line of work considered choosing preconstructed dictionaries, such as undecimated wavelets [149], steerable wavelets [145, 37, 136], contourlets [38, 39, 40, 70, 71], curvelets [146, 12], and others =-=[22, 123]-=-. These are generally suitable for stylized “cartoon-like” image content, assumed to be piecewise smooth and with smooth boundaries. Some of these papers provide detailed theoretical analysis establis... |

4 |
Aghanim, SZ and CMB reconstruction using generalized morphological component analysis
- Bobin, Moudden, et al.
(Show Context)
Citation Context ...ld generate a plausible solution ˆy1 = Ax ɛ 1, ˆy2 = Ax ɛ 2 to the separation problem. In fact, there have been several successful trials of this idea, first in acoustic and later in image processing =-=[122, 148, 147, 7, 8]-=-. An appealing image processing application that relies on MCA is inpainting, where missing pixels in an image are filled in, based on a sparse representation of the existing pixels [68]. MCA is neces... |

3 | Analysis of denoising by sparse approximation with random frame asymptotics - Fletcher, Rangan, et al. - 2005 |

2 | Face image compression using the K-SVD algorithm, submitted to - Bryt, Elad |

1 | On a Class of Optimization methods for Linear Least Squares with Non-Quadratic Regularization, to appear in Applied and Computational Harmonic Analysis - Elad, Matalon, et al. |

1 |
Construction of nearest points in the ℓp, p even and ℓ∞ norms
- Karlovitz
- 1970
(Show Context)
Citation Context ...j log(1+αx2j ) or ∑ j x2j /(α+x2j ). As an example, the FOCUSS method [84, 139, 138] uses ℓp for some fixed p ∈ (0, 1] and seeks a local minimum of the ℓp norm by iteratively reweighted least squares =-=[97]-=-. This is a practical strategy, but little is known about circumstances where it will be successful, i.e., when a numerical local minimum will actually be a good approximation to a global minimum of (... |