Results 1  10
of
62
KSVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
, 2006
"... In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signalatoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and inc ..."
Abstract

Cited by 930 (41 self)
 Add to MetaCart
In recent years there has been a growing interest in the study of sparse representation of signals. Using an overcomplete dictionary that contains prototype signalatoms, signals are described by sparse linear combinations of these atoms. Applications that use sparse representation are many and include compression, regularization in inverse problems, feature extraction, and more. Recent activity in this field has concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. Designing dictionaries to better fit the above model can be done by either selecting one from a prespecified set of linear transforms or adapting the dictionary to a set of training signals. Both of these techniques have been considered, but this topic is largely still open. In this paper we propose a novel algorithm for adapting dictionaries in order to achieve sparse signal representations. Given a set of training signals, we seek the dictionary that leads to the best representation for each member in this set, under strict sparsity constraints. We present a new method—the KSVD algorithm—generalizing the umeans clustering process. KSVD is an iterative method that alternates between sparse coding of the examples based on the current dictionary and a process of updating the dictionary atoms to better fit the data. The update of the dictionary columns is combined with an update of the sparse representations, thereby accelerating convergence. The KSVD algorithm is flexible and can work with any pursuit method (e.g., basis pursuit, FOCUSS, or matching pursuit). We analyze this algorithm and demonstrate its results both on synthetic tests and in applications on real image data.
From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images
, 2007
"... A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combin ..."
Abstract

Cited by 423 (37 self)
 Add to MetaCart
(Show Context)
A fullrank matrix A ∈ IR n×m with n < m generates an underdetermined system of linear equations Ax = b having infinitely many solutions. Suppose we seek the sparsest solution, i.e., the one with the fewest nonzero entries: can it ever be unique? If so, when? As optimization of sparsity is combinatorial in nature, are there efficient methods for finding the sparsest solution? These questions have been answered positively and constructively in recent years, exposing a wide variety of surprising phenomena; in particular, the existence of easilyverifiable conditions under which optimallysparse solutions can be found by concrete, effective computational methods. Such theoretical results inspire a bold perspective on some important practical problems in signal and image processing. Several wellknown signal and image processing problems can be cast as demanding solutions of undetermined systems of equations. Such problems have previously seemed, to many, intractable. There is considerable evidence that these problems often have sparse solutions. Hence, advances in finding sparse solutions to underdetermined systems energizes research on such signal and image processing problems – to striking effect. In this paper we review the theoretical results on sparse solutions of linear systems, empirical
Highly sparse representations from dictionaries are unique and independent of the sparseness measure
, 2003
"... ..."
On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them
, 2006
"... ..."
KSVD: Design of dictionaries for sparse representation
 IN: PROCEEDINGS OF SPARS’05
, 2005
"... In recent years there is a growing interest in the study of sparse representation for signals. Using an overcomplete dictionary that contains prototype signalatoms, signals are described by sparse linear combinations of these atoms. Recent activity in this field concentrated mainly on the study of ..."
Abstract

Cited by 45 (1 self)
 Add to MetaCart
(Show Context)
In recent years there is a growing interest in the study of sparse representation for signals. Using an overcomplete dictionary that contains prototype signalatoms, signals are described by sparse linear combinations of these atoms. Recent activity in this field concentrated mainly on the study of pursuit algorithms that decompose signals with respect to a given dictionary. In this paper we propose a novel algorithm – the KSVD algorithm – generalizing the KMeans clustering process, for adapting dictionaries in order to achieve sparse signal representations. We analyze this algorithm and demonstrate its results on both synthetic tests and in applications on real data.
Restricted Isometry Constants where p sparse
, 2011
"... recovery can fail for 0 <p ≤ 1 ..."
(Show Context)
Sparse and redundant modeling of image content using an imagesignaturedictionary
, 2007
"... Modeling signals by a sparse and redundant representations is drawing a considerable attention in recent years. Coupled with the ability to train the dictionary using signal examples, these techniques have been shown to lead to stateoftheart results in a series of recent applications. In this pa ..."
Abstract

Cited by 26 (2 self)
 Add to MetaCart
Modeling signals by a sparse and redundant representations is drawing a considerable attention in recent years. Coupled with the ability to train the dictionary using signal examples, these techniques have been shown to lead to stateoftheart results in a series of recent applications. In this paper we propose a novel structure of such a model for representing image content. The new dictionary is itself a small image, such that every patch in it (in varying location and size) is a possible atom in the representation. We refer to this as the ImageSignatureDictionary (ISD), and show how it can be trained from image examples. This novel structure enjoys several important features, such as shift and scale flexibilities, and smaller memory and computational requirements, compared to the classical dictionary approach. As a demonstration of these benefits, we present highquality image denoising results based on this new model.
Optimal nonlinear models for sparsity and sampling
 JOURNAL OF FOURIER ANALYSIS AND APPLICATIONS, SPECIAL ISSUE ON COMPRESSED SAMPLING
, 2008
"... Given a set of vectors (the data) in a Hilbert space H, we prove the existence of an optimal collection of subspaces minimizing the sum of the square of the distances between each vector and its closest subspace in the collection. This collection of subspaces gives the best sparse representation f ..."
Abstract

Cited by 16 (5 self)
 Add to MetaCart
Given a set of vectors (the data) in a Hilbert space H, we prove the existence of an optimal collection of subspaces minimizing the sum of the square of the distances between each vector and its closest subspace in the collection. This collection of subspaces gives the best sparse representation for the given data, in a sense defined in the paper, and provides an optimal model for sampling in union of subspaces. The results are proved in a general setting and then applied to the case of low dimensional subspaces of RN and to infinite dimensional shiftinvariant spaces in L²(Rd). We also present an iterative search algorithm for finding the solution subspaces. These results are tightly connected to the new emergent theories of compressed sensing and dictionary design, signal models for signals with finite rate of innovation, and the subspace segmentation problem.
Complexvalued sparse representation based on smoothed ℓ0 norm
 in Proceedings of ICASSP2008, Las Vegas
, 2008
"... In this paper we present an algorithm for complexvalued sparse representation. In our previous work we presented an algorithm for Sparse representation based on smoothednorm. Here we extend that algorithm to complexvalued signals. The proposed algorithm is compared to FOCUSS algorithm and it is e ..."
Abstract

Cited by 16 (5 self)
 Add to MetaCart
(Show Context)
In this paper we present an algorithm for complexvalued sparse representation. In our previous work we presented an algorithm for Sparse representation based on smoothednorm. Here we extend that algorithm to complexvalued signals. The proposed algorithm is compared to FOCUSS algorithm and it is experimentally shown that the proposed algorithm is about two or three orders of magnitude faster than FOCUSS while providing approximately the same accuracy. Index Terms — complexvalued sparse component analysis, overcomplete atomic decomposition. 1.
Blind compressed sensing
 IEEE TRANS. INF. THEORY
, 2011
"... The fundamental principle underlying compressed sensing is that a signal, which is sparse under some basis representation, can be recovered from a small number of linear measurements. However, prior knowledge of the sparsity basis is essential for the recovery process. This work introduces the conc ..."
Abstract

Cited by 15 (3 self)
 Add to MetaCart
The fundamental principle underlying compressed sensing is that a signal, which is sparse under some basis representation, can be recovered from a small number of linear measurements. However, prior knowledge of the sparsity basis is essential for the recovery process. This work introduces the concept of blind compressed sensing, which avoids the need to know the sparsity basis in both the sampling and the recovery process. We suggest three possible constraints on the sparsity basis that can be added to the problem in order to guarantee a unique solution. For each constraint, we prove conditions for uniqueness, and suggest a simple method to retrieve the solution. We demonstrate through simulations that our methods can achieve results similar to those of standard compressed sensing, which rely on prior knowledge of the sparsity basis, as long as the signals are sparse enough. This offers a general sampling and reconstruction system that fits all sparse signals, regardless of the sparsity basis, under the conditions and constraints presented in this work.