Results 1  10
of
72
Adaptive Greedy Approximations
"... The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the ..."
Abstract

Cited by 187 (0 self)
 Add to MetaCart
The problem of optimally approximating a function with a linear expansion over a redundant dictionary of waveforms is NPhard. The greedy matching pursuit algorithm and its orthogonalized variant produce suboptimal function expansions by iteratively choosing dictionary waveforms that best match the function's structures. A matching pursuit provides a means of quickly computing compact, adaptive function approximations. Numerical experiments show that the approximation errors from matching pursuits initially decrease rapidly, but the asymptotic decay rate of the errors is slow. We explain this behavior by showing that matching pursuits are chaotic, ergodic maps. The statistical properties of the approximation errors of a pursuit can be obtained from the invariant measure of the pursuit. We characterize these measures using group symmetries of dictionaries and by constructing a stochastic differential equation model. We derive a notion of the coherence of a signal with respect to a dict...
Quantized Overcomplete Expansions in R^N: Analysis, Synthesis, and Algorithms
 IEEE TRANSACTIONS ON INFORMATION THEORY
, 1998
"... Coefficient quantization has peculiar qualitative effects on representations of vectors in IR with respect to overcomplete sets of vectors. These effects are investigated in two settings: frame expansions (representations obtained by forming inner products with each element of the set) and matchi ..."
Abstract

Cited by 94 (15 self)
 Add to MetaCart
Coefficient quantization has peculiar qualitative effects on representations of vectors in IR with respect to overcomplete sets of vectors. These effects are investigated in two settings: frame expansions (representations obtained by forming inner products with each element of the set) and matching pursuit expansions (approximations obtained by greedily forming linear combinations). In both cases, based on the concept of consistency, it is shown that traditional linear reconstruction methods are suboptimal, and better consistent reconstruction algorithms are given. The proposed consistent reconstruction algorithms were in each case implemented, and experimental results are included. For frame expansions, results are proven to bound distortion as a function of frame redundancy r and quantization step size for linear, consistent, and optimal reconstruction methods. Taken together, these suggest that optimal reconstruction methods will yield O(1=r ) meansquared error (MSE), and that consistency is sufficient to insure this asymptotic behavior. A result on the asymptotic tightness of random frames is also proven. Applicability of quantized matching pursuit to lossy vector compression is explored. Experiments demonstrate the likelihood that a linear reconstruction is inconsistent, the MSE reduction obtained with a nonlinear (consistent) reconstruction algorithm, and generally competitive performance at low bit rates.
1 Dictionary Learning for Sparse Approximations with the Majorization Method
"... Abstract—In order to find sparse approximations of signals, an appropriate generative model for the signal class has to be known. If the model is unknown, it can be adapted using a set of training samples. This paper presents a novel method for dictionary learning and extends the learning problem by ..."
Abstract

Cited by 50 (10 self)
 Add to MetaCart
(Show Context)
Abstract—In order to find sparse approximations of signals, an appropriate generative model for the signal class has to be known. If the model is unknown, it can be adapted using a set of training samples. This paper presents a novel method for dictionary learning and extends the learning problem by introducing different constraints on the dictionary. The convergence of the proposed method to a fixed point is guaranteed, unless the accumulation points form a continuum. This holds for different sparsity measures. The majorization method is an optimization method that substitutes the original objective function with a surrogate function that is updated in each optimization step. This method has been used successfully in sparse approximation and statistical estimation (e.g. Expectation Maximization (EM)) problems. This paper shows that the majorization method can be used for the dictionary learning problem too. The proposed method is compared with other methods on both synthetic and real data and different constraints on the dictionary are compared. Simulations show the advantages of the proposed method over other currently available dictionary learning methods not only in terms of average performance but also in terms of computation time.
Matching Pursuit and Atomic Signal Models Based on Recursive Filter Banks
 IEEE Transactions on Signal Processing
, 1902
"... The matching pursuit algorithm can be used to derive signal decompositions in terms of the elements of a dictionary of timefrequency atoms. Using a structured overcomplete dictionary yields a signal model that is both parametric and signaladaptive. In this paper, we apply matching pursuit to the d ..."
Abstract

Cited by 47 (1 self)
 Add to MetaCart
(Show Context)
The matching pursuit algorithm can be used to derive signal decompositions in terms of the elements of a dictionary of timefrequency atoms. Using a structured overcomplete dictionary yields a signal model that is both parametric and signaladaptive. In this paper, we apply matching pursuit to the derivation of signal expansions based on damped sinusoids. It is shown that expansions in terms of complex damped sinusoids can be efficiently derived using simple recursive filter banks. We discuss a subspace extension of the pursuit algorithm which provides a framework for deriving realvalued expansions of real signals based on such complex atoms. Furthermore, we consider symmetric and asymmetric twosided atoms constructed from underlying onesided damped sinusoids. The primary concern is the application of this approach to the modeling of signals with transient behavior such as music; it is shown that timefrequency atoms based on damped sinusoids are more suitable for representing trans...
Wavelet Footprints: Theory, Algorithms, and Applications
 IEEE Trans. Signal Processing
, 2003
"... In recent years, waveletbased algorithms have been successful in different signal processing tasks. The wavelet transform is a powerful tool because it manages to represent both transient and stationary behaviors of a signal with few transform coefficients. Discontinuities often carry relevant sign ..."
Abstract

Cited by 46 (5 self)
 Add to MetaCart
(Show Context)
In recent years, waveletbased algorithms have been successful in different signal processing tasks. The wavelet transform is a powerful tool because it manages to represent both transient and stationary behaviors of a signal with few transform coefficients. Discontinuities often carry relevant signal information, and therefore, they represent a critical part to analyze. In this paper, we study the dependency across scales of the wavelet coefficients generated by discontinuities. We start by showing that any piecewise smooth signal can be expressed as a sum of a piecewise polynomial signal and a uniformly smooth residual (see Theorem 1 in Section II). We then introduce the notion of footprints, which are scale space vectors that model discontinuities in piecewise polynomial signals exactly. We show that footprints form an overcomplete dictionary and develop efficient and robust algorithms to find the exact representation of a piecewise polynomial function in terms of footprints. This also leads to efficient approximation of piecewise smooth functions. Finally, we focus on applications and show that algorithms based on footprints outperform standard wavelet methods in different applications such as denoising, compression, and (nonblind) deconvolution. In the case of compression, we also prove that at high rates, footprintbased algorithms attain optimal performance (see Theorem 3 in Section V).
Denoising by Sparse Approximation: Error Bounds Based on RateDistortion Theory
, 2006
"... If a signal x is known to have a sparse representation with respect to a frame, it can be estimated from a noisecorrupted observation y by finding the best sparse approximation to y. Removing noise in this manner depends on the frame efficiently representing the signal while it inefficiently repres ..."
Abstract

Cited by 45 (7 self)
 Add to MetaCart
If a signal x is known to have a sparse representation with respect to a frame, it can be estimated from a noisecorrupted observation y by finding the best sparse approximation to y. Removing noise in this manner depends on the frame efficiently representing the signal while it inefficiently represents the noise. The meansquared error (MSE) of this denoising scheme and the probability that the estimate has the same sparsity pattern as the original signal are analyzed. First an MSE bound that depends on a new bound on approximating a Gaussian signal as a linear combination of elements of an overcomplete dictionary is given. Further analyses are for dictionaries generated randomly according to a sphericallysymmetric distribution and signals expressible with single dictionary elements. Easilycomputed approximations for the probability of selecting the correct dictionary element and the MSE are given. Asymptotic expressions reveal a critical input signaltonoise ratio for signal recovery.
Sparse and structured decompositions of signals with the molecular matching pursuit
 IEEE Transactions on Speech and Audio Processing
"... algorithm for the decomposition of signals. The MMP is a practical solution which introduces the notion of structures within the framework of sparse overcomplete representations; these structures are based on the local dependency of significant timefrequency or timescale atoms. We show that thi ..."
Abstract

Cited by 34 (5 self)
 Add to MetaCart
(Show Context)
algorithm for the decomposition of signals. The MMP is a practical solution which introduces the notion of structures within the framework of sparse overcomplete representations; these structures are based on the local dependency of significant timefrequency or timescale atoms. We show that this algorithm is well adapted to the representation of real signals such as percussive audio signals. This is at the cost of a slight suboptimality in terms of the rate of convergence for the approximation error, but the benefits are numerous, most notably a significant reduction in the computational cost, which facilitates the processing of long signals. Results show that this algorithm is very promising for highquality adaptive coding of audio signals. Index Terms—Matching pursuit, overcomplete representations, parametric audio coding, timefrequency transforms. I.
Matching Pursuit With Damped Sinusoids
 In Proc. ICASSP
, 1997
"... The matching pursuit algorithm derives an expansion of a signal in terms of the elements of a large dictionary of timefrequency atoms. This paper considers the use of matching pursuit for computing signal expansions in terms of damped sinusoids. First, expansion based on complex damped sinusoids is ..."
Abstract

Cited by 31 (3 self)
 Add to MetaCart
The matching pursuit algorithm derives an expansion of a signal in terms of the elements of a large dictionary of timefrequency atoms. This paper considers the use of matching pursuit for computing signal expansions in terms of damped sinusoids. First, expansion based on complex damped sinusoids is explored; it is shown that the expansion can be efficiently derived using the FFT and simple recursive filterbanks. Then, the approach is extended to provide decompositions in terms of real damped sinusoids. This extension relies on generalizing the matching pursuit algorithm to derive expansions with respect to dictionary subspaces; of specific interest is the subspace spanned by a complex atom and its conjugate. Developing this particular case leads to a framework for deriving realvalued expansions of real signals using complex atoms. Applications of the damped sinusoidal decomposition include system identification, spectral estimation, and signal modeling for coding and analysismodifi...