G. Davis, S. Mallat and M. Avelaneda, Adaptive Greedy Approximations, Jour. of Constructive Approximation, vol. 13, No. 1, pp. 57-98, 1997  Home/Search   Document Details and Download   Summary   Related Articles   Check

....chances to find a function that closely fits the input signal, and thus capture most of its energy, grows with the dictionary size. The redundancy of the dictionary therefore leads the energy decay rate of the residual signal, which has been proven to be upper bounded by an exponential curve [7] [8] in MP decompositions. The contribution of each MP coefficient therefore clearly depends on its position within the signal representation. Based on the characterization of the energy decay curve, an exponentially bounded quantization (EUQ) scheme is proposed for the Matching Pursuit coefficients, ....

....speed directly depends on the dictionary, the decay rate of the residual energy can be bounded once the dictionary is known, even without a priori information about the input signal. The approximation error decay rate in Matching Pursuit has been shown to be bounded by an exponential [7] [8]. In other words, the decay of the residue norm is faster than an exponential decay curve whose rate depends on the dictionary only. From [7] there exists a decay parameter # 0 such that for all N 2 #N , 6) or, equivalently, m 1 2 # m f# , #m . 7) The decay rate can ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations, " Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....3 will describe the new shape descriptor we introduce, and the following shape recognition task will be described in section 4. 2. Describing images with Matching Pursuit The matching pursuit algorithm, first introduced for monodimensional signals by Mallat Zhang [9] is an iterative greedy [4] process that decomposes a function f in a Hilbert space H, using a redundant dictionary g ### of functions g # i usually called atoms. The goal of this section is not to describe MP in a complete manner, but rather to point out the basic concepts and the interesting properties of MP for ....

G. Davis, S. Mallat, and M. Avellaneda. Adaptive greedy approximations. Constructive Approximations, 1997. Springer-Verlag NY.

....at m th step a unit vector m and a function r m to minimize the error kf(x) m X j=1 r j ( j x)k L2 : This is one more example of Pure Greedy Algorithm. The Pure Greedy Algorithm and some other versions of greedy type algorithms have been intensively studied recently (see [B] DDGS] [DMA], Du] DT2] DT3] H] J1] J2] T14 24] In this section we discuss PGA and some its modi cations which make them more ready for implementation. We call this new type of greedy algorithms Weak Greedy Algorithms (see Introduction for de nitions of PGA and WGA) If H 0 is a nite ....

G. Davis, S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Constructive Approximation 13 (1997), 57-98. 55

....to its ability to extract the maximum signal energy in a few iterations. In other words, it corresponds to the decay rate of the residue and thus the coding efficiency of Matching Pursuit. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [2] [8]. From [9] there exists 0 and 0 such that for all m 0 : kR m 1 fk (1 2 2 ) 1 2 kR m fk ; 5) where 2 (0; 1] is an optimality factor. This factor depends on the algorithm that, at each iteration, searches for the best atom in the dictionary. The optimality factor is ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....to its ability to extract the maximum signal energy in a few iterations. In other words, it corresponds to the decay rate of the residue and thus the coding efficiency of the Matching Pursuit. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [1, 2]. From [3] there exists ### and ### such that for all # # # : ## ### ##### # # # # # # # # ## # ## # (5) where # # ### ## is an optimality factor. This factor depends on the algorithm that, at each iteration, searches for the best atom in the dictionary. The optimality factor # ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....to its ability to extract the maximum signal energy in a few iterations. In other words, it corresponds to the decay rate of the residue and thus the coding efficiency of Matching Pursuit. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [2] [8]. From [9] there exists ### and ### such that for all # # # : ## ### ##### # # # # # # # # ## # ## # (5) where # # ### ## is an optimality factor. This factor depends on the algorithm that, at each iteration, searches for the best atom in the dictionary. The optimality factor # is set ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....of the structural 4 redundancy of dictionaries is therefore proposed to quantify the compression properties of a dictionary. The redundancy factor leads the energy decay rate of the residual signal coded by Matching Pursuit, which has been proven to be upper bounded by an exponential curve [7 9]. In [10, 11] a redundancy formulation has been proposed in the context of frame expansion, but the same computation can not be applied to Matching Pursuit decomposition. The formulation proposed here is however general enough to be applied to any structured dictionary, and even in the context of ....

....speed depends directly on the dictionary set. The decay rate of the residual energy can thus be bounded once the dictionary is known, even without a priori information about the input signal. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [2, 9]. In other words, the decay of the residue norm is faster than an exponential decay curve whose rate depends on the dictionary only. From [7] there exists 0 such that for all N 0 kR N fk 2 N kfk ; 19) or, equivalently, kR m 1 fk 2 kR m fk ; 8m : 20) The decay rate 3 ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....to its ability to extract the maximum signal energy in a few iterations. In other words, it corresponds to the decay rate of the residue and thus the coding efficiency of the Matching Pursuit. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [1, 2]. From [3] there exists 0 and 0 such that for all m 0 : kR m 1 fk (1 2 2 ) 1 2 kR m fk ; 5) where 2 (0; 1] is an optimality factor. This factor depends on the algorithm that, at each iteration, searches for the best atom in the dictionary. The optimality factor is ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....of the structural 4 redundancy of dictionaries is therefore proposed to quantify the compression properties of a dictionary. The redundancy factor leads the energy decay rate of the residual signal coded by Matching Pursuit, which has been proven to be upper bounded by an exponential curve [7 9]. In [10, 11] a redundancy formulation has been proposed in the context of frame expansion, but the same computation can not be applied to Matching Pursuit decomposition. The formulation proposed here is however general enough to be applied to any structured dictionary, and even in the context of ....

....speed depends directly on the dictionary set. The decay rate of the residual energy can thus be bounded once the dictionary is known, even without a priori information about the input signal. The approximation error decay rate in Matching Pursuit have been shown to be bounded by an exponential [2, 9]. In other words, the decay of the residue norm is faster than an exponential decay curve whose rate depends on the dictionary only. From [7] there exists # such that for all N # # ## N f### # N #f# ; 19) or, equivalently, ## m## f### # ## m f# ; #m: 20) The decay rate 3 ....

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

....convergence rate) limit its applicability. For large and badly conditioned problems our technique will already produce some reasonable approximation before the conjugate gradient method has even nished its rst step. The above technique is a special case of a greedy algorithm as described in [1], 4] 2] 6] and [7] We use it here for solving a large linear system, but the analysis in section 3 shows that the notion of a dictionary is applicable here. Furthermore, it extends to cases with multiple instances of functions , or with radial basis functions of varying scale. We shall ....

....thin plate splines or multiquadrics at early stages, and one can go over to compactly supported functions when it comes to resolving local details. Numerical experiments in this direction are still to be carried out. The notion of a dictionary with respect to a greedy algorithm in the sense of [1], 4] 2] 6] and [7] applies here, and it is an interesting research area to pursue this connection further. 9 Numerical Experiments We start with a reproduction of the following Franke type function: f(x) 3 X j=0 a j exp( b j kx x j k 2 2 ) with the values j a j b j x j 0 1.0 0.1 ( ....

Davis, G., S. Mallat, and M. Avallaneda, Adaptive greedy approximations, Constr. Approx. 13 (1997) 737-785

....rate) limit its applicability. For large and badly conditioned problems our technique will already produce some reasonable approximation before the conjugate gradient method has even finished its first step. ffl The above technique is a special case of a greedy algorithm as described in [1], 6] 2] 8] and [9] We use it here for solving a large linear system, but the analysis in section 3 shows that the notion of a dictionary is applicable here. Furthermore, it extends to cases with multiple instances of functions Phi, or with radial basis functions of varying scale. We shall ....

....thin plate splines or multiquadrics at early stages, and one can go over to compactly supported functions when it comes to resolving local details. Numerical experiments in this direction are still to be carried out. The notion of a dictionary with respect to a greedy algorithm in the sense of [1], 6] 2] 8] and [9] applies here, and it is an interesting research area to pursue this connection further. 9. Numerical Experiments We start with a reproduction of the following Franke type function: f(x) 3 X j=0 a j exp(b j kx Gamma x j k 2 2 ) with the values j a j b j x j 0 ....

Davis, G., S. Mallat, and M. Avallaneda, Adaptive greedy approximations, Constr. Approx. 13 (1997) 737--785

....version of the algorithm is also introduced, and shown to converge) The new algorithm is illustrated by its application to the analysis of a vectocardiogram signal. EDICS Classifications: 1 MDIG, 2 SAMP. Corresponding author 2 I. Introduction The Matching Pursuit algorithm [10] 7] 12] [5] is an algorithm that has been used in such settings as signal and image processing when one has large amounts of information available from which one must select the most important features. This enables one to obtain a more easily analysed approximation to the given data. Versions of this ....

....the numerical example, illustrating the technique and highlighting the use of vectorial data (compared with the standard scalar data) in the analysis of cardiac frequency. II. The Matching Pursuit Algorithm A. The basic algorithm The standard Matching Pursuit algorithm, as given in [10] [5], allows one to construct approximations to an element of a Hilbert space H using vectors from a fixed set. In the notation of [12, Sec. 5.2] we are supplied with a dictionary of normalized vectors, fg ff : ff 2 Ag and an element f of H. For computational purposes, A is normally finite, but this ....

G. Davis, S. Mallat and M. Avellaneda, Adaptive greedy approximations, Constr. Approx. 13 (1997), 57--98.

....to a general three layer network # # # ## ## #### (3) with a dictionary of activation functions. Such networks can implement nonlinear : # ## approximations of desired i o functions ; in approximation theory and matching pursuit such # approximations have been of recent interest ([3,8 10]) 3. Neural complexity and information complexity It should be noted that the approximation (3) epitomizes the parallels between feedforward neural network theory and approximation theory which began to be noticed in the late eighties. Indeed, it shows effectively that feedforward neural nets ....

.... what is the complexity (number of hidden neurons ) of the smallest neural network which can approximate a given i o function # # #within error in a given error measure # This question has been studied in approximation theory ( 9,10] and wavelet and statistical theory (matching pursuit; see [3,8]) Results from these areas translate into interesting ones for neural networks with general computable activation functions The notion of information complexity used here# # (involving the number of function evaluations needed to approximate the i o function ) carries over# # # from computer ....

Davis, G., Stephane Mallat, and M. Avelaneda, Adaptive greedy approximations, J. Constructive Approximation 13 (1997), 57-98.

....suggests that kf n k exp( Gamman= CK) could be true here, but we have no proof. Such a result would be remarkable, since it has been empirically observed for time frequency dictionaries that the greedy algorithm works well in the beginning, and this cannot be explained by asymptotic results, [2]. The study of the performance on a clean linear combination of K atoms is the starting point for an analysis of the initial success. We will prove geometric convergence for K = 2. This simple case already shows that the Walsh dictionary is much better conditioned than a general dictionary. ....

Davis. G., S. Mallat, and M. Avellaneda, Adaptive greedy approximations, Constr. Approx. , (to appear)

No context found.

G. Davis, S. Mallat and M. Avelaneda, Adaptive Greedy Approximations, Jour. of Constructive Approximation, vol. 13, No. 1, pp. 57-98, 1997

No context found.

Geoff Davis, Stphane Mallat, and Marco Avellaneda. Adaptive greedy approximations. Journal of Constructive Approximation, 13:57--98, 1997.

No context found.

Davis G., Mallat S. and Avellaneda M., Adaptive Greedy Approximations, Journal of Constructive Approximations,vol. 13, pp. 5798, 1997.

No context found.

Geoff Davis, Stphane Mallat, and Marco Avellaneda. Adaptive greedy approximations. Journal of Constructive Approximation, 13:57--98, 1997.

No context found.

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

No context found.

G. Davis, S. Mallat, and M. Avellaneda, "Adaptive greedy approximations, " J. Constructive Approx., vol. 13, pp. 57--98, 1997.

No context found.

Davis G., Mallat S. and Avellaneda M., "Adaptive Greedy Approximations," Journal of Constructive Approximations, vol. 13, pp. 57--98, 1997.

No context found.

G. Davis, S. Mallat, and M. Avellaneda. Adaptive Greedy Approximations. Constructive Approximation, 13:57-98, 1997.

No context found.

G. Davis, S. Mallat, and M. Avellaneda. Adaptive Greedy Approximations. Constructive Approximation, 13:57-98, 1997.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC