L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression," The Annals of Statistics, Vol. 15, No. 2, pp. 880--882.  Home/Search   Document Not in Database   Summary   Related Articles   Check

.... jjR : 3.8) The residue R f is the approximation error of f after choosing n vectors in the dictionary and the energy of this error is given by (3. 8) In infinite dimensional spaces, the convergence of the error to zero is shown ( 10] to be a consequence of a theorem proved by Jones ([7]) lim m 1 m f jj = 0: 3.9) Hence f; g fl n g fl n ; 3.10) and we obtain an energy conservation f; g fl n j : 3.11) In finite dimensional signal spaces, the convergence is proved to be exponential ( 10] x4 Fast calculation Despite the apparent brute force ....

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, No. 2, pp. 880-882, 1987.

....algorithm are possible. An example is given by the weak greedy algorithm [16] which consists in modifying the atom selection rule by allowing to choose a slightly suboptimal candidate: #R m , g #m # t m sup g#D #R m , g# , t m # 1 . One can easily show that Matching Pursuit converges [17] and even converges exponentially in the strong topology in finite dimension, see [15] for a proof. Unfortunately this is not true in general in infinite dimension, even though this property holds for particular dictionaries [18] However, DeVore and Temlyakov [16] constructed a dictionary for ....

Jones L., "On a conjecture of huber concerning the convergence of projection pursuit regression," The Annals of Statistics, vol. 15, pp. 880--882, 1987.

....that, at each iteration, searches for the best atom in the dictionary. The optimality factor # is set to one when MP The decay parameter # decreases when the size of the signal space increases. However, at the limit of infinite dimensional spaces, the convergence is not exponential any more [11]. browses the complete dictionary at each iteration. The parameter # depends on the dictionary construction. It represents the ability of the dictionary functions to capture features of any input function f . The upper bound on the coefficient norm is reached in the worst case where the input ....

Jones L.K., "On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression," The Annals fo Statistics, vol. 15, no. 2, pp. 880--882, 1987.

....chosen to best match its residues. Although this decomposition is non linear, we maintain an energy conservation as though it were a linear, orthogonal decomposition. An important issue is to understand the behavior of the residue R f when m increases. By transposing a result proved by Jones [9] for projection pursuit algorithms [5] one can prove [10] that the matching pursuit algorithm converges, even in infinite dimensional spaces. 5 Theorem 2 Let f 2 H. The residue R f defined by the induction equation m 1 f jj = 0: 16) Hence : 18) When H is of finite ....

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, No. 2, p. 880-882, 1987.

....X d . This decomposition is obtained with a strategy similar to the matching pursuit approach. Readers further interested by projections pursuits are referred to a tutorial review written by Huber [10] The mathematical similarities of the two algorithms allow us to transpose a result of Jones [11] that proves the convergence of projection pursuit algorithms. Let us recall that V is the closed linear span of vectors in D. We denote by W the orthogonal complement of V in H. The orthogonal projectors over V and W are respectively written as Theorem 1 Let f 2 H. The residue R f de ned ....

....who helped us to develop the software. We are also grateful to Dave Donoho and Iain Johnstone for showing us the relations between this work and projection pursuit regressions. 31 Appendix A: Proof of Theorem 1 This appendix is a translation in the matching pursuit context of Jones s proof [11] for the convergence of projection pursuit regressions. Lemma 3 Let h n = R f; g n g n . For any n 0 and m 0, f j Proof: Since hm = R f; g m g m , f j = j R f; g m g m ; R f j = jjh m jj j g m ; R f j: 74) Equation (11) implies f j ....

L. K. Jones, \On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, No. 2, p. 880-882, 1987.

....examples of these successful methodologies were adaptively synthesized polynomial networks and projection pursuit. Barron and Barron [1] stated that an advantage of projection pursuit networks is that they have been amenable to theoretical examination of some of their approximation properties [4, 9, 16]. In particular, it is known that any square integrable function can be approximated by a theoretical analog of projection pursuit provided sufficiently many levels of the network are utilized. Intrator [14, 15] applied exploratory projection pursuit to do feature extraction in speech ....

L. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression," Annals Statist., vol. 15, pp. 880-882, 1987.

.... m 2 D is any satisfying jhf m 1 ; m ij t m sup g2D jhf m 1 ; gij; 2) f m : f m 1 hf m 1 ; m i m ; 3) G m (f; D) m X j=1 hf j 1 ; j i j : We note that in a particular case t k = t, k = 1; 2; this algorithm was considered in [J1]. We present convergence results and error estimates for PGA and WGA in Section 8. 7 Much less is known about greedy algorithms in the case of Banach space X. We discuss here two versions of generalization of PGA from Hilbert space H to Banach space X. The rst one is a straightforward ....

.... m and a function r m to minimize the error kf(x) m X j=1 r j ( j x)k L2 : This is one more example of Pure Greedy Algorithm. The Pure Greedy Algorithm and some other versions of greedy type algorithms have been intensively studied recently (see [B] DDGS] DMA] Du] DT2] DT3] H] [J1], J2] T14 24] In this section we discuss PGA and some its modi cations which make them more ready for implementation. We call this new type of greedy algorithms Weak Greedy Algorithms (see Introduction for de nitions of PGA and WGA) If H 0 is a nite dimensional subspace of H, we let PH0 ....

[Article contains additional citation context not shown here]

L. Jones, On a conjecture of Huber concerning the convergence of projection pursuit regression, The Annals of Statistics 15 (1987), 880-882.

....f . The upperbound on the coefficient norm is reached in the worst case where the input function is the farthest from 3 The decay parameter decreases when the size of the signal space increases. However, at the limit of infinite dimensional spaces, the convergence is not exponential any more [18]. FROSSARD, VANDERGHEYSNT AND KUNT : REDUNDANCY DRIVEN A POSTERIORI MP QUANTIZATION 9 any dictionary vector. Hence, is defined by : sup jhf; g n ij kfk: 22) This relationship is further developed in the next sections. B. New formulation of the redundancy factor The parameter can be ....

Jones L.K., "On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression," The Annals fo Statistics, vol. 15, no. 2, pp. 880--882, 1987.

....f . The upperbound on the coefficient norm is reached in the worst case where the input function is the farthest from # The decay parameter # decreases when the size of the signal space increases. However, at the limit of infinite dimensional spaces, the convergence is not exponential any more [18]. FROSSARD, VANDERGHEYSNT AND KUNT : REDUNDANCY DRIVEN A POSTERIORI MP QUANTIZATION 9 any dictionary vector. Hence, is definedby: ### ##f;g # ## # #f#: 22) This relationship is further developed in the next sections. B. New formulation of the redundancy factor The parameter can be ....

Jones L.K., "On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression," The Annals fo Statistics, vol. 15, no. 2, pp. 880--882, 1987.

.... approximation f i 1 by adding a term from the dictionary D Ridge which results in the largest decrease in approximation error; i.e. minimizes kf (f i 1 (hk; xi b) k L 2 over all choices of (k; b) It is known that when f 2 L 2 (D) with D a compact set, the greedy algorithm converges [15]; it is also known that for a relaxed variant of the greedy algorithm, the convergence rate can be controlled under certain assumptions [16, 1] There are unfortunately two problems with the conceptual basis of such results. First, they lack the constructive character which one ordinarily ....

L.K. Jones, On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Statist. 15 (1992), 880-882.

.... a term from the dictionary D Ridge which results in the largest decrease in approximation error; i.e. minimizes kf Gamma (f i Gamma1 ff Delta ae(hk; xi Gamma b) k L 2 over all choices of (k; ff; b) It is known that when f 2 L 2 (D) with D a compact set, the greedy algorithm converges [15]; it is also known that for a relaxed variant of the greedy algorithm, the convergence rate can be controlled under certain assumptions [16, 1] There are unfortunately two problems with the conceptual basis of such results. First, they lack the constructive character which one ordinarily ....

L.K. Jones, On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Statist. 15 (1992), 880--882.

....structural and computational viewpoints. We begin the section with a brief discussion of the theoretical properties of PPL. Then we carry out a detailed comparison of statistical performance via a simulation study. 13 4. 1 Theoretical Approximation Properties of PPLs At least two researchers [12, 19] have theoretically established the first level approximation properties of PPLs. In particular, they have shown that, with sufficiently many hidden units, any noise free square integrable function can be approximated arbitrarily well by a theoretical analog of PPLs. These kinds of results are ....

L.K. Jones. On a conjecture of Huber concerning the convergence of projection pursuit regression. The Annals of Statistics, Vol. 15, No. 2,880--882, 1987.

....k i ; R i fi k i R n f: 3.3) Hereafter we will denote h k i ; R i fi by ff i . CHAPTER 3. ADAPTIVE EXPANSIONS 21 3.2.2 Discussion Matching pursuit is similar to a class of algorithms used in statistics called projection pursuits. The proof of the convergence of projection pursuits given in [18] can be used to prove the convergence of matching pursuit in infinite dimensional spaces. In infinite dimensional spaces, the convergence can be quite slow. However, the convergence is exponential in finite dimensional spaces [7, x3:1] Since ff i is determined by projection, ff i k i R i 1 f ....

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression," The Annals of Statistics, Vol. 15, No. 2, pp. 880--882.

....after installation into the network, the convergence property of the constructive algorithm does not follow readily from the universal approximation capability of the network architecture. In some cases, convergence can still be shown to exist. For example, strong convergence of PPR is proved in [45], which states that if each new g n in (3) at stage n is given by the conditional expectation [42] g n (z) E(f Gamma f n Gamma1 ja T n X = z) and the projection direction a n is chosen as long as E(g n (a T n X) 2 ae sup b T b=1 E(g n (b T X) 2 ; where 0 ae 1 is fixed, ....

L.K. Jones. On a conjecture of Huber concerning the convergence of projection pursuit regression. The Annals of Statistics, 15(2):880--882, 1987.

....functions g k are determined from the data during training; w k represent the projection directions. Incidentally, this shows that continuous functions g k can be uniformly approximated by a sigmoidal MLP of one input. Therefore, the approximation capabilities of MLPs and PP are very similar [25, 63]. This architecture admits generalisations to several output variables [84] depending on whether the output share the common basis functions g k and, if not, whether the separate g k share common projection directions w k . ffl Generalised additive models: j = D; w k0 = 0; w k = e k ; v; fi) ....

L. K. Jones, On a conjecture of Huber concerning the convergence of projection pursuit regression, Annals of Statistics, 15 (1987), pp. 880--882.

....f r Gamma1 . It does not look for the best approximation from R r . The algorithm converges. That is, lim r 1 kf r k L 2 (IR n ;d ) 0 : However, because of the non linearity it is worth trying to mollify the demands of the algorithm while maintaining its convergence. In 1987, L. K. Jones [8] proved the following result. Let 0 ae 1 be fixed. Assume that at the rth step we have obtained g r (b r Delta x) where g r is optimal for b r (i.e. g r = gb r ) but b r is not quite an optimal direction. Assume, however, that we do know that kg r (b r Delta )k L ....

Jones, L. K., On a conjecture of Huber concerning the convergence of projection pursuit regression, Ann. Statist. 15 (1987), 880--882.

....of a network structure is a prerequisite for the convergence of its learning procedure. Attempts to solving the problem without considering these questions could be very time consuming if not fruitless. A. Known Result on Convergence of PPR The strong convergence of PPR has been proved in [37], which states that if each new g n in (1) at stage n is given by the conditional expectation [38] gn (z) E(f Gamma fn Gamma1 ja T n X = z) 6) and the projection direction an is chosen as long as E(gn (a T n X) 2 ae sup b T b=1 E(gn (b T X) 2 ; 5 The approximation error ....

L.K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, no. 2, pp. 880--882, 1987.

.... g fl k j 2 kR n fk 2 : 3:8) The residue R n f is the approximation error of f after choosing n vectors in the dictionary and the energy of this error is given by (3:8) For any f 2 H, the convergence of the error to zero is shown [58] to be a consequence of a theorem proved by Jones [36] lim n 1 kR n fk = 0: 3:9) Hence f = 1 X k=0 R k f; g fl k g fl k ; 3:10) kfk 2 = 1 X k=0 j R k f; g fl k j 2 : 3:11) In infinite dimensions, the convergence rate of this error can be extremely slow. In finite dimensions, let us prove that the convergence is ....

.... of the variables (X 1 ; Y 1 ) XK ; YK ) The algorithm works by iteratively projecting the function f onto a series of ridge functions, to obtain an expansion of the form f(x) 1 X j=1 g j (a T j x) 3:15) This projection pursuit algorithm was proved to converge strongly in [36], and from this result the proof of the convergence of the non orthogonal matching pursuit is derived. The projection pursuits differ significantly from matching pursuits in that the function f(x) is not known exactly, so its applicability and the numerical considerations for its implementation ....

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, No. 2, p. 880-882, 1987.

....X d . This decomposition is obtained with a strategy similar to the matching pursuit approach. Readers further interested by projections pursuits are referred to a tutorial review written by Huber [10] The mathematical similarities of the two algorithms allow us to transpose a result of Jones [11] that proves the convergence of projection pursuit algorithms. Let us recall that V is the closed linear span of vectors in D. We denote by W the orthogonal complement of V in H. The orthogonal projectors over V and W are respectively written as P V and PW . Theorem 1 Let f 2 H. The residue ....

....Orszag who helped us to develop the software. We are also grateful to Dave Donoho and Iain Johnstone for showing us the relations between this work and projection pursuit regressions. Appendix A: Proof of Theorem 1 This appendix is a translation in the matching pursuit context of Jones s proof [11] for the convergence of projection pursuit regressions. Lemma 3 Let h n = R n f; g fl n g fl n . For any n 0 and m 0, j hm ; R n f j 1 ff jjh m jj jjh n jj: 73) Proof: Since hm = R m f; g fl m g fl m , j hm ; R n f j = j R m f; g fl m g fl m ; R n f j = ....

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression", The Annals of Statistics, vol. 15, No. 2, p. 880-882, 1987.

No context found.

L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression," The Annals of Statistics, Vol. 15, No. 2, pp. 880--882.

No context found.

L. K. Jones, "On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression," The Ann. of Stat., vol. 15, No.2, pp. 880-882, 1987.

No context found.

Jones L.K., On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression, The Annals fo Statistics,vol. 15, no. 2, pp. 880882, 1987. ############ 165

No context found.

L. K. Jones, "On a Conjecture of Huber Concerning the Convergence of Projection Pursuit Regression," The Ann. of Stat., vol. 15, No.2, pp. 880-882, 1987.

No context found.

L. K. Jones, "On a conjecture of huber concerning the convergence of projection pursuit regression," Ann. Statist., vol. 15, no. 2, pp. 880--882, 1987.

No context found.

L.K. Jones. On a conjecture of Huber concerning the convergence of projection pursuit regression. Ann. Stat., 15(2):880--882, 1987.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC