A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. 33, pp. 222--230, Feb. 1985.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....that QRQ = #. It is easy to obtain the following necessary and su#cient conditions (see Appendix A.1) for convergence of the SPU LMS algorithm (3.7) i=1 1 which is independent of P and identical to that of LMS. We use the integrated MSE di#erence J = k=0 [# k ] introduced in [22] as a measure of the convergence rate and M( as a measure of misadjustment. The misadjustment factor is simply (see Appendix A.3) M( 3.8) which is the same as that of the standard LMS. Thus, we conclude that random update of subsets has no e#ect on the final excess mean squared ....

....used to determine the stability of G i (z) and not G k (z) G i (0) that was used in [46] Following the rest of the procedure as outlined in [46] exactly, we obtain the conditions for stability to be (3.7) A.2 Derivation of expression (3. 9) Here we follow the procedure in [22]. Assuming G k converges we have the expression for G# to be G# = P [2# ] 1 1# min . Then we have G k 1 = F (G k where F = I . Since # k = tr G k (# k ) tr (G k = tr (G 0 = tr (I F ) 1 (G 0 from which (3.9) follows. 137 ....

A. Feuer and E. Weinstein, "Convergence analysis of lms filter with uncorrelated gaussian data," IEEE Trans. Acoust., Speech, and Sig. Proc., vol. ASSP-33, no. 1, pp. 222--229, Feb. 1985.

....necessary and sufficient conditions 29 (see Appendix .1) for convergence of the SPU LMS algorithm 0 2 max (3.6) j( def = P N i=1 i 2 Gamma i 1 which is independent of P and identical to that of LMS. We used the integrated MSE difference J = P 1 k=0 [ k Gamma 1 ] introduced in [20] as a measure of the convergence rate and M( 1 Gamma min min as a measure of misadjustment. The misadjustment factor is simply (see Appendix .3) M( j( 1 Gamma j( 3.7) which is same as that of the standard LMS. Thus, we conclude that random update of subsets has no effect on the ....

.... G k (z) i G i (0) 1 Gamma z Gamma1 (1 Gamma 2 P i 3 2 P 2 i ) 109 that was used in [37] Following the rest of the procedure as outlined in [37] exactly, we obtain the conditions for stability to be (3.6) 2 Derivation of expression (3. 8) Here we follow the procedure in [20]. Assuming G k converges we have the expression for G1 to be G1 = P [2 Gamma 2 2 Gamma 2 2 11 T ] Gamma1 2 P 2 1 min Then we have G k 1 Gamma G1 = F (G k Gamma G1 ) where F = I Gamma 2 P 2 2 P 2 P 2 11 T . Since k = trfG k g we have 1 X k=0 ( k ....

A. Feuer and E. Weinstein, "Convergence analysis of lms filter with uncorrelated gaussian data," IEEE Trans. Acoust., Speech, and Sig. Proc., vol. ASSP-33, no. 1, pp. 222--229, Feb. 1985.

....on the data and the model are imposed in order to make the derivation and the results more tractable. The most important of these simplifying conditions are known collectively as the independence assumptions, which appear in many of the earlier works on LMS and in several recent ones (see, e.g. [1, 4, 5, 6]) Basically, one assumes the following. I 1. The zero mean sequences Phi y(k) x k Psi are related via a linear model of the form y(k) x T k w v(k) 3) for some unknown w , and where v(k) is zero mean with variance oe 2 v . In addition, it is assumed that the sequences Phi x ....

A. Feuer and E. Weinstein. Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-33(1):222--229, February 1985.

.... sufficiently small stepsizes between the ensemble average learning curve and the actual learning curve, it is now common in the literature to use the average of few independent experiments to predict or confirm theoretical results from simulation results (a few relatively recent examples include [2, 9], which use 10 20 independent experiments, and [7, 8] which use 100 independent experiments) In this paper, we show by examples and also analytically that for larger step sizes, it may be necessary to perform a considerably larger number of experiments to correctly approximate the average E ....

A. Feuer and E. Weinstein. Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Transactions on Acoustics, Speech, and Signal Processing, ASSP-33(1):222-- 229, February 1985.

....curve (we shall also give an analytical justification for this fact later in Thm. 2) It is thus common in the literature to use the average of a few independent repeated experiments to predict or confirm theoretical results by means of simulations (a few relatively recent examples include [3], 25] which use 10 20 independent experiments, and [24] 27] 28] which use 100 independent experiments) But what about larger step sizes Will EALCs still provide good approximations for the actual learning curves C. Examples Consider a length M = 10 LMS adaptive filter operating with ....

....= 0:08, and L = 100. Consider now the same LMS filter of length M = 10 but with the larger step size = 0:16 (and noise variance oe 2 v = 10 Gamma4 ) Since the independence assumptions are satisfied in this case, it is possible to compute the learning curve E e(n) 2 exactly, as follows [3], 4] Let the input covariance matrix be R Delta = Exnx T n . Under the conditions of the example, R = I . Define further the weight error covariance matrix Cn Delta = E wn w T n . With these definitions, the MSD and MSE are given by E k wn k 2 = Tr Cn and E e(n) 2 = Tr Gamma ....

A. Feuer and E. Weinstein. Convergence analysis of LMS filters with uncorrelated Gaussian data. IEEE Trans. Acoust. Speech Signal Process., ASSP-33(1):222--229, February 1985.

....nature of the input data vector guarantees that X k will be correlated with X j for jj Gamma kj L. However, many have observed that analyses using this assumption are reasonably accurate in predicting the statistical behavior of the LMS algorithm when the step size is a small value [1, 2, 3, 4, 5, 6, 7]. In a discussion of this assumption, Mazo has shown that the analysis using the independence assumption captures the first order behavior of the algorithm, thus validating this claim [2] However, it is not clear how small the step size must be to guarantee accuracy of the independence assumption ....

.... shown to be [5] k 1 = 1 Gamma 2fl 2 2 f(L Gamma 1)fl 2 2 fl 4 g) k 2 Loe 2 n fl 2 2 ; 10) where fl i is the ith moment of the input data, defined as fl i = E[x i k ] 11) For correlated Gaussian input data, the update for the weight error covariance matrix is given by [6] E[V k 1 V T k 1 ] E[V k V T k ] Gamma (RE[V k V T k ] E[V k V T k ]R) 2 (2RE[V k V T k ]R Rtr[RE[V k V T k ] 2 oe 2 n R: 12) From this equation, the excess MSE can be computed from Equation (9) 2.3 Derivation of the Exact Updates We describe a method for ....

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-33, no. 1, pp. 222-230, February 1985.

....assumption [6] is widely used in tS convergence analysis of 1 D LM fort wo main reasons. Thefirst is duet te simplificat6# in analysis obtysis under suchassumptCj9 The second is due t te good agreement bet ween tn analytFS; result obtlt usingti independenceassumptGF and experimenti result [5] [9]. Thoughto 2 D M E analysis will be significantg simplified when invokingti independenceassumpt j ts use of 2 D indexing scheme in tS weightg updat equat#5 of tS 2 D LMresult in a new problemtob isnot encountcou in tS 1 D case. Fortj 1 D LM , as well as for tr TDLM ,tj adaptG efilt5 9 weight ....

....is tSadapt5 e filtEF5 weightg ectSE is a scalarparametG tra contn S tn convergencerat of tS LMSalgoritG# and G j is tS inst9 tst9j gradient oftj MSE at itF6 atF6 j. From Eq. 1) it followstSj tS weightghSGF covariancematnc iscalculatC by aset of 1 Dfirst order di#erenceequatceSE Accordingt [8] [9], t] set of di#erence equatnce maintinS stnSjFj y under a general conditlS imposed on tS used std sizeparametF . FortE 2 D LMS, however, t,adaptC efilt 6 weight vect# updat equat9C is described bytj 2 Dfirst 458 IEICE TRANS. FUNDAMENTALS, VOL.E82 A, NO.3 MARCH 1999 order di#erence equatce ....

M.A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust.,Sp eech & Signal Process., vol.ASSP-33,pl.ASS 230, Feb. 1985.

....noise case can be similarly handled. We also assume that ffl the data matrix Xk is independent of X j for j 6= k, and ffl the noise vector Nk is independent of N j for j 6= k. These two assumptions are one form of the independence assumption commonly used in adaptive filtering analyses [6, 7]. These assumptions are also less restrictive than those used by Feuer in his analysis of the block LMS algorithm [5] With these assumptions, it can be shown that the current weight vector Wk is independent of the current data matrix and noise vector, a result that simplifies the analysis ....

.... N = 1 and RX = R (1;1) this analysis reduces to E[V k 1 V T k 1 ] E[V k V T k ] Gamma Gamma RXE[V k V T k ] E[V k V T k ]RX Delta 2 oe 2 n RX 2 (2RXE[V k V T k ]RX RX tr[RXE[V k V T k ] 12) a well known result for the LMS algorithm with Gaussian inputs [7]. Comparing (12) with the corresponding meansquare analysis in (7) 11) we note some significant differences between the single and multiple error cases. In particular, the LMS algorithm s mean square convergence equation in (12) can be diagonalized using the Hermitian whitening matrix Q ....

[Article contains additional citation context not shown here]

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-33, no. 1, pp. 222-230, February 1985.

....in equation (36) of [1] by a minus sign. 1 2 Analysis 2. 1 Evolution of the Filter Coefficients The analysis of the algorithm in (1) 2) for jointly Gaussian input and desired response signals follows other analyses of stochastic gradient adaptive algorithms when operating on these signals [4, 5, 6, 7]. A key difference in our analysis is the additional assumption that the coefficient vector W k is Gaussian distributed. For the LMS algorithm with Gaussian inputs, this assumption has been shown to be valid for vanishingly small step sizes [8] and it has been successfully used to analyze other ....

.... and taking expectations, we get E[W k 1 W T k 1 ] E[A k W k W T k A T k ] E[G k X k d 2 k X T k G T k ] E[A k W k d k X T k G T k ] E[G k W k d k X T k A T k ] 8) 2 By employing the standard independence assumptions, it can be shown through the results in [4] that the first term on the right hand side of (8) is given by E[A k W k W T k A T k ] E[W k W T k ] Gamma (RXXE[W k W T k ] E[W k W T k ]RXX ) 2 (2RXXE[W k W T k ]RXX RXX tr[R XXE[W k W T k ] 2oe 2 d 3oe 4 d )E[W k W T k ] Gamma 2 (CXdX E[W k W T k ....

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-33, no. 1, pp. 222-230, February 1985.

.... 4 = Ef(a Delta (k) Gamma ff) 2 g ff oe 2 s ( a Delta (0) Gamma ff) 2 Gamma ff oe 2 s ) 1 Gamma 4 ff oe 2 s ) k (12) provided that 0 ff 1 2oe 2 s (13) Equation (13) is a more stringent bound for ff since the mean square weight error is ensured to be finite [11]. The variance of the gain estimate, var(ff) which equals the steady state value of ffl ff (k) is thus given by var(ff) ff oe 2 s (14) It is seen that the value of var(ff) increases with the step size, ff , and the signal power, oe 2 s . III. Extension to Real Valued Delay When Delta ....

A.Feuer and E.Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol.33, pp.222-230, Feb. 1985.

....have been proposed, it is still one of the most efficient algorithms and, due to its simplicity, in widespread use. Current textbooks on adaptive filtering [3,5,10] deal extensively or even exclusively [13] with the LMS algorithm, and many recent publications are still devoted to its analysis [1,6,9,16]. Despite this huge material, a convincing elementary theory of the LMS algorithm is still lacking, and teaching of the discipline has not yet reached the wanted maturity. Current treatments contain concepts, like the independence assumption , which cannot be made plausible [2] and important ....

Feuer, A. and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data". IEEE Trans. Acoustics, Speech, and Signal Processing, vol. ASSP-33 (1985), p. 222-230.

....based on the LMS algorithm, such as the algorithm (22) is the selection of the step size parameter . Although in theory sufficient conditions on exist that guarantee the convergence of the LMS algorithm, these conditions depend on the knowledge of the eigenvalues of the correlation matrix R [2, 35, 36]. In the case of the LMS adaptive L filter (22) R is the correlation matrix of the ordered input vector. For example, the necessary and sufficient condition for the average L filter coefficient vector E [ a(k) to be convergent is 0 2 max (23) where max denotes the maximal eigenvalue ....

.... 2 max (23) where max denotes the maximal eigenvalue of matrix R [2] Furthermore, should satisfy the following more strict condition 0 2 3 tr[R] 2 3 Theta total input power (24) in order to achieve convergence of the average Mean Squared Error E [J(k) to a steady state value [36]. In (24) tr[ Delta] stands for the trace of the matrix inside brackets. In addition, when the adaptive filter is going to operate in a nonstationary environment (as is in image processing) inequalities (23) and (24) turn to be useless, since the correlation matrix R is time space varying. In ....

A. Feuer, and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoustics, Speech and Signal Processing, vol. ASSP-33, no. 1, pp. 222--230, February 1985.

....on the performance of the LMS algorithm, and the influence of the various components involved. It was found that the choice of determines the influence of the initialization, the steady state noise and the speed of convergence, all of which resembles the results on the classic LMS theory [21, 22, 35, 45]. The matrices H(t) and G(t; 1) are also influencing the convergence behavior of the algorithm, and even for the simple case of linear time invariant blur and no motion there are limitations emerging from the ill posedness of the model equations. C SG Block Circulant Matrix Diagonalization A ....

A. Feuer and E. Weinstein, "Convergence analysis of lms filters with uncorrelated gaussian data," IEEE Trans. Acoustics, Speech and Signal Processing, vol. 33, pp. 222--230, February 1985.

No context found.

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. 33, pp. 222--230, Feb. 1985.

No context found.

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-33, pp. 222--229, 1985.

No context found.

A. Feuer and E. Weinstein, " Convergence Analysis of LMS Filters with Uncorrelated Gaussian Data," IEEE transactions on Acoustics, Speech and Signal Processing, vol. ASSP-33, no. 1, pp. 222-230, February 1985.

No context found.

A. Feuer and E. Weinstein. Convergence analysis of LMS filters with uncorrelated Gaussian data, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 33, no. 2, pp. 222--229, 1985.

No context found.

A. Feuer and E. Weinstein, "Convergence analysis of LMS filters with uncorrelated Gaussian data," IEEE Trans. Acoust. Speech and Signal Processing, vol. ASSP--33, no. 1, pp. 222--230, Feb. 1985.

No context found.

A. Feuer and E. Weinstein, "Convergence Analysis of LMS Filters with Uncorrelated Gaussian Data", IEEE Trans. ASSP, Vol. 33, pp. 222-230, 1985.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC