Stability Bounds on Step-Size for the Partial Update

STABILITY BOUNDS ON STEP-SIZE FOR THE PARTIAL UPDATE LMS ALGORITHMMahesh Godavarti and Alfred O.Hero IIIDepartment of Electrical Engineering&Computer ScienceThe University of MichiganAnn Arbor,MI48105mgodavar@,hero@ABSTRACTPartial updating of LMSfilter coefficients is an effective methodfor reducing the computational load and the power consumptionin adaptivefilter implementations.Only in the recent past has anywork been done on deriving conditions forfilter stability,conver-gence rate,and steady state error for the Partial Update LMS al-gorithm.In[5]approximate bounds were derived on the step sizeparameter which ensure stability in-the-mean of the alternatingeven/odd index coefficient updating strategy.Unfortunately,dueto the restrictiveness of the assumptions,these bounds are unre-liable when fast convergence(large)is desired.In this paper,tighter bounds on are derived which guarantee convergence in-the-mean of the coefficient sequence for the case of wide sensestationary signals.1.INTRODUCTIONPartial updating of the LMS adaptivefilter has been proposed to re-duce computational costs[2,3,4].In this era of mobile computingand communications,such implementations are also attractive forreducing power consumption.However,theoretical performancepredictions on convergence rate and steady state tracking error aremore difficult to derive than for standard full update LMS.Accu-rate theoretical predictions are important as it has been observedthat the standard LMS conditions on the step size parameter fail toensure convergence of the partial update algorithm.The two types of partial update LMS algorithms that are preva-lent in the literature have been described in[5].They are referredto as the“Periodic LMS algorithm”and the“Sequential LMS al-gorithm”.An attempt was made to recalculate the bounds on thestep-size parameter for both mean and mean-square convergence.Due to simplifying assumptions,the bounds derived turned outto be the same as those for the standard LMS algorithm.It wasshown,however,that these bounds fail to predict situations wherethe Sequential LMS algorithm is unstable when implemented withthese standard step-size constraints.In this paper we derive bounds on the step-size parameter whichensures convergence in mean for the special case involving alter-nate even and odd coefficient updates.The bounds are based onand and taking expectations of the update equations we obtain(4)It can be easily verified that for and the necessary and sufficient condition on for stability of the recursion(4)is given by(6)As for,we have that so that if the upper bound in condition(6)is used to set in partial update LMS,divergence occurs.3.ALGORITHM DESCRIPTION AND ANALYSISIt is assumed that thefilter is a standard FIRfilter of even length,.For convenience,we start with some definitions.Let be the input sequence and let denote the coefficients of the adaptivefilter.Definewhere the terms defined above are for the instant.In addition, Let denote the desired response.In typical applications isa known training signal which is transmitted over a noisy channel with unknown FIR transfer function.In this paper we assume that itself obeys an FIR model given by where are the coefficients ofan FIR model given by.Hereis assumed to be a zero mean i.i.d sequence that is independent of the input sequence.This is a standard assumption used in the analysis of the standard LMS algorithm[1]which can be shown tobe reasonable for jointly stationary and.The coefficient updates for odd in the partial update LMS algorithm considered here are given by(7) and for even(8) where is the error and is defined to beWe also define coefficient error vectors aswhereAssuming that is a WSS random sequence,we analyse the convergence of the mean coefficient error vector.For regular LMS algorithm the recursion for is given by(9) where is the-dimensional identity matrix andis the input signal correlation matrix.The necessary and sufficient condition for stability of the recursion is given by(10) where is the maximum eigen-value of the input signal cor-relation matrix.For odd,combining the even and odd update equations and writing them in terms of,we obtain(11) where the elements of areWe next make the standard assumptions that and are mutually uncorrelated and that is independent of[1]. These assumptions are somewhat restrictive but greatly simplifythe analysis.Taking expectations,using the independence assump-tion on the sequences,the mutual independence assump-tion on and,and simplifying we obtain(12) where(13) and,,, and.Under the assumption of even integer and real w.s.s.it can be shown that.For even,combining the even and odd update equations and writing them in terms of,we obtain(14)where the elements of areTaking expectations,and using the same assumptions as above, we obtain(15) where(16)It can be shown that under the above assumptions onand,the convergence conditions for even and odd update equa-tions are identical.We therefore focus on(12).If we want to write the update equations for the regular LMS algorithm in the same form as(12)we would have(17) which is the same as(9)only expressed in a different form.It should be noted here that even thoughthe matrix on the right is the correlation matrix for a permuted formof the input signal and therefore is also an input signal correlation matrix with the same eigenvalues as.Now to ensure stability of(12),the eigenvalues ofshould lie inside the unit circle.To estimate the eigenvalues of we employ the Bauer-Fike theorem[6,p.321]which states that if is an eigenvalue of andthen(18)where denotes any of the-norms and. For convenience,we will choose.Now writing as wherewe have(19)that is so because on account of being an Hermitian matrix which admits a matrix of orthogonal eigenvectors.Now can be written as where(20)(21)Using the properties of the matrix norm[6,pp.56-57]we obtain(22) where is the largest eigenvalue of the matrix which is the correlation matrix of the permuted input signal.If we let,which for w.s.s.can be rewritten as(31)which is the sufficient condition ensuring convergence of(12)and is the main result of this paper.This condition when applied to the motivating example gives us the bound on:which satisfies(5). It should be noticed that as the signal becomes more correlated making the bound tighter.0.050.10.150.20.250.30.350.40.451002003004005006007008009001000C o e f f i c i e n t V a l u eNumber of IterationsCoefficient 1Coefficient 2Figure 1:Trajectory ofandfor-8e+07-6e+07-4e+07-2e+072e+074e+076e+078e+07010*******4005006007008009001000C o e f f i c i e n t V a l u eNumber of IterationsCoefficient 1Coefficient 2Figure 2:Trajectory ofandfor4.SIMULATIONSWe have plotted the evolution trajectory of the 2-tap filter consid-ered in Section 2for and in Figures 1and 2.For Figure 1was chosen according to condition (31)and for Figure 2was chosen according to (10)which is the con-dition given in [5]for convergence in-the-mean.For simulationpurposes we setwhere is a vector composed of the w.s.s.AR process with varianceequal to 1and AR coefficient,and is a white se-quence,with variance equal to ,independent of .We setwhere is a white sequence,with vari-ance equal to,independent of .As can be seen from Figure 2stricter conditions are needed for convergence in mean than those given by (10).5.CONCLUSIONWe have analyzed the alternating odd/even partial update LMS al-gorithm and we have derived stability bounds on step-size param-eter for wide sense stationary signals based on extremal prop-erties of the matrix -norm.While these may not be the weakest possible bounds,they do provide the user with a useful sufficientcondition on which ensures convergence in the mean.The anal-ysis also leads directly to an estimate of mean convergence rate.Mean-square convergence analysis was not undertaken in this pa-per as the primary motivation was to show that current bounds on step-size are not sufficient to guarantee convergence.Theoretical analysis in the manner considered here for the general case of “Se-quential LMS Algorithm”is more complicated but feasible.6.REFERENCES[1]S.T.Alexander,Adaptive signal processing ,Springer-Verlag,New York NY ,1986.[2]M.J.Gingell,B.G.Hay,and L.D.Humphrey,“A BlockMode Update Echo Canceller Using Custom LSI,”GLOBE-COM Conference Record,V ol.3,pp.1394-1397,Nov.1983.[3]M.Godavarti,“Implementation of a G.165Line Echo Can-celler on Texas Instruments’TMS320C3x and TMS320C54x chips,”ICSPAT Conference Proceedings,pp.65-69,Sep.1997.[4] D.Messerschmitt,D.Hedberg,C.Cole,A.Haoui and P.Winship,“Digital V oice Echo Canceller with a TMS32020,”Application report:SPRA129,Texas Instruments.[5]S.C.Douglas,“Adaptive Filers Employing Partial Updates,”IEEE Trans.Circuits and Systems-II:Analog and Digital Signal Processing,vol.44,pp.209-216,March 1997.[6]G.H.Golub and C.F.Van Loan,“Matrix Computations,”3edition,The Johns Hopkins University Press,1996.。