328 pp. 328-336

Finite word-length effects in recursive least squares algorithms with application to adaptive equalization

F u y u n L I N G *

Dimi t r i s M A N O L A K I S **

J o h n G. P R O A K I S **

Abstract

In this paper we provide a summary of recent and new results on finite word length effects in recursive least squares adaptive algorithms. We define the numerical accuracy and numerical stability of adaptive recursive least squares algorithms and show that these two properties are related to each other, but are not equivalent. The numerical stability of adaptive recursive least squares algorithms is analyzed theoretically and the numerical accuracy with finite word length is investigated by computer simulation. It is shown that the conventional recursive least squares algorithm gives poor numerical accuracy when a short word length is used..4 new form of a recursive least squares lattice algorithm is presented which is more robust to round-off errors compared to the conventional form. Optimum scali, Tg of recursive least squares algorithms for fixed- point impl:mentation is also considered.

Key words : Adaptive algorithm, Rounding error, Least squares method, Recursivity, Accuracy, Stability, Numerical calc :lation, Dig'tal simulation, Equalization, Error analys!s.

EFFETS DUS A LA PRI~CISION FINIE

DANS LES ALGORITHMES DES M~)INDRES CARRIES RI~CURSIFS.

APPLICATION A L'I~GALISATION ADAPTATIVE

Analyse

On prdsente un rdsumd des nouveaux rdsultats concernant les effets de la prdcision finie dans les algorithmes adaptatifs des moindres carrds rdcursifs. Pour ces algorithmes, on dJfinit la p: d :ision numdrique, la stabilitd nu:ndrique et ron montre que ces deux propridt-~s sont lides sans dtre dquivalentes. La stabilitd m:m.~rique est analysde thdoriquement et la prdcision numdrique sur des mots de longueur finie est examinFe par si,nulation s:tr ordinateur. On montre que l'algo-

rithme rdcursif classique des moindres carrFs atteint une faible prdcision numdrique lorsque des mots de longueur faible sont utilisds. On prdsente une nouvelle forme d'algorithme des moindres carrds en treillis qui se montre plus rdsistant aux erreurs d'arrondi que l'algorithme classique. On considkre aussi une normali- sation optimale des algorithmes des moindres carrds rdcursifs pour une implantation en virgule fixe.

Mots el6s : Algorithme adaptatif, Erreur arrondi, M6thode moindre carr6, R6cursivit6, Pr6cision, Stabilit6, Calcul num6- rique, Simulation num&ique, Egalisation, Calcul erreur.

Contents

I. Introduction.

I I . Definition of numerical stability and accuracy of adaptive estimation algorithms.

I I I . A feedback system model and numerical stabi- lity of LS estimation algorithms.

IV. ,4 comparison of the numerical accuracy of some adaptive algorithms.

V. The numerical accuracy of the LS lattice algo- rithm and the role of error-feedback in improving the numerical accuracy of Ls adaptive algorithms.

VI. Adaptive algorithms using mixed precision arithmetic.

VII . Fixed point implementation of adaptive algo- rithms and the problem of dynamic range.

VII I . Concluding remarks.

References (17 ref.).

I. INTRODUCTION

M a n y a p p l i c a t i o n s o f a d a p t i v e e s t ima t ion a lgo- r i t hms involve r ea l - t ime s ignal p rocess ing us ing a smal l genera l p u r p o s e c o m p u t e r or specia l p u r p o s e ha rdware . In such cases, r e d u c i n g the w o r d l eng th

* Codex Corporation, Mansfield, MA, 02048. ** Departmzat of Electrical and Computer Engineering, Northeastern University, 360 Huntington Avenue, Boston, Massachusetts 02115 (617) 437-216 3, U3A.

ANN. TI~L~COMMUN., 41, n ~ 5-6, 1986 1/9

F. LING. -- FINITE WORD-LENGTH EFFECTS IN RECURSIVE LS ALGORITHMS 329

of the computations and/or using fixed point arith- metic are of particular importance. The time consu- med on the computat ions or the area used in a VLSI implementation of the special purpose hardware are proport ional to the word length used in the computations for addition and proportional to its square for multiplication.

The price paid in using a small word length and fixed-point arithmetic is a degradation in the per- formance of the adaptive algorithms due to round-off error. The degradation is primarily a result of two factors. First, when a short word length is used, the algorithm yields a larger output error than the case where infinite precision is used in the computations. We refer to this effect as the numerical accuracy of the adaptive algorithm. The numerical accuracies of different adaptive algorithms can be compared by measuring their estimation errors when the same word length is used in the computations. Second, the round-off error may accumulate and increase with time, until it destroys the normal operation of the algorithm. We define this effect as the numerical stability of the adaptive algorithm.

These two effects of round-off errors in adaptive algorithms are related to each other. However they are not identical nor are they equivalent. In other words, some algorithms which are known to be inaccurate may be stable if the round-off error is not large enough to affect their normal operation. On the other hand, an algori thm that becomes unstable in a long run, could yield accurate results for a short data record.

In this paper, we investigate the effects of round- off errors in adaptive algorithms. The numerical accuracy and the numerical stability of adaptive algorithms are defined, examined and compared. The problems associated with the implementation of adaptive algorithms using fixed-point arithmetic are also discussed. In particular, the effect of the round-off error on the accuracy of estimation of the reflection coefficients of the LS lattice algorithm is considered. Based on this analysis, we propose a new error feedback formula to estimate the coeffi- cients which greatly improves the numerical accuracy of the LS lattice algorithms and related algorithms.

in numerical analysis for LS estimation algorithms as the degree to which the accuracy of the estimates is affected by the computat ional error (e.g. [1], [2]). More precisely, for the same amount of round-off error, an algorithm that yields a smaller estimation error is considered to be more stable than an algo- r i thm which yields a larger estimation error. This definition has been used in previously published papers, such as [3], where results in numerical analysis, e.g., condition number theory, have been used to analyze the numerical properties of adaptive algo- rithms. In this paper, we define this property as the numerical accuracy of the LS adaptive algorithms, since it reflects the estimation accuracy of the algo- rithms.

There exists another type o f stability problem in adaptive algorithms. It occurs when adaptive algo- rithms operate continuously, particularly, in tracking time-varying signal characteristics. Since in such cases the number of iterations may become very large, the stability of an adaptive algorithm should also be examined in its asymptotic behaviour. This problem either has not been considered in previous papers, or has been considered as a consequence o f the estimation accuracy. However, our investigation shows that this problem may or may not be related to the estimation accuracy. Hence, in this paper, we make the distinction between numerical accuracy and numerical stability in analyzing the effects of round-off errors in LS adaptive algorithms.

Our definition of the numerical stability of adaptive algorithms, as used in this paper, is akin to the defini- t ion of stability in system theory. It is defined as the boundedness of the estimation error, which can be viewed as the output of a system caused by a finite (bounded) round-off error at the input. This definition is a natural consequence of the fact that all the ada- ptive algorithms that we have considered are recursive in time. In other words, the estimated coefficients at t ime t are related to the same coefficients at time t - - 1. Hence, we may describe the recursive algorithm by using a feedback system model. The stability of the modeling system determines the stability of the adaptive algorithm. Below, we show how this model is used to investigate the numerical stability of different adaptive algorithms.

H. DEFINITION OF N U M E R I C A L STABILITY

AND ACCURACY O F ADAPTIVE ESTIMATION

A L G O R I T H M S

IH. A FEEDBACK S Y S T E M M O D E L AND N U M E R I C A L STABILITY

OF LS E S T I M A T I O N A L G O R I T H M S

Before we begin the discussion of the numerical properties of adaptive algorithms, we would like to distinguish the difference between the numerical stability and the numerical accuracy of an adaptive algorithm. The numerical stability has been defined

In general, we can use the following feedback system model to describe an adaptive algorithm :

(1) O(t) = f(0(t m 1), y(t), n(t) ),

where 0 is the parameter vector to be estimated, y(t) is the exact new signal coming in at time t and inde-

2/9 ANN. TI~LI~COMMUN., 41, n ~ 5-6, 1986

330 F. LING. - FINITE WORD-LENGTH EFFECTS IN RECURSIVE LS ALGORITHMS

pendent of O(t - - 1), and n(t) denotes the round-off error due to the finite word length.

The system may be linear or non linear. Because of n(t), the estimate 0(t) is not the same as its ideal value obtained by using infinite precision. We denote

the difference as 0(t). If, for any algorithm, this difference becomes unbounded with t, we say that the algorithm is unstable. Otherwise the algorithm is stable and we can determine its numerical accuracy

in steady state by evaluating the norm of 0(t). For a particular adaptive algorithm, equation (1)

can be determined f rom the algorithm itself. Below we consider the numerical stability o f some LS adaptive algorithms.

III.1. The numerical stability of the LS lattice algo- rithm.

The LS lattice algorithm has been discussed in a number of papers including [4], (*) and [6]. Since its numerical stability is related to time-recursions, we only need to examine the equations that involve time updates. In the algorithm, the crosscorrelation between the forward, backward and joint estimation errors and the variance are computed time-recursively. Since all these update equations have the same form, we consider the t ime-update equation o f the cross- correlation between the forward and backward errors, ks(t), as representative. The equation is as follows :

(2) ks+l(t) = w k m + d t - 1) § fs(t) b s ( t - 1)/~m(t--1) .

Note that the quantities in the second term of the right side of (2) are independent of km+l(t - - 1).

The corresponding system is illustrated in Figure 1, where n~, n2 and n 3 are round-off noises introduced in the computations. The system is a first order feedback system. Its stability is determined by its homogeneous part. It is obvious that this system is stable when w < 1. The variances rSm(t) and r~(t), in the LS lattice algorithm are also computed order- recursively [4]. Since all these other time-recursive equations in the algorithm are similar to (2), we conclude that the accumulated errors in the correla-

~n~(t) ln2(t) ~n3(t) fm (t) ~ ( t ) 13re(t- 1) Otto(t- 1) y t

FIG. 1. - - A feedback system model of the LS lattice algori thm.

Systdme d rdtroaction de l' algorithme des moindres carrds en treillis.

(*) PACK (J. D.), ~ATORIUS (E. H.). Least squares adaptive lattice algorithms. Nosc Tech. Rep. TR 423 (april 1979).

tions and variances are bounded. Hence, the LS lattice algorithm is always stable.

The LS lattice algori thm is claimed to have good numerical properties. This is true in the sense that it will never become unstable under our definition of numerical stability, as we have shown above. However, the estimation accuracy of the LS algorithm is another story as we will show later.

III.2. The numerical stability of the recursive least squares (Kalman) algorithm and its square-root form (L-D-U decomposition).

The recursive least squares (RLS), o r Kalman, algorithm given in [7] is known not to have good numerical properties. This is a consequence of the fact that the Kalman algorithm solves the normal equations in time-recursive form. As is well known, the accuracy of the solution of the normal equations is determined by the condition number of the co- variance matrix R(t). In the case where R(t) is ill- conditioned, the solution o f the normal equations yields an inaccurate solution, and so does the Kalman algorithm.

It is commonly believed that the RLS Kalman algorithm is numerically unstable and that the round- off error, which is small initially, will accumulate and eventually destroy the normal operation. F rom our investigation, we have shown that this is not the case, if an exponentially weighting factor is used. The analysis is as follows.

The main time recursion in the Kalman algorithm is the recursive estimation of the inverse of the corre- lation matrix R(t), denoted as P(t). We recall that it can be computed time-recursively as :

(3) [ f i ( t - - l )Y( t )Y ' ( t ) i ( t - -1) ]

I'(t) =lw I'(t--1) w+ Y' ( t ) i ( t - -1) Y(t) +n(t) ,

where if(t) and ff(t - - 1) are the computed approxi- mations of P(t) and P(t - - 1), respectively. They can be written as :

(4) if( t) : P(t) + ~P(t),

and :

(5) f f ( t - 1) = e ( t - 1) + ~ e ( t - 0,

respectively. We have taken account of the total effects of the round-off error and denoted it as n(t). Y(t) is the new data vector and it can be considered independent of P(t - - 1) in most cases.

The system described by (3) is non linear and direct analysis is difficult. I f n(t) in (3) is small, we can linearize the system by using an incremental model. Since the stability o f a system depends on the homo- geneous part, we need only to analyze this part. By substituting (4) and (5) into (3), we obtain :

ANN. TI~LI~COMMUN., 41, n ~ 5-6, 1986 3/9

F. L I N G . - F I N I T E W O R D = L E N G T H EFFECTS I N RECURSIVE LS A L G O R I T H M S 331

l[ (6) P ( t ) + 8P(t) = w P ( t - - 1) + 8 P ( t - - l ) - -

( P i t - - 1) + 8P( t - - 1) ) Y(t) Y ' ( t ) ( P ( t - - 1) + 8 P ( t - - 1))

w + Y ' ( t ) (P(t - - 1) + 8 P ( t - - 1) ) Y(t)

+ nft)] �9

Substituting the exact expression for P(t) into (6) and ignoring higher order terms of 8P(t), we can simplify (6) and, then, we obtain the time-recursion for 8P(t) ill the form :

1 [ P(t-- 1) Y(t) V'(t) ] (7) 8P(t) -~ w ,_1-- w-+ Y T - ( t ) P - ~ Y(t)] •

r i _ r(t) r ' ( t )P(t- I) I 8P(t m 1)

t w + Y ' ( t ) P ( t - 1) ~ '( t)] "

It is easy to show that �9

P ( t - 1) Y(t) Y ' ( t ) (8a) I - -

w + Y ' ( t ) P ( t - - 1) Y(t)

and :

= w P ( t ) P - 1 0 - 1 ) ,

Y(t) r ' ( t ) P(t - - 1) (8b) I = w P - t ( t - 1)P(t).

w + Y ' ( t ) P ( t - - 1) Y(t)

The result in (7) can be simplified further by substitut- ing (8a, b) into it. Thus, we obtain :

1 (9) 8P(t) ~- - [wP(t) P - l ( t - - 1)] 8 P ( t - - 1) •

W

[ w e - , f t - - 1) P(t)] .

Under steady state conditions, both P - l ( t ) and P - l ( t ~ 1) are approximately equal to R- I / (1 ~ w), where R is the autocorrelation matrix of the data vector Y(t). Hence, we can write :

(10) P(t) P - l ( t - - 1) ~ P - l ( t - - 1) P(t) ~ I.

With the aid of (10), the homogeneous equation in (9) becomes :

(11) 8P(t) = wSP(t m 1).

This indicates that the incremental system is stable when w < 1.

We have shown above that the exponentially weighted Kalman algorithm is numerically stable. This is true even though the Kalman algorithm is known to have poor numerical accuracy. On the other hand, the above analysis is correct only when the round-off error is small. If the round-off error exceeds a certain level, depending on the condition number of R, the algorithm will not work properly due to the estimation accuracy.

The above conclusion has been tested through computer simulation. We found that, when single precision floating point arithmetic is used, which has 22 bit for the mantissa, the Kalman algorithm is still stable after 106 iterations. The stability is demonstrated for different values of w (0.85 to 0.999), and for different signal characteristics including some signals with a correlation matrix close to singular.

The actual implementation of the time update of P(t) involves two steps. First, the Kalman gain vector is computed from P(t ~ 1) and Y(t). Then, P(t ) is computed from K(t). This sRuation has been consi- dered in [8]. A similar conclusion is obtaine:l there.

The square-root Kalman algorithm using the L D U decomposition can be analyzed similarly as indicated in [8]. It is shown there that the square-root Kalman algorithm is also numerically stable. In addition, we have tested its numerical stability through compu- ter simulation. When a word-length of 10-15 bits is used, with up to 1 0 6 iterations, the square-root Kalman algorithm does not show any instability.

We elaborate further on the numerical stability of the RLS Kalman algorithm and its square-root form. First, the key point for being stable is the use of the exponential weighting factor and the fact its value has to be less than 1. The original algorithm given by Kalman for estimation in a nonstationary environment would also have a numerical stability problem, since it does not use exponential weighting. Second, we did not observe the algorithms becoming unstable when we reduced the value of the weighting factor, although the autocovariance matrix, R(t), may be close to singular in that case. Finally, since we did not have the accumulation of the round-off error, the reinitialization for the square-root Kalman algorithm, suggested in [9], seems unnecessary to us. However, compared to the LS lattice algorithm and the RMGS algorithm, the square-root algorithm has a problem with dynamic range for a fixed point imple- mentation, which will be discussed later.

IH.3. The fast Kalman algorithm.

The poor numerical stability of the fast Kalman- type algorithms is well known, but the cause of this instability needs further investigation. Despite its instability, the fast Kalman algorithm provides good numerical accuracy when it operates on short data records, as will be shown in the next section. Hence, as we pointed out at the beginning of this paper when considering the application of adaptive algo- rithms we have to consider both their stability and their accuracy. As was reported in [10], we also observe that when the exponential weighting factor is reduced, the fast Kalman algorithm becomes unstable in fewer iterations.

IV. A COMPARISON OF THE NUMERICAL ACCURACY

OF SOME ADAPTIVE ALGORITHMS

We define the numerical accuracy of an adaptive algorithm in terms of its behaviour in the steady state. In other words, in numerical accuracy, we are

4/9 ANN. TI~LI~COMMUN., 41, n ~ 5=6, 1986

332 F. LING, - FINITE WORD-LENGTH EFFECTS IN RECURSIVE LS ALGORITHMS

interested in the size of the errors in the estimated quantities for a certain amount o f round-off error introduced in the computation, after the algorithm has passed the period of initial convergence. I f an algorithm is not stable, we can still consider its numeri- cal accuracy before the algorithm becomes unstable, as long as the instability happens after the algorithm has converged. A detailed analysis of the numerical accuracy for each adaptive algorithm is highly desi- rable but extremely involved. In this section we provide some simulation results to compare the numerical accuracy of some adaptive algorithms.

The simulation results given below are obtained f rom a simulated linear equalizer with 11 stages, using different adaptive algorithms. The description of the simulation can be found in [11]. The channel model we used here is a three tap transversal linear filter with the transfer function :

(12) H(z) = 0.251 -b 0.935 z -~ -[- 0.251 z -2.

The signal obtained f rom the channel has a maximum/ minimum eigenvalue ratio of its correlation matrix equal to 11. A weighting factor equal to 0.975 is used for the LS algorithms, and a step size equal to 0.025 is used for the (Widrow) LMS algorithm. Theoretically, the LS algorithms and the LMS algorithm will yield the same mean squared output error after conver- gence. The additive noise has a variance of 0.001. The mean squared output error of the equalizer is 0.0021, if the precision of the computat ion is infinite.

The simulation results on the numerical accuracy of some adaptive algorithms, including the square- root Kalman (LDU decomposition) algorithm, the fast Kalman algorithm, the LS lattice algorithm, the recursive modified Gram-Schmidt (RMGS) algorithm [12] and the LMS algorithm [13], are given in Table I. Fixed point arithmetic with precision f rom 7 bit to 15 bit (not including the sign bit) was used. The conventional Kalman algorithm does not work when the word-length is shorter than 16 bit, so no simulation results are provided.

The simulation results of the fast Kalman algo- r i thm are obtained by averaging the data f rom 100 runs, each run consisting of 300 iterations. Since the fast Kalman algorithm converges after about 150 itera- tions, we only use the output error f rom 200 to 300 ite-

rations. The algorithm diverges after about 500 iterations when a precision of ten bits was used.

The following comments and conclusions can be drawn from our simulation results.

i) Although it is known that the lattice structure is less sensitive to quantization error than the trans- versal structure for fixed coefficient filters [14], from the simulation results, we observe that the numerical accuracy of the LS lattice algorithm is similar to the accuracy of the LMS algorithm and worse than the accuracy of the square root Kalman algorithm, which uses the LDU decomposition. Both of the latter algorithms have a transversal structure. The reason for this and a method to remedy this problem is given in the next section.

ii) The fast Kalman algorithm, which is known to be unstable, is very accurate for a word-length greater than 10 bit, but it becomes unstable more quickly with less precision. This reinforces our pre- vious statement that numerical accuracy and stability are two different properties of adaptive algorithms and should be considered independently. Although the fast Kalman algorithm is not stable, it can be used for estimation over short data records, or as a fast start-up procedure, and then one can switch to the LMS algorithm, if necessary. The only effective way to ensure the stable operat ion o f the fast Kalman algorithm in a long run is to use a periodic reinitializa- tion procedure as proposed in [15].

iii) The signal used in the simulation has a maximum/ minimum eigenvalue ratio equal to 1 I. Hence, the correlation matrix is not close to singular. Under this condition we obtain satisfactory performance from the square-root Kalman algorithm. We have also tested the square-root Kalman algorithm when the correlation matrix of the input signal is close to singular. The performance was not as good as we have given above. For the latter case, we had to rescale the quantities in the square-root Kalman algorithm to make it operate properly. Some quantities will overflow when the word-length is less than 12 bit. This is because the dynamic range of the quantities depends on the characteristics of the input signal. On the other hand, the dynamic range of the parameters in the LS lattice algori thm is smaller and,

TABLE I. - - N u m e r i c a l accuracy o f adap t ive a lgo r i t hms ( M e a n s q u a r e d ou tpu t e r ro r • 10-3).

N u m b e r S qua re - roo t Fa s t K a l m a n LS latt ice RMGS LMS o f bi ts K a l m a n

15 12 10

9 8 7

2.17 2.33 6.14

17.6 75.3

2.17 2.21 3.34

t1

I*

2.18 3.09

25.2 187.0 365.0

2.17 2.36 6.21

220.0

2.30 2.30

19.0 77.2

311.0 1170.0

* T he a l g o r i t h m diverges . O u t p u t MSE is 2.1 • 10 -3 for infinite precis ion.

ANN. T~L~COMMUN., 41, n ~ 5-6, 1986 5/9

F. L I N G . - F INITE W O R D - L E N G T H EFFECTS IN RECURSIVE LS A L G O R I T H M S 333

hence, they are not as sensitive to the signal characte- ristics. This is another advantage of the LS lattice algorithm. We will discuss this point later.

iv) The RMGS algori thm and the multichannel LS lattice algorithm using the sequential processing stages yield similar performance to the single channel LS lattice algorithm. These algorithms are recommended when they are applicable.

V. T H E N U M E R I C A L ACCURACY OF T H E LS LATTICE A L G O R I T H M

AND T H E R O L E OF ERROR-FEEDBACK IN I M P R O V I N G T H E N U M E R I C A L ACCURACY

OF LS ADAPTIVE A L G O R I T H M S

The simulation results given in the previous section indicate that the conventional LS lattice algorithm does not provide accurate estimates. This conclusion has motivated us to investigate the cause of the inaccuracy and to find a remedy for this problem. Since the parameters estimated in the LS lattice algorithm are the reflection coefficients, the extra output error is mainly caused by the inaccurate estimation of these coefficients.

We recall that in the conventional form of the LS lattice algorithm, the forward and the backward reflection coefficients are computed in the following way : First, the crosscorrelation between the forward and backward prediction errors and their variances are computed. Then, the reflection coefficients are computed by dividing the crosscorrelation by the variances. Since the crosscorrelation and the variances cannot be calculated accurately when a short word length is used in the computat ion, the computed reflection coefficients are generally biased. This is the cause of the inaccuracy o f the LS lattice algorithm.

To reduce the bias in the computed coefficients, we modified the conventional LS lattice algorithm by deriving a new error-feedback formula. The LS lattice algorithm which employs this new formula to estimate its reflection coefficients is more robust to round-off errors.

The new formula to compute the forward reflection coefficient is :

gm(t- 1) km+x(t) - - k m + t ( t - - 1) + rbm( t __ 1) • (13) s - - J"

b m ( t - 1) fro+ l(t).

Since the estimation error f m + l ( t ) is used to estimate the coefficient k~+a( t ) , we call it the error-feedback estimation formula. The backward reflection, coeffi- cient and the jo int estimation coefficient can be computed similarly. F r o m these equations, we note

that the new formulas for estimating the reflection coefficients are similar to the stochastic gradient algorithm, if we consider ~m(t - - 1) l r~( t - - 1) in (13) as a variable step size. Its robustness to round-off error is mainly due to this similarity. Since the error feedback formula and its numerical properties have been discussed in other papers [11, 16, 17], we will make the following brief comments.

i) The estimation accuracy of the LS lattice algorithm depends on how its reflection coefficients are computed.

ii) Due to the error-feedback nature of the new estimation formulas, the estimation accuracy does not depend directly on the computat ion accuracy of the correlation coefficients. Since the variance only serves as a variable step size, its accuracy does not affect the accuracy of the reflection coefficients either. The fact that the Kalman and the fast Kalman algorithms also use an error feedback form to estimate their coefficient vector explains their better estimation accuracy compared to the conventional LS lattice algorithm.

iii) Some variations o f the gradient algorithm have been used to estimate the reflection coefficients in the gradient lattice algorithm. The robustness of the gradient lattice algori thm to round-off error, which has been reported in the literature, is probably attr ibuted to the error feedback form used in the estimation process. The above error-feedback formula for the LS lattice algorithm, which combines the opti- mum convergence of LS estimation and the robustness of the gradient algorithm, provides a better choice for adaptive filtering and estimation when a short word length is desired in an implementation.

iv) The lattice algorithms can only be used when the data vector has certain shifting properties. For applications where the shifting property does not exist, the recursive modified Gram-Schmidt algorithm [12], that we derived recently, may be used instead. The error-feedback formula can be easily applied to the RMGS algorithm and the sequential processing stage of the multichannel lattice algorithm. As a result, we not only obtain a more robust algorithm, but a more efficient algori thm as well. The compu- tational complexity is N 2 -k- 6 N for the RMGS algo- r i thm and 6 p2 _t_ 10 p for a p-channel lattice stage. This number is close to the computat ional complexity of the multichannel fast Kalman-type algorithms [10].

The decrease in sensitivity provided by the new error-feedback formula given above has been reinfor- ced by simulation results. Table II provides the output mean square error of the RMGS and the LS lattice algorithms using both the conventional form and the error-feedback form. Fixed point arithmetic with a word-length f rom 8 to 15 bit was used in the simulation. The improvement of the new formula is obvious f rom these results.

6/9 ANN. T~L~COMMUN., 41, n ~ 5-6, 1986

334 F. LING. - FINITE WORD-LENGTH EFFECTS IN RECURSIVE LS ALGORITHMS

TABLE II. - - Numerical accuracy of LS lattice and RMGS algorithms using the error feedback formula

(MSE • lO-a).

Number of bits

Floating point 15 12 10 8

R M G S

2.10 2.17 2.36 6.21

220.0

R M G S E F

2.10 2.16 2.19 2.81

20.6

L S L A

2.10 2.18 3.09

25.2 365.0

L S L A E F

2.10 2.16 2.22 3.09

31.7

S Q R K A L

2.10 2.17 2.33 6.14

75.3

R M G S

R M G S E F

L S L A

L S L A E F

S Q R K A L

Recursive modified Gram-Schmidt algorithm. Recursive modified Gram-Schmidt algorithm using error feedback formula. Least squares lattice algorithm. Least squares lattice algorithm using error feedback formula. Square-root Kalman algorithm (LDU decomposition).

VI. ADAPTIVE A L G O R I T H M S U S I N G MIXED P R E C I S I O N

A R I T H M E T I C

The estimation accuracy of adaptive algorithms is affected more by addition than by multiplication in the computation. On the other hand, multiplication needs more computat ion time. This suggests that we employ a longer word-length for addition than multiplication. This approach is well known in numeri- cal analysis and proves to be effective for adaptive algorithms.

The effects of a mixed precision arithmetic is given in Figure 2. As predicted, the precision for addition has a larger effect on the output error than the pre- cision for multiplication. The simulation results from an LMS algorithm are also given. It seems that using a long word length in addition for the LMS algorithm is more effective than for the lattice algo- rithm and the RMGS algorithm. In any case using a long word length in addition will help to improve the estimation accuracy without appreciably increasing the computat ional complexity.

VII. F IXED P O I N T I M P L E M E N T A T I O N OF A D A P T I V E A L G O R I T H M S

AND T H E P R O B L E M OF D Y N A M I C RANGE

When fixed point arithmetic is used to implement adaptive algorithms, one problem which we have to pay special attention to is proper scaling to prevent overflow. It is desirable to use some predetermined fixed scaling factor that does not change during operation of the adaptive filter or estimator. Conse- quently, we need to investigate the dynamic range

of the quantities in different adaptive algorithms and how they change with the input signal characteristics.

It is known that the dynamic range of the quantities in an adaptive algorithm depends on the power of the input signal. In general, it is desirable to normalize the power of the input signal. The normalization can be done by using an automatic gain control or a simpler scaling algorithm. In the sequel, we assume

Output SNR & o Same word-length for | multiplication and addition /

/ z~ 12 bits for addition 1 0 _ ~ ~ o 15 bits for addition

- 3 b 10 I , , , , . . . . . - 6 8 10 12 14

Word-length for multiplication

Output SNR o Same word-length for

multiplication and addition ' ~ bits for addition zx 12

10 -~ ~ o 15 bits for addition

1 0 - 2 ,

1 0 - 3 , ,

8 ' ~ ' 1'o 1~ 1~, Word-length for multiplication

FIG. 2. - - Numerical accuracy for adaptive algorithms using mixed precision arithmetic.

(a) Least squares lattice algorithm using the error feedback formula.

(b) LMS algorithm.

Precision num~rique pour les algorithmes adaptatifs utilisant la precision arithmdtique mixte.

(a) Algorithme des moindres carrds en treillis utilisant la for- mule de r~troaction d'erreur.

(b) Algorithme des moindres carr~s.

ANN. TI~L~COMMUN., 41, n ~ 5-6, 1986 7/9

F. L I N G . -- F INITE W O R D - L E N G T H EFFECTS IN RECURSIVE LS ALGORITHMS 335

the power of the input data and the desired signal have been normalized to unity.

Below, we consider the dynamic range problem for a number of LS adaptive algorithms.

VII.I. The normalized LS lattice algorithm [4].

It is known that all the quantities in the single channel normalized lattice algorithm have magnitudes less than one. This algorithm is free of the problem of overflow. However, the normalized lattice algo- rithm requires a number, proportional to N, of square- root operations. It also requires denormalization for certain applications. These two problems with the normalized lattice algorithm limit its practical appli- cations. Another problem with this algorithm is that the magnitude of the normalized errors is approxima- tely equal to the square root of (1 - - w). As a conse- quence, the effective word-length of the normalized errors is much shorter than the total word-length, when w is close to one. This increases the output noise and the bias of the reflection coefficients, as reported in [5].

VII.2. The Kalman algorithm.

The Kalman algorithm has poor numerical accu- racy. Although it is numerically stable, it does not work properly, when fixed-point arithmetic with a word-length shorter than 16 bit is used. We have never been able to implement it successfully by using fixed point arithmetic with fewer than 16 bits accuracy. Hence, we will not discuss it further.

VII.3. The square-root Kalman algorithm [3, 9].

The square-root Kalman algorithm that we have considered is the algorithm employing the L D U decomposition. That is, the inverse of the covariance matrix R- qt ) or P(t) is neither computed nor stored. Instead, the algorithm stores and updates its factors, the triangular matrix L(t) and the diagonal matrix D(t), where P(t) = L(t) D(t) L'(t). Since R(t) is an estimate of R/(1 - - w), where R is the correlation matrix of Y(t), P(t) can be viewed as an estimate of (1 - - w) R -1. If the normalized signals are used, the largest (in magnitude) of the elements of R(t) is of the order of 1/(1 - - w). However, the largest element of P(t) is difficult to predict. It is a function of the signal characteristics of Y(t), especially, when R is close to singular. The norm of R - l , I R-a]2, may be very large. For example, [R-112 = l / ~ . m t n , which is large when ~'min is close to zero. In a fixed-point implementation, we are mainly interested in ]R-~]| ; however, 1R-112 also gives us an indication of the magnitudes of the elements of R- x. As a consequence,

the magnitude of the elements of P(t) may become very large when R is close to a singular matrix. This is also the case when the L-D-U decomposition is used. The largest element of D(t) is also very large in the case above.

From the discussion above we conclude that the largest (in magnitude) of the quantities in the square- root Kalman algorithm depends on the signal charac- teristics. It is difficult to determine an upper bound on the magnitudes of the elements of P(t) without knowing the signal characteristics in advance. This is the difficulty in implementing the square-root Kalman algorithm in fixed point arithmetic. When used in a nonstationary environment, optimum scaling is extre- mely difficult. The scale factor that we used for the case where ~kmax/]kmln = 11 does not work well when a larger eigenvalue ratio is used. The optimum scaling for the square-root Kalman algorithm can be achieved when the signal characteristics are stationary and are known in advance.

VII.4. The unnormalized LS lattice algorithm.

Unlike the Kalman and the square-root Kalman algorithms, the LS lattice and the RMGS algorithms are easier to implement by using fixed point arithmetic. A simple predetermined fixed scaling ensures the algorithm will work properly. We first consider the LS lattice algorithm.

One nice property of the LS lattice algorithm is that, in general, the average magnitude of most quantities in one stage is less than the average magni- tude of the corresponding quantities in the preceding stages. For example, for the variances and correla- tions, we have that r~( t ) > r~ + 1(0 and k,,( t ) > km + l ( t ). It is enough to consider the quantities of the first stage.

Since fo(t) = bo(t) = y(t) in the LS lattice and E[y2(t)] = 1, we impose a scaling factor of 1/4 for fm(t) and b,,(t). Assuming that y(t) is Gaussian, the probability that ly(t)l > 4 is less than 10 -4. If this small-probability-event does occur, we can simply set a hard limiter by letting the quantity equal to + 4.

The average value of rSo(t) and rg(t) is about equal to 1[(1 - - w). Since rSo(t) and r~(t) vary we can scale them by a factor equal to an integer which is a power of 2 and close to (1 - - w)/2. Since w is usually known when we implement the algorithm, this scaling factor is easy to precompute. In the conventional LS lattice algorithm ko(t) is smaller than rSo(t). The scaling factor for rSo(t) can also be used for kin(t). If the error- feedback form of the LS lattice algorithm is used, we compute k~(t) and k~(t) instead of kin(t). From the Leviason algorithm we know that the reflection coefficients are always less than unity. Being the estimate of the reflection coefficients, k ~ ( t ) a n d k~(t) should also be less than 1. A scaling factor of 1/2 would be sufficient. Similar conclusions can be drawn for the joint estimator part.

8 /9 ANN. T~,LI~COMMUN., 41 , n ~ 5-6, 1986

336 F. LING. - FINITE WORD-LENGTH EFFECTS IN RECU'RSIVE LS ALGORITHMS

From the above discussion, we note that the scaling factors for the LS lattice do not depend on the signal characteristics. The only parameter which may affect the scaling is the weighting factor w, which is easy to predetermine. Hence, we conclude that the unnor- malized lattice algorithm is a better choice than the square-root Kalman algorithm for a fixed point implementation, especially when the adaptive estima- tor or filter is used in a nonstat ionary environment.

VII.5. The RMGS algorithm.

When the shifting property does not exist, the LS lattice algorithm cannot be used. A good alternative to the Kalman-type algorithms is the RMGS algorithm.

Since the RMGS algorithm performs the L D U

decomposition of R( t ) (see [12]), instead of R - l ( t ) ,

the values of the elements of these factor matrices do not depend on the signal characteristics. Hence, we may choose the optimum scaling factor without knowing signal characteristics in advance. Conse- quently, it is easier to implement the RMGS algorithm than the square-root Kalman algorithm. The optimum scaling factor can be chosen using the same method as for the LS lattice algorithm.

VIII. CONCLUDING REMARKS

We have provided a summary of recent and new results on finite word length effects in recursive least squares adaptive algorithms. The numerical accuracy and the numerical stability of adaptive recursive least squares algorithms were defined and evaluated. It was demonstrated that the conventional recursive least squares lattice algorithm has poor numerical accuracy when a short word length is used. To remedy the problem we devised a new form of a recursive least squares lattice algorithm which is more robust compared to round-off errors than the conventional form. Analytic results were obtained which explain the advantages of the new form of the recursive least squares algorithm.

Optimum scaling of recursive least squares algo- rithms for fixed-point realizations was also considered. We showed that the opt imum scaling for recursive least squares lattice algorithms and its extensions is insensitive to signal characteristics. On the other hand, the opt imum scaling for the Kalman algorithm and its square-root form ( L D U decomposition) is sensitive to signal characteristics. Consequently, the recursive least squares lattice algorithm (and its extensions) is more suitable for a fixed-point implementation.

Manuscr i t regu le 28 juin 1985, acceptd le 20 ddcembre 1985.

REFERENCES

[1] HOUSEHOLDER (A. S.). The theory of matrices in numerical analysis. Blaisdell Pub. Co., N Y 0964).

[2] CYBENKO (G.). The numerical stability of the Levinson- Durbin algorithm for Toeplitz systems of equations. SIAM J. Sci. Statist. Comput., USA (sep. 1980), 1.

[3] BIERMAN (G. J.). Factorization methods for discrete sequential estimation. Academic Press, New York (1971).

[4] LEE et aL Rccursive least squares ladder estimation algo- rithms. IEEE Trans. ASSP, USA (June 1981), 29, pp. 627- 641.

[5] SAMSON (C.), REDDY (V. 1.)'.). Fixed-point error analysis of the normalized ladder algorithm. IEEE Trans. ASSP, USA (Oct. 1983), 31, n ~ 5, pp. 1177-1191.

[6] PROAKIS (J. G.). Digital communications. McGraw-Hill, New York (1983).

[7] GODARD (D.). Channel equalization using a Kalman filter for fast data transmission. IBM J. Res. Develop., USA (May 1974), pp. 267-273.

[8] LJUNG (L.). Fast algorithms for integral equations and least-squares identification problems. Ph.D. Dissertation, Linkoping Univ., Linkoping, Sweden (1983).

[9] Hsu (F. M.). Square-root Kalman filtering for high- speed data received over fading dispersive HF channels. IEEE Trans. IT, USA (Sep. 1982), 28, n ~ 5, pp. 753-763.

[10] CIOVFI (J. M.). Fast transversal filters for communications appIications. Ph.D. Dissertation, Stanford Univ. (1984).

[11] LING (F.). Rapidly convergent adaptive filtering algorithms with applications to adaptive equalization and channel estimation. P h . D . Thesis. Northeastern Univ., Boston, Mass. (Sep. 1984).

[12] LING (F.), PROAKIS (J. G.). A recursive modified Grain- Schmidt algorithm with applications to least squares estimation and adaptive filtering. Proe. 1984 IEEE Inter- national Symposium on Circuits and Systems, Quebec, Canada (May 1984).

[13] WIDROW (B.) et al. Stationary and nonstationary learning characteristics of the LMS adaptive filter. Proc. IEEE, USA (Aug. 1976), 64, pp. 1156-1162.

[14] MARKEL (J. D.), GRAY (A. H.) Jr. Round-offnoise characte- ristics of a class of orthogonal polynomial structures. IEEE Trans. ASSP, USA (Oct. 1975), 23, pp. 473-486.

[15] LIN (D. W.). On digital implementation of the fast Kalman algorithm. IEEE Trans. ASSP, USA (Oct. 1984), 32, pp. 998-1005.

[16] LING (F.), MANOLAKIS (O.), PROAKIS (J. G.). Least squares lattice algorithms with direct updating of the reflection coefficients. Proc. 1985 GRETSI Internat. Signal Proc. Applicat., Nice, France (May 20-24, 1985).

[17] LING (F.), PROAKIS (J. G.). Numerical accuracy and stabi- lity. Two problems of adaptive algorithms caused by round-off error. Proc. 1984 1EEL Internat. Conf. ASSP, San Diego, CA (Mar. 1984).

ANN. T~L~COMMUN., 41, n ~ 5-6, 1986 9/9