Applied Mathematics and Computation 186 (2007) 184–192

www.elsevier.com/locate/amc

A finite-data-window least squares algorithmwith a forgetting factor for dynamical modeling q

Feng Ding *, Yongsong Xiao

Control Science and Engineering Research Center, Southern Yangtze University, Wuxi 214122, China

Abstract

In this paper, we extend the forgetting factor least squares and finite-data-window least squares identification algo-rithms, develop a finite-data-window least squares algorithm with a forgetting factor for dynamical system modeling,derive its recursive version, and also give its simplified form. We illustrate the advantages of the proposed algorithm withsimulation examples.� 2006 Elsevier Inc. All rights reserved.

Keywords: Recursive identification; Parameter estimation; Least squares algorithm; Forgetting factor; Finite-data window

1. Problem formulation

Some identification algorithms with their gains approach zero, e.g., the hierarchical algorithms [1,2], theauxiliary model algorithms [3–6], the multi-innovation algorithms [7,8], the bias compensation least squaresalgorithms [9], can give consistent estimation of time-invariant systems; other identification methods withnon-zero gains, such as the covariance resetting least squares and covariance modification least squares algo-rithms [10], can be used to track time-varying parameters; however, we believe that the most effective andimportant methods are the forgetting-factor least squares (FF-LS) algorithm [11] and the finite-data-windowleast squares algorithm (FDW-LS) [10,12]. How to retain the advantages of both in a single algorithm is themotivation in this work. We extend the FF-LS and FDW-LS algorithms, develop a least squares algorithmwith a forgetting factor over a finite-data window, and derive its recursive version. To the authors’ best knowl-edge, the contribution in this paper has not been reported in the literature.

Let us introduce some notation first. The superscript T denotes the matrix transpose; the symbol I standsfor an identity matrix of appropriate sizes; the norm of a matrix X is defined by kXk2 = tr[XXT]; 1n representsan n-dimensional column vector whose elements are all 1.

0096-3003/$ - see front matter � 2006 Elsevier Inc. All rights reserved.

doi:10.1016/j.amc.2006.06.133

q This research was supported by the National Natural Science Foundation of China (No. 60574051).* Corresponding author.

E-mail addresses: fding@sytu.edu.cn, dingf@tsinghua.edu.cn (F. Ding).

F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192 185

Consider the identification problem of the following system:

yðtÞ ¼ uTðtÞhþ vðtÞ: ð1Þ

Here y(t) is the system output, v(t) is a white noise with zero mean, h 2 Rn is the parameter vector to be iden-tified and uðtÞ 2 Rn is the regressive information vector consisting of the system input and output. Over thedata set {y(i),u(i) : 1 6 i 6 t}, minimizing the cost function

J 1ðhÞ :¼Xt

i¼1

½yðiÞ � uTðiÞh�2

leads to the recursive least squares (RLS) estimation algorithm,

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞhðt � 1Þ�;P�1ðtÞ ¼ P�1ðt � 1Þ þ uðtÞuTðtÞ; Pð0Þ ¼ p0I ;

where hðtÞ represents the estimate of h at time t, P(t) denotes the covariance matrix and p0 is a positiveconstant.

The RLS algorithm is unable to track time-varying parameters because of P(t)! 0 as t!1. In order totrack time-varying parameters, we introduce a forgetting factor k and minimize the weighted cost function

J 2ðhÞ :¼Xt

i¼1

kt�i½yðiÞ � uTðiÞh�2

to obtain the forgetting-factor recursive least squares (FF-RLS) algorithm:

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞhðt � 1Þ�;P�1ðtÞ ¼ kP�1ðt � 1Þ þ uðtÞuTðtÞ; 0 < k < 1; Pð0Þ ¼ p0I :

The RLS and FF-RLS algorithms use all input-output data {y(i), u(i) :1 6 i 6 t} up to and including time t

to compute the parameter estimate h. Another way is to use a finite data set {y(i),u(i) : t � p + 1 6 i 6 t} (themost recent data with window length p); then minimizing the cost function

J 3ðhÞ :¼Xt

i¼t�pþ1

½yðiÞ � uTðiÞh�2;

we may obtain a finite-data-window least squares algorithm. Further, minimizing the weighted cost functionwith data window length equaling p,

J 4ðhÞ :¼Xt

i¼t�pþ1

kt�i½yðiÞ � uTðiÞh�2;

will result in a least squares algorithm with a forgetting factor over a finite-data window.At this point, we see that if p = t, then J3 = J1 and J4 = J2; if k = 1, then J4 = J3 and J2 = J1; and if both

p = t and k = 1, then J4 = J3 = J2 = J1. Thus, J4 is a most general cost function.The objective of this paper is to derive a recursive identification algorithm to estimate the parameter vector

h by minimizing the cost function J4, and to study its performance via simulation.

2. The algorithm derivation

Here, we derive the recursive version of the least squares algorithm with a forgetting factor over a finite-data window (FF-FDW-LS algorithm), which does not require computing matrix inversion and can be imple-mented on-line.

186 F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192

Let l :¼ffiffiffikp

, and define the weighted data matrix and weighted output vector as

Hðp; tÞ ¼

uTðtÞluTðt � 1Þl2uTðt � 2Þ

..

.

lp�1uTðt � p þ 1Þ

266666664

3777777752 Rp�n; Yðp; tÞ ¼

yðtÞlyðt � 1Þl2yðt � 2Þ

..

.

lp�1yðt � p þ 1Þ

26666664

377777752 Rp:

Then J4 can be written as

J 4ðhÞ ¼ kYðp; tÞ �Hðp; tÞhk2:

Letting the derivative of J4 with respect to h be zero, we have

oJ 4ðhÞoh

¼ 2HTðp; tÞ½Yðp; tÞ �Hðp; tÞh� ¼ 0:

Provided that u(t) is persistently exciting (for large p), the least squares estimate of h is given by

hðtÞ ¼ ½HTðp; tÞHðp; tÞ��1HTðp; tÞYðp; tÞ: ð2Þ

It follows that:

hðt � 1Þ ¼ ½HTðp; t � 1ÞHðp; t � 1Þ��1HTðp; t � 1ÞYðp; t � 1Þ: ð3Þ

In order to find the recursive relation between hðtÞ and hðt � 1Þ, we need to define an intermediate ‘‘estimate’’

aðt � 1Þ :¼ ½HTðp � 1; t � 1ÞHðp � 1; t � 1Þ��1HTðp � 1; tÞYðp � 1; t � 1Þ; ð4Þ

which represents the estimate obtained by minimizing the weighted cost function J4 with data window lengthp � 1, namely,

J 5ðhÞ :¼Xt�1

i¼t�pþ1

kt�1�i½yðiÞ � uTðiÞh�2:

From the definitions of H(p, t) and Y(p, t), we have

Hðp; tÞ ¼ uTðtÞlHðp � 1; t � 1Þ

� �;

Yðp; tÞ ¼yðtÞ

lYðp � 1; t � 1Þ

� �;

Hðp; t � 1Þ ¼Hðp � 1; t � 1Þ

lpuTðt � pÞ

� �;

Yðp; t � 1Þ ¼Yðp � 1; t � 1Þ

lpyðt � pÞ

� �:

Define the covariance matrices

PðtÞ :¼ ½HTðp; tÞHðp; tÞ��1; ð5Þ

Paðt � 1Þ :¼ ½HTðp � 1; t � 1ÞHðp � 1; t � 1Þ��1: ð6Þ

It follows that:

P�1ðtÞ ¼ ½uðtÞlHTðp � 1; t � 1Þ� uTðtÞlHTðp � 1; t � 1Þ

� �¼ kHTðp � 1; t � 1ÞHðp � 1; t � 1Þ þ uðtÞuTðtÞ¼ kP�1

a ðt � 1Þ þ uðtÞuTðtÞ;

F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192 187

P�1ðt � 1Þ ¼ HTðp; t � 1ÞHðp; t � 1Þ

¼ ½HTðp � 1; t � 1Þlpuðt � pÞ�sHðp � 1; t � 1Þ

lpuTðt � pÞ

� �¼ HTðp � 1; t � 1ÞHðp � 1; t � 1Þ þ kpuðt � pÞuTðt � pÞ¼ P�1

a ðt � 1Þ þ kpuðt � pÞuTðt � pÞ;aðt � 1Þ :¼ ½HTðp � 1; t � 1ÞHðp � 1; t � 1Þ��1

HTðp � 1; t � 1ÞYðp � 1; t � 1Þ¼ Paðt � 1ÞHTðp � 1; t � 1ÞYðp � 1; t � 1Þ:

From (2), it is easy to get

hðtÞ ¼ ½HTðp; tÞHðp; tÞ��1HTðp; tÞYðp; tÞ

¼ PðtÞ½uðtÞlHTðp � 1; t � 1Þ� yðtÞlYðp � 1; t � 1Þ

� �¼ PðtÞ½kHTðp � 1; t � 1ÞYðp � 1; t � 1Þ þ uðtÞyðtÞ�¼ PðtÞ½kP�1

a ðt � 1ÞPaðt � 1ÞHTðp � 1; t � 1ÞYðp � 1; t � 1Þ þ uðtÞyðtÞ�¼ PðtÞ½kP�1

a ðt � 1Þaðt � 1Þ þ uðtÞyðtÞ�¼ kPðtÞP�1

a ðt � 1Þaðt � 1Þ þ PðtÞuðtÞyðtÞ¼ ½I � PðtÞuðtÞuTðtÞ�aðt � 1Þ þ PðtÞuðtÞyðtÞ¼ aðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞaðt � 1Þ�;

hðt � 1Þ ¼ ½HTðp; t � 1ÞHðp; t � 1Þ��1HTðp; t � 1ÞYðp; t � 1Þ

¼ Pðt � 1Þ½HTðp � 1; t � 1Þlpuðt � pÞ� Yðp � 1; t � 1Þlpyðt � pÞ

� �¼ Pðt � 1Þ½HTðp � 1; t � 1ÞYðp � 1; t � 1Þ þ kpuðt � pÞyðt � pÞ�¼ Pðt � 1Þ½P�1

a ðt � 1Þaðt � 1Þ þ kpuðt � pÞyðt � pÞ�;aðt � 1Þ ¼ Paðt � 1ÞP�1ðt � 1Þhðt � 1Þ � kpPaðt � 1Þuðt � pÞyðt � pÞ

¼ ½I � kpPaðt � 1Þuðt � pÞuTðt � pÞ�hðt � 1Þ � kpPaðt � 1Þuðt � pÞyðt � pÞ¼ hðt � 1Þ � kpPaðt � 1Þuðt � pÞ½yðt � pÞ þ uTðt � pÞhðt � 1Þ�:

From the above equations, we obtain the FF-FDW-RLS algorithm as follows:

hðtÞ ¼ aðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞaðt � 1Þ�;P�1ðtÞ ¼ kP�1

a ðt � 1Þ þ uðtÞuTðtÞ;aðt � 1Þ ¼ hðt � 1Þ � kpPaðt � 1Þuðt � pÞ½yðt � pÞ þ uTðt � pÞhðt � 1Þ�;P�1

a ðt � 1Þ ¼ P�1ðt � 1Þ � kpuðt � pÞuTðt � pÞ:

By applying the matrix inversion lemma,

ðAþ BCÞ�1 ¼ A�1 � A�1BðI þ CA�1BÞ�1CA�1;

they can be equivalently expressed as

hðtÞ ¼ aðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞaðt � 1Þ�; ð7Þ

PðtÞ ¼ 1

kPaðt � 1Þ � Paðt � 1ÞuðtÞuTðtÞPaðt � 1Þ

kþ uTðtÞPaðt � 1ÞuðtÞ

� �; ð8Þ

aðt � 1Þ ¼ hðt � 1Þ � kpPaðt � 1Þuðt � pÞ½yðt � pÞ þ uTðt � pÞhðt � 1Þ�; ð9Þ

Paðt � 1Þ ¼ Pðt � 1Þ þ Pðt � 1Þuðt � pÞuTðt � pÞPðt � 1Þk�p � uTðt � pÞPðt � 1Þuðt � pÞ : ð10Þ

188 F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192

To initialize the FF-FDW-RLS algorithm, we take h0 ¼ 0 or some small real vector, e.g., h0 ¼ 10�61n;P(0) = p0I with p0 being a large positive number, e.g., p0 = 106.

To summarize, we list the steps involved in the FF-FDW-RLS algorithm to compute hðtÞ as t increases:

1. Collect the input–output data {u(t),y(t) : t = 1,2, . . . ,L}, select data length p and forgetting factor k, andform u(t).

2. To initialize, let t = 1, hð0Þ ¼ 10�61n and P(0) = 106I.3. Compute Pa(t � 1) by (10), a(t � 1) by (9), P(t) by (8), and hðtÞ by (7), respectively.4. If t = L, then terminate the procedure and obtain the parameter estimate hðtÞ; otherwise, increment t by 1

and go to step 3.

The FF-FDW-RLS algorithm employs the idea of updating the estimate h using a fixed data batch with afinite window length p, and is capable of tracking time-varying parameters.

When k = 1, we obtain the finite-data-window recursive least squares algorithm:

hðtÞ ¼ aðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞaðt � 1Þ�;

PðtÞ ¼ Paðt � 1Þ � Paðt � 1ÞuðtÞuTðtÞPaðt � 1Þ1þ uTðtÞPaðt � 1ÞuðtÞ ;

aðt � 1Þ ¼ hðt � 1Þ � Paðt � 1Þuðt � pÞ½yðt � pÞ þ uTðt � pÞhðt � 1Þ�;

Paðt � 1Þ ¼ Pðt � 1Þ þ Pðt � 1Þuðt � pÞuTðt � pÞPðt � 1Þ1� uTðt � pÞPðt � 1Þuðt � pÞ :

For the FF-FDW-RLS algorithm in (7)–(10), it is clear that if the dimension, n, of the parameter vector h isvery large, then this algorithm will lead to increased computational complexity, which may be reduced byusing the hierarchical identification scheme [1,2,14–17].

3. An approximate FF-FDW-RLS algorithm

The FF-FDW-RLS algorithm is complicated because it involves the intermediate variable a(t � 1). Next,we give an approximate FF-FDW-RLS algorithm which is more efficient.

The FF-RLS algorithm can be expressed as

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞhðt � 1Þ�;

P�1ðtÞ ¼Xt

i¼1

kt�iuðiÞuTðiÞ; 0 < k < 1:

Comparing the cost functions J2(h) and J4(h), from the above two equations, we can see that the FF-FDW-RLS algorithm can be approximately simplified as

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞhðt � 1Þ�;

P�1ðtÞ ¼Xt

i¼t�pþ1

kt�iuðiÞuTðiÞ; 0 < k < 1:

The covariance matrix P(t) may be written in a recursive form

P�1ðtÞ ¼ uðtÞuTðtÞ þ kuðt � 1ÞuTðt � 1Þ þ � � � þ kp�1uðt � p þ 1ÞuTðt � p þ 1Þ¼ uðtÞuTðtÞ þ kP�1ðt � 1Þ � kpuðt � pÞuTðt � pÞ ¼: uðtÞuTðtÞ þ kP�1

1 ðt � 1Þ;

where

P�11 ðt � 1Þ :¼ P�1ðt � 1Þ � kp�1uðt � pÞuTðt � pÞ:

Table 1Comparing computational efficiency

Algorithms Number of multiplications Number of additions

I 7n2 + 8n + p + 2 6n2 + 2n

II 6n2 + 6n + p � 1 5n2 + n

F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192 189

Applying the matrix inversion lemma to the above two equations, the simplified FF-FDW-RLS algorithm canbe implemented by the following equations [13]:

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uTðtÞhðt � 1Þ�; ð11Þ

PðtÞ ¼ 1

kP1ðt � 1Þ � P1ðt � 1ÞuðtÞuTðtÞP1ðt � 1Þ

kþ uTðtÞP1ðt � 1ÞuðtÞ

� �; ð12Þ

P1ðt � 1Þ ¼ Pðt � 1Þ þ Pðt � 1Þuðt � pÞuTðt � pÞPðt � 1Þk�pþ1 � uTðt � pÞPðt � 1Þuðt � pÞ

; Pð0Þ ¼ p0I : ð13Þ

In Table 1, we compare computational efficiency of the two algorithms (I: FF-FDW-RLS; II: simplifiedFF-FDW-RLS), where numbers of multiplications and additions are for each iteration step, and n is the num-ber of parameters to be estimated. From Table 1, it is clear that the simplified FF-FDW-RLS algorithm iscomputationally more efficient than the FF-FDW-RLS algorithm.

The bounded convergence of the simplified FF-FDW-RLS algorithm for time-varying parameter systemswas studied in [13] by using the stochastic process theory and martingale theory [18–22]. Of course, we alsomay analyze convergence properties of the FF-FDW-RLS algorithm by these tools.

4. Extension to other models

The model in (1) is the most general parameter identification model in systems and control because manysystems can be written in the form of (1) by parameterization. For example, for the popular ARX systems:

AðzÞyðtÞ ¼ BðzÞuðzÞ þ vðtÞ; ð14Þ

where u(t) is the system input, the definitions of y(t) and v(t) are as before, and A(z) and B(z) are polynomialsin the shift operator z�1[z�1y(t) = y(t � 1)] with

AðzÞ ¼ 1þ a1z�1 þ a2z�2 þ � � � þ ana z�na ;

BðzÞ ¼ b1z�1 þ b2z�2 þ b3z�3 þ � � � þ bnb z�nb :

Define the parameter vector h to be identified and information vector u(t) as follows:

h0 ¼ ½a1; a2; . . . ; ana ; b1; b2; . . . ; bnb �T 2 Rnaþnb ;

u0ðtÞ ¼ ½�yðt � 1Þ;�yðt � 2Þ; . . . ;�yðt � naÞ; uðt � 1Þ; uðt � 2Þ; . . . ; uðt � nbÞ�T:

Then the ARX model in (14) can be written in the form of (1):

yðtÞ ¼ uT0 ðtÞh0 þ vðtÞ:

Also, for the Box–Jenkins models

AðzÞyðtÞ ¼ BðzÞuðtÞ þ DðzÞCðzÞ vðtÞ; ð15Þ

where

CðzÞ ¼ 1þ c1z�1 þ c2z�2 þ � � � þ cnc z�nc ;

DðzÞ ¼ 1þ d1z�1 þ d2z�2 þ � � � þ dnd z�nd :

Let

eðtÞ :¼ DðzÞCðzÞ vðtÞ:

190 F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192

It follows that:

eðtÞ ¼ uTn ðtÞhn þ vðtÞ;

with

hn ¼ ½c1; c2; . . . ; cnc ; d1; d2; . . . ; dnd �T 2 Rncþnd ;

unðtÞ ¼ ½�eðt � 1Þ;�eðt � 2Þ; . . . ;�eðt � ncÞ; vðt � 1Þ; vðt � 2Þ; . . . ; vðt � ndÞ�T:

Define

h ¼h0

hn

� �2 Rn; uðtÞ ¼

u0ðtÞunðtÞ

� �; n :¼ na þ nb þ nc þ nd :

Then Eq. (15) can be rewritten as

yðtÞ ¼ uT0 ðtÞh0 þ eðtÞ ¼ uT

0 ðtÞhþ uTn ðtÞhn þ vðtÞ ¼ uTðtÞhþ vðtÞ:

However, a difficulty is that the unmeasurable noise terms e(t � i) and v(t � i) appear in u(t). In order to esti-mate h by the FF-FDW-RLS algorithm, these unknown e(t � i) and v(t � i) are replaced with their corre-sponding estimates; thus, based on noise estimation, we obtain a modified FF-FDW-RLS algorithm asfollows:

hðtÞ ¼ hðt � 1Þ þ PðtÞuðtÞ½yðtÞ � uT ðtÞhðt � 1Þ�;

PðtÞ ¼ 1

kP1ðt � 1Þ � P1ðt � 1ÞuðtÞuT ðtÞP1ðt � 1Þ

kþ uT ðtÞP1ðt � 1ÞuðtÞ

� �;

P1ðt � 1Þ ¼ Pðt � 1Þ þ Pðt � 1Þuðt � pÞuT ðt � pÞPðt � 1Þk�pþ1 � uT ðt � pÞPðt � 1Þuðt � pÞ

; Pð0Þ ¼ p0I ;

uðtÞ ¼u0ðtÞuT

n ðtÞ

" #;

h0ðtÞhnðtÞ

" #¼ hðtÞ;

u0ðtÞ ¼ ½�yðt � 1Þ; . . . ;�yðt � naÞ; uðt � 1Þ; . . . ; uðt � nbÞ�T;unðtÞ ¼ ½�eðt � 1Þ; . . . ;�eðt � ncÞ; vðt � 1Þ; . . . ; vðt � ndÞ�T;

eðtÞ ¼ yðtÞ � uT0 ðtÞh0ðtÞ;

vðtÞ ¼ yðtÞ � uT ðtÞhðtÞ or vðtÞ ¼ eðtÞ � uTn ðtÞhnðtÞ;

h0ðtÞ ¼ ½a1ðtÞ; a2ðtÞ; . . . ; anaðtÞ; b1ðtÞ; b2ðtÞ; . . . ; bnbðtÞ�T;

hnðtÞ ¼ ½c1ðtÞ; c2ðtÞ; . . . ; cncðtÞ; d1ðtÞ; d2ðtÞ; . . . ; dnd ðtÞ�T:

5. Examples

Two examples are given for performance studies of the proposed algorithms.

Example 1. Consider the following time-invariant system:

AðzÞyðtÞ ¼ BðzÞuðtÞ þ vðtÞ;AðzÞ ¼ 1þ a1z�1 þ a2z�2 ¼ 1� 1:60z�1 þ 0:80z�2;

BðzÞ ¼ b1z�1 þ b2z�2 ¼ 0:40z�1 þ 0:30z�2;

h ¼ ½a1; a2; b1; b2�T ¼ ½�1:60; 0:80; 0:40; 0:30�T:

{u(t)} is taken as a persistent excitation signal sequence with zero mean and unit variance r2u ¼ 1:002, and

{v(t)} as a white noise sequence with zero mean and variance r2v ¼ 0:202. Apply the proposed algorithms in

(7)–(10) and (11)–(13), respectively, to estimate the parameters of this system, the parameter estimation errors

F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192 191

da versus t are shown in Fig. 1, where the data length p = 100, the forgetting factor k = 0.98, the absoluteparameter estimation error da ¼ khðtÞ � hk, and the noise-to-signal ratio dns = 29.38%.

Example 2. Consider the following time-varying system:

Fig. 1.

Fig. 2.

AðzÞyðtÞ ¼ BðzÞuðtÞ þ vðtÞ;AðzÞ ¼ 1þ a1z�1 þ a2z�2 ¼ 1� 1:60z�1 þ 0:80z�2;

BðzÞ ¼ b1z�1 þ b2z�2 ¼ b1z�1 þ 0:30z�2;

h ¼ ½a1; a2; b1; b2�T ¼ ½�1:60; 0:80; b1; 0:30�T;

b1 ¼0:75; t < 1000;

0:25; t P 1000:

�

Simulation conditions are similar to those of Example 1, the results are shown in Fig. 2, where when b1 = 0.75,dns = 19.49%; when b1 = 0.25, dns = 37.41%.

0 500 1000 1500 2000 2500 30000

0.02

0.04

0.06

0.08

0.1

0.12

t

δ a

The parameter estimation errors da vs. t. Solid line: the FF-FDW-RLS algorithm; dots: the simplified FF-FDW-RLS algorithm.

0 500 1000 1500 2000 2500 30000

0.1

0.2

0.3

0.4

0.5

t

δ a

The parameter estimation errors da vs. t. Solid line: the FF-FDW-RLS algorithm; dots: the simplified FF-FDW-RLS algorithm.

192 F. Ding, Y. Xiao / Applied Mathematics and Computation 186 (2007) 184–192

From Figs. 1 and 2, we can see that the FF-FDW-RLS and approximate FF-FDW-RLS algorithms havealmost the same performance, but the latter requires less computational effort.

6. Conclusions

This paper presents and develops the recursive least squares algorithm with a forgetting factor over a finite-data window, which is general and can be used to identify various time-varying and time-invariant systems.

Acknowledgements

The authors are grateful to Professor Tongwen Chen in the Department of Electrical and Computer Engi-neering at the University of Alberta, Edmonton, Alberta, Canada, for his helpful suggestions and carefulrevision.

References

[1] F. Ding, T. Chen, Hierarchical gradient-based identification of multivariable discrete-time systems, Automatica 41 (2) (2005) 315–325.[2] F. Ding, T. Chen, Hierarchical least squares identification methods for multivariable systems, IEEE Transactions on Automatic

Control 50 (3) (2005) 397–402.[3] F. Ding, T. Chen, Combined parameter and output estimation of dual-rate systems using an auxiliary model, Automatica 40 (10)

(2004) 1739–1748.[4] F. Ding, T. Chen, Identification of dual-rate systems based on finite impulse response models, International Journal of Adaptive

Control and Signal Processing 18 (7) (2004) 589–598.[5] F. Ding, T. Chen, Parameter estimation of dual-rate stochastic systems by using an output error method, IEEE Transactions on

Automatic Control 50 (9) (2005) 1436–1441.[6] F. Ding, T. Chen, Modeling and identification for multirate systems, Acta Automatica Sinica 31 (1) (2005) 105–122.[7] F. Ding, T. Chen, Performance analysis of multi-innovation gradient type identification methods, Automatica 43 (1) (2007) 1–15.[8] F. Ding, Y. Shi, T. Chen, Performance analysis of estimation algorithms of non-stationary ARMA processes, IEEE Transactions on

Signal Processing 54 (3) (2006) 1041–1053.[9] F. Ding, T. Chen, L. Qiu, Bias compensation based recursive least squares identification algorithm for MISO systems, IEEE

Transactions on Circuits and Systems–II: express briefs 53 (5) (2006) 349–353.[10] G.C. Goodwin, K.S. Sin, Adaptive Filtering, Prediction and Control, Prentice-Hall, Englewood Cliffs, NJ, 1984.[11] F. Ding, T. Chen, Performance bounds of forgetting factor least squares algorithm for time-varying systems with finite measurement

data, IEEE Transactions on Circuits and Systems–I: Regular Papers 52 (3) (2005) 555–566.[12] L. Ljung, System Identification: Theory for the User, second ed., Prentice-Hall, Englewood Cliffs, NJ, 1999.[13] F. Ding, T. Ding, D. Xiao, J. Yang, Bounded convergence of finite-data-window least squares identification algorithms for time-

varying systems, Acta Automatica Sinica 28 (5) (2002) 754–761.[14] F. Ding, T. Chen, Hierarchical identification of lifted state-space models for general dual-rate systems, IEEE Transactions on Circuits

and Systems–I: Regular Papers 52 (6) (2005) 1179–1187.[15] F. Ding, T. Chen, Iterative least squares solutions of coupled Sylvester matrix equations, Systems & Control Letters 54 (2) (2005) 95–

107.[16] F. Ding, T. Chen, Gradient based iterative algorithms for solving a class of matrix equations, IEEE Transactions on Automatic

Control 50 (8) (2005) 1216–1221.[17] F. Ding, T. Chen, On iterative solutions of general coupled matrix equations, SIAM Journal on Control and Optimization 44 (6)

(2006) 2269–2284.[18] F. Ding, T. Chen, Parameter estimation for dual-rate systems with finite measurement data, dynamics of continuous, Discrete and

Impulsive Systems, Series B: Applications & Algorithms 11 (1) (2004) 101–121.[19] F. Ding, T. Chen, Least squares based self-tuning control of dual-rate systems, International Journal of Adaptive Control and Signal

Processing 18 (8) (2004) 697–714.[20] F. Ding, T. Chen, Identification of Hammerstein nonlinear ARMAX systems, Automatica 41 (9) (2005) 1479–1489.[21] F. Ding, Y. Shi, T. Chen, Gradient-based identification methods for Hammerstein nonlinear ARMAX models, Nonlinear Dynamics

45 (1-2) (2006) 31–43.[22] F. Ding, P. Liu, Y. Shi, Convergence analysis of estimation algorithms of dual-rate stochastic systems, Applied Mathematics and

Computation 176 (1) (2006) 245–261.