CY3A2 System identification 1

Derivation of Recursive Least Squares

Given that is the collection

Thus the least squares solution is

Now what happens when we increase n by 1, when a new data point comes in, we need to re-estimate this requires repetitions calculations and recalculating the inverse (expensive in computer time and storage)

n

n

T

nn

T

nn

ˆ y 1

1

n

T

n

T

T

n 2

1

CY3A2 System identification 2

Lets look at the expression andand define

n

T

n

n

T

nnP 1

T

nnn

T

nn

n

i

T

ii

n

i

T

ii

T

n

T

T

n

n

T

nn

P

P

1

1

1

11

2

1

21

1

nnn

T

n

nn

n

iii

n

iii

n

n

n

T

n

y

yyy

y

y

y

11

1

11

2

1

21

y

y

n

T

ny

CY3A2 System identification 3

nnn

T

nnn

T

nnn

T

nn

T

nnyPPˆ

11

1

yyy

The least squares estimate at data n

111

1

11111

n

T

nnnn

T

nnn

ˆP,Pˆ yy Because

nnn

n

T

nnnnn

nnn

T

nnnnn

nnnnnn

Kˆ

ˆyPˆ

yˆˆPP

yˆPPˆ

1

11

11

1

1

1

1

T

nnnnPP

1

1

1 T

nnn

T

nn

T

n

11yy(1) (2)

(3)

(4)

( Substitute (4) into (3) )

( Applying (1) )

CY3A2 System identification 4

(8)

(7)

(6)

(5)

11

1

1

1

T

nnnn

nnn

nnnn

n

T

nnn

PP

PK

Kˆˆ

ˆy

RLS Equations are

But we still require a matrix inverse to be calculated in (8)

Matrix Inversion Lemma

If A, C, BCD are nonsigular square matrix ( the inverse exists) then

1111111 ------- DAB]DAB[C-A A BCD][A

CY3A2 System identification 5

The best way to prove this is to multiply both sides by [A+BCD]Now, in (8), identify A, B,C,D

nn

T

n

n

T

nnn

n

n

T

nnn

T

nnnn

------T

nnnn

T

n

nn

T

nnnn

P

PPP

PPPP

DAB]DAB[C-AAPP

D,C

B,PA

PP

1

11

1

1

1

111

11111111

1

1

1

11

1

1

1

1

:Note

CY3A2 System identification 6

I

DAB]DA[CBDACBCBCDAI

DAB]DA[CBCDAIBBCDAI

DAB]DAB[CBCDA

DAB]DAB[CBCDAI

DAB]DAB[C-AABCD] [A

------

------

-----

-----

------

1111111

111111

11111

11111

111111

Matrix inversion lemma is very important in convert LS into RLS. To prove the above,

CY3A2 System identification 7

(9)

(7)

(6)

(5)

nn

T

n

n

T

nnn

nn

nnn

nnnn

n

T

nnn

P

PPPP

PK

Kˆˆ

ˆy

1

11

1

1

1

1

RLS equations are

In practice, this recursive formula can be initiated by setting to a large diagonal matrix, and by letting be your best first guess.

0P

0̂

CY3A2 System identification 8

RLS with forgetting

We would like to modify the recursive least squares algorithm so that older data has less effect on the coefficient estimation. This could be done by biasing the objective function that we are trying to minimise (i.e. the squared error)

This same weighting function when used on an ARMAX model can be used to bias the calculation of the Pn matrix giving more recent values greater prominence, as follows.

where λ is chosen to be between 0 and 1.

V n i– i2

i 1=

n

=

n

i

T

ii

in

nP

1

1

CY3A2 System identification 9

When λ is 1 all time steps are of equal importance but as λ smaller less emphasis is given to older values. We can use this expression to derive a recursive form of weighted

The Matrix inversion lemma will then give a method of calculating given to get

1nP

nP

T

nnn

T

nn

n

i

T

ii

inn

i

T

ii

in

n

P

P

1

1

1

11

1

nn

T

n

n

T

nnn

nn P

PPPP

1

11

1

1

CY3A2 System identification 10

(4)

(3)

(2)

(1)

nn

T

n

n

T

nnn

nn

nnn

nnnn

n

T

nnn

P

PPPP

PK

Kˆˆ

ˆy

1

11

1

1

1

1

RLS Algorithm with forgetting factor: