System Representation by a set of Low Dimensional Models Miroslav Svítek

Czech Technical University in Prague, Faculty of Transportation Sciences, Konviktská 20, 110 00 Prague 1

1. Introduction The paper combines the approaches known in signal

processing area with the approaches typical for neural networks. In the signal processing community the signal parameters as mean values, correlation matrix, covariance matrix, regression parameters vector, etc. are estimated through measured data. The knowledge of such parameters results in the identification of the optimal LTI (linear time invariant) model describing the main system property [1]. The model with estimated parameters could be used for time interpolation, filtering, extrapolation, etc.

2. Sylvester's theorems

The decomposition methodology described in this chapter is based on the application of Sylvester's theorems on system transition matrix A of m-dimensional linear time invariant system described by state-space model:

nnn

nn1n

ux yuxx

•+•=•+•=+

DCBA (1)

where nnn y,u,x are m-dimensional state, input and output vectors in time interval n and A,B,C,D are state-space mm ⋅ matrices. Theorem 1: Sylvester theorem for distinct eigenvalues If A is square system transition matrix (1) and if i. represents one of the n distinct eigenvalues of A, and if P(A) is any polynomial of the matrix A, then ∑ .∑

. = .=

.

.−.•.−

•.=.−.

•.−•.=n

1i

n

ij ji

ji

n

1in

ijij

ii

)()(

)(P)(

)(Adj)(P)(PIAIAA

(2) where Adj(A) means the adjoin matrix that is formed by replacing each element of matrix A by its cofactor and then taking the transpose. Theorem 2: Sylvester theorem for repeated eigenvalues If A is square system transition matrix (1) and if i. represents an eigenvalue of A repeated is times, and if k is number of all distinct eigenvalues j. , and if P(A) is any polynomial of the matrix A, then

i

i

ik

)seigenvaluedistinct all(

1i k

ijj

1s

1s

i )(

)(Adj)(Pdd

)!1s()1()(P

.=.

=

.

−

−

∑.

.−.

•.−•..−

−= IAA

(3) where Adj(A) means the adjoin matrix and if all

eigenvalues are equal, then 1)(k

ijj =.−..

.

.

Definition 1: With respect to equation (2) and with assumption of distinct eigenvalues the special matrixes

n21 Z,....,Z,Z could be defined for matrix A:

.. .−.

•.−=

n

ij ji

ji )(

)(Z

IA (4), with following properties:

∑=

=

==•

.=•

n

1ii

iji

ji

.1Z

j,ifor ZZZ

j,ifor 0ZZ (5)

Proof: The proof of theorem 1 and 2 is done in [4] where the proof of matrix components for repeated eigenvalues is also presented. 3. Approximation of derivatives

In equation (3) the derivatives are necessary to be solved. In following theorem the form of derivatives approximation is presented together with the approximation error. Theorem 3: Approximation of derivatives Let f(x) is any function of x and

....dx

)x(fd,dx

)x(fd,dx

)x(df3

3

2

2

first, second, third, etc.

derivatives of function f(x), then the approximate derivatives could be expressed

( )

( )

( ).etc

,)h2x(f)hx(f2)hx(f2)h2x(fh21

dx)x(fd

,)hx(f)x(f2)hx(fh21

dx)x(fd

,)hx(f)hx(fh21

dx)x(df

33

3

22

2

−−−++−+

−+−+

−−+

(6)

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)

where for all derivatives the approximation error is proportional to ( )2h. . For better precision of approximation other approximate forms exist, e.g. for approximation error proportional to

( )4h. the following equations could be described:

( )

( )etc.

,)h2x(f)hx(f16)x(f30)hx(f16)h2x(fh121

dx)x(fd

,)h2x(f)hx(f8)hx(f8)h2x(fh12

1dx

)x(df

22

2

−−−+−+++−

−+−−+++−

(7) Proof: The proof is given in [4]. Theorem 4: Complex-step approximation of derivatives

Let f(x) is any function of x and dx

)x(df is first

derivative of function f(x), then the approximate derivatives with complex-step approximation could be expressed:

( ) ( )[ ] ( )[ ]

hhjxfIm

hhjxfImlim

dxxdf

0h

•+•+=.

(8) where Im[.] means imaginary part. Then the approximation error is proportional to ( )2h. . Proof: Let us express yjxz •+= and

( ) vjuzf •+= together with Cauchy-Riemann equations [6]:

yv

xu

.

.=..

(9), xv

yu

.

.−=..

(10)

If f is analytic function (satisfy the (9) and (10)) than we can use and rewrite (9) as follows:

( )( ) ( )

hyjxvhyjxvlim

xu

0h

•+−+•+=..

. (11)

Since function f is real function of a real variable than 0y = , ( ) ( )xfxu = and ( ) 0xv = . Equation (11) can be

than rewritten to (8). In order to determine the error involved in this approximation, the derivation based on Taylor series expansion is used (with pure imaginary step equal to hj• ):

( ) ( ) ( ) ( ) ( ) ........dx

xfd!3

1hjdx

xfd!2

1hdx

xdfhjxfhjxf 3

33

2

22 −•••+••−••+=•+

(12) Taking imaginary part of both sides of equation (12) and dividing this equation by h yields:

( ) ( )[ ] ( ) .....

dxxfd

!31h

hhjxfIm

dxxdf

3

32 +••+•+= (13)

Hence the approximation is ( )2h. . The higher order derivatives express in Theorem 3 could be transferred into complex-step method by using approximation of Cauchy's Integral Formula in general form ( Γ is a simple closed positively oriented contour that encloses z):

( ) ( )( ) ∑∫

−

=••π••

•π••

Γ+

•+

••

.−.

.•π•

=1m

0i mni2j

mi2j

1nn

n

e

erzf

rm!nd

zf

j2!n

dzzfd

(14) where m is the number of points used in the integration. the approximate of derivative of order

1m,....,1,0n −= could be fond by using ( 14). From complex variable theory, for a real function of the real variable that is analytic holds:

( ) ( ) vjuyjxfvjuyjxf •−=•−⇒•+=•+ (15).

4. Approximation of derivatives in Sylvester theorem for repeated eigenvalues (transformed eigenvalues method) The approximation of derivatives (6), (7) could be applied to Sylvester theorem for repeated eigenvalues (3) and the following theorem could be defined. Theorem 5: Approximation of P(A) with repeated eigenvalues of matrix A If matrix A has k all distinct eigenvalues (3) where d eigenvalues d. are repeated ds times and ch eigenvalues ch. are poorly distinct, then the polynomial function P(A) could be approximated by set of ch distinct and by set of t transformed distinct eignevalues t. ( hq,....,hq,..,hq,..,hq dddd1111 +.−.+.−. ) with

error proportional at least to )h(O 2 as follows:

∑ ∑∑

∑ ∑ .∑ .

= −=...

=

= −=. .∝ ∝

∝.

= .∝ ∝

∝

••.+.•+•.=

=.−•.+.

•.−••.+.•+

.−.•.−

•.=

d

1f

q

q,ff,fi

ch

1ii

d

1f

q

q

k

f ff,f

ch

1i

k

i ii

f

f

f

f

Z)h(PkZ)(P

)h()(

)h(Pk)()(

)(P)(PIAIA

A

(16) where fq depends on selected approximate form (6), (7), (8) and on the multiplicity of repeated eigenvalue f. ,

.,fk are weight constants of approximation form (6), (7),

(8) ..∝ ∝

∝. .−•.+.

•.−=

k

f f,f )h(

)(Z

IA are transformed

component matrices assigned to transformed distinct

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)

eigenvalues and h is selected small approximation parameter. Proof: If the Sylvester theorem for repeated eigenvalues (3) is used and if the approximation according to equation (6), (7) or (8) is applied then the part of equation (3) assigned to repeated eigenvalue i. could be approximated with

error proportional at least to )h(O 2 or more (depends on used approximate form, e.g. forms (6), (7) or (8)) as follows:

iiii

i

i

i

i

i

i

q,iiiq0,ii0q,iiiq

1s

1s

ik

ijj

1s

1s

i

Z)hq(Pk..Z)(Pk..Z)hq(Pk

)(Z)(Pdd

)!1s()1(

)(

)(Adj)(Pdd

)!1s()1(

••+.•++•.•++••−.•=

=

.•.•

.•

−−=

.−.

•.−•..−

−

−−

.=.−

−

.=..

−

−

.IA

(17) where hq,...,,..,hq iiiii •+..•−. are transformed eigenvalues assigned to is times repeated eigenvalue i. and

ii q,i0,iq,i Z,...,Z,..,Z − are transformed component matrices assigned to transformed eigenvalues

hq,...,,..,hq iiiii •+..•−. . 5. Approximation based on modified repeated eigenvalues (modified eigenvalues method)

The Sylvester' theorem came out from Lagrange interpolation polynomial with distinct polynomial roots. The repeated roots in Lagrange polynomial could be modified and approximated by distinct roots according to following table with defined approximation error (it is easy to extend the table for higher root multiplicity):

In case the parameter h is small enough the approximation error could be also small because of high power of h. This methodology could be applied to Sylvester's theorem and the modified eigenvalues could

be used instead of repeated ones and then the decomposition could be done with help of equation (2). As an example following table describes the multiplicity of original repeated eigenvalue and its modification to distinct ones: Tab. 2 Original repeated and modified distinct eigenvalues

Multiplicity of original repeated eigenvalues

Modified distinct eigenvalues

2. h ,h +.−. 3. h , ,h +..−. 4. 2h h, h,- ,h2 +.+..−.5. 2h h, , h,- ,h2 +.+...−.

etc. etc. The LTI dynamical system decomposition is based on

the application of Sylvester's theorems or its approximation to state-space matrix A (1) and by using the property of components matrices to calculate the transformed input, output and state-space vectors of one-dimensional models.

6. The LTI system decomposition with distinct eigenvalues of transition matrix

The LTI system decomposition with distinct eigenvalues could be done with help of following fundamental decomposition theorem. Theorem 6: Fundamental decomposition of LTI dynamical systems with distinct eigenvalues of transition matrix A Dynamical m-dimensional LTI dynamical system described by state-space model (1) with transition matrix A with distinct eigenvales could be decomposed into

2m one-dimensional LTI models where the component matrices assigned to transition matrix A are used as transformation matrices of state, input and output vectors (filter banks).

Tab.1: Modified roots in Lagrange polynomial [4]

modified roots/

repeated roots

(a-2h)

(a-h)

a

(a+h)

(a+2h)

approx.

error

2)ax( + 0 ))ha(x( −+ 0 ))ha(x( ++ 0 2h 3)ax( + 0 ))ha(x( −+ )ax( + ))ha(x( ++ 0 )ax(h2 + 4)ax( + ))h2a(x( −+

))ha(x( −+ 0 ))ha(x( ++ ))h2a(x( ++

22

4

)ax(h5h4

+−

−

5)ax( + ))h2a(x( −+

))ha(x( −+ )ax( + ))ha(x( ++ ))h2a(x( ++ 32

4

)ax(h5)ax(h4

+−−+

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)

Proof: The m-dimensional state-space model (1) with square

mm ⋅ matrices A, B, C, D1 could be decomposed with help of components matrices m21 Z,...,Z,Z into following form

nmn1nmn2n1

nmn2n1n,mn,2n,1

nmn2n1nmmn22n11

1nm1n21n1

uDZ...uDZxCZ..xCZxCZyZ....yZyZy..yy

uBZ..uBZuBZxZ...xZxZxZ...xZxZ

••++••+••++••+••==•++•+•=+++

••++••+••+.++.+.==+++ +++

(18) where form P(A)=A in equation (2) was used and the

matrix components property ∑=

=m

1ii 1Z (5) was taking

into account. By multiplying the equations (18) by component matrix jZ and by taking into account the

property jifor 0ZZ ji .=• and

jifor ZZZ iji ==• (5) the LTI dynamical system could be decomposed into m following sub-systems:

njnjnjn,j

njnjj1nj

uDZxCZyZy

uBZxZxZ

••+••=•=

••+••.=• +

(19) with transition value equal to j. . The transformed m-

dimensional state vectors nj xZ • { }m,...,2,1j . were obtained through filtering of state vector by component matrices (component matrices play the role of filter banks). Each component of transformed state vector

nj xZ • could be easily described as a first-order

dynamical model because the transition value j. is

common for all m transformed states nj xZ • . 7. Identification methods of low dimensional models

The identification task yields to estimation of s-dimensional vector unknown parameters:

[ ]m21m21m21m21 ,...,,,p,...p,p,b,...,b,b,a,...,a,aw ...=r

(20)

based on the knowledge of data vector: ]x,....,x,x,x,...,x,x[x 1i,m1i,21i,1i,mi,2i,1i −−−=r

(21)

where the transformed state-vector fulfill the conditions on m one-dimensional models:

1 for non-square matrices B,C,D, the zero elements could be completed as well as in vectors nn y,u to achieve the form (4.1)

•

•

.

..

=

•

−

−

−

1i,m

1i,2

1i,1

mmm

222

111

m

2

1

i,m

i,2

i,1

mmm

222

111

x..

xx

p..ba........p..bap..ba

0..00..........00..0

x..

xx

p..ba........p..bap..ba

(22) The parameters must be estimated under the pre-

defined composition rule:

•

•

=

i,m

i,1

mm

11

m,m1,m

m,11,1

i,m

i,1

x..

x

p..a......p..a

r..r......

r..r

x..

x

(23) where parameters m,m1,m1,1 r,....,r,...,r are given

beforehand. The (22) could be rewritten into following form:

( )( )

( )

•

•

.

..

−

•

=

−

−

−

1i,m

1i.2

1i,1

mmm

222

111

m

2

1

i,m

i.2

i,1

mmm

222

111

im

i2

i1

x..

xx

p..ba........p..bap..ba

0..00..........00..0

x..

xx

p..ba........p..bap..ba

w,xf..

w,xfw,xf

rr

rr

rr

(24) with the additional functions representing

composition rule: ( )

( )

•

•

−

=

+

i,m

i,1

mm

11

m,m1,m

m,11,1

i,m

i,1

in

i1m

x..

x

p..a......p..a

r..r......

r..r

x..

x

w,xf..

w,xf

rr

rr

(25) The (24) and (25) are set of n non-linear functions

that could be linearized by matrix Taylor series as follows (from (24) and (25) it arises that the functions

( ) ( )ini1 w,xf,....,w,xf rrrrmust converge to zero vector):

( )( )

( )

( )( )

( )

( ) ( ) ( )

( ) ( ) ( )

−

•

..

..

..

..

..

..

+

=

−

−

−

=

−

−

−

−

0..00

w..

ww

w..

ww

ww,xf..

ww,xf

ww,xf

..........w

w,xf..w

w,xfw

w,xf

w,xf..w,xfw,xf

w,xf..

w,xfw,xf

1i,s

1i,2

1i,1

i,s

i,2

i,1

wws

in

2

in

1

in

s

i1

2

i1

1

i1

1iin

1ii2

1ii1

iin

ii2

ii1

1irr

rrrrrr

rrrrrr

rr

rr

rr

rr

rr

rr

(26) In next part we mark the vector of unknown

parameters in i-time step as iwr :

[ ]Tisiii wwww ,,2,1 ...=r

(27)

The measurement vector will be marked as 1iz −r

and transition matrix between measurement and parameter vector as 1iD − . The vector 1iz −

rand matrix 1iD − are

computed from known data vector ixr and last estimate of parameters 1iw −

ras follows:

( )( )

( )

−

•

..

..

..

..

..

..

=

−

−

−

−

−

−

=

−

−

1iin

1ii2

1ii1

1i,s

1i,2

1i,1

wws

in

2

in

1

in

s

i1

2

i1

1

i1

1i

w,xf..w,xfw,xf

w..

ww

w)w,x(f..

w)w,x(f

w)w,x(f

..........w

)w,x(f..w

)w,x(fw

)w,x(f

z

1i

rr

rr

rr

rrrrrr

rrrrrr

r

rr

(28)

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)

1iwws

in

2

in

1

in

s

i1

2

i1

1

i1

1i

w)w,x(f..

w)w,x(f

w)w,x(f

..........w

)w,x(f..w

)w,x(fw

)w,x(f

D

−=

−

..

..

..

..

..

..

=

rr

rrrrrr

rrrrrr

(29) Based on representation (28) and (29) the

equation for extended Kalman filter [14] could be written:

ii1i1i ewDzrrr +•= −−

(30) where noise vector ie

r is supposed to be Gaussian

with zero mean and covariance matrix Q: [ ] [ ] ji ,cov j,i 0,cov ==.= Qeeee jiji

rrrr

(31) The time evolution of parameters vector is supposed

to be random walk: i,w1ii eww

rrr += − (32),

where noise vector i,wer is supposed to be Gaussian with zero mean and covariance W:

[ ] [ ] ji ,cov j,i 0,cov ,,,, ==.= Weeee jwiwjwiwrrrr

(33). The extended Kalman estimation filter could be for

the studied case written in following form: ( )

( ) 1T1ii1i

T1i1ii

1ii

1i1i1ii1ii

WDSDDSH

QSSwDzHww

−−−−−

−

−−−−

+••••=

+=•−•+= rrrr

(34)

where a priory information 11 S,wr must be known (estimated) in advanced.

8. Examples of low dimensional identification Let us define the LTI system with two repeated eigenvalues 121 == .. as follows with matrix (23)

−

=

+

•

−

=

−

−

25.0

xx

,ee

xx

2110

xx

2,0

1,0

i,2

i,1

1i,2

1i,1

i,2

i,1

(35)

−

=

11

11rrrr

2,21,2

2,11,1 (36)

In this example the parameter vector ),b,a,,b,a(w 222111 ..=r

will be estimated by identification method. The equations (26) could be

rewritten for studied example as follows (in this example m=2, n=4):

−•−+•−−•++•+

•+••.−+•+••.−+

=

−−−−

−−−−

−−−−−−−

−−−−−−−

−

−

−

−

i,1i,21i,21i,1i,11i,21i,1

i,1i,21i,21i,1i,11i,21i,1

1i,21i,21i,11i,21i,2i,21i,2i,11i,2

1i,21i,11i,11i,11i,1i,21i,1i,11i,1

1ii4

1ii3

1ii2

1ii1

xx)bb(x)aa(xx)bb(x)aa(

)xbxa(ˆxbxa)xbxa(ˆxbxa

)w,x(f)w,x(f)w,x(f)w,x(f

rr

rr

rr

rr

(37) where )ˆ,b,a,ˆ,b,a(w 1i,21i,21i,21i,11i,11i,11i −−−−−−− ..=r

is last estimate of parameters vector. In simulation mode the noise covariance was selected:

=

1001

Q (38)

In Fig. 1 the approximate and original evolution of state component i,1x is shown where the evolution of estimated vector parameters is shown in Fig. 2.

0 5 10 15 20 25 30-140

-120

-100

-80

-60

-40

-20

0

20Evolution of real and approximated system

Time interval

Evo

lutio

n of

real

and

app

roxi

mat

ed s

tate

com

pone

nt X

1

Fig.1 Approximate (+) and original evolution of state

components i,1x

0 10 20 30 40 50 60 70 80 90 100-0.5

0

0.5

1

1.5

2Evolution of estimated parameters

Time interval

Evo

lutio

n of

est

imat

ed p

aram

eter

s

Fig. 2 Evolution of estimated parameters with result:

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)

4457.0a 100,1 = , 5540.0b 100,1 = ,

0064.1ˆ100,1 =. , 4767.0a 100,2 = , 4768.0b 100,2 −= ,

9687.0ˆ100,2 =. The same method was used for system with two

complex conjugate eigenvalues 866.0j5.01 •+=. , 866.0j5.02 •−=. as follows

−

=

+

•

−

=

−

−

25.0

xx

,ee

xx

1110

xx

2,0

1,0

i,2

i,1

1i,2

1i,1

i,2

i,1

(39) Fig. 3 shows the evolution of approximate and original state component i,1x and Fig. 4 shows the evolution of parameters vector.

0 10 20 30 40 50 60 70 80 90 100-10

-8

-6

-4

-2

0

2

4

6

8

10Evolution of real and approximated system

Time interval

Evo

lutio

n of

real

and

app

roxi

mat

ed s

tate

com

pone

nt X

1

Fig.3 Approximate (+) and original

evolution of state components i,1x

0 10 20 30 40 50 60 70 80 90 100-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6Evolution of estimated parameters

Time interval

Evo

lutio

n of

est

imat

ed p

aram

eter

s

Fig.4 Evolution of estimated parameters with result: 3637.0a 100,1 = , 3626.0b 100,1 = ,

5165.0ˆ100,1 =. , 3619.0a 100,2 = ,

3588.0b 100,2 −= , 4923.0ˆ100,2 =.

9. Conclusion The presented results have shown the theory of LTI

systems decomposition for distinct and repeated eigenvalues of transition matrix A of state-space model together with identification algorithm. This theory vindicates much known practice of mixtures of low dimensional dynamical models to approximate the higher order dynamical system. In paper the direct proof of LTI dynamical system decomposition with distinct and repeated eigenvalues of matrix A was presented as fundamental decomposition theorem. The decomposition theory was demonstrated on numerical example together with identification method.

References

[1] Peterka V.: Bayesian Approach to System Identification.

In: Eykfoff P. (ed.): Trends and Progress in System Identification, Pergamon Press Oxford, 239-304.

[2] Function of matrices www.ece.utexas.edu/~ari/Courses/EE380K/Functions_of_Matrices.pdf]

[3] Martins J., Kroo I., Alonso J.: An Automated Method for Sensitivity Analysis Using Complex Variables, American Institute of Aeronautics and Astronautics, AIAA-2000-0689, 2000.

[4] Svítek M: The Decomposition Theory of LTI Systems, Neural Network World, 6/04, pp. 489-506, 2004.

Proceedings of the 5th WSEAS/IASME Int. Conf. on SYSTEMS THEORY and SCIENTIFIC COMPUTATION, Malta, September 15-17, 2005 (pp342-347)