Eigen-decomposition of a classof

Infinite dimensional tridiagonal matrices

Eigen-decomposition of a classof

Infinite dimensional tridiagonal matrices

G.V. Moustakides: Dept. of Computer Engineering, Univ. of Patras, GreeceB. Philippe: IRISA-INRIA, France

G.V. Moustakides: Dept. of Computer Engineering, Univ. of Patras, GreeceB. Philippe: IRISA-INRIA, France

22

OutlineOutline

Definition of the problem.

From finite to infinite dimensions.

Reduction of the problem to a finite dimensional d.e. and eigen-decomposition problem, using Fourier transform.

Numerical aspects regarding the solution of the d.e. and the computation of the final (infinite dimensional) eigenvectors.

Conclusion.

33

Definition of the problemDefinition of the problem

2

2

t

t

t

t

t

t

r

r

r

r

A

A D I A 0

A D I A

Q A D A

A D I A

0 A D I A

A

I:D:A:

r:

identity matrix diagonal matrix general matrix

real scalar

real matrices of dimensions NN}

44

Eigen-decompositionEigen-decomposition

Eigenvalues:Eigenvalues: There is an infinite number.

Eigenvectors:Eigenvectors: There is an infinite number and each eigenvector is of infinite size.

Goal:Goal: To reduce the infinite dimensional eigen-decomposition problem into a finite one.

55

From finite to infinite dimensionsFrom finite to infinite dimensions

t

t

t

tK

t

t

Kr

r

r

Kr

D I A

A A 0

A D I A

Q A D A

A D I A

0 A A

A D I

QK has dimensions: (2K+1)N (2K+1)N, therefore we have (2K+1)N eigenvalue-eigenvector pairs.

Typical values: N = 100-1000, K = 5-10.

66

( ) ( ) ( )K i i ik k k Q

QK has dimensions: (2K+1)N (2K+1)N

i(k) has dimensions: (2K+1)N 1.

k = -K,…,K, i = 1,…,N.

( , ) ( , )

( ,1) ( ,1)

( )( ,0) ( ,0)

( , 1) ( ,1)

( , ) ( , )

i i

t

i i

ii i

ti i

i i

k K k K

Kr

k k

kk k

k k

Kr

k K k K

D I A 0

A

D

A

0 A D I

i(k,l) has dimensions: N 1. k,l= -K,…,K, i=1,…,N.

77

( 1) ( , 1)

( , )

( 1) ( , 1)

( , 1)

( ) ( , )

( , 1)

ti

ti

ti

i

i i

i

l r k l

lr k l

l r k l

k l

k k l

k l

A D I A 0

0 A D I A 0

0 A D I A

Consider now the infinite dimensional problem by letting K

Ai (k,l+1) + (D+lrI)i (k,l) + Ati (k,l-1) = i(k)i (k,l)

Ai (k,l+1) + Di (k,l) + Ati (k,l-1) = (i(k) -lr)i (k,l)

88

Reduction to finite dimensionsReduction to finite dimensionsReduction to finite dimensionsReduction to finite dimensions

Ai(k,l+1)+Di(k,l)+Ati(k,l-1) = (i(k)-lr)i(k,l)

A,D: NNi(k,l): N1i=1,…,N, k,l= -,…,

Key IdeaKey Idea

i(k) = i + kr without loss of generality assume 0 i ri(k,l) = i(l-k)

Ai(l-k+1)+Di(l-k)+Ati(l-k-1) = (i-(l-k)r)i(l-k)

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -nri(n)

99

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -nri(n)

(2)

(1)

, ;(0)

( 1)

( 2)

i

i

i i

i

i

i , {i(n)}, i=1,…,N, 0 i r

(1) (0)

(0) ( 1)

( ) , ; ( 2 ) , ;( 1) ( 2)

( 2) ( 3)

( 3) ( 4)

i i

i i

i ii i

i i

i i

r r

1010

Fourier TransformFourier Transform

( ) ( ) jn

n

X x n e

Let …, x(-2), x(-1), x(0), x(1), x(2),… be a real sequence. Then we define its Fourier Transform as

ImportanImportantt

( )( ) jn

n

dXnx n e j

d

( 2 ) ( )X X

( ) ( )jn jk

n

x n k e e X

1111

Ai(n+1)+(D-iI)i(n)+Ati(n-1) = -rni(n)

( ) ( ) jni i

n

n e

( )( ) ( ) ( ) ( )j t j i

i i i i

de e jr

d

A D I A

1( )( )j t ji

i i

djr e e

d

D A A I

1212

1( )( )j t ji

i i

djr e e

d

D A A I

1( )( ), (0)j t jd

jr e ed

Ψ

D A A Ψ Ψ I

1

( ) ( ) (0)ijri ie

Ψ

i() as being the Fourier transform of a (vector) sequence isnecessarily periodic with period 2.

We need i and i(0) to solve it.

1313

TheoremTheorem

0

( )( ) ( ), (0)

( 2 ) ( ).

dXX X X

d

B

B B

Consider the following linear system of d.e.

Let Z() be the transition matrix of the d.e., that is

( )( ) ( ), (0)

d

d

Z

B Z Z I

then we know that X()= Z()X0.

0 0(2 )X X Z

The solution X() is periodic if and only if X(2)=X(0)

1414

1 2(2 ) (0) (2 ) (0) (0)ijri i ie

Z Ψ

1 2(2 ) (0) (0)ijri ie

Ψ

1( )( )j t ji

i i

djr e e

d

D A A I

1( )( ), (0)j t jd

jr e ed

Ψ

D A A Ψ Ψ I

1

( ) ( )ijre Z Ψ

1515

Steps to obtain Steps to obtain ((i i ,{,{i i ((nn)}), )}), i=i=1,…,1,…,NN

Compute the transition matrix () from the d.e.

1( )( ), (0)j t jd

jr e ed

Ψ

D A A Ψ Ψ I

Find the eigenvalue-eigenvector pairs i, i(0) of

(2 ) (0) (0), 1, ,i i i i N Ψ

Form the desired eigenvalue-FT(eigenvector) pairs as

1

, ( ) ( ) (0)2

ijrii i ir e

Ψ

Use Inverse Fourier Transform to recover the final infinite eigenvector {i(n)} from i().

1616

Numerical aspectsNumerical aspectsNumerical aspectsNumerical aspects

Numerical solution of the d.e.

1( )( ), (0)j t jd

jr e ed

Ψ

D A A Ψ Ψ I

Eigen-decomposition of (2).

Computation of the Inverse Fourier Transform of i() where

1

( ) ( ) (0)ijri ie

Ψ

1717

Numerical solution of the d.e.Numerical solution of the d.e.

1( )( ), (0)j t jd

jr e ed

Ψ

D A A Ψ Ψ I

( )( ) ( ), (0) , ( ) Hermitian

dj

d

Ψ

B Ψ Ψ I B

One can show that () is unitary, therefore any numerical solution should respect this structure. A possible scheme is

1

( / 2)

( ) ( , ) ( ), (0)

( , )

n n n

je

B

Ψ M Ψ Ψ I

M

1818

( / 2)( , ) je BM

1

3 3

( ) (1 ) ,

, (1 2 )

( , ) ( )

12 2 2 1.3512

3

n n

n

n n

Ψ M

M

M Ψ

3 Step Integration. Yoshida scheme

1( ) ( , ) ( ), (0)n n n Ψ M Ψ Ψ I 1 Step Integration

1

1

12 2

2

1 1( )

2 2

1 1 1 1( )

2 12 2 12

e P

P

X X I X I X

X I X X I X X

Pade 1

Pade 2

1919

Pade 1, 1 step intgr.

Pade 2, 1 step intgr.

Pade 2, 3 step intgr.

Pade 1, 3 step intgr.

2020

Eigen-decomposition of Eigen-decomposition of (2(2))

Since (2) is unitary there are special eigen-decomposition algorithms that require lower computational complexity than the corresponding algorithm for the general case.

1

( ) ( ) (0)i njri n n ie

Ψ

From this problem we obtain the pairs i, i(0), i=1,…,N.

Using the solution () of the differential equation we can compute the Discrete Fourier Transform of the eigenvectors

Notice that we obtain a sampled version of the required Fourier transform.

2121

Inverting the Fourier TransformInverting the Fourier Transform

( ) ( ) jn

n

X x n e

Let …, x(-2), x(-1), x(0), x(1), x(2),… with Fourier Transform

If x(n)=0 for n < 0 and n M, then the Fourier Transform is equal

1

0

( ) ( )M

jn

n

X x n e

then the finite sequence x(n), n =0,…, M-1, can be completely recovered from a sampled version of the Fourier transform. Specifically we need only the samples

2, 0,..., 1X n n M

M

2222

21

0

1 2( ) , 0,..., 1

M jnkM

k

x n X k e n MM M

Complexity O(M 2).

For M=2m popular Fast Fourier Transform (FFT).Complexity O(M log(M)).

Apply Inverse Discrete Fourier Transform to i(n),

this will yield the desired vectors i(n).

If only a small number of i(n) is significant, then we apply Inverse Discrete Fourier Transform only to a subset of the vectors i(n) produced by the solution of the d.e.

Inverce discrete Fourier Inverce discrete Fourier TransformTransform

2323

ConclusionConclusionConclusionConclusion

We have presented as special infinite dimensional eigen- decomposition problem.

With the help of the Fourier Transform this problem was transformed into a d.e. followed by an eigen-decomposition both of finite size.

We presented numerical techniques that efficiently solve all subproblems of the proposed solution.

2424

E n D

E n D

Questions please ?Questions please ?Questions please ?Questions please ?