1

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

Dynamic system classifier

Daniel Pumpe, Maksim Greiner, Ewald Muller,Torsten Enßlin

22.01.2016

2

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

OUTLINE

INTRODUCTION

MODEL TRAINING

MODEL SELECTION

CONCLUSION

3

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MOTIVATION- COMPLEX SYSTEMS

To classify complex dynamical systems

3

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MOTIVATION- COMPLEX SYSTEMS

To classify complex dynamical systems

3

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MOTIVATION- COMPLEX SYSTEMS

To classify complex dynamical systems

3

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MOTIVATION- COMPLEX SYSTEMS

To classify complex dynamical systems

4

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

THE GOAL

To classify complex dynamical systems

s1 system classes

?

s2

?

s3

?

d data

0 5 10 15 20time [2π/ω]

−1000

−500

0

500

1000

1500

x(t

)[m

]

5

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

BAYES THEOREM

”Information is what forces a change in belief” by Caticha

P(s | d) =P(d | s)P(s)P(d)

6

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

STOCHASTIC DIFFERENTIAL EQUATION (SDE)

oscillating dynamical systems

d2x (t)dt2 + γ

dx (t)dt

+ ω2x (t) = F (t)

Operator form

xt = R(s)tt′ ξt′

(R(s)

tt’

)−1= δ(2)

(t− t′

)− γtδ

(1) (t− t′)

+ ω0 eβtδ(t− t′

)

6

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

STOCHASTIC DIFFERENTIAL EQUATION (SDE)

complex dynamical systems

d2x (t)dt2 + γ (t)

dx (t)dt

+ ω0 eβ(t)x (t) = ξ (t)

Operator form

xt = R(s)tt′ ξt′

(R(s)

tt’

)−1= δ(2)

(t− t′

)− γtδ

(1) (t− t′)

+ ω0 eβtδ(t− t′

)

6

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

STOCHASTIC DIFFERENTIAL EQUATION (SDE)

complex dynamical systems

d2x (t)dt2 + γ (t)

dx (t)dt

+ ω0 eβ(t)x (t) = ξ (t)

Operator form

xt = R(s)tt′ ξt′

(R(s)

tt’

)−1= δ(2)

(t− t′

)− γtδ

(1) (t− t′)

+ ω0 eβtδ(t− t′

)

7

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

TRAINING DATA

0 5 10 15 20time [2π/ω0]

−6000

−4000

−2000

0

2000

4000

6000

8000x(t)[m

]

x1

x2

x3

x4

x5

x6

8

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

CONSTRUCTION OF THE LIKELIHOOD

βt γt

s signal field

R(s) Response operator

xt training data

9

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

THE LIKELIHOOD OF A SDE

βt γt

s

R(s)

xt

P(x|s) = G(

x,R(s)†ΞR(s))

I temporarily structuredcovariance

I characterizes anon-stationary processes

10

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

THE PRIOR

τβ τβ

βt γt

s

P(βt|Ω) = G (βt,Ω)

assuming statistical stationarity:

Ω =∑

k

eτkΩk

11

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

A HIERARCHICAL PRIOR MODEL

αβ, qβinverse Gamma Distribution

P(eτ |α, q)

σβ smoothness enforcing

P(τ |σ)

τβ

βt

11

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

A HIERARCHICAL PRIOR MODEL

αβ, qβinverse Gamma Distribution

P(eτ |α, q)

σβ smoothness enforcing

P(τ |σ)τβ

βt

12

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

THE MODEL TRAINING

αβ, qβ σβ

τβ

βt

αγ , qγ σγ

τγ

γt

s

R(s)

xt

training

13

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

TRAINING DATA

0 5 10 15 20time [2π/ω0]

−6000

−4000

−2000

0

2000

4000

6000

8000x(t)[m

]

x1

x2

x3

x4

x5

x6

14

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

RECONSTRUCTED βREC

−1.0

−0.5

0.0

0.5

1.0

β[ lo

gω2/ω

2 0

]

Orginal βx1,2

x1,2,3

x1,2,3,4

x1,2,3,4,5,

x1,2,3,4,5,6

0 5 10 15 20time [2π/ω0]

−1.0

−0.5

0.0

0.5

Res

idua

ls

15

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

RECONSTRUCTED γREC

−1.0

−0.5

0.0

0.5

1.0γ[ 1/

ω2 0

] Orginal γx1,2

x1,2,3

x1,2,3,4

x1,2,3,4,5

x1,2,3,4,5,6

0 5 10 15 20time [2π/ω0]

−1.0

−0.5

0.0

0.5

1.0

Res

idua

ls

16

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

RECONSTRUCTED P(k)

10−1 100 101

|k|

10−28

10−26

10−24

10−22

10−20

10−18

10−16

10−14

10−12

10−10

10−8

10−6

10−4

10−2

100P(k)

Original P (k)

x1,2

x1,2,3

x1,2,3,4

x1,2,3,4,5

x1,2,3,4,5,6

17

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MODEL SELECTION

d = ROBSx + n = ROBS R(s) ξ + n

s s1 s2 s3

R(s)

xt

d

classification

P(si|d) = P(d|si)P(si)P(d)

17

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MODEL SELECTION

d = ROBSx + n = ROBS R(s) ξ + n

s s1 s2 s3

R(s)

xt

d

classification

P(si|d) = P(d|si)P(si)P(d)

17

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MODEL SELECTION

d = ROBSx + n = ROBS R(s) ξ + n

s s1 s2 s3

R(s)

xt

d

classification

P(si|d) = P(d|si)P(si)P(d)

18

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

THE BAYESIAN NETWORK OF DSC

αβ , qβ σβ

τβ

βt

αγ , qγ σγ

τγ

γt

s s1 s2 s3

R(s)

xt

d

training

classification

19

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

MODEL SELECTION- THE SYSTEM CLASSES

−700

−350

0

350

700

x(t)[

m]

x1

x2

x3

0 5 10 15 20time [2π/ω0]

−1.0

−0.5

0.0

0.5

β[ lo

gω2/ω

2 0

] ,γ[ 1/

ω2 0

]

γrec

βrec

−80

−40

0

40

x(t)[

m]

x1

x2

x3

0 5 10 15 20time [2π/ω0]

−1.3

−0.8

−0.3

0.2

0.7

β[ lo

gω2/ω

2 0

] ,γ[ 1/

ω2 0

]γrec

βrec

20

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

TEST CASE- SNR= 10

∆i,j = logP(d|si)− logP(d|sj)

0 5 10 15 20time [2π/ω0]

−30

−20

−10

0

10

20

30x(t)[m

]

SNR=10 ∆i,j=1 ∆i,j=2 ∆i,j=3ds1 0 -4352 -1757

20

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

TEST CASE- SNR= 10

∆i,j = logP(d|si)− logP(d|sj)

0 5 10 15 20time [2π/ω0]

−30

−20

−10

0

10

20

30x(t)[m

]

SNR=10 ∆i,j=1 ∆i,j=2 ∆i,j=3ds1 0 -4352 -1757

21

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

TEST CASE- SNR= 0.01

∆i,j = logP(d|si)− logP(d|sj)

0 5 10 15 20time [2π/ω0]

−600

−400

−200

0

200

400

600

800x(t)[m

]

SNR=0.01 ∆i,j=1 ∆i,j=2 ∆i,j=3ds1 0 -6 -5

22

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

PERFORMANCE OF DSC

∆i,j = logP(d|si)− logP(d|sj)

SNR=0.01 ∆i,j=1 ∆i,j=2 ∆i,j=3ds1 0 -6 -5ds2 -8 0 -10ds3 0 0 0SNR=0.1ds1 0 -144 -55ds2 -151 0 -139ds3 -1 -12 -0SNR=10ds1 0 -4352 -1757ds2 -6355 0 -5724ds3 -60 -136 0

23

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

CONCLUSION

I DSC algorithm is established:1. Analyzes training data from system classes to construct

abstract classifying information2. Confronts data with the system classes, to state the

probability which system class explains observations best

I The classification ability of the DSC-algorithm hassuccessfully been demonstrated in realistic numerical tests

I The DSC-algorithm should be applicable to a wide rangeof model selection problems

Thanks for your attention!

24

25

INTRODUCTION MODEL TRAINING MODEL SELECTION CONCLUSION

CLASSIFICATION- THE LIKELIHOOD

d = ROBSx + n = ROBS R(s) ξ + n .

P(d|si) =

∫DxP(d|x)P(x|si)

=

∫Dx G (d− ROBSx,N)

× G (x,R(s)† Ξ R(s))

∝ 1√|D|

exp(

12

j†Dj)

withj = R(s)†R†OBSN−1d

andD−1 = R(s)†R†OBSN−1ROBS R(s) +Ξ−1.