Reproducing Non-negative Dynamic SystemsCourse project for EECS E6891

Zhuo Chen

March 12, 2014

Review on Non-negative dynamic system

What it is?

I (Non-negative) matrix decomposition technic

I Dynamic(dependency) constraint in the activation

Why reproducing this?

I Good performance(in paper)

I Not open source

Possible applications?

I Source separation

What it really is?

Derivation

Model:

hn = Ahn−1 ◦ ξnvn = Whn ◦ εn

where A ∈ RK×K ,hn ∈ RK , vn ∈ RF , W ∈ RF×K

P (ξ1, ...ξN ) =∏k,n

G(ξkn|αk, βk)

P (ε1, ..., εN ) =∏f,n

G(εfn|υf , δf )

Derivation

P (hn|Ahn−1) =∏k

G(hkn|αk,βk∑

j akjhj(n−1))

P (vn|Whn) =∏f,n

G(εfn|υf ,δf∑

k wfkhkn)

Taking the expectation

E(hkn|Ahn−1) =αk

βk

∑j

akjhj(n−1)

E(vfn|Whn) =υfδf

∑k

wfkhkn

Then we can form the objective function(negative likelihood).The goal is to minimize the objective function.

L = − log(V |WH)− log(P (H|A))

Derivation

For wfk(as example)

Lwfk= −(

∑n,f

δ logvfn∑

k wfkhkn− δ

vfn∑k wfkhkn

)

which is the IS-divergence between V and WH. Using thelogarithm inequality and the convexity of inverse functions, wecan form the lower bound for L

L(w′fk) ≤ L(wfk)−∑nf

∑k wfkhkn −

∑k w′fkhkn∑

k wfkhkn

+vfn

(∑

k wfkhkn)2(∑k

wfkhkn −∑k

w2fk

w′fkhkn)

By taking the derivative of w′fk, we can get the update rule forwfk.

Hierarchical extension

Source-filter extension

Implementation and resultI The implementation of basic NDS model in Matlab and

Python.I Result depend heavily on the initializationI Numerical problemsI Haven’t finish the larger scale experiment

One example output:

50 100 150 200 250 300 350

20

40

60

80

100

120

2 4 6 8 10 12 14 16 18 20

5

10

15

20

2 4 6 8 10 12 14 16 18 20

20

40

60

80

100

120

50 100 150 200 250 300 350

20

40

60

80

100

120

Todo list

I Optimizing the code

I HNDS and SFNDS

I Reproduce the results in the NDS paper(speechenhancement)

Thanks!

Questions?