Parameterized Complexity of Matrix Factorization

David P. Woodruff

IBM Almaden

Based on joint works with Ilya Razenshteyn, Zhao Song, Peilin Zhong

Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 1 / 50

Outline

Frobenius norm factorization can be done in polynomial time

Intractable matrix problems

I Nonnegative matrix factorization(NMF)

F Arora-Ge-Kannan-Moitra’12, Moitra’13

I Weighted low rank approximation(WLRA)

F Razenshteyn-Song-W’16

I `1 low rank approximation(`1LRA)

F Song-W-Zhong’16

Outline

Frobenius norm factorization can be done in polynomial timeIntractable matrix problems

F Arora-Ge-Kannan-Moitra’12, Moitra’13

F Song-W-Zhong’16

Outline

F Arora-Ge-Kannan-Moitra’12, Moitra’13I Weighted low rank approximation(WLRA)

F Song-W-Zhong’16

Outline

I Nonnegative matrix factorization(NMF)F Arora-Ge-Kannan-Moitra’12, Moitra’13

F Song-W-Zhong’16

Outline

F Razenshteyn-Song-W’16I `1 low rank approximation(`1LRA)

F Song-W-Zhong’16

Outline

I Weighted low rank approximation(WLRA)F Razenshteyn-Song-W’16

F Song-W-Zhong’16

Outline

F Song-W-Zhong’16

Outline

I `1 low rank approximation(`1LRA)F Song-W-Zhong’16

Frobenius Norm Matrix Factorization

QuestionGiven a matrix A ∈ Rn×n and k > 1, the goal is to output two matricesU ∈ Rn×k ,V ∈ Rk×n such that

‖UV − A‖2F 6 (1 + ε) minrank−k A ′

‖A ′ − A‖2F ,

where ‖A‖2F = (∑n

i=1∑n

j=1 A2i,j)

2, ε ∈ [0,1)

Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries

‖A ′ − A‖2F ,

i=1∑n

j=1 A2i,j)

2, ε ∈ [0,1)

Useful for data compression, easier to store factorizations U,V

Can be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries

‖A ′ − A‖2F ,

i=1∑n

j=1 A2i,j)

2, ε ∈ [0,1)

Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... time

Fails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries

‖A ′ − A‖2F ,

i=1∑n

j=1 A2i,j)

2, ε ∈ [0,1)

Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorization

Not robust to noise, outliers, or missing entries

‖A ′ − A‖2F ,

i=1∑n

j=1 A2i,j)

2, ε ∈ [0,1)

Useful for data compression, easier to store factorizations U,VCan be solved using the singular value decomposition (SVD) inmatrix multiplication n2.376... timeFails if you want a nonnegative factorizationNot robust to noise, outliers, or missing entries

Nonnegative Matrix Factorization

Example of NMF. .

. .Razenshteyn-Song-Woodruff-Zhong Parameterized Complexity of Matrix Factorization 5 / 50

Example of NMF. .

Given : A ∈ Rn×n, n = 4, k = 2

Example of NMF. .

Given : A ∈ Rn×n, n = 4, k = 2

0 1 1 11 2 1 12 4 2 23 5 2 2

Example of NMF. .

Given : A ∈ Rn×n, n = 4, k = 2

0 1 1 11 2 1 12 4 2 23 5 2 2

Question : Are there two matrices U,V> ∈ Rn×k , such thatUV = A,U > 0,V > 0

Example of NMF. .

Given : A ∈ Rn×n, n = 4, k = 2

0 1 1 11 2 1 12 4 2 23 5 2 2

0 11 12 23 2

· [1 1 0 00 1 1 1

Example of NMF. .

Given : A ∈ Rn×n, n = 4, k = 2

0 1 1 11 2 1 12 4 2 23 5 2 2

0 11 12 23 2

· [1 1 0 00 1 1 1

Main Question, Nonnegative Matrix Factorization

QuestionGiven matrix A ∈ Rn×n and k > 1, is there an algorithm that candetermine if there exist two matrices U,V> ∈ Rn×k ,

UV = A,U > 0,V > 0.

Or, are there any hardness results?

Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learningApplications

I Text mining, Spectral data analysis, Scalable Internet distanceprediction, Non-stationary speech denoising, Bioinformatics,Nuclear imaging, etc.

UV = A,U > 0,V > 0.

Equivalent to computing the nonnegative rank of A, rank+(A)

Fundamental question in machine learningApplications

UV = A,U > 0,V > 0.

Equivalent to computing the nonnegative rank of A, rank+(A)Fundamental question in machine learning

Applications

UV = A,U > 0,V > 0.

Rank vs. Nonnegative Rankrank(A) is the rank and rank+(A) is the nonnegative rank

I rank(A) 6 rank+(A)I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.

I rank(A) 6 rank+(A)

I Vavasis’09, determining whether rank+(A) = rank(A) is NP-hard.

1 1 0 01 0 1 00 1 0 10 0 1 1

1 1 01 0 10 1 00 0 1

·1 0 0 −1

0 1 0 10 0 1 1

rank(A) = 3,

1 1 0 01 0 1 00 1 0 10 0 1 1

1 1 01 0 10 1 00 0 1

·1 0 0 −1

0 1 0 10 0 1 1

rank(A) = 3, but rank+(A) = 4

Polynomial System Verifier

. .[Renegar’92, Basu-Pollack-Roy’96]

Given : a polynomial system P(x) over the real numbers

Given : a polynomial system P(x) over the real numbersv : #variables, x = (x1, x2, · · · , xv )

m : #polynomial constraints fi(x) > 0, ∀i ∈ [m]

d : maximum degree of all polynomial constraints

d : maximum degree of all polynomial constraintsH : the bitsizes of the coefficients of the polynomials

In (md)O(v) poly(H) time, candecide if there exists a solution to polynomial system P

Main Idea

1. Write minU,V>∈Rn×k ,U,V>0

‖UV − A‖2F as a polynomial system

that has poly(k) variables and poly(n) constraints and degree

2. Use polynomial system verifier to solve it

Algorithm. .

Given : A ∈ Rn×n, k ∈ N

Algorithm. .

Question :

Algorithm. .

Question : Are there matrices U,V> ∈ Rn×k such that

Algorithm. .

Question : Are there matrices U,V> ∈ Rn×k such thatUV = A

Algorithm. .

Question : Are there matrices U,V> ∈ Rn×k such thatUV = AU > 0, V > 0

Algorithm. .

Output :

Algorithm. .

Output : Yes or No

Algorithm. .

Output : Yes or Noin n2O(k)

time Arora-Ge-Kannan-Moitra’12

Algorithm. .

Output : Yes or Noin n2O(k)

time Arora-Ge-Kannan-Moitra’12in 2O(k3)nO(k2) time Moitra’13

k -SUM. .

k -SUM

k -SUM. .

k -SUM

Given: a set of n values s1, s2, · · · , sn each in the range [0,1]

k -SUM. .

k -SUM

Determine:

k -SUM. .

k -SUM

Determine: if there is a set of k numbers that sum to exactly k/2

k -SUM. .

k -SUM

k -SUM hardness [Patrascu-Williams’10]

k -SUM. .

k -SUM

Assume: 3-SAT on n variables cannot be solved in 2o(n) time

k -SUM. .

k -SUM

Assume: 3-SAT on n variables cannot be solved in 2o(n) time

then k -SUM cannot be solved in no(k) time

Hardness - Under Exponential Time Hypothesis. .

Output : Are there two matrices U,V> ∈ Rn×k , such thatUV = AU > 0,V > 0

Assume : Exponential Time Hypothesis[Impagliazzo-Paturi-Zane’98]

Requires : nΩ(k) time

Open Problems

The upper bound is nO(k2) while the lower bound is nΩ(k) - what isthe right answer?

Weighted Low Rank Approximation

Weighted Low Rank Approximation. .

Given : A ∈ Rn×n, W ∈ Rn×n, k ∈ N, ε > 0

Output :

Output : rank-k A ∈ Rn×n s.t.

AW A )− (

Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε) min

rank−k A ′‖W (A ′ − A)‖2F

AW A )− (

∑i,j

W 2i,j(A

′i,j − Ai,j)

AW A )− (

∑i,j

W 2i,j(A

′i,j − Ai,j)

AW A )− (

Output :

U,V> ∈ Rn×k s.t.‖W (UV − A)‖2F 6 (1 + ε)OPT

W A )− (

Motivation. .

Action Comedy Historical Cartoon Magical

Motivation. .

..........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Motivation. .

...........

Matrix Completion. .

Given : A ∈ R, ?n×n, Ω ⊂ [n]× [n] and k ∈ R

1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1

, k = 2

1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1

, k = 2

Output : AΩ s.t. rank(A) = k

1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1

, k = 2

1 1 0 01 1 0 00 0 1 10 0 1 1

1 0 1 00 1 0 11 0 1 00 1 0 1

1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1

, k = 2

Algorithms : Candes-Recht’09, Candes-Recht’10, Keshavan’12Hardt-Wootters’14, Jain-Netrapalli-Sanghavi’13Hardt’15, Sun-Luo’15

1 ? ? 0? 1 0 ?? 0 1 ?0 ? ? 1

, k = 2

Algorithms : Candes-Recht’09, Candes-Recht’10, Keshavan’12Hardt-Wootters’14, Jain-Netrapalli-Sanghavi’13Hardt’15, Sun-Luo’15

Hardness : Peeters’96, Gillis-Glineur’11Hardt-Meka-Raghavendra-Weitz’14

More Real-life Datasets. .

RateBeer

Documents

Amazon

Biology

RateBeer

Tasters Beers Ratings

Documents

Amazon

Biology

RateBeer

Documents

Words Books Frequency

Amazon

Biology

RateBeer

Documents

Amazon

Users Products Ratings

Biology

RateBeer

Documents

Amazon

Users Products Ratings

Biology

Genes Cells Levels

Main Results - Summary

Algorithm for Weighted low rank approximation(WLRA) problem

I W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r

Hardness for Weighted low rank approximation(WLRA) problem

Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columns

I W has r distinct columnsI W has rank at most r

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columns

I W has rank at most r

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Algorithm for Weighted low rank approximation(WLRA) problemI W has r distinct rows and columnsI W has r distinct columnsI W has rank at most r

Hardness for Weighted low rank approximation(WLRA) problem.

r Distinct Rows and Columns. .

Given : A ∈ Rn×n, W ∈ Rn×n, r ∈ N, k ∈ N, ε > 0

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

Output :

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

rank−k A ′‖W (A − A ′)‖2F

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

rank−k A ′

∑i,j

W 2i,j(Ai,j − A ′i,j)

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

Output : rank-k A ∈ Rn×n s.t.‖W (A − A)‖2F 6 (1 + ε)

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

OPTwith prob. 9/10

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) time

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) timefor an arbitrarily small constant γ > 0

1 1 1 3 3 41 1 1 3 3 42 2 2 5 5 02 2 2 5 5 02 2 2 5 5 07 7 7 4 4 6

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r/ε) timefor an arbitrarily small constant γ > 0fixed parameter tractable

r Distinct Columns. .

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

Output :

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

rank−k A ′

∑i,j

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

OPTwith prob. 9/10

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) time

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) timefor an arbitrarily small constant γ > 0

1 1 1 1 1 12 2 2 1 1 33 3 3 1 1 54 4 4 1 1 15 5 5 1 1 36 6 6 1 1 5

, r = 3

OPTwith prob. 9/10in O((nnz(A) + nnz(W )) · nγ) + n · 2O(k2r2/ε) timefor an arbitrarily small constant γ > 0fixed parameter tractable

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Output :

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

rank−k A ′

∑i,j

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

OPTwith prob. 9/10

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

OPTwith prob. 9/10

in nO(k2r/ε) time

Rank r. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

, r = 3

OPTwith prob. 9/10

in nO(k2r/ε) timepreviously only r=1 was known to be in polynomial time

Random-4SAT Hypothesis. .

[Feige’02, Goerdt-Lanka’04]

Given : random 4-SAT formula S on n variables

Given : random 4-SAT formula S on n variableseach clause has 4 literals

Given : random 4-SAT formula S on n variableseach clause has 4 literalseach of Θ(n4) clauses is picked ind. with prob. Θ(1/n3)m = Θ(n) is the number of clauses

(x4 ∨ x32 ∨ x9 ∨ x20)S1

Any algorithm that :

(x4 ∨ x32 ∨ x9 ∨ x20)S1

(x2 ∨ x13 ∨ x5 ∨ x25)S2

· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm

S = S1 ∧ S2 ∧ · · ·∧ Sm

Any algorithm that :outputs 1 with prob. 1 when S is satisfiableoutputs 0 with prob. > 1/2probability is over the input instances

(x4 ∨ x32 ∨ x9 ∨ x20)S1

(x2 ∨ x13 ∨ x5 ∨ x25)S2

· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm

S = S1 ∧ S2 ∧ · · ·∧ Sm

Any algorithm that :outputs 1 with prob. 1 when S is satisfiableoutputs 0 with prob. > 1/2probability is over the input instances

Requires : 2Ω(n) time

(x4 ∨ x32 ∨ x9 ∨ x20)S1

(x2 ∨ x13 ∨ x5 ∨ x25)S2

· · · · · ·(x11 ∨ x19 ∨ x24 ∨ x6)Sm

S = S1 ∧ S2 ∧ · · ·∧ Sm

Maximum Edge Biclique(MEB). .

Given : A bipartite graph G = (U,V ,E) with |U | = |V | = n

Output : A k1-by-k2 complete bipartite subgraph of G

Output : A k1-by-k2 complete bipartite subgraph of Gs.t. k1 · k2 is maximized

[Goerdt-Lanka’04]

Assume : Random 4-SAT hardness hypothesis

[Goerdt-Lanka’04]

there exist two constants ε1 > ε2 > 0 such that

[Goerdt-Lanka’04]

Any algorithm that : distinguishes between

bipartite graphs G = (U,V ,E) with |U | = |V | = n in two cases

[Goerdt-Lanka’04]

1. there is a clique of size > (n/16)2(1 + ε1)

[Goerdt-Lanka’04]

2. all bipartite cliques are of size 6 (n/16)2(1 + ε2)

[Goerdt-Lanka’04]

Requires :

[Goerdt-Lanka’04]

Requires : 2Ω(n) time

Reduction. .

MEB Given : U,V with |U | = |V | = n

Reduction. .

MEB Given : U,V with |U | = |V | = nOutput : a biclique s.t. k1k2 is maximized

Reduction. .

( complement ) |E |− k1k2 is minimized

Reduction. .

WLRA Given : A ∈ 0,1n×n,W ∈ Rn×n

Reduction. .

Output : u, v ∈ Rn s.t. ‖W (uv> − A)‖2F is minimized

Reduction. .

Ai,j =

if edge (Ui ,Vj) exists

otherwise

Reduction. .

Ai,j =

if edge (Ui ,Vj) exists

otherwiseW i,j =

Main Results - Hardness. .

Weighted Low Rank Approximation Hardness

Given : A ∈ Rn×n, distinct r columns W ∈ Rn×n

k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)

Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTwith prob. 9/10

k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)

Assume : Random-4SAT Hypothesis[Feige’02, Goerdt-Lanka’04]

k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)

∃ε0 > 0, for any algorithm with ε < ε0 and k > 1

k ∈ N, ε > 0,W ij ∈ 0,1,2, · · · ,poly(n)

∃ε0 > 0, for any algorithm with ε < ε0 and k > 1Requires : 2Ω(r) time

Algorithmic Techniques

I Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch

Algorithm, assuming a rank r weight matrix W

I Warmup, inefficient WLRA algorithmI “Guessing a sketch”

ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]

I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch

ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]

I Multiple regression sketchAlgorithm, assuming a rank r weight matrix W

ToolsI Polynomial system verifier [Renegar’92, Basu-Pollack-Roy’96]I Lower bound on the cost [Jeronimo-Perrucci-Tsigaridas’13]I Multiple regression sketch

Algorithm, assuming a rank r weight matrix WI Warmup, inefficient WLRA algorithm

I “Guessing a sketch”

Algorithm, assuming a rank r weight matrix WI Warmup, inefficient WLRA algorithmI “Guessing a sketch”

Polynomial System Verifier (Recall)

Given : a real polynomial system P(x)

Given : a real polynomial system P(x)v : #variables, x = (x1, x2, · · · , xv )

d : maximum degree of all polynomial constraints

It takes (md)O(v) poly(H) time todecide if there exists a solution to polynomial system P

Lower Bound on the Cost

. .[Jeronimo-Perrucci-Tsigaridas’13]

Given: a real polynomial system P(x)

v : #variables, x = (x1, x2, · · · , xv )

m : #polynomials constraints fi(x) > 0, ∀i ∈ [m]

d : maximum degree of all polynomials

d : maximum degree of all polynomialsH : the bitsizes of the coefficients of the polynomials

T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0

minimum value that nonnegative g takes over T ∩ Ball is

T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0

either 0 or > (2H + m)−dv

T : x ∈ Rv |f1(x) > 0, · · · , fm(x) > 0

either 0 or > (2H + m)−dv

g = (x1x2 − 1)2 + x22

Multiple Regression Sketch. .

Given :

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

Let x(j) = arg minx∈Rk×1

‖A(j)x − b(j)‖,∀j ∈ [m]

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

Choose : S to be a random Gaussian matrix

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

Choose : S to be a random Gaussian matrixDenote y(j) = arg min

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

Gaurantee :

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such that

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

Gaurantee : For all ε ∈ (0,1/2), one can set t = O(k/ε) such thatm∑

j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)

m∑j=1‖A(j)x(j) − b(j)‖22

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)

m∑j=1‖A(j)x(j) − b(j)‖22

with prob. 9/10

Given : A(1),A(2), · · · ,A(m) ∈ Rn×k

b(1),b(2), · · · ,b(m) ∈ Rn×1

‖A(j)x − b(j)‖,∀j ∈ [m]

y∈Rk×1‖SA(j)y − Sb(j)‖, ∀j ∈ [m]

j=1‖A(j)y(j) − b(j)‖22 6 (1 + ε)

m∑j=1‖A(j)x(j) − b(j)‖22

with prob. 9/10

Warmup, inefficient WLRA Algorithm. .

Aij ∈ 0,±1,±2, · · · ,±∆

W ij ∈ 0,1,2, · · · ,∆

Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPT

Output : rank-k A s.t. ‖W (A − A)‖2F 6 (1 + ε)OPTAlgorithm :

1. create 2nk variables for U,V> ∈ Rn×k

2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F

2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C

2. write polynomial g(x1, · · · , x2nk ) = ‖W (A − UV )‖2F3. pick C ∈ [L−,L+], run polynomial verifier g(x) 6 C4. optimize C by binary search over [L−,L+]

Time : 2Ω(nk)

How can we do better?

Time : 2Ω(nk)

How can we do better?

Time : 2Ω(nk)

How can we do better?polynomial verifier runs in (# constraints · degree)O(# variables)

Time : 2Ω(nk)

lower bound on cost (# constraints)−degreeO(# variables)

Time : 2Ω(nk)

write a polynomial with few #variables, i.e. poly(kr/ε)

Time : 2Ω(nk)

write a polynomial with few #variables, i.e. poly(kr/ε)without blowing up degree and #constraints too much

Main Idea

To reduce the number of variables to poly(kr/ε) :

1. Multiple regression sketch with O(k/ε) rows

2. Weight matrix W has rank at most r

Guess a Sketch. .

Aij ∈ 0,±1,±2, · · · ,±∆

Guess a Sketch. .

W ij ∈ 0,1,2, · · · ,∆

Guess a Sketch. .

Output : U,V> ∈ Rn×k s.t. ‖W (UV − A)‖2F 6 (1 + ε)OPT

Guess a Sketch. .

Algorithm : W j be j th column of WDW j be diagonal matrix with vector W j

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

Guess a Sketch. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 1 =

Guess a Sketch. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 2 =

Guess a Sketch. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 3 =

Guess a Sketch. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 4 =

Guess a Sketch. .

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 5 =

Guess a Sketch. .

......

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

DW 6 =

Guess a Sketch. .

Algorithm :

......

W A )− (

Guess a Sketch. .

Algorithm :

......

W A )− (

Guess a Sketch. .

Algorithm :

......

.n∑j=1‖DW j UV j − DW j Aj‖22

W A )− (

Guess a Sketch. .

Algorithm :

......

W A )− (

Guess a Sketch. .

Algorithm :

......

n∑i=1‖U iVDW i − AiDW i‖22

W A )− (

Guess a Sketch. .

Algorithm :

......

Sketch by Gaussian S,T> ∈ Rt×n

Guess a Sketch. .

Algorithm :

......

Sketch by Gaussian S,T> ∈ Rt×nn∑

j=1‖SDW j UV j − SDW j Aj‖22

Guess a Sketch. .

Algorithm :

......

n∑i=1‖U iVDW i T − AiDW i T‖22

Guess a Sketch. .

Algorithm :

......

Guess SDW j U ∈ Rt×k and VDW i T ∈ Rk×t

Guess a Sketch. .

Algorithm :

......

SDW 1U = S

Guess a Sketch. .

Algorithm :

......

SDW 2U = S

Guess a Sketch. .

Algorithm :

......

SDW 3U = S

Guess a Sketch. .

Algorithm :

......

SDW 4U = S

Guess a Sketch. .

Algorithm :

......

SDW 5U = S

Guess a Sketch. .

Algorithm :

......

SDW 6U = S

Guess a Sketch. .

Algorithm :

......

Create t × k variables for each of n SDW j U s?

Guess a Sketch. .

Algorithm :

......

×t × kn

Guess a Sketch. .

Algorithm :

......

×t × k

Guess a Sketch. .

Algorithm :

......

×t × k

Guess a Sketch. .

Algorithm : W j be j th column of W

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

Guess a Sketch. .

Algorithm : W j be j th column of W.

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

1 1 10 1 21 1 30 0 41 0 50 0 6

column span of W

Guess a Sketch. .

Algorithm : W j be j th column of W..

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

101010

W 1 = W 1

Guess a Sketch. .

Algorithm : W j be j th column of W...

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

111000

W 2 = W 2

Guess a Sketch. .

Algorithm : W j be j th column of W....

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

123456

W 3 = W 3

Guess a Sketch. .

Algorithm : W j be j th column of W.....

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

212010

101010

111000

W 4 = W 1 + W 2

Guess a Sketch. .

Algorithm : W j be j th column of W......

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

224466

101010

123456

W 5 = W 1 + W 3

Guess a Sketch. .

Algorithm : W j be j th column of W.......

1 1 1 2 2 20 1 2 1 2 31 1 3 2 4 40 0 4 0 4 41 0 5 1 6 50 0 6 0 6 6

234456

111000

123456

W 6 = W 2 + W 3

Guess a Sketch. .

Algorithm :....... n∑

j=1‖DW j UV j − DW j Aj‖22

Guess a Sketch. .

sketchn∑

1 + ε

Guess a Sketch. .

sketchn∑

1 + ε

create variables for SDW j U, ∀j ∈ [r ]

Guess a Sketch. .

sketchn∑

1 + ε

create t × k variables for SDW 1U

Guess a Sketch. .

sketchn∑

1 + ε

Guess a Sketch. .

sketchn∑

1 + ε

Guess a Sketch. .

sketchn∑

1 + ε

write SDW 4U as SDW 1U + SDW 2U

Guess a Sketch. .

sketchn∑

1 + ε

1 + 10 + 1

1 + 10 + 0

1 + 00 + 0

Guess a Sketch. .

sketchn∑

1 + ε

......

Guess a Sketch. .

sketchn∑

1 + ε

.......

Guess a Sketch. .

sketchn∑

1 + ε

........

1 + 10 + 2

1 + 30 + 4

1 + 50 + 6

Guess a Sketch. .

sketchn∑

1 + ε

.........

Guess a Sketch. .

sketchn∑

1 + ε

.........

Guess a Sketch. .

sketchn∑

1 + ε

.........

1 + 11 + 2

1 + 30 + 4

0 + 50 + 6

Guess a Sketch. .

sketchn∑

1 + ε

.........

Wrapping Up

Can use an explicit formula for the regression solution in terms ofthe variables created

The regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)

(# constraints · degree)O(# variables) = (O(n) ·O(nk))O(krt)

Wrapping Up

Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominator

Multiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)

Wrapping Up

Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variables

Time : nO(rk2/ε)

Wrapping Up

Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)

Wrapping Up

Can use an explicit formula for the regression solution in terms ofthe variables createdThe regression solution is a rational function, but can use tricks toclear the denominatorMultiple Regression Sketch + Bounded Rank Weight Matrix implya small number of variablesTime : nO(rk2/ε)

Open Problems

For a rank-r weight matrix W , the upper bound is nO(k2r/ε) but thelower bound is only 2Ω(r) - can we close this gap?

Can we prove a hardness result with respect to the parameter k ,e.g., a 2Ω(k) lower bound for WLRA problem?

Open Problems

For a rank-r weight matrix W , the upper bound is nO(k2r/ε) but thelower bound is only 2Ω(r) - can we close this gap?Can we prove a hardness result with respect to the parameter k ,e.g., a 2Ω(k) lower bound for WLRA problem?

`1 Low Rank Approximation

`1-Low Rank Approximation. .

Given : A ∈ Rn×n, k ∈ N, α > 1

Output :

A A−

Output : rank-k A ∈ Rn×n s.t.‖A − A‖1 6 α · min

rank−k A ′‖A ′ − A‖1

A A−

∑i,j|A ′i,j − Ai,j |

A A−

∑i,j|A ′i,j − Ai,j |

A A−

Output :

U ∈ Rn×k ,V ∈ Rk×n s.t.‖UV − A‖1 6 α · OPT

Why is `1-low Rank Approximation Interesting?

The problem was shown to be NP-hard by Gillis-Vavasis’15 and anumber of heuristics have been proposed

Asked in several places if there are any approximation algorithms

Why is `1-low Rank Approximation Interesting?

The problem was shown to be NP-hard by Gillis-Vavasis’15 and anumber of heuristics have been proposedAsked in several places if there are any approximation algorithms

Thought Experiments. .

Given : A set of points in 2-dimensions

Goal : Find a line to fit those points

Output : `2 minimizer line,

Output : `2 minimizer line, `1 minimizer line

Given : k = 1 and A =

4 0 0 00 1 1 10 1 1 10 1 1 1

Output : rank-1 ‖‖F−solution A =

4 0 0 00 0 0 00 0 0 00 0 0 0

4 0 0 00 1 1 10 1 1 10 1 1 1

4 0 0 00 0 0 00 0 0 00 0 0 0

4 0 0 00 1 1 10 1 1 10 1 1 1

4 0 0 00 0 0 00 0 0 00 0 0 0

Output : rank-1 ‖‖1−solution A =

0 0 0 00 1 1 10 1 1 10 1 1 1

4 0 0 00 1 1 10 1 1 10 1 1 1

4 0 0 00 0 0 00 0 0 00 0 0 0

Output : rank-1 ‖‖1−solution A =

0 0 0 00 1 1 10 1 1 10 1 1 1

Heuristics and Nearly `1 Setting

`1-setting

I Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,Kwak’08, Brooks-Dula-Boone’13

I Robust PCA Candes-Li-Ma-Wright’11,Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16

None of them can achieve better than poly(n) approximation ratio

`1-settingI Heuristic algorithms Ke-Kanade’05, Ding-Zhou-He-Zha’06,

Kwak’08, Brooks-Dula-Boone’13

I Robust PCA Candes-Li-Ma-Wright’11,Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16

Kwak’08, Brooks-Dula-Boone’13I Robust PCA Candes-Li-Ma-Wright’11,

Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16

Kwak’08, Brooks-Dula-Boone’13I Robust PCA Candes-Li-Ma-Wright’11,

Wright-Ganesh-Rao-Peng-Ma’09,Netrapalli-Niranjan-Sanghavi-Anandkumar-Jain’14,Yi-Park-Chen-Caramanis’16

Main Question, `1 Low Rank Approximation

QuestionGiven matrix A ∈ Rn×n, is there an algorithm that is able to output arank-k matrix A such that

‖A − A‖1 6 α · minrank−k A ′

‖A ′ − A‖1?

Or, are there any inapproximability results?

Main Results - Algorithms

Upper bound (Algorithm) for an arbitrary n × n matrix A

I poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm

F rank-2k O(1)-approximation, nO(k2) running time

Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) time

I O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm

Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.

I Bicriteria algorithm

Upper bound (Algorithm) for an arbitrary n × n matrix AI poly(k) log n-approximation, poly(n) timeI O(k)-approximation, nO(k)2O(k2) time.I Bicriteria algorithm

`1-Low Rank Approximation Hardness

Output : U ∈ Rn×k , V ∈ Rk×n s.t. ‖UV − A‖1 6 α ·OPTwith prob. 9/10

Assume : Exponential Time Hypothesis(ETH)[Impagliazzo-Paturi-Zane’98]

for arbitrarily small constant γ > 0

for arbitrarily small constant γ > 0for any algorithm running in nO(1) time

Requires : α > (1 + 1log1+γ n

Implies a 2Ω(1/ε1−γ) Hardness

Open Problems

The hardness result is for (1 + 1/ poly(log n)) approximation, canwe obtain a hardness result for O(1) approximation ?

We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?

Open Problems

We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?

We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?

Open Problems

We have an nO(k)-time O(k)-approximation algorithm, can eitherthe approximation or the time be improved?We have a poly(n)-time O(log n) · poly(k)-approximationalgorithm. Can the approximation be improved?

Conclusions

Studied intractable matrix factorization problems through the lensof parameterized complexity

I nonnegative matrix factorizationI weighted low rank approximationI `1-low rank approximation

Algorithm techniques

I polynomial system verifierI random matrices for dimensionality reduction to reduce the number

of variables

Hardness techniques

I random 4-SAT, ETH, etc.

Parameterized Complexity gives a way of coping with intractabilityfor emerging machine learning problems

Conclusions

I nonnegative matrix factorization

I weighted low rank approximationI `1-low rank approximation

of variables

Hardness techniques

Conclusions

I nonnegative matrix factorizationI weighted low rank approximation

I `1-low rank approximationAlgorithm techniques

of variables

Hardness techniques

Conclusions

of variables

Hardness techniques

Conclusions

of variablesHardness techniques

Conclusions

Algorithm techniquesI polynomial system verifier

I random matrices for dimensionality reduction to reduce the numberof variables

Hardness techniques

Conclusions

Algorithm techniquesI polynomial system verifierI random matrices for dimensionality reduction to reduce the number

of variables

Hardness techniques

Conclusions

Thank you!. .

Questions?

Cauchy Distribution. .

Cauchy distribution C(µ,γ), pdf f (x) = 1πγ

(x−µ)2+γ

For any three independent random variablesX ,Y ,Z drawn from Cauchy distribution.For any a,b ∈ R, aX + bY ∼ (|a|+ |b|)Z

Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix...

Transcript of Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix...

Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix...

Documents

Transcript of Parameterized Complexity of Matrix Factorization - IBM · Parameterized Complexity of Matrix...

Parameterized Complexity of 1-Planarity - Donald Bren

Parameterized Complexity News - fpt.wdfiles.comfpt.wdfiles.com/...parameterized-complexity-newsletter/...October.pdf · Parameterized Complexity News Newsletter of the Parameterized

Parameterized Domination in Circle Graphscommunity.dur.ac.uk/george.mertzios/papers/Conf/Conf_Parameteriz… · Keywords: circle graphs; domination problems; parameterized complexity;

Parameterized Complexity and the Understanding, Design ...shonan.nii.ac.jp/shonan/wp-content/uploads/2011/09/No.2013-2.pdf · Parameterized Complexity and the Understanding, Design,

Parameterized Complexity of Spare Capacity Allocation and ...ildi/pub/routing_jda.pdf2.2 Parameterized complexity A parameterized problem contains pairs of the form (I,k) where I is

Parameterized Complexity of Cardinality Constrained ...cgi.di.uoa.gr/~vassilis/co/cai_2007_max-k-...pdf · Parameterized Complexity of Cardinality Constrained Optimization Problems

Parameterized Complexity VS Approximation Algorithms · D. Marx “Parameterized Complexity and Approximation Algorithms” 1/9/2013 20 . Conclusions • Parameterized Complexity

Parameterized Complexity of Neighborhood Problems in Graphs …vraman/pub/dsswat.pdf · 2007-03-26 · parameterized complexity classes on general graphs, become ﬁxed pa-rameter

Nonnegative Matrix Factorization - Complexity, Algorithms ... · Nonnegative Matrix Factorization, Neural Computation 2012. Householder XIX Nonnegative Matrix Factorization: Complexity,

PARAMETERIZED COMPLEXITY OF DIRECTED STEINER TREEdaniello/papers/directedSteinerJ.pdf · background on parameterized complexity the reader is referred to the monographs [8, 13, 17,

Parameterized Complexity of Decision Problems in Non ...

Parameterized Complexity of Shift Bribery in Iterative ... · parameterized complexity with respect to three parameters: the number of candidates, the number of votes, and the number

Complexity of Liveness in Parameterized Systems · We study the ﬁne-grained complexity of liveness veriﬁcation for parameterized systems formulated in the leader contributor model.

The Parameterized Complexity of Graph Cyclabilityrutcor.rutgers.edu/~mkaminski/preprints/The parametrized...The Parameterized Complexity of Graph Cyclability? Petr A. Golovach1, Marcin

Keeping a Crowd Safe On the Complexity of Parameterized Verification

Parameterized complexity of DPLL search procedureseprints.whiterose.ac.uk/79311/8/para-dpll.pdf · Parameterized complexity is a branch of complexity theory where problems are analyzed

The Parameterized Complexity of Graph Cyclabilityosebje.famnit.upr.si/~martin.milanic/AGTAC 2015 slides/Maniatis.pdf · The Parameterized Complexity of Graph Cyclability Spyridon

Parameterized Complexity SAT · The Gentle Revolution of Parameterized Complexity • Bioinformatics, Operations Research, Optimization, Automated Reasoning, etc." • 4 Monographs"-[Downey&Fellows

AGAPE 2009 Notes on Parameterized Complexity€¦ · 1 Introduction to Parameterized Complexity 1.1 Two Forms of Fixed-Parameter Complexity Many natural computational problems are

The Parameterized Complexity of Cascading Portfolio Schedulingpapers.nips.cc/paper/8983-the-parameterized... · Parameterized Complexity. In parameterized algorithmics [6, 4, 3, 9]