Efficiently Solving Convex Relaxations M. Pawan Kumar University of Oxford for MAP Estimation Philip...

Efficiently Solving Convex Relaxations

M. Pawan Kumar

University of Oxford

for MAP Estimation

Philip Torr

Oxford Brookes University

Aim

2

5

4

2

6

3

3

7

0

1 1

0

0

2 3

1

1

4 1

0

a b c d

Label ‘0’

Label ‘1’

Labelling m = {1, 0, 0, 1}

Random Variables V = {a, b, c, d}

Label Set L = {0, 1}

• To solve convex relaxations of MAP estimation

Edges E = {(a, b), (b, c), (c, d)}

Aim

2

5

4

2

6

3

3

7

0

1 1

0

0

2 3

1

1

4 1

0Label ‘0’

Label ‘1’

Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3 = 13

Approximate using Convex Relaxations

Minimum Cost Labelling? NP-hard problem


a b c d

Aim

2

5

4

2

6

3

3

7

0

1 1

0

0

2 3

1

1

4 1

0Label ‘0’

Label ‘1’

Objectives

• Solve tighter convex relaxations – LP and SOCP

• Handle large number of random variables, e.g. image pixels


a b c d

Outline

• Integer Programming Formulation

• Linear Programming Relaxation

• Additional Constraints

• Solving the Convex Relaxations

• Results and Conclusions

Integer Programming Formulation

2

5

4

2

0

1 3

0

a b

Label ‘0’

Label ‘1’Unary Cost

Unary Cost Vector u = [ 5

Cost of a = 0

2

Cost of a = 1

; 2 4 ]

Labelling m = {1 , 0}

2

5

4

2

0

1 3

0Label ‘0’


Unary Cost Vector u = [ 5 2 ; 2 4 ]T


Label vector x = [ -1

a 0

1

a = 1

; 1 -1 ]T

Recall that the aim is to find the optimal x


a b

2

5

4

2

0

1 3

0Label ‘0’


Unary Cost Vector u = [ 5 2 ; 2 4 ]T


Label vector x = [ -1 1 ; 1 -1 ]T

Sum of Unary Costs = 12

∑i ui (1 + xi)


a b

2

5

4

2

0

1 3

0Label ‘0’

Label ‘1’Pairwise Cost


Pairwise Cost of a and a0 0

00

0Cost of a = 0 and b = 0

3

Cost of a = 0 and b = 11 0

00

0 0

10

3 0

Pairwise Cost Matrix P


a b

2

5

4

2

0

1 3

0Label ‘0’




0 0

00

0 3

1 0

00

0 0

10

3 0

Sum of Pairwise Costs14

∑ij Pij (1 + xi)(1+xj)


a b

2

5

4

2

0

1 3

0Label ‘0’




0 0

00

0 3

1 0

00

0 0

10

3 0

Sum of Pairwise Costs14

∑ij Pij (1 + xi +xj + xixj)

14

∑ij Pij (1 + xi + xj + Xij)=

X = x xT Xij = xi xj


a b

Constraints

• Uniqueness Constraint

∑ xi = 2 - |L|i a

• Integer Constraints

xi {-1,1}

X = x xT


x* = argmin 12

∑ ui (1 + xi) + 14

∑ Pij (1 + xi + xj + Xij)

∑ xi = 2 - |L|i a

xi {-1,1}

X = x xT

ConvexNon-Convex


Outline






Linear Programming Relaxation

x* = argmin 12

∑ ui (1 + xi) + 14


∑ xi = 2 - |L|i a

xi {-1,1}

X = x xT

Retain Convex PartSchlesinger, 1976

Linear Programming Relaxation

x* = argmin 12

∑ ui (1 + xi) + 14


∑ xi = 2 - |L|i a

Retain Convex PartSchlesinger, 1976

Xij [-1,1] 1 + xi + xj + Xij ≥ 0

∑ Xij = (2 - |L|) xij b

xi [-1,1]

Dual of the LP RelaxationWainwright et al., 2001

a b c

d e f

g h i

= (u, P)

a b c

d e f

g h i

a b c

d e f

g h i

1

2

3

4 5 6

1

2

3

4 5 6

ii

Dual of the LP RelaxationWainwright et al., 2001

a b c

d e f

g h i

= (u, P)

a b c

d e f

g h i

a b c

d e f

g h i

1

2

3

4 5 6

Q(1)

ii

Q(2)

Q(3)

Q(4) Q(5) Q(6)

max i Q(i)

Dual of LP

Tree-Reweighted Message PassingKolmogorov, 2005

a b c

d e f

g h i

a b c

d e f

g h i

1

2

3

4 5 6

Pick a variable

c b a a d g

a

Reparameterize such that ui are min-marginals

u1

u2

u3

u4

Only one pass of belief propagation

Tree-Reweighted Message PassingKolmogorov, 2005

a b c

d e f

g h i

a b c

d e f

g h i

1

2

3

4 5 6

Pick a variable

c b a a d g

a

Average the unary costs

(u1+u3)/2

Repeat for all variables

(u1+u3)/2

(u2+u4)/2 (u2+u4)/2

TRW-S

Outline






Cycle InequalitiesChopra and Rao, 1991

a

e f

b c

d

a

ed

xi

xjxk

At least two of them have the same sign

xixj xjxk xkxi

Xij Xjk XkiX = xxT

At least one of them is 1

Xij + Xjk + Xki -1

Cycle InequalitiesChopra and Rao, 1991

a

e f

b c

d

Xij + Xjk + Xkl - Xli -2

xj

b

fe

xi

xk

c

xl

Generalizes to all cycles

LP-C

Second-Order Cone ConstraintsKumar et al., 2007

a

e f

b c

d

xc = xi

xj

xk

Xc = 1

Xij

Xij Xik

Xjk

XjkXik

1

1

Xc = xcxcT

Xc xcxcT

1 • (Xc - xcxcT) 0

(xi+xj+xk)2 ≤ 3 + Xij + Xjk + Xki

SOCP-C

Second-Order Cone ConstraintsKumar et al., 2007

a

e f

b c

d

1 • (Xc - xcxcT) 0 SOCP-Q

xc = xi

xj

xk

Xc = 1

Xij

Xij Xik

Xjk

XjkXik

1

1

xl

Xil

Xjl

Xkl

Xil Xjl Xkl 1

Outline






Modifying the Dual

a b c

d e f

g h i

ii

max i Q(i)

1

2

3

a b c

d e f

g h i

4

5

6

a d g

b e h

c f i

+ j sj

+ j sj

1 2a b

d e

b c

e f

d e

g h

e f

h i

3 4

Modifying TRW-S

a b c

d e f

g h i

a d g

b e h

c f i

a b

d e

b c

e f

d e

g h

e f

h iPick a variable --- aPick a cycle/clique with a

ii

max i Q(i) + j sj

+ j sj

Can be solved efficiently

Run TRW-S for trees with a

REPEAT

Properties of the Algorithm

Algorithm satisfies the reparametrization constraint

Value of dual never decreases CONVERGENCE

Solution satisfies Weak Tree Agreement (WTA)

WTA not sufficient for convergence

More accurate results than TRW-S

Outline






4-Neighbourhood MRF

Test SOCP-C

50 binary MRFs of size 30x30

u ≈ N (0,1)

P ≈ N (0,σ2)

Test LP-C

4-Neighbourhood MRF

σ = 5

LP-C dominates SOCP-C

8-Neighbourhood MRF

Test SOCP-Q

50 binary MRFs of size 30x30

u ≈ N (0,1)

P ≈ N (0,σ2)

8-Neighbourhood MRF

σ = 5 /2

SOCP-Q dominates LP-C

Conclusions

• Modified LP dual to include more constraints

• Extended TRW-S to solve tighter dual

• Experiments show improvement

• More results in the poster

Future Work

• More efficient subroutines for solving cycles/cliques

• Using more accurate LP solvers - proximal projections

• Analysis of SOCP-C vs. LP-C

Questions?

Timings

Method Time/Iteration

BP 0.0027

TRW-S 0.0027

LP-C 7.7778

SOCP-C 8.8091

SOCP-Q 9.1170

Linear in the number of variables!!

Video Segmentation

Keyframe User Segmentation

Segment remaining video ….

Video Segmentation

Belief Propagation

Input

8175 25620 18314

Video Segmentation

-swap

Input

1187 1368 1289

Video Segmentation

-expansion

Input

2453 1266 1225

Video Segmentation

TRW-S

Input

6425 1309 297

Video Segmentation

LP-C

Input

719 264 294

Video Segmentation

SOCP-Q

Input

0 0 0

4-Neighbourhood MRF

σ = 1

4-Neighbourhood MRF

σ = 2.5

8-Neighbourhood MRF

σ = 1/2

8-Neighbourhood MRF

σ = 2.5 /2

Efficiently Solving Convex Relaxations M. Pawan Kumar University of Oxford for MAP Estimation Philip...

Documents

Transcript of Efficiently Solving Convex Relaxations M. Pawan Kumar University of Oxford for MAP Estimation Philip...