5

Maximizing submodular functions

Minimizing convex functions:

Polynomial time solvable!Minimizing submodular functions:

Polynomial time solvable!

Maximizing convex functions:

NP hard!Maximizing submodular functions:

NP hard!

But can get approximation guarantees

6

Example: Set cover

Node predictsvalues of positionswith some radius

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

For A µ V: z(A) = “area covered by sensors placed at A”

Formally: W finite set, collection of n subsets Si

µ WFor A µ V={1,…,n} define z(A) = |i2 A Si|

Want to cover floorplan with discsPlace sensorsin building

Possiblelocations

V

7

Set cover is submodular

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

SERVER

LAB

KITCHEN

COPYELEC

PHONEQUIET

STORAGE

CONFERENCE

OFFICEOFFICE

S1 S2

S1 S2

S3

S4 S’

S’

A={S1,S2}

B = {S1,S2,S3,S4}

z(A[{S’})-z(A)

z(B[{S’})-z(B)

¸

8

Example: Feature selection• Given random variables Y, X1, … Xn

• Want to predict Y from subset XA = (Xi1,…,Xik

)

Want k most informative features:

A* = argmax IG(XA; Y) s.t. |A| · k

where IG(XA; Y) = H(Y) - H(Y | XA)

Y“Sick”

X1

“Fever”X2

“Rash”X3

“Male”

Naïve BayesModel

Uncertaintybefore knowing XA

Uncertaintyafter knowing XA

9

Example: Submodularity of info-gain

Y1,…,Ym, X1, …, Xn discrete RVs

z(A) = IG(Y; XA) = H(Y)-H(Y | XA)

• z(A) is always monotonic• However, NOT always submodular

Theorem [Krause & Guestrin UAI’ 05]If Xi are all conditionally independent given Y,then z(A) is submodular!

Y1

X1

Y2

X2

Y3

X4X3

Hence, greedy algorithm works!

In fact, NO algorithm can do better than (1-1/e) approximation!

10

• People sit a lot• Activity recognition in

assistive technologies• Seating pressure as

user interface

Equipped with 1 sensor per cm2!

Costs $16,000!

Can we get similar accuracy with fewer,

cheaper sensors?

Leanforward

SlouchLeanleft

82% accuracy on 10 postures! [Tan et al]

Building a Sensing Chair [Mutlu, Krause, Forlizzi, Guestrin, Hodgins UIST ‘07]

11

How to place sensors on a chair?• Sensor readings at locations V as random variables• Predict posture Y using probabilistic model P(Y,V)• Pick sensor locations A* µ V to minimize entropy:

Possible locations V

Accuracy CostBefore 82% $16,000 After 79% $100

Placed sensors, did a user study:

Similar accuracy at <1% of the cost!

12

Bounds on optimal solution[Krause et al., J Wat Res Mgt ’08]

Submodularity gives data-dependent bounds on the performance of any algorithm

Sensi

ng q

ualit

y z

(A)

Hig

her

is b

ett

er

Water networks

data

0 5 10 15 200

0.2

0.4

0.6

0.8

1

1.2

1.4Offline

(Nemhauser)bound Data-dependent

bound

Greedysolution

Number of sensors placed

Summary (1)

• Minimization of submodular functions– Submodularity and convexity– Submodular Polyhedron– Symmetric submodular functions

Summary (2)

• Pseudo-boolean functions– Representation (polynomial, posiform, tableau, graph

cut)– Reduction to quadratic polynomial– Necessary and sufficient conditions for submodularity– Minimization of quadratic and cubic submodular

functions via graph cuts– Lower bound via roof duality

• LP via posiform representation• LP via linear relaxation• Max flow via symmetric graph construction

Further reading

• Combinatorial algorithms for submodular (and bisubmodular) function minimization

• More algorithms/bounds for maximizing submodular functions

• Linear and semidefinite relaxations

• Matroids, greedoids, intersection of matroids, polymatroids and more

• Generalized roof duality