EMIS 8373: Integer Programming

Complexity and Problem Reductionsupdated 23 April 2009

Decision Problems• The decision or recognition version of an

optimization problem is posed as a yes-no question.• Complexity Theory is based on a theoretical

computer called a Turing machine:– Given a binary input string, a program on a Turning

Machine either• “accepts” the input (returns a “yes” answer),• “rejects” the input (returns a “no” answer),• or fails to give an answer.

– A Turing machine is a very simple machine, but, logically speaking, has all the power of any digital computer.

slide 2

Minimum Spanning Tree (MST)

• Input– A (simple) graph G = (V, E)– Edge cost ce for each edge e E– Parameter k

• Decision Problem– Is there a set of edges T E such that |

T| = |V|-1, T contains no cycles, and ?

Te

e kc

slide 3

The Traveling Salesman Problem (TSP)

• Input– N is a set of cities {1, 2, …, n}– Travel time tij between cities i and j – Parameter k

• Decision Problem– Is there a permutation of N such that

?1

1)1(),()1(),( ktt

n

inii

slide 4

The Minimum-Cost Cycle-Cover Problem

• Input– A (simple) graph G = (V, E)– Edge cost ce for each edge e E– Parameter k

• Decision Problem– Is there a set of cycles C = {C1, C2, …, Cj} in G

such that ?and kCC CeCC Ce

Eei ii i

ce

slide 5

The Set Covering Problem• Input

– An m-row, n-column, zero-one matrix A (aij {0,1})

– A cost cj associated with each column j– Parameter k

• Decision Problem– Is there a solution to the system shown below?

njx

mixa

kxc

j

n

jjij

n

jjj

,,2,11,0

,,2,111

1

slide 6

The 0-1 Knapsack Problem• Input

– A set of n items– A weight ai associated with each item j

– A utility cj associated with each item j– Parameters b and k

• Decision Problem– Is there a solution to the system

?,,2,1 1,0,,11

njxbxakxc j

n

jjj

n

jjj

slide 7

Polynomial Certificates

• Given a “yes” instance X of a particular decision problem (e.g., TSP), a polynomial certificate C is a binary string such that

1. The length of the string, L(C), is bounded by a polynomial in the length of the binary representation of X, L(X) .

2. There is a polynomial-time certificate-checking algorithm that takes X and C as input and verifies that X is a “yes” instance.

slide 8

Certificate Example: TSP

3

1

1

32

5 4

4

2

35

75

17

2 3 4 51 4 1 7 22 3 5 73 3 54 1

tij Matrix Graph Representation

k = 14

slide 9

Certificate Example: TSP

• Using the graph representation, L(X) is O(n2) for a generic TSP with n cities.

• A certificate C for a generic TSP instance could be a network with n arcs forming a cycle corresponding to a tour of length k or less.– Note that L(C) is O(n).– We will show that C can be verified in polynomial

time.

slide 10

Certificate Example: TSP

3

1

1

32

5 4

4

2

35

75

17

G

k = 14

1

32

5 4

C

slide 11

AMPL Certificate-Checking Algorithm for TSP

param s default 0;param visited {V} default 0;for {(i, j) in C} {

let s := s + t[i,j];let visited[i] := 1;let visited[j] := 1;

}if s k and (sum{i in V} visited[i] == card(V)) then

printf “certificate verified\n”;

complexity = O(m+n)=O(n2)slide 12

Certificate Example: 0-1 Knapsack Problem

• L(X) = O(n)• A polynomial certificate could be a binary

vector x Bn indicating which items are selected.– L(C) = n– We can check the certificate in O(n) time by

checking that

slide 13

n

jjj

n

jjj

bxakxc11

.and

The Class NP• A decision problem q is said to be in the class NP

if for every “yes” instance X of q, there is a polynomial certificate.

• NP is the set of problems that can be verified or checked in polynomial time.– NP stands for Non-deterministic Polynomial. That is,

a problem is in NP if there exists a non-deterministic Touring machine that accepts all “yes” instances of q in polynomial time.

– MST, TSP, MCCP, Set Cover, and 0-1 Knapsack are all in NP

slide 14

The Class P

• P is the set of problems that can be solved in polynomial time.– P is a subset of NP– MST, MCNFP, and LP are all in P– It is unknown if TSP, MCCP, Set Cover, and

0-1 Knapsack are in P. – Does P = NP? Or are there problems that

cannot be solved in polynomial time?

slide 15

Polynomial Reductions: Example 1

• Hamiltonian Path Problem– Input: a graph G = (V, E), origin node s, and destination node t.– Decision Problem: Is there a simple path from s to t that visits

every node in V?• Longest Path Problem

– Input: a graph G = (V, E), origin node s, destination node t, edge cost cij for each edge and parameter k.

– Decision Problem: Is there a simple path from s to t with length k?

• Both problems are clearly in NP; the yes-instance certificate would be a feasible path from s to t.

slide 16

Polynomial Reductions: Example 1

• How does the difficulty of Longest Path compare to the difficulty of Hamiltonian Path?

• Given a graph G = (V, E)– F = {(i, j): i, jV, (i, j) E}. – Let H= (V, E F). – Let cij = 1 if (i, j) E and cij = 0 if (i, j) F.– Let k = |V|-1

• Claim: {G, s, t} is a “yes” instance of Hamiltonian Path if and only if {H, s, t, c, k} is a “yes” instance of Longest Path.

slide 17

Polynomial Reductions: Example 1

• If we have an algorithm for Longest Path, then we can use it to solve Hamiltonian Path.

• If the conversion from G to H is “efficient”, then solving Longest Path is “at least as difficult” as solving Hamiltonian Path.• L(G) = O(|V|2)• H has (|V|2-|V|)/2 edges• L(H) is bounded by a polynomial in L(G).

• We call the conversion a polynomial reduction from Hamiltonian Path to Longest Path.

slide 18

Analogy: LP is at Least as Hard as MCNFP

• We can solve Minimum Cost Network Flow Problems (MCNFP) by formulating them as LP.

• For a network G =(V, E), the corresponding LP has |E| decision variables and |V| + 2|E| constraints.

• The size of the resulting LP is bounded by a polynomial in the size of the network.

• Since we can solve LP in polynomial time, this implies that we can solve MCNFP in polynomial time. Thus, LP is at least as “difficult” as MCNFP.

slide 19

Converting Hamiltonian Path to Longest Path: Yes Instances

1

1

1

32

5 4

1

1

10

00

00

s

t

1

32

5 4s

t

G H

k = 4

Hamiltonian Path: (1,2),(2,3),(3,4),(4,5)

Path of Length 4: (1,2),(2,3),(3,4),(4,5)

slide 20

Converting Hamiltonian Path to Longest Path: No Instances

1

1

1

32

5 4

1

1

10

0

0

00

s

t

1

32

5 4s

t

G H

k =4

No Hamiltonian Path from s to t

The longest path from s to t has length 3: (1,2),(2,3),(3,4),(4,5)

slide 21

Proof (If Direction)• Suppose G is a “yes” instance of

Hamiltonian Path and let p be a simple Hamiltonian Path from s to t.

• p has |V|-1 edges.• Since each edge in p is in E, the cost of path

p in H is |V|-1.• Thus, the Longest Path instance resulting

from the conversion is also a “yes” instance.

slide 22

Proof (Only If Direction)• Suppose H is a “yes” instance of Longest Path

• Let p be a simple path of length |V|-1 from s to t.• p must have at least |V|-1 edges from E otherwise its

length would be less than |V|-1.• Since p is a simple path, it cannot have more than |

V|-1 edges. Otherwise, it would contain a cycle.• Since p has exactly |V|-1 edges and no cycles, it must

visit each node in V exactly once• Therefore, H is a “yes” instance of Longest Path

only if the original graph G is as “yes” instance of Hamiltonian Path.

slide 23

Polynomial Reductions

• If problems A and B are in NP, and if any instance Xa of A can be converted to (formulated as) an equivalent instance Xb of B such that L(Xb) is bounded by a polynomial in L(Xa), then A is said to be polynomially reducible to B.– An algorithm that solves B can also solve A with a

conversion “cost” that is polynomial in the size of the problem instance.

– This implies that B is at least as difficult as A.– It does not imply that A is at least as difficult as B.

slide 24