CSE 331: Review.

CSE 331: Review

Main Steps in Algorithm DesignProblem Statement

Algorithm

Real world problem

Problem Definition Precise mathematical def

“Implementation” Data Structures

Analysis Correctness/Run time

Stable Matching Problem

Gale-Shaply Algorithm

Stable Marriage problem

Set of men M and women W

Matching (no polygamy in M X W)

Perfect Matching (everyone gets married)

Instablity

m w

m’ w’

Preferences (ranking of potential spouses)

Stable matching = perfect matching+ no instablity

Gale-Shapley AlgorithmIntially all men and women are free

While there exists a free woman who can propose

Let w be such a woman and m be the best man she has not proposed to

w proposes to m

If m is free

(m,w) get engaged

Else (m,w’) are engaged

If m prefers w’ to w

w remains freeElse

(m,w) get engaged and w’ is free

Output the engaged pairs as the final output

At most n2 iterations

O(1) time implementation

GS algorithm: Firefly Edition

1

1

2

2

3

3

4

4

5

5

6

6

Mal

Wash

Simon

Inara

Zoe

Kaylee

GS algo outputs a stable matching

Lemma 1: GS outputs a perfect matching S

Lemma 2: S has no instability

Proof technique de jour

Source: 4simpsons.wordpress.com

Proof by contradiction

Assume the negation of what you want to prove

After some reasoning

Two obervations

Obs 1: Once m is engaged he keeps getting engaged to “better” women

Obs 2: If w proposes to m’ first and then to m (or never proposes to m) then she prefers m’ to m

Proof of Lemma 2

By contradiction

m w

m’ w’

Assume there is an instability (m,w’)

m prefers w’ to w

w’ prefers m to m’

w’ last proposed to m’

Contradiction by Case Analysis

Depending on whether w’ had proposed to m or not

Case 1: w’ never proposed to m

w’ prefers m’ to m

Assumed w’ prefers m to m’

Source: 4simpsons.wordpress.com

By Obs 2

By Obs 1

Case 2: w’ had proposed to m

Case 2.1: m had accepted w’ proposalm is finally engaged to w

Thus, m prefers w to w’4simpsons.wordpress.com

Case 2.2: m had rejected w’ proposal

m was engaged to w’’ (prefers w’’ to w’)

m is finally engaged to w (prefers w to w’’)

m prefers w to w’

4simpsons.wordpress.com

By Obs 1

By Obs 1

Overall structure of case analysis

Did w’ propose to m?

Did m accept w’ proposal?

4simpsons.wordpress.com

4simpsons.wordpress.com 4simpsons.wordpress.com

Graph Searching

BFS/DFS

O(m+n) BFS Implementation

BFS(s)

CC[s] = T and CC[w] = F for every w≠ s

Set i = 0Set L0= {s}

While Li is not empty

Li+1 = Ø

For every u in Li

For every edge (u,w)

If CC[w] = F then

CC[w] = TAdd w to Li+1

i++

Array

Linked List

Input graph as Adjacency list

Version in KT also

computes a BFS tree

An illustration

1

2 3

4 5

6

7

8

1 2 3 4 5 7 8 6

O(m+n) DFS implementation

BFS(s)

CC[s] = T and CC[w] = F for every w≠ s

Intitialize Q= {s}

While Q is not empty

Delete the front element u in Q

For every edge (u,w)

If CC[w] = F then

CC[w] = TAdd w to the back of Q

O(n)

O(1)

O(1)

Repeated nu timesO(nu)

Repeated at most once for each

vertex u

Σu O(nu) = O(Σu nu) =

O(m) O(1)

A DFS run using an explicit stack

1

2 3

4 5

6

7

8

1

2

4

5

6

3

8

7

3

5

3

7

Topological Ordering

Run of TopOrd algorithm

Greedy Algorithms

Interval Scheduling: Maximum Number of Intervals

Schedule by Finish Time

End of Semester blues

Monday Tuesday Wednesday Thursday Friday

Project

331 HWExam study

Party!

Write up a term paper

Can only do one thing at any day: what is the maximum number of tasks that you can do?


Set A to be the empty set

While R is not empty

Choose i in R with the earliest finish time

Add i to A

Remove all requests that conflict with i from R

Return A*=A

O(n log n) time sort intervals such that f(i) ≤ f(i+1)

O(n) time build array s[1..n] s.t. s[i] = start time for i

Do the removal on

the fly

The final algorithm


Project

331 HWExam study

Party!


Order tasks by their END time

Proof of correctness uses“greedy stays ahead”

Interval Scheduling: Maximum Intervals


Scheduling to minimize lateness


Project

331 HW

Exam study

Party!


All the tasks have to be scheduledGOAL: minimize maximum lateness

The Greedy Algorithm

(Assume jobs sorted by deadline: d1≤ d2≤ ….. ≤ dn)

f=s

For every i in 1..n do

Schedule job i from s(i)=f to f(i)=f+ti

f=f+ti

Proof of Correctness uses“Exchange argument”

Proved the following

Any two schedules with 0 idle time and 0 inversions have the same max lateness

Greedy schedule has 0 idle time and 0 inversions

There is an optimal schedule with 0 idle time and 0 inversions

Shortest Path in a Graph: non-negative edge weights

Dijkstra’s Algorithm

Shortest Path problem

Input: Directed graph G=(V,E)

Edge lengths, le for e in E

“start” vertex s in V

Output: All shortest paths from s to all nodes in V

100

155

s

u

w

5

s

u15

5

s

u

w

Dijkstra’s shortest path algorithm

Input: Directed G=(V,E), le ≥ 0, s in V

R = {s}, d(s) =0

While there is a x not in R with (u,x) in E, u in R

d’(w) = min e=(u,w) in E, u in R d(u)+le

Pick w that minimizes d’(w) Add w to Rd(w) = d’(w)

s

w

u

z

x

y

1

2

4

3

3

1

2

1

2

d(s) = 0

1

4

2 s

u

d(u) = 1

4

2

w

d(w) = 2

5

x

d(x) = 2

3

4

y

d(y) = 3

z

d(z) = 4

Shortest paths

Dijkstra’s shortest path algorithm (formal)

Input: Directed G=(V,E), le ≥ 0, s in V

S = {s}, d(s) =0

While there is a v not in S with (u,v) in E, u in S

Pick w that minimizes d’(w) Add w to Sd(w) = d’(w)

At most n iterations

O(m) time

O(mn) time bound is trivial

O(m log n) time implementation is possible

Proved that d’(v) is best when v is added

Minimum Spanning Tree

Kruskal/Prim

Minimum Spanning Tree (MST)

Input: A connected graph G=(V,E), ce> 0 for every e in E

Output: A tree containing all V that minimizes the sum of edge weights

Kruskal’s Algorithm

Joseph B. Kruskal

Input: G=(V,E), ce> 0 for every e in E

T = Ø

Sort edges in increasing order of their cost

Consider edges in sorted order

If an edge can be added to T without adding a cycle then add it to T

Prim’s algorithm

Robert Prim

Similar to Dijkstra’s algorithm

Input: G=(V,E), ce> 0 for every e in E

2

1 3

51

50

0.5

S = {s}, T = Ø

While S is not the same as V

Among edges e= (u,w) with u in S and w not in S, pick one with minimum cost

Add w to S, e to T

2

1 50

0.5

Cut Property Lemma for MSTs

S V \ S

Cheapest crossing edge is in all MSTs

Condition: S and V\S are non-empty

Assumption: All edge costs are distinct

Divide & Conquer

Sorting

Merge-Sort

Sorting

Given n numbers order them from smallest to largest

Works for any set of elements on which there is a total order

Mergesort algorithmInput: a1, a2, …, an Output: Numbers in sorted order

MergeSort( a, n )

If n = 2 return the order min(a1,a2); max(a1,a2)

aL = a1,…, an/2

aR = an/2+1,…, an

return MERGE ( MergeSort(aL, n/2), MergeSort(aR, n/2) )

An example run

MergeSort( a, n )


aL = a1,…, an/2

aR = an/2+1,…, an


151 100 19 2 8 34

511 19 100

1 19 51 100

2 8 43

2 3 4 8

1 2 3 4 8 19 51 100

CorrectnessInput: a1, a2, …, an Output: Numbers in sorted order

MergeSort( a, n )


aL = a1,…, an/2

aR = an/2+1,…, an


By induction

on n

Inductive step follows from correctness of MERGE

If n = 1 return the order a1

Counting Inversions

Merge-Count

Mergesort-Count algorithmInput: a1, a2, …, an Output: Numbers in sorted order+ #inversion

MergeSortCount( a, n )

If n = 2 return ( a1 > a2, min(a1,a2); max(a1,a2))

aL = a1,…, an/2 aR = an/2+1,…, an

return (c+cL+cR,a)

(cL, aL) = MergeSortCount(aL, n/2)

(cR, aR) = MergeSortCount(aR, n/2)

(c, a) = MERGE-COUNT(aL,aR) Counts #crossing-inversions+ MERGE

O(n)

T(2) = c

T(n) = 2T(n/2) + cn

O(n log n) time

If n = 1 return ( 0 , a1)

Closest Pair of Points

Closest Pair of Points Algorithm

Closest pairs of points

Input: n 2-D points P = {p1,…,pn}; pi=(xi,yi)

Output: Points p and q that are closest

d(pi,pj) = ( (xi-xj)2+(yi-yj)2)1/2

The algorithmInput: n 2-D points P = {p1,…,pn}; pi=(xi,yi)

Sort P to get Px and Py

Q is first half of Px and R is the rest

Closest-Pair (Px, Py)

Compute Qx, Qy, Rx and Ry

(q0,q1) = Closest-Pair (Qx, Qy)

(r0,r1) = Closest-Pair (Rx, Ry)

δ = min ( d(q0,q1), d(r0,r1) )

S = points (x,y) in P s.t. |x – x*| < δ

return Closest-in-box (S, (q0,q1), (r0,r1))

If n < 4 then find closest point by brute-force

Assume can be done in O(n)

O(n log n)

O(n)

O(n)

O(n)

O(n)

O(n log n) + T(n)

T(< 4) = c

T(n) = 2T(n/2) + cn

O(n log n) overall

Dynamic Programming

Weighted Interval Scheduling

Scheduling Algorithm

Weighted Interval Scheduling

Input: n jobs (si,ti,vi)

Output: A schedule S s.t. no two jobs in S have a conflict

Goal: max Σi in S vj

Assume: jobs are sorted by their finish time

A recursive algorithm

Compute-Opt(j)

If j = 0 then return 0

return max { vj + Compute-Opt( p(j) ), Compute-Opt( j-1 ) }

OPT(j) = max { vj + OPT( p(j) ), OPT(j-1) }

Proof of correctness by induction on jCorrect for j=0

= OPT( p(j) ) = OPT( j-1 )

Exponential Running Time1

23

4

5

p(j) = j-2

OPT(5)

OPT(3) OPT(4)

OPT(1) OPT(2)

OPT(1) OPT(1)

OPT(1)OPT(2)

OPT(1)

OPT(2)

OPT(3)Formal

proof: Ex.

Only 5 OPT values!

Bounding # recursionsM-Compute-Opt(j)

If j = 0 then return 0

M[j] = max { vj + M-Compute-Opt( p(j) ), M-Compute-Opt( j-1 ) }

If M[j] is not null then return M[j]

return M[j]

Whenever a recursive call is made an M value of assigned

At most n values of M can be assigned

O(n) overall

Property of OPT

OPT(j) = max { vj + OPT( p(j) ), OPT(j-1) }

Given OPT(1), …, OPT(j-1), one can compute OPT(j)

Recursion+ memory = IterationIteratively compute the OPT(j) values

M[0] = 0

M[j] = max { vj + M[p(j)], M[j-1] }

For j=1,…,n

Iterative-Compute-Opt

M[j] = OPT(j) O(n) run time

Knapsack Problem

Knapsack Algorithm

Subset Sum ProblemMaximize the weight packed into a bag

*ref: Images from http://minecraft.gamepedia.com

w=10

w=8

w=15

w=33

Capacity:W=40

w=12

w=5

w=12

w=17

http://minecraft.gamepedia.com/


Subset Sum Problem

Input: A set of n items each with weight wi > 0 and a capacity W

Output: A subset of the n items with maximum sum of weights under the constraint that the sum weights is ≤W

Knapsack ProblemMaximize the value packed into the bag

*ref: Images from http://minecraft.gamepedia.com

w=10v=1

w=8v=200

w=15v=76

w=35v=50

Capacity:W=40

w=12v=14

w=1000v=20

w=5v=19

w=18v=21



Knapsack Problem

Input: A set of n items each with weight wi > 0 and value vi>0A capacity W

Output: A subset of the n items with maximum sum of valueunder the constraint that the sum weights is ≤W

Dynamic Programming Algorithm:Subset Sum

OPT(i,w) = Maximum weight packed given the first i item and capacity w

OPT(i,w) = max { OPT(i-1,w), wi + OPT(i-1, w-wi)}

Pack item iDon’t pack item i

If item i can’t fit (wi>w) then OPT(i,w) = OPT(i-1,w)

For each OPT(i,w), decide if item i should be packed or not:

If item i can fit:

Output OPT(n,W): With some book keeping, can also output the packed set

Dynamic Programming Algorithm:Knapsack Problem

OPT(i,w) = Maximum value packed given the first i item and capacity w

OPT(i,w) = max { OPT(i-1,w), vi + OPT(i-1, w-wi)}

Pack item iDon’t pack item i

If item i can’t fit (wi>w) then OPT(i,w) = OPT(i-1,w)

Decide if item i should be packed or not:

If item i can fit:

Output OPT(n,W): With some book keeping, can also output the packed set

Runtime

OPT(i,w) = max { OPT(i-1,w), vi + OPT(i-1, w-wi)}

OPT(i,w) is computed in constant time

nW entries in OPT to be computed for an O(nW) runtime

Shortest Path in a Graph

Bellman-Ford

Shortest Path Problem

Input: (Directed) Graph G=(V,E) and for every edge e has a cost ce (can be <0)

t in V

Output: Shortest path from every s to t

1 1

100

-1000

899

s t

Shortest path has cost negative

infinity

Assume that G has no negative

cycle

Recurrence Relation

OPT(i,u) = cost of shortest path from u to t with at most i edges

OPT(i,u) = min { OPT(i-1,u), min(u,w) in E { cu,w + OPT(i-1, w)} }

Path uses ≤ i-1 edges Best path through all neighbors

P vs NP

P vs NP question

P: problems that can be solved by poly time algorithms

NP: problems that have polynomial time verifiable witness to optimal solution

Is P=NP?

Alternate NP definition: Guess witness and verify!

CSE 331: Review.

Documents

Transcript of CSE 331: Review.