LECTURE 4: D&C, DYNAMIC PROGRAMMING...
Transcript of LECTURE 4: D&C, DYNAMIC PROGRAMMING...
ADVANCED ALGORITHMS LECTURE 4: D&C, DYNAMIC PROGRAMMING
�1
LECTURE 4
ANNOUNCEMENTS
▸ Homework 0 grading (close to done)
▸ Homework 1 posted; due on Monday Sep 10, 11:59 PM
▸ Start early! (Post comments/questions on canvas…)
▸ Unless immediate, argue why algorithm is correct
▸ Three steps — algorithm description, proof of correctness, complexity analysis
�2
LECTURE 4
LAST CLASS
�3
▸ Divide and conquer
▸ Break instance into smaller pieces, solve pieces separately, combine solutions
▸ Instances disjoint in examples we saw — not necessary (today)
▸ Merge-sort algorithm
▸ Recurrences
▸ Guess and prove by induction
▸ Plug and chug
▸ Recurrence tree
link oniMaster theorem homepage
Akira Bazzi Theorem
LECTURE 4
DIVIDE AND CONQUER
�4
PROBLEM INSTANCE
Divide step
Solve sub-problems
FULL SOLUTIONCombine/“conquer”
LECTURE 4
TODAY’S PLAN
�5
▸ More examples — divide and conquer
▸ Continue integer multiplication
▸ Median finding
▸ Introduction — dynamic programming
LECTURE 4
EXAMPLE 2: MULTIPLYING INTEGERS
�6
Question: given two n digit numbers, how efficiently can we multiply them?
▸ Elementary school algorithm: takes time O(n2)
▸ Basic divide and conquer — recap
LECTURE 4
DIVIDE AND CONQUER
�7
A = B =X Y Z W
A = 10n/2X + YSome observations: B = 10n/2Z + W
AB = 10nXZ + 10n/2(YZ + XW ) + YW
▸ Divide and conquer alg: compute XZ, YZ, XW, YW, compute sum above
▸ Gives recurrence: T(n) = 4T(n/2) + O(n)
n
Mz Mz X OO O Mz Mz
El D 7L'mazn digits
LECTURE 4
SOLVING RECURRENCE
�8
Tht 4TH Oln
1Tfn 042 f
E g mastertheorem
Corl plug a chug
Can we do better than elementaryschool Arg
LECTURE 4
CAN WE DO BETTER?
�9
Reason to hope: we only need to find three quantities:
XZ, (XW + YZ), YW
LECTURE 4
CAN WE DO BETTER?
�10
Reason to hope: we only need to find three quantities:
XZ, (XW + YZ), YW
Kartsuba’s observation:
XW + YZ = (X + Y )(W + Z) − XZ − YW
r
a
we needonly THREE multiplications
TIM 3 TIZI Oln
LECTURE 4
OVERALL RUNNING TIME
�11
TIM 3M c n g
Cin 131C I 1347
log C n 1 z cn 1 32T ngN
I
cnfltZztfzIt.r.JCatEHHcn.fzfzreaani.s.s 37171
LECTURE 4
EVEN BETTER?
�12
▸ Can we do linear time?
▸ HW1 — will see something approaching linear
I 35N
mK ite2 N
a Fast Fourier Transform Ofmlog n time alg42
Open problem if we can do better
LECTURE 4
COMPUTING THE MEDIAN
�13
Question: given an unsorted list A[0], A[1], …, A[n-1], find the (n/2)th smallest element
Divide and conquer?
Sort thearray output Ithsmallest O n log n0cal
An
n
medians of the pieces don't givemuch meaningful information
i l
LECTURE 4
KEY INSIGHT: “APPROXIMATE MEDIAN” HELPS!
�14
AH AH Aln i
tfAf AIDANTAIND Ifs
Fddi 12 elements
t213 13
Observations
1 we can now restrict search to
a 2.13 sized array een
2 needtobe careful answer is
not median of sub arraybut f
1h
Smallest element
ila qb6n
Given an array some k find theKth
smallest elementSelect A K
1 a select A Z Select Bg Eall elements
in Eu
1 n T Iz Oh time takentofindsub array
Ent E t C n t B
n l t Zz t E t Ept c'n
B
t n
213
Select Ez Z
How does recursive callwork
B24313Bx
Let a be njrd smallestelement of B
myth smallestelement of B is the
smallest element of B
select B Z FRida select A K wilt
a find approx medianselect B k
tidesThaty
cursive call10h a
givesdetails new array B sat IBIL F IAI
LECTURE 4
MEDIAN OF MEDIANS IS APPROXIMATE MEDIAN!
�15
Trick
AIA Afi AIn
o
Tfn 1 Fo 101Mt Tff
LECTURE 4
DIVIDE AND CONQUER — OVERVIEW
�16
▸ Problem “cleanly” divides into not-too-many sub-problems
▸ Solutions easily (efficiently) combined
▸ Leads to simple, efficient solutions
▸ Analysis by induction, complexity by recurrences
LECTURE 4
OVERLAP IN RECURSIVE CALLS
�17
▸ Divide & conquer: usually sub-problems don’t interact “much”
▸ What if they do? Can run into wastefulness — solving same sub-problem many times…
▸ Motivation for next topic — dynamic programming
LECTURE 4
NEW PROBLEM: SUBSET SUM
�18
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
isAfol t Akid t AEsB
all subsets find if any sum to S
2 n time
LECTURE 4
DIVIDE AND CONQUER
�19
▸ Try 1: divide into two halves
I 4
Alo f AlnDT T T T
o s
is.tl1 n Sti TIE
ntogs
s 1logan
LECTURE 4
RUNNING TIME
�20
logn
LECTURE 4
TRY 2
�21
▸ See if first element is picked or not
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
Ilookfords in AID Aln B lookfor a sumof
a sumof s Afo in AID
Takes time 2hAlnD
LECTURE 4
TRY 2
�22
▸ See if first element is picked or not
▸ Is this really divide and conquer?
Problem: given a set of non-negative integers A[0], A[1], …, A[n-1], and an integer S, find if there is a subset of A[i] whose sum is S.
LECTURE 4
RUNNING TIME
�23
LECTURE 4
LOOKING MORE CLOSELY — EXAMPLE
�24
A[] = {1, 2, 3, 5, 7, 9, 10, 11}; S = 20
LECTURE 4
AVOIDING MULTIPLE SOLVES ON “SAME” INSTANCE
�25
▸ Store answers!
▸ How much are we storing?
▸ Running time