4 -1 Chapter 4 The Divide-and-Conquer Strategy. 4 -2 Outlines 4-1 The 2-Dimensional Maxima Finding...

4 -1

Chapter 4

The Divide-and-Conquer Strategy

4 -2

Outlines

4-1 The 2-Dimensional Maxima Finding Problem

4-2 The Closest Pair Problem

4-3 The Convex Hull Problem

4-4 The Voronoi Diagrams Constructed by the Divide-and-Conquer Strategy

4-5 Applications of the Voronoi Diagrams

4-6 The Fast Fourier Transform

4-7 The Experimental Results

4 -3

Introduction divide-and-conquer strategy

first divides a problem into two smaller sub-problems and each sub-problem is identical to its original problem, except its input size is smaller.

Both sub-problems are then solved and the sub-solutions are finally merged into the final solution.

These two sub-problems themselves can be solved by the divide-and-conquer strategy again.

Or, to put it in another way, these two sub-problems are solved recursively.

4 -4

A simple example

Finding the maximum of a set S of n numbers. Dividing the input into two sets, each set consisting of

n/2 numbers. Let us call these two sets S1 and S2.

Find the maximums of S1 and S2 respectively.

Let the maximum of Si be denoted as Xi, i =1, 2.

Then the maximum of S can be found by comparing X1 and X2. Whichever is the larger is the maximum of S.

4 -5

A simple example finding the maximum of a set S of n

numbers

4 -6

Time Complexity In general, the complexity T(n) of a divide-and-

conquer algorithm is determined by the following formulas:

where S(n) denotes the time steps needed to split the

problem into two sub-problems, M(n) denotes the time steps needed to merge two

sub-solutions and b is a constant.

T(n)=

2T(n/2)+S(n)+M(n) b

, n≧ c , n<c

4 -7

Time complexity:

Calculation of T(n): Assume n = 2k,

T(n) = 2T(n/2)+1= 2(2T(n/4)+1)+1= 4T(n/4)+2+1

:=2k-1T(2)+2k-2+…+4+2+1=2k-1+2k-2+…+4+2+1=2k-1 = n-1

T(n)= 2T(n/2)+1 1 , n>2 , n2

Time complexity

4 -8

A general divide-and-conquer algorithm

Step 1: If the problem size is small, solve this problem directly; otherwise, split the original problem into 2 sub-problems with equal sizes.

Step 2: Recursively solve these 2 sub-problems by applying this algorithm.

Step 3: Merge the solutions of the 2 sub- problems into a solution of the original problem.

4 -9

Time complexity of the general algorithm

Time complexity:

where S(n) : time for splitting M(n) : time for merging b : a constant

c : a constant e.g. Binary search e.g. quick sort e.g. merge sort

T(n)= 2T(n/2)+S(n)+M(n) b

, n c , n < c

4 -10

4.1 2-D maxima finding problem

Def : A point (x1, y1) dominates (x2, y2) if x1 > x2 and y1 > y2. A point is called a maxima if no other point dominates it

Maxima finding problem: find the maximal points among these n points.

Straightforward method : Compare every pair of points.

Time complexity: O(n2)

4 -11

Divide-and-conquer for maxima finding

The maximal points of SL and SR

Perpendicular line

Median point

4 -12

Merge Step The merging process is rather simple. Since the x-value of a point in SR is always

larger than the x-value of every point in SL.

A point in SL is a maxima if and only if its y-value is not less than the y-value of a maxima of SR.

4 4 -13

The algorithm: Input: A set of n planar points. Output: The maximal points of S.Step 1: If S contains only one point, return it as the

maxima. Otherwise, find a line L perpendicular to the X-axis which separates the set of points into two subsets SLand SR , each of which consisting of n/2 points.

Step 2: Recursively find the maximal points of SL and SR .

Step 3: Find the largest y-value of SR. Project the maximal points of SL onto L. Discard each of the maximal points of SL if its y-value is less than the largest y-value of SR .

4 4 -14

Time complexity: T(n) Step 1: O(n) median finding Step 2: 2T(n/2) Step 3: O(nlogn) :sorting n points

according to their y-value.

Assume n = 2k

T(n) = O(n log n) +O(nlog2n)= O(nlog2n)

T(n)=

2T(n/2)+O(n)+O(nlogn) 1

, n > 1 , n = 1

4 4 -15

Improvement We note that our divide-and-conquer strategy is dominated by

sorting in the merging steps. Somehow we are not doing a very efficient job because sorting sorting

should be done once and for all.should be done once and for all. That is, we should conduct a presorting. If this is done, the

merging complexity is O(n) and the total number of time steps needed is O(nlogn) + T(n)

where

and T(n) can be easily found to be O(nlogn). Thus the total time-complexity of using the divide-and-conquer strategy to find maximal points with presorting is O(nlogn).

T(n)=

2T(n/2)+O(n)+O(n) 1

, n > 1 , n = 1

4 4 -16

Merge Sort

1)()2/(2

1)1()(

nnnT

nnT

1)2/(2

1)(

ncnnT

ncnT

Rewrite:

Becomes A Recursion Tree

4 4 -17

Merge Sort (Recursion Tree)

1)2/(2

1)(

ncnnT

ncnT

)2/(2)( nTcnnT

4 4 -18


1)2/(2

1)(

ncnnT

ncnT

)2/(2)( nTcnnT

)4/(42

2 nTcn

cn

4 4 -19


1)2/(2

1)(

ncnnT

ncnT

)2/(2)( nTcnnT

?...4

42

2

cncncn

)4/(42

2 nTcn

cn

4 4 -20


1)2/(2

1)(

ncnnT

ncnT

)2/(2)( nTcnnT

?...4

42

2

cncncn

)4/(42

2 nTcn

cn

nin

ilg,1

2when

4 4 -21


1)2/(2

1)(

ncnnT

ncnT

cnncnnT lg)(

)lg()( nnnT

4 4 -22

Recurrence

1)2/(2

1)(

ncnnT

ncnT

is a recurrence.

Recurrence: an equation that describes a function in terms of its value on smaller functions

4 4 -23

Recurrence (Examples)

0

0

)1(

0)(

n

n

nTcnT

0)1(

00)(

nnTn

nnT

1

1

)2/(2)(

n

n

cnT

cnT

1

1

)/()(

n

n

cnbnaT

cnT

4 4 -24

Solving Recurrences

Substitution method

Iteration Method

Recursion-tree method

Master method

4 4 -25

Substitution Method

Guess the form of the solution Use mathematical induction to find the

constants and show that the solution works

Examples: T(n) = 2T(n / 2) + (n) T(n) = (n lg n)

We already know

4 4 -26

Substitution Method




T(n) = 2T(n / 2) + n ???

4 4 -27

Substitution Method




T(n) = 2T(n / 2) + n T(n) = (n lg n)

Need to Prove by Induction

4 4 -28

Changing Variables

Sometimes, changing variables may help Examples:

T(n) = 2T(n / 2) + n T(n) = O(n lg n)

We already know

4 4 -29

Changing Variables


T(n) = 2T(n / 2) + n T(n) = O(n lg n)

nnTnT lg)(2)(

4 4 -30

Changing Variables


T(n) = 2T(n / 2) + n T(n) = O(n lg n)

nnTnT lg)(2)( mnornm 2,lg Let

4 4 -31

Changing Variables


T(n) = 2T(n / 2) + n T(n) = O(n lg n)


mmSmS )2/(2)(

4 4 -32

Changing Variables


T(n) = 2T(n / 2) + n T(n) = O(n lg n)


mmSmS )2/(2)(

T(n) = S(m) = O(m lg m) = O(lg n lg lg n)

4 4 -33

Iteration Method Expand the recurrence Work some algebra to express as a

summation Evaluate the summation

4 4 -34

Iteration Method (Example)

0)1(

00)(

nnTc

nnT

4 4 -35


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

4 4 -36


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2)

4 4 -37


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

4 4 -38


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3)

4 4 -39


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3) = 3c + T(n-3)

4 4 -40


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3) = 3c + T(n-3)

= ...

4 4 -41


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3) = 3c + T(n-3)

= …

= kc + T(n-k) Set k = ?

4 4 -42


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3) = 3c + T(n-3)

= …

= kc + T(n-k) Set k = n

4 4 -43


0)1(

00)(

nnTc

nnT

T(n) = c + T(n-1)

= c + c + T(n-2) = 2c + T(n-2)

= 2c + c + T(n-3) = 3c + T(n-3)

= …

= kc + T(n-k)

= nc + T(0) = nc

T(n) = (n)

Set k = n

4 4 -44


0)1(

00)(

nnTn

nnT

4 4 -45


T(n) = n + T(n-1)

0)1(

00)(

nnTn

nnT

4 4 -46


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

0)1(

00)(

nnTn

nnT

4 4 -47


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

0)1(

00)(

nnTn

nnT

4 4 -48


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

= …

0)1(

00)(

nnTn

nnT

4 4 -49


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

= …

= n + n-1 + n-2 + … + n-(k-1) + T(n-k)

0)1(

00)(

nnTn

nnT

4 4 -50


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

= …

= n + n-1 + n-2 + … + n-(k-1) + T(n-k)

0)1(

00)(

nnTn

nnT

Set k = n

4 4 -51


T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

= …

= n + n-1 + n-2 + … + n-(k-1) + T(n-k)

= n + n-1 + n-2 + … + 1 + T(0)

0)1(

00)(

nnTn

nnT

Set k = n

4 4 -52

T(n) = n + T(n-1)

= n + n-1 + T(n-2)

= n + n-1 + n-2 + T(n-3)

= …

= n + n-1 + n-2 + … + n-(k-1) + T(n-k)

= n + n-1 + n-2 + … + 1 + T(0)

=


0)1(

00)(

nnTn

nnT

2

)1( nnT(n) = (n2)

4 4 -53

Recursion-Tree Method Expand the recurrence Construct a recursion-tree Sum the costs

4 4 -54

Recursion-Tree Method (Example)

2)4/(3)( cnnTnT

4 4 -55


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

4 4 -56


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

4 4 -57


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

?...

16

3

16

3 22

22

cncncn

4 4 -58


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)4/(316

3...

16

3

16

3

21

22

22

iii

nTcn

cncncn

4 4 -59


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)4/(316

3...

16

3

16

3

21

22

22

iii

nTcn

cncncn

nin

i 4log,14

4 4 -60


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

4 4 -61


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

)(

)1()1(33log

3loglog

4

44

n

nTn

4 4 -62


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)(16

3 3log1log

0

2 4

4

ncnn

i

i

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

4 4 -63


Recall)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)(16

3 3log1log

0

2 4

4

ncnn

i

i

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

)(1)16/3(

1)16/3( 3log2log

4

4

ncnn

1

11

0

r

rr

nn

i

i

4 4 -64


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)(16

3 3log1log

0

2 4

4

ncnn

i

i

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

)(1)16/3(

1)16/3( 3log2log

4

4

ncnn

)(16

3 3log2

0

4ncni

i

4 4 -65


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)(16

3 3log1log

0

2 4

4

ncnn

i

i

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

)(1)16/3(

1)16/3( 3log2log

4

4

ncnn

)(16

3 3log2

0

4ncni

i

)()16/3(1

1 3log2 4ncn

4 4 -66


)4/(3

)4/(3)(2

2

nTcn

cnnTnT

)16/(916

3 22 nTcncn

)(16

3 3log1log

0

2 4

4

ncnn

i

i

)1(316

3...

16

3

16

3

4

4

log21log

22

22

Tcn

cncncn

nn

)(1)16/3(

1)16/3( 3log2log

4

4

ncnn

)(16

3 3log2

0

4ncni

i

)()16/3(1

1 3log2 4ncn

)(13

16 3log2 4ncn

T(n) = O(n2)

4 4 -67

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

4 68

Algorithms Analysis

Master Theorem

4 4 -69

Master Theorem*** Provide a “cookbook” method for solving

recurrences Divide-and-conquer algorithm

An algorithm that divides the problem of size n into a subproblems, each of size n / b

4 4 -70

Master Theorem11

)()/()(

banda

nfbnaTnT

)()(),()( loglog aa bb nnTthennOnfIf

)lg()(),()( loglog nnnTthennnfIf aa bb

))(()(

),()/(),()( log

nfnTthen

ncfbnafifandnnfIf ab

1.

2.

3.

1,0 c

4 4 -71

Notes on Master Theorem

Some technicalities:In case 1, f (n) must be polynomially smaller than

by a factor of

In case 3, f (n) must be polynomially larger than by a factor of

The three cases doesn’t cover all possibilities of f (n). Can’t use Master Theorem

abn log 0, n

abn log 0, n

4 4 -72

Using the Master Method

nnTnT )3/(9)(

11

)()/()(

banda

nfbnaTnT

4 4 -73


nnTnT )3/(9)(

11

)()/()(

banda

nfbnaTnT

nnfba )(,3,9

11

)()/()(

banda

nfbnaTnT

4 4 -74


nnTnT )3/(9)(

nnfba )(,3,9

)( 229loglog 3 nnnn ab

11

)()/()(

banda

nfbnaTnT

4 4 -75


nnTnT )3/(9)(

nnfba )(,3,9


1),()( 9log3 wherenOnf

11

)()/()(

banda

nfbnaTnT

4 4 -76


nnTnT )3/(9)(

nnfba )(,3,9



)()(),()( loglog aa bb nnTthennOnfIf Use Case 1:

11

)()/()(

banda

nfbnaTnT

4 4 -77


nnTnT )3/(9)(

nnfba )(,3,9



)()(),()( loglog aa bb nnTthennOnfIf Use Case 1:

)()( 2nnT

11

)()/()(

banda

nfbnaTnT

4 4 -78


1)3/2()( nTnT

11

)()/()(

banda

nfbnaTnT

4 4 -79


1)3/2()( nTnT

1)(,2/3,1 nfba

11

)()/()(

banda

nfbnaTnT

4 4 -80


1)3/2()( nTnT

1)(,2/3,1 nfba

101loglog 2/3 nnn ab

11

)()/()(

banda

nfbnaTnT

4 4 -81


1)3/2()( nTnT

1)(,2/3,1 nfba


)1()()( log abnnf

11

)()/()(

banda

nfbnaTnT

4 4 -82


1)3/2()( nTnT

11

)()/()(

banda

nfbnaTnT

1)(,2/3,1 nfba


)1()()( log abnnf

Use Case 2: )lg()(),()( loglog nnnTthennnfIf aa bb

11

)()/()(

banda

nfbnaTnT

4 4 -83


1)3/2()( nTnT

1)(,2/3,1 nfba


)1()()( log abnnf

Use Case 2: )lg()(),()( loglog nnnTthennnfIf aa bb

)(lg)( nnT

11

)()/()(

banda

nfbnaTnT

4 4 -84


nnnTnT lg)4/(3)(

11

)()/()(

banda

nfbnaTnT

4 4 -85


nnnTnT lg)4/(3)( nnnfba lg)(,4,3

11

)()/()(

banda

nfbnaTnT

4 4 -86



)( 793.03loglog 4 nOnn ab

11

)()/()(

banda

nfbnaTnT

4 4 -87



)( 793.03loglog 4 nOnn ab 2.0),()( 3log4 wherennf

11

)()/()(

banda

nfbnaTnT

4 4 -88




4/3),(lg)4/3()4/lg()4/(3)/( cforncfnnnnbnaf

11

)()/()(

banda

nfbnaTnT

4 4 -89





Use Case 3:))(()(

),()/(),()( log

nfnTthen


11

)()/()(

banda

nfbnaTnT

4 4 -90





Use Case 3:))(()(

),()/(),()( log

nfnTthen


)lg()( nnnT

11

)()/()(

banda

nfbnaTnT

4 4 -91


nnnTnT lg)2/(2)(

11

)()/()(

banda

nfbnaTnT

4 4 -92


nnnTnT lg)2/(2)(

nnnfba lg)(,2,2

11

)()/()(

banda

nfbnaTnT

4 4 -93


nnnTnT lg)2/(2)(

nnnfba lg)(,2,2

But is not polynomially larger than

nnnf lg)( nn ab log

nnnnnnf ab lg/)lg(/)( log is asymptotically less than0, n

Master Method doesn’t Apply

11

)()/()(

banda

nfbnaTnT

4 4 -94

Extended Master Method 1band1a

f(n)aT(n/b)T(n)

)lg()(

,0),lg()(1log

log

nnnTthen

knnnfIfka

ka

b

b

4 4 -95

Extended Master Method 11

)()/()(

banda

nfbnaTnT

)lg()(

,0),lg()(1log

log

nnnTthen

knnnfIfka

ka

b

b

Back to the previous recurrence:

nnnTnT lg)2/(2)(

4 4 -96

Extended Master Method 11

)()/()(

banda

nfbnaTnT

)lg()(

,0),lg()(1log

log

nnnTthen

knnnfIfka

ka

b

b

Back to the previous recurrence:

nnnTnT lg)2/(2)(

1),lg()( log kandnnnf ab

)lg()( 2 nnnT

4 4 -97

Proof of the Master Theorem

1,1 ba

Lemma 4.2

ibnifnfbnaT

nifnT

)()/(

1)1()(If

Then

1log

0

log )/()()(n

j

jjab

b bnfannT

4 4 -98

Proof of Master Theorem

)/()()( bnaTnfnT

4 4 -99


)/()()( bnaTnfnT

)/()/()( 22 bnTabnafnf

4 4 -100


)/()()( bnaTnfnT


?)/()/()( 22 bnfabnafnf

4 4 -101


)/()()( bnaTnfnT


)/()/(

)/()/()(11

22

kkkk bnTabnfa

bnfabnafnf

4 4 -102


)/()()( bnaTnfnT


)/()/(

)/()/()(11

22

kkkk bnTabnfa

bnfabnafnf

)1()1( Twhen nk blog

4 4 -103


)/()()( bnaTnfnT


)/()/(

)/()/()(11

22

kkkk bnTabnfa

bnfabnafnf

)1()1( Twhen nk blog)()1()1()1( loglogloglog aaan bbbb nnTnTa

4 4 -104


)/()()( bnaTnfnT


)/()/(

)/()/()(11

22

kkkk bnTabnfa

bnfabnafnf

)1()1( Twhen nk blog)()1()1()1( loglogloglog aaan bbbb nnTnTa

1log

0

log )/()()(n

j

jjab

b bnfannT Lemma 4.2 proved

4 4 -105

Proof of Master Theorem (Lemma 4.2)

4 4 -106


1,1 ba

Lemma 4.3

1log

0

)/()(n

j

jjb

bnfang

1.

2.

3.

)()(),()( loglog aa bb nOngthennOnfIf

)lg()(),()( loglog nnngthennnfIf aa bb

))(()(,)()/( nfngthenbnandncfbnafIf

1,0 c

4 4 -107

Proof of the Master Theorem For case 1:

)()( log abnOnf

))/(()/( log ajj bbnObnf

1log

0

log

)(n

j

a

jj

b b

b

naOng

4 4 -108


)()( log abnOnf


1log

0

log

)(n

j

a

jj

b b

b

naOng

1log

0log

log1log

0

log n

j

j

aa

n

j

a

jj

b

b

b

b b

b

abn

b

na

1log

0

log )(n

j

jab

b bn

1

1loglog

b

bn

na

b

b

1

1log

b

nn ab

4 4 -109


)()( log abnOnf


1log

0

log

)(n

j

a

jj

b b

b

naOng

1log

0log

log1log

0

log n

j

j

aa

n

j

a

jj

b

b

b

b b

b

abn

b

na

1log

0

log )(n

j

jab

b bn

1

1loglog

b

bn

na

b

b

1

1log

b

nn ab

Since

)()( loglog aa bb nOnOn

4 4 -110


)()( log abnOnf


1log

0

log

)(n

j

a

jj

b b

b

naOng

1log

0log

log1log

0

log n

j

j

aa

n

j

a

jj

b

b

b

b b

b

abn

b

na

1log

0

log )(n

j

jab

b bn

1

1loglog

b

bn

na

b

b

1

1log

b

nn ab abnOng log)(

Case 1 Proved

4 4 -111


)()( log abnnf

We have ))/(()/( log ajj bbnbnf

4 4 -112


)()( log abnnf

We have

1log

0

log

)(n

j

a

j

jb b

b

nang

))/(()/( log ajj bbnbnf

4 4 -113


)()( log abnnf

We have

1log

0

log

)(n

j

a

j

jb b

b

nang


1log

0log

log1log

0

log n

j

j

a

an

j

a

j

jb

b

b

b b

b

an

b

na

1log

1

log 1n

j

ab

bn

nn bab loglog

4 4 -114


)()( log abnnf

We have

1log

0

log

)(n

j

a

j

jb b

b

nang


1log

0log

log1log

0

log n

j

j

a

an

j

a

j

jb

b

b

b b

b

an

b

na

1log

1

log 1n

j

ab

bn

nn bab loglog )lg()log()( loglog nnnnng a

ba bb

Case 2 Proved

4 4 -115


)()/( ncfbnaf

)()/()/( nfacbnf

4 4 -116


)()/( ncfbnaf

)()/()/( nfacbnf Repeat j time:

)()/()()/()/( nfcbnfaornfacbnf jjjjj

4 4 -117


)()/( ncfbnaf



1log

0

1log

0

)()/()(n

j

jn

j

jjbb

nfcbnfang

Therefore

4 4 -118


)()/( ncfbnaf



1log

0

1log

0

)()/()(n

j

jn

j

jjbb

nfcbnfang

Therefore

0

)(j

jcnf

4 4 -119


)()/( ncfbnaf



1log

0

1log

0

)()/()(n

j

jn

j

jjbb

nfcbnfang

Therefore

))((1

1)( nfO

cnf

0

)(j

jcnf

Case 3 Proved

Lemma 4.3 Proved

4 4 -120


1,1 ba

Lemma 4.4

ibnifnfbnaT

nifnT

)()/(

1)1()(If

)()(),()( loglog aa bb nnTthennOnfIf

)lg()(),()( loglog nnnTthennnfIf aa bb

))(()(

),()/(),()( log

nfnTthen


1.

2.

3.

1,0 c

4 4 -121

Proof of the Master Theorem By combing Lemma 4.2 & 4.3

For Case 1:

For Case 2:

For Case 3:

))(())(()()( log nfnfnnT ab

)()()()( logloglog aaa bbb nnnnT

)lg()lg()()( logloglog nnnnnnT aaa bbb

because )()( log abnnfLemma 4.4 Proved

4 4 -122

Proof of the Master Theorem Lemma 4.2, 4.3, and 4.4 proved the Master

Theorem We skip the proof when floors and ceilings are

used in the Master Theorem

)()/()(

)()/()(

nfbnaTnT

nfbnaTnT

4 4 -123

Master Theorem

4 4 -124

4 4 -125

4 4 -126

4 4 -127

4 4 -128


4 4 -129


4 4 -130

4.2 The closest pair problem Given a set S of n points, find a pair of

points which are closest together. 1-D version :

Solved by sortingTime complexity :

O(n log n)

2-D version

4 4 -131

D&C We first partition the set S into SL and SR such that every point in

SL lies to the left of every point in SR and the number of points in SL is equal to that in SR.

Find a vertical line L perpendicular to the x-axis such that S is cut into two equal sized subsets.

Solving the closest pair problems in SL and SR respectively, we shall obtain dL and dR where dL and dR denote the distances of the closest pairs in SL and SR respectively.

Let d = min(dL, dR).

If the closest pair (Pa„ Pb) of S consists of a point in SL and a point in SR, then Pa and Pb must lie within a slab centered at line L and bounded by lines L-d and L+d.

Merge Step: examine points in slab.

4 4 -132

Points in Slab During the merging step, we may examine only points

in the slab. Although in average, the number of points within the

slab may not be too large, in the worst case, there can be as many as n points within the slab.

Thus the brute-force way to find the closest pair in the slab needs calculating n2/4 distances and comparisons.

This kind of merging step will not be good for our divide-and-conquer algorithm.

Fortunately, as will be shown in the following, the merging step can be accomplished in 0(n) time. (how?)

4 4 -133

Merge Step If a point P in SL and a point Q in SR constitute a

closest pair, the distance between P and Q must be less than d. Hence we do not have to consider a point too far away from P. Consider Figure 4-5.

We only have to examine the shaded area in Figure 4-5. If P is exactly on line L, this shaded area will be the largest.

Even in this case, this shaded area will be contained in the d2d rectangle A as shown in Figure 5-6.

Thus we only have to examine points within this rectangle A.

4 4 -134

Merge Step

For each point P in the slab, we only have to examine limited number of points in the other half of the slab.

Without losing generality, we may assume that P is within the left-half of the slab.

Let the y-value of P be denoted as yp. For P, we only have to examine points in the other half of the slab whose y-values are within yp+d and yp -d.

There will be at most six such points as discussed above (why?).

O(n) Step

4 4 -135

at most 6 points in area A:

d/2

d/2

One box contains one point.If s,s’ S have the property that d(s,s’)<d, then s and s’ are within 15 positions of each other in the sorted list Sy.

Sort points by x-values and sort points by y-values.

LBox

4 4 -136

The algorithm: Input: A set of n planar points. Output: The distance between two closest

points. Step 1: Sort points in S according to their y-values

and x-values.Step 2: If S contains only one points, return

infinity(∞) as their distance.Step 3: Find a median line L perpendicular to the

X-axis to divide S into two subsets, with equal sizes, SL and SR.

Step 4: Recursively apply Step 2 and Step 3 to solve the closest pair problems of SL and SR. Let dL(dR) denote the distance between the closest pair in SL (SR). Let d = min(dL, dR).

4 4 -137

Step 5: For a point P in the half-slab bounded by L-d and L, let its y-value by denoted as yP . For each such P, find all points in the half-slab bounded by L and L+d whose y-value fall within yP +d and yP -d. If the distance d between P and a point in the other half-slab is less than d, let d=d . The final value of d is the answer.

Time complexity: O(n log n) Step 1: O(n log n) Steps 2~5:

T(n) = O(n log n)

T(n)= 2T(n/2)+O(n)+O(n) 1

, n > 1 , n = 1

4 4 -138

4.3 The convex hull problem

The convex hull of a set of planar points is the smallest convex polygon containing all of the points.

concave polygon:

convex polygon:

Convex Hull 139

Convex Polygon A convex polygon is a nonintersecting polygon whose internal angles are

all convex (i.e., less than ) In a convex polygon, a segment joining two vertices of the polygon lies

entirely inside the polygon The convex hull of a set of points is the smallest convex polygon

containing the points Think of a rubber band snapping around the points

convex nonconvex

Convex Hull 140

Special Cases

The convex hull is a segment

Two points All the points

are collinear The convex hull is

a point there is one

point All the points

are coincident

Convex Hull 141

Applications

Motion planning Find an optimal route that avoids obstacles for a

robot Geometric algorithms

Convex hull is like a two-dimensional sorting

obstacle

startend

Convex Hull 142

Computing the Convex Hull The following method computes the convex hull of a set of points

Phase 1: Find the lowest point (anchor point)Phase 2: Form a nonintersecting polygon by sorting the points

counterclockwise around the anchor pointPhase 3: While the polygon has a nonconvex vertex, remove it

The orientation of a triplet (p1, p2, p3) of points in the plane is counterclockwise, clockwise, or collinear, depending on whether , (p1,p2,p3) is positive, negative, or zero, respectively.

4 -143

Convex Hull 144

Orientation The orientation of three points in the plane

is clockwise, counterclockwise, or collinear orientation(a, b, c)

clockwise (CW, right turn) counterclockwise (CCW, left turn) collinear (COLL, no turn)

The orientation of three points is characterized by the sign of the determinant (a, b, c), whose absolute value is twice the area of the triangle with vertices a, b and c

a

b

c

a

c

b

c

ba

CW

CCW

COLL

1

1

1

),,(

cc

bb

aa

yx

yx

yx

cba

4 4 -145

Convex Hull 146

Sorting by Angle

Computing angles from coordinates is complex and leads to numerical inaccuracy

We can sort a set of points by angle with respect to the anchor point a using a comparator based on the orientation function

b c orientation(a, b, c) CCW b c orientation(a, b, c) COLL b c orientation(a, b, c) CW

a

bc

CCW

a

cb

CW

a

bc

COLL

Convex Hull 147

Removing Nonconvex Vertices

Testing whether a vertex is convex can be done using the orientation function

Let p, q and r be three consecutive vertices of a polygon, in counterclockwise order

q convex orientation(p, q, r) CCW q nonconvex orientation(p, q, r) CW or COLL

pq

r

qr

p

Gift Wrapping Algorithm We can identify a particular point, say one

with minimum y-coordinate, that provides an initial starting configuration for an algorithm that computes the convex hull.

The gift wrapping algorithm for computing the convex hull of a set of points in the plane is based on just such a starting point.

4 -148

Gift Wrapping 1.View the points as pegs implanted in a level field, and imagine that

we tie a rope to the peg corresponding to the point a with minimum y-coordinate (and minimum x-coordinate if there are ties). Call a the anchor point, and note that a is a vertex of the convex hull.

2. Pull the rope to the right of the anchor point and rotate it counterclockwise until it touches another peg, which corresponds to the next vertex of the convex hull.

3. Continue rotating the rope counterclockwise, identifying a new vertex of the convex hull at each step, until the rope gets back to the anchor point.

4 4 -149

4 4 -150

Graham Scan Algorithm

4 4 -151

Graham Scan

4 4 -152

4 4 -153

4 4 -154

Convex Hull 155

Graham Scan The Graham scan is a

systematic procedure for removing nonconvex vertices from a polygon

The polygon is traversed counterclockwise and a sequence H of vertices is maintained

for each vertex r of the polygonLet q and p be the last and second

last vertex of H

while orientation(p, q, r) CW or COLLremove q from Hq pp vertex preceding p in H

Add r to the end of H

p

q

r

H

pqr

H

p

qr

H

Convex Hull 156

Analysis Computing the convex hull of a set of points takes O(n

log n) time Finding the anchor point takes O(n) time Sorting the points counterclockwise around the

anchor point takes O(n log n) time Use the orientation comparator and any sorting

algorithm that runs in O(n log n) time (e.g., heap-sort or merge-sort)

The Graham scan takes O(n) time Each point is inserted once in sequence H Each vertex is removed at most once from

sequence H

4 4 -157

The divide-and-conquer strategy to solve the problem:

4 4 -158

convex hull problem To find a convex hull, we may use the divide-and-

conquer. The set of planar points hav into two subsets SL

and SR by a line perpendicular to the x-axis. Convex hulls for SL and SR are now constructed

and they are denoted as Hull(Sl),Hull(Sr) respectively.

To combine Hull(SL) and Hull(SR) into one convey use the Graham scan.

4 4 -159

Graham scan An interior point of Hull(Sl) is selected. Consider the point as the origin. Then each other point forms a polar angle with

interior point. All of the points are now sorted with respect to

these polar angle. The Graham scan examines the points one by one

and eliminates the points which cause reflexive angles, as illustrated in Figure 4-10.

4 4 -160

Reflexive angle

Reflexive angle

4 4 -161

e.g. points b and f need to be deleted.

Final result:

reflexive angles

4 4 -162

The merging procedure: 1. Select an interior point p.2. There are 3 sequences of points which have increasing

polar angles with respect to p.(1) g, h, i, j, k(2) a, b, c, d(3) f, e

3. Merge these 3 sequences into 1 sequence:g, h, a, b, f, c, e, d, i, j, k.

4. Apply Graham scan to examine the points one by one and eliminate the points which cause reflexive angles.

4 4 -163

The divide-and-conquer strategy to solve the problem:

4 4 -164

Divide-and-conquer for convex hull Input : A set S of planar points Output : A convex hull for SStep 1: If S contains no more than five

points, use exhaustive searching to find the convex hull and return.

Step 2: Find a median line perpendicular to the X-axis which divides S into SL and SR ; SL lies to the left of SR .

Step 3: Recursively construct convex hulls for SL and SR. Denote these convex hulls by Hull(SL) and Hull(SR) respectively.

4 4 -165

Step 4: Apply the merging procedure to merge Hull(SL) and Hull(SR) together to form a convex hull.

Time complexity: T(n) = 2T(n/2) + O(n) = O(n log n)

4 4 -166

4.4 The Voronoi diagram problem e.g. The Voronoi diagram for two & three

points

Each Lij is the perpendicular bisector of the line.

4 4 -167

Definition of Voronoi diagrams Def : Given two points Pi, Pj S, let

H(Pi,Pj) denote the half plane containing Pi. The Voronoi polygon associated with Pi is defined as

jiji PPHiV

),()(

pi

4 4 -168

Given a set of n points, the Voronoi diagram consists of all the Voronoi polygons of these points.

The vertices of the Voronoi diagram are called Voronoi points and its segments are called Voronoi edges.

4 4 -169

Delaunay triangulation

The straight line dual of a Voronoi diagram is called the Delaunay triangulation, in honor of a famous French mathematician. There is a line segment connecting Pi and Pj in a Delaunay triangulation if and only if the Voronoi polygons of Pi and Pj share the same edge.

4 4 -170

Application of Voronoi Diagram

Voronoi diagrams are very useful for many purposes:

We can solve the so called all closest pairs problem by extracting information from the Voronoi diagram.

A minimal spanning tree can also be found from the Voronoi diagram.

4 4 -171

Example for constructing Voronoi diagrams

Divide the points into two parts.

4 4 -172

Merging two Voronoi diagrams Merging along the piecewise linear

hyperplane HP

4 4 -173

Property of HP If a point P is within the left(right) side of

HP, the nearest neighbor of P must be a point in SL(SR).

After discarding all of VD(SL) to the right of HP and all of VD(SR) to the left of HP, we obtain the resulting Voronoi diagram as shown in Figure 5-19.

4 4 -174

After merging

The final Voronoi diagram

4 4 -175

Divide-and-conquer for Voronoi diagram

Input: A set S of n planar points. Output: The Voronoi diagram of S.Step 1: If S contains only one point,

return.Step 2: Find a median line L perpendicular

to the X-axis which divides S into SL and SR such that SL (SR) lies to the left(right) of L and the sizes of SL and SR are equal.

4 4 -176

Step 3: Construct Voronoi diagrams of SL and SR recursively. Denote these Voronoi diagrams by VD(SL) and VD(SR).

Step 4: Construct a dividing piece-wise linear hyperplane HP which is the locus of points simultaneously closest to a point in SL and a point in SR. Discard all segments of VD(SL) which lie to the right of HP and all segments of VD(SR) that lie to the left of HP. The resulting graph is the Voronoi diagram of S.

(See details on the next page.)

4 4 -177

Mergeing Two Voronoi Diagrams into One Voronoi Diagram

Input: (a) SL and SR where SL and SR are divided by a perpendicular line L.

(b) VD(SL ) and VD(SR ).

Output: VD(S) where S = SL ∩SR

Step 1: Find the convex hulls of SFind the convex hulls of SLL and S and SRR, denoted as Hull(SL) and Hull(SR), respectively. (A special algorithm for finding a convex hull in this case will by given later.)

4 4 -178

Step 2: Find segments and which join HULL(SL ) and HULL(SR ) into a convex hull (Pa and Pc belong to SL and Pb and Pd belong to SR) Assume that lies above . Let x = a, y = b, SG= and HP = .

Step 3: Find the perpendicular bisector of SG. Denote it by BS. Let HP = HP∪{BS}. If SG = , go to Step 5; otherwise, go to Step 4.

dcPPba PP

baPP dcPP

yxPP

dcPP

4 4 -179

Step 4: The ray from VD(SL ) and VD(SR) which BS first intersects with must be a perpendicular bisector of either or for some z. If this ray is the perpendicular bisector of , then let SG = ; otherwise, let SG = . Go to Step 3.

Step 5: Discard the edges of VD(SL) which extend to the right of HP and discard the edges of VD(SR) which extend to the left of HP. The resulting graph is the Voronoi diagram of S = SL∪SR.

zxPP zy PP

zy PP zxPP

yz PP

4 4 -180

Properties of Voronoi Diagrams Def : Given a point P and a set S of

points, the distance between P and S is the distance between P and Pi which is the nearest neighbor of P in S.

The HP obtained from the above algorithm is the locus of points which keep equal distances to SL and SR .

The HP is monotonic in y.

4 4 -181

The relationship between a horizontal line H and SL and SR.

Each horizontal line H intersects with HP at one and only on point.

4 4 -182

The HP is monotonic in y.

horizontal line

(1) a, c belong to SL and b belong to SR

(2) a, c belong to SR and b belong to SL

4 4 -183

# of edges of a Voronoi diagram 3n - 6, where n is # of points.

Reasoning:i. # of edges of a planar graph with n

vertices 3n - 6.ii. A Delaunay triangulation is a planar

graph.iii. Edges in Delaunay triangulation edges in Voronoi diagram.

1 1

# of Voronoi edges

Voronoi vertices

Voronoi edge

Corollary: If G is a connected planar simple graph with E edges and V vertices where V≧3, then E≦3V-6.

4 4 -184

# of Voronoi vertices

# of Voronoi vertices 2n – 4 (upper bound). Reasoning:

i. Let F, E and V denote # of face(region), edges and vertices in a planar graph.

Euler’s relation: F = E - V + 2.ii. In a Delaunay triangulation,

V = n, E 3n – 6 F = E - V + 2 3n - 6 - n + 2 = 2n - 4.

Voronoi vertices

Reference: Rosen pp. 606~607.

4 4 -185

Construct a convex hull from a Voronoi diagram

After a Voronoi diagram is constructed, a convex hull can by found in O(n) time.

Connecting the points associated with the infinite raysinfinite rays.

4 4 -186

Construct Convex Hull from Voronoi diagram

Step 1: Find an infinite ray by examining all Voronoi edges. O(n)

Step 2: Let Pi be the point to the left of the infinite ray. Pi is a convex hull vertex. Examine the Voronoi polygon of Pi to find the next infinite ray.

Step 3: Repeat Step 2 until we return to the Starting ray.

4 4 -187

Time complexity Time complexity for merging 2 Voronoi

diagrams: Total: O(n)

Step 1: O(n) Step 2: O(n) Step 3 ~ Step 5: O(n)

(at most 3n - 6 edges in VD(SL) and VD(SR) and at most n segments in HP)

Time complexity for constructing a Voronoi diagram: O(n log n)because T(n) = 2T(n/2) + O(n)=O(n log

n)

4 4 -188

Finding lower bound by problem transformation

instance of A

transformation T(tr1)

instance of B

T(A) T(B) solver of B

answer of A

transformation

T(tr2)

answer of B

Problem A reduces to problem B (AB) iff A can be solved by using any algorithm

which solves B.If AB, B is more difficult.

Note: T(tr1) + T(tr2) < T(B) T(A) T(tr1) + T(tr2) + T(B) O(T(B))

4 4 -189

Lower bound of the Voronoi diagram

Let us consider a set of points on a straight line. The Voronoi diagram of such a set of points consists of a

set of bisecting lines as shown in Figure 5-26. After these lines have been constructed, a linear scanning

of these Voronoi edges will accomplish the function of sorting.

In other words, the Voronoi diagram problem can not be easier than the sorting problem.

A lower bound of the Voronoi diagram problem is therefore Q(nlogn) and the algorithm is consequently optimal.

4 4 -190

Lower bound The lower bound of the Voronoi

diagram problem is (n log n). sorting Voronoi diagram problem

The Voronoi diagram for a set of points on a straight line

4 4 -191

5.5 Applications of Voronoi diagrams

The Euclidean nearest neighbor searching problem.

The Euclidean all nearest neighbor problem.

4 4 -192

The Euclidean nearest neighbor searching problem.

The Euclidean nearest neighbor searching problem is defined as follows: We are given a set of n planar points: P1, P2 , …,Pn, and a testing point P. Our problem is to find a nearest neighbor of P among Pi’s and the distance used is the Euclidean distance.

A straightforward method is to conduct an exhaustive search. This algorithm would be an O(n) algorithm.

Using the Voronoi diagram, we can reduce the searching time to O(logn) with preprocessing time O(nlogn).

4 4 -193

Note that the Voronoi diagram divides the entire plane into regions R1, R2 Rn. Within each region Ri, there is a point Pi.

If a testing point falls within region Ri, then its nearest neighbor, among all points, is Pi.

Therefore, we may avoid an exhaustive search by simply transforming the problem into a region location problem.

That is, if we can determine which region Ri a testing point is located, we can determine a nearest neighbor of this testing point.

4 4 -194

A Voronoi diagram is a planar graph. Our first step is to sort these Voronoi vertices

according to their y-values. The Voronoi vertices are labeled V1, V2,…, V6

according to their decreasing y-values. For each Voronoi vertex, a horizontal line is drawn passing this vertex.

These horizontal lines divide the entire space into slabs.

4 4 -195

slab

region regionregion

R2 R3

R4 R6R1

4 4 -196

Euclidean nearest neighbor searching algorithm

(1) Conduct a binary search to determine which slab this testing point is located. Since there are at most 0(n) Voronoi vertices, this can be done in O(logn) time.

(2) Within each slab, conduct a binary search to determine which region this point is located in. Since there are at most 0(n) Voronoi edges, this can be done in O(logn) time.

The total searching time is O(logn). It is easy to see that the preprocessing time is O(nlogn),

essentially the time needed to construct the Voronoi diagram.

4 4 -197

The Euclidean all nearest neighbor problem.

We are given a set of n planar points P1, P2, ..., Pn.

The Euclidean closest pair problem is to find a nearest neighbor of every Pi.

Properties: If Pj is a nearest neighbor of Pi, then Pi and Pj share

the same Voronoi edge. Moreover, the midpoint of segment PiPj is located

exactly on this commonly shared Voronoi edge.

4 4 -198

Proof We shall show this property by contradiction. Suppose that Pi and Pj do not share the same Voronoi edge.

By the definition of Voronoi polygons, the perpendicular bisector of PiPj must be outside of the Voronoi polygon associated with Pi.

Let P,Pi intersect the bisector at M and some Voronoi edge at N, as illustrated in Figure 5-28.

4 4 -199

Euclidean all nearest neighbor problem

Given the above property, the Euclidean all nearest neighbor problem can be solved by examining every Voronoi edge of each Voronoi polygon.

Since each Voronoi edge is shared by exactly two Voronoi polygons, no Voronoi edge is examined more than twice.

That is, this Euclidean all nearest neighbor problem can be solved in linear time after the Voronoi diagram is constructed.

Thus this problem can be solved in O(nlogn) time.

4 4 -200

4 4 -201

Fast Fourier transform (FFT)

Fourier transform

A(f) = a(t)e2iftdt Inverse Fourier transform

a(t) = A(f)e-2iftdf Discrete Fourier transform(DFT)

Given a0, a1, …, an-1

Aj = ake2ijk/n ,0 j n-1

Inverse DFT

ak = Aje-2ijk/n ,0 k n-1

1

2

0 1 k n

1

0 1n j n

4 4 -202

DFT and waveform Any periodic waveform can be decomposed into

the linear sum of sinusoid functions (sine or cosine).

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-4

-3

-2

-1

0

1

2

3

4

7 15 48 56f( )頻率

))15(2cos(3))7(2cos()( tttf See the left part:

4 4 -203

An application of the FFT polynomial multiplication

Polynomial multiplication:

The straightforward product requires O(n2) time. DFT notations:

.},,,{ of DFTinverse theis },,,{

10 ,

...

.},,,{ of DFT theis },,,{

1 ,10 ,Let

...

110110

1

11

2210

110110

11

2210

nn

knk

nn

nn

njj

nn

bbbaaa

njwha

xbxbxbbxh

aaabbb

wnjwfb

xaxaxaaxf

xgxfxhxcxgxaxfn

k

kk

n

j

jj

,1

0

1

0

4 4 -204

Fast polynomial multiplicationStep 1: Let N be the smallest integer that N=2q and N2n-1. Step 2: Compute FFT of

Step 3: Compute FFT of

Step 4:Step 5:

Time complexity: O(NlogN)=O(nlogn), N<4n.

Nijj ewNjwgwf /2 ,10 ),( Compute

.}0,,0,0,,,,{ 110 N

naaa

.}0,,0,0,,,,{ 110 N

nccc

.)( of tscoefficien the

are numbers of sequence resulting The

)}.(,),(),({ of DFTinverse Compute

)()(Let 110

xh

whwhwh

wgwfwhN

jjj

4 4 -205

FFT algorithm A straightforward method to calculate

DFT requires O(n2) time. DFT can be solved by the divide-and-

conquer strategy (FFT). Let w=e2i/n. i.e. wn=1, wn/2=-1.

Aj = ake2ijk/n , 0 j n-1

= akwjk

e.g. n = 8, let e2i/8= w= ei/4

0 1 k n

0 1 k n

sincos iei

4 4 -206

FFT algorithm when n=4

n =4, let e2i/4=w = ei/2 (w4 = 1, w2 = -1) A0 = a0 + a1 + a2 + a3

A1 = a0 + a1w + a2w2 + a3w

3

A2 = a0 + a1w2 + a2w

(2)(2) + a3w(2)(3)

= a0 + a1w2 + a2w

4 + a3w6

A3 = a0 + a1w3 + a2w

(3)(2) + a3w(3)(3)

= a0 + a1w3 + a2w

6 + a3w9

4 4 -207

Rewrite as:

A0 = a0 + a2 + (a1 + a3)

A2 = a0 + a2w4 + (a1w

2 + a3w6)

= a0 + a2 - (a1 + a3) When we calculate A0 , we shall calculate (a0 +

a2) and (a1 + a3). Later, A2 can be easily found. Similarly,

A1 = (a0 + a2w2) + a1w + a3w

3

A3 = (a0 + a2w6) + (a1w

3 + a3w9)

= (a0 + a2w2) - (a1w + a3w

3)

4 4 -208

n = 8, let e2i/8= w = ei/4 (w8 = 1, w4 = -1)A0 = a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7

A1= a0+a1w+a2w2+a3w

3+a4w4+a5w

5+a6w6+a7w

7

A2 = a0+a1w2+a2w

4+a3w6+a4w

8+a5w10+a6w

12+a7w14

A3 = a0+a1w3+a2w

6+a3w9+a4w

12+a5w15+a6w

18+a7w21

A4 = a0+a1w4+a2w

8+a3w12+a4w

16+a5w20+a6w

24+a7w28

A5 = a0+a1w5+a2w

10+a3w15+a4w

20+a5w25+a6w

30+a7w35

A6 = a0+a1w6+a2w

12+a3w18+a4w

24+a5w30+a6w

36+a7w42

A7 = a0+a1w7+a2w

14+a3w21+a4w

28+a5w35+a6w

42+a7w49

FFT algorithm when n=8

4 4 -209

After reordering, we haveA0=(a0+a2+a4+a6)+(a1+a3 +a5 +a7)

A4=(a0+a2+a4+a6)-(a1+a3 +a5 +a7)

A1=(a0+a2w2+a4w

4+a6w6)+w(a1+a3w

2+a5w4+a7w

6)

A5=(a0+a2w2+a4w

4+a6w6)-w(a1+a3w

2+a5w4+a7w

6)

A2=(a0+a2w4+a4w

8+a6w12)+w2(a1+a3w

4+a5w8+a7w

12)

A6=(a0+a2w4+a4w

8+a6w12)-w2(a1+a3w

4+a5w8+a7w

12)

A3=(a0+a2w6+a4w

12+a6w18)+w3(a1+a3w

6+a5w9+a7w

18)

A7=(a0+a2w6+a4w

12+a6w18)-w3(a1+a3w

6+a5w9+a7w

18) Rewrite as:

A0 = B0 + C0 A4 = B0 － C0

A1 = B1 + C1 A5 = B1 － C1

A2 = B2 + C2 A6 = B2 － C2

A3 = B3 + C3 A7 = B3 － C3

4 4 -210

let x=w2 =e2i/4= ei/2 (x4 = 1, x2 = -1) We can recursively apply the same method to calculate

Bi’s and Ci’s.

B0 = a0+a2+a4+a6

B1 = a0+a2w2+a4w

4+a6w6

= a0+a2x+a4x2+a6x

3

B2 = a0+a2w4+a4w

8+a6w12

= a0+a2x2+a4x

4+a6x6

B3 = a0+a2w6+a4w

12+a6w18

= a0+a2x3+a4x

6+a6x9

Thus, {B0, B1, B2, B3} is the DFT of {a0, a1, a2, a3}.

4 4 -211

General FFT

In general, let w = e2i/n (assume n is even)(wn = 1, wn/2 = -1)

Aj = a0+a1wj+a2w

2j+…+an-1w(n-1)j

={a0+a2w2j+a4w

4j+…+an-2w(n-2)j} +

{a1wj+a3w

3j+…+an-1w(n-1)j }

= Bj + Cj

Aj+n/2 = a0+a1wj+n/2+a2w

2j+n+a3w3j+3n/2+…+

an-1w(n-1)j+(n(n-1)/2)

=a0-a1wj+a2w

2j-a3w3j+…+an-2w

(n-2)j-an-1w(n-1)j

= Bj - Cj

4 4 -212

Divide-and-conquer (FFT) Input : a0 , a1 , …, an-1, n = 2k Output : Aj, j=0, 1, 2, …, n-1,

where Aj = 0kn-1ake2ijk/n

Step 1: If n=2, compute

A0 = a0 + a1,

A1 = a0 - a1, and return.

Step 2: Divide each aj, 0 j n/2 - 1 into two sequences: Oj and Ej, where Oj(Ej), consists of odd-numbered (even-numbered) terms of Aj.

4 4 -213

Step 3: Recursively calculate the sums of terms in Oj and Ej . Denote the sum of terms of Oj and Ej by Bj and Cj, respectively.

Step 4: Compute Aj by the following formual :

Aj = Bj + Cj for 0 j n/2 - 1

Aj+n/2 = Bj - Cj for 0 j n/2 - 1.

Time complexity :T(n) = 2T(n/2) + O(n)

= O(n log n)

4 4 -214

5.8 Matrix multiplication

Let A, B and C be n n matrices

C = AB

C(i, j) = A(i, k)B(k, j)

The straightforward method to perform a matrix multiplication requires O(n3) time.

1 k n

4 4 -215

Divide-and-conquer approach

C = AB

C11 = A11 B11 + A12 B21

C12 = A11B12 + A12 B22

C21 = A21 B11 + A22 B21

C22 = A21 B12 + A22 B22 Time complexity:

(# of additions : n2)

We get T(n) = O(n3)

C11 C12 = A11 A12 B11 B12 C21 C22 = A21 A22 B21 B22

T(n) = b 8T(n/2)+cn2 , n 2 , n > 2

4 4 -216

P = (A11 + A22)(B11 + B22)

Q = (A21 + A22)B11

R = A11(B12 - B22)

S = A22(B21 - B11)

T = (A11 + A12)B22

U = (A21 - A11)(B11 + B12)

V = (A12 - A22)(B21 + B22). C11 = P + S - T + V

C12 = R + T

C21 = Q + S

C22 = P + R - Q + U

Strassen’s matrix multiplicaiton

4 4 -217

Time complexity 7 multiplications and 18 additions or subtractions Time complexity:

T(n) = an2 + 7T(n/2)= an2 + 7(a(n/2)2 + 7T(n/4))= an2 + (7/4)an2 + 72T(n/4)= … := an2(1 + 7/4 + (7/4)2+…+(7/4)k-1+7kT(1)) cn2(7/4)log2n+7log2n, c is a constant= cnlog24+log27-log24 +nlog27 = O(nlog27) O(n2.81)

T(n) = b 7T(n/2)+an2 , n 2 , n > 2

4 -1 Chapter 4 The Divide-and-Conquer Strategy. 4 -2 Outlines 4-1 The 2-Dimensional Maxima Finding...

Documents

Transcript of 4 -1 Chapter 4 The Divide-and-Conquer Strategy. 4 -2 Outlines 4-1 The 2-Dimensional Maxima Finding...