CSC 201CSC 201Analysis and Design of AlgorithmsAnalysis and Design of Algorithms

Lecture 05:Lecture 05:Analysis of time Complexity of Analysis of time Complexity of Sorting AlgorithmsSorting Algorithms

Dr.Surasak MungsingDr.Surasak MungsingE-mail: Surasak.mu@spu.ac.th

04/24/23 1

04/24/232

Sorting AlgorithmsSorting Algorithms Bin Sort Radix Sort Insertion Sort Shell Sort Selection Sort Heap Sort Bubble Sort Quick Sort Merge Sort

04/24/233

04/24/234

Bin SortBin Sort

A 2 C 5B 4 J 3H 3 I 4D 4 E 3 F 0 G 4

F A EHJ

BDGI

CBin 0 Bin 1 Bin 2 Bin 3 Bin 4 Bin 5

F 0 E 3A 2 C 5G 4 I 4H 3 J 3 B 4 D 4

(a) Input chain

(b) Nodes in bins

(c) Sorted chain

04/24/23 5

Radix Sort with r=10 and d=3Radix Sort with r=10 and d=3

216 521 425 116 91 515 124 34 96 24(a) Input chain

24 34 91 96 116 124 216 425 515 521(d) Chain after sorting on most significant digit

515 216 116 521 124 24 425 34 91 96(c) Chain after sorting on second-least significant digit

521 91 124 34 24 425 515 216 116 96(b) Chain after sorting on least significant digit

04/24/23 6

Insertion Sort Concept

04/24/237

04/24/238

Shell Sort Algorithm

04/24/239

Shell Sort Algorithm

04/24/2310

Shell Sort Algorithm

04/24/2311

04/24/23 12

Shell Sort Demohttp://e-learning.mfu.ac.th/mflu/1302251/demodemo//Chap03Chap03//ShellSortShellSort//

ShellSortShellSort..htmlhtml

04/24/2313

Selection Sort Concept

04/24/23 14

04/24/2315

Heap Sort Algorithm

04/24/2316

04/24/23 17

Bubble Sort Concept

04/24/23 18

04/24/2319

Quick sortQuick sort

4313

8131

92

5765

75

260

4313

8131

92

5765

75

260

650

3113

2657

4392 75

81

Select pivot

Partition

The fastest known sorting algorithm in practice.

04/24/2320

Quick sortQuick sort

0 3126 5743 9275 816513

650

3113

2657

4392 75

81

0 3126 574313 65 9275 81

Quick sort small Quick sort large

04/24/23 21

Quick sort

04/24/2322

Quick Sort Partitions

04/24/2323

Quick sort

04/24/2324

External Sort: A simple merge

04/24/2325

Merge Sort

04/24/2326

Merge Sort

04/24/2327

04/24/23 28

04/24/2329

Analysis of Insertion SortAnalysis of Insertion Sort

for (int i = 1; i < a.length; i++){// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t;}

04/24/2330

ComplexityComplexity

Space/Memory Time

Count a particular operation Count number of steps Asymptotic complexity

04/24/2331

Comparison CountComparison Count

for (int i = 1; i < a.length; i++){// insert a[i] into a[0:i-1] int t = a[i]; int j; for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j]; a[j + 1] = t;}

04/24/2332

Comparison CountComparison Count

for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];

How many comparisons are made?

04/24/2333

Comparison CountComparison Count

for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];

number of compares depends on a[]s and t as well as on i

04/24/2334

Comparison CountComparison Count

Worst-case count = maximum countBest-case count = minimum countAverage count

04/24/2335

Worst-Case Comparison CountWorst-Case Comparison Count

for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];

a = [1, 2, 3, 4] and t = 0 4 comparesa = [1,2,3,…,i] and t = 0 i compares

04/24/2336

Worst-Case Comparison CountWorst-Case Comparison Countfor (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t <

a[j]; j--) a[j + 1] = a[j];

total compares = 1 + 2 + 3 + … + (n-1)

= (n-1)n/2

= O(n2)

n

(i-1)i=2

T(n) =

04/24/2337

Average-Case Comparison Average-Case Comparison CountCount

for (int i = 1; i < n; i++) for (j = i - 1; j >= 0 && t < a[j]; j--) a[j + 1] = a[j];

n

i=2

T =i-12

= O(n2)

04/24/2338

4313

8131

92

5765

75

260

4313

8131

92

5765

75

260

650

3113

2657

4392 75

81

Select pivot

Partition

Analysis of Quick Sort

04/24/2339

MainMainQuick SortQuick Sort

RoutineRoutine private static void quicksort( Comparable [ ] a, int left, int right ) {/* 1*/ if( left + CUTOFF <= right ) {/* 2*/ Comparable pivot = median3( a, left, right ); // Begin partitioning/* 3*/ int i = left, j = right - 1;/* 4*/ for( ; ; ) {/* 5*/ while( a[ ++i ].compareTo( pivot ) < 0 ) { }/* 6*/ while( a[ --j ].compareTo( pivot ) > 0 ) { }/* 7*/ if( i < j )/* 8*/ swapReferences( a, i, j ); else/* 9*/ break; }/*10*/ swapReferences( a, i, right - 1 ); // Restore pivot/*11*/ quicksort( a, left, i - 1 ); // Sort small elements/*12*/ quicksort( a, i + 1, right ); // Sort large elements } else // Do an insertion sort on the subarray/*13*/ insertionSort( a, left, right ); }

04/24/2340

Worst-Case AnalysisWorst-Case Analysisเวลาท่ีใชใ้นการ run quick sort เท่ากับเวลาท่ีใชใ้นการทำา recursive call 2 ครัง้ + linear time ท่ีใชใ้นการเลือก pivot ซึ่งทำาให ้basic quick sort relation เท่ากับ

T(n) = T(i) + T(n-i-1) + cnในกรณี Worst- case เชน่ การท่ี pivot มค่ีาน้อยท่ีสดุเสมอ เวลาท่ีใชใ้นการทำา recursion คือ

T(n) = T(n-1) + cn n>1T(n-1)= T(n-2)+c(n-1)T(n-2)= T(n-3)+c(n-2)

… T(2) = T(1)+c(2)

รวมเวลาทัง้หมดT(n) = T(1) + c

n

i=2i = O(n2)

04/24/2341

The pivot is in the middle; T(n) = 2 T(n/2) + cn

Divide both sides by n;

Add all equations;

BestBest-Case Analysis-Case Analysis

T(n/2)

n/2

T(n)

n= + c

T(2)

2

T(1)

1= + c

T(n/4)

n/4

T(n/2)n/2

= + c

T(n/4)

n/4

T(n/8)

n/8= + c

…

T(n)

n

T(1)

1= + c log n

T(n) = c n logn + n = O(n log n)

04/24/2342

AverageAverage-Case Analysis (1/4)-Case Analysis (1/4)

2N

N -1

j=0

T( j ) + cNT(N) =

NT(N) =N -1

j=0

T( j ) + cN22

(N-1) T(N-1) = 2 + c(N – 1)2T( j )N -2

j=0

Average time of T(i) and T(N-i-1) is 1N

N -1

j=0

T( j )

Total time; T(N) = T(i) + T(N-i-1) + c N …………..(1)

Therefore

…………..(2)

…………..(3)

…………..(4)

…………..(5)

04/24/2343

AverageAverage-Case Analysis-Case Analysis (2/4)(2/4)

…………..(6)NT(N) – (N-1) T(N-1) = 2T(N-1) +2cN - c(5) – (4);

(7) divides by N(N+1);T(N)

N+1

T(N-1)

N

2c

N+1= + …………..(8)

Rearrange terms in equation and ignore c on the right-hand side;

NT(N) = (N+1) T(N-1) +2cN …………..(7)

04/24/2344

AverageAverage-Case Analysis-Case Analysis (3/4)(3/4)T(N)

N+1

T(N-1)

N

2c

N+1= +

T(N-2)

N-1

T(N-3)

N-2

2c

N-1= +

T(N-1)

N

T(N-2)

N-1

2c

N= +

…………..(8)

…………..(9)

…………..(10)...

T(2)

3

T(1)

2

2c

3= + …………..(11)

Sun equations (8) to (11); …………..(12)N +1

i=3

T(N)

N+1

T(1)

22c= +

04/24/2345

AverageAverage-Case Analysis-Case Analysis (4/4)(4/4)

N +1

i=3

T(N)

N+1

T(1)

22c= + …………..(12)

Sum in equation (12) ia approximately logC(N+1)+ - 3/2, which is Euler’s constant 0.577

T(N) = O(Nlog N)

…………..(13)

and

T(N)

N+1= O(log N)therefore

…………..(14)

In summary, time complexity of Quick sort algorithm for Average-Case is

T(n) = O(n log n)

Apr 24, 2023 46