Chapter 7: Sorting Insertion Sort Shell Sort CS 340 Page 112 Heap Sort Merge Sort Quick Sort Best...

Click here to load reader

  • date post

    14-Dec-2015
  • Category

    Documents

  • view

    243
  • download

    3

Embed Size (px)

Transcript of Chapter 7: Sorting Insertion Sort Shell Sort CS 340 Page 112 Heap Sort Merge Sort Quick Sort Best...

  • Slide 1

Slide 2 Chapter 7: Sorting Insertion Sort Shell Sort CS 340 Page 112 Heap Sort Merge Sort Quick Sort Best Case Sorting Algorithm Analysis Slide 3 CS 340 Page 113 Sorting While sorting an array of elements is itself a significant problem, examining this problem serves several other purposes: A large variety of sorting algorithms have been developed, so we can explore many algorithm design techniques. Formal analysis of these algorithms is rather straightforward, so our intuitive feel for time complexity can be expanded. Worst-case, best-case, and average-case analyses are possible with most of these algorithms, showing us that algorithms that might be advantageous under certain conditions can be quite disadvantageous under other conditions. Memory limitations can affect the ability to apply certain sorting algorithms, so we can also examine the effects of external system limitations upon algorithm design and implementation. Slide 4 CS 340 Page 114 Insertion Sort // Note: The list itself will be the n elements // in A[1] through A[n]. A sentinel value // of INT_MIN will be placed in A[0]. template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // Defined elsewhere as smallest // possible value of type Etype. for (p = 1; p < n; p++) { itemToInsert = A[p]; for (j = p; itemToInsert < A[j-1]; j--) A[j] = A[j-1]; A[j] = itemToInsert; } // Note: The list itself will be the n elements // in A[1] through A[n]. A sentinel value // of INT_MIN will be placed in A[0]. template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // Defined elsewhere as smallest // possible value of type Etype. for (p = 1; p < n; p++) { itemToInsert = A[p]; for (j = p; itemToInsert < A[j-1]; j--) A[j] = A[j-1]; A[j] = itemToInsert; } Sort the first k elements and then insert the next element by sequentially finding its properly location, sliding each element greater than the next element over by one slot. Slide 5 CS 340 Page 115 Insertion Sort Example Catwoman Joker MrFreeze Penguin Riddler Scarecrow Two-Face Clayface Egghead PoisonIvy Joker Penguin MrFreeze Scarecrow Riddler Catwoman Two-Face Clayface Egghead PoisonIvy Joker MrFreeze Penguin Scarecrow Riddler Catwoman Two-Face Clayface Egghead PoisonIvy Joker MrFreeze Penguin Riddler Scarecrow Catwoman Two-Face Clayface Egghead PoisonIvy Catwoman Joker MrFreeze Penguin Riddler Scarecrow Two-Face Clayface Egghead PoisonIvy Catwoman Clayface Joker MrFreeze Penguin Riddler Scarecrow Two-Face Egghead PoisonIvy Catwoman Clayface Egghead Joker MrFreeze Penguin Riddler Scarecrow Two-Face PoisonIvy Catwoman Clayface Egghead Joker MrFreeze Penguin PoisonIvy Riddler Scarecrow Two-Face Slide 6 CS 340 Page 116 Insertion Sort Analysis template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } Slide 7 CS 340 Page 117 Insertion Sort Analysis (continued) Best case time complexity (i.e., inner loop iterates minimally): 3+ p=1,n- 1 (2+4+6)+3 = 6+12(n-1) = 12n-6 time units, which is O(n). Since there actually is an array requiring this time (i.e., an array with its elements initially in order), the best case is also (n). Average case time complexity: The average number of inversions in a list is n(n-1) (an inversion is a pair of array slots i and j where I A[j]), and this algorithm only removes one inversion per inner loop iteration. Thus, the average case is (n 2 ), and, by the worst-case argument above, it is also O(n 2 ). template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } Slide 8 CS 340 Page 118 Shell Sort Sort the list in layered sublists, so when sorting the list as a whole, its already reasonably close to being sorted. // Note: The list itself will be the n // elements in A[1] through A[n]. // No meaningful value will be // placed in A[0]. void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) for (i = inc + 1; iinc; j -= inc) { if (temp < A[j - inc]) A[j] = A[j - inc]; else break; } A[j] = temp; } // Note: The list itself will be the n // elements in A[1] through A[n]. // No meaningful value will be // placed in A[0]. void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) for (i = inc + 1; iinc; j -= inc) { if (temp < A[j - inc]) A[j] = A[j - inc]; else break; } A[j] = temp; } Slide 9 CS 340 Page 119 Shell Sort Example Increment: 6 Increment: 3 Increment: 1 Note that for each increment value, the algorithm performs an insertion sort on every subarray of values spaced increment apart. Also note that this implementation uses increment values n/2, n/4, n/8, , 1, known as Shells increments. A more strategic choice of increment values (i.e., all values relatively prime), would yield better overall performance.LeiaGreedoLandoAckbarR2D2C3PODarthYodaLukeJabbaHanObiwanChewie LeiaDarthChewieGreedoYodaLandoLukeAckbarJabbaR2D2HanC3POObiwan ChewieDarthLeiaGreedoYoda LandoLukeAckbarJabbaHanR2D2 C3POObiwan ChewieAckbarDarthJabbaLeia GreedoHanYodaR2D2 LandoC3POLukeObiwan AckbarChewieDarthJabbaLeia GreedoHanR2D2Yoda C3POLandoLukeObiwan AckbarGreedoC3POChewieHanLandoDarthR2D2LukeJabbaYodaObiwanLeia AckbarC3POChewieDarthGreedoHanJabbaLandoLeiaLukeObiwanR2D2Yoda Slide 10 CS 340 Page 120 void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 iterations; 3 TU each // (1 division, 1 assignment, // and 1 comparison) for (i = inc + 1; iinc; j -= inc) // (i/inc) iterations; 3 TU each: 1 // subtraction, 1 assignment, and 1 // comparison (with first iteration // having one less subtraction) { if (temp < A[j - inc]) // At most 12 TU: 2 subtractions, 3 A[j] = A[j - inc]; // multiplications, 3 additions, 3 else // memory accesses (including the break; // assignment), and 1 comparison } A[j] = temp; // 3 TU: 1 multiplication, 1 addition, } // and 1 assignment } void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 iterations; 3 TU each // (1 division, 1 assignment, // and 1 comparison) for (i = inc + 1; iinc; j -= inc) // (i/inc) iterations; 3 TU each: 1 // subtraction, 1 assignment, and 1 // comparison (with first iteration // having one less subtraction) { if (temp < A[j - inc]) // At most 12 TU: 2 subtractions, 3 A[j] = A[j - inc]; // multiplications, 3 additions, 3 else // memory accesses (including the break; // assignment), and 1 comparison } A[j] = temp; // 3 TU: 1 multiplication, 1 addition, } // and 1 assignment } Shell Sort Analysis Slide 11 CS 340 Page 121 void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 [email protected] for (i = inc + 1; iinc; j -= inc) // (i/inc) [email protected] { if (temp < A[j - inc]) // At A[j] = A[j - inc]; // most else // 12 break; // TU } A[j] = temp; // 3TU } void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 [email protected] for (i = inc + 1; iinc; j -= inc) // (i/inc) [email protected] { if (temp < A[j - inc]) // At A[j] = A[j - inc]; // most else // 12 break; // TU } A[j] = temp; // 3TU } Shell Sort Analysis (continued) Best case time complexity (i.e., inner loop iterates once): p=1,log(n)-1 (3+ i=p+1,n( 3+4+3+5+3)) = p=1,logn (3+18(n-p)) = 3logn+18nlogn- 9logn(1+logn) time units, which is O(nlogn). Since there actually is an array requiring this time (i.e., an array with its elements initially in order), the best case is also (nlogn). Note: Some other (relatively prime) Shell sort increments have (n 1.5 ) time complexity. This is O(n 2 ). There actually is an array requiring this time (i.e., an array with the largest n/2 elements in the even positions and the smallest n/2 elements in the odd positions), so the worst case is also (n 2 ). Slide 12 CS 340 Page 122 Heap Sort Insert the list into a maximum heap, so removals will involve quick access to the next largest element in the list. template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; for ( ; 2*iA[2*i])) ? (2*i+1) : (2*i); if (temp < A[childIndex]) A[i] = A[childIndex]; else break; } A[i] = temp; } template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j--)// Set list up as percDown(A, j, n);// a max-heap. for (int i = n; i >= 2; i--) { temp = A[1]; A[1] = A[i]; A[i] = temp;// (i.e., deleteMax) percDown(A, 1, i-1); } template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; for ( ; 2*iA[2*i])) ? (2*i+1) : (2*i); if (temp < A[childIndex]) A[i] = A[childIndex]; else break; } A[i] = temp; } template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j--)// Set list up as percDown(A, j, n);// a max-heap. for (int i = n; i >= 2; i--) { temp = A[1]; A[1] = A[i]; A[i] = temp;// (i.e., deleteMax) percDown(A, 1, i-1); } Slide 13 CS 340 Page 123 Heap Sort Example Original List: Homer Marge Maggie Bart Lisa Wiggum Burns Moe Barney Flanders Krusty Milhouse Lovejoy Skinner Original List: Homer Marge Maggie Bart Lisa Wiggum Burns Moe Barney Flanders Krusty Milhouse Lovejoy Skinner After Converting Into Max-Heap: Rest Of Heap Sort: Skinner Moe Milhouse Lisa Krusty Marge Burns Bart Barney Flanders Homer Maggie Lovejoy Wiggum Wiggum Moe Skinner Lisa Krusty Marge Milhouse Bart Barney Flanders Homer Maggie Lovejoy Burns Moe Lovejoy Milhouse Lisa Krusty Marge Burns Bart Barney Flanders Homer Maggie Skinner Wiggum Milhouse Lovejoy Marge Lisa Krusty Maggie Burns Bart Barney Flanders Homer Moe Skinner Wiggum Marge Lovejoy Maggie Lisa Krusty Homer Burns Bart Barney Flanders Milhouse Moe Skinner Wiggum Maggie Lovejoy Homer Lisa Krusty Flanders Burns Bart Barney Marge Milhouse Moe Skinner Wiggum Lovejoy Lisa Homer Bart Krusty Flanders Burns Barney Maggie Marge Milhouse Moe Skinner Wiggum Lisa Krusty Homer Bart Barney Flanders Burns Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Krusty Burns Homer Bart Barney Flanders Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Homer Burns Flanders Bart Barney Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Flanders Burns Barney Bart Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Burns Bart Barney Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Bart Barney Burns Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Barney Bart Burns Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Slide 14 CS 340 Page 124 template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; // 4 TU for ( ; 2*iA[2*i])) ? (2*i+1) : (2*i); // At most 16 TU if (temp < A[childIndex]) // At most 11 TU A[i] = A[childIndex]; else break; } A[i] = temp; // 4 TU } // TOTAL FOR PERCDOWN: At most 30log(n-i)+7 TU template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j-) // n/2 iterations; 3 TU each (except first) percDown(A, j, n); // At most 30log(n-j)+7 TU (by above analysis) for (int i = n; i >= 2; i--) // (n-i iterations; 2 TU each) { temp = A[1]; A[1] = A[i]; A[i] = temp; // 13 TU percDown(A, 1, i-1); // 30log(i-2)+8 TU } template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; // 4 TU for ( ; 2*iA[2*i])) ? (2*i+1) : (2*i); // At most 16 TU if (temp < A[childIndex]) // At most 11 TU A[i] = A[childIndex]; else break; } A[i] = temp; // 4 TU } // TOTAL FOR PERCDOWN: At most 30log(n-i)+7 TU template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j-) // n/2 iterations; 3 TU each (except first) percDown(A, j, n); // At most 30log(n-j)+7 TU (by above analysis) for (int i = n; i >= 2; i--) // (n-i iterations; 2 TU each) { temp = A[1]; A[1] = A[i]; A[i] = temp; // 13 TU percDown(A, 1, i-1); // 30log(i-2)+8 TU } Worst case time complexity: j=1,n/2 (3+30log(n-j)+7)-1+ i=2,n (2+13+30log(i-2)+8) (5n + 15nlogn 1) + (23n 23 + 30nlogn) = 45nlogn + 28n - 24 time units, which is O(nlogn). Since either changing an array into a max-heap or re-heapifying it as maximal elements are removed requires this time, the worst case is also (nlogn). Heap Sort Analysis Slide 15 CS 340 Page 125 Merge Sort Recursively sort both halves of the list, and then quickly merge the two sorted halves to form the entire sorted list. template void mergeSort(Etype A[], const int n) { Etype Acopy[n+1]; int size; for (int k = 1; k