FOR TREE SORT, EACH INSERTION TAKES LOGARITHMIC TIME AT MOST, SO FOR n INSERTIONS worstTime( n ) IS...

CHAPTER 12

SORTING

ASSUMPTIONS:

COMPARISON-BASED SORTS ONLY

operator< USED FOR COMPARISONS SORT INTO ASCENDING ORDER

INSERTION SORT:

ASSUME THE ITEMS TO BE SORTEDARE IN A CONTAINER THATSUPPORTS RANDOM-ACCESSITERATORS.

template <class RandomAccessIterator>void __insertion_sort(RandomAccessIterator first, RandomAccessIterator last){ if (first == last)

return; for (RandomAccessIterator i = first + 1; i != last; ++i)

// Put *i into its proper position between first// (inclusive) and i (inclusive).

template <class RandomAccessIterator> void __insertion_sort(RandomAccessIterator first, RandomAccessIterator last) { if (first == last)

return; for (RandomAccessIterator i = first + 1; i != last; ++i) // Put *i into its proper position between first // (inclusive) and i (inclusive). __linear_insert(first, i, value_type(first)); } RETURNS THE TYPE OF first

EXAMPLE:

80, 60, 90, 75, 55, 90

DURING THE FIRST LOOPITERATION, 60 IS INSERTED WHEREIT BELONGS RELATIVE TO 80.

80, 60, 90, 75, 55, 90

60, 80

60, 80, 90, 75, 55, 90

60, 80, 90

60, 80, 90, 75, 55, 90

60, 75, 80, 90

60, 75, 80, 90, 55, 90

55, 60, 75, 80, 90

55, 60, 75, 80, 90, 90

IF *i < *first, SAVE *i, COPY (BACKWARD)EVERY ITEM FROM first THROUGH i – 1INTO first + 1 THROUGH i, AND THENCOPY *i INTO *first.

FOR EXAMPLE,

38, 55, 71, 71, 83, 29, …

STEP 1: SAVE 29 AND COPY BACKWARD:

38, 38, 55, 71, 71, 83, …

STEP 2: COPY 29 TO first:

29, 38, 55, 71, 71, 83, …

OTHERWISE, *i >= first. SIFT *i DOWNBY COMPARING IT TO *(i – 1), *(i – 2),…, UNTIL THE PROPER PLACE FOR*iIS FOUND.

EXAMPLE: 38, 55, 85, 88, 75

iWE GET: 38, 55, 75, 85, 88

template <class RandomAccessIterator, class T> inline void __linear_insert (RandomAccessIterator first, RandomAccessIterator last, T*) { T value = *last; if (value < *first) { copy_backward(first, last, last + 1); *first = value; } else __unguarded_linear_insert(last, value); }

template <class RandomAccessIterator, class T>void __unguarded_linear_insert (RandomAccessIterator last, T value) { RandomAccessIterator next = last; --next; while (value < *next) { *last = *next; last = next--; } *last = value; }

“Unguarded” MEANS THERE IS NO PROTECTIONNEEDED AGAINST AN INFINITE LOOP BECAUSETHIS METHOD IS CALLED ONLY IF *first >= *i.

WORST CASE: CONTAINER INDESCENDING ORDER.

LET n = last – first.

IN __insertion_sort, THERE ARE n – 1ITERATIONS OF THE for LOOP.DURING EACH OF THOSEITERATIONS, copy_backward ISCALLED. WITHIN copy_backward, THETOTAL NUMBER OF LOOPITERATIONS IS 1 + 2 + 3 + … + n – 1.

n-11 + 2 + 3 + ... + n-2 + n-1 = i = n(n-1) / 2 i=1

worstTime(n) MAX NUMBER OF LOOPITERATIONS = n – 1 + n (n-1) / 2

worstTime(n) IS QUADRATIC IN n.

ON AVERAGE, DURING EACHITERATION OF THE for LOOP IN__insertion_sort, THE while LOOP IN__unguarded_linear_insert WILL BEEXECUTED (i – first) / 2 TIMES. SO THETOTAL NUMBER OF while-LOOPITERATIONS WILL BE

1/2 + 2/2 + 3/2 + … + (n-1)/2 = n(n-1) / 4

averageTime(n) AVERAGE NUMBEROF LOOP ITERATIONS

n – 1 + n (n-1) / 4

averageTime(n) IS QUADRATIC IN n.

HOW FAST CAN WE SORT?

A DECISION TREE IS A BINARY TREEIN WHICH EACH NON-LEAFREPRESENTS A COMPARISONBETWEEN TWO ITEMS ANDEACH LEAF REPRESENTS A SORTEDSEQUENCE OF THOSE ITEMS.

LEFT BRANCH: YESRIGHT BRANCH: NO

EXAMPLE: APPLY INSERTION SORTTO a1, a2, a3.

a1 < a2?

a2 < a3? a1 < a3?

a1 a2 a3 a1 < a3? a2 a1 a3 a2 < a3?

a1 a3 a2 a3 a1 a2 a2 a3 a1 a3 a2 a1

A DECISION TREE HAS ONE LEAFFOR EACH PERMUTATION OF THE nELEMENTS TO BE SORTED.

THE NUMBER OF PERMUTATIONSOF n ELEMENTS IS ?

SO A DECISION TREE TO SORT nELEMENTS MUST HAVE n! LEAVES.

BY THE BINARY TREE THEOREM,FOR ANY NON-EMPTY TREE t,

leaves(t) <= 2height(t)

SINCE n! = leaves(t), WE MUST HAVE

n! <= 2height(t)

WHICH IMPLIES THAT

log2(n!) <= height(t)

IN THE CONTEXT OF A DECISIONTREE, height(t) REPRESENTS THEMAXIMUM NUMBER OFCOMPARISONS NEEDED TO SORTTHE n ITEMS.

SO log2(n!) <= THE MAXIMUM NUMBEROF COMPARISONS TO SORT nELEMENTS.

IN OTHER WORDS,

worstTime(n) >= log2(n!)

BY EXERCISE 12.6,

O(log2(n!) ) = O(n log n)

WE CONCLUDE THAT, FOR ANYCOMPARISON-BASED SORT,worstTime(n) CAN BE NO BETTER THANO(n log n).

TO PUT IT ANOTHER WAY, IFworstTime(n) IS O(n log n) FOR SOMECOMPARISON-BASED SORT METHOD,THEN O(n log n) IS THE SMALLESTUPPER BOUND OF worstTime(n).

SO worstTime(n) CANNOT BE BETTERTHAN O(n log n) FOR ANYCOMPARISON-BASED SORT.

WHAT CAN WE SAY ABOUTaverageTime(n)?

averageTime(n) >= average number of comparisons = total number of comparisons / n!

IN A DECISION TREE, WHAT IS THETOTAL NUMBER OF COMPARISONSEQUAL TO?

THE TOTAL NUMBER OFCOMPARISONS IS EQUAL TO THESUM OF ALL ROOT-TO-LEAF PATHLENGTHS.

E(t) IS THE SUM OF ALL ROOT-TO-LEAF PATH LENGTHS IN t. SO THEAVERAGE NUMBER OF COMPARISONSIS

E(t) / n!

IN A DECISION TREE, THE NUMBEROF LEAVES IS n!. SO, BY THEEXTERNAL PATH LENGTH THEOREM,

E(t) / n! >= (n! / 2) floor (log2(n!)) / n!

= (1 / 2) floor (log2(n!))

floor(log2(n!)) IS O(log n!), WHICH IS

O(n log n).

FOR ANY COMPARISON-BASED SORT,averageTime(n) CAN BE NO BETTERTHAN O(n log n).

EXERCISE: WHICH OF THE FOLLOWING FOLLOWFROM WHAT WE HAVE DONE SO FAR IN THISCHAPTER ABOUT COMPARISON-BASED SORTS?

1. THERE IS A SORT FOR WHICH worstTime(n) ISQUADRATIC IN n.

2. IF, FOR SOME SORT, worstTime(n) ISQUADRATIC IN n, averageTime(n) MUST ALSO BEQUADRATIC IN n.

3. IF, FOR SOME SORT, worstTime(n) IS O(n log n),averageTime(n) MUST ALSO BE O(n log n).

4. IF, FOR SOME SORT, averageTime(n) IS O(n log n),worstTime(n) MUST ALSO BE O(n log n).

FAST SORTS

TREE SORT:

template<class ForwardIterator>void tree_sort (ForwardIterator first, ForwardIterator last);

EACH ITEM IS INSERTED INTO ANINITIALLY EMPTY multiset. THEN THEITEMS ARE COPIED BACK INTO THEORIGINAL CONTAINER.

EXAMPLE:

string my_array[ ] = {“good”, “true”, “right”, “honest”, “kind”};tree_sort (my_array, my_array + 5);

HERE IS THE RED-BLACK TREE:

honest true

good kind

THESE ITEMS ARE COPIED BACKINTO my_array, WHICH NOW HAS:

“good”, “honest”, “kind”, “right”, “true”

IN DEFINING A multiset, ONE OF THETEMPLATE ARGUMENTS IS THE ITEMTYPE, SO WE START WITH

template<class ForwardIterator>void tree_sort (ForwardIterator first, ForwardIterator last){ if (first != last) tree_sort_aux (first, last, *first);} // algorithm tree_sort

template<class ForwardIterator, class T>void tree_sort_aux (ForwardIterator first, ForwardIterator last, T){

multiset< T, less< T > > tree_set;

ForwardIterator itr;

for (itr = first; itr != last; itr++)tree_set.insert (*itr);

copy (tree_set.begin( ), tree_set.end( ), first);} // tree_sort_aux

FOR TREE SORT, EACH INSERTION TAKESLOGARITHMIC TIME AT MOST, SO FOR nINSERTIONS worstTime(n) IS O(n log n).

THEREFOREaverageTime(n) IS O(n log n)

HEAP SORT

ASSUME WE HAVE A CONTAINER cTHAT SUPPORTS RANDOM-ACCESSITERATORS. TO PERFORM HEAPSORT ON c:

make_heap (c.begin( ), c.end( ));sort_heap (c.begin( ), c.end( ));

THE GENERIC ALGORITHM make_heap(SEE CHAPTER 11) INTERPRETS THECONTAINER AS A HEAP. STARTINGNEAR THE BACK OF THE HEAP, ATTHE HIGHEST-INDEXED NON-LEAF,AND WORKING BACK TO THE ROOT,CONVERT EACH SUBTREE INTO AHEAP.

FOR EXAMPLE, SUPPOSE THECONTAINER IS A VECTOR WITH THEFOLLOWING ITEMS:

55, 82, 17, 46, 92, 78, 61, 33

WHEN A HEAP STRUCTURE ISSUPERIMPOSED ON THIS VECTOR,WE GET:

46 92 78 61

MAKING EACH SUBTREE INTO AHEAP IS STRAIGTFORWARDBECAUSE THE LEFT AND RIGHTSUBTREES ARE ALREADY HEAPS.NOTE THAT A LEAF ISAUTOMATICALLY A HEAP.

WE START WITH THE SUBTREEROOTED AT 46, THEN 17, 82, AND 55.

46 92 78 61

THIS BECOMES:

46 92 17 61

46 92 78 61

THIS BECOMES:

46 82 17 61

THIS BECOMES:

46 55 17 61

NOW THE GENERIC ALGORITHMsort_heap TAKES OVER: THE GENERICALGORITHM pop_heap ISREPEATEDLY CALLED UNTIL THEHEAP CONSISTS OF A SINGLE ITEM.

// Precondition: the items in the container, from the range// first (inclusive) to last (exclusive), form a// heap.// Postcondition: the items in the container, from the range// first (inclusive) to last (exclusive), are in// ascending order. The worstTime(n) is// O(n log n).void sort_heap (RandomAccessIterator first, RandomAccessIterator last){ while (last - first > 1) pop_heap (first, last--);} // sort_heap

RECALL THAT pop_heap INSERTS THE POPPED ITEMAT POSITON last.

55 61 78 82

ANALYSIS:FOR make_heap,worstTime(n) IS LINEAR IN n.

FOR pop_heap, worstTime(n) ISLOGARITHMIC IN n. SO FORsort_heap, AND FOR HEAP SORT,worstTime(n) IS O(n log n).THEREFORE, FOR HEAP SORTaverageTime(n) IS O(n log n), AND THISIS MINIMAL.

EXERCISE: SHOW THE STEPS INPERFORMING HEAP SORT ON ANARRAY WITH

1, 2, 3, 4, 5, 6, 7, 8, 9, 10

MERGE SORT

MERGE SORT IS THE SORT METHODFOR THE list CLASS. HERE IS THEMETHOD INTERFACE:

// Postcondition: This list is in ascending order// according to operator<. The// worstTime(n) is O(n log n).void sort( );

BEFORE WE GET TO THE DETAILSOF THE sort METHOD, WE NEED ANAUXILIARY METHOD:

// Precondition: Both this list and x are sorted lists,// ordered by operator<.// Postcondition: This list is a sorted list that contains all of// the items that were, before this call,// either in this list or in x. And x is now// empty. If, before this call, this list had// size m and x had size n, then// worstTime(m, n) is O(max (m, n)).void merge (list<T>& x);

FOR EXAMPLE, SUPPOSE WE HAVE

list1: 38, 46, 52, 97

list2: 20, 60, 85, 89, 92, 101, 105, 119

AFTER A CALL (WITHIN THE listCLASS OR SUBCLASS) TO

list1.merge (list2);

THE CONTENTS OF THE TWO LISTSARE:

list1: 20, 38, 46, 52, 60, 85, 89, 92, 97, 101, 105, 119

list2:

THE MERGING IS ACCOMPLISHED BYPOINTER ADJUSTMENTS: NO ITEMSARE MOVED!

RECALL THE ORGANIZATION OF LISTNODES:

prev data next prev data next

IN EACH list_node, THE data FIELDHOLDS THE ITEM, prev POINTS TOTHE PREVIOUS list_node, AND next

POINTS TO THE NEXT list_node.

ALSO, IN THE list CLASS’SEMBEDDED iterator CLASS, THERE ISONLY ONE FIELD, A POINTER.

THE merge METHOD DEFINES TWOITERATORS: first1, POINTING TO ANODE IN THE CALLING OBJECT, ANDfirst2, POINTING TO A NODE IN THEPARAMETER x.

INITIALLY:

38, 46, 52, 97

first1

20, 60, 85, 89, 92, 101, 105, 119

first2

NOW WE LOOP UNTIL THE END OFONE OF THE LISTS IS REACHED.DURING EACH LOOP ITERATION:

IF (*first1 < *first2)TRANSFER, BY POINTER ADJUSTMENTS,*first2 TO JUST BEFORE first1; first2++;

ELSEfirst1++;

BECAUSE 20 = *first2 < *first1 = 38:

20, 38, 46, 52, 97

first1

60, 85, 89, 92, 101, 105, 119

first2

BECAUSE 38, 46, AND 52 ARE < 60:

20, 38, 46, 52, 97

first1

60, 85, 89, 92, 101, 105, 119

first2

BECAUSE 60, 85, 89, AND 92 ARE < 97:

20, 38, 46, 52, 60, 85, 89, 92, 97

first1

101, 105, 119

first2

BECAUSE 97 < 101:

20, 38, 46, 52, 60, 85, 89, 92, 97

first1

101, 105, 119

first2

BECAUSE WE ARE AT THE END OFTHE FIRST LIST, WE EXIT THE LOOP,AND TRANSFER THE REMAININGITEMS IN THE SECOND LIST TO THEFIRST LIST.

FINALLY, INCREMENT THE FIRSTLIST’S SIZE BY THE SIZE OF THESECOND LIST, AND DESTROY THESECOND LIST.

template <class T>void list<T>::merge(list<T>& x){ iterator first1 = begin(); iterator last1 = end(); iterator first2 = x.begin(); iterator last2 = x.end(); while (first1 != last1 && first2 != last2)

if (*first2 < *first1) { iterator next = first2; transfer(first1, first2, ++next); first2 = next;

++first1; if (first2 != last2) transfer(last1, first2, last2); length += x.length; x.length = 0;}

IF BOTH LISTS HAVE SIZE n, THENUMBER OF LOOP ITERATIONSRANGES FROM n (IF ALL THE ITEMSIN ONE LIST ARE LESS THAN THEFIRST ITEM IN THE OTHER LIST) TO2n – 1 (IF THE TWO LISTS AREINTERLEAVED, SUCH AS 10, 30, 50,…AND 20, 40, 60, …). SO worstTime(n) ANDaverageTime(n) ARE LINEAR IN n.

THE sort METHOD REPEATEDLYMERGES SUBLISTS INTO DOUBLE-SIZED SUBLISTS. SUBLISTS OF SIZE 1ARE AUTOMATICALLY SORTED, SOTHE FIRST MERGING IS BETWEENTHE FIRST AND SECOND ELEMENTS(THAT IS, SUBLISTS OF SIZE 1) OFTHE CALLING LIST OBJECT.

THE MERGED SUBLISTS ARESTORED IN counter, AN ARRAY OF 64LISTS. FOR i = 0, 1, … 63, counter [i] ISINITIALLY EMPTY, AND ALWAYSCONTAINS A LIST OF SIZE 0, 2i, OR2i+1. IF counter [i] HAS SIZE 2i+1, THATLIST IS MERGED WITH counter [i + 1].

FOR EXAMPLE, SUPPOSE WE STARTWITH

90, 70, 50, 60, 80, 40, 10, 30, 20

HERE IS A SEQUENCE OF SNAPSHOTSOF THE counter ARRAY. THERE ISALSO A TEMPORARY LIST, carry,THAT SERVES AS AN INTERMEDIARYBETWEEN THE CALLING OBJECT ANDTHE counter ARRAY, AND BETWEENTHE SUBLISTS IN THE counter ARRAY.