The Art of Data Structures Sortingrsarkis/csc162/_static/lectures/Sorting.pdfBubble Sort Exchanging...

The Art of Data Structures Sorting

Richard E Sarkis CSC 162: The Art of Data Structures

Class Administrivia

Agenda

• To be able to explain and implement various sorting algorithms

• Bubble

• Selection

• Insertion

• Shell

• Merge

• Quick

Sorting

Sorting

• This is the process of organizing data in some particular order

• Numbers, increasing order

• Words, alphabetically

• etc

• Some algorithms benefit with pre-sorted data, e.g. binary search

Sorting

• Sorting is an important area of computer science

• Many sorting algorithms have been developed, and analyzed

• Sorting can take significant time, and is related to the number of items to process

Sorting

• Sorting requires two main operations:

• Comparisons of items to see if they are out of order; comparisons will be an important metric

• Exchange of items can be a costly operation, and also an important metric

Bubble Sort

Bubble Sort bubble_sort: The First Pass

What Is Algorithm Analysis?Searching

Sorting

The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort

bubbleSort: The First Pass

54 26 93 17 77 31 44 55 20

26 54 93 17 77 31 44 55 20

26 54 93 17 77 31 44 55 20

26 54 17 93 77 31 44 55 20

26 54 17 77 93 31 44 55 20

26 54 17 77 31 93 44 55 20

26 54 17 77 31 44 93 55 20

26 54 17 77 31 44 55 93 20

Exchange

No Exchange

Exchange

Exchange

Exchange

Exchange

Exchange

Exchange

26 54 17 77 31 44 55 20 9393 in place

after first pass

First pass

Algorithm Analysis

Bubble Sort Exchanging Two Values in Python


Sorting


Exchanging Two Values in Python

93 44

i j

temp

1st 3rd

2nd

93 44

Most programming languages require a 3-stepprocess with an extra storage location.

In Python, exchange can be done astwo simultaneous assignments.

Algorithm Analysis

Bubble Sort Implementation

def bubble_sort(alist): for passnum in range(len(alist)-1,0,-1): for i in range(passnum): if alist[i]>alist[i+1]: temp = alist[i] alist[i] = alist[i+1] alist[i+1] = temp

Bubble Sort Modified Implementation (cont.)

def bubble_sorted(alist): exchanges = True passnum = len(alist)-1 while passnum > 0 and exchanges: exchanges = False for i in range(passnum): if alist[i] > alist[i+1]: exchanges = True temp = alist[i] alist[i] = alist[i+1] alist[i+1] = temp passnum = passnum-1

Selection Sort

Selection Sort


Sorting


selectionSort

54 26 93 17 77 31 44 55 20

26 54 20 17 77 31 44 55 93

93 is largest

77 is largest

26 54 20 17 55 31 44 77 93 55 is largest

26 54 20 17 44 31 55 77 93 54 is largest

26 31 20 17 44 54 55 77 93 44 is largeststays in place

26 31 20 17 44 54 55 77 93 31 is largest

26 17 20 31 44 54 55 77 93 26 is largest

20 17 26 31 44 54 55 77 93 20 is largest

17 20 26 31 44 54 55 77 93 17 oklist is sorted

Algorithm Analysis

Selection Sort Implementation

def selection_sort(alist): for fillslot in range(len(alist)-1, 0, -1): position_of_max = 0 for location in range(1, fillslot+1): if alist[location] > alist[position_of_max]: position_of_max = location

temp = alist[fillslot] alist[fillslot] = alist[position_of_max] alist[position_of_max] = temp

Insertion Sort

Insertion Sort


Sorting


insertionSort

54 26 93 17 77 31 44 55 20

26 54 93 17 77 31 44 55 20

26 54 93 17 77 31 44 55 20

17 26 54 93 77 31 44 55 20

17 26 54 77 93 31 44 55 20

17 26 31 54 77 93 44 55 20

17 26 31 44 54 77 93 55 20

17 26 31 44 54 55 77 93 20

17 20 26 31 44 54 55 77 93

Assume 54 is a sortedlist of 1 item

inserted 26

inserted 93

inserted 17

inserted 77

inserted 31

inserted 44

inserted 55

inserted 20

Algorithm Analysis

Insertion Sort insertion_sort: The Fifth Pass


Sorting


insertionSort: Fifth Pass of the Sort

17 26 54 77 93 31 44 55 20

17 26 54 77 93 44 55 20

Need to insert 31

back into the sorted list

93>31 so shift it

to the right

17 26 54 77 93 44 55 20 77>31 so shift it

to the right

17 26 54 77 93 44 55 20 54>31 so shift it

to the right

17 26 31 54 77 93 44 55 20 26<31 so insert 31

in this position

Algorithm Analysis

Insertion Sort Implementation (cont.)

def insertion_sort(alist): for index in range(1, len(alist)): currentvalue = alist[index] position = index

while position > 0 and alist[position-1] > currentvalue: alist[position] = alist[position-1] position = position-1

alist[position] = currentvalue

Shell Sort

Shell Sort With Increments of Three

5.3. Sorting 215

1 def insertionSort(alist ):2 for index in range(1,len(alist )):3

4 currentvalue = alist[index]5 position = index6

7 while position >0 and alist[position -1]> currentvalue:8 alist[position ]=alist[position -1]9 position = position -1

10

11 alist[position ]= currentvalue

Listing 5.12: insertionSort

This can be seen in Figure 5.18. This list has nine items. If we usean increment of three, there are three sublists, each of which can be sortedby an insertion sort. After completing these sorts, we get the list shownin Figure 5.19. Although this list is not completely sorted, something veryinteresting has happened. By sorting the sublists, we have moved the itemscloser to where they actually belong.

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

sublist 1

sublist 2

sublist 3

Figure 5.18: A Shell Sort with Increments of Three

Figure 5.20 shows a final insertion sort using an increment of one; inother words, a standard insertion sort. Note that by performing the earliersublist sorts, we have now reduced the total number of shifting operationsnecessary to put the list in its final order. For this case, we need only fourmore shifts to complete the process.

Shell Sort After Sorting Each Sublist216 Chapter 5. Searching and Sorting

17 26 93 44 77 31 54 55 20

54 26 93 17 55 31 44 77 20

54 26 20 17 77 31 44 55 93

sublist 1 sorted

sublist 2 sorted

sublist 3 sorted

17 26 20 44 55 31 54 77 93 after sorting sublistsat increment 3

Figure 5.19: A Shell Sort after Sorting Each Sublist

17 26 20 44 55 31 54 77 93 1 shift for 20

17 20 26 44 55 31 54 77 93 2 shifts for 31

17 20 26 31 44 55 54 77 93 1 shift for 54

17 20 26 31 44 54 55 77 93 sorted

Figure 5.20: ShellSort: A Final Insertion Sort with Increment of 1

We said earlier that the way in which the increments are chosen is theunique feature of the shell sort. The function shown in Listing 5.13 uses adi↵erent set of increments. In this case, we begin with n

2 sublists. On thenext pass, n

4 sublists are sorted. Eventually, a single list is sorted with the

Shell Sort A Final Insertion Sort with Increment of 1

216 Chapter 5. Searching and Sorting

17 26 93 44 77 31 54 55 20

54 26 93 17 55 31 44 77 20

54 26 20 17 77 31 44 55 93

sublist 1 sorted

sublist 2 sorted

sublist 3 sorted

17 26 20 44 55 31 54 77 93 after sorting sublistsat increment 3

Figure 5.19: A Shell Sort after Sorting Each Sublist

17 26 20 44 55 31 54 77 93 1 shift for 20

17 20 26 44 55 31 54 77 93 2 shifts for 31

17 20 26 31 44 55 54 77 93 1 shift for 54

17 20 26 31 44 54 55 77 93 sorted

Figure 5.20: ShellSort: A Final Insertion Sort with Increment of 1

We said earlier that the way in which the increments are chosen is theunique feature of the shell sort. The function shown in Listing 5.13 uses adi↵erent set of increments. In this case, we begin with n

2 sublists. On thenext pass, n

4 sublists are sorted. Eventually, a single list is sorted with the

Shell Sort Initial Sublists for a Shell Sort

5.3. Sorting 217

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

sublist 1

sublist 2

sublist 3

54 26 93 17 77 31 44 55 20 sublist 4

Figure 5.21: Initial Sublists for a Shell Sort

basic insertion sort. Figure 5.21 shows the first sublists for our exampleusing this increment.

The following invocation of the shellSort function shows the partiallysorted lists after each increment, with the final sort being an insertion sortwith an increment of one.

>>> alist=[54,26,93,17,77,31,44,55,20]

>>> shellSort(alist)

After increments of size 4 the list is

[20, 26, 44, 17, 54, 31, 93, 55, 77]


[20, 17, 44, 26, 54, 31, 77, 55, 93]


[17, 20, 26, 31, 44, 54, 55, 77, 93]

At first glance you may think that a shell sort cannot be better thanan insertion sort, since it does a complete insertion sort as the last step. Itturns out, however, that this final insertion sort does not need to do verymany comparisons (or shifts) since the list has been pre-sorted by earlierincremental insertion sorts, as described above. In other words, each passproduces a list that is “more sorted” than the previous one. This makes thefinal pass very e�cient.

Although a general analysis of the shell sort is well beyond the scopeof this text, we can say that it tends to fall somewhere between O(n) andO(n2), based on the behavior described above. For the increments shown

Shell Sort Implementation

def shell_sort(alist): sublistcount = len(alist)//2 while sublistcount > 0: for startposition in range(sublistcount): gap_insertion_sort(alist, startposition, sublistcount)

print("After increments of size", sublistcount, "The list is", alist)

sublistcount = sublistcount // 2

Shell Sort Implementation

def gap_insertion_sort(alist, start, gap): for i in range(start+gap, len(alist), gap): currentvalue = alist[i] position = i

while position >= gap and \ alist[position-gap] > currentvalue: alist[position] = alist[position-gap] position = position-gap

alist[position] = currentvalue

Merge Sort

Merge Sort Splitting and Merging


Sorting


Splitting and Merging in a Merge Sort

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

54 26 93 17 77 31 44 55 20

55 20

Algorithm Analysis

Merge Sort Splitting and Merging


Sorting


Splitting and Merging in a Merge Sort

17 20 26 31 44 54 55 77 93

17 26 54 93

20 31 44 55 77

26 54 17 93 31 77

20 44 55

54 26 93 17 77 31 44

20 55

55 20

Algorithm Analysis

Merge Sort Implementation

def merge_sort(alist): print("Splitting ", alist) if len(alist) > 1: mid = len(alist)//2 lefthalf = alist[:mid] righthalf = alist[mid:]

merge_sort(lefthalf) merge_sort(righthalf)

i = 0 j = 0 k = 0

Merge Sort Implementation (cont.)

while i < len(lefthalf) and j < len(righthalf): if lefthalf[i] < righthalf[j]: alist[k] = lefthalf[i] i = i+1 else: alist[k] = righthalf[j] j = j+1 k = k+1

while i < len(lefthalf): alist[k] = lefthalf[i] i = i+1 k = k+1

while j < len(righthalf): alist[k] = righthalf[j] j = j+1 k = k+1 print("Merging ", alist)

Quick Sort

Quick Sort The First Pivot Value


Sorting


The First Pivot Value for a Quick Sort

54 26 93 17 77 31 44 55 20 54 will be thefirst pivot value

Algorithm Analysis

Quick Sort Finding the Split Point for 54


Sorting


Finding the Split Point for 54

54 26 93 17 77 31 44 55 20

leftmark rightmark

54 26 93 17 77 31 44 55 20

leftmark rightmark

54 26 93 17 77 31 44 55 20

leftmark rightmark

54 26 20 17 77 31 44 55 93

leftmark rightmark

54 26 20 17 44 31 77 55 93

leftmarkrightmark

26<54 move to right93>54 stop

now rightmark20<54 stop

54 26 20 17 77 31 44 55 93

leftmark rightmark

exchange 20 and 93

77>54 stop44<54 stop

exchange 77 and 44

77>54 stop31<54 stop

rightmark<leftmarksplit point found

exchange 54 and 31

leftmark and rightmark will converge on split point

now continue moving leftmark and rightmark

until they cross

Algorithm Analysis

Quick Sort Completing the Partition Process to Find

the Split Point for 54


Sorting


Completing the Partition Process to Find the SplitPoint for 54

31 26 20 17 44 54 77 55 93 54 is in place

<54 >54

31 26 20 17 44 77 55 93

quicksort left half quicksort right half

Algorithm Analysis

Quick Sort Implementation

def quick_sort(alist): quick_sort_helper(alist, 0, len(alist)-1)

def quick_sort_helper(alist, first, last): if first<last:

splitpoint = partition(alist,first,last)

quick_sort_helper(alist, first, splitpoint-1) quick_sort_helper(alist, splitpoint+1, last)

Quick Sort Implementation (cont.)

def partition(alist, first, last): pivotvalue = alist[first]

leftmark = first+1 rightmark = last

done = False while not done: while leftmark <= rightmark and \ alist[leftmark] <= pivotvalue: leftmark = leftmark + 1

while alist[rightmark] >= pivotvalue and \ rightmark >= leftmark: rightmark = rightmark -1

# Continued on next slide...

Quick Sort Implementation (cont.)

if rightmark < leftmark: done = True else: temp = alist[leftmark] alist[leftmark] = alist[rightmark] alist[rightmark] = temp

temp = alist[first] alist[first] = alist[rightmark] alist[rightmark] = temp

return rightmark

Analysis

Analysis

• A bubble sort, a selection sort, and an insertion sort are O(n2) algorithms

• A shell sort improves on the insertion sort by sorting incremental sub-lists

• It falls between O(n) and O(n2)

• A merge sort is O(n log n), but requires additional space for the merging process

Analysis

• A quick sort is O(n log n), but may degrade to O(n2) if the split points are not near the middle of the list

• It does not require additional space

Questions?

The Art of Data Structures Sortingrsarkis/csc162/_static/lectures/Sorting.pdfBubble Sort Exchanging...

Documents

Transcript of The Art of Data Structures Sortingrsarkis/csc162/_static/lectures/Sorting.pdfBubble Sort Exchanging...