Sorting

•Sorting:–Task of rearranging data in an order.–Order can be:

•Ascending Order:–1,2,3,4,5,6,7,8,9

•Descending Order:–9,8,7,6,5,4,3,2,1

•Lexicographic Order:–Dictionary Order

»ada, bat, cat, mat, max, may, min

Sorting

•Terminologies:

2 5 6 4 3 2 5 1 5

1 2 2 3 4 5 5 5 6

Stable Sort

Unsorted

Sorted

Sorting

•Terminologies:

2 5 6 4 3 2 5 1 5

1 2 2 3 4 5 5 5 6

Unsorted Array A

Sorted Array B

2 5 6 4 3 2 5 1 5

1 2 2 3 4 5 5 5 6

Unsorted Array A

Sorted Array A

Not In Place

In Place Sorting

Sorting•Terminologies:

–Stable Sort:•A list of unsorted data may contain two or more equal data.•If a sorting method maintains,

–The same relative position of their occurrences in the sorted list, then it is called:–Stable Sort

–In Place Sort:•Suppose a set of data to be stored is stored in an array A. •If a sorting method takes place,

–Within the array A only, that is, without using any other extra storage space, then it is called:–In Place Sorting Method

»Memory efficient because it does not require extra memory space.

Sorting

•Methods of Sorting:–Various sorting methods are:

•Bubble Sort•Selection Sort•Insertion Sort•Quick Sort•Merge Sort•Heap Sort

Bubble Sort30

20

70

10

Pass-1

20

30

70

10

20

30

70

10

20

30

10

70

20

30

10

70

Pass-2

20

30

10

70

20

10

30

70

20

10

30

70

20

10

30

70

Pass-3

10

20

30

70

10

20

30

70

10

20

30

70

1

2

3

4

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

n = 4Array A

Bubble Sort

Pass-1

Pass-2

Pass-3

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

for i = 1 to n-1 do //Controls number of passes

for j = 1 to n-1 do //Controls comparisons in each pass

if(A[ j ] > A[ j+1 ]), then

Swap(A[ j ], A[ j+1 ])

EndIf

EndFor

EndFor

Stop

Any optimization possible?

n = 4

Bubble Sort30

20

70

10

Pass-1

20

30

70

10

20

30

70

10

20

30

10

70

20

30

10

70

Pass-2

20

30

10

70

20

10

30

70

20

10

30

70

Pass-3

10

20

30

70

1

2

3

4

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[1] vs A[2]

Bubble Sort

Pass-1

Pass-2

Pass-3

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[1] vs A[2]

n = 4

for i = 1 to n-1 do //Controls number of passes

for j = 1 to n-i do //Controls comparisons in each pass

if(A[ j ] > A[ j+1 ]), then

Swap(A[ j ], A[ j+1 ])

EndIf

EndFor

EndFor

Stop

i = 1

i = 2

i = 3

j = 1 to 3

j = 1 to 2

j = 1 to 1

Bubble Sort•Algorithm:

–BubbleSort

•Input:–Array A[1...n] where n is the number of elements.

•Output:–Array A with all elements in ascending sorted order.

•Data Structure:–Array A[1...n]

Bubble SortSteps:

for i = 1 to n-1 do //Controls the number of passes

for j = 1 to n-i do //Controls the comparisons in each pass

if(A[ j ] > A[ j+1 ]), then

//Logic to swap

temp = A[ j ]

A[ j ] = A[ j+1 ]

A[ j+1 ] = temp

EndIf

EndFor

EndFor

Stop

Tracing of Bubble Sort

30

20

10

40

Pass-1

20

30

10

40

20

10

30

40

20

10

30

40

20

10

30

40

Pass-2

10

20

30

40

10

20

30

40

10

20

30

40

Pass-3

10

20

30

40

1

2

3

4

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[1] vs A[2]

Tracing of Bubble Sort

40

30

20

10

Pass-1

30

40

20

10

30

20

40

10

30

20

10

40

30

20

10

40

Pass-2

20

30

10

40

20

10

30

40

20

10

30

40

Pass-3

10

20

30

40

1

2

3

4

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[1] vs A[2]

Tracing of Bubble Sort

10

20

30

40

Pass-1

10

20

30

40

10

20

30

40

10

20

30

40

10

20

30

40

Pass-2

10

20

30

40

10

20

30

40

10

20

30

40

Pass-3

10

20

30

40

1

2

3

4

A[1] vs A[2]

A[2] vs A[3]

A[3] vs A[4]

A[1] vs A[2]

A[2] vs A[3]

A[1] vs A[2]

Tracing of Bubble Sort

55

77

99

33

22

88

66

44

1

2

3

4

5

6

7

8

Exercise

Bubble Sort

•Analysis of Algorithm BubbleSort:–Performance of any sorting algorithm depends on:

•Number of comparisons•Number of movements (Swappings)

Analysis of algorithm BubbleSort

Summary

Case Complexity

Best Case

List already in sorted order.T(n) = n2

Worst Case

List sorted in reverse order.T(n) = n2

Average Case

List sorted in random order.T(n) = n2

Selection Sort

1 2 3 4 5

InputArray A

min_val

min_loc

Pass-1

Assumption: Element atlocation 1 should be minimum.

40 30 50 10 20

1 2 3 4 5

40 30 50 10 20

1 2 3 4 5

10 30 50 40 20Pass-2

Assumption: Element atlocation 2 should be minimum.

min_val

min_loc

Minimum elementshould have been

at location 1.

Is it at same location? No

SWAP

Minimum elementshould have been

at location 2.

Is it at same location? No

SWAP

1 2 3 4 5

10 20 50 40 30

Selection Sort

Pass-3

Assumption: Element atlocation 3 should be minimum.

min_val

min_loc

Minimum elementshould have been

at location 3.

Is it at same location? No

SWAP

1 2 3 4 5

10 20 30 40 50Pass-4

Assumption: Element atlocation 4 should be minimum.

min_val

min_loc

Minimum elementshould have been

at location 4.

Is it at same location? Yes

NO SWAP

Output1 2 3 4 5

10 20 30 40 50

1 2 3 4 5

10 20 30 40 50

Selection SortInput

Array A

Pass-1

Assumption: Element at location 1should be minimum.

Minimum elementshould have been

at location 1.

Is it at same location? No

SWAPmin_loc

min_val

1 2 3 4 5

50 40 30 20 10

1 2 3 4 5

50 40 30 20 10

min_loc

min_val

min_loc

min_val

min_loc

min_val

min_loc

min_val

Pass-2

Assumption: Element at location 2should be minimum.

Minimum elementshould have been

at location 2.

Is it at same location? No

SWAP1 2 3 4 5

10 40 30 20 50

min_loc

min_val

min_loc

min_val

min_loc

min_val

Pass-3

Assumption: Element at location 3should be minimum.

Minimum elementshould have been

at location 3.

Is it at same location? Yes

NO SWAP1 2 3 4 5

10 20 30 40 50

min_loc

min_val

Selection Sort

Pass-4

Assumption: Element at location 4should be minimum.

Minimum elementshould have been

at location 4.

Is it at same location? Yes

NO SWAP1 2 3 4 5

10 20 30 40 50

min_loc

min_val

Output54321

10 20 30 40 5054321

10 20 30 40 50

Selection SortFor i = 1 to n-1, do

min_val = A[ i ], min_loc = i

For j = i+1 to n, do

If(min_val > A[ j ]), then

min_val = A[ j ]

min_loc = j

EndIf

EndFor

If(i != min_loc)

Swap(A[ i ], A[min_loc])

EndIf

EndFor

Stop

To sort 5 elements,4 passes were required.

To sort n elements,n-1 passes are required.

In 1st pass, value at location 1will be smallest.In 2nd pass, value at location 2will be smallest.

In i-th pass, value at location iwill be smallest.

In 1st pass, we need to docomparisons from 2nd value.In 2nd pass, we need to do comparisons from 3rd value.

In ith pass, we need to startcomparisons from (i+1)thvalue upto n.

Selection Sort•Algorithm:

–SelectionSort

•Input:–Array A[1...n] where n is the number of elements.

•Output:–Array A with all elements in ascending sorted order.

•Data Structure:–Array A[1...n]

Selection SortSteps:

For i = 1 to n-1, do //Controls the number of passes

min_val = A[ i ], min_loc = i

For j = i+1 to n, do //Controls the comparisons in each pass

If(min_val > A[ j ]), then

min_val = A[ j ]

min_loc = j

EndIf

EndFor

If(i != min_loc)

//Swap A[ i ] and A[min_loc]

temp = A[ i ]

A[ i ] = A[min_loc]

A[min_loc] = temp

EndIf

EndFor

Selection SortInput

Array A

Pass-1

Assumption: Element at location 1should be minimum.

Minimum elementshould have been

at location 1.

Is it at same location? Yes

NO SWAPmin_loc

min_val

1 2 3 4 5

10 20 30 40 50

1 2 3 4 5

10 20 30 40 50

Pass-2

Assumption: Element at location 2should be minimum.

Minimum elementshould have been

at location 2.

Is it at same location? Yes

NO SWAP1 2 3 4 5

10 20 30 40 50

min_loc

min_val

Pass-3

Assumption: Element at location 3should be minimum.

Minimum elementshould have been

at location 3.

Is it at same location? Yes

NO SWAP1 2 3 4 5

10 20 30 40 50

min_loc

min_val

Selection Sort

Pass-4

Assumption: Element at location 4should be minimum.

Minimum elementshould have been

at location 4.

Is it at same location? Yes

NO SWAP1 2 3 4 5

10 20 30 40 50

min_loc

min_val

Output54321

10 20 30 40 5054321

10 20 30 40 50

Tracing of Selection Sort

InputArray A 54321

40 60 50 10 20

Pass-154321

40 60 50 10 20

Min

5432110 60 50 40 20

Pass-254321

10 60 50 40 20

Min

5432110 20 50 40 60

Pass-354321

10 20 50 40 60

Min

5432110 20 40 50 60

Pass-454321

10 20 40 50 60

Min

5432110 20 40 50 60

No Swap

4 6 7 5 9 3 1 8 2

1 2 3 4 5 6 7 8 9

Array A

Tracing of Selection Sort

Analysis of algorithm SelectionSort

Summary

Case Complexity

Best Case

List already in sorted order.T(n) = n2

Worst Case

List sorted in reverse order.T(n) = n2

Average Case

List sorted in random order.T(n) = n2

Insertion Sort

•Insertion Sort:–Based on a method called:

•Bridge Player–The way Bridge Player sort their hands.–Picking up one card at a time, placing into its appropriate position.

Insertion Sort

Array A

1 2 3 4 5

Array B

1 2 3 4 5

40 30 50 10 20

40 30 50 10 20

Insertion Sort B[1] = A[1]

For i = 2 to n, do

//Pick the element

KEY = A[ i ]

//Find appropriate location by comparison

location = i

While(location > 1) AND (KEY < B[location-1]), do

location = location - 1

EndWhile

//Shift elements if required

j = i

While( j > location )

B[ j ] = B [ j – 1 ]

j = j - 1

EndWhile

//Place the element

B[location] = KEY

EndFor

1st element - Direct

For all other elements

1. Pick the element

2. Find appropriateposition / location.

3. Shift existing elements if required.

4. Place the element.

Insertion Sort•Algorithm:

–InsertionSort

•Input:–Array A[1...n] where n is the number of elements.

•Output:–Array B[1...n] with elements sorted in ascending order.

•Data Structure:–Array A[1...n] and B[1...n]

B[1] = A[1]

For i = 2 to n, do

//Pick the element

KEY = A[ i ]

//Find appropriate location by comparison

location = i

While(location > 1) AND (KEY < B[location-1]), do

location = location - 1

EndWhile

//Shift elements if required

j = i

While( j > location )

B[ j ] = B [ j – 1 ]

j = j - 1

EndWhile

//Place the element

B[location] = KEY

EndFor

Steps ofInsertionSort

Tracing of Insertion SortArray A

1 2 3 4 5

40 30 50 10 20

Array B 40

30 40

30 40 50

10 30 40 50

10 30 40 5020

0 1 2 3 4Iteration

Tracing of Insertion Sort

5 6 4 7 9 8 3 1 2Array A

1 2 3 4 5 6 7 8 9

Analysis of algorithm InsertionSort

Summary

Case Complexity

Best Case

List already in sorted order.T(n) = n

Worst Case

List sorted in reverse order.T(n) = n2

Average Case

List sorted in random order.T(n) = n2

SortingDivide & Conquer

Break all the sticks into 2 parts and arrange them in order.

Solution:

1) Divide:

Separate the sticks from the bundle of sticks so that it could be broken.

2) Conquer (Solve / Win):

Break each and every stick into 2 parts (which can be easily done)

3) Combine:

Arrange all the parts in an order.

Quick Sort

•Quick Sort:–To sort an array, it uses the concept of:

•Divide & Conquer

–Process:•Divide a large list/array into a number of smaller lists/arrays.•Sort them separately.•Combine the results to get the sorted list.

Quick SortDivide & Conquer

Problem

Divide

P1 P2 P3 Pn. . . . . . . .

Combine

Solution

Solve

Quick Sort

11 25 44 22 99

1 2 3 4 5 6 7

left

loc

Scan from right to left

right

33 88

rightright

loc

left

Scan from left to right

left left

loc

right

XX

While(A[loc] <= A[right])While(A[loc] >= A[left])

right = right - 1left = left + 1

Quick Sort

22 44 33 99 88

1 2 3 4 5

left right

loc

Scan from right to left

rightrightright

right

While(A[loc] <= A[right])

right = right - 1

AND (loc < right)

Scan from left to right

While(A[loc] >= A[left])

left = left + 1

AND (loc > left)

Quick Sort

44 88 22 33 99

1 2 3 4 5

left right

loc

Scan from right to left

right

loc

left

Scan from left to right

loc

right

loc

left

While (A[loc] <= A[right]) AND loc < rightright = right - 1

While (A[loc] >= A[left]) AND loc > leftleft = left + 1

Quick Sort

Partition Algorithm

Initialize loc to left

loc = left

While(left < right), do

//Scan from right to left

While(A[loc] <= A[right]) AND (loc < right), do

right = right - 1

EndWhile

If(A[loc] > A[right]), then

Swap(A[loc], A[right])

loc = right

left = left + 1

EndIf

//Scan from left to right

While(A[loc] >= A[left]) AND (loc > left), do

left = left + 1

EndWhile

If(A[loc] < A[left]), then

Swap(A[loc], A[left])

loc = left

right = right - 1

EndIf

EndWhile

Scan from right to left

Check as to why the scanning stopped.

Check as to why the scanning stopped.

Repeat the scannings till left and right do not meet.

Scan from left to right

Quick Sort

44 88 22 33 99

1 2 3 4 5

left right

loc

Scan from right to left

right

loc

left

Scan from left to right

loc

right

loc

left

While (A[loc] <= A[right]) AND loc < rightright = right - 1

While (A[loc] >= A[left]) AND loc > leftleft = left + 1

Quick Sort

33 22

1 2

left right

loc

44

3

88 99

4 5

left right

locloc

left

22

1

left rightloc

right

99

5

left rightloc

Quick Sort•Algorithm:

–QuickSort

•Input:–L:

•Lower Bound of Array A.

–U: •Upper Bound of Array A.

•Output:–Array A with elements sorted in ascending order.

•Data Structure:–Array.

Steps:

left = L, right = U

if(left < right), then

loc = Partition(left, right)

QuickSort(left, loc – 1)

QuickSort(loc + 1, right)

EndIf

Stop

Quick Sort

Quick Sort•Algorithm:

–Partition

•Input:–left:

•Index of the leftmost element in Array A.

–right: •Index of the rightmost element in Array A.

•Output:–loc:

•Final position of the pivot element.

•Data Structure:–Array.

•Remark:–Element located at left is taken as the pivot element.

loc = left

While(left < right), do

//Scan from right to left

While(A[loc] <= A[right]) AND (loc < right), do

right = right - 1

EndWhile

If(A[loc] > A[right]), then

Swap(A[loc], A[right])

loc = right

left = left + 1

EndIf

//Scan from left to right

While(A[loc] >= A[left]) AND (loc > left), do

left = left + 1

EndWhile

If(A[loc] < A[left]), then

Swap(A[loc], A[left])

loc = left

right = right - 1

EndIf

EndWhile

Tracing of Quick Sort44 88 22 33 99

1 2 3 4 5

44

33 22

1 2

88 99

4 5

33 88

22

1

99

5

Quick Sort44 88 22 33 99

1 2 3 4 5

22 33 44 88 99

1 2 3 4 5

99 88 44 33 22

1 2 3 4 5

Which is the best case of QuickSort out of above 3 options?

Best Case would be the case in which equal partitions are done.

Analysis of algorithm QuickSort

Summary

Case Complexity

Worst Case

List already in sorted order.T(n) = n2

Worst Case

List sorted in reverse order.T(n) = n2

Best Case / Average Case

List sorted in random order.T(n) = n log2n

Tracing of Quick Sort

55 88 22 99 44 11 66 77 33

1 2 3 4 5 6 7 8 9

Merge Sort

•Merge Sort:–Like Quick Sort, it also works on the basic principle of:

•Divide & Conquer

–It uses a concept / technique called:•Merging

Merging

1 2 3

1 2 3 4 5 6

1 2 3 4 5 6 7 8 9

i n1

j n2

k

Sorted Array A

Sorted Array B

Array C

20 40 60

10 30 50 70 90 110

10 20 30 40 50 60 70 90 110

j

k

i

k

j

k

i

k

j

k

i

4

k

j

k

j

k

j

k

7

10

Merge Sort

Merging Algorithm

Initialize i, j, k to 1

i = 1, j = 1, k = 1

While( i <= n1 ) AND ( j <= n2 ), do

If ( A[ i ] <= B [ j ] )

C[ k ] = A[ i ]

i = i + 1, k = k + 1

Else

C[ k ] = B[ j ]

j = j + 1, k = k + 1

EndIf

EndWhile

If ( i > n1 ), then

While( j <= n2 )

C[ k ] = B[ j ]

j = j + 1, k = k + 1

EndWhile

Else If ( j > n2 ), then

While( i <= n1 )

C[ k ] = A[ i ]

i = i + 1, k = k + 1

EndWhile

EndIf

Compare elements

from both the arraysuntil 1 array finishes.

Now check whicharray has finished.Move the elementsof other array oneby one.

Merge Sort

60 20 40 50 10 30

1 2 3 4 5 6

L Rmid

60 20 40

1 2 3

50 10 30

4 5 6

L mid R L mid R

60 20

1 2

40

3

50 10

4 5

30

6

L mid R L R L mid R L R

60

1

20

2

50

4

10

5

L R L R L R L R

Array A

Merge Sort

60 20 40 50 10 30

1 2 3 4 5 6

60 20 40

1 2 3

50 10 30

4 5 6

60 20

1 2

40

3

50 10

4 5

30

6

60

1

20

2

50

4

10

5

Array A

20 60

1 2

20 40 60

1 2 3

10 50

4 5

10 30 50

4 5 6

10 20 30 40 50 60

1 2 3 4 5 6

Output

Merge Sort•Algorithm:

–MergeSort

•Input:–L:

•Lower Bound of Array A.

–R: •Upper Bound of Array A.

•Output:–Array A with elements sorted in ascending order.

•Data Structure:–Array.

Steps:

if(L< R), then

mid = floor((L+R) / 2)

MergeSort(L, mid)

MergeSort(mid + 1, R)

Merge(L, mid, R)

EndIf

Stop

MergeSort

Merge Sort•Algorithm:

–Merge•Input:

–L: •Lower Bound of 1st sub-array of Array A.

–mid:•Upper Bound of 1st sub-array of Array A.•mid + 1 will be lower bound of 2nd sub-array of Array A.

–R: •Upper Bound of 2nd sub-array of Array A.

•Output:–Two sub-arrays are merged and sorted in the array A.

•Data Structure:–Array.

•Assumption:–Extra storage space of array C.

MergeAlgorithm

Steps:

i = L, j = mid + 1, k = L

While( i <= mid ) AND ( j <= R ), do

If ( A[ i ] <= A [ j ] )

C[ k ] = A[ i ]

i = i + 1, k = k + 1

Else

C[ k ] = A[ j ]

j = j + 1, k = k + 1

EndIf

EndWhile

If ( i > mid ), then

While( j <= R )

C[ k ] = A[ j ]

j = j + 1, k = k + 1

EndWhile

Else If ( j > R ), then

While( i <= mid )

C[ k ] = A[ i ]

i = i + 1, k = k + 1

EndWhile

EndIf

For m = L to k – 1, do

A[ m ] = C[ m ]

m = m + 1

EndFor

Quick Sort v/s Merge Sort

if(left < right), then

loc = Partition(left, right)

QuickSort(left, loc – 1)

QuickSort(loc + 1, right)

//Not required

EndIf

Stop

if(L< R), then

mid = floor((L+R) / 2)

MergeSort(L, mid)

MergeSort(mid + 1, R)

Merge(L, mid, R)

EndIf

Stop

Quick Sort Merge Sort

Divide

Conquer

Combine

HARD DIVISION, EASY COMBINATION EASY DIVISION, HARD COMBINATION

Equal sub-division is not alwaysguaranteed.

Both the sub-problems are of almostequal size always.