8/12/2019 Sorting Insertion Sort Merge Sort
1/45
Sorting
How to sort data efficiently?
8/12/2019 Sorting Insertion Sort Merge Sort
2/45
Sorting
The sorting problem is to arrange a sequence of
records so that the values of their key fields form
a nondecreasing sequence.
That is, given records r1, r2, . . . ,rn, with keyvalues k1, k2, . . . , kn, respectively, we must
produce the same records in an order
ri1, ri2 , . . . , rin
such that ki1 ki2 . kin
. The records all need not have distinct values
8/12/2019 Sorting Insertion Sort Merge Sort
3/45
Insertion Sort
Insertion sort is a simple sorting algorithm
that is appropriate for small inputs
Its very time consuming so if the size of
input is large, Insertion Sort is not a likely
choice for sorting that data.
Merge Sort, QuickSort are faster
algorithms for sorting for that matter.
8/12/2019 Sorting Insertion Sort Merge Sort
4/45
Insertion Sort contin.
The main idea of insertion sort is
Start by considering the first two elements of the
array data. If found out of order, swap them
Consider the third element and insert it into itsproper position among the first three elements
Consider the fourth element and place it in its
proper position among the first four elementsand so on..
After each ith iteration, first i elements should be
sorted
8/12/2019 Sorting Insertion Sort Merge Sort
5/45
Insertion Sort (With an
Example) In e game of cards, a player gets 13 cards.
He keeps them in the sorted order in his hand for his
ease.
A player looks at the first two cards, sorts them andkeeps the smaller card first and then the second.
Suppose that two cards were 9 and 8, the player swap
them and keep 8 before 9.
Now he takes the third card. Suppose, it is 10, then it isin its position. If this card is of number 2, the player will
pick it up and put it on the start of the cards.
Then he looks at the fourth card and inserts it in the first
three cards (that he has sorted) at a proper place.
8/12/2019 Sorting Insertion Sort Merge Sort
6/45
Insertion (With an Example)
cont. He repeats the same process with all the
cards and finally gets the cards in a sorted
order. Thus in this algorithm, we keep the
left part of the array sorted and takeelement from the right and insert it in the
left part at its proper place. Due to this
process of insertion, it is called insertionsorting.
8/12/2019 Sorting Insertion Sort Merge Sort
7/45
Insertion Sort Cont
8/12/2019 Sorting Insertion Sort Merge Sort
8/45
Insertion Sort cont..
The array consists of the elements 19, 12, 5 and 7.
We take the first two numbers i.e. 19 and 12. As we see 12 is less
than 19, so we swap their positions. Thus 12 comes at index 0 and
19 goes to index 1.
Now we pick the third number i.e. 5. We have to find the position ofthis number by comparing it with the two already sorted numbers.
These numbers are 12 and 19. We see that 5 is smaller than these
two. So it should come before these two numbers. Thus the proper
position of 5 is index 0. To insert it at index 0, we shift the numbers
12 and 19 before inserting 5 at index 0. Thus 5 has come at its
position.
Now we pick the number 7 and find its position between 5 and 12.
To insert 7 after 5 and before 12, we have to shift the numbers 12
and 19 to the right. After this shifting, we put number 7 at its
position. Now the whole array has been sorted so the process stops
here.
8/12/2019 Sorting Insertion Sort Merge Sort
9/45
Insertion Sort Cont.
void insertionSort(int *arr, int N)
{
int pos, count, val;
for(count=1; count < N; count++)
{
val = arr[count];
for(pos=count-1; pos >= 0; pos--)
if (arr[pos] > val)
arr[pos+1]=arr[pos];else break;
arr[pos+1] = val;
}
}
8/12/2019 Sorting Insertion Sort Merge Sort
10/45
Insertion Sort Analysis
To insert the last element we need at most N-1 comparisons and N-
1 movements.
To insert the N-1stelement we need N-2 comparisons and N-2
movements.
.
To insert the 2ndelement we need 1 comparison and one
movement.
To sum up:
2* (1 + 2 + 3 + N - 1) = 2 * (N - 1)* N / 2 = (N-1)*N = (N2)
If the greater part of the array is sorted, the complexity is almostO(N)
The average complexity is proved to be = (N2)
8/12/2019 Sorting Insertion Sort Merge Sort
11/45
Insertion Sort an O(N^2)
Algorithm Insertion Sort in an O(N^2) algorithm.
Similarly, there are other O(N^2)
algorithms for sorting like Selection Sort,
Bubble Sort
O(N^2) is not a very favorable amount of
time we would like to spend on sorting.
Lets try to understand O(nlogn) algorithms
for sorting
8/12/2019 Sorting Insertion Sort Merge Sort
12/45
O(nlogn) Sorting Algorithms
MergeSort
Quick Sort
Heap Sort
Reading 11.1 (MergeSort), 11.2
(QuickSort) from GTM
8/12/2019 Sorting Insertion Sort Merge Sort
13/45
Merge Sort
MergeSort Algorithms fall under divide and
conquercategory
The divide and conquer strategy is well known in
wars. The philosophy of this strategy is , divideyour enemy into parts and then conquer these
parts. To conquer these parts is easy, as these
parts cannot resist or react like a big united
enemy. The same philosophy is applied in theabove algorithms. To understand the divide and
conquer strategy in sorting algorithm, lets
consider an example.
8/12/2019 Sorting Insertion Sort Merge Sort
14/45
Merge Sort Cont
Divide and Conquer Example Suppose we have an unsorted array of
numbers is given below.
8/12/2019 Sorting Insertion Sort Merge Sort
15/45
Merge Sort Cont
Divide and Conquer Example Now we split this array into two parts
8/12/2019 Sorting Insertion Sort Merge Sort
16/45
Merge Sort Cont
Divide and Conquer Example Now we have two parts of the array. We
sort these parts separately. Suppose we
sort these parts with an elementary sort
algorithm. These parts may be sorted inthe following manner.
8/12/2019 Sorting Insertion Sort Merge Sort
17/45
Merge Sort Cont
Divide and Conquer Example After this we merge these two parts and
get the sorted array as shown below.
8/12/2019 Sorting Insertion Sort Merge Sort
18/45
Simple Analysis of Divide and
Conquer Let see few analysis to confirm the usefulness
of the divide and conquer technique.
To sort the halves approximate time is
(n/2)^2+(n/2)^2 To merge the two halves approximate time is n
So, for n=100,
divide and conquer takes approximately: =(100/2)^2 + (100/2)^2 + 100 = 2500 + 2500 +
100 = 5100
8/12/2019 Sorting Insertion Sort Merge Sort
19/45
8/12/2019 Sorting Insertion Sort Merge Sort
20/45
Divide and Conquer
Advantages con. So the whole operation of sorting using this
divide and conquer technique in insertion sort
will take around
(100/2)^2 + (100/2)^2+100 = 5100.Clearly the time spent (5100) after applying
divide and conquer mechanism is significantly
lesser than the previous time (10000). It is
reduced approximately to half of the previous
time. This example shows the usefulness of
divide and conquer technique.
8/12/2019 Sorting Insertion Sort Merge Sort
21/45
8/12/2019 Sorting Insertion Sort Merge Sort
22/45
Mergesort
Merge-sort is based on an algorithmic design pattern called div ide-and-
conquer .
The divide-and-conquer pattern consists of the following three steps:
1. Divide: If the in pu t size is smaller than a certain thresho ld (say, one
or two elements), solve the problem directly using a straightforward method
and return the solution obtained. Otherwise, divide the input data into two or more disjoint subsets.
2. Recur: Recursively solve the subp roblems associated with the
subsets.
3. Conquer : Take the solut ions to the subprob lems and merge them
into a solution to the original problem.
8/12/2019 Sorting Insertion Sort Merge Sort
23/45
Mergesort Con
To sort a sequence S with n elements using the three divide-and-
conquer
steps, the merge-sort algorithm proceeds as follows:
1. Divide: If S has zero or one element, return S immediately; it is
alreadysorted. Otherwise (S has at least two elements), remove all the
elements
from S and put them into two sequences, S1 and S2, each
containing about
half of the elements of S; that is, S1 contains the first n/2elementsof S,
and S2 contains the remaining n/2elements.
2. Recur: Recursively sort sequences S1 and S2.
3. Conquer : Put back the elements into S by merging the sorted
sequences S1 and S2 into a sorted sequence.
8/12/2019 Sorting Insertion Sort Merge Sort
24/45
Mergesort Con
8/12/2019 Sorting Insertion Sort Merge Sort
25/45
Mergesort Con.
8/12/2019 Sorting Insertion Sort Merge Sort
26/45
Mergesort Con
Merging Operation
We have two sorted array and anotherempty array whose size is equal to the
sum of sizes of two sorted arrays.
8/12/2019 Sorting Insertion Sort Merge Sort
27/45
Merging Operation Con.
8/12/2019 Sorting Insertion Sort Merge Sort
28/45
Merging Operation con
8/12/2019 Sorting Insertion Sort Merge Sort
29/45
8/12/2019 Sorting Insertion Sort Merge Sort
30/45
Mergesort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
31/45
Mergesort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
32/45
Mergesort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
33/45
Mergesort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
34/45
Mergsort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
35/45
Mergsort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
36/45
Mergesort Con
Divide and Recur Step
8/12/2019 Sorting Insertion Sort Merge Sort
37/45
Mergesort Con
Food for Thought !
Proposition
The merge-sort tree associated with anexecution of mergesort on a sequence of
size n has height log n. Why???
8/12/2019 Sorting Insertion Sort Merge Sort
38/45
The Running Time for
Merging Let n1 and n2 be the number of elements of S1
and S2, respectively.
Algorithm merge has three while loops
The key observation is that during eachiteration of one of the loops, one element is
copied or moved from either S1 or S2 into S
(and that element is no longer considered).
Since no insertions are performed into S1 or S2,
this observation implies that the overall number
of iterations of the three loops is n1 +n2. Thus,
the running time of algorithm merge is O(n1+n2).
8/12/2019 Sorting Insertion Sort Merge Sort
39/45
The Running Time of Merging
con.
8/12/2019 Sorting Insertion Sort Merge Sort
40/45
Th R i Ti f M
8/12/2019 Sorting Insertion Sort Merge Sort
41/45
The Running Time of Merge-
Sort con. let us analyze the running time of the
entire merge-sort algorithm, assuming it is
given an input sequence of n elements.
For simplicity, we restrict our attention to
the case where n is a power of 2.
we analyze the merge-sort algorithm by
referring to the merge-sort tree T. (RecallFigures 11.2 through 11.4.)
Th R i Ti f M
8/12/2019 Sorting Insertion Sort Merge Sort
42/45
The Running Time of Merge-
Sort cont We call the t ime spent at a node v o f T
the runn ing t ime of the recu rsive cal l
assoc iated w ith v, exc lud ing the time
taken waiting for the recursive callsassociated with the children of v to
terminate. In other words, the time spent
at node v includes the running times of thedivide and conquer steps, but excludes the
running time of the recur step.
Th R i Ti f M
8/12/2019 Sorting Insertion Sort Merge Sort
43/45
The Running Time of Merge-
Sort cont. the conquer step, which consists of
merging two sorted subsequences, also
takes linear time, independent of whether
we are dealing with arrays or linked lists.That is, letting i denote the depth of node
v, the time spent at node v is O(n/2^i),
since the size of the sequence handled bythe recursive call associated with v is
equal to n/2^i.
Th R i Ti f M
8/12/2019 Sorting Insertion Sort Merge Sort
44/45
The Running Time of Merge-
Sort con.. Given our definition of time spent at a node,
the running time of merge-sort is equal to the
sum of the times spent at the nodes of T.
Observe that T has exactly 2^i nodes at depth i.This simple observation has an important
consequence, for it implies that the overall time
spent at all the nodes of T at depth i is O(2^i
n/2^i), which is O(n). By Proposition 11.1(consult GTM Chapter 11, topic 11.1), the height
of T is logn. Thus, since the time spent at each
of the log n+1 levels of T is O(n), we have the
following result.
Th R i Ti f M
8/12/2019 Sorting Insertion Sort Merge Sort
45/45
The Running Time of Merge-
Sort con.Algorithm merge-sort sorts a sequence S
of size n in O(nlogn) time, assuming two
elements of S can be compared in O(1)
time
Top Related