Sorting and searching · Searching algorithms Sorting algorithms Merge Sort Radboud University...

IntroductionSearching algorithms

Sorting algorithmsMerge Sort

Radboud University Nijmegen

Sorting and searching

Alexandra Silva

[email protected]

http://www.cs.ru.nl/~alexandra

Institute for Computing and Information SciencesRadboud University Nijmegen

13 February 2013

Alexandra 13 February 2013 Lesson 2 1 / 31

http://www.cs.ru.nl/~




Recap

Last week’s message

• There are different types of functions

• They have very different growths

• Different algorithms are associated with different functions

• Why analyzing time complexity is important

• Which type of abstractions are done in the analysis





Example I : Linear search in a vector

int linsearch(int *v, int a, int b, int k)

int i;

i=a;

while ((i<=b) && (v[i]!=k))

i++;

if (i>b)

return -1;

else return i;

Cost

c1c2c3c4c5c5

repetitions

1m+1m111

m is the number of time i++ executes.Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.







int i;

i=a;

while ((i<=b) && (v[i]!=k))

i++;

if (i>b)

return -1;

else return i;

Cost

c1c2c3c4c5c5

repetitions

1m+1m111

m is the number of time i++ executes.

Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.







int i;

i=a;

while ((i<=b) && (v[i]!=k))

i++;

if (i>b)

return -1;

else return i;

Cost

c1c2c3c4c5c5

repetitions

1m+1m111

m is the number of time i++ executes.Obs1: These value depends on the while loop condition being true– the position i is in-between a and b, that is: 0 ≤ m ≤ b − a + 1.





Total Execution Time

T (n) = c1 + c2(m + 1) + c3m + c4 + c5

For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why? Depends on when the element is found.

Best case scenario: The value k is in the first position (k =

v[a]). Then:T (n) = (c1 + c2 + c4 + c5)

T(n) is constant

Worst case scenario: the value in not found.

T (n) = c1+c2(n+1)+c3(n)+c4+c5 = (c2+c3)n+(c1+c2+c4+c5)

T(n) is linear






T (n) = c1 + c2(m + 1) + c3m + c4 + c5

For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why?

Depends on when the element is found.


v[a]). Then:T (n) = (c1 + c2 + c4 + c5)

T(n) is constant



T(n) is linear






T (n) = c1 + c2(m + 1) + c3m + c4 + c5

For a given n (the size of the input to search, n = b − a + 1), thevalue T (n) can vary: why? Depends on when the element is found.


v[a]). Then:T (n) = (c1 + c2 + c4 + c5)

T(n) is constant



T(n) is linear





Example II : search for duplicates

void dup(int *v,int a,int b)

int i,j;

for (i=a;i<b;i++)

for (j=i+1;j<=b;j++)

if (v[i]==v[j])

printf("%d equals to %d\n",i,j);

Cost

c1c2c3c4

repetitions

NS1S2S2

N = b− a + 1 (input size); S1 =∑b−1

i=a (ni + 1); S2 =∑b−1

i=a ni andni = b − i number of times the inner for executes, for each i .





Total time of execution

T (N) = c1N + c2S1 + c3S2 + c4S2

In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:

S2 =N−1∑i=1

N − i = (N − 1)N − (N−1)N2 = 1

2N2 − 1

2N

S1 =N−1∑i=1

(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1

2N2 − 1

2N − 1

T (N) is a quadratic function: T (N) = k2N2 + k1N + k0.






T (N) = c1N + c2S1 + c3S2 + c4S2

In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.

Let us take a = 1 and b = N. Then:

S2 =N−1∑i=1

N − i = (N − 1)N − (N−1)N2 = 1

2N2 − 1

2N

S1 =N−1∑i=1

(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1

2N2 − 1

2N − 1







T (N) = c1N + c2S1 + c3S2 + c4S2

In this algorithm the best and worst case coincide: no matter whatthe size of the input, the for cycles are executed the same numberof times.Let us take a = 1 and b = N. Then:

S2 =N−1∑i=1

N − i = (N − 1)N − (N−1)N2 = 1

2N2 − 1

2N

S1 =N−1∑i=1

(N − i + 1) = (N − 1)(N + 1)− (N−1)N2 = 1

2N2 − 1

2N − 1






Some remarks

We simplified the analysis:

• we used abstract costs ci instead of concrete times

We simplify even further: for a large enough N the term with N2 isso much larger than the other terms that we can say that theexecution time of dup is approximated by

T (N) = kN2

And in fact k is typically also not relevant. So, using the notationfrom last time, we can say that the execution time is

Θ(N2)





Some further remarks

Analisys of worst case is useful:

• it’s a guarantee: upper limit for any input

• it occurs often (more than one would wish; e.g. search ofinexistent information)

• often the average case is very close to the worst case

Analysis of the average case involves (advanced) probabilistic studyof the dimension and type of input (e.g. what is the probability avector is almost sorted?; what is the average size? )

Simplifying further: we will in general assume that, for a givendimension, all inputs occur with the same probability.





Intermezzo

We can classify algorithms in terms of the strategy they use tosolve a problem:

• Incremental – (e.g insertion sort, which we will see next).

• Divide and conquer – (e.g mergesort and quicksort).

• Greedy – (e.g Minimum Spanning Tree).

• Dinamyc programming – (e.g All-Pairs-Shortest- Path).

• Randomized algorithms – (e.g randomized quicksort).





Case study I: Insertion sort

Problem: You’re given a list of numbers (integers) and you wantto order them. We want to do this on a list given in an array whichis to be re-ordered ( 6= construct a new list).

• look at the cards

• pick the lower one of a club;put it to the left; search forthe next card of that cluband continue until you haveall the cards inside each clubordered.





Case study I: Insertion sort –example





Case study I: Insertion sort – algorithm

void insertion_sort(int A[])

for (j=2 ; j<=N ; j++)

key = A[j]; // store the current elm in a tmp record

i = j-1;

while (i>0 && A[i] > key)

/* search for the position where A[j] belongs */

A[i+1] = A[i]; /* shift the rest of the greater

elements down */

i--;

A[i+1] = key; // place the element





Case study I: Insertion sort – analysis

Execution time will depend on several things.

• dimension of the list

• how ordered it is already

But let’s again take it a step at a time . . .





Case study I: Insertion sort – analysis

void insertion_sort(int A[])

for (j=2 ; j<=N ; j++)

key = A[j];

i = j-1;

while (i>0 && A[i] > key)

A[i+1] = A[i];

i--;

A[i+1] = key;

Cost

c1c2c3c4c5c6

c7

repetitions

NN − 1N − 1S1S2S2

N − 1

where S1 =∑N

j=2 nj ; S2 =∑N

j=2(nj − 1), where nj is the numberof times the while is executed, for each value of j





Total execution time

T (N) = c1N+c2(N−1)+c3(N−1)+c4S1+c5S2+c6S2+c7(N−1)

Depending on the input the time can vary.

Best case:

The vector is already sorted

nj = 1 for j = 2, . . . ,N; Hence S1 = N − 1 and S2 = 0;

Thus:

T (N) = (c1 + c2 + c3 + c4 + c7)N − (c2 + c3 + c4 + c7)

T (N) linear: in the best case we have an execution time in Θ(N).






T (N) = c1N+c2(N−1)+c3(N−1)+c4S1+c5S2+c6S2+c7(N−1)

Depending on the input the time can vary.

Best case: The vector is already sorted

nj = 1 for j = 2, . . . ,N; Hence S1 = N − 1 and S2 = 0;

Thus:

T (N) = (c1 + c2 + c3 + c4 + c7)N − (c2 + c3 + c4 + c7)

T (N) linear: in the best case we have an execution time in Θ(N).






Worst case:

The vector is ordered backwards (from the greatestto the smallest element.)

nj = j for j = 2, . . . ,N; Hence

S1 =N∑j=2

j =N(N + 1)

2− 1 and S2 =

N∑j=2

(j − 1) =N(N − 1)

2;

Thus:

T (N) = 12(c4 + c5 + c6)N2 + (c1 + c2 + c3 + .5c4 − .5c5 − .5c6 + c7)N−(c2 + c3 + c4 + c7)

T (N) quadratic: worst case has an execution time in Θ(N2).We can say the time of execution is in Ω(N) and O(N2).






Worst case: The vector is ordered backwards (from the greatestto the smallest element.)

nj = j for j = 2, . . . ,N; Hence

S1 =N∑j=2

j =N(N + 1)

2− 1 and S2 =

N∑j=2

(j − 1) =N(N − 1)

2;

Thus:









nj = j for j = 2, . . . ,N; Hence

S1 =N∑j=2

j =N(N + 1)

2− 1 and S2 =

N∑j=2

(j − 1) =N(N − 1)

2;

Thus:


T (N) quadratic: worst case has an execution time in Θ(N2).

We can say the time of execution is in Ω(N) and O(N2).







nj = j for j = 2, . . . ,N; Hence

S1 =N∑j=2

j =N(N + 1)

2− 1 and S2 =

N∑j=2

(j − 1) =N(N − 1)

2;

Thus:







More algorithms!

• Up to here: how to analyze the execution time of anon-recursive algorithm.

• Now: Divide and conquer – (mergesort and quicksort).

1 algorithms2 solving recurrence relations

• Randomized algorithms – (randomized quicksort).





Dive and Conquer

Divide and conquer strategy

• Divide the problem into n sub-problems (smaller instances ofthe original problem)

• Conquer the sub-problems• trivial for small sizes• use the same strategy, otherwise

• Combine the solutions of the sub-problems into the solution ofthe original problem

Implementation is typically recursive (why?)





Merge sort

It uses a divide and conquer strategy

• Divide the vector in two vectors of similar size

• Conquer: recursively order the two vectors (trivialfor. . .

vectors of size 1)

• Combine two ordered vectors into a new ordered vector.Auxiliary merge function.

1 The merge function gets as input two ordered sequencesA[p...q] and A[q+1...r]

2 In the end of the execution we get the sequence A[p...r]

ordered.





Merge sort

It uses a divide and conquer strategy

• Divide the vector in two vectors of similar size

• Conquer: recursively order the two vectors (trivialfor. . . vectors of size 1)

• Combine two ordered vectors into a new ordered vector.Auxiliary merge function.

1 The merge function gets as input two ordered sequencesA[p...q] and A[q+1...r]

2 In the end of the execution we get the sequence A[p...r]

ordered.





Mergesort

void merge_sort(int A[], int p, int r)

if (p < r) /* base case ? */

q = (p+r)/2; /* Divide */

merge_sort(A,p,q); /* Conquer */

merge_sort(A,q+1,r); /* Conquer */

merge(A,p,q,r); /* Combine */

Question: What is the dimension of each sequence in the dividestep? q = b(p + r)/2c

Initial call on an array with n elements: merge sort(A,1,n).





Mergesort



q = (p+r)/2; /* Divide */




Question: What is the dimension of each sequence in the dividestep?

q = b(p + r)/2c






Mergesort



q = (p+r)/2; /* Divide */




Question: What is the dimension of each sequence in the dividestep? q = b(p + r)/2c






Example I





Example II

Array with number of elements not a multiple of 2





Remains to do. . .

merge procedure

Input: Array A and indices p, q, r such that

• p ≤ q < r .

• Subarray A[p..q] is sorted and subarray A[q + 1..r ] is sorted.By the restrictions on p, q, r , neither subarray is empty.

Output: The two subarrays are merged into a single sortedsubarray in A[p..r ].





Merge

Idea behind merging: Think of two piles of cards.Each pile is sorted and placed face-up on a table with the smallestcards on top. We will merge these into a single sorted pile,face-down on the table. A basic step:

• Choose the smaller of the two top cards.

• Remove it from its pile, thereby exposing a new top card.

• Place the chosen card face-down onto the output pile.

• Repeatedly perform basic steps until one input pile is empty.Once one input pile empties, just take the remaining input pileand place it face-down onto the output pile.

What is the complexity of this algorithm?





Merge

• Each basic step should take constant time, since we check justthe two top cards.

• There are ≤ n basic steps, since each basic step removes onecard from the input piles, and we started with n cards in theinput piles.

• Therefore, this procedure should take Θ(n) time.





The merge procedurevoid merge(int A[], int p, int q, int r)

int L[MAX], R[MAX];

int n1 = q-p+1; /* length of array A[p..q] */

int n2 = r-q; /* length of array A[q+1..r] */

for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */

for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */

L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */

what are the sentinels good for?

i = 1; j = 1;

for (k=p ; k<=r ; k++)

if (L[i] <= R[j]) /* put the next smallest

A[k] = L[i]; i++; element in the array */

else

A[k] = R[j]; j++;

merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.





The merge procedurevoid merge(int A[], int p, int q, int r)

int L[MAX], R[MAX];

int n1 = q-p+1; /* length of array A[p..q] */

int n2 = r-q; /* length of array A[q+1..r] */

for (i=1 ; i<=n1 ; i++) L[i] = A[p+i-1]; /* array LEFT */

for (j=1 ; j<=n2 ; j++) R[j] = A[q+j]; /* array RIGHT */

L[n1+1] = MAXINT; R[n2+1] = MAXINT; /* sentinels */

what are the sentinels good for?

i = 1; j = 1;

for (k=p ; k<=r ; k++)

if (L[i] <= R[j]) /* put the next smallest

A[k] = L[i]; i++; element in the array */

else

A[k] = R[j]; j++;

merge executes in Θ(n) time, where n = r − p + 1 = the numberof elements being merged.Alexandra 13 February 2013 Lesson 2 26 / 31




Example

merge(A,9,12,16)





Example c’d

merge(A,9,12,16)





Complexity analysis

Algorithm with a recursive call: running time can usually bedescribed as a recurrence.

How does that work?

Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.

T (n): time of execution (in the worst case) for a input of size n;

If n = 1 then T (n) is constant: T (n) ∈ Θ(1).





Complexity analysis

Algorithm with a recursive call: running time can usually bedescribed as a recurrence. How does that work?

Let’s for a moment assume that the size of the input is a power of2. In each division step the sub-arrays have exactly size n/2.

T (n): time of execution (in the worst case) for a input of size n;

If n = 1 then T (n) is constant: T (n) ∈ Θ(1).





Complexity analysis

Otherwise:

1 Divide computing the middle of the vector is done in constanttime: Θ(1).

2 Conquer two problems of size n/2 are solved: 2T (n/2).

3 Compose the merge function executes in linear time: Θ(n)

Hence:

T (n) =

Θ(1) if n = 1

Θ(1) + 2T (n/2) + Θ(n) if n > 1

and now we want to conclude that T (n) ∈ Θ(??).





Merge vs Insertion sort

Running ahead: we will show that T (n) ∈ Θ(n lg n).

Cf. Insertion sort in Θ(n2).


Sorting and searching · Searching algorithms Sorting algorithms Merge Sort Radboud University...

Documents

Transcript of Sorting and searching · Searching algorithms Sorting algorithms Merge Sort Radboud University...