1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity...
-
Upload
brendan-joseph -
Category
Documents
-
view
224 -
download
0
Transcript of 1 Sorting Algorithms (Basic) Search Algorithms BinaryInterpolation Big-O Notation Complexity...
1
Sorting Algorithms (Basic)Search Algorithms
BinaryInterpolation
Big-O NotationComplexity
Sorting, Searching, Recursion Intro to Algorithms
Selection Sortfor int’sfor string’s
Templated Functions
Recursion
2
Sorting Elementary sorting algorithms
– Bubble– Insertion*– Selection
3
Bubble Sort
const size_t N = <constant>;int A[N];// ‘A’ then is populated // Sort followsfor (size_t i = N - 1; i >= 1; --i) for (size_t j = 0; j < i; ++j) if (A[j] > A[j + 1]) std::swap (A[j], A[j + 1]);
Complexity?
4
Insertion Sort
for (i = 1; i < N; ++i){ // Invariant: A[0..i) sorted v = A[i]; // Element to place j = i; // Index to place ‘v’ while (j >= 1 && v < A[j–1]) { A[j] = A[j–1]; --j; } // Invariant: v >= A[j-1] or j == 0
A[j] = v;}
5
Selection Sort
// Find N-1 smallest and placefor (i = 0; i < N - 1; ++i) { min = i; for (j = i+1; j < N; ++j) if (A[j] < A[min])
min = j; //if (i != min) swap (A[i], A[min]);}
6
Searching Sequential
– Examine 1st element, 2nd, …– Assume int A[N];– int* pos = std::find (A, A+N, val);
Binary– Requires sequence be sorted– More efficient – quantify– int* pos = lower_bound (A, A+N, val);
// *pos >= val or pos @ end– bool found = binary_search (A, A+N, val);
Interpolation– Sorted sequence– O(lg lg (N)) average
7
Binary Search If data are sorted can use Binary Search Look at middle element
– If match, return location– If smaller, search left sub-array– If larger, search right sub-array
8
Binary Search
Case 1: Matchif (target == midValue)
return mid;
m idfirs t
target
C as e 1: target = m id valueS earc h is d o ne
las t-1 las t
9
Binary Search
Case 2: Target smaller than middle value// Search the left sublist
if (target < midValue)
<reposition last to mid><search
sublist arr[first]…arr[mid-1]
las t-1firs t
target
C as e 2: target < m id valueS earc h lo w er s ub lis t
m id -1 las t
10
Binary Search
Case 3: Target larger than middle value// Search upper sublist if (target > midValue)
<reposition first to mid+1><search sublist arr[mid+1]…arr[last-
1]>
C as e 3: target > m id valueS earc h up p er s ub lis t
las t-1new firs t = m id + 1
firs t
target
las t
11
Illustrating the Binary Search- Successful Search
1. Search for target = 23Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.
Since target = 23 > midvalue = 12, step 2 searches the upper sublist with first = 5 and last = 9.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
12
Illustrating the Binary Search- Successful Search
Step 2:Indices first = 5, last = 9, mid = (5+9)/2 = 7.
Since target = 23 < midvalue = 33, step 3 searches the lower sublist with first = 5 and last = 7.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
13
Illustrating the Binary Search- Successful Search
Step 3: Indices first = 5, last = 7, mid = (5+7)/2 = 6.
Since target = midvalue = 23, a match is found at index mid = 6.
m id
-7 3 5 8 12 16arr0 1 2 3 4 5
23 33 55
6 7 8 9
14
Illustrating the Binary Search- Unsuccessful Search
Search for target = 4. Step 1: Indices first = 0, last = 9, mid = (0+9)/2 = 4.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
Since target = 4 < midvalue = 12, step 2 searches the lower sublist with first = 0 and last = 4.
15
Illustrating the Binary Search- Unsuccessful Search
Step 2: Indices first = 0, last = 4, mid = (0+4)/2 = 2.
Since target = 4 < midvalue = 5, step 3 searches the lower sublist with first = 0 and last 2.
m id
-7 3 5 8 12 16arr0 1 2 3 4 5
23 33 556 7 8
16
Illustrating the Binary Search- Unsuccessful Search
Step 3: Indices first = 0, last = 2, mid = (0+2)/2 = 1.
Since target = 4 > midvalue = 3, step 4 should search the upper sublist with first = 2 and last =2. However, since first >= last, the target is not in the list and we return index -1.
m id
-7 3 5 8 12 16arr
0 1 2 3 4 5
23 33 55
6 7 8 9
17
Binary Search Algorithm
int binSearch (const int A[],
int first, int last,
int target)
{int midIndex;int midValue;
18
Binary Search Algorithm
// Test for nonempty sublistwhile (first < last)
{midIndex = (first+last)/2;midValue = A[mid];if (target == midValue)
return midIndex;// Determine which sublist to
// search
19
Binary Search Algorithm
else if (target < midValue)last = midIndex;
elsefirst = midIndex + 1;
}return -1;
}// Note half-open range convention
20
Binary Search (Observation) midIndex = first + 1 * (last – first) / 2; Implicit assumption? Can we do better?
21
Interpolation Searchint iSearch (const vector <int>& A, int v) {
int loc, s = 0, e = A.size () - 1;
if (v < A[s]) return (-1);
if (v >= A[e]) s = e;
while (s < e) {
loc = s + (e - s) * (v – A[s]) / (A[e] – A[s]);
if (v > A[loc]) { s = loc + 1; }
else if (v < A[loc]) { e = loc - 1; }
else return (loc);
}
if (v != A[s]) s = -1;
return (s);
}
22
Algorithm Complexity Big-O notation – machine-independent means
for expressing efficiency of an algorithm Count key operations or stmts, and relate this
count to problem size using growth fn. f(N) = O(g(N)) if there are positive constants c
and n0 such that f(N) <= c * g(N) when N >= n0
Express op count as function of input size– f1(N) = 3N2 + 50N + 120– f2(N) = 1000 lg(N) + 50– f3(N) = 5
f1(N) = O(N2); n0 = 1, c = 200 3N2 + 50N + 120 <= 200N2, for N >= n0
f3(N) = O(1); n0 = 1, c = 6
23
Big-O Notation
24
Constant Time Algorithms
An algorithm is O(1) (constant time) when its running time is independent of input size
push_back on vector involves a simple assignment statement complexity O(1) (if capacity sufficient)
fro nt rear
D irec t In se r t a t R ear
25
Linear Time AlgorithmsAn algorithm is O(N) when its running time is proportional to input size
S equential S earch fo r the Minim um E lem ent in an A rray
32 46 8 12 3
m in im u m elem en t fou n d in th e list a fter n com p a rison s
n = 51 2 3 4 5
26
Polynomial AlgorithmsAlgorithms with running time O(N2) are
quadratic– practical only for relatively small values of N;
whenever N triples, the running time increases by ?
Algorithms with running time O(N3) are cubic– efficiency is generally poor; tripling N
increases the running time by ?
27
Algorithm Complexity Chart
n log2n n log2n n2 n3 2n 2 1 2 4 8 4 4 2 8 16 64 16 8 3 24 64 512 256 16 4 64 256 4096 65536 32 5 160 1024 32768 4294967296 128 7 896 16384 2097152 3.4 x 1038
1024 10 10240 1048576 1073741824 1.8 x 10308
65536 16 1048576 4294967296 2.8 x 1014 Forget it!
28
Logarithmic Time AlgorithmsThe logarithm of N, base 2, is commonly used when analyzing computer algorithmsEx. log2(2) = lg (2) = 1
log2(75) = lg (75) ~= 6.2288
When compared to functions N and N2, the function lg N grows very slowly
nn 2
lo g 2n
29
Function Templates Same logic is appropriate for different types
– Templatize function Example: selection sort logic identical for
floats, ints, strings, etc.
30
Selection Sort AlgorithmInteger Version
void selectionSort (int A[], int n) {
. . .int temp;
// int temp used for the exchangefor (pass = 0; pass < n-1; ++pass)
{. . .
if (A[j] < A[smallIndex])
// Compare integer elements . . .
}}
31
Selection Sort AlgorithmString Version
void selectionSort (string A[], int n) {
. . .string temp;
// string temp used for the exchange
for (pass = 0; pass < n-1; ++pass) {
. . .if (A[j] <
A[smallIndex])// compare string elements
. . .
}
}
32
Template Syntax Include keyword template followed list of formal
types enclosed in angle brackets In the argument list, each type is preceded by
the keyword typename (or class)
// Argument list with multiple template
// types template <typename T, typename U,
typename V, ...>
33
Template Syntax Example
template <typename T>void selectionSort (T A[], int n){ int smallIndex;
// index of smallest element in the // sublist
int pass, j;T temp;
34
Template Syntax Example
// pass has the range 0 to n-2for (pass = 0; pass < n-1; +
+pass){
// scan the sublist starting at // index pass
smallIndex = pass;
// j traverses the sublist // A[pass+1] to A[n-1]
for (j = pass+1; j < n; ++j) // update if smaller element found
35
Template Syntax Example
if (A[j] < A[smallIndex])
smallIndex = j;// if smallIndex and
pass are not // the same location, exchange the
// smallest item in the sublist with // A[pass]
if (smallIndex != pass)
swap (A[pass],
A[smallIndex]);}
}
36
Recursion Must ensure
– Recursive calls work on smaller problem– Eventually reach base case
Many problems have naturally recursive solutions (difficult to express iteratively)
37
Recursion Examples Power function Binary search Towers of Hanoi Fibonacci numbers (*)
38
Recursive Definition of the Power Function
Recursive definition– exponent n = 0 (base case)– n 1 which assumes we already know xn-1
Compute successive powers of x by multiplying the previous value by x
1,*
0,11 nxx
nx n
n
39
Implementing the Recursive Power Function
Recursive power:double power (double x, int n)
// n is a non-negative integer{
if (n == 0)return 1.0; // base case
elsereturn x * power (x, n-1);
// recursive step} // runtime? how to improve?
Solving Recurrence T(N) = T(N-1) + 1; T(0) = 0 Solve:
– T(N) = T(N-1) + 1– T(N-1) = T(N-2) + 1– T(N-2) = T(N-3) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = N = O(N)– (not good enough)
40
41
Binary Search// Search A [s, e)int bs (int A[], int s, int e, int v){ if (s >= e) return -1; int m = (s + e) / 2; if (v == A[m]) return m; if (v < A[m]) return bs (A, s, m, v); else return bs (A, m+1, e, v);}
Improvement
Reformulate definition to cut problem in half each time (?)
Suppose N = 2k for k >= 0 T(N) = T(N/2) + 1; T(0) = 0 Solve:
– T(N) = T(N/2) + 1– T(N/2) = T(N/4) + 1– T(N/4) = T(N/8) + 1– …– T(1) = T(0) + 1– T(0) = 0– :: T(N) = lg(N) + N = O(lg(N))
42
43
Solving the Tower of Hanoi Puzzle using Recursion
N eed le A
. . . . . . . .
N eed le CN eed le B N eed le C
. . . . . . . .
N eed le BN eed le A
Move N-1 discs from A to B (using B as temp)Move disc N from A to CMove N-1 discs from B to C (using A as temp)
Recurrence?
44
Fibonacci Numbers using Iteration
int fibIter (int n){ // Don’t use recursion!
if (n == 0 || n == 1)return n;
// Store previous two Fib #s int oneBack = 0, twoBack = 1,
current;
45
Fibonacci Numbers using Iteration
// Compute successive terms
for (int i = 2; i <= n; ++i) {
current = oneBack + twoBack;twoBack = oneBack;
oneBack = current; }
return current;}// Counting adds: N >= 2// T(N) = N-1 = O(N)