ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165
-
Upload
julian-murphy -
Category
Documents
-
view
21 -
download
0
description
Transcript of ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165
![Page 1: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/1.jpg)
ICS 353Design and Analysis of Algorithms
Section 01 – 09:20-10:20am – 24:165
Summer Semester 2005 - 2006 (053)
King Fahd University of Petroleum & Minerals
Information & Computer Science Department
Summer Semester 2005 - 2006 (053)
King Fahd University of Petroleum & Minerals
Information & Computer Science Department
![Page 2: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/2.jpg)
• Instructor: Dr. Wasfi Al-Khatib الخطيب وصفي• Office: (22) 133-1
• Office hours: – SMW 08:10 – 09:10 am. – UT 10:20 – 11:20 am.
• Phone: 1715
• email: [email protected]
Important Preliminaries
![Page 3: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/3.jpg)
Important Preliminaries (Cont.)
• Prerequisite: ICS 202: Data Structures
• Prerequisites by Topic: – Data structures including linked lists, arrays,
stacks, queues, trees, and graphs, and– Knowledge of a high level structured language.
![Page 4: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/4.jpg)
Text Book
Introduction to Algorithms: Design Techniques and analysis
By
M. Alsuwaiyel.
![Page 5: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/5.jpg)
Course Goals
• To provide the students with the following:
– The fundamentals of algorithms and algorithmic techniques,
– The ability to decide on the suitability of a specific technique for a given problem,
– The ability to analyze the complexity of a given algorithm,
– The ability to design efficient algorithms for new situations, using as building blocks the techniques learned, and
– Introducing the concept of NP-complete problems.
![Page 6: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/6.jpg)
Course Contents
• Basic Concepts in Algorithmic Analysis, Chapter1• Mathematical preliminaries, Chapter 2 + • Review of Data Structures, Chapter 3 • Advanced Data Structures, Chapter 4 • Induction, Chapter 5 • Divide and Conquer, Chapter 6• Dynamic Programming, Chapter 7• Greedy Algorithms, Chapter 8• Graph Traversal, Chapter 9• NP- Complete Problems, Chapter 10
![Page 7: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/7.jpg)
Grading Policy
Homeworks 10%
Quizzes 15%
Pop Quizzes 10%
Two Major Exams 30%
Final Exam 35%
Absences [-4,2]%
![Page 8: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/8.jpg)
Important Dates
Task Date [and Time] Location Weight
Quiz 1 Saturday July 1, 2006 In class 3%
Quiz 2 Saturday July 8, 2006 In class 3%
Help Session Friday July 14, 2006 4:00-5:00pm TBA N/A
Major Exam I Saturday July 15, 2006 7:00-8:30pm In class 15%
Quiz 3 Saturday July 22, 2006 In class 3%
Quiz 4 Saturday July 29, 2006 In class 3%
Help Session Friday August 4, 2006 7:00-8:30pm TBA N/A
Major Exam II Saturday August 5, 2006 7:00-8:30pm In class 15%
Quiz 5 Saturday August 12, 2006 In class 3%
Help Session Tuesday August 15, 2006 TBA TBA N/A
Final Exam Wednesday August 16, 2006 7:00-9:00pm TBA 35%
![Page 9: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/9.jpg)
Final Important Remarks• Homeworks:
– Due at the beginning of class.– Late homeworks are not accepted.– Discussing homeworks with others (especially on WebCT) is
highly encouraged. However, copying homeworks is NOT permitted and will be considered CHEATING.
– Homeworks are crucial for learning and obtaining good grades.• Quizzes: 15 minute. Each covers material given since the
last quiz or major exam.• Pop Quizzes: 5-10 minute. Each covers material given
during the same lecture.• Exams HWs, & quizzes are generally CHALLENGING.• MAKE USE of my office hours.
![Page 10: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/10.jpg)
Attendance• Attendance will be checked each class. • A bonus of 0.5 point (out of the maximum 100) will be
given for each “additional” hour attended above the 38 hours– This shall come from the help sessions conducted for the two
major exams and the final exam.• Unexcused Absences Policies:
– The first three absences are FREE of charge.– The fourth absence is worth – 2 of your maximum 100 total. – Each subsequent absence, up to the seventh absence, is worth
– 0.5.– The eighth absence will result in an automatic DN grade.– An unexcused absence can become an excused absence
ONLY by an official letter from the Dean of Student’s office.
![Page 11: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/11.jpg)
Your 24-Hour Right
• One has 24 hours to object to the grade of a homework, [pop] quiz or a major exam starting from the end of the class time in which the graded exam papers have been distributed.– i.e. if you were not present in class during the
distribution of exam papers, you loose this right.
• If for some reason you cannot see me in person, within this period, send me an email requesting an appointment. The email, though, should be sent within the 24-hour time period.
![Page 12: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/12.jpg)
Cairo Envelopes
How much can they increase your letter grade?
NONE…………..even if you bring an elder!
Examples: – Slept after Fajr– Checkpoints long lines– I got married / I had a divorce– I am graduating and the job is waiting for me
![Page 13: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/13.jpg)
So, What Can Increase my Grade?
• Active participation– In Class– On WebCT– In my office hours
• The bottom line is: Trying
![Page 14: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/14.jpg)
Cheating Policies
• Cheating in one category will result in a zero for the whole category. For example, if cheating is confirmed in homework 3, this will result in 0 out of 10% [which means that there is no need to submit any additional homework].
• An additional 5% will be deducted from the overall total (out of 100).
![Page 15: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/15.jpg)
What is an algorithm?
• An algorithm is defined as a finite set of steps, each of which may require one or more operations and if carried out on a set of inputs, will produce one or more outputs after a finite amount of time.
• Examples of Algorithms• Examples of computations that are not
algorithms
![Page 16: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/16.jpg)
Properties of Algorithms
• Definiteness: It must be clear what should be done.• Effectiveness: Each step must be such that it can, at
least in principle, be carried out by a person using pencil and paper in a finite amount of time. E.g. integer arithmetic.
• An algorithm produces one or more outputs and may have zero or more externally supplied inputs.
• Finiteness: Algorithms should terminate after a finite number of operations.
![Page 17: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/17.jpg)
Our Objective
• Find the most efficient algorithm for solving a particular problem.
• In order to achieve the objective, we need to determine:– How can we find such algorithm?– What does it mean to be an efficient algorithm?– How can one tell that it is more efficient than
other algorithms?
![Page 18: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/18.jpg)
In the First Chapter
• We will answer the following two questions– What does it mean to be an efficient algorithm?– How can one tell that it is more efficient than
other algorithms?
based on some easy-to-understand searching and sorting algorithms that we may have seen earlier.
![Page 19: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/19.jpg)
Searching Problem
• Assume A is an array with n elements A[1], A[2], … A[n]. For a given element x, we must determine whether there is an index j; 1 ≤ j ≤ n, such that x = A[j]
• Two algorithms, among others, address this problem– Linear Search– Binary Search
![Page 20: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/20.jpg)
Linear Search Algorithm
Algorithm: LINEARSEARCH
Input: array A[1..n] of n elements and an element x.
Output: j if x = A[j], 1 ≤ j ≤ n, and 0 otherwise.1. j 12. while (j < n) and (x A[j])3. j j + 14. end while
5. if x = A[j] then return j else return 0
![Page 21: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/21.jpg)
Analyzing Linear Search
• One way to measure efficiency is to count how many statements get executed before the algorithm terminates
• One should keep an eye, though, on statements that are executed “repeatedly”.
• What will be the number of “element” comparisons if x– First appears in the first element of A– First appears in the middle element of A– First appears in the last element of A– Doesn’t appear in A.
![Page 22: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/22.jpg)
Binary Search
• We can do “better” than linear search if we knew that the elements of A are sorted, say in non-decreasing order.
• The idea is that you can compare x to the middle element of A, say A[middle].– If x < A[middle] then you know that x cannot be an
element from A[middle+1], A[middle+2], …A[n]. Why?
– If x > A[middle] then you know that x cannot be an element from A[1], A[2], …A[middle-1]. Why?
![Page 23: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/23.jpg)
Binary Search AlgorithmAlgorithm: BINARYSEARCH
Input: An array A[1..n] of n elements sorted in nondecreasing order and an element x.
Output: j if x = A[j], 1 ≤ j ≤ n, and 0 otherwise.1. low 1; high n; j 02. while (low ≤ high) and (j = 0)
3. mid (low + high)/24. if x = A[mid] then j mid5. else if x < A[mid] then high mid - 1
6. else low mid + 17. end while
8. return j
![Page 24: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/24.jpg)
Worst Case Analysis of Binary Search• What to do: Find the maximum number of element
comparisons• How to do:
– The number of “element” comparisons is equal to the number of iterations of the while loop in steps 2-7. HOW?
– How many elements of the input do we have in the• First iteration
• Second iteration
• Thrid iteration
• …
• ith iteration
– The last iteration occurs when the size of input we have =
![Page 25: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/25.jpg)
Theorem
• The number of comparisons performed by Algorithm BINARYSEARCH on a sorted array of size n is at most 1log n
![Page 26: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/26.jpg)
Merging Two Sorted Lists
• Problem Description: Given two lists (arrays) that are sorted in non-decreasing order, we need to merge them into one list sorted in non-decreasing order.
• Example:
3 7 9 12 1 2 4 13 14
Input
Output
1 2 3 4 7 9 12 13 14
![Page 27: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/27.jpg)
Algorithm MERGE
Algorithm: MERGEInput: An array A[1..m] of elements and three
indices p, q and r, with 1 ≤ p ≤ q <r ≤ m, such that both the subarrays A[p..q] and A[q + 1..r] are sorted individually in nondecreasing order.
Output: A[p..r] contains the result of merging the two subarrays A[p..q] and A[q + 1..r].
Comment: B[p..r] is an auxiliary array.
![Page 28: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/28.jpg)
Algorithm MERGE (Cont.)
1. s ← p; t ← q + 1; k ← p2. while s ≤ q and t ≤ r3. if A[s] ≤ A[t] then4. B[k] ← A[s]5. s ← s + 16. else7. B[k] ←A[t]8. t ← t + 19. end if10. k ← k + 111. end while
12. if (s = q + 1) then B[k..r] ← A[t..r]
13. else B[k..r] ← A[s..q]
14. end if15. A[p..r] ← B[p..r]
![Page 29: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/29.jpg)
Analyzing MERGE
• Assuming arrays A[p,q] and A[q+1,r]– The least number of comparisons is
which occurs when
– The most number of comparisons is which occurs when
– The number of element assignments performed is
![Page 30: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/30.jpg)
Selection Sort
Algorithm: SELECTIONSORTInput: An array A[1..n] of n elements.Output: A[1..n] sorted in nondecreasing order.1. for i 1 to n - 12. k i3. for j i + 1 to n
{Find the index of the ith smallest element}
4. if A[j] < A[k] then k j5. end for6. if k i then interchange A[i] and A[k]7. end for
![Page 31: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/31.jpg)
Selection Sort Example
42 85 9
![Page 32: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/32.jpg)
Selection Sort Example
42 85 9
45 82 9
i k
1 2
54 82 92 5
94 82 53 5
94 82 54 4
![Page 33: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/33.jpg)
Analyzing Selection Sort
• We need to find the number of comparisons carried out in line #4:– For each iteration of the outer for loop, how many
times is line #4 executed?– Therefore, in total, line #4 is executed
• The number of element Interchanges (swaps):– Minimum:– Maximum: NOTE: The number of element assignments is 3
times the number of element interchanges
![Page 34: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/34.jpg)
Insertion Sort
Algorithm: INSERTIONSORTInput: An array A[1..n] of n elements.Output: A[1..n] sorted in nondecreasing order.1. for i 2 to n2. x A[i]3. j i - 14. while (j > 0) and (A[j] > x)5. A[j + 1] A[j]6. j j - 17. end while8. A[j + 1] x9. end for
![Page 35: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/35.jpg)
Insertion Sort Example
42 85 9
![Page 36: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/36.jpg)
Insertion Sort Example
42 85 9x=2
45 8 2 9x=9
45 8 2 9x=8
45 9 2 8x=4
94 8 2 5
![Page 37: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/37.jpg)
Analyzing Insertion Sort
• The minimum number of element comparisons is which occurs when
• The maximum number of element comparisons is which occurs when
• The number of element assignments is
![Page 38: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/38.jpg)
Bottom-Up Merge Sort
• Informally, the algorithm does the following– 1. Divide the array into pairs of elements (with
possibly single elements in case the number of elements is )
– 2. Merge each pair in non-decreasing order (with possibly a single “pair” left)
– 3. Repeat step 2 until there is only one “pair” left.
![Page 39: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/39.jpg)
Bottom-Up Merge Sort Example
42 318 1075 9 12 6
![Page 40: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/40.jpg)
Bottom-Up Merge Sort Example
5 2 9 8 4 12 7 1 3 6 10
52 98 4 12 71 3 6 10
52 98 4 1271 3 6 10
52 984 1271 3 6 10
52 984 1271 3 6 1042 318 1075 9 12 6
![Page 41: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/41.jpg)
Algorithm BOTTOMUPSORT
Algorithm: BOTTOMUPSORTInput: An array A[1..n] of n elements.Output: A[1..n] sorted in nondecreasing order. 1. t ← 1 2. while t < n 3. s ← t; t ← 2s; i ← 0 4. while i + t ≤ n 5. MERGE(A, i + 1, i + s, i + t) 6. i ← i + t 7. end while 8. if i + s < n then 9. MERGE(A, i + 1, i+ s, n)10. end while
![Page 42: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/42.jpg)
Analyzing Algorithm BOTTOMUPSORT
• With no loss of generality, assume that the size of the array, n, is a power of 2.– In the first iteration, we have pairs that are
merged using element comparisons.– In the second iteration, we have pairs that are
merged using– ….– In the jth iteration, we have pairs that are
merged using
– The outer while loop is executed times.– Therefore,
![Page 43: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/43.jpg)
Analyzing Algorithm BOTTOMUPSORT
• What about the number of element assignments?
![Page 44: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/44.jpg)
Time Complexity
• One way of measuring the performance of an algorithm is how fast it executes. The question is how to measure this “time”?– Is having a digital stop watch suitable?
![Page 45: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/45.jpg)
Order of Growth• As measuring time is subjective to many factors, we
look for a more “objective” measure, i.e. the number of operations
• Since counting the exact number of operations is cumbersome, sometimes impossible, we can always focus our attention to asymptotic analysis, where constants and lower-order terms are ignored.– E.g. n3, 1000n3, and 10n3+10000n2+5n-1 are all “the same”
– The reason we can do this is that we are always interested in comparing different algorithms for arbitrary large number of inputs.
![Page 46: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/46.jpg)
Example
Growth rate for some function
![Page 47: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/47.jpg)
Example
Growth rate for same previous functions showing larger input sizes
![Page 48: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/48.jpg)
Running Times for Different Sizes of Inputs of Different Functions
![Page 49: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/49.jpg)
Asymptotic Analysis: Big-oh (O())
• Definition: For T(n) a non-negatively valued function, T(n) is in the set O(f(n)) if there exist two positive constants c and n0 such that T(n) cf(n) for all n > n0.
• Usage: The algorithm is in O(n2) in [best, average, worst] case.
• Meaning: For all data sets big enough (i.e., n>n0), the algorithm always executes in less than or equal to cf(n) steps in [best, average, worst] case.
![Page 50: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/50.jpg)
Big O()
• O() notation indicates an upper bound.
• Usually, we look for the tightest upper bound:– while T(n) = 3n2 is in O(n3), we prefer O(n2).
![Page 51: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/51.jpg)
Big O() Examples
• Example 1: Find c and n0 to show that
T(n) = (n+2)/2 is in O(n)
• Example 2: Find c and n0 to show that T(n)=c1n2+c2n is in O(n2)
• Example 3: T(n) = c. We say this is in O(1).
![Page 52: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/52.jpg)
Asymptotic Analysis: Big-Omega (())
• Definition: For T(n) a non-negatively valued function, T(n) is in the set (g(n)) if there exist two positive constants c and n0 such that T(n) >= cg(n) for all n > n0.
• Meaning: For all data sets big enough (i.e., n > n0), the algorithm always executes in more than or equal to cg(n) steps.
() notation indicates a lower bound.
![Page 53: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/53.jpg)
() Example
• Find c and n0 to show that T(n) = c1n2 + c2n is in (n2) .
![Page 54: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/54.jpg)
Asymptotic Analysis: Big Theta (())
• When O() and () meet, we indicate this by using () (big-Theta) notation.
• Definition: An algorithm is said to be (h(n)) if it is in O(h(n)) and it is in (h(n)).
![Page 55: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/55.jpg)
Example
• Show that log(n!) is in (n log n).
![Page 56: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/56.jpg)
Complexity Classes and small-oh (o())
• Using () notation, one can divide the functions into different equivalence classes, where f(n) and g(n) belong to the same equivalence class if f(n) = (g(n))
• To show that two functions belong to different equivalence classes, the small-oh notation has been introduced
• Definition: Let f(n) and g(n) be two functions from the set of natural numbers to the set of non-negative real numbers. f(n) is said to be in o(g(n)) if for every constant c > 0, there is a positive integer n0 such that f(n) < cg(n) for all n n0.
![Page 57: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/57.jpg)
Simplifying Rules• If f(n) is in O(g(n)) and g(n) is in O(h(n)), then
f(n) is in O(h(n))
• If f(n) is in O(kg(n)) for any constant k > 0, then f(n) is in ………
• If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)), then (f1 + f2)(n) is in ………
• If f1(n) is in O(g1(n)) and f2(n) is in O(g2(n)) then f1(n)f2(n) is in ………
• You can safely “globally” replace O with or in the above, where the above rules will still hold.
![Page 58: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/58.jpg)
Very Useful Simplifying Rule
• Let f(n) and g(n) be be two functions from the set of natural numbers to the set of non-negative real numbers such that:
Thenif L < then f(n) is in
if L > 0 then f(n) is in if 0 < L < then f(n) is in if L = 0 then f(n) is in
)(
)(lim0
ng
nfL
n
![Page 59: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/59.jpg)
Space Complexity
• Space complexity refers to the number of memory cells needed to carry out the computational steps required in an algorithm excluding memory cells needed to hold the input.
• Compare additional space needed to carry out SELECTIONSORT to that of BOTTOMUPSORT if we have an array with 2 million elements!
![Page 60: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/60.jpg)
Examples
• What is the space complexity for– Linear search– Binary search– Selection sort– Insertion sort– Merge (that merges two sorted lists)– Bottom up merge sort
![Page 61: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/61.jpg)
Estimating the Running Time of an Algorithm
• As mentioned earlier, we need to focus on counting those operations which represent, in general, the behavior of the algorithm
• This is achieved by– Counting the frequency of basic operations.
• Basic operation is an operation with highest frequency to within a constant factor among all other elementary operations
– Recurrence Relations
![Page 62: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/62.jpg)
Counting the Frequency of Basic Operations
• Sometimes, it is easier to compute the frequency of an operation that is a good representative of the overall time complexity of the algorithm– For example, Algorithm MERGE.
• Counting the number of iterations – The number of iterations in a while loop and/or a for
loop is a good indication of the total number of operations
![Page 63: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/63.jpg)
Example 1
sum = 0;
for (j=1; j<=n; j++)
for (i=1; i<=j; i++)
sum++;
for (k=0; k<n; k++)
A[k] = k;
![Page 64: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/64.jpg)
Example 2
for j := 1 to n do sum[j] := 0;
for i := 1 to j2 do
sum[j] := sum[j] + i;
end for;
end for;
return sum[1..n];
![Page 65: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/65.jpg)
Example 3
count := 0;
for i := 1 to n do
m := n/i for j := 1 to m do
count := count + 1 ;
end for;
end for;
return count;
![Page 66: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/66.jpg)
Example 4
count := 0; while n >= 1 do for j := 1 to n do execute_algorithm_x; count := count + 1; end for n := n / 2; end while return count;
![Page 67: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/67.jpg)
Examples 5 & 6
sum1 = 0;for (k=1; k<=n; k*=2) for (j=1; j<=n; j++) sum1++;
sum2 = 0;for (k=1; k<=n; k*=2) for (j=1; j<=k; j++) sum2++;
![Page 68: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/68.jpg)
Example 7
count := 0; for i := 1 to n do j := 2; while j <= n do j := j2; count := count + 1; end while end for; return count;
![Page 69: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/69.jpg)
Recurrence Relations
• The number of operations can be represented as a recurrence relation.
• There are very well known techniques, other than expanding the recurrence relation, which we will study in order to solve these recurrences
![Page 70: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/70.jpg)
Example• Recursive Merge Sort
MergeSort(A,p,r) if p < r then
q := (p+r)/2; MergeSort(A,p,q);
MergeSort(A,q+1,r);
Merge(A,p,q,r);
end if;
– What is the call to sort an array with n elements?
– Let us assume that the overall cost of sorting n elements is T(n), assuming that n is a power of two.
• If n = 1, do we know T(n)?
• What is the cost of MergeSort(A,p,q)?
• What is the cost of MergeSort(A,q+1,r)?
• What is the cost of Merge(A,p,q,r)?
![Page 71: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/71.jpg)
Average Case Analysis
• Probabilities of all inputs is an important piece of prior knowledge in order to compute the number of operations on average
• Usually, average case analysis is lengthy and complicated, even with simplifying assumptions.
k
i 1
![Page 72: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/72.jpg)
Computing the Average Running Time
• The running time in this case is taken to be the average time over all inputs of size n.– Assume we have k inputs, where each input
costs Ci operations, and each input can occur with probability Pi, 1 i k, the average running time is given by
k
iiiCP
1
![Page 73: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/73.jpg)
Average Case Analysis of Linear Search
• Assume that the probability that key x appears in any position in the array (1, 2, …, n) or does not appear in the array is equally likely
– This means that we have a total of ……… different inputs, each with probability ………
– What is the number of comparisons for each input?
– Therefore, the average running time of linear search = ………
![Page 74: ICS 353 Design and Analysis of Algorithms Section 01 – 09:20-10:20am – 24:165](https://reader036.fdocuments.in/reader036/viewer/2022062512/56812cab550346895d915b9f/html5/thumbnails/74.jpg)
Average Case Analysis of Insertion Sort
• Assume that array A contains the numbers from 1..n ( i.e. elements are distinct)
• Assume that all n! permutations of the input are equally likely.
• What is the number of comparisons for inserting A[i] in its proper position in A[1..i]? What about on average?
• Therefore, the total number of comparisons on average is