Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks...
-
Upload
julie-matthews -
Category
Documents
-
view
222 -
download
0
description
Transcript of Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks...
![Page 1: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/1.jpg)
Introduction to Algorithm Analysis Concepts
15-211 Fundamental Data Structures and Algorithms
Aleks Nanevski and Margaret Reid-Miller
January 15, 2003
based on the lecture by Peter Lee
![Page 2: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/2.jpg)
Plan
TodayIntroduction to some basic concepts in
the design of data structures lists, related algorithms and analysis
Reading:For today: Chapter 5 and 7.1-7.3For next time: Chapter 18 and 19
![Page 3: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/3.jpg)
Homework 1 is available!
See the Blackboard Due Monday, Jan.19, 11:59pm
![Page 4: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/4.jpg)
A First Data Structure
![Page 5: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/5.jpg)
Lists of integers
Operations:create a new empty listreturn the length of the listadd an integer to the end of the list…
3 2 7 1 4
![Page 6: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/6.jpg)
Implementing lists
How shall we implement this? What design process could we use?
One answer:Think mathematicallyThink inductively
![Page 7: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/7.jpg)
Induction
Recall proofs by induction: If trying to prove that a property
P(n) holds for 0, 1, 2, …, thenProve the base case of P(0)For n>0, assume P(n-1), show that
P(n) holds
![Page 8: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/8.jpg)
Inductive definitions A great deal of computer science
can and must be defined inductively Example: factorial
fact(n) = 1 * 2 * ... * n Inductive definition:
fact(0) = 1fact(n) = n * fact(n-1), for n>0
Base case
Inductive case
![Page 9: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/9.jpg)
Implementing lists
How shall we implement this? What design process could we use?
One answer:Think mathematicallyThink inductively
![Page 10: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/10.jpg)
Inductive definitions
An integer list is eitheran empty list, oran integer paired with an integer list
Base case
Inductive case
![Page 11: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/11.jpg)
Integer lists in Java
An integer list is eitheran empty list, oran integer paired with an integer list
use null
define a new IntList class
The inductive definition gives us guidance on ways to implement integer lists in Java
One possibility (not really the best):
![Page 12: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/12.jpg)
Integer lists in Javapublic class IntList { int head; IntList tail;
public IntList(int n, IntList l) { head = n; tail = l; }…
An integer list is eitheran empty list, oran integer paired with an integer list
![Page 13: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/13.jpg)
Integer lists in Java Example:
L.head == 3 How to access 7?
L = 3 7 1 0 9head
tail
headtail
![Page 14: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/14.jpg)
Integer lists in Java Example:
L.head == 3 How to access 7?
L.tail.head == 7
L.tail.tail.tail.head == ?
L = 3 7 1 0 9head
tail
headtail
![Page 15: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/15.jpg)
Integer lists in Java Example:
L.head == 3 How to access 7?
L.tail.head == 7
L.tail.tail.tail.head == 0
L = 3 7 1 0 9head
tail
headtail
![Page 16: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/16.jpg)
How about the length operation?
![Page 17: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/17.jpg)
Another inductive definition
The length of a list L is0, if L is the empty list1 + length of the tail of L, otherwise
![Page 18: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/18.jpg)
Implementing length()
public class ListOps {
public static int length(IntList l) { if (l == null) return 0; else return 1 + length(l.tail); } …
![Page 19: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/19.jpg)
The add operation Attach n onto the end of list L
Inductive definition: the singleton list containing n, if L is the
empty list
L = 3 7 1 0 9 n
![Page 20: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/20.jpg)
The add operation Attach n onto the end of list L
Inductive definition: the singleton list containing n, if L is the
empty listotherwise, a list whose head is ? tail is ?
L = 3 7 1 0 9 n
![Page 21: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/21.jpg)
The add operation Attach n onto the end of list L
Inductive definition: the singleton list containing n, if L is the empty listotherwise, a list whose
head is the head of L, and tail is M
where M is the result of appending n onto the end of the tail of L
L = 3 7 1 0 9 n
![Page 22: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/22.jpg)
Implementing add()
public class ListOps { …
public static IntList add(int n, IntList l){ if (l == null) return new IntList(n, null); else return new IntList(l.head,add(n,l.tail)); } …
![Page 23: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/23.jpg)
Running time
How much time does it take to compute length() and add()?
No, not “wall-clock” time.
![Page 24: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/24.jpg)
The “step” In order to abstract from a
particular piece of hardware, operating system, and language, we will focus on counting the number of steps of an algorithm
A “step” should execute in constant timeThat is, its execution time should not
vary much when the size of the input varies
![Page 25: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/25.jpg)
Constant-time operationspublic static int length(IntList l) { if (l == null) return 0; else return 1 + length(l.tail); }
This is the only operation in length() that does not run in a constant amount of time.Hence, we want to know how many times this operation is invoked.
![Page 26: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/26.jpg)
Constant-time operationspublic static int length(IntList l) { if (l == null) return 0; else return 1 + length(l.tail); }
Each call to length() requires at most a constant amount of time plus the time for a recursive call on the tailSo, the “steps” we want are the number of recursive calls
![Page 27: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/27.jpg)
length()
How many steps for length()? Inductive reasoning:
0 steps (i.e. recursive calls), if l == null1 + #steps for l.tail, otherwise
For a list with N elements, length() requires N steps
We say that length() is linear time.
![Page 28: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/28.jpg)
Why do we care about “steps”?n 100n sec 7n2 sec 2n sec
1 100 s 7 s 2 s5 .5 ms 175 s 32 s
10 1 ms .7 ms 1 ms45 4.5 ms 14 ms 1 year
100 100 ms 7 sec 1016 year1,000 1 sec 12 min --
10,000 10 sec 20 hr --1,000,000 1.6 min .22 year --
![Page 29: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/29.jpg)
Our goal
Our goal is to compare algorithms against each otherNot compute the “wall-clock” time
We will also want to know if an algorithm is “fast”, “slow”, or maybe so slow as to be impractical
![Page 30: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/30.jpg)
What about add()?
public static IntList add(int n, IntList l) { if (l == null) return new IntList(n, null); else return new IntList(l.head, add(n, l.tail));}
![Page 31: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/31.jpg)
What about add()?
public static IntList add(int n, IntList l) { if (l == null) return new IntList(n, null); else return new IntList(l.head, add(n, l.tail));}
Answer: also linear time.
![Page 32: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/32.jpg)
Let’s Try Something a Bit Harder…
![Page 33: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/33.jpg)
Reverse
The reversal of a list L is:L, if L is emptyotherwise, the head of L added to the
end of M where M is the reversal of the tail of L
![Page 34: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/34.jpg)
Implementing reverse()
public static IntList reverse(IntList l) { if (l == null) return null; else { IntList r = reverse (l.tail); return add (l.head, r);}
![Page 35: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/35.jpg)
How many steps? How many “steps” does reverse
take? Think back to the inductive
definition:The reversal of a list L is:
L, if L is empty otherwise, the head of L added to M
• where M is the reversal of the tail of L
![Page 36: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/36.jpg)
Running time for reverse
The running time is given by the following recurrence equation:
t(n) = 1, if n=0t(n) = n + t(n-1)
time required to reverse the tail
time required to add head to the end
Solving for t would tell us how many steps it takes to reverse a list of length n
![Page 37: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/37.jpg)
Reverset(0) = 1
t(n) = n + t(n-1)
public static IntList reverse(IntList l) { if (l == null) return null; else { IntList r = reverse(l.tail); return add (l.head, r); }}
![Page 38: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/38.jpg)
Solving recurrence equations A common first step is to use repeated
substitution:t(n) = n + t(n-1) = n + (n-1) + t(n-2) = n + (n-1) + (n-2) + t(n-3)and so on… = n + (n-1) + (n-2) + (n-3) + … + 1
![Page 39: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/39.jpg)
Gauss says that this is easy…
t(n) = n + (n-1) + (n-2) + … +1 = n(n+1)/2
But how on earth did he come up with this beautiful little closed-form solution?
![Page 40: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/40.jpg)
Incrementing series
By the way, this is an arithmetic series that comes up over and over again in computer science, because it characterizes many nested loops:
for (i=1; i<n; i++) { for (j=1; j<i; j++) { f(); }}
![Page 41: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/41.jpg)
Mathematical handbooks
For really common series like this one, standard textbooks and mathematical handbooks will usually provide closed-form solutions.So, one way is simply to look up the
answer. Another way is to try to think
visually…
![Page 42: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/42.jpg)
Visualizing it
n
n
0 1 2 3 …
12
3
…Area: n2/2
Area of the leftovers: n/2
So: n2/2 + n/2 = (n2+n)/2 = n(n+1)/2
![Page 43: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/43.jpg)
Proving it Yet another approach is to start
with an answer or a guess, and then verify it by induction.t(1) = 1(1+1)/2 = 1Inductive case:
for n>1, assume t(n-1) = (n-1)(n-1+1)/2 = (n2 - n)/2
then t(n) = n + (n2 - n)/2 = (n2 + n)/2 = n(n+1)/2
![Page 44: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/44.jpg)
Gauss says that this is easy...Fold the sequence in half1 n sums up to: n+12 n-1 sums up to: n+13 n-2 sums up to: n+1
4 n-3 sums up to: n+1 5 n-4 ...6 n-5... ... total: (n+1)*n/2
![Page 45: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/45.jpg)
Summations
Arithmetic and geometric series come up everywhere in analysis of algorithms.
Some series come up so frequently that every computer scientist should know them by heart.
![Page 46: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/46.jpg)
Quadratic time Very roughly speaking,
f(n) = n(n+1)/2 grows at about the same rate as
g(n) = n2
In such cases, we say that reverse() runs in quadratic time(we’ll be more precise about this later
in the course)
![Page 47: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/47.jpg)
How about Sorting?
Everybody knows how to sort an array, but we have singly linked lists.
As always, think inductively:
sort(nil) = nilsort(L) = insert the head into the right
place in sort(tail(L))
![Page 48: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/48.jpg)
Ordered Insert
Need to insert element in order, in an already sorted lists.
2 5 10 20 50
12 2 5 12 10 20 50
![Page 49: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/49.jpg)
Code for ordered insert
The running time depends on the position of x in the new list.
But in the worst case this could take n steps.
public IntList order_insert(int x,IntList l) { if (x <= l.head) return new IntList(x, l); else { IntList t = order_insert(x, l.tail); return new IntList(l.head, t); }}
![Page 50: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/50.jpg)
Analysis of sort()sort(nil) = nilsort(L) = insert the head into the right
place in sort(tail(L))
t(0) = 1t(n) = n + t(n-1)
which we already know to be “very roughly” n2, or quadratic time.
![Page 51: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/51.jpg)
Insertion sort
for i = 2 to n do
insert a[i] in the proper place
in a[1:i-1]
This is yet another example of a doubly-nested loop…
for i=2, this sorts a[1:2]for i=3, this sorts a[1:3]...for i=n, this sorts a[1:n]
![Page 52: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/52.jpg)
How fast is insertion sort?
We’ve essentially counted the number of computation steps in the worst case.
But what happens if the elements are nearly sorted to begin with?
![Page 53: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/53.jpg)
A preview of some questions
Question: Insertion sort takes n2 steps in the worst case, and n steps in the best case. What do we expect in the average case? What is meant by “average”?
Question: What is the fastest average time that we could ever hope to sort in? How could we prove our answer?
![Page 54: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/54.jpg)
Worst-case analysis
We’ll have much more to say, later in the course, about “worst-case” vs “average-case” vs “expected case” performance.
![Page 55: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/55.jpg)
Better sorting
The sorting algorithm we have just shown is called insertion sort.
It is OK for very small data sets, but otherwise is slow.
Later we will look at several sorting algorithms that run in many fewer steps.
![Page 56: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/56.jpg)
Quiz Break
![Page 57: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/57.jpg)
Doubling summationLike the incrementing summation, sums of powers of 2 are also encountered frequently in computer science.
What is the closed-form solution for this sum? Prove your answer by induction.
![Page 58: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/58.jpg)
Hint 1: Visualizing it
Imagine filling a glass by halves…
2n-1
2n-2
2n-32n-42n-5
![Page 59: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/59.jpg)
Hint 2: Visualizing it A somewhat geekier hint:
term in binary20 121 1022 10023 100024 10000…
What is the sum of this column of binary numbers?
![Page 60: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/60.jpg)
Proving it Base case:
When n=1, then 20 = 21-1 Induction step, when n>1.
Assume true for n’<n, consider n
By the IH, then
![Page 61: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/61.jpg)
How does this series appear in programming?
Think of its recurrence equation:
Can you think of an algorithm whose running time is characterized by this series?
t(0) = 1t(n) = 2 * t(n-1) + 1
![Page 62: Introduction to Algorithm Analysis Concepts 15-211 Fundamental Data Structures and Algorithms Aleks Nanevski and Margaret Reid-Miller January 15, 2003.](https://reader035.fdocuments.in/reader035/viewer/2022062504/5a4d1b827f8b9ab0599bb6ee/html5/thumbnails/62.jpg)
Summary Counting constant-time “steps” of
computation Linear time and quadratic time Recurrence equations Sums of geometric series Simple list algorithms Next time: Programming tips