Post on 11-Jan-2016
description
1
Abstract Data TypesStack, Queue
Amortized analysis
Cormen: Ch 10, 17 (11, 18)
2
ADT is an interface
• It defines– the type of the data stored– operations, what each operation does
(not how)– parameters of each operation
3
ADT
Application
Implementation of theData structure
חוזה בין מתכנת האפליקציה
ומיישם מבנה הנתונים
ממשק
4
ADT : how to work with
• Advantage: – Application programming can be done
INDEPENDENTLY of implementation– If care about efficiency (complexity):
• BE CAREFUL!!• Using the ADT the “wrong way” might
become quite inefficient!!!
5
Example: Stacks
• Push(x,S) : Insert element x into S• Pop(S) : Delete the last (time) element
inserted into S• Empty?(S): Return yes if S is empty• Top(S): Return the last element inserted into
S• Size(S)• Make-stack()
6
The Stack Data Abstraction
push
push
push
7
The Stack Data Abstraction
push
push
push
pop
push
Last in,First out.
8
The Stack: Applications??1. REAL-LIFE:
1. Rifle
2. Some document handling?
Last in,First out.
1. Computers/ Communications:
1. In some applications – LIFO preferred on FIFO
2. Program control + algorithm == a KEY
9
• Evaluate an expression in postfix or Reverse Polish Notation
Infix Postfix
(2+ 3) * 5 2 3 + 5 *
( (5 * (7 / 3) ) – (2 * 7) ) 5 7 3 / * 2 7 *-
A stack application
Application
Implementation
חוזה בין מתכנת האפליקציה
ומיישם מבנה הנתונים
ממשק
10
2 3 + 5 *
A stack application
23
Application
Implementation
חוזה בין מתכנת האפליקציה
ומיישם מבנה הנתונים
ממשק
11
2 3 + 5 *
A stack application
55
12
2 3 + 5 *
A stack application
25
13
Pseudo = כאילו
Combination of: 1. Programming (like) commands2. “free style English”
Allows to conveniently express and algorithm.
See Cormen (one page) for his language constructs. No need for FORMAL syntax. (can deviate
Cormen)
A Word on Pseudo-code
14
S ← make-stack()while ( not eof ) do B ← read the next data;
if B is an operand then push(B,S) else X ← pop(S)
Y ← pop(S) Z ← Apply the operation B on X and Y push(Z,S)return(top(S))
Pseudo-code
15
Implementation
• We will be interested in algorithms to implement the ADT..
• And their efficiency..
Application
Implementation
חוזה בין מתכנת האפליקציה
ומיישם מבנה הנתונים
ממשק
16
Using an array
12 1 3A
A[0] A[1]
t
The stack is represented by the array A and variable t
A[2] A[N-1]
1213
17
Using an array
12 1 3A
A[0] A[1]
t
make-stack(): Allocates the array A, which is of some fixed size N, sets t ← -1
The stack is represented by the array A and variable t
A[2] A[N-1]
18
Operations
12 1 3A
t
size(S): return (t+1)
empty?(S): return (t < 0)
top(S): if empty?(S) then error else return A[t]
A[N-1]A[0] A[1]
A[2]
19
Pop
12 1 3A
t
pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)
A[N-1]A[0] A[1]
A[2]
pop(S)
20
Pop
12 1 3A
t
A[N-1]A[0] A[1]
A[2]
pop(S): if empty?(S) then error else e ←A[t] t ← t – 1 return (e)
pop(S)
21
Push
12 1 3A
t
push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x
A[N-1]A[0] A[1]
A[2]
push(5,S)
22
Push
12 1 5A
t
push(x,S): if size(S) = N then error else t ←t+1 A[t] ← x
A[N-1]A[0] A[1]
A[2]
push(5,S)
23
Implementation with lists
12 1 5
top
size=3
x.element
x.nextx
24
Implementation with lists
12 1 5
top
make-stack(): top ← null size ← 0
size=3
25
Operations
12 1 5
top
size(S): return (size)
empty?(S): return (top = null)
top(S): if empty?(S) then error else return top.element
size=3
26
Pop
12 1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=3
27
Pop
12 1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=2
28
Garbage collection
1 5
top
pop(S): if empty?(S) then error else e ←top.element top ← top.next size ← size-1 return (e)
pop(S)
size=2
29
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
30
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
31
Push
1 5
topsize=2
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
32
Push
1 5
topsize=3
push(x,S): n = new node n.element ←x n.next ← top top ← n size ← size + 1 push(5,S)
5
33
Analysis
• Bound the running time of an operation on the worst-case
• As a function of the “size”, n, of the data structure
• Example: T(n) < 4n+7• Too detailed, most cases we are just
interested in the order of growth
34
Why order of growth
• Precise cost can change from computer to computer
• From Programmer to programmer
• Anyhow MOORE may go down by factor of 1.5 -2 next year
35
Big-O
) ו- כך ש:c - קיים ) ( ( ))T n O f n0n
0)()( nnnfcnT
דוגמא:2
( ) ( 1)T n n
)(4)1(222nOnn
36
Big-O
))(()( ngOnf
cg(n)
f(n)
n0
38
The running time of our stack and queue operations
• Each operation takes O(1) time
39
Stacks via extendable arrays
• Array implementation difficulties:• Pick large N: wasted space • Pick small N: stack is limited • Solution:
– Pick moderate N, and:
– When the array is full we will double its size (N) (DOUBLING)
40
Push
push(x,S):
/* N is size of array*/ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
12 1A
t
A[N-1]A[0] A[1]
41
Push
push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
12 1 3 3A 4 5 7 3 2 8 1
A[N-1]A[0] A[1]
t
push(5,S)
42
Push
push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
push(5,S)
43
Push
push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
12 1 3 3 4 5 7 3 2 8 1
t
push(5,S)
44
Push
push(x,S): if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
t
A[2N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
push(5,S)
45
Push
push(x,S):
/* N is array size */ if size(S) = N then {allocate a new array of size 2N copy the old array to the new one; N ← 2N } t ←t+1 A[t] ← x
5
A[2N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
push(5,S)
46
Analysis
• An operation may take O(n) worst case time !
• But that cannot happen often..
47
Amortized Analysis
• How long it takes to do m operations in the worst case ?
• Well, O(nm)
• Yes, but can it really take that long ?
48
49
x
50
x x
51
x x
x x x
52
x x
x x x x
53
x x
x x x x
x x x x x
54
x x
x x x x
x x x x x x
55
x x
x x x x
x x x x x x x
56
x x
x x x x
x x x x x x x x
57
x x
x x x x
x x x x x x x x
x x x x x x x x x
58
x x
x x x x
x x x x x x x x
x x x x x x x x x x
59
x x
x x x x
x x x x x x x x
x x x x x x x x x x x
60
x x x x x x x x
x x x x x x x x x
After N-1 pushes that cost 1 we will have a push that costs 2N+1
Let N be the size of the array we just copied (half the size of the new array)
Total time 3N for N pushes, 3 per push
N
x x x x
62
Theorem: A sequence of m operations on a stack takes O(m) time
We proved:
63
Amortized Analysis (The bank’s view)
• You come to do an operation with a bunch of tokens (amortized cost of the operation)
64
Operation
65
Operation
You have to pay for each unit of work by a token (from your POCKET)
67
Operation
If you have more tokens than work you put the remaining tokens in the bank
68
Operation
69
Operation
If we have more work than tokens we pay with token from the bank
70
Operation
If we have more work than tokens we pay with tokens from the bank
71
Amortized Analysis (The bank’s view)
• Suppose we prove that the bank is never in a deficit
• The total work is no larger than the total number of tokens
• 3*m in our example if m is the number of operations
73
“Easy” push
• Each push comes with 3 tokens (pocket)
• If it’s an easy push then we pay with one token for the single unit of work, and put two in the bank.
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
74
5
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
“Easy” push• Imagine that the tokens in the bank
are placed on the items of the stack• We put one token on the item just
inserted and another token on an arbitrary item without a token
75
5 6
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
76
5 6 6 7 1 10 4 67 2 5 7
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
77
“hard” push
• Tokens from the bank “pay” for copying the array into the larger array
5 6 6 7 1 10 4 67 2 5 7
A[N-1]
12 1 3 3A 4 5 7 3 2 8 1
A[0] A[1]
t
78
• Then you pay a token and put two in the bank as an “easy” push
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 A
t
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4
t
“hard” push
79
Need to prove• The balance is never negative:• When we get to an expensive push there
are enough tokens to pay for copying• By induction: prove that after the i-th push
following a copying there are 2i tokens in the bank
• N/2 easy pushes before the hard push N tokens are at bank at the hard push.
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4
t
Starting with empty
array
80
How many tokens we spent ?
• Each operation spent 3 tokens
81
Summary
• So the total # of tokens is 3m
THM: A sequence of m operations takes O(m) time !! (we finished the proof)
• Each operation takes O(1) amortized time
82
Theorem: A sequence of m operations on a stack takes O(m) time
proof (3) .
• We formalize the bank as a potential function
83
Amortized(op) = actual(op) +
Define
Let be a potential function
(can represent the amount in the bank)
Out of pocket = on Operation + on (bank)
84
Amortized(op1) = actual(op1) + 1- 0
Amortized(op2) = actual(op2) + 2- 1
Amortized(opm) = actual(opm) + m- (m-1)
……
+
iAmortized(opi) = iactual(opi) + m- 0
iAmortized(opi) iactual(opi) if m- 0 0
We need to find that gives a nontrivial statement
Out of pocket = on Operation + on (bank)
85
Example: Our extendable arrays
Define: the potential of the stack
2( 1) 2( 1) 0( )
0
t N if t NS
Otherwise
5 6 6 7 1 10 4 67 2 5 712 1 3 3 4 5 7 3 2 8 1 4
t
86
Amortized(push) = actual(push) + =
1 + 2(t’+1) – N - (2(t+1) – N) = 1+2(t’-t) = 3
Amortized(push) = N + 1 + =
N + 1 + (2 - N) = 3
Can we gain (make money)? No! #pop ≤ # push amortz>= 2
With copying (hard):
Without copying (easy):
[ N + 1 + (after -
before)]Amortized(pop) = 1 + =
1 + 2(t-1) – N - (2t – N) = -1
87
Amortized(op1) = actual(op1) + 1- 0
Amortized(op2) = actual(op2) + 2- 1
Amortized(opn) = actual(opn) + n- (n-1)
……
+
iAmortized(opi) = iactual(opi) + n- 0
iAmortized(opi) iactual(opi) if n- 0 03m ≥
88
Summary
• So the total # of tokens is 3m
THM: A sequence of m operations starting from an empty stack takes O(m) time !!
• Each operations take O(1) amortized time
89
Queue
• Inject(x,Q) : Insert last element x into Q• Pop(Q) : Delete the first element in Q• Empty?(Q): Return yes if Q is empty• Front(Q): Return the first element in Q• Size(Q)• Make-queue()
90
The Queue Data Abstraction
inject
inject
91
The Queue Data Abstraction
inject
inject
inject
inject
popFirst in,First out (FIFO).
92
Using an array
12 1 4 2A 5
A[0] A[1]
t
pop(Q)
A[2] A[N-1]
93
Using an array
1 4 2 5A
A[0] A[1]
t
pop(Q)
A[2] A[N-1]
94
Using an array
1 4 2 5A
A[0] A[1]
t
A[2] A[N-1]
This would be inefficient if we insist that elements span a prefix of the array
USE a MOVING ARRAY
95
Using an array
12 1 4 2A 5
A[0] A[1]
r
A[2] A[N-1]
f
A
A[0] A[1]
r
A[2]
f
Empty queue f=r
96
Using an array
12 1 4 2A 5
A[0] A[1]
r
pop(Q)
A[2] A[N-1]
f
97
Using an array
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)
98
Using an array
1 4 2A 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)
99
Using an array
1 4 2A 5 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)
100
Using an array
2A 5 5 5
A[0] A[1]
A[2] A[N-1]
f r
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
101
Using an array
A 5 5 5 5
A[0] A[1]
r
A[2] A[N-1]
f
pop(Q)inject(5,Q)inject(5,Q)pop(Q)pop(Q)pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
Fall off the edge?
102
Make the array “circular”
5 5 A 5 5
A[0] A[1]
r
A[2] A[N-1]
f
Pop(Q), inject(5,Q), pop(Q), inject(5,Q),……….
103
Operations
empty?(Q): return (f = r)
top(Q): if empty?(Q) then error else return A[f]
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
104
Operations
size(Q): if (r >= f) then return (r-f) else return N-(f-r)
1 4 2A 5
A[0] A[1]
A[2] A[N-1]
f r
105
Operations
size(Q): if (r >= f) then return (r-f) else return N-(f-r)
5 5 A 5 5
A[0] A[1]
r
A[2] A[N-1]
f
106
Pop
pop(Q)
1 4 2A 5
A[0] A[1]
A[2]
f r
pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)
107
Pop
pop(Q): if empty?(Q) then error else e ←A[f] f ← (f + 1) mod N return (e)
pop(Q)
1 4 2A 5
A[0] A[1]
A[2]
f r
108
Inject
inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N
inject(5,Q)
4 2A 5
A[0] A[1]
A[2]
f r
Why?
f=r empty
109
Inject
inject(x,Q): if size(Q) = N-1 then error else A[r] ← x r ← (r+1) mod N
inject(5,Q)
4 2A 5 5
A[0] A[1]
A[2]
f r
Or could do
doubling
110
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
111
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
4
112
Implementation with lists
12 1 5
headsize=3
tail
inject(4,Q)
4
Complete the details by yourself
113
Continue in DS2….Slide 26
114
Double ended queue (deque)
• Push(x,D) : Insert x as the first in D• Pop(D) : Delete the first element of D• Inject(x,D): Insert x as the last in D• Eject(D): Delete the last element of D• Size(D)• Empty?(D)• Make-deque()
115
Implementation with doubly linked lists
5
head
size=2
tail
13
x.next
x.element
x.prev
x
116
Empty list
head
size=0
tail
We use two sentinels here to make the code simpler
117
Push
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
118
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
119
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
120
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=1
tail
push(4,D)
4
121
push(x,D): n = new node n.element ←x n.next ← head.next (head.next).prev ← n head.next ← n n.prev← head size ← size + 1
5
head
size=2
tail
push(4,D)
4
122
Implementations of the other operations are similar
• Try by yourself