Amortized analysis

Amortized analysis

Amortized analysis

In amortized analysis the goal is to bound the worst casetime of a sequence of operations on a data structure.

If n operations take T (n) time (worst case), theamortized cost of an operation is T (n)/n.

In the aggregate analysis method, we directly boundT (n).

Amortized analysis

Stack with Multipop operation

A stack with Multipop operation supports the standard stackoperations (Push, Pop, IsEmpty) and a Multipop operation:

Multipop(S , k)

(1) while not IsEmpty(S) and k > 0(2) Pop(S)(3) k ← k − 1

Given n operations (starting with an empty stack):

Let n′ be the number of times Pop is called within theMultipop calls.

The total cost is Θ(n + n′).

There are at most n push operations.

Therefore, n′ ≤ n.

Total cost is Θ(n). Amortized cost is Θ(n)/n = Θ(1).

Amortized analysis

Binary counter

Suppose we store a counter in an array A[0..k − 1] ofbits, where A[0] is the least significant bit.

We implement operation Increment(A) which increasesthe value of the counter by 1, as follows.

Increment(A)

(1) i ← 0(2) while i < A.length and A[i ] = 1(3) A[i ]← 0(4) i ← i + 1(5) if i < A.length(6) A[i ]← 1

To bound the time complexity of n increment operations(starting with A containing only zeros), it suffices tobound the number of bit flips in these operations.

Amortized analysis

Binary counter

0 0 0 0 0 00 0 0 0 0 10 0 0 0 1 00 0 0 0 1 10 0 0 1 0 00 0 0 1 0 1

01234567

0 0 0 1 1 00 0 0 1 1 10 0 1 0 0 08

A[0]A[1]A[2]A[3]A[4]A[5]

bit flip freq. number of flips0 1 n1 1/2 bn/2c2 1/4 bn/4c3 1/8 bn/8c

...i 1/2i bn/2ic

...

The total number of flips isk−1∑i=0

bn/2ic <∞∑i=0

n/2i = n∞∑i=0

1/2i = 2n.

Total cost of n increments is Θ(n). Amortized cost Θ(1).

Amortized analysis

Accounting method

Each operation has a cost which is its actual timecomplexity.

When an operation is performed, we charge a certainamount, which may be different than the cost.

Part of a charge can be stored in objects of the datastructures.

To pay the cost of an operation, we can use both thecharge or stored credit.

The total cost of the operation is at most the sum of thecharges.

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

push(S,4)$2

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,4)

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,5)

5 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

push(S,2)

5 $12 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

multipop(S,2)

5 $12 $1

Amortized analysis

Stack with Multipop operation

For stack with Multipop, the costs and charges of theoperations are as follows (we assume that Pop/Multipop arenot called when the stack is empty).

operation cost chargePush 1 2Pop 1 0Multipop k ′ 0

k ′ is number of elements popped

For a push operation, we use $1 to pay for the cost, andthe remaining $1 is stored in the pushed element.

The cost of Pop/Multipop is payed using the dollarsstored at the popped elements.

The total cost of n operations is at most 2n.

The amortized time complexity of an operation is Θ(1).

4 $1

multipop(S,2)

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 0 0A[0]A[1]A[2]A[3]A[4]A[5]

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 0 1A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 1 0A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 0 1 1A[0]A[1]A[2]A[3]A[4]A[5]

$1$1

Amortized analysis

Binary counter

The cost of an Increment(A) operation is the numberof bit flips.

The charge of an Increment(A) operation is $2.

We keep the following invariant: each 1 in A stores $1.

To pay the cost of Increment(A), we use the dollarsstored at the 1’s that were changed to 0’s, and $1 fromthe charge (to pay for the 0 that was changed to 1).

The remaining $1 of the charge is stored at the 0 thatwas changed to 1.

0 0 0 1 0 0A[0]A[1]A[2]A[3]A[4]A[5]

$1

Amortized analysis

Potential function method

Consider a sequence of n operations.

Let ci be the cost of the i -th operation in the sequence.

Let Di be the data-structure after the i -th operation.

Define a potential function Φ: {D0, . . . ,Dn} → R.

The amortized cost of the i -th operation isci = ci + Φ(Di)− Φ(Di−1).

n∑i=1

ci =n∑

i=1

(ci+Φ(Di)−Φ(Di−1)) =n∑

i=1

ci+Φ(Dn)−Φ(D0)

If Φ(D0) = 0 and Φ(Di) ≥ 0 for all i ,∑n

i=1 ci ≤∑n

i=1 ci .

To bound∑n

i=1 ci , we compute the sum∑n

i=1 ci .

This method is a variation of the accounting method. Φ(Di) isthe total stored credit. ci is the charge of the i -th operation.

Amortized analysis

Binary counter

Define Φ(Di) to be the number of ones in the array.

Recall that ci is the number of bit flips performed by thei -th operation.

The i -th operation changes A[0], . . . ,A[ci − 2] from oneto zero, and A[ci − 1] from zero to one. Therefore,

Φ(Di)− Φ(Di−1) = 1− (ci − 1) = 2− ci

ci = ci + (2− ci) = 2.∑ni=1 ci = 2n.

1 1 0 0 1 1A[0]A[1]A[2]A[3]A[4]A[5]

1 1 0 1 0 0A[0]A[1]A[2]A[3]A[4]A[5]

Di-1 Di

Amortized analysis

Hash table

Suppose that we use a hash table with chaining.

Let α = n/m, where n is the number of elements in thetable and m is the size of the table.

We want to maintain the invariant 1/4 ≤ α ≤ 1:

α ≤ 1 gives Θ(1) expected search time.α ≥ 1/4 gives Θ(n) space.

Assume we only have insertions (no deletions).

We can maintain the invariant by performing rehashingwhen the m + 1-th element is inserted to the table: Wecreate a new table of size 2m and move all elements tothe new table.

Amortized analysis

Hash table

The cost of Insert is $1 for inserting the element to thetable, and m dollars for performing rehashing.

The charge of Insert is $3.

When we insert an element to the table, we use $1 fromthe charge to pay for the cost, and we store the remaining$2 in the table.

When rehashing is performed, the cost is payed usingdollars stored in the table.

Immediately after rehashing, no dollars are stored in thetable.

The next rehashing occurs after m insertions, so the tablestores 2m dollars at the time of the next rehashing.

Amortized analysis

Example

3

T

 6

$2

Amortized analysis

Example

T

 

insert(T,4)6

3

$0

Amortized analysis

Example

T

 

insert(T,4)6

4 3

$2

Amortized analysis

Example

T

 

insert(T,2)6

4 32

$4

Amortized analysis

Example

T

 

insert(T,7)

6

3

4

2

$0

Amortized analysis

Example

T

 

insert(T,7)

6

3

4

27

$2

Amortized analysis

Example

T

 

insert(T,7)

6

3

4

27

$2

Amortized analysis

BB[α] tree — definition

Let size(v) be the number of vertices in the subtree of v .A vertex v is of bounded balance α if

size(v .left) ≥ bα·size(v)c and size(v .right) ≥ bα·size(v)cA BB[α] tree (α < 0.5) is a binary search tree such thatevery vertex v is of bounded balance α.The height of a BB[α] tree T is at most log1/(1−α) |T |.

α = 1/3

9

1 1

35

3 1

114

2 6

10

168

14

12

20

Amortized analysis

Insertion and deletion

To insert/delete an element to a BB[α] tree, insert/deleteas in BST. Let w be the highest vertex that is not ofbounded balance α. If w exists, replace the subtree of wby a balanced tree containing the same elements.The time complexity is Θ(height(T ) + size(w)), which isΘ(|T |) in the worst case.

α = 1/3

9

1 1

35

3 1

114

2 6

10

168

14

12

20

Amortized analysis

Insertion and deletion

To insert/delete an element to a BB[α] tree, insert/deleteas in BST. Let w be the highest vertex that is not ofbounded balance α. If w exists, replace the subtree of wby a balanced tree containing the same elements.The time complexity is Θ(height(T ) + size(w)), which isΘ(|T |) in the worst case.

α = 1/3

10

1 1

36

4 1

12

w

3

4

2 6

10

168

14

12

20

Amortized analysis

Insertion and deletion

To insert/delete an element to a BB[α] tree, insert/deleteas in BST. Let w be the highest vertex that is not ofbounded balance α. If w exists, replace the subtree of wby a balanced tree containing the same elements.The time complexity is Θ(height(T ) + size(w)), which isΘ(|T |) in the worst case.

α = 1/3

10

1 1

36

3 2

11 12 10

16

8 14

12

203

4

6

Amortized analysis

Amortized complexity

We analyze the cost of n insertions (w/o deletions).The cost of insert is height(T ) + size(w), where w is thevertex whose subtree was replaced (or the new leaf).The charge of insert is(

1 +1

1− 2α

)log1/(1−α) |T |+

2

1− 2α.

We use height(T ) ≤ log1/(1−α) |T | dollars from thecharge to pay for the height(T ) cost.In each node on the path from the root to the new leaf,we put 1/(1− 2α) dollars from the charge. This requires≤ 1

1−2α log1/(1−α) |T | dollars.We use the dollars stored in w and 2/(1− 2α) dollarsfrom the charge to pay for the size(w) cost.We need to show that w contains at least size(w)− 2

1−2αdollars.

Amortized analysis

Analysis

Consider the tree before the balancing of w .Let M = size(w).Without loss of generality, size(w .left) ≥ size(w .right).Since w is not of bounded balance αsize(w .right) = bαMc − 1

size(w .left) = M − size(w .right)− 1 = M − bαMcConsider the insert operations of elements to the subtreeof w since the last balancing operation involving w .The least number of such insertions is when all elementswere inserted to w .left.In that case, after last balancing operation involving w ,

size(w .right) = bαMc − 1 and size(w .left) ≤ bαMc.Therefore, the number of insert operations is at least

(M − bαMc)− bαMc ≥ M(1− 2α)− 2.Amortized analysis

Analysis

Each insert operation of an element to the subtree of wadds 1/(1− 2α) dollars to w .

Therefore, w contains at least

1

1− 2α(M(1− 2α)− 2) = M − 2

1− 2α

dollars.

The charge of insert is Θ(log |T |). Therefore, the cost of ninsert operations is Θ(n log n).

Amortized analysis