Practical Session No. 14 –

Topological sort ,Amortized Analysis

Topological sort

Topological-

Sort

Ordering of vertices in a directed acyclic graph (DAG) G=(V,E)

such that if there is a path from v to u in G, then v appears before u

in the ordering.

There could be many solutions, for example:

1. call DFS to compute f[v]

2. As the visit in each vertex is finished (blackened), insert it to the

head of a linked list

3. Return the linked list of the vertices

Time Complexity: O(|V|+|E|)

Question 1

2 lists are given:

A - courses list. Every student must study all the courses in A.

B – Prerequisites. B contains tuples (a, b) (where a and b are in A), indicating that

course a must be taken before course b. The prerequisites dependencies are acyclic.

Design a schedule for each of the following students.

1. A lazy student who wants to take only one course in a semester.

2. A student who wants to take all the courses in A in the minimal number of

semesters. The student is willing to take any number of courses in a semester.

Example: A = { Alg, Eng, Ds1, Ds2, Mat, Ph1, Ph2 }

B = { (Alg, Ds2), (Ds1, Ds2), (Mat, Ds1), (Ph1, Ph2) }

Optional output for student no. 1:

Semester 1: Eng

Semester 2: Mat

Semester 3: Alg

Semester 4: Ds1

Semester 5: Ds2

Semester 6: Ph1

Semester 7: Ph2

Optional output for student no. 2:

Semester 1: Eng, Mat, Ph1, Alg

Semester 2: Ds1, Ph2

Semester 3: Ds2

Solution: Assume that the courses names are integers in the range [1..n], n is known (n is not

constant). The relations between the courses will be represented by a directed graph

G = (V, E), where V is the set of courses and if course i is a prerequisite of course j, E

will contain the edge (i, j). The graph will be represented as an adjacency list.

The graph for the given example is:

Algorithm for a lazy student: Print the courses in a way that course i won't be

printed before course j, if j is a prerequisite of i. In other words, sort the courses

topologically.

Complexity: O(|V| + |E|)

Now, let's observe another algorithm for topological sort of a DAG in O(|V|+|E|).

Find in-degree of all the vertices and keep a list of vertices with in-degree=0 -

O(|V|+|E|)

While V is not empty do:

o Extract a vertex v with in-degree of 0 from the list - O(1)

o output v and remove it from G, along with its edges,

o Reduce the in-degree of each node u such as (v,u) was an edge in G and add to

the list vertices with in-degree=0, if necessary - O(d(v))

Algorithm for a hard working student: A variation of the above topological sort

algorithm with a slight change. In each semester k, execute the algorithm on all the

nodes with degree 0 simultaneously (instead of dealing with one source at each stage,

all the sources will be dealt and printed in one semester.)

(note that the topological sort is not necessary because after we find the vertexes with

in-degree 0, we can go to the next vertexes through the edges.)

Complexity: O(|V| + |E|)

Question 2

Given a DAG (directed acyclic graph) G=(V,E) with weighted edges. Suggest an

O(|V|+|E|) time algorithm for finding the weight of the maximum weight path in G.

Solution: 1. Topologically sort V - O(V+E).

(We'll denote the topologically ordered vertex list as (v1,v2, …, vn)).

2. For every vertex v, create a list of all in-edges, i.e., list of all the vertices u, such

that (u,v)E - O(V+E).

3. Define a new array A for which A[i] = the maximal weighted path that ends in vi.

By definition, A[i] = max{A[j]+w(j,i) : (j,i) E }. - O(V+E).

4. Start by computing A[i] for all i = 1..n, where v1, v2, …, vn is the topological

ordering.

Since the graph was topologically sorted, when computing A[i], all the values A[j]

of

it’s neighbors j were already computed (all i's neighbors appear before it in the

topological ordering).

5. After computing A[i] for all the nodes, find vk for which A[k] is maximal.

MaxPath(G)

v1, v2, …, vn ← TopologicalSort(G)

Create-in-edges-lists()

A ← new array [1..n]

A[1] ← 0

for i ← 2 to n do

A[i] ← max{A[j]+w(j,i) : j in_edges_list[i] }

return max{A[1],A[2],…,A[n]}

We design the algorithm

1. Express the solution to a problem in terms of solutions to smaller problems.

2. Solve all the smallest problems first and put their solutions in a table

3. Solve the next larger problems, and so on, up to the problem, we originally wanted

to

solve.

4. Each problem should be easily solvable by looking up and combining solutions of

smaller problems in the table.

Time Complexity: O(|V|+|E|)

Amortized Analysis

Refers to finding the average running time per operation, over a worst-case sequence

of operations. Amortized analysis differs from average-case performance in that

probability is not involved; amortized analysis guarantees the time per operation over

worst-case performance. In an amortized analysis, we consider the total time of a

sequence of operations. Even if some single operations might be very expensive.

Aggregate analysis determines the upper bound T(n) on the total cost of a sequence

of n operations, then calculates the amortized cost of an operation to be T(n) / n.

Accounting method determines the individual cost of each operation, combining its

immediate execution time and its influence on the running time of future

operations. Usually, many short-running operations accumulate a "debt" of

unfavorable state in small increments, while rare long-running operations decrease

it drastically. The costs of actions are represented by tokens. One token represent

an operation of one action in the computer. The cheap operations actually account

for more token in the accounting method than they actually are (usually still O(1)

but with higher constant), and when executing cheap immediate operations, the

tokens are accumulated to credit for later “paying" for actual expensive operations.

Question 3: Incrementing a Bit String

A is a bit string. What is the time complexity of increment operation using Amortized

analysis? Analyze number of bit flips (Show with both methods: aggregate and

accounting).

Solution: Aggregate Method:

Assume n increments starting from all 0s.

A[0] flips every increment for n flips.

A[1] flips every 2nd time for <= n/2 flips.

A[2] flips every 4th time for <= n/4 flips.

A[i] flips every 2ith time for <= n/2^i flips.

Number of flips <= n +n/2+n/4+...=2n O(n)

Therefore amortized cost of each operation is 2 = O(1)

Accounting Method:

Assume n increments starting from all 0s.

Increment flips exactly one bit from 0 to 1.

Assign an amortized cost of 2 tokens per increment.

Increment(A[0…m-1])

i 0

while i < m and A[i] = 1

A[i]0

ii + 1

if i < m A[i]1

Both tokens are assigned to the bit that is flipped from 0 to 1.

Use one token immediately for flip from 0 to 1.

Save other token for when it is flipped back to 0.

All bit flips are accounted for, so the total cost of 2n is <= number of bit flips.

Question 4: Clearable Table Data Structure We would like to create an ADT, called clearable table, of a table which support the

following operations.

add(e) Insert a new element to the next unoccupied cell.

clear() Empty the table (delete all the elements from the table).

Clearable table is implemented with a table of size N.

Prove that the amortized cost of add(e), clear() is O(1) in each of the two

methods(aggregate and accounting)

Solution: Accounting Method:

Let S be a table of size N.

Assume that n operations of clear and add have been performed on S (at the beginning

S was empty).

Let us assume we pay one token for an action that is performed in constant time

(basic operation).

We will define a cost to each of the operations:

add - 2 tokens.

clear - 0 tokens.

We set more tokens to add than it actually cost, and we set fewer tokens to clear.

After an element is inserted into the table, it has a spare token. Therefore, meanwhile

the clear operation, each element pays the extra token for its deletion from the table.

We get a cost of O(1) for each operation

Aggregate Method:

Assume that n operations of clear and add have been performed on S (In the

beginning S was empty).

For each operation of inserting an element to table, we could delete the element at

most one time.

Therefore the number of elements that was deleted from the table is at most n.

The total cost of n operations is O(n), and the cost of single operation is O(n)/n=O(1).

Question 5 You must implement a queue using only (a constant number of) stacks, you may only

use O(1) extra space besides the stacks. All operations work in O(1) amortized time.

Solution: We use 2 stacks, A and B.

Accounting Method:

Let us define the cost of each operation:

Enqueue(x) - 3 tokens.

Dequeue() - 1 token.

Each element enqueued has 2 spare tokens to later "pay" for moving it from stack B to

stack A when needed.

An element is moved from stack B to A at most once, and hence when executing

Enqueue(x), the moving of the elements between stacks is already "paid" for.

Thus all operations cost O(1) amortized time.

Aggregate Method:

Assume that n operations Enqueue(x) and Dequeue() have been performed on the

queue (In the beginning S was empty).

Enqueue(x) always costs O(1), Dequeue() costs (2k+1) where k is the number of

Enqueue( ) operations performed right before it. (k=0 if it comes after another

Dequeue() or is the first operation executed on the empty stack).

So if we group every Dequeue() operation with all k Enqueue(x) operations right

before it we get that the sum of the costs of these (k+1) operations is (1+1+1….+1) +

(2k+1) = 3k+1 =< 4k.

Thus grouping all operations in a sequence of blocks of 1 Dequeue() operation

preceded with all Enqueue( ) operations right before it we have the total cost of n

operations =<4n = O(n), and the cost of single operation is O(n)/n=O(1).

Dequeue()

if (A is empty)

while(B is not empty)

push(A,pop(B))

if (A is not empty)

return pop(A)

Enqueue(x)

push(B,x)

Question 6

אנו מעוניינים להוסיף לו איברים בעזרת , בהינתן מערך מאותחל

.addהפונקציה

בגודל אנו מאתחלים מערך חדש, כאשר מוסיפים איבר למערך מלא

.כפול ומעתיקים את כל האיברים מן המערך הישן אל החדש

:לדוגמא

.מכפילים את גודל המערך והשלישית והחמישית אנ, בדוגמא ניתן לראות כי בפעולת ההוספה השנייה

.נתחו את זמן הריצה לשיעורין בשיטת הצבירה והחיוב

:פתרון

:שיטת הצבירה

:למערך הינה iמחיר פעולת ההוספה של האיבר ה , נשים לב כי

:פעולות הוספה nכעת נסכום את מחירי

.בממוצע O(1)הינה addלכן העלות לשיעורין של פונקציה

:שיטת החיוב

:מטבעות אינה מספיקה 2בכ addתמחור של הפונקציה

i 1 2 3 4 5 6 7 8 9 10 si 1 2 4 4 8 ci 1 2 3 1 5 c'i 2 2 2 2 2 bi 1 1 0 1 -2

כדי לכסות את פעולת ב המטבעות מספיק 3ב addתמחור של הפונקציה

:העתקת המערךi 1 2 3 4 5 6 7 8 9 10 si 1 2 4 4 8 8 8 8 16 16 ci 1 2 3 1 5 1 1 1 9 1 c'i 3 3 3 3 3 3 3 3 3 3 bi 2 3 3 5 3 5 7 9 3 4

מטבע ראשון להכנסת האיבר הi למערך.

מטבע שני להזזתi בפעם הראשונה לאחר הכנסתו למערך.

2הינו החזקה הגדולה ביותר של כאשר מטבע שלישי נתרם לאיבר ה

.iלפני הכנסת האיבר ה כבר כלומר עבור איבר קודם במערך שהועתק .iהקטנה מ

. כלומר 3nנקבל זמן ריצה חסום על ידי addפעולות nלכן עבור