Greedy algo revision 2

1

Greedy Algorithims

It is animated pic

"Greed, for lack of a better word, is good" - Gordon Gecko, Wall Street (1987)

With a goal of reaching the largest-sum, at each step, the greedy algorithm will choose what appears to be the optimal immediate choice, so it will choose 12 instead of 3 at the second step, and will not reach the best solution, which contains 99.

Revision-2 (7 Dec, 2014)

2Revision-2 (7 Dec, 2014)

3

Currency Exchange Problem is Greedy

The currency algorithm is optimal in that it uses the fewest number of currency notes among all possible ways to make change for a given amount


4Revision-2 (7 Dec, 2014)

Example: To make change for the amount x = 49 (RUPEES).

Use E = 49/20 = 2. The remainder = 49 – 20E = 9

C= 9/5 = 1. Then the remainder = 9 – 5C = 4,

B = 4/2 = 2 . Finally, the remainder = 4 – 2B = 2,

B = 2/2 =0.

The total number of Currency used = E+E+C +B+ B = 5.

1=A2=B5=C10=D20=E

Revision-2 (7 Dec, 2014) 5

Here’s how the Greedy Algorithm can ruin your morning


If you’ve ever grown impatient waiting for a bus and instead started walking to your destination only to have a bus pass you 10 minutes late, you have fallen prey to the Greedy Algorithm. Sure, it feels good to start walking, but long-term, you’ll be both exhausted and late when you arrive at your destination.

In fact, sometimes it’s better to walk backwards, away from your final destination. That’s called retrograde progress. How can that be true? If the subway is two blocks behind you, and it’s rush hour in New York City.

Explanation of the previous slides


Greedy people pick short term gains over long term gains all the time. Which sometimes work , sometimes counter productive


In the last decades, similar problems called vehicle routing problems have become crucial for many companies, like Fedex and UPS. And these are so difficult, that companies asked help from applied mathematicians! Since then, applied mathematicians have enjoyed the beautifully combinatorial structure of these problems, as well as the millions of dollars companies have been willing to pay for answers!


The VRP (vehicle Routing Problem) is actually known to as hard as the TSP. This means that, when the number of cities isn’t small, there is virtually no hope to solve the VRP exactly in a reasonable amount of time, even with tomorrow’s greatest computer power. In fact, so far, no best solution is known for problems with more than a few hundreds of cities to visit!


To make Santa Claus’ problem a vehicle routing problem, we merely have to assume that Santa Claus’ reindeers cannot carry too heavy quantities. So, for instance, they can only carry presents for 5 cities. This will force Santa to come back to his toy factory several times to reload his sleigh. Typically, a Santa’s journey on the night of December 24th could look as follows:

Assumption to solve VRP


The most straightforward heuristics is the greedy algorithm.

What’s a greedy algorithm?It consists in choosing the next best step without much thinking, like a greedy Santa who is given a bunch of candies to choose from. He’d repeatedly eat his favorite one among the remaining candies, until he’s full… or until Mother Christmas tells him to stop!

It sounds like a great way to choose the best candies!


Note, this is not best solution, if you are interested for the best solution , visit http://www.science4all.org/le-nguyen-hoang/vehicle-routing-problem/

Knapsack Algorithem

The Greedy Method


Given n objects each have a weight wi and a value vi , and given a knapsack of total capacity W. The problem is to pack the knapsack with these objects in order to maximize the total value of those objects packed without exceeding the knapsack’s capacity. More formally, let xi denote the fraction of the object i to be included in the knapsack, 0 xi 1, for 1 i n. The problem is to find values for the xi such that

Note that we may assume because otherwise, we would choose xi = 1 for each i which would be an obvious optimal solution.

n

iii

n

iii vxWwx

11maximized. is and

n

ii Ww

1

The Knapsack Problem:

19


(Max value/weight ratio) Sort the objects based on the value to weight ratios, from the highest to the lowest, then select. Example: Given n = 5 objects and a knapsack capacity W = 100 as in Table I..

Fractional knapsack problem

W 10 20 30 40 50 SUMV 20 30 66 40 60

V/W 2.0 1.5 2.2 1.0 1.2

x[i] 1 1 1 0 0.8 164

20


The Optimal Knapsack Algorithm:Input: an integer n, positive values wi and vi , for 1 i n, and another positive value W.

Output: n values xi such that 0 xi 1 and

Algorithm (of time complexity O(n lgn))(1) Sort the n objects from large to small based on the ratios vi/wi . We assume the arrays w[1..n] and v[1..n] store the respective weights and values after sorting.(2) initialize array x[1..n] to zeros.(3) weight = 0; i = 1(4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1 (4.2) else x[i] = (W – weight) / w[i] (4.3) weight = weight + x[i] * w[i] (4.4) i++

n

iii

n

iii vxWwx

11maximized. is and

21

22

100 KG50

KG60 PKR

40 KG

40 PKR

30 KG

66 PKR

20 KG

30 PKR10

KG20 PKR


i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0 0 0 0 0

(4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1

(4.2) else x[i] = (W – weight) / w[i]

(4.3) weight = weight + x[i] * w[i](4.4) i++

W=0 n=5before , weight=0i=1W[i]=?X[i]=?After, weight=?

23


i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0 0 0 0





W=0 n=5before , weight=0i=1W[i]=30X[i]=1After, weight=30

24


i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0 0 0







i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0 0






-26


i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0, 0.8 0





W=100 n=5before , weight=60i=4W[i]=50X[i]=0.8After, weight=110

27


i 1 2 3 4 5w 30 10 20 50 40

v 66 20 30 60 40

V/W 2.2 2.0 1.5 1.2 1.0

0 0, 1 0, 1 0, 1 0, 0.8 0





W=100 n=5before , weight=110Algorithm stops

28


(1) Sort the n objects from large to small based on the ratios vi/wi . We assume the arrays w[1..n] and v[1..n] store the respective weights and values after sorting.(2) initialize array x[1..n] to zeros. (3) weight = 0; i = 1 (4) while (i n and weight < W) do

(4.1) if weight + w[i] W then x[i] = 1 (4.2) else x[i] = (W – weight) / w[i] (4.3) weight = weight + x[i] * w[i] (4.4) i++

Algorithm (of time complexity O(n lgn))

29


If the items are already sorted into decreasing order of vi / wi, then the while-loop takes a time in O(n); Therefore, the total time including the sort is in O(n log n).If we keep the items in heap with largest vi/wi at the root. Then• creating the heap takes O(n) time• while-loop now takes O(log n) time (since heap property

must be restored after the removal of root)

Although this data structure does not alter the worst-case, it may be faster if only a small number of items are need to fill the knapsack.

Analysis of Fractional KnapSack

31

Huffman Code - Greedy

0

0

0

1

1

1

A

B

C D


Simplest type of strategy:

1. Take a step that makes the problem smaller. 2. iterate.

Difficulty: Prove that this leads to an optimal solution.

This is not always the case!

Idea

32

Computer Data Encoding: How do we represent data in binary?

Historical Solution:Fixed length codes.Encode every symbol by a unique binary string of a

fixed length. Examples: ASCII (7 bit code), EBCDIC (8 bit code), …

Example (that works) –Huffman code

33

American Standard Code for Information Interchange

3 -34

ASCII Example:

35

AABCAA

A A B C A A1000001 1000001 1000010 1000011 1000001 1000001

dead beef cafe deeded dad. dad faced a faded cab. dad acceded. dad be bad.

There are 12 a's, 4 b's, 5 c's, 19 d's, 12 e's, 4 f's, 17 spaces, and 4 periods, for a total of 77 characters.

Fix Lend Code• 77*3= 231

bits

Variable Length Code=

Total space usage in bits:

38

Assume an ℓ bit fixed length code.

For a file of n characters

Need nℓ bits.

Desiderata:

39

Construct a variable length code for a given file with the following properties:

1. Prefix code.2. Using shortest possible codes.3. Efficient.4. As close to entropy as possible.

Idea

40

Consider a binary tree, with: 0 meaning a left turn 1 meaning a right turn.

0

0

0

1

1

1

A

B

C D

Idea

41

Consider the paths from the root to each of the leaves A, B, C, D:

A : 0 B : 10 C : 110 D : 111

0

0

0

1

1

1

A

B

C D

Observe:

42

1. This is a prefix code, since each of the leaves has a path ending in it, without continuation.

2. If the tree is full then we are not “wasting” bits.

3. If we make sure that the more frequent symbols are closer to the root then they will have a smaller code.

0

0

0

1

1

1

A

B

C D

Revision-2 (7 Dec, 2014)43

Greedy Algorithm Example:

F E D C B A

60 50 40 30 20 10

Alphabet: A, B, C, D, E, F

Frequency table:

Total File Length: 210

48

http://www.cs.utexas.edu/users/djimenez/utsa/cs3343/lecture13.html

49

Huffman’s AlgorithmHuffman (C)

n = the size of Cinsert all the elements of C into Q,

using the value of the node as the priorityfor i in 1..n-1 do

z = a new tree nodex = Extract-Minimum (Q)y = Extract-Minimum (Q)left node of z = xright node of z = yf[z] = f[x] + f[y]Insert (Q, z)

end forreturn Extract-Minimum (Q) as the complete tree

Each priority queue operation (e.g. heap): O(log n)

In each iteration: one less subtree.

Initially: n subtrees.

Total: O(n log n) time.

50

Exercises

SOLVE BFS; DFS using Greedy WAY


Breadth-First Search (BFS):

BFS(v) // visit all nodes reachable from node v (1) create an empty FIFO queue Q, add node v to Q

(2) create a boolean array visited[1..n], initialize all values to false except for visited[v] to true(3) while Q is not empty (3.1) delete a node w from Q (3.2) for each node z adjacent from node w

if visited[z] is false then add node z to Q and set visited[z] to true

12 4

35

6

Node search order starting with node 1, including two nodes not reached

The time complexity is O(n+e) with n nodes and e edges, if the adjacency lists are used. This is because in the worst case, each node is added once to the queue (O(n) part), and each of its neighbors gets considered once (O(e) part).

Depth-First Search (DFS):

(1) create a boolean array visited[1..n], initialize all values to false except for visited[v] to true

(2) call DFS(v) to visit all nodes reachable via a path

DFS(v) for each neighboring nodes w of v do

if visited[w] is false then set visited[w] to true; call DFS(w) // recursive call

12 5

3 64

Node search order starting with node 1, including two nodes not reached

The algorithm’s time complexity is also O(n+e) using the same reasoning as in the BFS algorithm.


Minimum Spanning Tree


Definition: A tree is a connected undirected graph withno simple circuits.• Since a tree cannot have a simple circuit, a tree cannotcontain multiple edges or loops.• Therefore, any tree must be a simple graph.• Theorem: An undirected graph is a tree if and only ifthere is a unique simple path between any of itsvertices.

What is Tree

Yes Yes

No No


A spanning tree of a graph is just a subgraph thatcontains all the vertices and is a tree.A graph may have many spanning trees

On a connected graph G=(V, E), aspanning tree:1. • is a connected subgraph2. • acyclic3. is a tree (|E| = |V| - 1)4. contains all vertices of G5. (spanning)

Properties of spanning Tree

Spanning Trees

Suppose you have a connected undirected graph– Connected: every node is reachable from every other node– Undirected: edges do not have an associated direction• ...then a spanning tree of the graph is a connected subgraph inwhich there are no cycles• Total of 16 different spanning trees for the graph below

I can’t find others, can you? Make them 16


The Minimum Spanning Tree for a given graph is the Spanning Tree of minimum cost for that graph.

7

1

5 34

2

1

32

This is minimum spanning tree


Kruskal’s Prims

Finding Minimum Spanning Tree

Algorithms :

REVISION-2 (7 DEC, 2014) 59

8

1

5

7

4

2

6

Edge first1. Arrange all edges in a list (L) in non-decreasing order2. Select edges from L, and include that in set T, avoid cycle.3. Repeat 3 until T becomes a tree that covers all vertices

4

15

161313

14

14

12

14

15

16

1412

Kruskal’s algorithm


8

1

5

7

3

2

6

4

15

161313

14

14

12

14

15

16

1412

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16


8

1

5

7

4

2

6

3

15

161313

14

14

12

14

15

16

1412

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16


8

1

5

7

4

2

6

3

15

161313

14

14

12

14

15

16

1412

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

Skip, because it makes cycle


8

1

5

7

4

2

6

3

15

161313

14

14

12

14

15

16

1412

1,2 123,4 121,8 134,5 132,7 143,6 147,8 145,6 145,8 156,7 151,4 162,3 16

MST is formed, Stops


1

5

7

2

3

6

Prim’s algorithm

1.Start form any arbitrary vertex

2.Find the edge that has minimum weight form all known vertices

3.Stop when the tree covers all vertices

4

15

161313

14

14

12

14

15

16

1412


1

5

7

2

3

6

4

15

161313

14

14

12

14

15

16

1412

1

Question, All edges connected to 1, which is the shortest one? See next slide


1

5

7

2

3

6

4

15

161313

14

14 14

15

16

1412

1

3

12

Question, All edges connected to 1, 3 , which is the shortest one? See next slide


1

5

7

2

3

6

4

15

1613

14

14 14

15

16

1412

1

3

12

813

Question, All edges connected to 1, 3 ,8 which is the shortest one? See next slide


1

5

7

2

3

6

4

15

1613

14

14

15

16

1412

1

3

12

813

714

Question, All edges connected to 1, 3 , 7, 8 which is the shortest one? See next slideNote: if two edges has same length, choose any one of them until there is no cycle


1

5

7

2

3

6

4

1613

14

14

15

16

1412

1

3

12

813

714

515

Question, All edges connected to 1, 3 ,7,8 and 5 which is the shortest one? See next slide


1

5

7

2

3

6

4

16

14

14

15

16

1412

1

3

12

813

714

515

2

13


1

5

7

2

3

6

4

16

14

14

15

16

14

1

3

12

813

714

515

2

13

4

12


1

5

7

2

3

6

4

16

14

14

15

16

14

1

3

12

813

714

515

2

13

4

12

146

Stop, because you are done here


B

DC

G

A

F

E

HI

3 20

5

8

100

32

31

19

1411

44 14

99

54

Prim’s algorithm

1.Start form any arbitrary vertex

2.Find the edge that has minimum weight form all known vertices

3.Stop when the tree covers all vertices


B

DC

G

A

F

E

HI

3 20

5

8

100

32

31

19

1411

44 14

99

54

ACurrent:

Next Step:Consider all possible edges connected to Node A and choose the shortest edgeA= 3, 44

A= 3


B

DC

G

A

F

E

HI

20

5

8

100

32

31

19

1411

44 14

99

54

B

A

3 Previous Step

Next Step:Consider all possible edges connected to either Node A or Node B and choose the shortest edgeA= 44B= 20,54A= 20


B

DC

G

A

F

E

HI

5

8

100

32

31

19

1411

44 14

99

54

B

A

3Previous Step

Next Step:Consider all possible edges connected to either Node A , B, or C and choose the shortest edgeA= 44B= 54C= 5,100C= 5

C20


B

DC

G

A

F

E

HI

8

100

32

31

19

1411

44 14

99

54

B

A

3Previous Step

Next Step:Consider all possible edges connected to either Node A, B, C, D and choose the shortest edgeA= 44 B= 54, C= 100, D= 8,14D=8

C20

D5


B

DC

G

A

F

E

HI

100

32

31

19

1411

44

99

54

B

A

3Previous Step

C20

D5

F8

Next Step:Consider all possible edges connected to either Node A , B, C, D, F and choose the shortest edgeA= 44, B= 54, C= 100, D= 14, F= 32,99D=14

14


B

DC

G

A

F

E

HI

100

32

31

19

1411

44

99

54

B

A

3Previous Step

C20

D5

F8

E

14

Next Step:Consider all possible edges connected to either Node A , B, C, D, E, F and choose the shortest edgeA= 44 , B= 54, C= 100, D= 54, F= 32,99, 100 , E= 11, 44E=11


B

DC

G

A

F

E

HI

100

32

31

19

14

44

99

54

B

A

3Previous Step

C20

D5

F8

E

14

Next Step:Consider all possible edges connected to either Node A, B, C, D, E, F, H and choose the shortest edgeA= 44 , B= 54, C= 100, D= 54, E= 44, F= 32,99,100, H= 14,19, E= 44H=14

H

11


B

DC

G

A

F

E

HI

100

32

31

44

99

54

B

A

3Previous Step

C20

D5

F8

E

14

Next Step:Consider all possible edges connected to either Node A,B, C, D, E, F, G, H and choose the shortest edgeA= 44 , B= 54, C=100, D=54, E= 44, 99, F= 32, 99, 100, H= 19, G= 31, 32, H=19

H

11G

19

14


B

DC

G

A

F

E

HI

100

32

3114

44

99

54

B

A

3Previous Step

C20

D5

F8

E

14

Next Step: nothing, nodes are occupied

H

11G

19

19

I14


B

DC

G

A

F

E

HI

3 20

5

8

100

32

31

19

1411

44 14

99

54Practice the same example But with different vertex selection, choose D in this case


B

DC

G

A

F

E

HI

3 20

5

8

100

32

31

19

1411

44 14

99

54

D


B

DC

G

A

F

E

HI

3 20

8

100

32

31

19

1411

44 14

99

54

DC5

Find the edge that has minimumweight form all known vertices


B

DC

G

A

F

E

HI

3 20

100

32

31

19

1411

44 14

99

54

DC5

F8



B

DC

G

A

F

E

HI

3 20

100

32

31

19

1411

44

99

54

DC5

F14

E

8



B

DC

G

A

F

E

HI

3 20

100

32

31

19

14

44

99

54

DC5

F14

E

8

H

11 Find the edge that has minimumweight form all known vertices


B

DC

G

A

F

E

HI

3 20

100

32

31

19

44

99

54

DC5

F14

E

8

H

11G

14



B

DC

G

A

F

E

HI

3 20

100

32

31

44

99

54

DC5

F14

E

8

H

11G

14

I19



B

DC

G

A

F

E

HI

3

100

32

31

44

99

54

DC5

F14

E

8

H

11G

14

I19

20B



B

DC

G

A

F

E

HI

100

32

31

44

99

54

DC5

F14

E

8

H

11G

14

I19

20B

A

3


Compare Prim and Kruskal Complexity

Kruskal: O(NlogN)comparison sort for edges

Prim: O(NlogN)search the least weight edge for every vertice

Revision-2 (7 Dec, 2014) 100

Why do we need MST?

• a reasonable way for clustering points in space into naturalgroups• can be used to give approximate solutions to hard problemsSuppose you want to provide solution to :– electric power, Water, telephone lines,network setup• To minimize cost, you could connect locations using MST• However, MST is not necessary the shortest path and it does not apply to cycle

Revision-2 (7 Dec, 2014) 101

MST don’t solve TSP

• Travel salesman problem (TSP) can notbe solved by MST salesman needs to go home (what’s the cost going home?)• TSP is a cycle ( Bad News)• Use MST to approximate• solve TSP by exhaustive approach- try every

permutation on cyclic graph

Greedy algo revision 2

Education

Transcript of Greedy algo revision 2