Greedy Algorithm

CSC2105: AlgorithmsCSC2105: AlgorithmsGreedy MethodGreedy Method

Mashiour RahmanMashiour [email protected]@aiub.edu

American International University BangladeshAmerican International University Bangladesh

Mashiour Rahman

AIUB::CSC2105::Algorithms Greedy 2

The Greedy Method The Greedy Method The greedy methodThe greedy method is a general algorithm design paradigm, is a general algorithm design paradigm, built on the following elements:built on the following elements:

configurations: different choices, collections, or values to find

an objective function: a score assigned to configurations, which we want to either maximize or minimize

It is usually used for optimization problem. That is, problem that It is usually used for optimization problem. That is, problem that required to find a best solution. In making a decision, required to find a best solution. In making a decision, a greedy a greedy algorithm always chooses the option that looks best at the algorithm always chooses the option that looks best at the momentmoment..

It works best when applied to problems with the It works best when applied to problems with the greedy-greedy-choicechoice property: property: A globally-optimal solution can always be A globally-optimal solution can always be found by a series of local improvements from a starting found by a series of local improvements from a starting configuration.configuration.

Mashiour Rahman


Greedy StepsGreedy StepsOn each step in the algorithm, the choice must be:On each step in the algorithm, the choice must be:

FeasibleFeasible - i.e. it satisfies the problems constraints.

Locally optimalLocally optimal – i.e. it has to be the best local choice among all feasible choices available at the step.

Irrevocable Irrevocable – i.e. once made, it cannot be changed on subsequent steps of the algorithm.

““Greed, for lack of a better word, is good! Greed, for lack of a better word, is good! Greed is right! Greed works!”Greed is right! Greed works!”

Mashiour Rahman


General form of Greedy General form of Greedy MethodMethod

In general, a greedy algorithm is in the form:In general, a greedy algorithm is in the form:

Let A be the empty set;

Repeat

Choose the “best” c such that

A +{c} is a feasible solution;

Insert c into A;

Until A is a solution.

We call We call cc a a candidatecandidate..

If a set of candidate If a set of candidate AA could be extended to a solution, then could be extended to a solution, then we say that we say that AA is is feasiblefeasible..

A feasible set is A feasible set is promisingpromising if it could be extended to the if it could be extended to the optimal solution. optimal solution.

Mashiour Rahman


Making ChangeMaking ChangeProblem: A dollar amount to reach and a collection of coin amounts to use to get Problem: A dollar amount to reach and a collection of coin amounts to use to get there.there.

Configuration: Configuration:

Input: Given an integer Input: Given an integer SS. A . A solutionsolution is a set of coins such that the total is equal is a set of coins such that the total is equal SS. . The coins must be drawn from { 1, 5, 10, 25} cents.The coins must be drawn from { 1, 5, 10, 25} cents.

Output: The solution which has minimum number of coins.Output: The solution which has minimum number of coins.

Objective function: Minimize number of coins returned.Objective function: Minimize number of coins returned.

Greedy solution:Greedy solution:

Let A be the empty set.

Repeat

let c be the highest valued coin such that

c + total_amount (A) <= S.

Insert c into A.

Until total_amount ( A ) = S.

S=36 {25,10,1}S=36 {25,10,1}

S=24 {10,10,1,1,1,1}S=24 {10,10,1,1,1,1}

Mashiour Rahman


Proof by inductionProof by induction..

First note that First note that AA is always is always feasiblefeasible. This is because we can always use the 1-cent . This is because we can always use the 1-cent coin to top-up.coin to top-up.

basis:basis: AA= ø is promising.= ø is promising.

hypo:hypo: AA is promising. is promising.

step:step:

Suppose Suppose AA is promising. Let is promising. Let cc be the highest candidate. We want to show by be the highest candidate. We want to show by contradiction that contradiction that A+{c}A+{c} is promising. If is promising. If A +{c}A +{c} is not promising, is not promising,

since since AA is promising, then there exist a set is promising, then there exist a set BB of coins, such that of coins, such that

(a) (a) A+BA+B is promising; is promising;

(b) all coins in (b) all coins in BB is smaller than is smaller than cc, and , and

(c) total value of (c) total value of BB is larger or equal to is larger or equal to cc..

Suppose Suppose cc=5, then from (b) & (c), the only possible =5, then from (b) & (c), the only possible BB is is

{1,1,1,1,1}, {1,1,1,1,1,1},.......{1,1,1,1,1}, {1,1,1,1,1,1},.......

In each case, we can replace a number of coins by a 5-cents coin.In each case, we can replace a number of coins by a 5-cents coin.

This contradict the fact that This contradict the fact that A+BA+B is promising. is promising.

ClaimClaim: in the greedy algorithm, the : in the greedy algorithm, the set Aset A is always is always promising.promising.

Mashiour Rahman


(a) (a) A+BA+B is promising; is promising;

(b) all coins in (b) all coins in BB is smaller than is smaller than cc, and , and

(c) total value of (c) total value of BB is larger or equal to is larger or equal to cc..

Suppose Suppose cc=10, then from (b) & (c) the only possible =10, then from (b) & (c) the only possible BB is is

•{5,5}, {5,1,1,1,1,1}, {5,5,5,...}{5,5}, {5,1,1,1,1,1}, {5,5,5,...}

•In each case, we can replace a number of coins by a 10-cents coin.In each case, we can replace a number of coins by a 10-cents coin.

•This contradict the fact that This contradict the fact that A+BA+B is promising. is promising.

Suppose Suppose cc=25, then from (b) & (c) the only possible =25, then from (b) & (c) the only possible BB is is

•{10,10,5}, {10,10,10}, {10,5,5,5},....{10,10,5}, {10,10,10}, {10,5,5,5},....

•If If BB contains at most two 10-cents, then by (c), there must be sufficient coins contains at most two 10-cents, then by (c), there must be sufficient coins in in BB to sum more than or equal to 25 cents. In fact, there are coins to make to sum more than or equal to 25 cents. In fact, there are coins to make up exactly 25 cents. These coins can be replaced by a single 25-cents coin. up exactly 25 cents. These coins can be replaced by a single 25-cents coin.

•This contradicts the fact that This contradicts the fact that A+BA+B is promising. is promising.

•If If BB contains three or more 10-cents coins, we can replace 3 of them with 2 contains three or more 10-cents coins, we can replace 3 of them with 2 coins: 25-cents and 5-cents. coins: 25-cents and 5-cents.

•Thus contradicting the fact that Thus contradicting the fact that A+BA+B is promising. is promising.

ClaimClaim: in the greedy algorithm, the set A is always : in the greedy algorithm, the set A is always promising.promising.

Mashiour Rahman


The Fractional Knapsack The Fractional Knapsack ProblemProblem

Given:Given: A set A set SS of of nn items, with each item items, with each item ii having having

bi - a positive benefit

wi - a positive weight

Goal:Goal: Choose items with maximum total value but with weight at most Choose items with maximum total value but with weight at most WW..

The value of an item is its benefit/weight ratio.

If we are allowed to take fractional amounts, then this is the If we are allowed to take fractional amounts, then this is the fractional fractional knapsack problemknapsack problem..

let xi denote the amount we take of item i, 0 xi wi

Objective: maximize

Constraint:

Si

iii wbx )/(

Si

i Wx

Mashiour Rahman


ExampleExampleGiven:Given: A set A set SS of of nn items, with each item items, with each item ii having having

bi - a positive benefit

wi - a positive weight

Goal:Goal: Choose items with maximum total value but with weight Choose items with maximum total value but with weight at most at most WW..

Weight:Weight:Benefit:Benefit:

11 22 33 44 55

4 ml4 ml 8 ml8 ml 2 ml2 ml 6 ml6 ml 1 ml1 ml

$12$12 $32$32 $40$40 $30$30 $50$50

Items:Items:

Value:Value: 33($ per ml)($ per ml)

44 2020 55 505010 ml

Solution:Solution:• 1 ml of 51 ml of 5• 2 ml of 32 ml of 3• 6 ml of 46 ml of 4• 1 ml of 21 ml of 2

““knapsack”knapsack”

Mashiour Rahman


The Fractional Knapsack The Fractional Knapsack AlgorithmAlgorithm

Greedy choice:Greedy choice: Keep taking item with highest Keep taking item with highest valuevalue (benefit / weight ratio (benefit / weight ratio bbii / w / wii ))

Run time:Run time: O(n log n). Why? O(n log n). Why?

Use a max-heap priority queueUse a max-heap priority queue

Algorithm fractionalKnapsack(S, W)

Input: set S of items with benefit bi and weight wi; maximum weight W

Output: amount xi of each item i to maximize benefit with weight at most W

for each item i in S

xi 0vi bi / wi {value}

w 0 {total weight}

while w < W

remove item i with highest vi

xi min{wi , W - w}

w w + min{wi , W - w}

Mashiour Rahman


Task SchedulingTask SchedulingGiven:Given: a set a set TT of of nn tasks, each having: tasks, each having:

A start time, si

A finish time, fi (where si < fi)

Goal:Goal: Perform all the tasks using a minimum number of Perform all the tasks using a minimum number of “machines.”“machines.”

1 98765432

Machine 1

Machine 3

Machine 2

Mashiour Rahman


ExampleExampleGiven:Given: a set a set TT of of nn tasks, each having: tasks, each having:

A start time, si

A finish time, fi (where si < fi)

[1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start)

Goal:Goal: Perform all tasks on minimum number of machines Perform all tasks on minimum number of machines

1 98765432

Machine 1

Machine 3

Machine 2

Mashiour Rahman


Task Scheduling Task Scheduling AlgorithmAlgorithm

Greedy choice:Greedy choice: consider tasks by their start time and use as few machines as consider tasks by their start time and use as few machines as possible with this order.possible with this order.

Run time:Run time: O(n lg n). Why?

Correctness:Correctness: Suppose there is a better schedule. Suppose there is a better schedule.

We can use k-1k-1 machines

The algorithm uses kk

Let ii be first task scheduled on machine kk

Machine ii must conflict with k-1k-1 other tasks

But that means there is no non-conflicting schedule using k-1k-1 machines

AlgorithmAlgorithm taskScheduletaskSchedule((TT))

Input:Input: set set TT of tasks with start time of tasks with start time ssii and finish time and finish time ffiiOutput:Output: non-conflicting schedule with minimum number of non-conflicting schedule with minimum number of machinesmachines

mm 00 {no. of machines}{no. of machines}whilewhile T is not empty T is not empty dodo

remove task i with smallest sremove task i with smallest sii

if if there’s a machine j for i there’s a machine j for i thenthenschedule i on machine jschedule i on machine j

elseelse mm m + 1m + 1schedule i on machine mschedule i on machine m

Greedy Algorithm

Documents

Transcript of Greedy Algorithm