Design and Analysis of Computer Algorithm Lecture 5-1
description
Transcript of Design and Analysis of Computer Algorithm Lecture 5-1
April 22, 2023
Design and Analysis of Computer AlgorithmLecture 5-1
Pradondet NilaguptaDepartment of Computer Engineering
This lecture note has been modified from lecture note for 23250 by Prof. Francis Chin
Design and Analysis of Computer Algorithm 2April 22, 2023
Greedy Method
Design and Analysis of Computer Algorithm 3April 22, 2023
Topics Cover
The General Method Activity -Selection Problem Optimal storage on Tapes Knapsack problem Minimal Spanning Tree Single Source shortest paths
Design and Analysis of Computer Algorithm 4April 22, 2023
Greedy Method: Definition
An algorithm which always takes the best immediate, or local, solution while finding an answer. Greedy algorithms will always find the overall, or globally, optimal solution for some optimization problems, but may find less-than-optimal solutions for some instances of other problems.
Design and Analysis of Computer Algorithm 5April 22, 2023
Example of Greedy Method (1/4)
Prim's algorithm and Kruskal's algorithm are greedy algorithms which find the globally optimal solution, a minimum spanning tree. In contrast, any known greedy algorithm to find an Euler cycle might not find the shortest path, that is, a solution to the traveling salesman problem.
Dijkstra's algorithm for finding shortest paths is another example of a greedy algorithm which finds an optimal solution.
Design and Analysis of Computer Algorithm 6April 22, 2023
Example of Greedy Method (2/4)
If there is no greedy algorithm which always finds the optimal solution for a problem, one may have to search (exponentially) many possible solutions to find the optimum. Greedy algorithms are usually quicker, since they don't consider possible alternatives.
Design and Analysis of Computer Algorithm 7April 22, 2023
Example of Greedy Method (3/4)
Consider the problem of making change: Coins of values 25c, 10c, 5c and 1c Return 63c in change
– Which coins? Use greedy strategy:
– Select largest coin whose value was no greater than 63c
– Subtract value (25c) from 63 getting 38– Find largest coin … until done
Design and Analysis of Computer Algorithm 8April 22, 2023
Example of Greedy Method (4/4)
At any individual stage, select that option which is “locally optimal” in some particular sense
Greedy strategy for making change works because of special property of coins
If coins were 1c, 5c and 11c and we need to make change of 15c?– Greedy strategy would select 11c coin followed by
4 1c coins– Better: 3 5c coins
Design and Analysis of Computer Algorithm 9April 22, 2023
Greedy Algorithm
Start with a solution to a small subproblem Build up to a solution to the whole problem Make choices that look good in the short term
Disadvantage: Greedy algorithms don’t always work ( Short term solutions can be diastrous in the long term). Hard to prove correct
Advantage: Greedy algorithm work fast when they work. Simple algorithm, easy to implement
Design and Analysis of Computer Algorithm 10April 22, 2023
Greedy Algorithm
Procedure GREEDY(A,n)// A(1:n) contains the n inputs//
solution //initialize the solution to empty//for i 1 to n do
x SELECT(A)if FEASIBLE(solution,x)
then solution UNION(solution,x)endifrepeatreturn(solution)
end GREEDY
Design and Analysis of Computer Algorithm 11April 22, 2023
Activity-Selection Problem
The problem is to select a maximum-size set of mutally compatible activities.
Example We have a set S = { 1,2,…,n} of n proposed activities t
hat wish to use a resource, such as a lecture hall, which can be used by only one activities at a time.
Design and Analysis of Computer Algorithm 12April 22, 2023
Example
i si fi
1 0 62 3 53 1 44 2 135 3 86 12 147 8 118 8 129 6 1010 5 711 5 9
Design and Analysis of Computer Algorithm 13April 22, 2023
Brute Force
Try every all possible solution Choose the largest subset which is feasible Ineffcient (2n) choices
Design and Analysis of Computer Algorithm 14April 22, 2023
Greedy Approach
Sort by finish time
Design and Analysis of Computer Algorithm 15April 22, 2023
Activity-Selection Problem Pseudo code
Greedy_Activity_Selector(s,f)1 n <- length[s]2 A <- {1}3 j <- 14 for i <- 2 to n5 do if si > fj
6 then A <- A U {i}7 j <- i8 return A
It can schdule a set S of n activities in (n) time, assuming that the activities were already sorted
Design and Analysis of Computer Algorithm 16April 22, 2023
Proving the greedy algorithm correct
We assume that the input activities are in order by increasing finishing time
f1 < f2 < … < fn Activities #1 has the earliest finish time then it must b
e in an optimal solution.
k
1
1 possible solution
Activitiy 1
Design and Analysis of Computer Algorithm 17April 22, 2023
Proving (cont.)
k
1
Eliminate the activities which has a start time early than the finish time of activity 1
Design and Analysis of Computer Algorithm 18April 22, 2023
Proving (cont.)
1
Greedy algorithm produces an optimal solution
Design and Analysis of Computer Algorithm 19April 22, 2023
Element of the Greedy Strategy
Question? How can one tell if a greedy algorithm will solve
a particular optimization problem? No general way to tell!!! There are 2 ingredients that exhibited by most
problems that lend themselves to a greedy strategy– The Greedy Choice Property– Optimal Substructure
Design and Analysis of Computer Algorithm 20April 22, 2023
The Greedy Choice Property
A globally optimal solution can be arrived at by making a locally optimal (greedy) choice.
Make whatever choice seems best at the moment. May depend on choice so far, but not depend on any
future choices or on the solutions to subproblems
Design and Analysis of Computer Algorithm 21April 22, 2023
Optimal Substructure
An optimal solution to the problem contains within it optimal solutions to subproblems
Design and Analysis of Computer Algorithm 22April 22, 2023
Optimal Storage on Tapes
There are n programs that are to be stored on a computer tape of length L.
Each program i has a length li , 1 i n All programs are retrieved equally often, the expected
or mean retrieval time (MRT) is
nj
jtn1
)/1(
Design and Analysis of Computer Algorithm 23April 22, 2023
Optimal Storage on Tapes (cont.)
We are required to find a permutation for the n programs so that when they are stored on tape in the order the MRT is minimized.
Minimizing the MRT is equivalent to minimizing
jknjliID
k11
)(
Design and Analysis of Computer Algorithm 24April 22, 2023
Example
Let n = 3 and (l1,l2,l3) = (5,10,3)
Ordering I D(I)1,2,3 5 + 5 + 10 + 5 + 10 + 3 = 381,3,2 5 + 5 + 3 + 5 + 3 + 10 = 312,1,3 10 + 10 + 5 + 10 + 5 + 3 = 432,3,1 10 + 10 + 3 + 10 + 3 + 5 = 413,1,2 3 + 3 + 5 + 3 + 5 + 10 = 293,2,1 3 + 3 + 10 + 3 + 10 + 5 = 34
Design and Analysis of Computer Algorithm 25April 22, 2023
The Greedy Solution
Make tape emptyfor i := 1 to n do
grab the next shortest fileput it next on tape
The algorithm takes the best short term choice without checking to see weather it is the best long term decision.
Design and Analysis of Computer Algorithm 26April 22, 2023
Optimal Storage on Tapes (cont.)
Theorem 4.1 If l1 l2 … ln then the ordering ij = j, 1 j n
minimizes
Over all possible permutation of the ij See proof on text pp.154-155
n
k
k
jli j1 1
Design and Analysis of Computer Algorithm 27April 22, 2023
Knapsack Problem
We are given n objects and a knapsack. Object i has a weight wi and the knapsack has a capacity M.
If a fraction xi, 0 xi 1, of object I is placed into the knapsack the a profit of pixi is earned.
The objective is to obtain a filling of the knapsack that maximizes the total weight of all chosen objects to be at most M
maximize
subject to
and 0 xi 1, 1 I n
ni
iixp1
Mxwni
ii 1
Design and Analysis of Computer Algorithm 28April 22, 2023
Example
1020
30
50
$60 $100 $120
Item 1
Item 2
Item 3
knapsack
Design and Analysis of Computer Algorithm 29April 22, 2023
Knapsack 0/1
30
20
$120
$100
Total =$220
20
10
$100
$60
=$160
30
10
$120
$60
=$180
Design and Analysis of Computer Algorithm 30April 22, 2023
Fractional Knapsack
Taking the items in order of greatest value per pound yields an optimal solution
20
10
$100
$60
=$240Total
2030
$80
Design and Analysis of Computer Algorithm 31April 22, 2023
Optimal Substructure
Both fractional knapsack and 0/1 knapsack have an optimal substructure.
Design and Analysis of Computer Algorithm 32April 22, 2023
Example Fractional Knapsack (cont.)
There are 5 objects that have a price and weight list below, the knapsack can contain at most 100 Lbs.
Method 1 choose the least weight first– Total Weight = 10 + 20 + 30 + 40 = 100
– Total Price = 20 + 30 + 66 + 40 = 156
Price ($US) 20 30 66 40 60weight (Lbs.) 10 20 30 40 50
Design and Analysis of Computer Algorithm 33April 22, 2023
Example Fractional Knapsack (cont.)
Method 2 choose the most expensive first– Total Weight = 30 + 50 + 20 = 100
– Total Price = 66 + 60 + 20 = 146
Price ($US) 20 30 66 40 60weight (Lbs.) 10 20 30 40 50
half
Design and Analysis of Computer Algorithm 34April 22, 2023
Example Fractional Knapsack (cont.)
Method 3 choose the most price/ weight first– Total weight = 30 + 10 + 20 + 40 = 100
– Total Price = 66 + 20 + 30 + 48 = 164
Price ($US) 20 30 66 40 60weight (Lbs.) 10 20 30 40 50price/weight 2 1.5 2.2 1 1.2
Design and Analysis of Computer Algorithm 35April 22, 2023
More Example on fractional knapsac
Consider the following instance of the knapsack problem: n = 3, M = 20, (p1,p2,p3) = 25,24,15 and (w1,w2,w3) = (18,15,10)
(x1,x2,x3)1) (1/2,1/3,1/4) 16.5 24.252) (1,2/15,0) 20 28.23) ( 0,2/3,1) 20 314) ( 0,1,1/2) 20 31.5
iixw iixp
Design and Analysis of Computer Algorithm 36April 22, 2023
The Greedy Solution
Define the density of object Ai to be wi/si. Use as much of low density objects as possible. That is, process each in increasing order of density. If the whole thing ts, use all of it. If not, fill the remaining space with a fraction of the current object,and discard the rest.
First, sort the objects in nondecreasing density, so that wi/si w i+1/s i+1 for 1 i < n.
Then do the following
Design and Analysis of Computer Algorithm 37April 22, 2023
PseudoCode
Procedure GREEDY_KNAPSACK(P,W,M,X,n)//P(1:n) and W(1:n) contain the profits and weights respectively of the n obj
ects ordered so that P(I)/W(I) > P(I+1)/W(I+1). M is the knapsack size and X(1:n) is the solution vector//
real P(1:n), W(1:n), X(1:n), M, cu;integer I,n;
x ; //initialize solution to zero //cu M; // cu = remaining knapsack capacity //for i to n do
if W(i) > cu then exit endifX(I) 1;cu c - W(i);
repeatif I < n then X(I) cu/W(I) endif
End GREEDY_KNAPSACK
Design and Analysis of Computer Algorithm 38April 22, 2023
Proving Optimality
Let p1/w1 > p2/w2 > … > pn/wn
Let X = (x1,x2,…,xn) be the solution generated by GREEDY_KNAPSACK
Let Y = (y1,y2,…,yn) be any feasible solution We want to show that
0p)yx(n
1iiii
Design and Analysis of Computer Algorithm 39April 22, 2023
Proving Optimality (cont.)
If all the xi are 1, then the solution is clearly optimal (It is the only solution) otherwise, let k be the smallest number such that xk < 1.
1 1 1 1 1 1 1 1 1 1 1 1
1 2 n
1 1 .. .. 1 xk0 0 .. .. 0 0
1 2 nk
Design and Analysis of Computer Algorithm 40April 22, 2023
Proving Optimality (cont.)
n
1iiii p)yx(
1 1 .. .. 1 xk0 0 .. .. 0 0
1 2 nk
1k
1i i
iiii wpw)yx(
k
kkkk w
pw)yx(
n
1ki i
iiii wpw)yx(
Design and Analysis of Computer Algorithm 41April 22, 2023
Proving Optimality (cont.)
Consider each of these block
1k
1i i
iiii wpw)yx(
1k
1i k
kiii wpw)yx(
k
kkkk w
pw)yx( k
kkkk w
pw)yx(
n
1ki i
iiii wpw)yx(
n
1ki k
kiii wpw)yx(
Design and Analysis of Computer Algorithm 42April 22, 2023
Proving Optimality (cont.)
n
1iiii p)yx(
n
1i k
kiii wpw)yx(
n
1iiii
k
k w)yx(wp
= W0p)yx(
n
1iiii
Since W always > 0, therefore