Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za...
-
Upload
ezra-tyler -
Category
Documents
-
view
220 -
download
1
Transcript of Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za...
![Page 1: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/1.jpg)
Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques
Theory of Algorithms:Theory of Algorithms:Greedy TechniquesGreedy Techniques
James Gain and Edwin Blake{jgain | edwin} @cs.uct.ac.za
Department of Computer Science
University of Cape Town
August - October 2004
![Page 2: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/2.jpg)
ObjectivesObjectives
To introduce the mind set of greedy techniques
To show some examples of greedy algorithms: Change Making
Huffman Coding
To discuss the strengths and weaknesses of greedy techniques
“Greed, for lack of a better word, is good! Greed is right! Greed works!” Gordon Grecko (Michael Douglas) in Wall Street
![Page 3: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/3.jpg)
Greedy AlgorithmsGreedy Algorithms
Optimization problems solved through a sequence of choices that are:1. Feasible - satisfy problem constraints
2. Locally Optimal - best choice among all feasible options for that step
3. Irrevocable - no backing out
A Greedy grab of the best alternative, hoping that a sequence of locally optimal steps will lead to a globally optimal solution
Even if not optimal, sometimes an approximation is acceptable
Not all optimization problems can be approached in this manner!
![Page 4: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/4.jpg)
Applications of the Greedy StrategyApplications of the Greedy Strategy
Optimal solutions: Change making Minimum Spanning Trees (MST) - Prim’s and Kruskal’s
Algorithms Single-source shortest paths - Dijkstra’s Algorithm Simple scheduling problems Huffman codes
Approximations: Traveling Salesman Problem (TSP) Knapsack problem Other combinatorial optimization problems
![Page 5: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/5.jpg)
Change MakingChange Making
Problem: give change for a specific amount n, with the least number of coins of the denominations d1 > d2 > … > dm
Example: Smallest change for R2.54 using R5,
R2, R1, 50c, 20c, 10c, 5c, 2c, 1c R2 + 50c + 2c + 2c = R2.54
Algorithm: At any step choose the coin of the largest denomination
that doesn’t exceed the remaining total; Repeat
Note: Optimal for reasonable sets of coins
![Page 6: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/6.jpg)
Minimum Spanning Tree ProblemMinimum Spanning Tree Problem
Problem: Given n points, connect them in the cheapest way so that there is a path between any pair of points
Spanning Tree (of a connected graph G ): A connected acyclic subgraph of G that includes all of G’s vertices.
Minimum Spanning Tree (of a weighted graph G ): A spanning tree of G with minimum total weight
Algorithms: Prim’s and Kruskal’s Algorithms Example: 3
42
14
26
1
3
3
42
1
2
1
3
![Page 7: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/7.jpg)
Shortest Paths ProblemShortest Paths Problem
Single Source Shortest Paths Problem: Given a weighted graph G, find the shortest paths from
a source vertex s to each of the other vertices Solution:
Dijkstra’s Algorithm (a relative of Prim’s MST) for positive weights
3
42
14
26
1
3
2
1
2
42
1
2
3
31
4
![Page 8: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/8.jpg)
Text CompressionText Compression
Variable length encoding: Compresses text by assigning codes of different length to characters
based on their probability of occurrence
Used by Samuel Morse in constructing telegraph codes
Prefix-free Codes: Codes are unique in that no code is a prefix for a code of another
character
Tree for Binary Codes: Leaves are characters
Left edge codes 0, right edge codes 1
The code of a character is a simple walk from root to leaf
![Page 9: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/9.jpg)
Algorithm for Huffman CodingAlgorithm for Huffman Coding
Invented by David Huffman as part of a class assignment while he was an undergraduate
Algorithm: Construct a Huffman Tree that assigns shorter strings to higher
frequencies. This defines a Huffman Code
1. Initialize n one-node trees labeled with the characters of the alphabet. Record the frequency (weight) of each character in the root
2. REPEAT until a single tree is obtained:
Find two trees with the smallest weight
Make them the left and right sub-tree of a new tree and record the sum of their weights
in the root
![Page 10: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/10.jpg)
Example: Huffman CodingExample: Huffman Coding
0.55C0.15
A0.4
B0.45
C0.15
A0.4
B0.45
0.55
C0.15
A0.4
B0.45
1.0
Char Code
A 01
B 1
C 00
![Page 11: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/11.jpg)
Notes on Huffman CodingNotes on Huffman Coding
Compression Ratio: Standard measure of compression
CR = (x - y) / y * 100%, where x is compressed and y is uncompressed
Typically, 20-80% in the case of Huffman Coding
Yields optimal compression provided: The probabilities of character occurrences are
independent
Probabilities are known in advance
![Page 12: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/12.jpg)
Strengths and Weaknesses of Strengths and Weaknesses of Greedy TechniquesGreedy Techniques
Strengths: Intuitively simple and appealing
Weaknesses: Only applicable to optimization problems
Doesn’t always produce an optimal solution
![Page 13: Theory of Algorithms: Greedy Techniques James Gain and Edwin Blake {jgain | edwin} @cs.uct.ac.za Department of Computer Science University of Cape Town.](https://reader035.fdocuments.in/reader035/viewer/2022062805/5697c0071a28abf838cc5e2b/html5/thumbnails/13.jpg)
Summary To-DateSummary To-Date
Introduction 1.1-1.4
Fundamentals of the Analysis of Algorithm Efficiency
2.1-2.4, 2.6, 2.7
Brute Force 3.1-3.4
Divide-and-Conquer 4.1-4.3, 4.5, 4.6
Decrease-and-Conquer 5.1-5.6
Transform-and-Conquer 6.1, 6.5, 6.6
Space and Time Tradeoffs 7.1-7.2
Greedy Techniques 9.4