Algorithm Design - Heinz Nixdorf Institut · 2012-05-31 · Friedhelm Meyer auf der Heide 1 HEINZ...
Transcript of Algorithm Design - Heinz Nixdorf Institut · 2012-05-31 · Friedhelm Meyer auf der Heide 1 HEINZ...
Friedhelm Meyer auf der Heide 1
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Welcome to the course
Algorithm Design
Summer Term 2011
Friedhelm Meyer auf der Heide
Lecture 13, 15.7.2011
Friedhelm Meyer auf der Heide 2
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityTopics
- Divide & conquer
- Dynamic programming
- Greedy Algorithms
- Randomized Algorithms
- Approximation Algorithms
- Online Algorithms
Friedhelm Meyer auf der Heide 3
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 1:
Divide & Conquer
Friedhelm Meyer auf der Heide 4
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityDivide & Conquer
Divide the problem in a number of smaller subproblems.
Conquer the subproblems by solving them recursively.
Combine the results of the subproblem into a solution
of the original problem.
Our Examples:
Integer multiplication, matrix multiplication (HW), closest pair
of points in the plane, Fast Fourier transform, ….(HW)
Friedhelm Meyer auf der Heide 5
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 2:
Dynamic Progamming
Friedhelm Meyer auf der Heide 6
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityDynamic programming
You probably know dynamic programming algorithms for
optimal evaluation of the product of many matrices, and for the
word problem for context-free languages (CYK-algorithm)
Examples:
String Distance, Shortest Paths, Knapsack, Minimum Weight
Triangulation (HW), Optimal Search Trees (HW)
Friedhelm Meyer auf der Heide 7
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 8
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 9
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 10
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 11
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 3:
Greedy Algorithms
Friedhelm Meyer auf der Heide 12
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityGreedy Algorithms
Examples:Activity Selection (or Interval Scheduling), Scheduling to Minimize Lateness
Theoretical Foundations of the Greedy Method: Matroid TheoryExamples for Matroids: Linear independent vectors, Minimum Spanning Trees, Bipartite matching with maximum left side
Friedhelm Meyer auf der Heide 13
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
For which problems are Greedy-Algorithms
optimal?
Consider a finite set E and a system U of subsets of E. (E,U) is called a
subset-system, if the following holds:
(i) ? 2 U
(ii) For each B 2 U, also each subset of B is in U.
B2 U is called maximal, if no proper superset of B is in U.
The Optimization problem corresponding to (E,U) is :
Given a weight function w:EQ+ ,compute a maximal set B 2 U with
maximizes w(B) = §e2 B w(e).
(Minimization problems are described analagously.)
Friedhelm Meyer auf der Heide 14
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Systems of Subsets and the
Canonical Greedy Algorithm
Canonical Greedy ((E,U))
(1) Sort E such that w(e1) ¸ … ¸ w(en).
(2) B?.
(3) For k=1 to n
if B [ {ek} 2 U then B B [ {ek}
(4) Return B
For which types of subset-systems is the canonical Greedy
algorithm optimal?
Friedhelm Meyer auf der Heide 15
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityExample 2
Problem: Minimum spanning tree (MST)
Input: a graph G=(V,E) with positive edge weights w:EQ+ .
Output: a spanning tree with minimum weight.
MST is a subset-system:
- E : the set of edges,
- U : all subsets of E that do not contain a cycle
form a subset-system
Friedhelm Meyer auf der Heide 16
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityGreedy-Algorithms und Matroids
A system of subsets (E,U) is a matroid, if in addition, the following exchange
property holds:
(iii) For all A,B 2 U with |A|<|B|, there is x2 B-A such that A[{x} 2 U.
Remark: All maximal sets of a matroid have the same size. (homework)
We will see:
- The system of subsets for 0-1 knapsack (Example 1) is not a matroid.
- The systems of subsets for examples 0,2,3 are a matroids.
Theorem: Consider a system of subsets (E,U). canonical Greedy is optimal
for (E,U) for every weight function w, if and only if (E,U) is a matroid.
Kruskal‟s algorithm is optimal for MST (example 2)!
Friedhelm Meyer auf der Heide 17
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 4.
Randomised Algorithms
Friedhelm Meyer auf der Heide 18
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityRandomized Algorithms
… use the outcome of random experiments (“coin flips”) to guide the execution of the algorithm.
For a fixed input, runtime and output may be random variables.
Friedhelm Meyer auf der Heide 19
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityRandomized Algorithms
Examples:
Quicksort, Skip Lists, Contention Resolution (symmetry -breaking), Global Minimum Cut (contraction algorithm), Random Variables and their Expectations (Guessing Cards, Coupon Collector)
Chernoff Bounds, example Load Balancing
Friedhelm Meyer auf der Heide 20
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityCuts in graphs
Let G=(V,E) be a connected, undirected graph.
A cut in G is defined by a disjoint partition of V into two sets A
and B. Its size is the number of edges between A and B.
Goal: Compute a cut with minimum size.
A cut of size 6.
A B
Friedhelm Meyer auf der Heide 21
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 22
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 23
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityTools from Probability Theory: Chernoff Bounds
Prob(|X-¹|)¸ ± ¹) is „inverse exponential“ in n !!!!
Friedhelm Meyer auf der Heide 24
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 5.
Approximation Algorithms
Friedhelm Meyer auf der Heide 25
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 26
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
26
Approximation Algorithms
Examples:
Greedy Techniques (Load-Balancing, Center Selection )
Pricing Method (Vertex Cover)
Linear Programming and Rounding (Vertex Cover, generalized Load-Balancing )
Polynomial Time Approximation Scheme (Knapsack )
Friedhelm Meyer auf der Heide 27
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 28
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 29
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
15.07.2011 29
Friedhelm Meyer auf der Heide 30
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 31
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 32
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityApproximation via “rounding” linear programs
Friedhelm Meyer auf der Heide 33
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 34
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
minimize
Friedhelm Meyer auf der Heide 35
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 36
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 37
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide 38
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Chapter 6.
Online Algorithms
Friedhelm Meyer auf der Heide 39
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityOnline Algorithms
An online algorithm is one that can process its input piece-by-piece,
without having the entire input available from the start.
In contrast, an offline algorithm is given the whole problem data from the
beginning and is required to output an answer which solves the problem
at hand.
Input: a sequence of requests.
Task: process the requests as efficiently as possible
Online:
i„th request has to be processed before future requests are known
Offline:
All requests are known in advance
Friedhelm Meyer auf der Heide 40
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityHow to measure quality of online algorithms?
1. Assume some a priori knowledge about request
sequence, e.g., „requests are chosen randomly“
2. Assume worst case measure, compare online cost to
offline cost
Online : standard competitive analysis – competitive ratio
Online randomized:
Friedhelm Meyer auf der Heide 41
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Examples:
Paging, page migration, data management in
networks
Friedhelm Meyer auf der Heide 42
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Page migration – Classical online problem
• processors connected by a network
There are costs of communication associated with each
edge. Cost of communication between pair of nodes =
cost of the cheapest path between these nodes.
• Costs of communication fulfill the triangle inequality.
Page Migration Model (1)
v1
v2
v3
v4
v7
v6
v5
Friedhelm Meyer auf der Heide 43
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Alternative view:
• processors in a metric space
Indivisible memory page of size in the local memory of
one processor (initially at )
Page Migration Model (2)
v1
v2
v3
v4
v7
v6
v5
Friedhelm Meyer auf der Heide 44
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityPage Migration Model (3)
Input: sequence of processors,
dictated by a request adversary
- processor which wants to access (read or write)
one unit of data from the memory page.
After serving a request an algorithm may move the page
to a new processor.
v1
v2
v3
v4
v7
v6
v5
Friedhelm Meyer auf der Heide 45
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityPage Migration (cost model)
Cost model:
The page is at node .
Serving a request issued at costs .
Moving the page to node costs .
Friedhelm Meyer auf der Heide 46
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityA randomized algorithm
Memoryless coin-flipping algorithm CF [Westbrook 92]
Theorem: CF is 3-competitive against an adaptive-online
adversary (may see the outcomes of the coinflips).
Remark: This ratio is optimal against adaptive-online
adversary.
In each step, after serving a request issued at ,
move page to with probability .
Friedhelm Meyer auf der Heide 47
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityDeterministic algorithm
Algorithm Move-To-Min (MTM) [Awerbuch, Bartal, Fiat 93]
Theorem: MTM is 7-competitive
Remark: The currently best deterministic algorithm achieves
competitive ratio of 4.086
After each steps, choose to be the node
which minimizes , and move to .
( is the best place for the page in the last steps)
Friedhelm Meyer auf der Heide 48
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Data management in networks
Friedhelm Meyer auf der Heide 49
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Scenario
Networks have low bandwidth, global objects are small, access is
fine grained.
• typical for parallel processor networks, partially also for the internet.
• bottleneck: link-congestion
task: distribute global objects (maybe dynamically) among
processors such that
• an application (sequence of read/write access to global
variables) can be executed using small link-congestion
• storage overhead is small.
- Exploit Locality -
Friedhelm Meyer auf der Heide 50
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Basic Strategy
Design strategy for trees
Produce strategy for target-network by tree embedding
Friedhelm Meyer auf der Heide 51
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Dynamic Model
Application: Sequence of read / write requests from processors
to objects. Each processor decides solely based on
its local knowledge.
distributed online-strategy
Goal: Develop strategy that produces only by a factor c more
congestion than an optimal offline strategy.
c-competitive strategy
(and by a factor m more storage per processor
(m, c) – competitive strategy )
Friedhelm Meyer auf der Heide 52
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Dynamic strategy for trees
v writes to x :
v creates (or updates) copy of x in v,
and invalidates all other copies (consistency!)
v reads x:
v reads the closest copy of x and creates copies in
every processor on the path back to v.
(Remark: Data Tracking in trees is easy!)
Friedhelm Meyer auf der Heide 53
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
Friedhelm Meyer auf der Heide 54
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
Friedhelm Meyer auf der Heide 55
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
Friedhelm Meyer auf der Heide 56
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
v2
Friedhelm Meyer auf der Heide 57
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
v2
Friedhelm Meyer auf der Heide 58
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
v2
v3
Friedhelm Meyer auf der Heide 59
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
v2
v3
vk
Friedhelm Meyer auf der Heide 60
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Example and Analysis
Consider phase write (v0), read (v1), read (v2), ... , read (vk-1), write (vk)
v0
v1
v2
v3
Each strategy has to use each link of the red subtree at least once.
Our strategy uses each of these links at most three times.
Strategy is 3-competitive for trees
vk
Friedhelm Meyer auf der Heide 61
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Other networks
Idea: Simulate suitable tree in target-network M.
tree embedding:
Goals: - small dilation
(in order to reduce overall load)
- randomized embedding
(in order to reduce congestion)
Goals contradict?!?
Friedhelm Meyer auf der Heide 62
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Tree embedding
Example: nxn-mesh1 2 30
M„
v
• leaves: nodes of the mesh
• link-capacity: # links leaving the submesh
Randomized, locality preserving embedding!
Friedhelm Meyer auf der Heide 63
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Result for meshes
The static and dynamic strategies are
O (log(n))-competitive in nxn-meshes, w.h.p.
Finding an optimal static placement for several variables is
NP-hard already on 3x3-meshes.
Friedhelm Meyer auf der Heide 64
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Some Results
competitive ratio w.h.p.
d-dimensional meshes O(d log n)
Fat Trees O(log n)
Hypercubes O(log n)
SE Networks
De Bruijn Networks
Direct Butterflies
Indirect Butterflies
Arbitrary Networks
O(log n)
O(log n)
O(log n)
O(1)
polylog n (Räcke 2002)
Friedhelm Meyer auf der Heide 65
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and ComplexityI wish you ….
… not too much
stress with exams
and some time for relaxing holydays.
Friedhelm Meyer auf der Heide 66
HEINZ NIXDORF INSTITUTE
University of Paderborn
Algorithms and Complexity
Friedhelm Meyer auf der Heide
Heinz Nixdorf Institute
& Computer Science Department
University of Paderborn
Fürstenallee 11
33102 Paderborn, Germany
Tel.: +49 (0) 52 51/60 64 80
Fax: +49 (0) 52 51/62 64 82
E-Mail: [email protected]
http://www.upb.de/cs/ag-madh
Thank you for
your attention!