Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne COS 423: Theory of...
-
date post
20-Dec-2015 -
Category
Documents
-
view
219 -
download
1
Transcript of Princeton University COS 423 Theory of Algorithms Spring 2002 Kevin Wayne COS 423: Theory of...
Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
COS 423: Theory of Algorithms
Princeton University
Spring, 2001
Kevin Wayne
2
Algorithm. (webster.com) A procedure for solving a mathematical problem (as of finding the
greatest common divisor) in a finite number of steps that frequently involves repetition of an operation.
Broadly: a step-by-step procedure for solving a problem or accomplishing some end especially by a computer.
"Great algorithms are the poetry of computation."
Etymology. "algos" = Greek word for pain. "algor" = Latin word for to be cold. Abu Ja'far al-Khwarizmi's = 9th century Arab scholar.
– his book "Al-Jabr wa-al-Muqabilah" evolved into today's high school algebra text
Theory of Algorithms
3
Imagine: A World With No Algorithms
Fast arithmetic. Cryptography.
Quicksort. Databases.
FFT. Signal processing.
Huffman codes. Data compression.
Network flow. Routing Internet packets.
Linear programming. Planning, decision-making.
4
What is COS 423?
Introduction to design and analysis of computer algorithms. Algorithmic paradigms. Analyze running time of programs. Data structures. Understand fundamental algorithmic problems. Intrinsic computational limitations. Models of computation. Critical thinking.
Prerequisites. COS 226 (array, linked list, search tree, graph, heap, quicksort). COS 341 (proof, induction, recurrence, probability).
5
Administrative Stuff
Lectures: (Kevin Wayne) Monday, Wednesday 10:00 - 10:50, COS 104.
TA's: (Edith Elkind, Sumeet Sobti)
Textbook: Introduction to Algorithms (CLR).
Grading: Weekly problem sets. Collaboration, no-collaboration. Class participation, staff discretion. Undergrad / grad.
Course web site: courseinfo.princeton.edu/courses/COS423_S2001/ Fill out questionnaire.
6
Approximate Lecture Outline
Algorithmic paradigms. Divide-and-conquer. Greed. Dynamic programming. Reductions.
Analysis of algorithms. Amortized analysis.
Data structures. Union find. Search trees and extensions.
Graph algorithms. Shortest path, MST. Max flow, matching.
7
Approximate Lecture Outline
NP completeness. More reductions. Approximation algorithms.
Other models of computation. Parallel algorithms. Randomized algorithms.
Miscellaneous. Numerical algorithms. Linear programming.
Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
College Admissions
Sample problem.
Algorithm.
Analysis.
References:
The Stable Marriage Problem by Dan Gusfield and Robert Irving, MIT Press, 1989.
Introduction to Algorithms by Jon Kleinberg and Éva Tardos.
9
College Admissions
Goal: Design a self-reinforcing college admissions process.
Given a set of preferences among colleges and applicants, can we assign applicants to colleges so that for every applicant X, and every college C that X is not attending, either:
C prefers every one of its admitted students to X; X prefers her current situation to the situation in which she is
attending college C.
If this holds, the outcome is STABLE. Individual self-interest prevents any applicant / college to
undermine assignment by joint action.
Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
Love, Marriage, and Lying
Standard disclaimer.
11
Stable Matching Problem
Problem: Given N men and N women, find a "suitable" matching between men and women.
Participants rate members of opposite sex. Each man lists women in order of preference from best to worst.
Zeus Bertha AmyDiane Erika Clare
Yancey Amy ClareDiane Bertha Erika
Xavier Bertha ClareErika Diane Amy
Wyatt Diane AmyBertha Clare Erika
Victor Bertha DianeAmy Erika Clare
Man 1st 2nd 3rd 4th 5th
Men’s Preference List
worstbest
12
Stable Matching Problem
Problem: Given N men and N women, find a "suitable" matching between men and women.
Participants rate members of opposite sex. Each man lists women in order of preference from best to worst. Each woman lists men in order of preference.
Erika Yancey ZeusWyatt Xavier Victor
Diane Victor YanceyZeus Xavier Wyatt
Clare Wyatt YanceyXavier Zeus Victor
Bertha Xavier YanceyWyatt Victor Zeus
Amy Zeus WyattVictor Yancey Xavier
Woman 1st 2nd 3rd 4th 5th
Women’s Preference List
worstbest
13
Stable Matching Problem
Problem: Given N men and N women, find a "suitable" matching between men and women.
PERFECT MATCHING: everyone is matched monogamously. – each man gets exactly one woman– each woman gets exactly one man
STABILITY: no incentive for some pair of participants to undermine assignment by joint action.
– in matching M, an unmatched pair (m,w) is UNSTABLE if man m and woman w prefer each other to current partners
– unstable pair could each improve by dumping spouses and eloping
STABLE MATCHING = perfect matching with no unstable pairs.(Gale and Shapley, 1962)
14
Lavender assignment is a perfect matching.Are there any unstable pairs?
Men’s Preference List Women’s Preference List
Zeus
Yancey
Xavier
Man
A
B
A
1st
B
A
B
2nd
C
C
C
3rd
Clare
Bertha
Amy
Woman
X
X
Y
1st
Y
Y
X
2nd
Z
Z
Z
3rd
Yes. Bertha and Xavier form an unstable pair. They would prefer each other to current partners.
B
X
Example
15
Example
Green assignment is a stable matching.
A
B
A
C X
X
Y
Y
Y
X
Z
Z
Z
Men’s Preference List Women’s Preference List
Zeus
Yancey
Xavier
Man 1st
B
A
B
2nd
C
C
3rd
Clare
Bertha
Amy
Woman 1st 2nd 3rd
16
Example
Orange assignment is also a stable matching.
A X
X
Y
Y
Z
Z
Men’s Preference List Women’s Preference List
Zeus
Yancey
Xavier
Man 1st
A
B
2nd
C
C
3rd
Clare
Bertha
Amy
Woman 1st 2nd 3rd
B
A
B
C
X
Y
Z
17
B
Preference List
Bob
Chris
Adam C
A
B
D
D
Doofus A B C
D
C
A
Stable Roommate Problem
Not obvious that any stable matching exists.
Consider related "stable roommate problem." 2N people. Each person ranks others from 1 to 2N-1. Assign roommate pairs so that no unstable pairs.
C
A
B
D
No stable matching.
All 3 perfect matchings have unstable pair.
E.g., A-C forms unstable pair in lavender matching.
C
A
1st 2nd 3rd
18
Propose-And-Reject Algorithm
Intuitive method that guarantees to find a stable matching.
Initialize each person to be free.
while (some man m is free and hasn't proposed to every woman)
w = first woman on m's list to whom m has not yet proposed
if (w is free)
assign m and w to be engaged
else if (w prefers m to her fiancé m')
assign m and w to be engaged, and m' to be free
else
w rejects m
Gale-Shapley Algorithm (men propose)
19
Implementation and Running Time Analysis
Engagements. Maintain two arrays wife[m], and husband[w]; set equal to 0 if
participant is free. Store list of free men on a stack (queue).
Preference lists. For each man, create a linked list of women, ordered from favorite to
worst.– men propose to women at top of list, if rejected goto next
For each woman, create a "ranking array" such that mth entry in array is woman's ranking of man m.
– allows for queries of the form: does woman w prefer m to m' ?
Resource consumption. Time = (N2). Space = (N2). Optimal.
wmwm to proposed has :),(
20
Men's Preference List
Wyatt
Victor
Man 1st
A
B
2nd
C
D
3rd
C
B
AZeus
Yancey
Xavier C
D
A
B
B
A
D
C
4th
E
E
5th
A
D
E
E
D
C
B
E
Women's Preference List
Bertha
Amy
Man 1st
W
X
2nd
Y
Z
3rd
Y
X
VErika
Diane
Clare Y
Z
V
W
W
V
Z
X
4th
V
W
5th
V
Z
X
Y
Y
X
W
Z
A Worst Case Instance
Number of proposals n(n-1) + 1. Algorithm terminates when last woman gets first proposal.
Number of proposals = n(n-1) + 1 for following family of instances.
21
Proof of Correctness
Observation 1. Men propose to their favorite women first.
Observation 2. Once a woman is matched, she never becomes unmatched. She only "trades up."
Fact 1. All men and women get matched (perfect). Suppose upon termination Zeus is not matched. Then some woman, say Amy, is not matched upon termination. By Observation 2, Amy was never proposed to. But, Zeus proposes to everyone, since he ends up unmatched.
(contradiction)
22
Proof of Correctness
Observation 1. Men propose to their favorite women first.
Observation 2. Once a woman is matched, she never becomes unmatched. She only "trades up."
Fact 2. No unstable pairs. Suppose (Amy, Zeus) is an unstable pair: each prefers
each other to partner in Gale-Shapley matching S*.
Case 1. Zeus never proposed to Amy. Zeus must prefer Bertha to Amy (Observation 1) (Amy, Zeus) is stable. (contradiction)
Case 2. Zeus proposed to Amy. Amy rejected Zeus (right away or later) Amy prefers Yancey to Zeus (women only trade up) (Amy, Zeus) is stable (contradiction)
Bertha-Zeus
Amy-Yancey
S*
23
Understanding the Solution
For a given problem instance, there may be several stable matchings. Do all executions of Gale-Shapley yield the same stable matching?
If so, which one?
Fact 3. Yes. Gale-Shapley finds MAN-OPTIMAL stable matching! Man m is a valid partner of woman w if there exists some stable
matching in which they are married. Man-optimal assignment: every man receives best valid partner.
– simultaneously best for each and every man– there is no stable matching in which any single man individually
does better
24
Proof of Fact 3
Proof. Suppose, for sake of contradiction, some man is
paired with someone other than best partner.– since men propose in decreasing order of
preference, some man is rejected by valid partner Let Yancey be first such man, and let Amy be first valid partner
that rejects him. When Yancey is rejected, Amy forms (or reaffirms) engagement
with man, say Zeus, whom she prefers to Yancey. Let Bertha be Zeus' partner in S. Zeus not rejected by any valid partner at the point when Yancey is
rejected by Amy (since Yancey is first to be rejected by valid partner). Thus, Zeus prefers Amy to Bertha.
But Amy prefers Zeus to Yancey. Thus (Amy, Zeus) is unstable pair in S.
Bertha-Zeus
Amy-Yancey
S
25
Understanding the Solution
Fact 4. Gale-Shapley finds WOMAN-PESSIMAL matching. Each woman married to worst valid partner.
– simultaneously worst for each and every woman.– there is no stable matching in which any single woman
individually does worse
Proof. Suppose (Amy, Zeus) matched in S*, but Zeus is not worst valid
partner for Amy. There exists stable matching S in which Amy is paired with man,
say Yancey, whom she likes less than Zeus. Let Bertha be Zeus' partner in S. Zeus prefers Amy to Bertha (man optimality). (Amy, Zeus) form unstable pair in S. Bertha-Zeus
Amy-Yancey
S
26
Understanding the Solution
Fact 5. The man-optimal stable matching is weakly Pareto optimal. There is no other perfect matching (stable or unstable), where
every man does strictly better.
Proof. Let Amy be last woman in some execution of Gale-Shapley (men
propose) algorithm to receive a proposal. No man is rejected by Amy since algorithm terminates when last
woman receives first proposal. No man matched to Amy will be strictly better off than in man-
optimal stable matching.
27
Extensions: Unacceptable Partners
Yeah, but in real-world every woman is not willing to marry every man, and vice versa?
Some participants declare others as "unacceptable."(prefer to be alone than with given partner)
Algorithm extends to handle partial preference lists.
Matching S unstable if there exists man m and woman w such that: m is either unmatched in S, or strictly prefers w to his partner in S w is either unmatched in S, or strictly prefers m to her partner in S.
Fact 6. Men and women are each partitioned into two sets: those that have partners in all stable matchings; those that have partners in none.
28
Extensions: Sets of Unequal Size
Also, there may be an unequal number of men and women. E.g., |M| = 100 men, |W| = 90 women. Algorithm extends. WLOG, assume |W| < |M|.
Matching S unstable if there exists man m and woman w such that: m is either unmatched in S, or strictly prefers w to his partner in S; w is either unmatched in S, or strictly prefers m to her partner in S.
Fact 7. All women are matched in every stable matching. Men are partitioned into two subsets:
men who are matched in every stable matching; men who are matched in none.
29
Extensions: Limited Polygamy
What about limited polygamy? E.g., Bill wants 3 women. Algorithm extends.
Matching S unstable if there exists man m and woman w such that: either w is unmatched, or w strictly prefers m to her partner; either m does not have all its "places" filled in the matching, or m
strictly prefers w to at least one of its assigned residents.
30
Application: Matching Residents to Hospitals
Sets of unequal size, unacceptable partners, limited polygamy.
Matching S unstable if there exists hospital h and resident r such that: h and r are acceptable to each other; either r is unmatched, or r prefers h to her assigned hospital; either h does not have all its places filled in the matching, or h
prefers r to at least one of its assigned residents.
31
Application: Matching Residents to Hospitals
Matching medical school residents to hospitals. (NRMP) Hospitals ~ Men (limited polygamy allowed). Residents ~ Women. Original use just after WWII (predates computer usage). Ides of March, 13,000+ residents.
Rural hospital dilemma. Certain hospitals (mainly in rural areas) were unpopular and
declared unacceptable by many residents. Rural hospitals were under-subscribed in NRMP matching. How can we find stable matching that benefits "rural hospitals"?
Rural Hospital Theorem: Rural hospitals get exactly same residents in every stable
matching!
32
Deceit: Machiavelli Meets Gale-Shapley
Is there any incentive for a participant to misrepresent his/her preferences?
Assume you know men’s propose-and-reject algorithm will be run. Assume that you know the preference lists of all other participants.
Fact 8. No, for any man yes, for some women!
A X
X
Y
Y
Z
Z
Men’s Preference List Women’s True Preference List
Zeus
Yancey
Xavier
Man 1st
A
B
2nd
C
C
3rd
Clare
Bertha
Amy
Woman 1st 2nd 3rd
B
A
B
C
X
Y
Z
X
Z
Y
Y
Z
X
Amy Lies
Clare
Bertha
Amy
Woman 1st 2nd 3rd
X
Y
Z
X
Y
Z
X
Y
Z
33
Lessons Learned
Powerful ideas learned in COS 423. Isolate underlying structure of problem. Create useful and efficient algorithms.
Sometimes deep social ramifications. Historically, men propose to women. Why not vice versa? Men: propose early and often. Men: be more honest. Women: ask out the guys. Theory can be socially enriching and fun! CS majors get the best partners!!!
Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
Love, Marriage, and Lying: Extra Slides
35
Example With a Unique Stable Matching
Red matching is unique stable matching.
C Y
Z
X
X
Y
X
Men’s Preference List Women’s Preference List
Zeus
Yancey
Xavier
Man 1st
B
B
2nd
C
A
3rd
Clare
Bertha
Amy
Woman 1st 2nd 3rd
B
C
A
A
Z
Y
Z
36
How to Represent Men and Women
Represent men and women as integers between 0 and N-1. 0 through N-1 since C array indices start at 0. Could use struct if we want to carry around more information,
e.g., name, age, astrological sign.
37
How to Represent Marriages
Use array to keep track of marriages.
int wife[N];
int husb[N];
for (m = 0; m < N; m++)
wife[m] = -1;
for (w = 0; w < N; w++)
husb[w] = -1;
unmatched man if1
womanto matched man if][wife
m
wmwm
unmatched w womanif1
womanto matched man if][husb
wmmw
38
Filling in Some of the Code
while (marriages < N)
for (m = 0; wife[m] != -1; m++) ;
while (-1 == wife[m]) let w be man m’s favorite women to whom he has not yet proposed
if (-1 == husb[w]) husb[m] = w; wife[w] = m; marriages++;
else if (w prefers m to current fiancé) m' = husb[w]; wife[m'] = -1; husb[m] = w; wife[w] = m;
if (w unmatched) m and w get engaged
while (m unmatched)
m' = current fiancé of wm' now unmatchedm and w get engaged
find unmatched man
39
Men’s Preference List
Man 0th 1st 2nd 3rd 4th
0 1 0 3 4 21 3 1 0 2 42 1 4 2 3 03 0 3 2 1 44 1 3 0 4 2
Representing the Preference Lists
Use 2D-array to represent preference lists. 2D-array is array of arrays. mp[m][i] = w if man m’s ith favorite woman is w. wp[w][i] = m if woman w’s ith favorite man is m.
int mp[N][N];
int wp[N][N];
mp[1][0] = 3man 1 likes woman 3 the best
40
Initializing the Preference Lists
Could read from stdin.
We’ll assign random lists for each man and woman. Use randomPermutation function from Lecture P2. Need N random permutations for men, and N for women.
int mp[N][N];
int wp[N][N];
for (m = 0; m < N; m++) randomPermutation(mp[m], N);
for (w = 0; w < N; w++) randomPermutation(wp[w], N);
mp[m] is man m’s preference list array.
41
Dumping
Does woman w=2 prefer man m1=3 to man m2=0?
Yes, m1 appears on woman w’spreference list before m2.
for (i = 0; i < N; i++) {
if (wp[w][i] == m1) YES
if (wp[w][i] == m2) NO
}
search preference list sequentially until m1 or m2 found
Women’s Preference List
Woman 0th 1st 2nd 3rd 4th
0 4 0 1 3 21 2 1 3 0 42 1 2 3 4 03 0 4 3 2 14 3 1 4 2 0
42
Keeping Track of Men’s Proposals
Unmatched man proposes to most favorable woman to whom he hasn’t already proposed.
How do we keep track of which woman a man has proposed to? Men propose in decreasing order of preference. Suffices to keep track of number of proposals in array.
propose[m] = i if man m has proposed to i woman already.
initialize arrayint props[N];
for (i = 0; i < N; i++) props[i] = 0;
43
Keeping Track of Men’s Proposals
Unmatched man proposes to most favorable woman to whom he hasn’t already proposed.
How do we keep track of which woman a man has proposed to? Men propose in decreasing order of preference. Suffices to keep track of number of proposals in array.
propose[m] = i if man m has proposed to i woman already.
while (-1 == wife[n]) {
w = mp[m][props[m]];
props[m]++; . . .}
props[m] is ranking of next woman on preference list
make next proposal to woman mp[m][props[m]]
find next woman to propose to
44
Try Out The Code
#include <stdio.h>#include <stdlib.h>#include <assert.h>#define N 500
int main(void) { int mp[N][N]; /* mp[m][i] = w if man m's ith favorite woman is w */ int wp[N][N]; /* wp[w][i] = m if woman w's ith favorite man is m */ int wife[N]; /* wife[m] = w if m married to w */ int husb[N]; /* husb[w] = m if m married to w */ int props[N]; /* props[m] = i if man m has proposed to i women */
int marriages = 0; /* number of couples matched so far */ int m, w;
/* initialize men */ for (m = 0; m < N; m++) { props[m] = 0; wife[m] = -1; randomPermutation(mp[m], N); }
/* initialize women */ for (w = 0; w < N; w++) { husb[w] = -1; randomPermutation(wp[w], N); }
marriage.c
45
Try Out The Code
while (marriages < N) { /* find first unmatched man */ for (m = 0; wife[m] != -1; m++) ;
printf("man %d proposing:\n", m); /* propose to next women on list until successful */ while (-1 == wife[m]) { w = mp[m][props[m]]; printf(" to woman %d", w); props[m]++; /* woman w unmatched */ if (-1 == husb[w]) { printf(" accepted\t(woman %d previously unmatched)\n", w); husb[w] = m; wife[m] = w; marriages++; }
/* woman w prefers m to current mate */ else if (wr[w][m] < wr[w][husb[w]]) { printf(" accepted\t(woman %d dumps man %d)\n", w, husb[w]); wife[husb[w]] = -1; husb[w] = m; wife[m] = w; } /* otherwise m rejected by w */ else printf(" rejected\t(woman %d prefers %d)\n", w, husb[w]); } }
marriage.c
46
Try Out The Code
Observation: code isREALLY slow for large N.
printf("Stable matching\n"); for (m = 0; m < N; m++) printf("%5d %5d\n", m, wife[m]);
return 0;}
marriage.c% gcc marriage.c% a.out
man 0 proposing: to woman 4 accepted (woman 4 previously unmatched)man 1 proposing: to woman 0 accepted (woman 0 previously unmatched)man 2 proposing: to woman 2 accepted (woman 2 previously unmatched)man 3 proposing: to woman 2 rejected (woman 2 prefers 2) to woman 3 accepted (woman 3 previously unmatched)man 4 proposing: to woman 2 accepted (woman 2 dumps man 2)man 2 proposing: to woman 3 accepted (woman 3 dumps man 3)man 3 proposing: to woman 0 rejected (woman 0 prefers 1) to woman 4 rejected (woman 4 prefers 0) to woman 1 accepted (woman 1 previously unmatched)
Stable matching 0 4 1 0 2 3 3 1 4 2
Unix
47
Why So Slow?
Does woman w=2 prefer man m1=3 to man m2=0?
Need to search through row 2 to find answer.This is repeated many times.If N is large, this can be very expensive.
Women’s Preference List
Woman 0th 1st 2nd 3rd 4th
0 4 0 1 3 21 2 1 3 0 42 1 2 3 4 03 0 4 3 2 14 3 1 4 2 0
48
Men’s Preference List
Man 0th 1st 2nd 3rd 4th
0 1 0 3 4 21 3 1 0 2 42 1 4 2 3 03 0 3 2 1 44 1 3 0 4 2
mp[1][0] = 3man 1 likes woman 3 best
Men’s Rankings
Man 0 1 2 3 4
0 1st 0th 4th 2nd 3rd
1 2nd 1st 3rd 0th 4th
2 4th 0th 2nd 3rd 1st
3 0th 3rd 2nd 1st 4th
4 2nd 0th 4th 1st 3rd
mr[1][3] = 0man 1 likes woman 3 best
An Auxiliary Data Structure
Create a 2D array that stores men’s ranking of women. mr[m][w] = i if man m’s ranking of woman w is i. wr[w][m] = i if woman w’s ranking of man m is i.
49
Men’s Rankings
Man 0 1 2 3 4
0 1st 0th 4th 2nd 3rd
1 2nd 1st 3rd 0th 4th
2 4th 0th 2nd 3rd 1st
3 0th 3rd 2nd 1st 4th
4 2nd 0th 4th 1st 3rd
mr[m][w1] = 2mr[m][w2] = 4
if (mr[m][w1] < mr[m][w2])
YES
else
NO
An Auxiliary Data Structure
Create a 2D array that stores men’s ranking of women. mr[m][w] = i if man m’s ranking of woman w is i. wr[w][m] = i if woman w’s ranking of man m is i.
Does man m = 3 prefer woman w1 = 2 to woman w2 = 4?
50
Check if Marriage is Stable
Check if husb[N] and wife[N] correspond to a stable marriage. Good warmup and useful for debugging. Check every man-woman pair to see if they’re unstable. Use ranking arrays.
int isStable(int husb[], int wife[],
int mr[N][N], int wr[N][N]) {
int m, w;
for (m = 0; m < N; m++)
for (w = 0; w < N; w++)
if (mr[m][w] < mr[m][wife[m]]) &&
(wr[w][m] < wr[w][husb[w]])
return 0;
return 1;
}
isStable
m prefers w to current wife
w prefers m to current husband
51
Check if Marriage is Stable
Check if husb[N] and wife[N] correspond to a stable marriage. Good warmup and useful for debugging. Check every man-woman pair to see if they’re unstable. Use ranking arrays.
Time/space tradeoff for using auxiliary ranking arrays. Disadvantage: requires twice as much memory (storage). Advantage: dramatic speedup in running time
(using 400 MHz Pentium II with N = 10,000). 1 second using ranking arrays. 2 hours by searching preference list sequentially!