Complexity Theory - Stanford University
Transcript of Complexity Theory - Stanford University
![Page 1: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/1.jpg)
Complexity TheoryPart One
![Page 2: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/2.jpg)
It may be that since one is customarily concerned with existence, […] finiteness, and so forth, one is not inclined to take seriously the question of the existence of a better-than-finite algorithm.
- Jack Edmonds, “Paths, Trees, and Flowers”
![Page 3: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/3.jpg)
![Page 4: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/4.jpg)
It may be that since one is customarily concerned with existence, […] finiteness, and so forth, one is not inclined to take seriously the question of the existence of a better-than-finite algorithm.
- Jack Edmonds, “Paths, Trees, and Flowers”
![Page 5: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/5.jpg)
It may be that since one is customarily concerned with existence, […] finiteness, and so forth, one is not inclined to take seriously the question of the existence of a better-than-finite algorithm.
- Jack Edmonds, “Paths, Trees, and Flowers”
![Page 6: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/6.jpg)
It may be that since one is customarily concerned with existence, […] decidability, and so forth, one is not inclined to take seriously the question of the existence of a better-than-decidable algorithm.
- Jack Edmonds, “Paths, Trees, and Flowers”
![Page 7: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/7.jpg)
A Decidable Problem
Presburger arithmetic is a logical system for reasoning about arithmetic.
∀x. x + 1 ≠ 0
∀x. ∀y. (x + 1 = y + 1 → x = y)
∀x. x + 0 = x
∀x. ∀y. (x + y) + 1 = x + (y + 1)
(P(0) ∧ ∀y. (P(y) → P(y + 1))) → ∀x. P(x)
Given a statement, it is decidable whether that statement can be proven from the laws of Presburger arithmetic.
Any Turing machine that decides whether a statement in Presburger arithmetic is true or false has to move its tape head at least times on some inputs of length n (for some fixed constant c ≥ 1).
22𝑐𝑛
![Page 8: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/8.jpg)
For Reference
Assume c = 1.
220= 2
221= 4
222= 16
223= 256
224= 65536
225= 18446744073709551616
226= 340282366920938463463374607431768211456
![Page 9: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/9.jpg)
The Limits of Decidability
The fact that a problem is decidable does not mean that it is feasibly decidable.
In computability theory, we ask the question
What problems can be solved by a computer?
In complexity theory, we ask the question
What problems can be solvedefficiently by a computer?
In the remainder of this course, we will explore this question in more detail.
![Page 10: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/10.jpg)
Where We've Been
● The class R represents problems that can besolved by a computer.
● The class RE represents problems where “yes”answers can be verified by a computer.
The mapping reduction can be used to findconnections between problems.
![Page 11: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/11.jpg)
Where We're Going
● The class P represents problems that can besolved efficiently by a computer.
● The class NP represents problems where “yes”answers can be verified efficiently by acomputer.
![Page 12: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/12.jpg)
RegularLanguages CFLs
All Languages
R RE
![Page 13: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/13.jpg)
All Languages
RERegularLanguages CFLs R
![Page 14: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/14.jpg)
Undecidable Languages
RegularLanguages
CFLs REfficientlyDecidable
Languages
![Page 15: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/15.jpg)
The Setup
In order to study computability, we needed to answer these questions:
• What is “computation?”
• What is a “problem?”
• What does it mean to “solve” a problem?
To study complexity, we need to answer these questions:
• What does “complexity” even mean?
• What is an “efficient” solution to a problem?
![Page 16: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/16.jpg)
Measuring Complexity
Suppose that we have a decider D for some language L.
How might we measure the complexity of D?
• Number of states.
• Size of tape alphabet.
• Size of input alphabet.
• Amount of tape required.
• Number of steps required.
• Number of times a given state is entered.
• Number of times a given symbol is printed.
• Number of times a given transition is taken.
• (Plus a whole lot more...)
![Page 17: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/17.jpg)
Measuring Complexity
Suppose that we have a decider D for some language L.
How might we measure the complexity of D?
• Number of states.
• Size of tape alphabet.
• Size of input alphabet.
• Amount of tape required.
• Amount of time required.
• Number of times a given state is entered.
• Number of times a given symbol is printed.
• Number of times a given transition is taken.
• (Plus a whole lot more...)
![Page 18: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/18.jpg)
What is an efficient algorithm?
![Page 19: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/19.jpg)
Searching Finite Spaces
• Many decidable problems can be solved by searching over a large but finite space of possible options.
• Searching this space might take a staggeringly long time, but only finite time.
• From a decidability perspective, this is totally fine.
• From a complexity perspective, this may be totally unacceptable.
![Page 20: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/20.jpg)
A Sample Problem
4 3 11 9 7 13 5 6 1 12 2 8 0 10
![Page 21: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/21.jpg)
A Sample Problem
Goal: Find the length of the longest increasing subsequence of this
sequence.
4 3 11 9 7 13 5 6 1 12 2 8 0 10
![Page 22: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/22.jpg)
A Sample Problem
Goal: Find the length of the longest increasing subsequence of this
sequence.
4 3 11 9 7 13 5 6 1 12 2 8 0 10
![Page 23: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/23.jpg)
A Sample Problem
Goal: Find the length of the longest increasing subsequence of this
sequence.
4 3 11 9 7 13 5 6 1 12 2 8 0 10
![Page 24: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/24.jpg)
A Sample Problem
Goal: Find the length of the longest increasing subsequence of this
sequence.
4 3 11 9 7 13 5 6 1 12 2 8 0 10
![Page 25: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/25.jpg)
Longest Increasing Subsequences
One possible algorithm: try all subsequences, find the longest one that's increasing, and return that.
There are 2n subsequences of an array of length n.
(Each subset of the elements gives back a subsequence.)
Checking all of them to find the longest increasing subsequence will take time O(n · 2n).
Nifty fact: the age of the universe is about 4.3 × 1026
nanoseconds old. That's about 285 nanoseconds.
Practically speaking, this algorithm doesn't terminate if you give it an input of size 100 or more.
![Page 26: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/26.jpg)
A Different Approach
![Page 27: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/27.jpg)
Patience Sorting
4 3 11 9 7 13 5 6 1 12 2 8 0 10
4
3
11
9
7
13
5
6
1
12
2
8
0
10
Place each number on top of a pile.
Put each number on top of the first pile whose top value is larger than it. (If you
can’t, make a new pile.)
Then, add a link to the top number in the previous pile.
![Page 28: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/28.jpg)
Patience Sorting
4 3 11 9 7 13 5 6 1 12 2 8 0 10
4
3
11
9
7
13
5
6
1
12
2
8
0
10
Trace backwards from the top of the last pile. The numbers you visit form one of the longest increasing subsequences of your
original sequence.
![Page 29: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/29.jpg)
Patience Sorting
4 3 11 9 7 13 5 6 1 12 2 8 0 10
4
3
11
9
7
13
5
6
1
12
2
8
0
10
Trace backwards from the top of the last pile. The numbers you visit form one of the longest increasing subsequences of your
original sequence.
![Page 30: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/30.jpg)
Patience Sorting
34 11 9 7 13 5 6 1 12 2 8 0 10
4
3
11
9
7
13
5
6
1
12
2
8
0
10
Trace backwards from the top of the last pile. The numbers you visit form one of the longest increasing subsequences of your
original sequence.
![Page 31: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/31.jpg)
Longest Increasing Subsequences
Theorem: There is an algorithm that can find the longest increasing subsequence of an array in time O(n²).
It’s the previous patience sorting algorithm, with some clever implementation tricks.
This algorithm works by exploiting particular aspects of how longest increasing subsequences are constructed. It's not immediately obvious that it works correctly.
Phenomenal Exercise 1: Prove that this procedure always works!
Phenomenal Exercise 2: Show that you can actually implement this same algorithm in time O(nlog n).
![Page 32: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/32.jpg)
Another Problem
E
A
F
C
D
B
![Page 33: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/33.jpg)
Another Problem
E
A
F
C
D
B
![Page 34: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/34.jpg)
Another Problem
E
A
F
C
D
B
Goal: Determine the length of the shortest path from A to F in
this graph.
![Page 35: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/35.jpg)
Shortest Paths
It is possible to find the shortest path in a graph by listing off all sequences of nodes in the graph in ascending order of length and finding the first that's a path.
This takes time O(n · n!) in an n-node graph.
For reference: 29! nanoseconds is longer than the lifetime of the universe.
![Page 36: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/36.jpg)
Shortest Paths
Theorem: It's possible to find the shortest path between two nodes in an n-node, m-edge graph in time O(m + n).
Proof idea: Use breadth-first search!
The algorithm is a bit nuanced. It uses some specific properties of shortest paths and the proof of correctness is nontrivial.
![Page 37: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/37.jpg)
For Comparison
Longest increasing subsequence:
Naive: O(n · 2n)
Fast: O(n²)
Shortest path problem:
Naive: O(n · n!)
Fast: O(n + m).
![Page 38: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/38.jpg)
Defining Efficiency
• When dealing with problems that search for the “best” object of some sort, there are often at least exponentially many possible options.
• Brute-force solutions tend to take at least exponential time to complete.
• Clever algorithms often run in time O(n), or O(n2), or O(n3), etc.
![Page 39: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/39.jpg)
Polynomials and Exponentials
• An algorithm runs in polynomial time if its runtime is some polynomial in n.
• That is, time O(nk) for some constant k.
• Polynomial functions “scale well.”
• Small changes to the size of the input do not typically induce enormous changes to the overall runtime.
• Exponential functions scale terribly.
• Small changes to the size of the input induce huge changes in the overall runtime.
![Page 40: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/40.jpg)
The Cobham-Edmonds Thesis
A language L can be decided efficiently if
there is a TM that decides it in polynomial time.
Equivalently, L can be decided efficiently ifit can be decided in time O(nk) for some k ∈ ℕ.
Like the Church-Turing thesis, this is not a theorem!
It's an assumption about the nature of efficient computation, and it is
somewhat controversial.
![Page 41: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/41.jpg)
The Cobham-Edmonds Thesis
Efficient runtimes:
4n + 13
n3 – 2n2 + 4n
n log log n
“Efficient” runtimes:
n1,000,000,000,000
10500
Inefficient runtimes:
2n
n!
nn
“Inefficient” runtimes:
n0.0001 log n
1.000000001n
![Page 42: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/42.jpg)
Why Polynomials?
Polynomial time somewhat captures efficient computation, but has a few edge cases.
However, polynomials have very nice mathematical properties:
• The sum of two polynomials is a polynomial. (Running one efficient algorithm, then another, gives an efficient algorithm.)
• The product of two polynomials is a polynomial. (Running one efficient algorithm a “reasonable” number of times gives an efficient algorithm.)
• The composition of two polynomials is a polynomial. (Using the output of one efficient algorithm as the input to another efficient algorithm gives an efficient algorithm.)
![Page 43: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/43.jpg)
The Complexity Class P
The complexity class P (for polynomial time) contains all problems that can be solved in polynomial time.
Formally:
P = { L | There is a polynomial-time decider for L }
Assuming the Cobham-Edmonds thesis, a language is in P if it can be decided efficiently.
![Page 44: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/44.jpg)
Examples of Problems in P
All regular languages are in P.
• All have linear-time TMs.
All CFLs are in P.
• Requires a more nuanced argument (the CYK algorithm or Earley's algorithm.)
And a ton of other problems are in P as well.
Curious? Take CS161!
![Page 45: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/45.jpg)
Undecidable Languages
RegularLanguages
CFLs REfficientlyDecidable
Languages
![Page 46: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/46.jpg)
Undecidable Languages
RegularLanguages
CFLs RP
![Page 47: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/47.jpg)
What can't you do in polynomial time?
![Page 48: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/48.jpg)
start
end
How many simple paths are there from the start node to the end node?
![Page 49: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/49.jpg)
, , ,
How many subsets of this set are there?
![Page 50: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/50.jpg)
An Interesting Observation
There are (at least) exponentially many objects of each of the preceding types.
However, each of those objects is not very large.
Each simple path has length no longer than the number of nodes in the graph.
Each subset of a set has no more elements than the original set.
This brings us to our next topic...
![Page 51: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/51.jpg)
What if you need to search a large space for a single object?
![Page 52: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/52.jpg)
Verifiers – Again
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
Does this Sudoku problem have a solution?
![Page 53: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/53.jpg)
Verifiers – Again
2
4
3
6
7
8
1
5
9
5
9
8
4
1
3
6
7
2
7
1
6
5
9
2
3
8
4
9
8
1
7
5
6
2
4
3
6
7
2
3
4
1
5
9
8
4
3
5
2
8
9
7
6
1
1
6
9
8
3
5
4
2
7
8
5
4
1
2
7
9
3
6
3
2
7
9
6
4
8
1
5
Does this Sudoku problem have a solution?
![Page 54: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/54.jpg)
4 3 11 9 7 13 5 6 1 12 2 8 0 109 3 11 4 2 13 5 6 1 12 7 8 0 10
Verifiers – Again
Is there an ascending subsequence of length at least 7?
![Page 55: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/55.jpg)
4 3 11 9 7 13 5 6 1 12 2 8 0 109 3 11 4 2 13 5 6 1 12 7 8 0 10
Verifiers – Again
Is there an ascending subsequence of length at least 7?
![Page 56: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/56.jpg)
Verifiers – Again
Is there a simple path that goes through every node exactly once?
![Page 57: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/57.jpg)
Verifiers – Again
Is there a simple path that goes through every node exactly once?
1
2
5
4
6
3
![Page 58: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/58.jpg)
Verifiers
Recall that a verifier for L is a TM V such that
V halts on all inputs.
w ∈ L iff ∃c ∈ Σ*. V accepts ⟨w, c⟩.
![Page 59: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/59.jpg)
Polynomial-Time Verifiers
A polynomial-time verifier for L is a TM V such that
V halts on all inputs.
w ∈ L iff ∃c ∈ Σ*. V accepts ⟨w, c⟩.
V's runtime is a polynomial in |w| (that is, V's runtime is O(|w|k) for some integer k)
![Page 60: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/60.jpg)
The Complexity Class NP
The complexity class NP (nondeterministic polynomial time) contains all problems that can be verified in polynomial time.
Formally:
NP = { L | There is a polynomial-time verifier for L }
The name NP comes from another way of characterizing NP. If you introduce nondeterministic Turing machines and appropriately define “polynomial time,” then NP is the set of problems that an NTM can solve in polynomial time.
Useful fact: NP ⊊ R. Come talk to me after class if you’re curious why!
![Page 61: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/61.jpg)
P = { L | there is a polynomial-time
decider for L }
NP = { L | there is a polynomial-time
verifier for L }
![Page 62: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/62.jpg)
R = { L | there is a polynomial-time
decider for L }
RE = { L | there is a polynomial-time
verifier for L }
![Page 63: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/63.jpg)
We know that R ≠ RE.
So does that mean P ≠ NP?
![Page 64: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/64.jpg)
And now...
![Page 65: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/65.jpg)
The
Biggest Question
in
Theoretical Computer Science
![Page 66: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/66.jpg)
P ≟ NP
![Page 67: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/67.jpg)
P = { L | There is a polynomial-timedecider for L }
NP = { L | There is a polynomial-timeverifier for L }
P ⊆ NP
Polynomial-TimeDecider for L
yes!
no!
![Page 68: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/68.jpg)
P = { L | There is a polynomial-timedecider for L }
NP = { L | There is a polynomial-timeverifier for L }
P ⊆ NP
Polynomial-TimeVerifier for L
yes!
no!
![Page 69: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/69.jpg)
P NP
Which Picture is Correct?
![Page 70: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/70.jpg)
P NP
Which Picture is Correct?
![Page 71: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/71.jpg)
P ≟ NP
The P ≟ NP question is the most important question in theoretical computer science.
With the verifier definition of NP, one way of phrasing this question is
If a solution to a problem can be checked efficiently,can that problem be solved efficiently?
An answer either way will give fundamental insights into the nature of computation.
![Page 72: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/72.jpg)
Why This Matters
The following problems are known to be efficiently verifiable, but have no known efficient solutions:
Determining whether an electrical grid can be built to link up some number of houses for some price (Steiner tree problem).
Determining whether a simple DNA strand exists that multiple gene sequences could be a part of (shortest common supersequence).
Determining the best way to assign hardware resources in a compiler (optimal register allocation).
Determining the best way to distribute tasks to multiple workers to minimize completion time (job scheduling).
And many more.
If P = NP, all of these problems have efficient solutions.
If P ≠ NP, none of these problems have efficient solutions.
![Page 73: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/73.jpg)
Why This Matters
If P = NP:
A huge number of seemingly difficult problems could be solved efficiently.
Our capacity to solve many problems will scale well with the size of the problems we want to solve.
If P ≠ NP:
Enormous computational power would be required to solve many seemingly easy tasks.
Our capacity to solve problems will fail to keep up with our curiosity.
![Page 74: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/74.jpg)
What We Know
Resolving P ≟ NP has proven extremely difficult.
In the past 45 years:
• Not a single correct proof either way has been found.
• Many types of proofs have been shown to be insufficiently powerful to determine whether P ≟ NP.
• A majority of computer scientists believe P ≠ NP, but this isn't a large majority.
• Interesting read: Interviews with leading thinkers about P ≟ NP:
• http://web.ing.puc.cl/~jabaier/iic2212/poll-1.pdf
![Page 75: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/75.jpg)
The Million-Dollar Question
The Clay Mathematics Institute has offered a $1,000,000 prize to anyone who proves or
disproves P = NP.
![Page 76: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/76.jpg)
“My hunch is that [P ≟ NP] will be solved by a young researcher who is not
encumbered by too much conventional wisdom about how to attack the problem.”
– Prof. Richard Karp(The guy who first popularized the P ≟ NP problem.)
![Page 77: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/77.jpg)
“There is something very strange about this problem, something very philosophical. It is
the greatest unsolved problem in mathematics […] It is the raison d’être of abstract computer
science, and as long as it remains unsolved, its mystery will ennoble the field.”
-Prof. Jim Owings(Computability/Complexity theorist)
![Page 78: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/78.jpg)
What do we know about P ≟ NP?
![Page 79: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/79.jpg)
Adapting our Techniques
![Page 80: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/80.jpg)
P = { L | there is a polynomial-time
decider for L }
NP = { L | there is a polynomial-time
verifier for L }
![Page 81: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/81.jpg)
R = { L | there is a polynomial-time
decider for L }
RE = { L | there is a polynomial-time
verifier for L }
![Page 82: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/82.jpg)
We know that R ≠ RE.
So does that mean P ≠ NP?
![Page 83: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/83.jpg)
A Problem
The R and RE languages correspond to problems that can be decided and verified, period, without any time bounds.
To reason about what's in R and what's in RE, we used two key techniques:
Universality: TMs can run other TMs as subroutines.
Self-Reference: TMs can get their own source code.
Why can't we just do that for P and NP?
![Page 84: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/84.jpg)
Theorem (Baker-Gill-Solovay): Any proof that purely relies on universality and self-reference cannot resolve P ≟ NP.
Proof: Take CS154!
![Page 85: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/85.jpg)
So how are we going toreason about P and NP?
![Page 86: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/86.jpg)
NPPREG
Problems in NP vary widely in theirdifficulty, even if P = NP.
How can we rank the relative difficultiesof problems?
![Page 87: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/87.jpg)
Reducibility
![Page 88: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/88.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
![Page 89: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/89.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
![Page 90: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/90.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
A matching, but not a maximum matching.
![Page 91: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/91.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
A maximum matching.
![Page 92: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/92.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
![Page 93: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/93.jpg)
Maximum Matching
Given an undirected graph G, a matching in G is a set of edges such that no two edges share an endpoint.
A maximum matching is a matching with the largest number of edges.
![Page 94: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/94.jpg)
Maximum Matching
Jack Edmonds' paper “Paths, Trees, and Flowers” gives a polynomial-time algorithm for finding maximum matchings.
He’s the guy with the quote about “better than decidable.”
Using this fact, what other problems can we solve?
![Page 95: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/95.jpg)
Domino Tiling
![Page 96: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/96.jpg)
Domino Tiling
![Page 97: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/97.jpg)
Domino Tiling
![Page 98: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/98.jpg)
Domino Tiling
![Page 99: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/99.jpg)
Domino Tiling
![Page 100: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/100.jpg)
Domino Tiling
![Page 101: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/101.jpg)
Solving Domino Tiling
![Page 102: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/102.jpg)
Solving Domino Tiling
![Page 103: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/103.jpg)
Solving Domino Tiling
![Page 104: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/104.jpg)
Solving Domino Tiling
![Page 105: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/105.jpg)
Solving Domino Tiling
![Page 106: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/106.jpg)
Solving Domino Tiling
![Page 107: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/107.jpg)
Solving Domino Tiling
![Page 108: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/108.jpg)
Solving Domino Tiling
![Page 109: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/109.jpg)
In Pseudocode
boolean canPlaceDominoes(Grid G, int k) {
return hasMatching(gridToGraph(G), k);
}
![Page 110: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/110.jpg)
Intuition:
Tiling a grid with dominoes can't be“harder” than solving maximum matching,
because if we can solve maximummatching efficiently, we can solve domino
tiling efficiently.
![Page 111: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/111.jpg)
Another Example
![Page 112: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/112.jpg)
Reachability
● Consider the following problem:
Given an directed graph G and nodes sand t in G, is there a path from s to t?
● This problem can be solved in polynomialtime (use DFS or BFS).
![Page 113: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/113.jpg)
Converter Conundrums
Suppose that you want to plug your laptop into a projector.
Your laptop only has a VGA output, but the projector needs HDMI input.
You have a box of connectors that convert various types of input into various types of output (for example, VGA to DVI, DVI to DisplayPort, etc.)
Question: Can you plug your laptop into the projector?
![Page 114: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/114.jpg)
Converter Conundrums
ConnectorsRGB to USB
VGA to DisplayPortDB13W3 to CATV
DisplayPort to RGBDB13W3 to HDMIDVI to DB13W3S-Video to DVIFireWire to SDI
VGA to RGBDVI to DisplayPort
USB to S-VideoSDI to HDMI
![Page 115: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/115.jpg)
Converter Conundrums
ConnectorsRGB to USB
VGA to DisplayPortDB13W3 to CATV
DisplayPort to RGBDB13W3 to HDMIDVI to DB13W3S-Video to DVIFireWire to SDI
VGA to RGBDVI to DisplayPort
USB to S-VideoSDI to HDMI
VGA RGB USB
DisplayPort DB13W3 CATV
X DVIHDMI S-Video
FireWire SDI
![Page 116: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/116.jpg)
Converter Conundrums
VGA RGB USB
DisplayPort DB13W3 CATV
X DVIHDMI S-Video
FireWire SDI
![Page 117: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/117.jpg)
Converter Conundrums
VGA RGB USB
DisplayPort DB13W3 CATV
X DVIHDMI S-Video
FireWire SDI
![Page 118: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/118.jpg)
Converter Conundrums
VGA RGB USB
DisplayPort DB13W3 CATV
X DVIHDMI S-Video
FireWire SDI
![Page 119: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/119.jpg)
Converter Conundrums
VGA RGB USB
DisplayPort DB13W3 CATV
X DVIHDMI S-Video
FireWire SDI
ConnectorsRGB to USB
VGA to DisplayPortDB13W3 to CATV
DisplayPort to RGBDB13W3 to HDMIDVI to DB13W3S-Video to DVIFireWire to SDI
VGA to RGBDVI to DisplayPort
USB to S-VideoSDI to HDMI
![Page 120: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/120.jpg)
In Pseudocode
boolean canPlugIn(List<Plug> plugs) {
return isReachable(plugsToGraph(plugs),VGA, HDMI);
}
![Page 121: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/121.jpg)
Intuition:
Finding a way to plug a computer into aprojector can't be “harder” than
determining reachability in a graph, sinceif we can determine reachability in a graph,we can find a way to plug a computer into a
projector.
![Page 122: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/122.jpg)
Intuition:
Problem A can't be “harder” than problemB, because solving problem B lets us solve
problem A.
bool solveProblemA(string input) {return solveProblemB(transform(input));
}
![Page 123: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/123.jpg)
bool solveProblemA(string input) {return solveProblemB(transform(input));
}
● If A and B are problems where it'spossible to solve problem A using thestrategy shown above*, we write
A ≤p B.
● We say that A is polynomial-timereducible to B.
* Assuming that transform* runs in polynomial time.
![Page 124: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/124.jpg)
bool solveProblemA(string input) {return solveProblemB(transform(input));
}
● This is a powerful general problem-solvingtechnique. You’ll see it a lot in CS161.
![Page 125: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/125.jpg)
● If A ≤p
B and B ∈ P, then A ∈ P.
If L1
≤P
L2
and L2
∈ NP, then L1
∈ NP.
P
Polynomial-Time Reductions
![Page 126: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/126.jpg)
● If A ≤p
B and B ∈ P, then A ∈ P.
If L1
≤P
L2
and L2
∈ NP, then L1
∈ NP.
P
Polynomial-Time Reductions
![Page 127: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/127.jpg)
● If A ≤p
B and B ∈ P, then A ∈ P.
If L1
≤P
L2
and L2
∈ NP, then L1
∈ NP.
P
Polynomial-Time Reductions
![Page 128: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/128.jpg)
● If A ≤p
B and B ∈ P, then A ∈ P.
● If A ≤p
B and B ∈ NP, then A ∈ NP.
P
Polynomial-Time Reductions
![Page 129: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/129.jpg)
Polynomial-Time Reductions
● If A ≤p
B and B ∈ P, then A ∈ P.
● If A ≤p
B and B ∈ NP, then A ∈ NP.
NPP
![Page 130: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/130.jpg)
Polynomial-Time Reductions
● If A ≤p
B and B ∈ P, then A ∈ P.
● If A ≤p
B and B ∈ NP, then A ∈ NP.
NPP
![Page 131: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/131.jpg)
Polynomial-Time Reductions
● If A ≤p
B and B ∈ P, then A ∈ P.
● If A ≤p
B and B ∈ NP, then A ∈ NP.
NPP
![Page 132: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/132.jpg)
This ≤ₚ relation lets us rank the relative difficulties of problems in P and NP.
What else can we do with it?
![Page 133: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/133.jpg)
Next Time
NP-Completeness
What are the hardest problems in NP?
![Page 134: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/134.jpg)
Next TimeRight now
NP-Completeness
What are the hardest problems in NP?
![Page 135: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/135.jpg)
Complexity TheoryPart Two
![Page 136: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/136.jpg)
Recap from Last Time
![Page 137: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/137.jpg)
The Complexity Class P
The complexity class P (polynomial time) is defined as
P = { L | There is a polynomial-time decider for L }
Intuitively, P contains all decision problems that can be solved efficiently.
This is like class P, except with “efficiently” tacked onto the end.
![Page 138: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/138.jpg)
The Complexity Class NP
The complexity class NP (nondeterministic polynomial time) contains all problems that can be verified in polynomial time.
Formally:
NP = { L | There is a polynomial-time verifier for L }
Intuitively, NP is the set of problems where “yes” answers can be checked efficiently.
This is like the class RE, but with “efficiently” tacked on to the definition.
![Page 139: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/139.jpg)
The Biggest Unsolved Problem inTheoretical Computer Science:
P ≟ NP
![Page 140: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/140.jpg)
Theorem (Baker-Gill-Solovay): Any proof that purely relies on universality and self-reference cannot resolve P ≟ NP.
Proof: Take CS154!
![Page 141: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/141.jpg)
So how are we going toreason about P and NP?
![Page 142: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/142.jpg)
Intuition:
Problem A can't be “harder” than problemB, because solving problem B lets us solve
problem A.
bool solveProblemA(string input) {return solveProblemB(transform(input));
}
![Page 143: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/143.jpg)
bool solveProblemA(string input) {return solveProblemB(transform(input));
}
● If A and B are problems where it'spossible to solve problem A using thestrategy shown above*, we write
A ≤p B.
● We say that A is polynomial-timereducible to B.
* Assuming that transform* runs in polynomial time.
![Page 144: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/144.jpg)
Polynomial-Time Reductions
● If A ≤p
B and B ∈ P, then A ∈ P.
● If A ≤p
B and B ∈ NP, then A ∈ NP.
NPP
![Page 145: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/145.jpg)
This ≤ₚ relation lets us rank the relative difficulties of problems in P and NP.
What else can we do with it?
![Page 146: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/146.jpg)
New Stuff!
![Page 147: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/147.jpg)
Satisfiability
A propositional logic formula φ is called satisfiable if there is some assignment to its variables that makes it evaluate to true.
p ∧ q is satisfiable.
p ∧ ¬p is unsatisfiable.
p → (q ∧ ¬q) is satisfiable.
An assignment of true and false to the variables of φ that makes it evaluate to true is called a satisfying assignment.
![Page 148: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/148.jpg)
SAT
The boolean satisfiability problem (SAT) is the following:
Given a propositional logicformula φ, is φ satisfiable?
Formally:
SAT = { ⟨φ⟩ | φ is a satisfiable PL formula }
![Page 149: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/149.jpg)
Finding Cliques
Does this graph contain a k-clique?
![Page 150: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/150.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
![Page 151: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/151.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
Could we somehow take these rules and encode them as a propositional logic formula?
![Page 152: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/152.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 153: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/153.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
Take your graph and define the following
propositional variables. The variable c3
represents choosing node c as the 3rd
node of your clique.
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 154: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/154.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 155: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/155.jpg)
Finding Cliques
a b
dc
Imagine we’re looking for a 3-clique. That means we need one variable in each of these groups to
be true.
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 156: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/156.jpg)
Finding Cliques
a b
dc
(a1 ∨ b1 ∨ c1 ∨ d1)
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 157: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/157.jpg)
Finding Cliques
a b
dc
The first node of your clique is either a or b or
c or d.
(a1 ∨ b1 ∨ c1 ∨ d1)
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 158: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/158.jpg)
Finding Cliques
a b
dc
(a1 ∨ b1 ∨ c1 ∨ d1)
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 159: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/159.jpg)
Finding Cliques
a b
dc
(a1 ∨ b1 ∨ c1 ∨ d1) ∧
(a2 ∨ b2 ∨ c2 ∨ d2)
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 160: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/160.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
(a1 ∨ b1 ∨ c1 ∨ d1) ∧
(a2 ∨ b2 ∨ c2 ∨ d2)∧
(a3 ∨ b3 ∨ c3 ∨ d3)
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 161: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/161.jpg)
Finding Cliques
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a b
dc
(a1 ∨ b1 ∨ c1 ∨ d1) ∧
(a2 ∨ b2 ∨ c2 ∨ d2)∧
(a3 ∨ b3 ∨ c3 ∨ d3)
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 162: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/162.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 163: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/163.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
We need to ensure we don’t pick a pair of nodes that don’t have an edge between them. In this
graph, the missing edges are
{a, c} and {b, c}
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 164: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/164.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
We can’t choose both a and c because there’s no edge between them.
for all i, j:(¬ai ∨ ¬cj)
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 165: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/165.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
for all i, j:(¬ai ∨ ¬cj)
for all i, j:(¬bi ∨ ¬cj)
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 166: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/166.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 167: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/167.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
Altogether, finding an assignment of true false values to these variables that satisfies these
constraints would amount to finding a clique of desired size in our graph.
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 168: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/168.jpg)
Finding Cliques
a b
dc
What is a k-clique?
A set of k nodes
Such that there’s an edge between every pair of nodes
One example that works is to assign a1, b2, and
d3 to be true and all other variables to be false.
a1 a2 a3 a4
b1 b2 b3 b4
c1 c2 c3 c4
d1 d2 d3 d4
![Page 169: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/169.jpg)
Intuition:
Finding a k-clique can’t be any harder than solving SAT, because we can take
any graph and encode it as a propositional logic formula where
finding a satisfying assignment corresponds to finding a k-clique.
![Page 170: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/170.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
Does this Sudoku puzzlehave a solution?
![Page 171: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/171.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
![Page 172: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/172.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
Take your puzzle and let the variable xi,j,k
represent filling in cell (i, j) with value k.
![Page 173: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/173.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
for all i, j:xi,j,1∨ … ∨ xi,j,9
∧for all k ≠ l:
(¬xi,j,k ∨ ¬xi,j,l)
All cells (i, j) should have some value between 1 and 9.
![Page 174: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/174.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
for all i, j:xi,j,1∨ … ∨ xi,j,9
∧for all k ≠ l:
(¬xi,j,k ∨ ¬xi,j,l)
And each cell should only be assigned one value.
![Page 175: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/175.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
![Page 176: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/176.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
for all k in [1, 9]:(x1,1,k ∨ … ∨ x1,9,k)
In the first row, every value k has to be assigned to one of the cells.
![Page 177: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/177.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
Add similar constraints for the other rows, columns, and 3x3 squares!
![Page 178: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/178.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
![Page 179: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/179.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
(x1,3,7 ∧ x1,3,7 ∧ x1,3,7 ∧ …)
![Page 180: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/180.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
(x1,3,7 ∧ x1,5,6 ∧ x1,3,7 ∧ …)
![Page 181: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/181.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
(x1,3,7 ∧ x1,5,6 ∧ x1,6,1 ∧ …)
![Page 182: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/182.jpg)
Solving Sudoku
3
6
8
1
5
1
7
7
5
2
3
4
1
2
4
6
3
1
8
3
5
7
1
9
8
5
7
5
2
2
7
9
4
8
What is a Sudoku solution?
●An assignment of numbers 1 through 9 to a 9×9 grid
●Such that each number appears exactly once in each row, column, and 3×3 square
●Subject to some existing constraints (numbers that have been filled in already)
Altogether, finding an assignment of true false values to these variables that satisfies these constraints would amount to finding a solution to our
Sudoku puzzle.
![Page 183: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/183.jpg)
Intuition:
Solving Sudoku can’t be any harder than solving SAT, because we can take any
Sudoku puzzle and encode it as a propositional logic formula where finding a satisfying assignment corresponds to
finding a puzzle solution.
![Page 184: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/184.jpg)
SAT
Finding cliques
Solving Sudoku
![Page 185: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/185.jpg)
SAT
Finding cliques
Solving Sudoku
Job scheduling
Tetris
Longest common
subsequence
Finding Hamiltonian
pathsIt turns out that a lot of other problems can
also be reduced to SAT.
![Page 186: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/186.jpg)
Key Observations:
(1) SAT is versatile – being able to solve SAT allows us to solve many other problems.
(2) The fact that lots of problems reduce to SAT suggests we can gauge the difficulty of these problems by looking at the difficulty of SAT.
![Page 187: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/187.jpg)
An Analogy: Running Really Fast
![Page 188: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/188.jpg)
For people A and B, we say A ≤ᵣ B ifA’s top running speed is at most B’s top speed.
(Intuitively: B can run at least as fast as A.)
We say that person P is CS103-fast if
∀A ∈ CS103. A ≤ᵣ P.
(How fast are you if you’re CS103-fast?)
We say that person P is CS103-complete ifP ∈ CS103 and P is CS103-fast.
(How fast are you if you’re CS103-complete?)
CS103 CS103-fastCS103-complete
Usain
Bolt
PaulaRadcliffe
Fastest runner in
CS103
Tied for fastest in
CS103
![Page 189: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/189.jpg)
For languages A and B, we say A ≤ₚ B ifA reduces to B in polynomial time.
(Intuitively: B is at least as hard as A.)
We say that a language L is NP-hard if
∀A ∈ NP. A ≤ₚ L.
(How hard is a problem that’s NP-hard?)
We say that a language L is NP-complete ifL ∈ NP and L is NP-hard.
(How hard is a problem that’s NP-complete?)
NP NP-hardNP-complete
LD
ATMHardest
problem in NP
Tied for hardest in
NP
![Page 190: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/190.jpg)
Intuition: The NP-complete problems are the hardest problems in NP.
If we can determine how hard those problems are, it would tell us a lot about
the P ≟ NP question.
![Page 191: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/191.jpg)
The Tantalizing Truth
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem X. Since L is NP-complete, we knowthat X ≤p L. Since L ∈ P and X ≤p L, we see that X ∈ P. Sinceour choice of X was arbitrary, this means that NP ⊆ P, soP = NP. ■Intuition: This means the hardest
problems in NP aren’t actually that hard. We can solve them in
polynomial time. So that means we can solve all problems in NP in
polynomial time.
![Page 192: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/192.jpg)
The Tantalizing Truth
NPP
NPC
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem X. Since L is NP-complete, we knowthat X ≤p L. Since L ∈ P and X ≤p L, we see that X ∈ P. Sinceour choice of X was arbitrary, this means that NP ⊆ P, soP = NP. ■
![Page 193: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/193.jpg)
The Tantalizing Truth
NPP
NPC
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem X. Since L is NP-complete, we knowthat X ≤p L. Since L ∈ P and X ≤p L, we see that X ∈ P. Sinceour choice of X was arbitrary, this means that NP ⊆ P, soP = NP. ■
![Page 194: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/194.jpg)
The Tantalizing Truth
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem X. Since L is NP-complete, we knowthat X ≤p L. Since L ∈ P and X ≤p L, we see that X ∈ P. Sinceour choice of X was arbitrary, this means that NP ⊆ P, soP = NP. ■
NPP
NPC
![Page 195: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/195.jpg)
The Tantalizing Truth
P = NP
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem X. Since L is NP-complete, we knowthat X ≤p L. Since L ∈ P and X ≤p L, we see that X ∈ P. Sinceour choice of X was arbitrary, this means that NP ⊆ P, soP = NP. ■
![Page 196: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/196.jpg)
The Tantalizing Truth
P = NP
Theorem: If any NP-complete language is in P, then P = NP.
Proof: Suppose that L is NP-complete and L ∈ P. Now considerany arbitrary NP problem A. Since L is NP-complete, we knowthat A ≤ₚ L. Since L ∈ P and A ≤ₚ L, we see that A ∈ P. Sinceour choice of A was arbitrary, this means that NP ⊆ P, soP = NP. ■
![Page 197: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/197.jpg)
The Tantalizing Truth
Theorem: If any NP-complete language is not in P, then P ≠ NP.
Proof: Suppose that L is an NP-complete language not in P. SinceL is NP-complete, we know that L ∈ NP. Therefore, we knowthat L ∈ NP and L ∉ P, so P ≠ NP. ■
Intuition: This means the hardest problems in NP are so hard that
they can’t be solved in polynomial time. So the hardest problems in NP
aren’t in P, meaning P ≠ NP.
![Page 198: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/198.jpg)
The Tantalizing Truth
NP
PNPC
Theorem: If any NP-complete language is not in P, then P ≠ NP.
Proof: Suppose that L is an NP-complete language not in P. SinceL is NP-complete, we know that L ∈ NP. Therefore, we knowthat L ∈ NP and L ∉ P, so P ≠ NP. ■
![Page 199: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/199.jpg)
How do we even know NP-complete
problems exist in the first place?
![Page 200: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/200.jpg)
Theorem (Cook-Levin): SAT is NP-complete.
Proof Idea: To see that SAT ∈ NP, show how to make a polynomial-time verifier for it. Key idea: have the certificate be a satisfying assignment.
To show that SAT is NP-hard, given a polymomial-time verifier V for an arbitrary NP language L, for any string w you can construct a polynomially-sized formula φ(w) that says “there is a certificate c where V accepts ⟨w, c⟩.” This formula is satisfiable if and only if w ∈ L, so deciding whether the formula is satisfiable decides whether w is in L. ■-ish
Proof: Take CS154!
![Page 201: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/201.jpg)
Why All This Matters
Resolving P ≟ NP is equivalent to just figuring out how hard SAT is.
SAT ∈ P ↔ P = NP
We've turned a huge, abstract, theoretical problem about solving problems versus checking solutions into the concrete task of seeing how hard one problem is.
You can get a sense for how little we know about algorithms and computation given that we can't yet answer this question!
![Page 202: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/202.jpg)
Why All This Matters
You will almost certainly encounter NP-hard problems in practice – they're everywhere!
If a problem is NP-hard, then there is no known algorithm for that problem that
• is efficient on all inputs,
• always gives back the right answer, and
• runs deterministically.
Useful intuition: If you need to solve an NP-hard problem, you will either need to settle for an approximate answer, an answer that's likely but not necessarily right, or have to work on really small inputs.
![Page 203: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/203.jpg)
Sample NP-Hard Problems
Computational biology: Given a set of genomes, what is the most probable evolutionary tree that would give rise to those genomes? (Maximum parsimony problem)
Game theory: Given an arbitrary perfect-information, finite, two-player game, who wins? (Generalized geography problem)
Operations research: Given a set of jobs and workers who can perform those tasks in parallel, can you complete all the jobs within some time bound? (Job scheduling problem)
Machine learning: Given a set of data, find the simplest way of modeling the statistical patterns in that data (Bayesian network inference problem)
Medicine: Given a group of people who need kidneys and a group of kidney donors, find the maximum number of people who survive. (Cycle cover problem)
Systems: Given a set of processes and a number of processors, find the optimal way to assign those tasks so that they complete as soon as possible (Processor scheduling problem)
![Page 204: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/204.jpg)
Coda: What if P ≟ NP is resolved?
![Page 205: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/205.jpg)
Time-Out for Announcements!
![Page 206: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/206.jpg)
Please evaluate this course on Axess.
Your feedback really makes a difference.
![Page 207: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/207.jpg)
The Big Picture
![Page 208: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/208.jpg)
Take a minute to reflect on your journey.
![Page 209: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/209.jpg)
Set Theory
Power Sets
Cantor’s Theorem
Direct Proofs
Parity
Proof by Contrapositive
Proof by Contradiction
Modular Congruence
Number Theory
Propositional Logic
First-Order Logic
Logic Translations
Logical Negations
Propositional Completeness
Vacuous Truths
Tournament Graphs
Binary Relations
Equivalence Relations
Equivalence Classes
Systems of Representatives
Strict Orders
Functions
Injections
Surjections
Bijections
Inverse Functions
Permutations
Graphs
Connectivity
Graph Automorphisms
Vertex Covers
Bipartite Graphs
Mathematical Induction
Loop Invariants
Complete Induction
Tiling Problems
Bezout's Identity
Euclid's Algorithm
Hypercubes
Formal Languages
DFAs
Regular Languages
Closure Properties
NFAs
Subset Construction
Kleene Closures
Monoids
5-Tuples
Regular Expressions
State Elimination
Distinguishability
Myhill-Nerode Theorem
Nonregular Languages
Extended Transition Functions
Equivalence Relation Indices
Axiom of Choice
Context-Free Grammars
Turing Machines
Church-Turing Thesis
TM Encodings
Universal Turing Machines
Self-Reference
Decidability
Recognizability
Self-Defeating Objects
Undecidable Problems
Halting Problem
Verifiers
Diagonalization Language
Complexity Class P
Complexity Class NP
P ≟ NP Problem
Polynomial-Time Reducibility
NP-Completeness
![Page 210: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/210.jpg)
You’ve done more than just checka bunch of boxes off a list.
![Page 211: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/211.jpg)
You’ve given yourself the foundationto tackle problems from all over
computer science.
![Page 212: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/212.jpg)
Kinda sorta like a left inverse!
Hey, you’ve seen this before!
From CS255
![Page 213: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/213.jpg)
From CS145
Cartesian products!
Set-builder notation!
![Page 214: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/214.jpg)
From CS251
Whoa, it’s a function!
![Page 215: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/215.jpg)
From CS143It’s a CFG!
It’s an automaton derived from a CFG!
![Page 216: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/216.jpg)
From CS221
It’s a DFA!
![Page 217: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/217.jpg)
From CS161
It’s FOL and functions!
![Page 218: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/218.jpg)
From CS224W
First-order definitions on
graphs!Set difference and cardinality!
![Page 219: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/219.jpg)
From CS242
It’s a CFG!
![Page 220: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/220.jpg)
From CS166
Fibonacci numbers!
Trees!
Formal proofs!
![Page 221: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/221.jpg)
From CS238
Hey, we know that one!
![Page 222: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/222.jpg)
From CS144
It’s a generalization of DFAs!
![Page 223: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/223.jpg)
From CS168
Reducibility!
A Myhill-Nerode-style argument!
![Page 224: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/224.jpg)
From CS124
New definitions on graphs!What do graphs with
these properties look like?
Transform some object to make it closed under
some operation!
![Page 225: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/225.jpg)
From CS154
Using Turing machines to define intrinsic information
content!
![Page 226: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/226.jpg)
From CS246
Functions, set union, and set cardinality!
![Page 227: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/227.jpg)
From CS140
These are binary relations!This is a strict order!
![Page 228: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/228.jpg)
You’ve given yourself the foundationto tackle problems from all over
computer science.
![Page 229: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/229.jpg)
There’s so much more to explore.
Where should you go next?
![Page 230: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/230.jpg)
Course Recommendations
Theoryland
CS154
Phil 151
Phil 152
Math 107
Math 108
Math 120
Math 113
Math 161
Math 152
Applications
CS124
CS143
CS161
CS224W
CS243
CS246
CS242
CS251
CS255
Functions
RelationsGraphs
Symmetries
Number Theory
Set Theory
Computability
Languages /Automata
Graphs
Functions
![Page 231: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/231.jpg)
Your Questions
![Page 232: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/232.jpg)
“As computers become more ubiquitous, it seems that knowledge about computers
and how they work does not. How can we address this? Does LinkedIn have a perspective on this? Your personal
thoughts?”
Though there’s still a lot we don’t know about computation, the amount that we do know is increasing every day! CS research (theoretical and applied) is
constantly discovering new things about what’s possible with a computer and new algorithms, new applications of existing techniques, etc.
![Page 233: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/233.jpg)
“If I get an awful grade on the final exam, is it possible to still do okay-ish in the class (if I've been keeping up with the problem
sets)?”
I don’t think this is the right question to be asking right now: 1) If you’ve been keeping up with the problem sets, there’s no reason to expect that you’ll do poorly on the exam, and 2) the final hasn’t happened yet so the outcome is
still very much in your hands! Instead of thinking of what could happen if the exam goes badly, focus your energy on actively targeting your weak spots and
working to make the exam go well.
![Page 234: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/234.jpg)
“What is the high-level knowledge map for math used in CS? PSets do give a glimpse into how much math is involved in many areas of CS but what is the big picture?”
We saw some specific examples earlier today, here’s a broader overview*
* Huge caveat: this is largely based on which areas of CS I’ve personally been exposed to! There’s lots that I don’t have much experience in.
![Page 235: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/235.jpg)
AI
NLP
Compilers
Information Theory
Databases
Linear Algebra
Automata,languages
Logic
Number Theory
Crypto
NetworkingProbability
Functions
Set theory
Graphs
![Page 236: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/236.jpg)
AI
NLP
Compilers
Information Theory
Databases
Linear Algebra
Automata,languages
Logic
Number Theory
Crypto
NetworkingProbability
Functions
Set theory
Graphs
![Page 237: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/237.jpg)
AI
NLP
Compilers
Information Theory
Databases
Linear Algebra
Automata,languages
Logic
Number Theory
Crypto
NetworkingProbability
Functions
Set theory
Graphs
Math is everywhere in CS :D You can choose how deeply you want to study the theory side of things
(yay abstractions!)
![Page 238: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/238.jpg)
“What does doing research in CS look like? Like for bench science, we will conduct experiments to test an idea. It's hard to imagine CS researches as sitting in an
office and coding all day long.”
Similar process, just different tools! CS research can take many forms: coming up with new ways of modeling and predicting phenomena, designing new
algorithms and proving that they meet certain runtime/space constraints, finding new ways of applying CS to other fields (education, healthcare, transportation, …)
Would be happy to connect you to folks here in the department who are doing CS research!
![Page 239: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/239.jpg)
“Out of all the CS courses offered at Stanford...why CS103? What makes this
course more compelling to you than others?”
:)We get to take these abstract, complex, philosophical ideas (the nature of
computation, infinity, truth) and make them accessible and tangible.As a class in the CS core @ Stanford, CS103 presents the unique opportunity to get people excited about computer science from a completely different
perspective.
![Page 240: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/240.jpg)
Final Thoughts
![Page 241: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/241.jpg)
A Huge Round of Thanks!
![Page 242: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/242.jpg)
There are more problems tosolve than there are programs
capable of solving them.
![Page 243: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/243.jpg)
There is so much more to explore and so many big questions to ask – many of
which haven't been asked yet!
![Page 244: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/244.jpg)
Theory
Practice
![Page 245: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/245.jpg)
You now know what problems we can solve, what problems we can't solve, and what
problems we believe we can't solve efficiently.
![Page 246: Complexity Theory - Stanford University](https://reader034.fdocuments.in/reader034/viewer/2022050123/626d8cfccad0b5176532f594/html5/thumbnails/246.jpg)
Our questions to you:
What problems will you choose to solve?
Why do those problems matter to you?
And how are you going to solve them?