David Evans cs.virginia/evans
description
Transcript of David Evans cs.virginia/evans
David Evanshttp://www.cs.virginia.edu/evans
CS588: Security and PrivacyUniversity of VirginiaComputer Science
Lecture 15:Complexity Notes
This is selected from CS200 lectures 16,30, 37, 38 fromhttp://www.cs.virginia.edu/cs200/lectures/
CS588 Spring 2005 2
What does really mean?•O(x) – it is no more than x work
(upper bound)(x) – work scales as x (tight
bound)(x) – it is at least x work
(lower bound)If O(x) and (x) are true,
then (x) is true.
CS588 Spring 2005 3
Meaning of O (“big Oh”)
f(x) is O (g (x)) means:There is a positive constant
c such that c * f(x) < g(x)
for all but a finite number of x values.
CS588 Spring 2005 4
O Examplesf(x) is O (g (x)) means:
There is a positive constant c such that c * f(x) < g(x)
for all but a finite number of x values.
x is O (x2)? Yes, c = 1 works fine.
10x is O (x)? Yes, c = .09 works fine.
x2 is O (x)? No, no matter what c we pick,cx2 > x for big enough x
CS588 Spring 2005 5
Lower Bound: (Omega)f(x) is (g (x)) means:
There is a positive constant c such that
c * f(x) > g(x) for all but a finite number of
x values. Difference from O – this was <
CS588 Spring 2005 6
f(x) is (g (x)) means:There is a positive constant c such that
c * f(x) > g(x) for all but a finite number of x values.
f(x) is O (g (x)) means:There is a positive constant c such that
c * f(x) < g(x) for all but a finite number of x values.
Examples
• x is (x)– Yes, pick c = 2
• 10x is (x)– Yes, pick c = 1
• Is x2 (x)?– Yes!
• x is O(x)– Yes, pick c = .5
• 10x is O(x)– Yes, pick c = .09
• x2 is not O(x)
CS588 Spring 2005 7
Tight Bound: (Theta)
f(x) is (g (x)) iff:
f(x) is O (g (x)) and f(x) is (g (x))
CS588 Spring 2005 8
Examples• 10x is (x)
– Yes, since 10x is (x) and 10x is O(x)•Doesn’t matter that you choose different
c values for each part; they are independent
• x2 is/is not (x)?– No, since x2 is not O (x)
• x is/is not (x2)?– No, since x2 is not (x)
CS588 Spring 2005 9
Measuring Work
• When we say a procedure or a problem is O(f(n)) what are we counting?
• Need a model computer to count steps
CS588 Spring 2005 10
How should we model a Computer?
Apollo Guidance Computer (1969)
Colossus (1944)
IBM 5100 (1975)
Cray-1 (1976)
CS588 Spring 2005 11
Modeling Computers• Input
– Without it, we can’t describe a problem
• Output– Without it, we can’t get an answer
• Processing– Need some way of getting from the input
to the output
• Memory– Need to keep track of what we are doing
CS588 Spring 2005 12
Modeling Input
Engelbart’s mouse and keypad
Punch Cards
Altair BASIC Paper Tape, 1976
CS588 Spring 2005 13
Simplest Input
• Non-interactive: like punch cards and paper tape
• One-dimensional: just a single tape of values, pointer to one square on tape0 0 1 1 0 0 1 0 0 0
How long should the tape be?
Infinitely long! We are modeling a computer, not building one. Our model should not have silly practical limitations (like a real computer does).
CS588 Spring 2005 14
Modeling Output
• Blinking lights are cool, but hard to model
• Output is what is written on the tape at the end of a computation
Connection Machine CM-5, 1993
CS588 Spring 2005 15
Modeling Processing• Evaluation Rules
– Given an input on our tape, how do we evaluate to produce the output
• What do we need:– Read what is on the tape at the current
square– Move the tape one square in either
direction– Write into the current square
0 0 1 1 0 0 1 0 0 0
Is that enough to model a computer?
CS588 Spring 2005 16
Modeling Processing
• Read, write and move is not enough• We also need to keep track of what
we are doing:– How do we know whether to read, write
or move at each step?– How do we know when we’re done?
• What do we need for this?
CS588 Spring 2005 17
Finite State Machines
1Start 2
HALT
10
1#
0
CS588 Spring 2005 18
Hmmm…maybe we don’t need those infinite tapes
after all?
1Start 2
HALT
(not a paren
)#
not a paren
)
ERROR
What if thenext input symbolis ( in state 2?
CS588 Spring 2005 19
How many states do we need?
1Start 2
HALT
(not a paren
)#
not a paren
)
ERROR
3
not a paren
(
)
4(
)
not a paren
(
CS588 Spring 2005 20
Finite State Machine
• There are lots of things we can’t compute with only a finite number of states
• Solutions:– Infinite State Machine
• Hard to describe and draw
– Add a tape to the Finite State Machine
CS588 Spring 2005 21
FSM + Infinite Tape• Start:
– FSM in Start State– Input on Infinite Tape– Pointer to start of input
• Move:– Read one input symbol from tape– Follow transition rule from current state
• To next state• Write symbol on tape, and move L or R one square
• Finish:– Transition to halt state
CS588 Spring 2005 22
Matching Parentheses• Repeat until halt:
– Find the leftmost )•If you don’t find one, the parentheses match,
write a 1 at the tape head and halt.
– Replace it with an X– Look left for the first (
•If you find it, replace it with an X (they matched)•If you don’t find it, the parentheses didn’t match
– end write a 0 at the tape head and halt
CS588 Spring 2005 23
Matching Parentheses
1Start
HALT
Input: )Write: XMove: L
), X, L
2: look for (
X, X, RX, X, L
(, X, R
#, 0, ##, 1, #
(, (, R
Will this report thecorrect result for (()?
CS588 Spring 2005 24
Matching Parentheses
1Start
HALT
), X, L
2: look for (
(, (, R
(, X, R
#, 0, ##, 1, #
X, X, L
X, X, R
#, #, L
3: look for (X, X, L
#, 1, #(, 0, #
CS588 Spring 2005 25
Turing Machine• Alan Turing, On computable numbers:
With an application to the Entscheidungsproblem, 1936– Turing developed the machine
abstraction to show the halting problem really leads to a contradiction
– Our informal argument, depended on assuming we could do if and everything else except halts?
CS588 Spring 2005 26
Describing Turing Machinesz z z z z z z z z z z z z z z zz z z z
TuringMachine ::= < Alphabet, Tape, FSM >Alphabet ::= { Symbol* }Tape ::= < LeftSide, Current, RightSide >OneSquare ::= Symbol | #Current ::= OneSquareLeftSide ::= [ Square* ]RightSide ::= [ Square* ]
Everything to left of LeftSide is #.Everything to right of RightSide is #.
1
Start
HALT
), X, L
2: look for (
#, 1, -
), #, R
(, #, L
(, X, R
#, 0, -
Finite State Machine
CS588 Spring 2005 27
Describing Finite State Machines
TuringMachine ::= < Alphabet, Tape, FSM >FSM ::= < States, TransitionRules, InitialState, HaltingStates >States ::= { StateName* }InitialState ::= StateName must be element of StatesHaltingStates ::= { StateName* } all must be elements of StatesTransitionRules ::= { TransitionRule* }TransitionRule ::= < StateName, ;; Current State OneSquare, ;; Current square StateName, ;; Next State OneSquare, ;; Write on tape Direction > ;; Move tapeDirection ::= L, R, #
1Start
HALT
), X, L
2: look for (
), #, R (, #, L
(, X, R
#, 0, ##, 1, #
Transition Rule is a procedure:StateName X OneSquare StateName X OneSquare X Direction
CS588 Spring 2005 28
Example Turing
Machine
TuringMachine ::= < Alphabet, Tape, FSM >FSM ::= < States, TransitionRules, InitialState, HaltingStates >Alphabet ::= { (, ), X }States ::= { 1, 2, HALT }InitialState ::= 1HaltingStates ::= { HALT }TransitionRules ::= { < 1, ), 2, X, L >,
< 1, #, HALT, 1, # >, < 1, ), #, R >, < 2, (, 1, X, R >,
< 2, #, HALT, 0, # >, < 2, ), #, L >,}
1Start
HALT
), X, L
2: look for (
), #, R (, #, L
(, X, R
#, 0, ##, 1, #
CS588 Spring 2005 29
Enumerating Turing Machines
• Now that we’ve decided how to describe Turing Machines, we can number them
• TM-5023582376 = balancing parens• TM-57239683 = even number of 1s• TM-3523796834721038296738259873 = Photomosaic
Program• TM-3672349872381692309875823987609823712347823 = WindowsXP
Not the real numbers – they would be much
bigger!
CS588 Spring 2005 30
Complexity Class P
Class P: problems that can be solved in polynomial time by a deterministic TM.
O (nk) for some constant k.
Easy problems like sorting, making a photomosaic using duplicate tiles, simulating the universe are all in P.
CS588 Spring 2005 31
Deterministic Computing
• All our computing models so far are deterministic: you know exactly what to do every step– TM: only one transition rule can match
any machine configuration, only one way to follow that transition rule
– Lambda Calculus: always -reduce the outermost redex
CS588 Spring 2005 32
Nondeterministic Computing• Allows computer to try different options at
each step, and then use whichever one works best
• Make a Finite State Machine non-deterministic: doesn’t change computing time
• Make a Turing Machine non-deterministic: might change computing time
• No one knows whether it does or not•This is the biggest open problem in Computer
Science
CS588 Spring 2005 33
Making FSM’s Nondeterministic
1 2 30 1
0 0
1
HALT
#
CS588 Spring 2005 34
Nondeterministic FSM
1 2 3
HALT
#
0, 1
0 1
Input Read Possible States
0 { 1, 2 }
00 { 1, 2 }
001 { 1, 3 }
0010 { 1, 2 }
00101 { 1, 3 }
00101# { HALT }
CS588 Spring 2005 35
Power of NFSM
• Can NFSMs computing anything FSMs cannot compute?
• Can NFSMs compute faster than FSMs?
No – you can always convert an NFSM to a FSM bymaking each possible set of states in the NFSM into onestate in the new FSM. (Number of states may increase as 2s)
No – both read input one letter at a time, and transitionautomatically to the next state. (Both are always (n) wheren is the number of symbols in the input.)
CS588 Spring 2005 36
Nondeterministic TMs
• Multiple transitions possible at every step
• Follow all possible steps:– Each possible execution maintains its
own state and its own infinite tape– If any possible execution halts (in an
accepting state), the NTM halts, and the tape output of that execution is the NTM output
CS588 Spring 2005 37
Complexity ClassesClass P: problems that can be solved in polynomial time by a deterministic TM.
O (nk) for some constant k.Easy problems like simulating the universe are all in P.
Class NP: problems that can be solved in polynomial time by a nondeterministic TM
Hard problems like the pegboard puzzle sorting are in NP (as well as all problems in P).
CS588 Spring 2005 38
Problem Classes
PNP
Decidable
Undecidable
Sorting: (n log n)
Simulating Universe: O(n3)
Cracker Barrel: O(2n) and (n)
Fill tape with 2n *s: (2n)
Halting Problem: ()
CS588 Spring 2005 39
P = NP?• Is there a polynomial-time solution to
the “hardest” problems in NP?• No one knows the answer!• The most famous unsolved problem in
computer science and math• Listed first on Millennium Prize
Problems– win $1M if you can solve it – (also an automatic A+ in this course)
CS588 Spring 2005 40
If P NP:
PNP
Decidable
Undecidable
Sorting: (n log n)
Simulating Universe: O(n3)
Cracker Barrel: O(2n) and (n)
Fill tape with 2n *s: (2n)
Halting Problem: ()
CS588 Spring 2005 41
If P = NP:
PNP
Decidable
Undecidable
Sorting: (n log n)
Simulating Universe: O(n3)
Cracker Barrel: O(2n) and (n)
Fill tape with 2n *s: (2n)
Halting Problem: ()
CS588 Spring 2005 42
Smileys Problem
Input: n square tilesOutput: Arrangement of the tiles in a square, where the colors and shapes match up, or “no, its impossible”.
Thanks to Peggy Reed for making the Smiley Puzzles!
CS588 Spring 2005 44
How much work is the Smiley’s Problem?
• Upper bound: (O)O (n!)Try all possible permutations
• Lower bound: () (n)Must at least look at every tile
• Tight bound: ()
No one knows!
CS588 Spring 2005 45
NP Problems
• Can be solved by just trying all possible answers until we find one that is right
• Easy to quickly check if an answer is right – Checking an answer is in P
• The smileys problem is in NPWe can easily try n! different answers
We can quickly check if a guess is correct (check all n tiles)
CS588 Spring 2005 46
Is the Smiley’s Problem in P?
No one knows!
We can’t find a O(nk) solution.We can’t prove one doesn’t exist.
CS588 Spring 2005 47
This makes a huge difference!
1
100
10000
1E+06
1E+08
1E+10
1E+12
1E+14
1E+16
1E+18
1E+20
1E+22
1E+24
1E+26
1E+28
1E+30
2 4 8 16 32 64 128
n! 2n
n2
n log n
today
2032
time since “Big Bang”
log-log scale
Solving a large smileys problem either takes a few seconds, or more time than the universe has been in existence. But, no one knows which for sure!
CS588 Spring 2005 48
Who cares about Smiley puzzles?
If we had a fast (polynomial time) procedure to solve the smiley puzzle, we would also have a fast procedure to solve the 3/stone/apple/tower puzzle:
3
CS588 Spring 2005 49
3SAT Smiley
Step 1: Transform into smileys
Step 2: Solve (using our fast smiley puzzle solving procedure)
Step 3: Invert transform (back into 3SAT problem
CS588 Spring 2005 50
The Real 3SAT Problem(also can be quickly
transformed into the Smileys Puzzle)
CS588 Spring 2005 51
Propositional GrammarSentence ::= Clause
Sentence Rule: Evaluates to value of Clause
Clause ::= Clause1 Clause2
Or Rule: Evaluates to true if either clause is true
Clause ::= Clause1 Clause2
And Rule: Evaluates to true iff both clauses are true
CS588 Spring 2005 52
Propositional GrammarClause ::= Clause
Not Rule: Evaluates to the opposite value of clause (true false)
Clause ::= ( Clause )Group Rule: Evaluates to value of clause.
Clause ::= NameName Rule: Evaluates to value associated with Name.
CS588 Spring 2005 53
PropositionExample
Sentence ::= ClauseClause ::= Clause1 Clause2 (or)
Clause ::= Clause1 Clause2 (and)
Clause ::= Clause (not)
Clause ::= ( Clause )Clause ::= Name
a (b c) b c
CS588 Spring 2005 54
The Satisfiability Problem (SAT)
• Input: a sentence in propositional grammar
• Output: Either a mapping from names to values that satisfies the input sentence or no way (meaning there is no possible assignment that satisfies the input sentence)
CS588 Spring 2005 55
SAT Example
SAT (a (b c) b c) { a: true, b: false, c: true } { a: true, b: true, c: false }
SAT (a a) no way
Sentence ::= ClauseClause ::= Clause1 Clause2 (or)
Clause ::= Clause1 Clause2 (and)
Clause ::= Clause (not)
Clause ::= ( Clause )Clause ::= Name
CS588 Spring 2005 56
The 3SAT Problem• Input: a sentence in propositional
grammar, where each clause is a disjunction of 3 names which may be negated.
• Output: Either a mapping from names to values that satisfies the input sentence or no way (meaning there is no possible assignment that satisfies the input sentence)
CS588 Spring 2005 57
3SAT / SAT
Is 3SAT easier or harder than SAT?
It is definitely not harder than SAT, since all 3SAT problemsare also SAT problems. Some SAT problems are not 3SAT problems.
CS588 Spring 2005 58
3SAT Example
3SAT ( (a b c) (a b d)
(a b d) (b c d ) ) { a: true, b: false, c: false, d: false}
Sentence ::= ClauseClause ::= Clause1 Clause2 (or)
Clause ::= Clause1 Clause2 (and)
Clause ::= Clause (not)
Clause ::= ( Clause )Clause ::= Name
CS588 Spring 2005 59
3SAT Smiley• Like 3/stone/apple/tower puzzle, we
can convert every 3SAT problem into a Smiley Puzzle problem!
• Transformation is more complicated, but still polynomial time.
• So, if we have a fast (P) solution to Smiley Puzzle, we have a fast solution to 3SAT also!
CS588 Spring 2005 60
NP Complete• Cook and Levin proved that 3SAT was NP-
Complete (1971)• A problem is NP-complete if it is as hard
as the hardest problem in NP• If 3SAT can be transformed into a different
problem in polynomial time, than that problem must also be NP-complete.
• Either all NP-complete problems are tractable (in P) or none of them are!
CS588 Spring 2005 61
NP-Complete Problems• Easy way to solve by trying all possible
guesses• If given the “yes” answer, quick (in P) way to
check if it is right– Solution to puzzle (see if it looks right)– Assignments of values to names (evaluate logical
proposition in linear time)
• If given the “no” answer, no quick way to check if it is right– No solution (can’t tell there isn’t one)– No way (can’t tell there isn’t one)
CS588 Spring 2005 62
Traveling Salesperson Problem
– Input: a graph of cities and roads with distance connecting them and a minimum total distant
– Output: either a path that visits each with a cost less than the minimum, or “no”.
• If given a path, easy to check if it visits every city with less than minimum distance traveled
CS588 Spring 2005 63
Graph Coloring Problem– Input: a graph of nodes with edges
connecting them and a minimum number of colors
– Output: either a coloring of the nodes such that no connected nodes have the same color, or “no”.
If given a coloring, easy to check if it no connected nodes have the same color, and the number of colors used.
CS588 Spring 2005 64
Pegboard Problem- Input: a configuration of n pegs on a
cracker barrel style pegboard- Output: if there is a sequence of jumps that leaves a single peg, output that sequence of jumps. Otherwise, output false.If given the sequence of jumps, easy (O(n)) to check it is correct. If not, hard to know if there is a solution. Proof that variant of this
problem is NP-Complete isattached to today’s notes.
CS588 Spring 2005 65
Minesweeper Consistency Problem
– Input: a position of n squares in the game Minesweeper– Output: either a assignment of bombs to squares, or “no”.
• If given a bomb assignment, easy to check if it is consistent.
CS588 Spring 2005 66
Drug Discovery Problem– Input: a set of proteins,
a desired 3D shape– Output: a sequence of
proteins that produces the shape (or impossible)
Note: US Drug sales = $200B/year
If given a sequence, easy (not really) to check if sequence has the right shape.
Caffeine
CS588 Spring 2005 67
Complexity and Cryptography
• Next class