Computational Models — Lecture...

Computational Models — Lecture 51

Handout Mode

Ronitt Rubinfeld and Iftach Haitner.

Tel Aviv University.

March 28/30, 2016

1Based on frames by Benny Chor, Tel Aviv University, modifying frames by Maurice Herlihy, Brown University. Also including

modifications of Yishay Mansour.

Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 1 / 48

Talk Outline

I Algorithmic issues for CFL

I Chomsky Normal Form

I Pumping Lemma for context free languages

I Push Down Automata (PDA)

Next week:

I Equivalence of CFGs and PDAs

I Sipser’s book, 2.1, 2.2 & 2.3

Last time

I context-free languages

I context-free grammars

Formal Definition

A context-free grammar is a 4-tuple (V ,Σ,R,S), where

I V is a finite set of variables

I Σ is a finite set of terminals

I R is a finite set of rules: each rule is a variable and a finite string ofvariables and terminals.

I S is the start symbol.

I If u and v are strings of variables and terminals,and A→ w is a rule of the grammar,then uAv yields uwv , written uAv → uwv .

I u ∗→ v if u = v , or u → u1 → . . .→ uk → v for some sequenceu1,u2, . . . ,uk

Definition 1

The language of the grammar G, denoted L(G), is {w ∈ Σ∗ : S ∗→ w}

where ∗→ is determined by G.Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 4 / 48

Part I

Checking Membership

Checking Membership in a CFL

Challenge

Given a CFG G and a string w , decide whether w ∈ L(G)?

Initial Idea: Design an algorithm that tries all derivations.

Problem: If G does not generate w , we’ll never stop.

Possible solution: Use special grammars that are:

I just as expressive!

I better for checking membership.

Chomsky Normal Form (CNF)

A simplified, canonical form of context free grammars.G = (V ,Σ,R,S) is in a CNF, if every rule in in R has one of the followingforms:

A→ a, A ∈ V ∧ a ∈ ΣA→ BC, A ∈ V ∧ B,C ∈ V \ {S}S → ε.

Simpler to analyze: each derivation adds (at most) a single terminal, S onlyappears once, ε appears only at the empty word

What does parse tree look like?

Most internal nodes are degree 2 (except parents of leaves, whichare degree 1)

CNF: Theorem

Theorem 2Any context-free language is generated by a context-free grammar inChomsky Normal Form.

Proof Idea:

I Add new start symbol S0.

I Convert “long rules” to proper form.

I Eliminate all ε rules of the form A→ ε.

I Eliminate all “unit” rules of the form A→ B.

I Patch up rules so that grammar generates the same language.

Add new start symbol

Add new start symbol S0 and rule S0 → S

(Guarantees that new start symbol is never on right hand side of a rule)e.g.

S → A | ab | εA → baA | S

becomes

S0 → SS → A | ab | εA → baA | S

Convert "long rules": Phase 1 – no mixing ofterminals/nonterminals, and no multiple terminals

S → ccAbA | bc | bA → a | bb

becomes

S → XcXcAXbA | XbXc | bA → a | XbXb

Xc → cXb → b

Convert "long rules": Phase 2 – multiple nonterminals

S → AAAB

becomes

S → AN1

N1 → AN2

N2 → AB

Eliminate "ε-rules"

Repeat until all A→ ε rules are gone:

I remove A→ ε

I for any rule of form R → AB or R → BA, add R → B.

I for any rule of form R → AA add R → A and R → ε (unless R → ε hasalready been removed).

I for any rule of form R → A add R → ε (unless R → ε has already beenremoved.)

(Alternative description: Let W be the set of variables A such that A ∗→ ε. Foreach A ∈W , (1) remove A→ ε if present, (2) for any rule of form R → AB orR → BA, add R → B, even if A = B. Don’t need to add R → ε since R ∈W ).

Eliminate "unit rules"

Repeat until all unit rules removed

I remove some A→ B

I for each B → U add A→ U (unless A→ U was previously removed unitrule)

(Alternative description: Create a directed graph with nodes corresponding tovariables and edge from A to B if A→ B is a rule. For each strong componentin the graph, replace all variables by a single variable.)

Cleanup

Delete all “unreachable” rules, e.g.:

I delete all A→ A rules

I for each rule with A on LHS, make sure that A appears on RHS of somerule that is reachable from start variable.

I for each rule with A on RHS, make sure that A also appears on LHS of arule.

I for each variable A, make sure it can reach a terminal.

CNF: Example

S → ASA | aBA → B | SB → b | ε

Is transformed into:

S0 → AA1 | UB | a | SA | ASS → AA1 | UB | a | SA | ASA → b | AA1 | UB | a | SA | AS

A1 → SAU → aB → b

CNF: Bounded Derivation Length

Lemma 3For a CNF grammar G and w ∈ L(G) with |w | = n ≥ 1, it holds that w has aderivation of length 2n − 1.

Proof? consider the parsing tree for w

Advantage: Easier to check whether w ∈ L(G)

Checking Membership for Grammars in CNF Form

Given a CNF grammar G = (V ,Σ,R,S), we build a function Derive(A, x) thatreturns TRUE iff A ∗→ x .

Algorithm 4 (Derive(A, x))

I If x = ε: if A→ ε ∈ R (i.e., A = S) return TRUE, otherwise return FALSE.

I If |x | = 1: if A→ x ∈ R return TRUE, otherwise return FALSE.

I For each A→ BC and each partition x = xLxR (i.e. xL = x1...xj andxR = xj+1...x|x|):

I Call Derive(B, xL) and Derive(C, xR).I Return TRUE if both return TRUE.

I Return FALSE.

Test whether w ∈ L(G) by calling Derive(S,w)

Correctness?

I Procedure Derive can also output a parse tree for wI Have we critically used that G is in CNF?

Time Complexity of Derive

What is the time complexity T : N 7→ N of Derive?

I Each recursive call tests |R| rules and n partitions.I T (n) ≤ |R| · n · 2T (n − 1)I T (n) ∈ O((|R| · n)n).

Still exponential...

Efficient Algorithm

I Keep in memory the results of Derive(A, x).

I Main observation: Number of different inputs is |V | · n2. why???I Only |V | · n2 calls, each takes O(|R| · n).I T (n) ∈ O(|R| · n3 · |V |).

I Polynomial time!

I This approach is called Dynamic Programming

Basic idea:

I If number of different inputs is limited, say I(n).I Each run (excluding recursive calls) takes at most R(n) timeI Total running time is bounded by T (n) ≤ R(n)I(n).

Part II

Non-Context-Free Languages

Proving a Language is not a CFL

I The pumping lemma for finite automata and Myhill-Nerode theorem areour tools for showing that languages are not regular.

I We will now show a similar pumping lemma for context-free languages.

I It is slightly more complicated . . .

Pumping Lemma for CFL (also known as, the uvxyz Theorem)

Theorem 5

For any CFL L there exists ` ∈ N (“critical length"), such that for any w ∈ Lwith |w | ≥ `, there exist u, v , x , y , z ∈ Σ∗ such that w = uvxyz and

I For every i ≥ 0: uv ixy iz ∈ LI |vy | > 0, (“non-triviality")

I |vxy | ≤ ` (extra property that is helpful for us later!)

Basic Intuition

Let L be a CFL and a let w be a “very long” string in L. Then w must have a“tall” parse tree.

u v x y z

Hence, some root-to-leaf path must repeat a symbol. Why is that so?We have: T ∗→ uRz, R ∗→ vRy , and R ∗→ x .But then the second R could also produce vRy , giving uv2xy2z!

Proof of Thm 5

Let G be a CFG and let L = L(G).

I Let b be the max number of symbols in right-hand-side of any rule (whatis b for a CNF grammar?).

Since no node in a parse tree of G has more than b children, at depth dsuch tree has at most bd leaves.

I Let |V | be the number of variables in G, and set ` = b|V |+2.

Let w be a string with |w | ≥ `, and let T be parse tree for w (with respect toG) with fewest nodes

I T has height ≥ |V |+ 2

I Some path in T has length ≥ |V |+ 2

I Such path repeats a variable R

Proof of Thm 5 cont.

Set w = uvxyz

u v x y z

I Each occurrence of R produces a string

I Upper produces string vxy

I Lower produces string x

Proving uv ixy iz ∈ L for all i > 1

Replacing smaller by larger yields uv ixy iz, for i > 0.

u v x y z

Proving uv ixy iz ∈ L for i = 0

Replacing larger by smaller yields uxz.

Together, they establish:

I uv ixy iz ∈ L for all i ≥ 0

Proving |vy | > 0

If v and y are both ε, then

is a parse tree for w with fewer nodes than T , a contradiction.

Proving |vxy | ≤ `

u v x y z

R |V|+2

b |V|+2

I Without loss of generality both occurrences of R lie in bottom |V |+ 1variables on the path.

I The upper occurrence of R (from now on R1) generates vxy .I Subtree rooted at R1 is of height at most |V |+ 2.

Hence, |vxy | ≤ b|V |+2 = `.Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 29 / 48

Non CFL Example (1)

Claim 6L1 = {anbncn : n ∈ N} is not a CFL.

Proof: By contradiction. Assume L1 is a CFL with grammar G, let ` be thecritical length of G and consider w = a`b`c`. Let u, v , x , y , z be the stringswith w = uvxyz guaranteed by Thm 5 for w .

I Note that neither v nor y contain

I both a’s and b’s, orI both b’s and c’s,

(otherwise uv2xy2z would have out-of-order symbols).

I But if v and y contain only one letter, then uv2xy2z is imbalanced

Non CFL Example (2)

Claim 7

L2 = {aibjck : 0 ≤ i ≤ j ≤ k} is not context free.

Proof: By contradiction. Assume L2 is a CFL with grammar G, let ` be thecritical length of G and consider w = a`b`c`. Let u, v , x , y , z be the stringswith w = uvxyz guaranteed by Thm 5 for w .

I Neither v nor y contains two distinct symbols, because otherwiseuv2xy2z would have out-of-order symbols.

I vxy cannot be all the same letter (if a or b, can pump "up", if c canpump "down").

I |vxy | ≤ `, so either

I v contains only a’s and y contains only b’s, but then uv2xy2z hastoo few c’s.

I v contains only b’s and y contains only c’s, but then uv0xy0z hastoo many a’s.

♣Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 31 / 48

Non CFL Example (3)

Claim 8L3 = {ww : w ∈ {0,1}∗} is not context-free.

Proof:By contradiction. Assume L3 is a CFL with grammar G, let ` be the criticallength of G and consider w = 0`1`0`1`. Let u, v , x , y , z be the strings withw = uvxyz guaranteed by Thm 5 for w .

I Recall that |vxy | ≤ `I Assuming vxy is in the first half of w , then uv2xy2z “moves" a 1 into the

first position of second half.

I Assuming vxy is in the second half, then uv2xy2z “moves" a 0 into thelast position of first half.

I Assuming vxy straddles the midpoint, then pumping down to uxz yields0`1i0j1` where i and j cannot both be `.

♣Note that {wwR : w ∈ {0,1}∗} is a CFL.

Part III

Push-Down Automata

String Generators and String Acceptors

I Regular expressions are string generators – they tell us how to generateall strings in a language L

I Finite Automata (DFA, NFA) are string acceptors – they tell us if aspecific string w is in L

I CFGs are string generators

I Are there string acceptors for CFLs?

I YES! Push-down automata

A Finite Automaton

A PushDown Automaton

(ignore the ’$’ sign)Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 36 / 48

Example 1 — PDA for L1 = {0n1n : n ≥ 0}Informally:

1. Read input symbols

1.1 Push each read 0 on the stack1.2 Pop a 0 for each read 1

2. Accept if stack is empty after last symbol read, and no 0 appears after 1

Recall that L1 is not regular

Example 2 — PDA for L2 = {aibjck : i = j ∨ i = k}Informally:

Read and push a’sEither pop and match with b’s or pop and match with c’s

A non-deterministic choice

Pushdown Automaton (PDA) — Formal Definition

A PDA is a 6-tuple (Q,Σ, Γ, δ,q0,F ), where

I Q is a finite set called the states,

I Σ is a finite set called the input alphabet,

I Γ is a finite set called the stack alphabet,

I δ : Q × Σε × Γε → P(Q × Γε) is the transition function,2

I q0 ∈ Q is the starting state, and

I F ⊆ Q is the set of accepting states.

2Xε := X ∪ {ε}.Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture 5 March 28/30, 2016 39 / 48

The language accepted by a PDA

I A pushdown automaton (PDA) M accepts a string w , if there is a“computation" of M on w (see next slide) that leads to an accepting state.

I The language accepted by M, denoted L(M), is the set of all stringsw ∈ Σ∗ accepted by M.

I A (non-deterministic) PDA may have many computations on a singlestring

Model of Computation

The following is with respect to M = (Q,Σ, Γ, δ,q0,F ).

Definition 9 (δ∗)

For w ∈ Σ∗ let δ̂(q,w , s) be all pairs (q′, s′) ∈ Q × Γ∗ for which existw ′1, . . . ,w

′m ∈ Σε, states r1, . . . , rm ∈ Q and strings s0, s1, . . . sm ∈ Γ∗ s.t.:

1. w = w ′1, . . . ,w′m, r0 = q, rm = q′, s0 = s and sm = s′

2. For every i ∈ {0, . . . ,m − 1} exist a,b ∈ Γε and t ∈ Γ∗ s.t.:

2.1 (ri+1,b) ∈ δ(ri ,w ′i+1,a)2.2 si = at and si+1 = bt

Namely, (q′, s′) ∈ δ̂(q0,w , ε) if after reading w (possibly with in-betweenε moves), M can find itself in state q′ and stack value s′.

I M accepts w ∈ Σ∗ if ∃q′ ∈ F such that (q′, t) ∈ δ̂(q0,w , ε) for some t .

Diagram Notation

When drawing the automata diagram, we use the following notation

I Transition a,b → c from state q to q′ means (q′, c) ∈ δ(q,a,b),and informally means the automata

I read a from inputI pop b from stackI push c onto stack

I Meaning of ε transitions ((informally):

I a = ε : don’t read inputI b = ε: don’t pop any symbolI c = ε: don’t push any symbol

A PDA for L1 = {0n1n : n ≥ 0}

Claim 100011 ∈ L(P).

Proof: takew ′1 = ε w ′2 = 0 w ′3 = 0 w ′4 = 1 w ′5 = 1 w ′6 = ε

s0 = ε s1 = $ s2 = 0$ s3 = 00$ s4 = 0$ s5 = $ s6 = εr0 = q1 r1 = q2 r2 = q2 r3 = q2 r4 = q3 r5 = q3 r6 = q4

A PDA for L1 = {0n1n : n ≥ 0}We want to show that L(P) = L1 = {0n1n : n ≥ 0}What do we need to prove?

Claim 11

I δ̂(q1, ε, ε) = {(q1, ε), (q2, $)}.

I δ̂(q1,0k , ε) = {(q2,0k $)}, for k ≥ 1.

I δ̂(q1,0k 1i , ε) = {(q3,0k−i $)}, for k > i ≥ 1.

I δ̂(q1,0k 1k , ε) = {(q3, $), (q4, ε)}, for k ≥ 1.

I δ̂(q1,w , ε) = ∅, for w 6∈ {0k 1i |k ≥ i ≥ 0}.

Knowing when stack is empty

It is convenient to be able to know when the stack is empty, but there is nobuilt-in mechanism to do that.

Solution

1. Start by pushing $ onto stack.

2. When you see it again, stack is empty.

A PDA for L2 = {aibjck : i = j ∨ i = k}

A PDA for L2 = {aibjck : i = j ∨ i = k}, cont.

I Non-determinism is essential here!

I Unlike finite automata, non-determinism does add power.

I But we saw deterministic algorithm to deicide any CFL (and as we seelater, CFLs are exactly the languages decided by PDAs)!

I How to prove that non-determinism adds power?

...I Does not seem trivial or immediate.

I Another example: L = {xnyn : n ≥ 0} ∪ {xny2n : n ≥ 0} is accepted bya non-deterministic PDA, but not by a deterministic one. (Proof? Book!)

PDA Languages

The Push-Down Automata Languages, LPDA, is the set of all languages thatcan be described by some PDA:

I LPDA = {L(M) : M is a PDA}

It is immediate that LPDA ) LDFA: every DFA is just a PDA that ignores thestack.

I LCFG ⊆ LPDA ?

I LPDA ⊆ LCFG ?

I LPDA = LCFG !!!

Computational Models — Lecture...

Documents

Transcript of Computational Models — Lecture...

Iftach Haitner Microsoft Research New England

1 On the Power of the Randomized Iterate Iftach Haitner, Danny Harnik, Omer Reingold.

Yevgeniy Dodis Iftach Haitner Aris Tentes On the (In)Security of RSA Signatures 1.

Ronitt Rubinfeld - People | MIT CSAILS. Ravi Kumar (graduated 1997, Cornell University). Currently at Google. Tugkan Batu (graduated 2001, Cornell University). Currently at London

On Testing Convexity and Submodularity Michal Parnas Dana Ron Ronitt Rubinfeld.

Testing Properties of Collections of Distributionspeople.csail.mit.edu/ronitt/papers/Revision1OfTR10-157.pdf · Testing Properties of Collections of Distributions Reut Levi∗ Dana

A very D ennison S w iftach S ystem s.pfs.averydennison.com/content/.../product-families/Fasteners/docs/... · A very D ennison¨ S w iftach¨ S ystem s. Inform. Identify. Package.

The Bloomier Filter Bernard Chazzelle Princeton Un., NEC Lab. Joe KillianNEC Lab. Ronitt RubinfeldNEC Lab. Ayellet TalTechnion, Princeton Un. Presented.

Ideal Parent Structure Learning School of Engineering & Computer Science The Hebrew University, Jerusalem, Israel Gal Elidan with Iftach Nachman and Nir.

Iftach Pearlman City of Leesburg, GIS Keyetta Jackson Lake County, GIS Francis Franco Metropolitan Planning Organization, GIS.

1 LumiCal Optimization Simulations Iftach Sadeh Tel Aviv University Collaboration High precision design May 6 th 2008.

A Linear Lower Bound on the Communication Complexity of Single-Server PIR Weizmann Institute of Science Israel Iftach HaitnerJonathan HochGil Segev.

A Clustering Algorithm for LumiCal Halina Abramowicz, Ronen Ingbir, Sergey Kananov, Aharon Levy, Iftach Sadeh Tel Aviv University DESY Collaboration High.

Computational Models - Lecture 111tau-cm2013.wdfiles.com/local--files/course-schedule/Blecture11_h.pdf · Iftach Haitner and Yishay Mansour (TAU) Computational Models Lecture 11 May

Computational Models — Lecture 91tau-cm2015b.wdfiles.com/local--files/course-schedule/Blecture9_h.pdf · Ronitt Rubinfeld and Iftach Haitner (TAU) Computational Models, Lecture

Iftach Haitner and Yishay Mansour.tau-cm2012.wdfiles.com/local--files/course-schedule/Blecture7.pdf · Computational Models - Lecture 71 Iftach Haitner and Yishay Mansour. Tel Aviv

Something for almost nothing: Advances in sublinear time algorithms Ronitt Rubinfeld MIT and Tel Aviv U.

Finding Collisions in Interactive Protocols A Tight Lower Bound on the Round Complexity of Statistically-Hiding Commitments Iftach Haitner, Jonathan Hoch,

Approximating the MST Weight in Sublinear Time Bernard Chazelle (Princeton) Ronitt Rubinfeld (NEC) Luca Trevisan (U.C. Berkeley)

Computational Models - Lecture 111tau-cm2015.wdfiles.com/local--files/course-schedule/lecture11_B.pdf · Computational Models - Lecture 111 Handout Mode Guest LecturerL: Iftach Haitner