1

Unit 3

Regular Operations

Deterministic and Nondeterministic Automata

Reading: Sipser, chapter 1

2

Reminder – Regular Languages

A regular language is a language for which there exists a recognizing finite automaton.

Notes:

1. For every symbol a in the language La = {a}is regular.

2. For every word w=a1a2…an over the language Lw = {w} is regular.

Proof: Draw DFAs.

3

Regular Operations

• Union

• Intersection

• Complement

• Minus

• Concatenation

4

Union of Two Regular Languages

Theorem (Sipser 1.12): If L1 and L2 are regular

languages, then so is L1 L2.

(The regular languages are „closed‟ under

the union operation.)

Proof idea: L1 and L2 are regular, therefore there

are two FAs A and B, with L1=L(A) and L2=L(B).

Using these two DFAs, we will make a third

automaton M such that L(M) = L1 L2.

5

Proof Idea

L1={w{0,1}* | w ends with 0}

L2={w{0,1}* | |w| is odd}

p0

p1

1

01

0

A:

q0

q1

0,10,1

B:

p1

p0

6

Proof Idea

L1={w{0,1}* | w ends with 0}

L2={w{0,1}* | |w| is odd}

1

01

0

A:

q0

q1

0,10,1

B:

1101111011

11011 11011

11011

1101111011 11011

11011 11011

1101111011

11011

11011

7

The Union Automaton

Let A recognizes L1 where A = (QA, , A, q0A, FA),

B recognizes L2 where B = (QB, , B, q0B, FB).

Construct M = ( Q, , , q0, F) to recognize L1 L2

Basic idea:

• The automaton M runs A and B in „parallel‟ on a string w.

• At the end, the final state (r1,r2) indicates whether wL1 (does r1FA?) or wL2 (does r2FB?)

8

1. Q={ (rA,rB) QAQB } (rA in QA and rB in QB) .

2. is the same for both A and B (otherwise =AB)

3. For each (rA,rB)Q and , define

((rA,rB),)=(A(rA,),B(rB,)).

4. q0 is the pair (q0A,q0B)

5. F = { (rA,rB) | rAFA or rBFB

Alternatively F = FAQB QAFB .

The Union Automaton (cont.)

9

Example:

Let ={0,1},

L1={words of odd length},

L2={words ending with a 0}.

Define L1 L2 and build the union automaton.

Answer: In class

10

Union - Formal Proof

We have to prove that L(M)= L1 L2.

To prove the equality we have to prove that:

• L1 L2 L(M).

and

• L(M) L1 L2.

11

L1 L2 L(M).

• If w L1L2 then wL1 or wL2

• That means:

‟A(q0A,w)=r1 and ‟B(q0B,w)=r2

where r1FA or r2FB

• So ‟((q0A,q0B),w)= (‟A(q0A,w), ‟B(q0B,w)) = (r1,r2)

where (r1,r2) (FAQB) or (r1,r2)(QAFB).

• By definition of F:

‟(q0,w)=(r1,r2) (FAQB)(QAFB)=F

• That means wL(M).

12

L(M) L1 L2

• If wL(M) then ‟(q0,w)= ‟((q0A,q0B),w)=

=(‟A(q0A,w), ‟B(q0B,w)) F

• By definition of F:

(‟A(q0A,w), ‟B(q0B,w))F means

‟A(q0A,w)FA or ‟B(q0B,w))FB

• So w belongs to L1 or to L2.

• By definition of union: w L1 L2

• That means L(M) L1 L2.

13

Intersection

Theorem:

The class of regular languages is closed under

the intersection operation:

If L1 and L2 are regular, so is L1L2.

Basic idea:

• proof by construction: construct an automaton

M that uses in parallel the automaton A of L1

and the automaton B of L2 and accepts word if

both automata stop in accepting states.

14

Intersection - example

Example: Let ={0,1},

L1={words of odd length},

L2={words ending with a 0}.

Define L1 L2 and describe graphically the intersection automaton.

Answer: In class

15

The Intersection Automaton

Let A recognizes L1 where A=(QA, , A, q0A, FA)

and B recognizes L2 where B=(QB, , B, q0B, FB)

Construct M = ( Q, , , q0, F) to recognize L1 L2,

1. Q={ (rA,rB) QAQB} .

2. = AB

3. For each (rA,rB)Q and , define

((rA,rB),)=(A(rA,),B(rB,)).

16

4. q0 = (q0A,q0B)

5. F = { (rA,rB) | rAFA and rBFB}

Alternatively F = FAFB.

Proof: Similar to the proof for Union.

The Intersection Automaton

17

Complement (~L)

Theorem:

The class of regular languages is closed under the complement operation.

If L is regular, so is ~L (the complement language of L).

Proof: By construction. Construct an automaton M that uses the automaton A of L and rejects all words A accepts and vice versa.

18

The Complement Automaton

Let A recognize L where A = (QA,,A,q0A,FA).

Construct M = ( Q, , , q0, F) that recognizes ~L.

1. Q= QA

2. = A

3. is identical to A

4. q0 =q0A

5. F = { q | q FA } or F = QA - FA.

Easy to verify: w L(A) iff wL(M)

19

The Set-minus Operation

Theorem: The class of regular languages is closed under the minus operation:

If L1 and L2 are regular, so is L1-L2.

Proof: Based on the fact that

L1-L2 = L1(~L2)

So?

20

Concatenation of L1 and L2

Definition: L1◦L2 = { xy | xL1 and yL2 }

Example: {a,b}◦{0,11} = {a0,a11,b0,b11}

Theorem:

If L1 and L2 are regular languages, then so is

L1◦L2. (The regular languages are „closed‟

under concatenation.)

21

Proving Concatenation Thm.

Consider the concatenation:

{1,01,11,001,011,…}◦{0,000,00000,…}

(That is: the binary strings that end with a “1”,

followed by an odd number of 0‟s.)

Basic idea: Run an automaton for L2 after an

automaton for L1.

Problem: given a string w=01010, how does

the automaton know where the L1 prefix ends

and the L2 suffix starts?

We need an M with ‘lucky guesses’.

22

Non-determinism

In order to prove that regular language are

closed under concatenation (and for many other

usages) we use a new concept:

Non-determinism

23

Nondeterminism

Nondeterministic machines are capable of

being lucky (or they can be assisted with

an oracle).

A nondeterministic finite automaton

can have transition rules like the following:

q1 q2

q1

q21

q21

24

q1 q2 q3 q4

0,1

1 0, 1

0,1

A Nondeterministic Automaton

q4q3q2

25

q1

0,1

1 0, 1

0,1

010101010101

0101

01010101 0101 010101010101 0101010101010101 010101010101

A Nondeterministic Automaton

26

DeterministicComputation

Non-DeterministicComputation

accept (or reject)

accept

reject

27

A Nondeterministic Automaton

This automaton accepts “0110”, because

there exists a path that leads to an

accepting state, namely:

q1 q1 q2 q3 q4 q40 1 1 0

q1 q2 q3 q4

0,1

1 0, 1

0,1

28

A Nondeterministic Automaton

The string 1 is rejected: on “1” the

automaton can only reach: {q1,q2,q3}.

q1 q2 q3 q4

0,1

1 0, 1

0,1

29

Nondeterminism ~ Parallelism

• For any string w, the nondeterministicautomaton can be in a set of possible states.

• If the final set of the possible states contains an accepting state, then the automaton accepts the string.

• The automaton processes the input in aparallel fashion. Its computational path is no longer a line, but a tree.

30

Def. of Nondeterministic FA (NFA)

• A nondeterministic finite automaton (NFA) M is

defined by a 5-tuple M=(Q,,,q0,F), with

– Q: finite set of states

– : finite alphabet

– : transition function :QP(Q)

– q0Q: start state

– FQ: set of accepting states

31

Nondeterministic :QP(Q)

• The function :QP(Q) is the crucial

difference. It means: “When reading symbol „a‟

while in state q, one can go to one of the states

in (q,a)Q.”

• The in = {} takes care of the

empty string transitions.

• (q,a) can be an empty set.

Example:

A={Q, , , q1, F} s.t.

• Q={q1,q2,q3,q4}

• ={0,1}

• is given in the table:

• q1 is the start state

• F={q4}

32

q1 q2 q3 q4

0,1

1 0, 1

0,1

A:

0 1

q1 {q1} {q1,q2}

q2 {q3} {q3}

q3 {q4}

q4 {q4} {q4}

33

The Language of an NFA

• Informally:

The language of an NFA A is a set of words that

A accepts (recognizes), i.e. when reading them it

might stop in an accepting state.

• Formally:

L(A)={w | w*, ‟(q0,w)F}

means:

There exists a path from q0 to rF

for the input string w.

?

34

The language of an NFA

Even more formally: A nondeterministic FA

M = (Q,,,q,F) accepts a string w = w1…wn if

and only if we can rewrite w as y1…ym with

yi and there is a sequence r0…rm of states

in Q such that:

1) r0=q0

2) ri+1 (ri,yi+1) for all i=0,…,m–1

3) rm F

35

Lets Practice NFA’s

[Sipser 1.5]: Give NFAs with the specified number of states that recognize the following languages over the alphabet ={0,1}:

1. { w | w ends with 00}, three states.

2. {0}; two states.

3. { w | w contains even number of 0s, or exactly two 1s}, six states.

4. { w | w contains even number of 0s, and exactly two 1s}, six states.

5. {0n | nN }, one state.

36

Additional Practice Problems

1. Describe an NFA for L={0i1j | i mod 3 = 1, j mod 2 = 1}

2. Give a formal description of this NFA:

3. What is the language of this NFA?

11 0,

11 0,

Additional Practice Problems

• Let ={1,2,3}, let L be the languages of all words over in which the letters form a non-decreasing sequence. L={w | w 1*2*3*}

Example: {113, 2333, 1123}L, {1132, 321} L.

• Describe

– An NFA,

– An NFA with no -transitions,

– A DFA, that recognizes L.

37

38

DFA vs. NFA

• Can an NFA be more powerful than an DFA

(What do we mean by powerful)?

• We will show that every NFA has an equivalent

DFA (thus NFA has the same power as DFA).

• However, NFAs can be much simpler.

39

How does an NFA work?

• A nondeterministic computation is a tree of

possibilities.

• The root of the tree is the start state of the

automaton.

• Each branching point is a computation at

which automaton has multiple choices.

• An NFA accepts an input string if there exists

at least one computation that leads to an

accept state.

40

DeterministicComputation

Non-DeterministicComputation

accept (or reject) accept

reject

0, 1

0, 1

41

Equivalence of FA

Definition: Two automata are equivalent if they

recognize the same language.

Example:A:

B:

L(A)=L(B)0, 1 0, 1

0, 10, 1

42

NFA vs. DFA

Theorem: Every NFA has an equivalent DFA.

Proof idea: By construction - given an NFA we

will build an equivalent DFA.

43

How can we do it?

• Consider an example in which we can get from some state q on an input letter a to more than one state.

• The next state - (q,a) - is one of {qi, qj, qk}.

q

a

qk

a

qjqi

a

44

• It means that the next state in

an NFA is one state among

several candidates.

• But in a DFA only one state

can be the next state.

How can we do it? (cont.)

qjqi qk

q

a aa

Solution:

One state in a DFA = a set

of states in an NFA:

(q,a)= {qi, qj, qk}.

45

• What states can we reach starting from the state q on input a.

How can we do it? (cont.)

qjqi qk

q

a aa

ql qm qn qp

46

Formally:

‟(q, a)= ((q,a),)= ({qi,qj,qk},) = (qi,)(qj,) (qk,)={ql,qm,qn,qp}

Basic idea:

The next state in a DFA is the set of all the states in an NFA to which you can get from the set of all the current states in an NFA.

How can we do it? (cont.)

qjqi qk

q

a aa

ql qm qn qp

47

Let A be an NFA where A = (QA, , A, q0A, FA) .

We construct a DFA M=(Q, , , q0, F)

equivalent to A, as follows:

• Q = P(QA)

• =

• For each RQ and ,

• q0 = q0A

• F = { RQ | there exists rR such that rFA}

NFA DFA (with no transitions)

Rr

A rR

,,

qi, qk, ql

48

Explanation

a

(R,a)=S,

SF if (SFA)

qt

qpql

qm

qn

qs

qp

R:

(R,a)

49

Example 1 (no - transition)

Convert the following nondeterministic finite automaton into a deterministic one

q0 q1q2

1 0

0,1

q3

0

1

Solution: In class

50

What about -transitions?

Definition:

The - closure of a state is a set of all the states reachable from the current state using -transitions only.

E(q)= { pQ | ‟(q,i) = p, i0}

Informally:

Use all the transitions you can. Add the states you reach to -closure. Try to reach as many states as you can.

51

E(q0) = {q0, q1, q2}

E(q1) = {q1, q2}

E(q2) = {q2}

E(q3) = {q3, q4}

E(q4) = {q4}

The - closure

q0 q1 q2 q3 q4

52

Let A be an NFA where A = (QA, , A, q0A, FA) .

We construct a DFA M=(Q, , , q0, F)

equivalent to A, as follows:

• Q = P(QA)

• =

For each RQ and ,

q0 = E(q0A)

• F = { RQ | there exists rR such that rFA}

NFA DFA

Rr

A rER

,,

qi, qk, ql

53

Explanation

(R,a)

aqt

qpql

qm

qn

qs

qp

R:

*

*

qa

qb

qc

54

NFA=DFA

• The automaton we have is a deterministic automaton equivalent to A (need to prove)

Conclusion:

A language is regular iff some NFA recognizes it.

• Proof:

– Regular DFA NFA

– NFA DFA regular

55

Example 2 (with - transition)

Convert the following nondeterministic finite automaton into a deterministic one

Solution: In class

q0 q1q2

q3

0 1

0

0

0

56

Formal Proof of Construction

• Recall: We suggested a conversion of NFA

A=(QA,,A,q0A,FA) into a DFA M=(Q, , , q0,F)

• We need to show that L(M)=L(A).

• We first show that for each word w*,

‟A(E(q0A),w) = ‟(q0,w).

• Tool: induction on word‟s length.

proof in class (and next slide)

57

’A(E(q0A),w) = ’(q0,w)• Substitute E(q0A)=q0

• Induction basis: |w|=1 and w=a .

‟A(q0,a) = (by definition of ‟) rq0 E(A(r,a))

= (by definition of ) (q0,a).

• Induction hypothesis: Assume the theorem is true any word w, |w|<n: ‟A(q0,w) = ‟(q0,w)

• Induction step: Show that the theorem is true for wa.

aaa

aa

wqwqrE

rEwq

wqr

A

wqr

AA

AA

,',,',

,,

00

,'

,

0

'

0

0'

by definition of ’ (NFA) by hypothesis

by definition of for M by definition of ’ (DFA)

58

L(M)=L(A)

wL(A)

‟A(q0,w)=P and P FA

by the previous theorem by definition of F

‟(q0,w) =P and PF

wL(M)

59

• Recall: A language is regular iff some NFA recognizes it.

• Let M1 recognize L1 where M1= (Q1, , 1, q01, F1) and let

M2 recognize L2 where M2 = (Q2, , 2, q02, F2) .

• Construct M to recognize L1◦L2, where M = ( Q, , , q0, F).

q01

F1

q02

M1 M2

F2

Back to Concatenation (finally )

60

Concatenation

M = ( Q, , , q0, F) where

Q=Q1Q2; = ; q0=q01 ; F= F2

=12 {for each state qF1, (q,)=q02}

q01 q02

M1 M2

M

F1 F2

61

Concatenation

• M starts by simulating M1 for a while

• Then M moves non-deterministically from a final state of M1 to the initial state of M2

• Then M simulates M2 for the rest of the input.

62

Example:

Consider the concatenation:

{1,01,11,001,011,…}◦{0,000,00000,…}

(That is: the binary strings that end with a “1”,

followed by an odd number of 0‟s.)

In Class exercise:

Draw an NFA for L1◦L2

63

Concatenation - Proof

L(M)=L(M1◦M2)

• The basic idea of the proof is:

• Let w*, then ‟(q0,w)=q for some qF iff

w=uv where ‟(q01,u)=f for some fF1 and

‟(q02,v)=q for some qF2.

• Therefore M accepts w iff M1 accepts u and

M2 accepts v.

64

Union

• We already saw that the regular languages are

closed under the union operation.

• We will prove it again using a simple non-

deterministic automaton.

• Let M1 recognize L1 where M1=(Q1, , 1, q01, F1)

and M2 recognize L2 where M2=(Q2, , 2, q02, F2)

• Construct M to recognize L1 L2, where M=( Q,

, , q0, F).

65

UnionWe construct M in a way that uses non-determinism to

“guess” whether the input word w is in L1 or L2 and then

process the string exactly as the corresponding

automaton (M1 or M2).

q01

F1

q02

F2

M1 M2

66

Union

M = ( Q, , , q0, F) where:

• Q=Q1Q2{q0}

•

• =12

{(q0,)={q01,q02} }

• q0

• F=F1F2

q01

F1

q02

F2

M1

M2

q0

M

67

Union - Proof

L(M)=L(M1) ∪ L(M2)

The basic idea of the proof is:

• if w* , then ‟(q0,w)=q for some qF iff

either ‟(q01,w)=q for some qF1 or

‟(q02,w)=q for some qF2

• Hence M accepts w iff M1 accepts w or M2

accepts w.

68

Kleene (star) Closure

Theorem: If L is regular then L* is also regular.

Proof:

• Let M1 recognize L1 where M1 = (Q1,,1,q01,F1).

• We construct M=(Q,,,q0,F) such that

L(M)=L1*.

Kleene (star) closure

M = ( Q, , , q0, F) where

• Q=Q1{q0}; = ; q0=q0 ; F= {q0} F1

• =1 {(q0, )=q01} { qF, (q,)=q01}

q01

F1

M1

q0

M

70

Kleene (star) closure

Theorem: L(M)=L1*

Proof: Homework.

Directions:

1. If wL1* then there exists a k such that

w L1k …

2. If wL(M) then its process ends in a state

from F and there exists a k such that w

„visits‟ k times in states from F.