Models of Computationby Dr. Michael P. Frank, University of Florida

Modified and extended by Longin Jan Latecki, Temple University

Rosen 7Rosen 7thth ed., Ch. 13.1 ed., Ch. 13.1

Modeling Computation

• An algorithm:An algorithm:– A description of a computational procedure.A description of a computational procedure.

• Now, how can we model the computer Now, how can we model the computer itself, and what it is doing when it carries itself, and what it is doing when it carries out an algorithm?out an algorithm?– For this, we want to model the abstract process For this, we want to model the abstract process

of of computationcomputation itself. itself.

Early Models of Computation

• Recursive Function TheoryRecursive Function Theory– Kleene, Church, Turing, Post, 1930’sKleene, Church, Turing, Post, 1930’s

• Turing Machines Turing Machines – Turing, 1940’s– Turing, 1940’s• RAM Machines RAM Machines – von Neumann, 1940’s– von Neumann, 1940’s• Cellular Automata Cellular Automata – von Neumann, 1950’s– von Neumann, 1950’s• Finite-state machines, pushdown automataFinite-state machines, pushdown automata

– various people, 1950’svarious people, 1950’s• VLSI models VLSI models – 1970s– 1970s• Parallel RAMs, etc. Parallel RAMs, etc. – 1980’s– 1980’s

§11.1 – Languages & Grammars

• Phrase-Structure GrammarsPhrase-Structure Grammars

• Types of Phrase-Structure GrammarsTypes of Phrase-Structure Grammars

• Derivation TreesDerivation Trees

• Backus-Naur FormBackus-Naur Form

Computers as Transition Functions

• A computer (or really any physical system) can be A computer (or really any physical system) can be modeled as having, at any given time, a specific state modeled as having, at any given time, a specific state ssSS from some (finite or infinite) from some (finite or infinite) state space state space SS..

• Also, at any time, the computer receives an Also, at any time, the computer receives an input symbolinput symbol iiII and produces an and produces an output symboloutput symbol ooOO. . – Where Where II and and OO are sets of symbols. are sets of symbols.

• Each “symbol” can encode an arbitrary amount of data.Each “symbol” can encode an arbitrary amount of data.

• A computer can then be modeled as simply being a A computer can then be modeled as simply being a transition function transition function TT::SS××II → → SS××OO..– Given the old state, and the input, this tells us what the Given the old state, and the input, this tells us what the

computer’s new state and its output will be a moment later.computer’s new state and its output will be a moment later.• Every model of computing we’ll discuss can be viewed Every model of computing we’ll discuss can be viewed

as just being some special case of this general picture.as just being some special case of this general picture.

Language Recognition Problem

• Let a Let a language language LL be any set of some arbitrary objects be any set of some arbitrary objects ss which will be dubbed “sentences.”which will be dubbed “sentences.”– That is, the “legal” or “grammatically correct” sentences of the That is, the “legal” or “grammatically correct” sentences of the

language.language.

• Let the Let the language recognition problemlanguage recognition problem for for LL be: be:– Given a sentence Given a sentence ss, is it a legal sentence of the language , is it a legal sentence of the language LL? ?

• That is, is That is, is ssLL??

• Surprisingly, this simple problem is as general as our Surprisingly, this simple problem is as general as our very notion of computation itself! very notion of computation itself!

Vocabularies and Sentences

• Remember the concept of strings Remember the concept of strings ww of of symbols symbols ss chosen from an alphabet chosen from an alphabet ΣΣ??– An alternative terminology for this concept: An alternative terminology for this concept:

• SentencesSentences σσ of of wordswords υυ chosen from a chosen from a vocabulary vocabulary VV..

– No essential difference in concept or notation!No essential difference in concept or notation!

• Empty sentence (or string): Empty sentence (or string): λλ (length 0) (length 0)

• Set of all sentences over Set of all sentences over VV: : Denoted Denoted VV**..

Grammars

• A formal A formal grammargrammar GG is any compact, precise is any compact, precise mathematical definition of a language mathematical definition of a language LL..– As opposed to just a raw listing of all of the language’s As opposed to just a raw listing of all of the language’s

legal sentences, or just examples of them.legal sentences, or just examples of them.

• A grammar implies an algorithm that would A grammar implies an algorithm that would generate all legal sentences of the language.generate all legal sentences of the language.– Often, it takes the form of a set of recursive definitions.Often, it takes the form of a set of recursive definitions.

• A popular way to specify a grammar recursively is A popular way to specify a grammar recursively is to specify it as a to specify it as a phrase-structure grammar.phrase-structure grammar.

PSG Example – English Fragment

We have We have GG = ( = (VV, , TT, , SS, , PP)), where:, where:• VV = { = {(sentence)(sentence), , (noun phrase)(noun phrase), ,

(verb phrase)(verb phrase), , (article)(article), , (adjective)(adjective),, (noun)(noun), , (verb)(verb), , (adverb)(adverb), , aa, , thethe, , largelarge,,

hungryhungry, , rabbitrabbit, , mathematicianmathematician, , eatseats, , hopshops,, quicklyquickly, , wildlywildly}}

• TT = { = {aa, , thethe, , largelarge, , hungryhungry, , rabbitrabbit, , mathematicianmathematician,, eatseats, , hopshops, , quicklyquickly, , wildlywildly}}

• SS = = (sentence)(sentence)• PP = = (see next slide)(see next slide)

Productions for our Language

PP = { = { (sentence)(sentence) → → (noun phrase)(noun phrase) (verb phrase)(verb phrase),,(noun phrase)(noun phrase) → → (article) (adjective) (noun)(article) (adjective) (noun),,(noun phrase)(noun phrase) → → (article) (noun)(article) (noun),,(verb phrase)(verb phrase) → → (verb) (adverb)(verb) (adverb),,(verb phrase)(verb phrase) → → (verb)(verb), , (article)(article) → → aa, , (article)(article) → → thethe,,(adjective)(adjective) → → largelarge, , (adjective)(adjective) → → hungryhungry,,(noun)(noun) → → rabbitrabbit, , (noun)(noun) → → mathematicianmathematician,,(verb)(verb) → → eatseats, , (verb)(verb) → → hopshops,,(adverb)(adverb) → → quicklyquickly, , (adverb)(adverb) → → wildlywildly } }

Backus-Naur Form

sentencesentence noun phrasenoun phrase verb phraseverb phrasenoun phrasenoun phrase articlearticle [[adjectiveadjective]] nounnounverb phraseverb phrase verbverb [[adverbadverb]]articlearticle a a | | thetheadjectiveadjective large large | hungry| hungrynounnoun rabbit rabbit | mathematician| mathematicianverbverb eats eats | | hopshopsadverbadverb quickly quickly | | wildlywildly

Square brackets []mean “optional”

Vertical barsmean “alternatives”

A Sample Sentence Derivation

(sentence)(sentence) (noun phrase) (verb phrase) (noun phrase) (verb phrase)

(article) (adj.) (noun) (verb phrase)(article) (adj.) (noun) (verb phrase) (art.) (adj.) (noun) (verb) (adverb)(art.) (adj.) (noun) (verb) (adverb) thethe (adj.) (noun) (verb) (adverb)(adj.) (noun) (verb) (adverb)

the the largelarge (noun) (verb) (adverb)(noun) (verb) (adverb) the large rabbit the large rabbit (verb) (adverb)(verb) (adverb)

the large rabbitthe large rabbit hopshops (adverb)(adverb) the large rabbit hops quicklythe large rabbit hops quickly

On each step,we apply a production to a fragment of the previous sentence template to get a new sentence template. Finally, we end up with a sequence of terminals (real words), that is, a sentence of our language L.

04/19/23 13

Derivation Tree

Phrase-Structure Grammars• A A phrase-structure grammarphrase-structure grammar (abbr. PSG) (abbr. PSG)

GG = ( = (VV,,TT,,SS,,PP)) is a 4-tuple, in which: is a 4-tuple, in which:– VV is a vocabulary (set of words) is a vocabulary (set of words)

• The “template vocabulary” of the language.The “template vocabulary” of the language.– TT VV is a set of words called is a set of words called terminalsterminals

• Actual words of the language.Actual words of the language.• Also, Also, NN : :≡ ≡ VV − − TT is a set of special “words” called is a set of special “words” called

nonterminalsnonterminals. (Representing concepts like “noun”). (Representing concepts like “noun”)– SSNN is a special nonterminal, the is a special nonterminal, the start symbol.start symbol.– PP is a set of is a set of productionsproductions (to be defined). (to be defined).

• Rules for substituting one sentence fragment for another.Rules for substituting one sentence fragment for another.

A phrase-structure grammar is a special case of the more general concept of a string-rewriting system, due to Post.

Productions

• A A production pproduction pPP is a pair is a pair pp=(=(bb,,aa) of sentence fragments ) of sentence fragments ll, , rr (not necessarily in (not necessarily in LL), which may generally contain a mix ), which may generally contain a mix of both terminals and nonterminals.of both terminals and nonterminals.– We often denote the production as We often denote the production as bb → → aa..

• Read “Read “bb goes to goes to aa” (like a directed graph edge)” (like a directed graph edge)

– Call Call bb the “before” string, the “before” string, a a the “after” string.the “after” string.– It is a kind of recursive definition meaning that It is a kind of recursive definition meaning that

If If lbr lbr LLTT, then , then larlar LLTT.. ((LLTT = sentence = sentence “templates”)“templates”)

• That is, if That is, if lbrlbr is a legal sentence template, then so is is a legal sentence template, then so is larlar..• That is, we can substitute That is, we can substitute aa in place of in place of bb in any sentence template. in any sentence template.

• A phrase-structure grammar imposes the constraint that each A phrase-structure grammar imposes the constraint that each ll must contain a nonterminal symbol. must contain a nonterminal symbol.

Languages from PSGs

• The recursive definition of the language The recursive definition of the language LL defined defined by the PSG: by the PSG: GG = ( = (VV, , TT, , SS, , PP))::– Rule 1: Rule 1: SS LLTT ( (LLTT is is LL’s ’s template languagetemplate language))

• The start symbol is a sentence template (member of The start symbol is a sentence template (member of LLTT).).

– Rule 2: Rule 2: ((bb→→aa))PP: : ll,,rrVV*: *: lbrlbr LLTT → → larlar LLTT

• Any production, after substituting in any fragment of any Any production, after substituting in any fragment of any sentence template, yields another sentence template.sentence template, yields another sentence template.

– Rule 3: Rule 3: ((σσ LLTT: ¬: ¬nnNN: : nnσσ) → ) → σσLL• All sentence templates that contain no nonterminal symbols All sentence templates that contain no nonterminal symbols

are sentences in are sentences in LL..

Abbreviatethis usinglbr lar.(read, “lar is directly derivable from lbr”).

Example

• Let Let GG = ({ = ({aa, , bb, , AA, , BB, , SS}, }, {{aa, , bb}, }, SS,, { {SS → → ABaABa, , AA → → BBBB, , BB → → abab, , ABAB → →

bb})})..• One possible derivation in this grammar is:One possible derivation in this grammar is:

SS ABaABa AabaAaba BBabaBBaba BababaBababa abababa.abababa.

V T

P

Derivability

• Recall that the notation Recall that the notation ww00 ww11 means that means that ((bb→→aa))PP: : ll,,rrV*V*: : ww00 = = lbrlbr ww1 1 = = larlar..– The template The template ww11 is directly derivable from is directly derivable from ww00..

• If If ww22,…,…wwnn-1-1: : ww00 ww11 ww22 … … wwnn, then we , then we write write ww00 * * wwnn, and say that , and say that wwnn is derivable from is derivable from ww00. . – The sequence of steps The sequence of steps wwii wwii+1+1 is called a is called a derivation of derivation of

wwnn from from ww00..• Note that the relation Note that the relation * is just the transitive * is just the transitive

closure of the relation closure of the relation ..

A Simple Definition of L(G)

• The language The language LL((GG)) (or just (or just LL) that is ) that is generated by a given phrase-structure generated by a given phrase-structure grammar grammar GG=(=(VV,,TT,,SS,,PP)) can be defined by: can be defined by:

LL((GG) = {) = {ww TT* | * | SS * * ww}}

• That is, That is, LL is simply the set of strings of is simply the set of strings of terminals that are derivable from the start terminals that are derivable from the start symbol.symbol.

Language Generated by a Grammar

• Example: Let Example: Let GG = ({ = ({SS,,AA,,aa,,bb},{},{aa,,bb}, }, SS,,{{SS → → aAaA, , SS → → bb, , AA → → aaaa})}). What is . What is LL((GG))??

• Easy: We can just draw a treeEasy: We can just draw a treeof all possible derivations.of all possible derivations.– We have: We have: SS aAaA aaaaaa..– and and SS bb..

• Answer: Answer: LL = { = {aaaaaa, , bb}}..

S

aA b

aaaExample of asentence diagram.

Generating Infinite Languages

• A simple PSG can easily generate an A simple PSG can easily generate an infinite language.infinite language.

• Example: Example: SS → 11→ 11SS, , SS → 0 ( → 0 (TT = {0,1}). = {0,1}).• The derivations are:The derivations are:

– SS 0 0– SS 11 11SS 110 110– SS 11 11SS 1111S 1111S 11110 11110– and so on…and so on…

L = {(11)*0} – theset of all strings consisting of somenumber of concaten-ations of 11 with itself,followed by 0.

Another example

• Construct a PSG that generates the language Construct a PSG that generates the language LL = = {{00nn11nn | | nnNN}}..– 00 and and 11 here represent symbols being concatenated here represent symbols being concatenated nn

times, not integers being raised to the times, not integers being raised to the nnth power.th power.

• Solution strategy:Solution strategy: Each step of the derivation Each step of the derivation should preserve the invariant that the number of should preserve the invariant that the number of 00’s = the number of ’s = the number of 11’s in the template so far, and ’s in the template so far, and all all 00’s come before all ’s come before all 11’s.’s.

• Solution:Solution: SS → → 00SS11, , SS → → λλ..

Types of Grammars - Chomsky hierarchy of languages

• Venn Diagram of Grammar Types:Venn Diagram of Grammar Types:

Type 0 – Phrase-structure GrammarsType 1 –

Context-Sensitive

Type 2 –Context-Free

Type 3 –Regular

Defining the PSG Types

• Type 1: Context-Sensitive PSG:Type 1: Context-Sensitive PSG:– All after fragments are either longer or empty:All after fragments are either longer or empty:

if b if b →→ a, then a, then ||bb| | ≤≤ | |aa| or a=| or a=λλ

• Type 2: Context-Free PSG:Type 2: Context-Free PSG:– All before fragments have length 1: All before fragments have length 1:

if b if b →→ a, then a, then ||bb| = 1 and| = 1 and bb NN

• Type 3: Regular PSGs:Type 3: Regular PSGs:– All after fragments are either single terminals, or a All after fragments are either single terminals, or a

pair of a terminal followed by a nonterminal.pair of a terminal followed by a nonterminal. if b if b →→ a, then a, then aa TT or or aa TNTN..

Classifying grammars

Given a grammar, we need to be able to find the Given a grammar, we need to be able to find the smallest class in which it belongs. This can be smallest class in which it belongs. This can be determined by answering three questions: determined by answering three questions:

Are the left hand sides of all of the productions single Are the left hand sides of all of the productions single non-terminals? non-terminals?

• If yes, does each of the productions create at most If yes, does each of the productions create at most one non-terminal and is it on the right?one non-terminal and is it on the right? Yes – regularYes – regular No – context-free No – context-free

• If not, can any of the rules reduce the length of a If not, can any of the rules reduce the length of a string of terminals and non-terminals?string of terminals and non-terminals?

Yes – unrestricted Yes – unrestricted No – context-sensitive No – context-sensitive

A A regular grammarregular grammar is one where each production is one where each production takes one of the following forms:takes one of the following forms:

A A →→ aa,,

A A →→ aBaB..The grammar G: The grammar G:

SS → 1→ 1<First1>, <First1><First1>, <First1> → 1→ 1<Second1>, <Second1>,

<Second1><Second1> → 1→ 1<First1>, <Second1><First1>, <Second1> → 0→ 0

is regular. What is L(G)?is regular. What is L(G)?

Regular grammars (Type 3)

04/19/23 27

The grammar:The grammar: SS → → 00SS11, , SS → → λλ

is not regular, it is context-freeis not regular, it is context-free

Only Only one nonterminalone nonterminal can appear on the right side can appear on the right side and it must be at the and it must be at the right endright end of the right side. of the right side.

Therefore, the productions Therefore, the productions

A A →→ aBc aBc and and S S →→ TU TU

are are notnot part of a regular grammar, part of a regular grammar,

but the production but the production A A →→ bA bA is.is.

Grammar

Productions of the form:

xA String of variables and terminals

),,,( PSTVG

Variables Terminalsymbols

Startvariable

Variable

Context-Free Grammars (Type 2)

ExampleExample• The grammar:The grammar: SS → → 00SS11, , SS → → λλ is context-free.is context-free.

• Another example of a context free language is Another example of a context free language is { { aannbbmmccn+mn+m | n,m | n,m 0}0} . . • This is not a regular language, but it is context free as it can be This is not a regular language, but it is context free as it can be

generated by the following CFG (Context Free Grammar):generated by the following CFG (Context Free Grammar):– S → aSc | B S → aSc | B – B → bBc | B → bBc | λλ

The language { The language { aannbbnnccnn | n | n 1} is context-sensitive 1} is context-sensitive but not context freebut not context free..A grammar for this language is given by:A grammar for this language is given by:

S S →→ aSBC | aBC aSBC | aBCCB CB →→ BC BC aB aB →→ ab abbB bB →→ bb bbbC bC →→ bc bccC cC →→ cc cc

A example derivation in this grammar is:A example derivation in this grammar is:

S S aSBC aSBC

aaBCBCaaBCBC (using (using S S →→ aBC) aBC)

aabCBCaabCBC (using (using aB aB →→ ab ab))

aabBCCaabBCC (using (using CB CB →→ BC BC))

aabbCCaabbCC (using (using bB bB →→ bb) bb)

aabbcCaabbcC (using (using bC bC →→ bc bc))

aabbccaabbcc (using (using cC cC →→ cc cc))

  which derives which derives aa22bb22cc22..

04/19/23 31

Context-Sensitive Grammar (Type 1)

A A context-sensitive grammarcontext-sensitive grammar is a is a formal grammar GG = ( = (VV, T, , T, SS, , PP) such that all rules in ) such that all rules in PP are of the form are of the form

ααAAβ → αγβ β → αγβ

with with AA in in N=V-TN=V-T (i.e., (i.e., AA is single is single nonterminal) and α and β in ) and α and β in V (i.e., α and β strings of nonterminals and V (i.e., α and β strings of nonterminals and terminals) and γ is ) and γ is in V+ (i.e., γ a nonempty string of nonterminals and in V+ (i.e., γ a nonempty string of nonterminals and terminals), plus a rule of the formterminals), plus a rule of the form

S → S → λλ

with with λλ the empty string, is allowed if S does not appear on the the empty string, is allowed if S does not appear on the right side of any rule.right side of any rule.

04/19/23 32

Every context-sensitive grammar which does not generate the empty string can be transformed into an equivalent one in Kuroda normal form:AB → CD or

A → BC or

A → B or

A → α

where A, B, C and D are nonterminal symbols

and α is a terminal symbol.