Programming Language Implementation Lexical and Syntax Analysis Part II
description
Transcript of Programming Language Implementation Lexical and Syntax Analysis Part II
![Page 1: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/1.jpg)
Dr. M Shafique Introduction 1
Programming Language Implementation
Lexical and Syntax Analysis
Part II
![Page 2: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/2.jpg)
Dr. M Shafique Introduction 2
Outline
• Overview of parsing• Introduction
• Parsing
• Some more details • Lexical analysis
• Parsing
![Page 3: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/3.jpg)
Dr. M Shafique Introduction 3
Reference
• Compilers: Principles, Techniques, and ToolsA. V. Aho, R. Sethi, and J. D. UllmanAddison-Wesley Publishing Company 1988
Chapters 1, 2, 3, 4, and 5
![Page 4: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/4.jpg)
Dr. M Shafique Introduction 4
Introduction
• A programming language can be defined by describing its• Syntax and
• Semantics
• Grammar-oriented compilation technique • Syntax-directed translation
• Example• Infix expressions translated to post-fix expressions
• Input to output mapping
• 9 -5 +2 to 95-2+
![Page 5: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/5.jpg)
Dr. M Shafique Introduction 5
Example
• Syntax1. e e + d
2. e e – d
3. e d
4. d 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
• Tokens• + - 0 1 2 3 4 5 6 7 8 9
![Page 6: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/6.jpg)
Dr. M Shafique Introduction 6
Parsing
• Parsing is a process to determine if a string of tokens can be generated by a grammar
• Parsing methods• Top-down parsing
• Bottom-up parsing
![Page 7: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/7.jpg)
Dr. M Shafique Introduction 7
Parsing• Top-down parsing
1. At node n (labeled with nonterminal A), select one of the productions for A and construct children at n for the symbols on the RHS of the production
2. Find the next node at which a subtree is to be constructed
• Recursive-descent parsing is a top-down syntax analysis method in which a set of recursive procedures are executed to process the input• A procedure is associated with each non terminal of a
grammar• Left recursive rules can loop forever
![Page 8: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/8.jpg)
Dr. M Shafique Introduction 8
Parsing
• Bottom-up parsing
Bottom-up parsing constructs a aprse tree for an input string of tokens beginning at the leaves and working up towards the root.• Shift-reduce parsing
• Operator precedence parsing
• LR parsing
![Page 9: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/9.jpg)
Dr. M Shafique Introduction 9
Parsing
• Removing left recursion• Example
• Left-recursive grammar
A A α | β
• Equivalent grammar without left recursion
A β R
R αR | ε
![Page 10: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/10.jpg)
Dr. M Shafique Introduction 10
Some Important Basic Definitions
lexical: Of or relating to the morphemes of a language.
morpheme: A meaningful linguistic unit that cannotbe divided into smaller meaningful parts.
lexical analysis: The task concerned with breaking aninput into its smallest meaningful units, called tokens.
![Page 11: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/11.jpg)
Dr. M Shafique Introduction 11
Some Important Basic Definitions
syntax: The way in which words are put togetherto form phrases, clauses, or sentences. The rulesgoverning the formation of statements in a programminglanguage.
syntax analysis: The task concerned with fitting asequence of tokens into a specified syntax.
parsing: To break a sentence down into its componentparts of speech with an explanation of the form, function,and syntactical relationship of each part.
![Page 12: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/12.jpg)
Dr. M Shafique Introduction 12
Some Important Basic Definitions
parsing = lexical analysis + syntax analysis
semantic analysis: The task concerned with calculating the program’s meaning.
![Page 13: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/13.jpg)
Dr. M Shafique Introduction 13
Regular Expressions
Symbol: a A regular expression formed by a.
Alternation:M | N A regular expression formed by M or N.
Concatenation:M • N A regular expression formed by M followed by N.
Epsilon: The empty string.
Repetition:M* A regular expression formed by zero or
more repetitions of M.
![Page 14: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/14.jpg)
Dr. M Shafique Introduction 14
Building a Recognizer for a Language
General approach:
1. Build a deterministic finite automaton (DFA) from regular expression E
2. Execute the DFA to determine whether an input string belongs to L(E)
Note: The DFA construction is done automatically by a tool such as lex.
![Page 15: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/15.jpg)
Dr. M Shafique Introduction 15
Finite Automata
A nondeterministic finite automaton A = {S, , s0, F, move }consists of:1. A set of states S2. A set of input symbols (the input symbol alphabet)3. A state s0 that is distinguished as the start state4. A state F distinguished as the accepting state5. A transition function move that maps state-symbol pairs into sets of state.
In a Deterministic Finite State Automata (DFA), the functionmove maps each state-symbol pair into a unique state.
![Page 16: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/16.jpg)
Dr. M Shafique Introduction 16
Finite Automata
A Deterministic Finite Automaton (DFA):
A Nondeterministic Finite Automaton (NFA):
0 1 2 3a
b
b bstart
0 1 2 3
a
a
b
b bstart
What languages areaccepted by theseautomata?
b*abb
(a|b)*abb
(Aho,Sethi,Ullman, pp. 114)
![Page 17: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/17.jpg)
Dr. M Shafique Introduction 17
Another NFA
start
a
b
a
b
An -transition is taken without consuming any character from the input.
What does the NFA above accepts?
aa*|bb*
(Aho,Sethi,Ullman, pp. 116)
![Page 18: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/18.jpg)
Dr. M Shafique Introduction 18
Constructing NFA
It is very simple. Remember that a regular expression is formed by the use of alternation, concatenation, and repetition.
How do we define an NFA that accepts a regular expression?
Thus all we need to do is to know how to build the NFAfor a single symbol, and how to compose NFAs.
![Page 19: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/19.jpg)
Dr. M Shafique Introduction 19
Composing NFAs with Alternation
The NFA for a symbol a is: ai fstart
Given two NFA N(s) and N(t)
N(s)
N(t)
(Aho,Sethi,Ullman, pp. 122)
starti
f
, the NFA N(s|t) is:
![Page 20: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/20.jpg)
Dr. M Shafique Introduction 20
Composing NFAs with Concatenation
start
Given two NFA N(s) and N(t), the NFA N(st) is:
N(s) N(t)i f
(Aho,Sethi,Ullman, pp. 123)
![Page 21: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/21.jpg)
Dr. M Shafique Introduction 21
Composing NFAs with Repetition
The NFA for N(s*) is
N(s)
fi
(Aho,Sethi,Ullman, pp. 123)
![Page 22: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/22.jpg)
Dr. M Shafique Introduction 22
Properties of the NFA
• Following this construction rules, we obtain an NFA N(r) with these properties:• N(r) has at most twice as many states as the number of
symbols and operators in r;• N(r) has exactly one starting and one accepting state;• Each state of N(r) has at most one outgoing transition on a
symbol of the alphabet or at most two outgoing -transitions.
(Aho,Sethi,Ullman, pp. 124)
![Page 23: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/23.jpg)
Dr. M Shafique Introduction 23
How to Parse a Regular Expression?
Given a DFA, we can generate an automaton that recognizes the longest substring of an inputthat is a valid token.
Using the three simple rules presented, it is easyto generate an NFA to recognize a regular expression.
Given a regular expression, how do we generatean automaton to recognize tokens?
Create an NFA and convert it to a DFA.
![Page 24: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/24.jpg)
Dr. M Shafique Introduction 24
a An ordinary character stands for itself.
The empty string.
Another way to write the empty string.
M | N Alternation, Choosing from M or N.
M N Concatenation, an M followed by an N.
M* Repetition (zero or more times).
M+ Repetition (one or more times).
M? Optional, zero or one occurrence of M.
[a -zA -Z] Character set alternation.
. Stands for any single character except newline.
“a.+*” Quotation, a string in quotes stands for itself
literally.
Regular expression notation: An Example
(Appel, pp. 20)
![Page 25: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/25.jpg)
Dr. M Shafique Introduction 25
if {return IF;}
[a - z] [a - z0 - 9 ] * {return ID;}
[0 - 9] + {return NUM;}
([0 - 9] + “.” [0 - 9] *) | ([0 - 9] * “.” [0 - 9] +) {return REAL;}
(“--” [a - z]* “\n”) | (“ ” | “ \n ” | “ \t ”) + {/* do nothing*/}
. {error ();}
(Appel, pp. 20)
Regular expressions for some tokens
![Page 26: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/26.jpg)
Dr. M Shafique Introduction 26
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
The NFA for a symbol i is: i1 2start
The NFA for the regular expression if is:
f 31start 2i
The NFA for a symbol f is: f 2start 1
IF
if {return IF;}
![Page 27: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/27.jpg)
Dr. M Shafique Introduction 27
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
a-z 21start
ID
[a-z] [a-z0-9 ] * {return ID;}
0-9
a-z
![Page 28: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/28.jpg)
Dr. M Shafique Introduction 28
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
0-9 21start
NUM
[0 - 9] + {return NUM;}
0-9
![Page 29: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/29.jpg)
Dr. M Shafique Introduction 29
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
1start
REAL
([0 - 9] + “.” [0 - 9] *) | ([0 - 9] * “.” [0 - 9] +) {return REAL;}
0-9
0-9
2 3.
0-9
0-950-94
.
![Page 30: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/30.jpg)
Dr. M Shafique Introduction 30
Building Finite Automatas for Lexical Tokens
(Appel, pp. 21)
1start
/* do nothing */
(“--” [a - z]* “\n”) | (“ ” | “ \n ” | “ \t ”) + {/* do nothing*/}
- 2
a-z
- 3 4\n
\n\t
5blank \n
\tblank
![Page 31: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/31.jpg)
Dr. M Shafique Introduction 31
ID
1 20 - 9 0 - 9
NUM
0 - 9
1 2 3
4 5
0 - 9
0 - 9 0 - 9
0 - 9
REAL
1 2 43
5
a-z\n- -
blank, etc.blank, etc.
White space
21any but \n
error
IF
1 2a-z a-z
0-9
Building Finite Automatas for Lexical Tokens
1 2i f
3
.
.
(Appel, pp. 21)
![Page 32: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/32.jpg)
Dr. M Shafique Introduction 32
Conversion of NFA into DFA
(Appel, pp. 27)
What states can be reached from state 1 without consuming a character?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 33: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/33.jpg)
Dr. M Shafique Introduction 33
Conversion of NFA into DFA
What states can be reached from state 1 without consuming a character?
{1,4,9,14} form the -closure of state 1
(Appel, pp. 27)
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 34: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/34.jpg)
Dr. M Shafique Introduction 34
Conversion of NFA into DFA
What are all the state closures in this NFA?
closure(1) = {1,4,9,14}closure(5) = {5,6,8}closure(8) = {6,8}closure(7) = {7,8}
(Appel, pp. 27)
closure(10) = {10,11,13}closure(13) = {11,13}closure(12) = {12,13}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 35: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/35.jpg)
Dr. M Shafique Introduction 35
Conversion of NFA into DFA
Given a set of NFA states T, the -closure(T) is theset of states that are reachable through -transiton from
any state s T.
Given a set of NFA states T, move(T, a) is theset of states that are reachable on input a
from any state sT.
(Aho,Sethi,Ullman, pp. 118)
![Page 36: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/36.jpg)
Dr. M Shafique Introduction 36
Problem Statement for Conversion of NFA into DFA
Given an NFA find the DFA with the minimum number of states that has the same behavior as the NFA for all inputs.
If the initial state in the NFA is s0, then theset of states in the DFA, Dstates, is initialized with a
state representing -closure(s0).
(Aho,Sethi,Ullman, pp. 118)
![Page 37: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/37.jpg)
Dr. M Shafique Introduction 37
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 38: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/38.jpg)
Dr. M Shafique Introduction 38
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = {5,15}
-closure({5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 39: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/39.jpg)
Dr. M Shafique Introduction 39
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
1-4-9-14
Now we need to compute:
move(1-4-9-14,a-h) = {5,15}
-closure({5,15}) = {5,6,8,15}
a-h 5-6-8-15
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
![Page 40: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/40.jpg)
Dr. M Shafique Introduction 40
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
![Page 41: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/41.jpg)
Dr. M Shafique Introduction 41
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = {2,5,15}
-closure({2,5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
![Page 42: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/42.jpg)
Dr. M Shafique Introduction 42
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, i) = {2,5,15}
-closure({2,5,15}) = {2,5,6,8,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
![Page 43: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/43.jpg)
Dr. M Shafique Introduction 43
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
![Page 44: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/44.jpg)
Dr. M Shafique Introduction 44
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = {5,15}
-closure({5,15}) = ?
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
![Page 45: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/45.jpg)
Dr. M Shafique Introduction 45
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, j-z) = {5,15}
-closure({5,15}) = {5,6,8,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z
![Page 46: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/46.jpg)
Dr. M Shafique Introduction 46
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, 0-9) = {10,15}
-closure({10,15}) = {10,11,13,15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
![Page 47: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/47.jpg)
Dr. M Shafique Introduction 47
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}move(1-4-9-14, other) = {15}
-closure({15}) = {15}
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
15other
![Page 48: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/48.jpg)
Dr. M Shafique Introduction 48
Conversion of NFA into DFA
(Appel, pp. 27)
Dstates = {1-4-9-14}
The analysis for 1-4-9-14is complete. We mark it andpick another state in the DFAto analyse.
2 3 84 5 6 7
139 10 11 1214 15
1
a-z
0-90-9
a-z
0-9i
f
IF
error
NUM
ID
anycharacter
1-4-9-14
a-h 5-6-8-15
2-5-6-8-15i
j-z10-11-13-15
0-9
15other
![Page 49: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/49.jpg)
Dr. M Shafique Introduction 49
The corresponding DFA
5-6-8-15
2-5-6-8-15
10-11-13-15
3-6-7-8
11-12-13
6-7-8
15
1-4-9-14
a-e, g-z, 0-9
a-z,0-9
a-z,0-9
0-9
0-9
f
i
a-h
j-z
0-9
other
ID
ID
NUM NUM
IF
error
ID
a-z,0-9
(Appel, pp. 29)
See pp. 118 of Aho-Sethi-Ullmanand pp. 29 of Appel.
![Page 50: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/50.jpg)
Dr. M Shafique Introduction 50
Lexical Analyzer and Parser
lexicalanalyzer
Syntaxanalyzer
symboltable
get nexttoken
(Aho,Sethi,Ullman, pp. 160)
token: smallest meaningful sequence of characters of interest in source program
SourceProgram
get nextchar
next char next token
(Contains a record for each identifier)
![Page 51: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/51.jpg)
Dr. M Shafique Introduction 51
Definition of Context-Free Grammars
A context-free grammar G = (T, N, S, P) consists of:1. T, a set of terminals (scanner tokens).2. N, a set of nonterminals (syntactic variables generated
by productions).3. S, a designated start nonterminal.4. P, a set of productions. Each production has the form,
A::= , where A is a nonterminal and is a sentential form , i.e., a string of zero or more grammar symbols (terminals/nonterminals).
![Page 52: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/52.jpg)
Dr. M Shafique Introduction 52
Syntax Analysis
Syntax Analysis Problem Statement: To find a derivation sequence in a grammar G for the input token stream (or say that none exists).
![Page 53: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/53.jpg)
Dr. M Shafique Introduction 53
Tree nodes represent symbols of the grammar (nonterminals or terminals) and tree edges represent derivation steps.
Parse trees
A parse tree is a graphical representation of a derivation sequence of a sentential form.
![Page 54: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/54.jpg)
Dr. M Shafique Introduction 54
Derivation
E E + E | E E | ( E ) | - E | id
Given the following grammar:
Is the string -(id + id) a sentence in this grammar?
Yes because there is the following derivation:
E -E -(E) -(E + E) -(id + id)
Where reads “derives in one step”.
(Aho,Sethi,Ullman, pp. 168)
![Page 55: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/55.jpg)
Dr. M Shafique Introduction 55
Derivation
E E + E | E E | ( E ) | - E | id
Lets examine this derivation:
E -E -(E) -(E + E) -(id + id)
E E
E-
E
E-
E( )
E
E-
E( )
+E E
E
E-
E( )
+E E
id idThis is a top-down derivationbecause we start building theparse tree at the top parse tree
(Aho,Sethi,Ullman, pp. 170)
![Page 56: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/56.jpg)
Dr. M Shafique Introduction 56Which derivation tree is correct?
Another Derivation Example
Find a derivation for the expression: id + id idE E
+E E
E
+E E
E E
E
+E E
E E
id id
id
E E
E E
E
E E
+E E
E
E E
+E E
id id
id
E E + E | E E | ( E ) | - E | id
(Aho,Sethi,Ullman, pp. 171)
![Page 57: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/57.jpg)
Dr. M Shafique Introduction 57
According to the grammar, both are correct.
Another Derivation Example
Find a derivation for the expression: id + id idE
+E E
E E
id id
id
E
+E E
E E
id id
id
A grammar that produces more than oneparse tree for any input sentence is saidto be an ambiguous grammar.
E E + E | E E | ( E ) | - E | id
(Aho,Sethi,Ullman, pp. 171)
![Page 58: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/58.jpg)
Dr. M Shafique Introduction 58
Left Recursion
Consider the grammar:E E + T | TT T F | FF ( E ) | id
A top-down parser might loop forever when parsingan expression using this grammar
E E
+E T
E
+E T
+E T
E
+E T
+E T
+E T
(Aho,Sethi,Ullman, pp. 176)
![Page 59: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/59.jpg)
Dr. M Shafique Introduction 59
Left Recursion
Consider the grammar:E E + T | TT T F | FF ( E ) | id
A grammar that has at least one production of the formA A is a left recursive grammar.
Top-down parsers do not work with left-recursivegrammars.
Left-recursion can often be eliminated by rewriting thegrammar.
(Aho,Sethi,Ullman, pp. 176)
![Page 60: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/60.jpg)
Dr. M Shafique Introduction 60
Left Recursion
This left-recursivegrammar:
E E + T | TT T F | FF ( E ) | id
Can be re-written to eliminate the immediate left recursion:
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
(Aho,Sethi,Ullman, pp. 176)
![Page 61: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/61.jpg)
Dr. M Shafique Introduction 61
Predictive Parsing
Consider the grammar:
stm if expr then stmt else stmt | while expr do stmt | begin stmt_list end
A parser for this grammar can be written with the following simple structure: switch(gettoken())
{ case if: …. break;
case while: …. break;
case begin: …. break;
default: reject input;}
Based only on the first token,the parser knows which rule to use to derive a statement.
Therefore this is called apredictive parser.
(Aho,Sethi,Ullman, pp. 183)
![Page 62: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/62.jpg)
Dr. M Shafique Introduction 62
Left Factoring
The following grammar:
stmt if expr then stmt else stmt | if expr then stmt
Cannot be parsed by a predictive parser that looksone element ahead.
But the grammar can be re-written:
stmt if expr then stmt stmt’stmt‘ else stmt |
Where is the empty string.
(Aho,Sethi,Ullman, pp. 178)
Rewriting a grammar to eliminate multiple productionsstarting with the same token is called left factoring.
![Page 63: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/63.jpg)
Dr. M Shafique Introduction 63
A Predictive Parser
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
Grammar:
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
ParsingTable:
(Aho,Sethi,Ullman, pp. 188)
![Page 64: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/64.jpg)
Dr. M Shafique Introduction 64
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
E
$
$ OUTPUT:
E
T
E’
$
T E’
PARSINGTABLE:
![Page 65: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/65.jpg)
Dr. M Shafique Introduction 65
T
E’
$
T
E’
$
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 186)
![Page 66: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/66.jpg)
Dr. M Shafique Introduction 66
(Aho,Sethi,Ullman, pp. 188)
T
E’
$
T
E’
$
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
id
T’
E’
$id
PARSINGTABLE:
![Page 67: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/67.jpg)
Dr. M Shafique Introduction 67
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F
T’
E’
$
F T’
T E’
id
T’
E’
$id
Action when Top(Stack) = input $ : Pop stack, advance input.
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 188)
![Page 68: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/68.jpg)
Dr. M Shafique Introduction 68
A Predictive Parser
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
STACK:
id idid+ INPUT:
Predictive ParsingProgram
$ OUTPUT:
E
F T’
T E’
id
T’
E’
$
E’
$
PARSINGTABLE: (Aho,Sethi,
Ullman, pp. 188)
![Page 69: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/69.jpg)
Dr. M Shafique Introduction 69
A Predictive Parser
E
F T’
T E’
id
T+ E’
F T’
id F T’
id
The predictive parser proceedsin this fashion emiting thefollowing productions:
E’ +TE’T FT’F idT’ FT’F idT’ E’
When Top(Stack) = input = $the parser halts and accepts the
input string. (Aho,Sethi,Ullman, pp. 188)
![Page 70: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/70.jpg)
Dr. M Shafique Introduction 70
LL(k) Parser
This parser parses from left to right, and does aleftmost-derivation. It looks up 1 symbol ahead to choose its next action. Therefore, it is known asa LL(1) parser.
An LL(k) parser looks k symbols ahead to decideits action.
![Page 71: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/71.jpg)
Dr. M Shafique Introduction 71
The Parsing Table
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
Given this grammar:
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
PARSINGTABLE:
How is this parsing table built?
![Page 72: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/72.jpg)
Dr. M Shafique Introduction 72
FIRST and FOLLOW
We need to build a FIRST set and a FOLLOW setfor each symbol in the grammar.
FIRST() is the set of terminal symbols that can begin any string derived from .
The elements of FIRST and FOLLOW areterminal symbols.
FOLLOW() is the set of terminal symbols that can follow :
t FOLLOW() derivation containing t
(Aho,Sethi,Ullman, pp. 189)
![Page 73: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/73.jpg)
Dr. M Shafique Introduction 73
Rules to Create FIRST
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If X is a terminal, FIRST(X) = {X}
FIRST(id) = {id}FIRST() = {}FIRST(+) = {+}
SETS:
2. If X , then FIRST(X)3. If X Y1Y2 ••• Yk
FIRST(() = {(}FIRST()) = {)}
FIRST rules:
*and Y1 ••• Yi-1 and a FIRST(Yi)
then a FIRST(X)
FIRST(F) = {(, id}FIRST(T) = FIRST(F) = {(, id}FIRST(E) = FIRST(T) = {(, id}
FIRST(E’) = {} {+, }FIRST(T’) = {} {, }
(Aho,Sethi,Ullman, pp. 189)
![Page 74: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/74.jpg)
Dr. M Shafique Introduction 74
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {$}
FOLLOW(E’) = { ), $}
SETS:
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW rules:
{ ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(T) = { ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
A and B are non-terminals, and are strings of grammar symbols
(Aho,Sethi,Ullman, pp. 189)
![Page 75: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/75.jpg)
Dr. M Shafique Introduction 75
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS: 3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW rules:
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(T) = { ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)
{+, ), $}
(Aho,Sethi,Ullman, pp. 189)
![Page 76: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/76.jpg)
Dr. M Shafique Introduction 76
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
*
(Aho,Sethi,Ullman, pp. 189)
![Page 77: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/77.jpg)
Dr. M Shafique Introduction 77
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
2. If A B, and a FIRST() and a then a FOLLOW(B)3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(F) = {+, ), $}
(Aho,Sethi,Ullman, pp. 189)
![Page 78: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/78.jpg)
Dr. M Shafique Introduction 78
Rules to Create FOLLOW
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
1. If S is the start symbol, then $ FOLLOW(S)
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
SETS:
FOLLOW rules:
FOLLOW(T) = {+, ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
3. If A B and a FOLLOW(A) then a FOLLOW(B)
FOLLOW(T’) = {+, ), $}
3a. If A B and and a FOLLOW(A) then a FOLLOW(B)
* FOLLOW(F) = {+, ), $}
2. If A B, and a FIRST() and a then a FOLLOW(B)
{+, , ), $}
(Aho,Sethi,Ullman, pp. 189)
![Page 79: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/79.jpg)
Dr. M Shafique Introduction 79
Rules to Build Parsing Table
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
![Page 80: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/80.jpg)
Dr. M Shafique Introduction 80
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 81: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/81.jpg)
Dr. M Shafique Introduction 81
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 82: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/82.jpg)
Dr. M Shafique Introduction 82
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 83: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/83.jpg)
Dr. M Shafique Introduction 83
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 84: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/84.jpg)
Dr. M Shafique Introduction 84
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A),
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 85: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/85.jpg)
Dr. M Shafique Introduction 85
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A),
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 86: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/86.jpg)
Dr. M Shafique Introduction 86
E TE’E’ +TE’ | T FT’T’ FT’ | F ( E ) | id
GRAMMAR:
FOLLOW(E) = {), $}
FOLLOW(E’) = { ), $}
FOLLOW SETS:
FOLLOW(T) = {+, ), $}
FOLLOW(T’) = {+, ), $}FOLLOW(F) = {+, , ), $}
FIRST(F) = {(, id}FIRST(T) = {(, id}FIRST(E) = {(, id}
FIRST(E’) = {+, }FIRST(T’) = { , }
FIRST SETS:
PARSINGTABLE:
1. If A : if a FIRST(), add A to M[A, a]2. If A : if FIRST(), add A to M[A, b] for each terminal b FOLLOW(A), 3. If A : if FIRST(), and $ FOLLOW(A), add A to M[A, $]
INPUT SYMBOL NON- TERMINAL id + * ( ) $
E E TE’ E TE’ E’ E’ +TE’ E’ E’ T T FT’ T FT’ T’ T’ T’ *FT’ T’ T’ F F id F (E)
(Aho,Sethi,Ullman, pp. 190)
Rules to Build Parsing Table
![Page 87: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/87.jpg)
Dr. M Shafique Introduction 87
Bottom-Up and Top-Down Parsers
Top-down parsers: starts constructing the parse tree at thetop (root) of the tree and move down towards the leaves.• Easy to implement by hand, but • Works with restricted grammars.
example: predictive parsers
Bottom-up parsers: builds the nodes on the bottom of theparse tree first and moves up towards the root.• Suitable for automatic parser generation• Handle a larger class of grammars.
examples: shift-reduce parser (or LR(k) parsers)
(Aho,Sethi,Ullman, pp. 195)
![Page 88: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/88.jpg)
Dr. M Shafique Introduction 88
Bottom-Up Parser
A bottom-up parser, or a shift-reduce parser, beginsat the leaves and works up to the top of the tree.
The reduction steps trace a rightmost derivationin reverse.
S aABeA Abc | bB d
Consider the Grammar:
We want to parse the input string abbcde.
(Aho,Sethi,Ullman, pp. 195)
![Page 89: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/89.jpg)
Dr. M Shafique Introduction 89
Bottom-Up Parser Example
a dbb cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
![Page 90: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/90.jpg)
Dr. M Shafique Introduction 90
Bottom-Up Parser Example
a dbb cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
![Page 91: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/91.jpg)
Dr. M Shafique Introduction 91
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
(Aho,Sethi,Ullman, pp. 195)
![Page 92: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/92.jpg)
Dr. M Shafique Introduction 92
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
We are not reducing here in this example.
A parser would reduce, get stuck and then backtrack!
(Aho,Sethi,Ullman, pp. 195)
![Page 93: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/93.jpg)
Dr. M Shafique Introduction 93
Bottom-Up Parser Example
a dbA cINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A
b
$
ProductionS aABeA Abc
A bB d
c
A
b
(Aho,Sethi,Ullman, pp. 195)
![Page 94: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/94.jpg)
Dr. M Shafique Introduction 94
Bottom-Up Parser Example
a dAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
(Aho,Sethi,Ullman, pp. 195)
![Page 95: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/95.jpg)
Dr. M Shafique Introduction 95
Bottom-Up Parser Example
a dAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
(Aho,Sethi,Ullman, pp. 195)
![Page 96: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/96.jpg)
Dr. M Shafique Introduction 96
Bottom-Up Parser Example
a BAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
(Aho,Sethi,Ullman, pp. 195)
![Page 97: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/97.jpg)
Dr. M Shafique Introduction 97
Bottom-Up Parser Example
a BAINPUT:
Bottom-Up ParsingProgram
e OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
a
S
e
(Aho,Sethi,Ullman, pp. 195)
![Page 98: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/98.jpg)
Dr. M Shafique Introduction 98
Bottom-Up Parser Example
SINPUT:
Bottom-Up ParsingProgram
OUTPUT:
A c
A
b
$
ProductionS aABeA Abc
A bB d
b
B
d
a
S
e
This parser is known as an LR Parser because it scans the input from Left to right, and it constructs
a Rightmost derivation in reverse order.
(Aho,Sethi,Ullman, pp. 195)
![Page 99: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/99.jpg)
Dr. M Shafique Introduction 99
Bottom-Up Parser Example
The scanning of productions for matching withhandles in the input string, and backtracking makesthe method used in the previous example veryinefficient.
Can we do better?
![Page 100: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/100.jpg)
Dr. M Shafique Introduction 100
LR Parser Example
Input
Stack
LR ParsingProgram
action goto
Output
(Aho,Sethi,Ullman, pp. 217)
![Page 101: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/101.jpg)
Dr. M Shafique Introduction 101
LR Parser Example
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
The following grammar:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
Can be parsed with this actionand goto table
(Aho,Sethi,Ullman, pp. 219)
![Page 102: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/102.jpg)
Dr. M Shafique Introduction 102
LR Parser Example
id idid +INPUT: $
STACK: E0
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
GRAMMAR:
OUTPUT:
(Aho,Sethi,Ullman, pp. 220)
![Page 103: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/103.jpg)
Dr. M Shafique Introduction 103
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E5
id
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 104: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/104.jpg)
Dr. M Shafique Introduction 104
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 105: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/105.jpg)
Dr. M Shafique Introduction 105
OUTPUT:
E3
F
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 106: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/106.jpg)
Dr. M Shafique Introduction 106
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 107: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/107.jpg)
Dr. M Shafique Introduction 107
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 108: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/108.jpg)
Dr. M Shafique Introduction 108
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
E7
2
T
0
T
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 109: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/109.jpg)
Dr. M Shafique Introduction 109
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E5
id
7
2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 110: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/110.jpg)
Dr. M Shafique Introduction 110
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E7
2
T
0action goto State
id + * ( ) $ E T F 0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
F
id
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 111: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/111.jpg)
Dr. M Shafique Introduction 111
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
E10
F
7
2
T
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 112: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/112.jpg)
Dr. M Shafique Introduction 112
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 113: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/113.jpg)
Dr. M Shafique Introduction 113
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
2
T
0
T
T F
F
id
idaction goto State
id + * ( ) $ E T F 0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 114: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/114.jpg)
Dr. M Shafique Introduction 114
OUTPUT:
0
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 115: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/115.jpg)
Dr. M Shafique Introduction 115
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
1
E
0
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 116: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/116.jpg)
Dr. M Shafique Introduction 116
OUTPUT:
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
T
T F
F
id
id
E
6
+
1
E
0
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 117: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/117.jpg)
Dr. M Shafique Introduction 117
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
5
id
6
+
1
E
0
F
id
GRAMMAR:
(Aho,Sethi,Ullman, pp. 220)
![Page 118: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/118.jpg)
Dr. M Shafique Introduction 118
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
OUTPUT:
T
T F
F
id
id
E
6
+
1
E
0
F
id
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
(Aho,Sethi,Ullman, pp. 220)
![Page 119: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/119.jpg)
Dr. M Shafique Introduction 119
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
3
F
6
+
1
E
0
F
id
GRAMMAR:
T
(Aho,Sethi,Ullman, pp. 220)
![Page 120: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/120.jpg)
Dr. M Shafique Introduction 120
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
OUTPUT:
T
T F
F
id
id
E
6
+
1
E
0
F
id
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
(Aho,Sethi,Ullman, pp. 220)
![Page 121: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/121.jpg)
Dr. M Shafique Introduction 121
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
9
T
6
+
1
E
0
F
id
GRAMMAR:
T
E
+
(Aho,Sethi,Ullman, pp. 220)
![Page 122: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/122.jpg)
Dr. M Shafique Introduction 122
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
0
GRAMMAR:
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
F
id
T
E
+
(Aho,Sethi,Ullman, pp. 220)
![Page 123: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/123.jpg)
Dr. M Shafique Introduction 123
LR Parser Example
id idid +INPUT: $
STACK:
(1) E E + T(2) E’ T(3) T T F(4) T F(5) F ( E ) (6) F id
LR ParsingProgram
action goto State id + * ( ) $ E T F
0 s5 s4 1 2 3 1 s6 acc 2 r2 s7 r2 r2 3 r4 r4 r4 r4 4 s5 s4 8 2 3 5 r6 r6 r6 r6 6 s5 s4 9 3 7 s5 s4 10 8 s6 s11 9 r1 s7 r1 r1 10 r3 r3 r3 r3 11 r5 r5 r5 r5
OUTPUT:
T
T F
F
id
id
E
1
E
0
F
id
GRAMMAR:
T
E
+
(Aho,Sethi,Ullman, pp. 220)
![Page 124: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/124.jpg)
Dr. M Shafique Introduction 124
Constructing Parsing Tables
All LR parsers use the same parsing program thatwe demonstrated in the previous slides.
What differentiates the LR parsers are the action and the goto tables:
Simple LR (SLR): succeeds for the fewest grammars, but is the easiest to implement.
Canonical LR: succeeds for the most grammars, but is the hardest to implement. It splits states when necessary to prevent reductions that would get the parser stuck.
Lookahead LR (LALR): succeeds for most common syntacticconstructions used in programming languages, but producesLR tables much smaller than canonical LR.
(See AhoSethiUllman pp. 221-230).
(See AhoSethiUllman pp. 236-247).
(See AhoSethiUllman pp. 230-236).
(Aho,Sethi,Ullman, pp. 221)
![Page 125: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/125.jpg)
Dr. M Shafique Introduction 125
Using Lex
Lexcompiler
Lexsource
programlex.l
lex.yy.c
Ccompiler
lex.yy.c a.out
a.outInput
stream
sequenceof
tokens
(Aho-Sethi-Ullman, pp. 258)
![Page 126: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/126.jpg)
Dr. M Shafique Introduction 126
Using Yacc
Yacccompiler
YaccSpecificationtranslate.y
y.tab.c
Ccompiler
y.tab.c a.out
a.outInput
output
(Aho-Sethi-Ullman, pp. 258)
yacc translate.y
cc y.tab.c -ly
![Page 127: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/127.jpg)
Dr. M Shafique Introduction 127
Parsing Action Conflicts
If the grammar specified is ambiguous, yacc willreport parsing action conflicts.
These conflicts can be reduce/reduce conflicts orshift/reduce conflicts.
Yacc has rules to resolve such conflicts automatically(see AhoSethiUllman, pp. 262-264), but the resultingparser might not have the behavior intended by thegrammar writer.
Whenever you see a conflict report, rerun yacc withthe -v flag, examine the y.output file, and re-writeyour grammar to eliminate the conflicts.
(Aho-Sethi-Ullman, pp. 262)
![Page 128: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/128.jpg)
Dr. M Shafique Introduction 128
Three-Address StatementsA popular form of intermediate code used in optimizing compilers is three-address statements (or variations, such as quadruples).
Source statement:x = a + b c + d
Three address statements with temporaries t1 and t2:
t1 = b ct2 = a + t1
x = t2 + d
• Intermediate Code GenerationRead Aho-Sethi-Ullman: Chapter 8.1 ~ 8.3, 8.7
(Aho-Sethi-Ullman, pp. 466)
![Page 129: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/129.jpg)
Dr. M Shafique Introduction 129
Lexical Analyzer (Scanner)+
Syntax Analyzer (Parser)+ Semantic Analyzer
Abstract Syntax Tree with attributes
Intermediate-code Generator
Non-optimized Intermediate Code
FrontEnd
ErrorMessage
Front End of a Compiler
![Page 130: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/130.jpg)
Dr. M Shafique Introduction 130
Component-Based Approach to Building Compilers
Target-1 Code Generator Target-2 Code Generator
Intermediate-code Optimizer
Language-1 Front End
Source programin Language-1
Language-2 Front End
Source programin Language-2
Non-optimized Intermediate Code
Optimized Intermediate Code
Target-1 machine code Target-2 machine code
![Page 131: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/131.jpg)
Dr. M Shafique Introduction 131
Advantages of Using an Intermediate Language
1. Retargeting - Build a compiler for a new machine by attaching a new code generator to an existing front-end.
2. Optimization - reuse intermediate code optimizers in compilers for different languages and different machines.
Note: the terms “intermediate code”, “intermediate language”, and “intermediate representation” are all used interchangeably.
![Page 132: Programming Language Implementation Lexical and Syntax Analysis Part II](https://reader035.fdocuments.in/reader035/viewer/2022062807/568150c0550346895dbee1ab/html5/thumbnails/132.jpg)
Dr. M Shafique Introduction 132
position := initial + rate * 60
Th
e P
has
es o
f a
Co
mp
iler
lexical analyzer
id1 := id2 + id3 * 60
syntax analyzer
:=
id1 +
id2 *
id3 60
semantic analyzer
:=
id1 +
id2 *
id3 inttoreal
60
intermediate code generator
temp1 := inttoreal (60)temp2 := id3 * temp1temp3 := id2 + temp2id1 := temp3
code optimizer
temp1 := id3 * 60.0id1 := id2 + temp1
code generator
MOVF id3, R2MULF #60.0, R2MOVF id2, R1ADDF R2, R1MOVF R1, id1