CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester...
Transcript of CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester...
![Page 1: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/1.jpg)
CS 335: Top-down ParsingSwarnendu Biswas
Semester 2019-2020-II
CSE, IIT Kanpur
Content influenced by many excellent references, see References slide for acknowledgements.
![Page 2: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/2.jpg)
Example Expression Grammar
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚 𝑇𝑒𝑟𝑚
𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝐹𝑎𝑐𝑡𝑜𝑟
𝐹𝑎𝑐𝑡𝑜𝑟 → 𝐸𝑥𝑝𝑟 | num | name
CS 335 Swarnendu Biswas
pri
ori
ty
![Page 3: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/3.jpg)
Derivation of name + name × nameSentential Form Input
𝐸𝑥𝑝𝑟 ↑ name + name × name
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
𝐹𝑎𝑐𝑡𝑜𝑟 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
name + 𝑇𝑒𝑟𝑚 ↑ name + name × name
name + 𝑇𝑒𝑟𝑚 name ↑ +name × name
name + 𝑇𝑒𝑟𝑚 name +↑ name × name
name + 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + 𝐹𝑎𝑐𝑡𝑜𝑟 × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name + name ↑× name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name + name ×↑ name
name + name × name name + name ×↑ name
name + name × name name + name × name ↑
CS 335 Swarnendu Biswas
![Page 4: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/4.jpg)
Derivation of name + name × nameSentential Form Input
𝐸𝑥𝑝𝑟 ↑ name + name × name
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
𝐹𝑎𝑐𝑡𝑜𝑟 + 𝑇𝑒𝑟𝑚 ↑ name + name × name
name + 𝑇𝑒𝑟𝑚 ↑ name + name × name
name + 𝑇𝑒𝑟𝑚 name ↑ +name × name
name + 𝑇𝑒𝑟𝑚 name +↑ name × name
name + 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + 𝐹𝑎𝑐𝑡𝑜𝑟 × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name +↑ name × name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name + name ↑× name
name + name × 𝐹𝑎𝑐𝑡𝑜𝑟 name + name ×↑ name
name + name × name name + name ×↑ name
name + name × name name + name × name ↑
CS 335 Swarnendu Biswas
The current input terminal being scanned is called the lookahead symbol
![Page 5: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/5.jpg)
Derivation of name + name × name
CS 335 Swarnendu Biswas
𝑙𝑚𝑆𝑡𝑎𝑟𝑡
𝑙𝑚𝐸𝑥𝑝𝑟 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
𝑇𝑒𝑟𝑚
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
![Page 6: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/6.jpg)
Derivation of name + name × name
CS 335 Swarnendu Biswas
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
𝐹𝑎𝑐𝑡𝑜𝑟
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
𝐹𝑎𝑐𝑡𝑜𝑟
name
![Page 7: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/7.jpg)
Derivation of name + name × name
CS 335 Swarnendu Biswas
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑙𝑚 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
Term
𝐹𝑎𝑐𝑡𝑜𝑟
name
𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
𝐹𝑎𝑐𝑡𝑜𝑟
name
name
![Page 8: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/8.jpg)
General Idea of Top-down Parsing
Start with the root (start symbol) of the parse tree
Grow the tree downwards by expanding productions at the lower levels of the tree
• Select a nonterminal and extend it by adding children corresponding to the right side of some production for the nonterminal
Repeat till
• Lower fringe consists only terminals and the input is consumed
Top-down parsing basically finds a leftmost derivation for an input string
CS 335 Swarnendu Biswas
![Page 9: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/9.jpg)
General Idea of Top-down Parsing
Start with the root of the parse tree
Grow the tree by expanding productions at the lower levels of the tree
• Extend a nonterminal by adding children corresponding to the right side of some production for the nonterminal
Repeat till
• Lower fringe consists only terminals and the input is consumed
• Mismatch in the lower fringe and the remaining input stream
• Selection of a production may involve trial-and-error
• Wrong choice of productions while expanding nonterminals
• Input character stream is not part of the language
CS 335 Swarnendu Biswas
![Page 10: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/10.jpg)
Leftmost Top-down Parsing Algorithmroot = node for Start symbol
curr = root
push(null) // Stack
word = nextWord()
while (true):
if curr ∈ Nonterminal:
pick next rule 𝐴 ⟶ 𝛽1𝛽2…𝛽𝑛 to expand curr
create nodes for 𝛽1, 𝛽2, …, 𝛽𝑛 as children of curr
push(𝛽𝑛, 𝛽𝑛−1, 𝛽1)
curr = 𝛽1
if curr == word:
word = nextWord()
curr = pop()
if word == eof and curr == null:
accept input
else
backtrack
CS 335 Swarnendu Biswas
![Page 11: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/11.jpg)
Implementing Backtracking
• Extend the previous algorithm to backtrack• Set curr to parent and delete the children
• Expand the node curr with untried rules if any• Create child nodes for each symbol in the right hand of the production
• Push those symbols onto the stack in reverse order
• Set curr to the first child node
• Move curr up the tree if there are no untried rules
• Report a syntax error when there are no more moves
CS 335 Swarnendu Biswas
![Page 12: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/12.jpg)
Example of Top-down ParsingRule # Sentential Form Input
𝐸𝑥𝑝𝑟 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
3 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
6 𝐹𝑎𝑐𝑡𝑜𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
9 name+ 𝑇𝑒𝑟𝑚 ↑ name+ name × name
name+ 𝑇𝑒𝑟𝑚 name ↑ +name × name
name+ 𝑇𝑒𝑟𝑚 name+↑ name × name
4 name+ 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
6 name+ 𝐹𝑎𝑐𝑡𝑜𝑟 × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
9 name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+ name ↑× name
name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+ name ×↑ name
9 name+ name × name name+ name ×↑ name
name+ name × name name+ name × name ↑
CS 335 Swarnendu Biswas
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
2 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚
3 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚
4 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
5 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟
6 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟
7 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
8 𝐹𝑎𝑐𝑡𝑜𝑟 → num
9 𝐹𝑎𝑐𝑡𝑜𝑟 → name
![Page 13: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/13.jpg)
Example of Top-down ParsingRule # Sentential Form Input
𝐸𝑥𝑝𝑟 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
3 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
6 𝐹𝑎𝑐𝑡𝑜𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
9 name+ 𝑇𝑒𝑟𝑚 ↑ name+ name × name
name+ 𝑇𝑒𝑟𝑚 name ↑ +name × name
name+ 𝑇𝑒𝑟𝑚 name+↑ name × name
4 name+ 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
6 name+ 𝐹𝑎𝑐𝑡𝑜𝑟 × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
9 name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+↑ name × name
name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+ name ↑× name
name+ name × 𝐹𝑎𝑐𝑡𝑜𝑟 name+ name ×↑ name
9 name+ name × name name+ name ×↑ name
name+ name × name name+ name × name ↑
CS 335 Swarnendu Biswas
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
2 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚
3 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚
4 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
5 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟
6 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟
7 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
8 𝐹𝑎𝑐𝑡𝑜𝑟 → num
9 𝐹𝑎𝑐𝑡𝑜𝑟 → name
How does a top-down parser choose which rule to apply?
![Page 14: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/14.jpg)
Example of Top-down Parsing
Rule # Sentential Form Input
𝐸𝑥𝑝𝑟 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 +⋯ ↑ name+ name × name
1 … ↑ name+ name × name
1 … ↑ name+ name × name
CS 335 Swarnendu Biswas
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
2 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚
3 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚
4 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
5 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟
6 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟
7 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
8 𝐹𝑎𝑐𝑡𝑜𝑟 → num
9 𝐹𝑎𝑐𝑡𝑜𝑟 → name
![Page 15: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/15.jpg)
Example of Top-Down Parsing
Rule # Sentential Form Input
𝐸𝑥𝑝𝑟 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 ↑ name+ name × name
1 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚 + 𝑇𝑒𝑟𝑚 +⋯ ↑ name+ name × name
1 … ↑ name+ name × name
1 … ↑ name+ name × name
CS 335 Swarnendu Biswas
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
2 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚
3 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚
4 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
5 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟
6 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟
7 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
8 𝐹𝑎𝑐𝑡𝑜𝑟 → num
9 𝐹𝑎𝑐𝑡𝑜𝑟 → name
A top-down parser can loop indefinitely with left-recursive grammar
![Page 16: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/16.jpg)
Left Recursion
• A grammar is left-recursive if it has a nonterminal 𝐴 such that there is
a derivation 𝐴 ֜+𝐴𝛼 for some string 𝛼
• Direct left recursion: There is a production of the form 𝐴 → 𝐴𝛼
• Indirect left recursion: First symbol on the right-hand side of a rule can derive the symbol on the left
CS 335 Swarnendu Biswas
We can often reformulate a grammar to avoid left recursion
![Page 17: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/17.jpg)
Remove Left Recursion
CS 335 Swarnendu Biswas
𝐴 → 𝐴𝛼1 𝐴𝛼2 … |𝐴𝛼𝑚 𝛽1 … |𝛽𝑛
𝐴 → 𝛽1𝐴′|𝛽2𝐴
′|…| 𝛽𝑛𝐴′
𝐴′ → 𝛼1𝐴′ 𝛼2𝐴
′ … |𝛼𝑚𝐴′|𝜖
![Page 18: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/18.jpg)
Remove Left Recursion
CS 335 Swarnendu Biswas
𝐸 → 𝐸 + 𝑇 | 𝑇𝑇 → 𝑇 ∗ 𝐹 | 𝐹𝐹 → 𝐸 | id
𝐸 → 𝑇𝐸′
𝐸′ → +𝑇𝐸′
𝑇 → 𝐹𝑇′
𝑇′ →∗ 𝐹𝑇′
𝐹 → 𝐸 |id
![Page 19: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/19.jpg)
Non-Left-Recursive Expression Grammar
CS 335 Swarnendu Biswas
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 + 𝑇𝑒𝑟𝑚
2 𝐸𝑥𝑝𝑟 → 𝐸𝑥𝑝𝑟 − 𝑇𝑒𝑟𝑚
3 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚
4 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 × 𝐹𝑎𝑐𝑡𝑜𝑟
5 𝑇𝑒𝑟𝑚 → 𝑇𝑒𝑟𝑚 ÷ 𝐹𝑎𝑐𝑡𝑜𝑟
6 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟
7 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
8 𝐹𝑎𝑐𝑡𝑜𝑟 → num
9 𝐹𝑎𝑐𝑡𝑜𝑟 → name
Rule # Production
0 𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
1 𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
2 𝐸𝑥𝑝𝑟′ → + 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
3 𝐸𝑥𝑝𝑟′ → − 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
4 𝐸𝑥𝑝𝑟′ → 𝜖
5 𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
6 𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
7 𝑇𝑒𝑟𝑚′ →÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
8 𝑇𝑒𝑟𝑚′ → 𝜖
9 𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟)
10 𝐹𝑎𝑐𝑡𝑜𝑟 → num
11 𝐹𝑎𝑐𝑡𝑜𝑟 → name
![Page 20: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/20.jpg)
Indirect Left Recursion
• There is a left recursion because 𝑆 → 𝐴𝑎 → 𝑆𝑑𝑎
CS 335 Swarnendu Biswas
𝑆 → 𝐴𝑎 | 𝑏𝐴 → 𝐴𝑐 𝑆𝑑 𝜖
![Page 21: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/21.jpg)
Eliminating Left Recursion
• Input: Grammar 𝐺 with no cycles or 𝜖−productions
• AlgorithmArrange nonterminals in some order 𝐴1, 𝐴2, … , 𝐴𝑛for 𝑖 ← 1…𝑛
for 𝑗 ← 1 to 𝑖 − 1
If ∃ a production 𝐴𝑖 → 𝐴𝑗𝛾
Replace 𝐴𝑖 → 𝐴𝑗𝛾 with one or more productions that expand 𝐴𝑗
Eliminate the immediate left recursion among the 𝐴𝑖 productions
CS 335 Swarnendu Biswas
![Page 22: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/22.jpg)
Eliminating Left Recursion
• Input: Grammar 𝐺 with no cycles or 𝜖−productions
• AlgorithmArrange nonterminals in some order 𝐴1, 𝐴2, … , 𝐴𝑛for 𝑖 ← 1…𝑛
for 𝑗 ← 1 to 𝑖 − 1
If ∃ a production 𝐴𝑖 → 𝐴𝑗𝛾
Replace 𝐴𝑖 → 𝐴𝑗𝛾 with one or more productions that expand 𝐴𝑗
Eliminate the immediate left recursion among the 𝐴𝑖 productions
CS 335 Swarnendu Biswas
Loop invariant at the start of outer iteration 𝑖
∀𝑘 < 𝑖, no production expanding 𝐴𝑘 has 𝐴𝑙 in its righthand side for all 𝑙 < 𝑘
![Page 23: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/23.jpg)
Eliminating Indirect Left Recursion
CS 335 Swarnendu Biswas
𝑆 → 𝐴𝑎 | 𝑏𝐴 → 𝐴𝑐 𝑆𝑑 𝜖
𝑆 → 𝐴𝑎 | 𝑏𝐴 → 𝑏𝑑𝐴′ | 𝐴′
𝐴′ → 𝑐𝐴′ 𝑎𝑑𝐴′ 𝜖
![Page 24: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/24.jpg)
Cost of Backtracking
Backtracking is expensive
• Parser expands a nonterminal with the wrong rule
• Mismatch between the lower fringe of the parse tree and the input is detected
• Parser undoes the last few actions
• Parser tries other productions if any
CS 335 Swarnendu Biswas
![Page 25: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/25.jpg)
Avoid Backtracking
• Parser is to select the next rule • Compare the curr symbol and the next input symbol called the lookahead
• Use the lookahead to disambiguate the possible production rules
• Backtrack-free grammar is a CFG for which the leftmost, top-down parser can always predict the correct rule with one word lookahead • Also called a predictive grammar
CS 335 Swarnendu Biswas
![Page 26: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/26.jpg)
FIRST Set
• Intuition• Each alternative for the leftmost nonterminal leads to a distinct terminal
symbol
• Which rule to choose becomes obvious by comparing the next word in the input stream
• Given a string 𝛾 of terminal and nonterminal symbols, FIRST(𝛾) is the set of all terminal symbols that can begin any string derived from 𝛾• We also need to keep track of which symbols can produce the empty string
• FIRST: (𝑁𝑇 ∪ 𝑇 ∪ 𝜖, EOF ) → (𝑇 ∪ 𝜖, EOF )
CS 335 Swarnendu Biswas
![Page 27: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/27.jpg)
Steps to Compute FIRST Set
1. If 𝑋 is a terminal, then FIRST 𝑋 = {𝑋}
2. If 𝑋 → 𝜖 is a production, then 𝜖 ∈ FIRST(𝑋)
3. If 𝑋 is a nonterminal and 𝑋 → 𝑌1𝑌2… 𝑌𝑘 is a productionI. Everything in FIRST(𝑌1) is in FIRST 𝑋
II. If for some 𝑖, 𝑎 ∈ FIRST(𝑌𝑖) and ∀1 ≤ 𝑗 < 𝑖, 𝜖 ∈ FIRST(𝑌𝑗), then 𝑎 ∈FIRST(𝑋)
III. If 𝜖 ∈ FIRST(𝑌1…𝑌𝑘), then 𝜖 ∈ FIRST(𝑋)
CS 335 Swarnendu Biswas
![Page 28: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/28.jpg)
FIRST Set
• Generalize FIRST relation to string of symbols
FIRST 𝑋𝛾 → FIRST 𝑋 if 𝑋 ↛ 𝜖
FIRST 𝑋𝛾 → FIRST 𝑋 ∪ FIRST 𝛾 if 𝑋 → 𝜖
CS 335 Swarnendu Biswas
![Page 29: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/29.jpg)
Compute FIRST Set
CS 335 Swarnendu Biswas
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟) | num | name
![Page 30: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/30.jpg)
Compute FIRST Set
FIRST 𝐸𝑥𝑝𝑟 = {(, name, num}
FIRST 𝐸𝑥𝑝𝑟′ = {+,−, 𝜖}
FIRST 𝑇𝑒𝑟𝑚 = {(, name, num}
FIRST 𝑇𝑒𝑟𝑚′ = {𝜖 ×,÷}
FIRST 𝐹𝑎𝑐𝑡𝑜𝑟 = {(, name,num}
CS 335 Swarnendu Biswas
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟) | num | name
![Page 31: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/31.jpg)
FOLLOW Set
• FOLLOW(𝑋) is the set of terminals that can immediately follow 𝑋• That is, 𝑡 ∈ FOLLOW(𝑋) if there is any derivation containing 𝑋𝑡
CS 335 Swarnendu Biswas
𝑆
𝐴 𝑎 𝛽𝛼
𝑐 𝛾…
Terminal 𝑐 is in FIRST(𝐴) and 𝑎is in FOLLOW(𝐴)
![Page 32: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/32.jpg)
Steps to Compute FOLLOW Set
1. Place $ in FOLLOW(𝑆) where 𝑆 is the start symbol and $ is the end marker
2. If there is a production 𝐴 → 𝛼𝐵𝛽, then everything in FIRST(𝛽)except 𝜖 is in FOLLOW(𝐵)
3. If there is a production 𝐴 → 𝛼𝐵, or a production 𝐴 → 𝛼𝐵𝛽 where FIRST(𝛽) contains 𝜖, then everything in FOLLOW(𝐴) is in FOLLOW(𝐵)
CS 335 Swarnendu Biswas
![Page 33: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/33.jpg)
Compute FOLLOW Set
CS 335 Swarnendu Biswas
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟) | num | name
![Page 34: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/34.jpg)
Compute FOLLOW Set
FOLLOW 𝐸𝑥𝑝𝑟 = {$, )}
FOLLOW 𝐸𝑥𝑝𝑟′ = {$,)}
FOLLOW 𝑇𝑒𝑟𝑚 = {$, +,−, )}
FOLLOW 𝑇𝑒𝑟𝑚′ = {$,+,−, )}
FOLLOW 𝐹𝑎𝑐𝑡𝑜𝑟 = {$, +,−,×,÷, )}
CS 335 Swarnendu Biswas
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚 𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟 𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → (𝐸𝑥𝑝𝑟) | num | name
![Page 35: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/35.jpg)
Conditions for Backtrack-Free Grammar
• Consider a production 𝐴 → 𝛽
FIRST+ = ቊFIRST 𝛽 if 𝜖 ∉ FIRST(𝛽)
FIRST 𝛽 ∪ FOLLOW 𝐴 otherwise
• For any nonterminal 𝐴 where 𝐴 → 𝛽1|𝛽2|…| 𝛽𝑛, a backtrack-free grammar has the property FIRST+ 𝐴 → 𝛽𝑖 ∩ FIRST+ 𝐴 → 𝛽𝑗 = 𝜙, ∀1 ≤ 𝑖, 𝑗 ≤ 𝑛, 𝑖 ≠ 𝑗
CS 335 Swarnendu Biswas
![Page 36: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/36.jpg)
Backtracking
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → name
| name [ 𝐴𝑟𝑔𝑙𝑖𝑠𝑡 ]
| name ( 𝐴𝑟𝑔𝑙𝑖𝑠𝑡 )
𝐴𝑟𝑔𝑙𝑖𝑠𝑡 → 𝐸𝑥𝑝𝑟 𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠
𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠 → , 𝐸𝑥𝑝𝑟 𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠
| 𝜖
CS 335 Swarnendu Biswas
![Page 37: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/37.jpg)
Backtracking
𝑆𝑡𝑎𝑟𝑡 → 𝐸𝑥𝑝𝑟
𝐸𝑥𝑝𝑟 → 𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′
𝐸𝑥𝑝𝑟′ → +𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′
−𝑇𝑒𝑟𝑚𝐸𝑥𝑝𝑟′ 𝜖
𝑇𝑒𝑟𝑚 → 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′
𝑇𝑒𝑟𝑚′ →× 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′
÷ 𝐹𝑎𝑐𝑡𝑜𝑟𝑇𝑒𝑟𝑚′ 𝜖
𝐹𝑎𝑐𝑡𝑜𝑟 → name
| name [ 𝐴𝑟𝑔𝑙𝑖𝑠𝑡 ]
| name ( 𝐴𝑟𝑔𝑙𝑖𝑠𝑡 )
𝐴𝑟𝑔𝑙𝑖𝑠𝑡 → 𝐸𝑥𝑝𝑟 𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠
𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠 → , 𝐸𝑥𝑝𝑟 𝑀𝑜𝑟𝑒𝐴𝑟𝑔𝑠
| 𝜖
CS 335 Swarnendu Biswas
Not all grammars are backtrack free
![Page 38: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/38.jpg)
Left Factoring
• Left factoring is the process of extracting and isolating common prefixes in a set of productions
• Algorithm
CS 335 Swarnendu Biswas
𝐹𝑎𝑐𝑡𝑜𝑟 → 𝑛𝑎𝑚𝑒 𝐴𝑟𝑔𝑢𝑚𝑒𝑛𝑡𝑠𝐴𝑟𝑔𝑢𝑚𝑒𝑛𝑡𝑠 → 𝐴𝑟𝑔𝐿𝑖𝑠𝑡 𝐴𝑟𝑔𝐿𝑖𝑠𝑡 𝜖
𝐴 → 𝛼𝛽1 𝛼𝛽2 … 𝛼𝛽𝑛 𝛾1 𝛾2 … |𝛾𝑗
𝐴 → 𝛼𝐵|𝛾1 𝛾2 … |𝛾𝑗𝐵 → 𝛽1 𝛽2 … |𝛽𝑛
![Page 39: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/39.jpg)
Key Insight in Using Top-Down Parsing
• Efficiency depends on the accuracy of selecting the correct production for expanding a nonterminal• Parser may not terminate in the worst case
• A large subset of the context-free grammars can be parsed without backtracking
CS 335 Swarnendu Biswas
![Page 40: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/40.jpg)
Recursive-Descent Parsing
CS 335 Swarnendu Biswas
![Page 41: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/41.jpg)
Recursive-Descent Parsing
• Recursive-descent parsing is a form of top-down parsing that mayrequire backtracking
• Consists of a set of procedures, one for each nonterminal
CS 335 Swarnendu Biswas
void A() {Choose an A-production 𝐴 → 𝑋1𝑋2…𝑋𝑘for 𝑖 ← 1…𝑘
if 𝑋𝑖 is a nonterminalcall procedure 𝑋𝑖()
else if 𝑋𝑖 equals the current input symbol 𝑎advance the input to the next symbol
else // error
}
![Page 42: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/42.jpg)
Limitations with Recursive-Descent Parsing
• Consider a grammar with two productions 𝑋 → 𝛾1 and 𝑋 → 𝛾2• Suppose FIRST(𝛾1) ∩ FIRST(𝛾2) ≠ 𝜙
• Say 𝑎 is the common terminal symbol
• Function corresponding to 𝑋 will not know which production to use on input token 𝑎
CS 335 Swarnendu Biswas
![Page 43: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/43.jpg)
Recursive-Descent Parsing with Backtracking
• To support backtracking • All productions should be tried in some order
• Failure for some production implies we need to try remaining productions
• Report an error only when there are no other rules
CS 335 Swarnendu Biswas
![Page 44: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/44.jpg)
Predictive Parsing
• Special case of recursive-descent parsing that does not require backtracking• Lookahead symbol unambiguously determines which production rule to use
• Advantage is that the algorithm is simple and the parser can be constructed by hand
CS 335 Swarnendu Biswas
𝑠𝑡𝑚𝑡 → expr ;| if 𝑒𝑥𝑝𝑟 𝑠𝑡𝑚𝑡| for 𝑜𝑝𝑡𝑒𝑥𝑝𝑟 ; 𝑜𝑝𝑡𝑒𝑥𝑝𝑟 ; 𝑜𝑝𝑡𝑒𝑥𝑝𝑟 𝑠𝑡𝑚𝑡| other
𝑜𝑝𝑡𝑒𝑥𝑝𝑟 → 𝜖 | expr
![Page 45: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/45.jpg)
Pseudocode for a Predictive Parser
void stmt() {switch(lookahead) {
case expr:match(expr); match(‘;’); break;
case if:match(if); match(‘(‘); match(expr); match(‘)’); stmt(); break;
case for:match(for); match(‘(‘); optexpr(); match(‘;’); optexpr(); match(‘;’); optexpr(); match(‘)’); stmt(); break;
case other:match(other); break;
default:report(“syntax error”);
}}
CS 335 Swarnendu Biswas
![Page 46: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/46.jpg)
LL(1) Grammars
• Class of grammars for which no backtracking is required• First L stands for left-to-right scan, second L stands for leftmost derivation
• There is one lookahead token
• No left-recursive or ambiguous grammar can be LL(1)
• In LL(k), k stands for k lookahead tokens• Predictive parsers accept LL(k) grammars
• Every LL(1) grammar is a LL(2) grammar
CS 335 Swarnendu Biswas
![Page 47: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/47.jpg)
Nonrecursive Table-Driven Predictive Parser
CS 335 Swarnendu Biswas
Parsing Table 𝑀
Predictive Parsing Program
a + b $Input
OutputStack X
Y
Z
$
![Page 48: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/48.jpg)
Predictive Parsing Algorithm• Input: String 𝑤 and parsing table 𝑀 for grammar 𝐺
• Algorithm:Let 𝑎 be the first symbol in 𝑤Let 𝑋 be the symbol at the top of the stack while 𝑋 ≠ $:
if 𝑋 == 𝑎:pop the stack and advance the input
else if 𝑋 is a terminal or 𝑀[𝑋, 𝑎] is an error entry:error
else if 𝑀 𝑋, 𝑎 == 𝑋 → 𝑌1𝑌2…𝑌𝑘:output the production pop the stackpush 𝑌𝑘𝑌𝑘−1…𝑌1 onto the stack
𝑋 ← top stack symbol
CS 335 Swarnendu Biswas
![Page 49: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/49.jpg)
Predictive Parsing Table
CS 335 Swarnendu Biswas
Nonterminal id + * ( ) $
𝐸 𝐸 → 𝑇𝐸′ 𝐸 → 𝑇𝐸′
𝐸′ 𝐸′ → +𝑇𝐸′ 𝐸′ → 𝜖 𝐸′ → 𝜖
𝑇 𝑇 → 𝐹𝑇′ 𝑇 → 𝐹𝑇′
𝑇′ 𝑇′ → 𝜖 𝑇′ →∗ 𝐹𝑇′ 𝑇′ → 𝜖 𝑇′ → 𝜖
𝐹 𝐹 → id 𝐹 → (𝐸)
𝐸 → 𝑇𝐸′
𝐸′ → +𝑇𝐸′ | 𝜖𝑇 → 𝐹𝑇′
𝑇′ →∗ 𝐹𝑇′ | 𝜖𝐹 → 𝐸 | id
![Page 50: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/50.jpg)
Construction of a Predictive Parsing Table
• Input: Grammar 𝐺
• Algorithm:• For each production 𝐴 → 𝛼 in 𝐺,
• For each terminal 𝑎 in FIRST 𝛼 , add 𝐴 → 𝛼 to 𝑀[𝐴, 𝑎]
• If 𝜖 is in FIRST 𝛼 , then for each terminal 𝑏 in FOLLOW(𝐴), add 𝐴 → 𝛼 to 𝑀 𝐴, 𝑏
• If 𝜖 is in FIRST 𝛼 and $ is in FOLLOW(𝐴), add 𝐴 → 𝛼 to 𝑀[𝐴, $]
• No production in 𝑀[𝐴, 𝑎] indicates error
CS 335 Swarnendu Biswas
![Page 51: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/51.jpg)
Working of Predictive ParserMatched Stack Input Action
𝐸$ id+ id ∗ id$
𝑇𝐸′$ id+ id ∗ id$ Output 𝐸 → 𝑇𝐸′
𝐹𝑇′𝐸′$ id+ id ∗ id$ Output 𝑇 → 𝐹𝑇′
id𝑇′𝐸′$ id+ id ∗ id$ Output 𝐹 → id
id 𝑇′𝐸′$ +id ∗ id$ Match id
id 𝐸′$ +id ∗ id$ Output 𝑇′ → 𝜖
id +𝑇𝐸′$ +id ∗ id$ Output 𝐸′ → +𝑇𝐸′
id+ 𝑇𝐸′$ id ∗ id$ Match +
id+ 𝐹𝑇′𝐸′$ id ∗ id$ Output 𝑇 → 𝐹𝑇′
id+ id𝐓′𝐸′$ id ∗ id$ Output 𝐹 → id
CS 335 Swarnendu Biswas
![Page 52: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/52.jpg)
Working of Predictive ParserMatched Stack Input Action
…
id+ id𝑇′𝐸′$ id ∗ id$ Output 𝐹 → id
id+ id 𝑇′𝐸′$ ∗ id$ Match id
id+ id ∗ 𝐹𝑇′𝐸′$ ∗ id$ Output 𝑇′ →∗ 𝐹𝑇′
id+ id∗ 𝐹𝑇′𝐸′$ id$ Match ∗
id+ id∗ id𝑇′𝐸′$ id$ Output 𝐹 → id
id+ id∗id 𝑇′𝐸′$ $ Match id
id+ id∗id 𝐸′$ $ Output 𝑇′ → 𝜖
id+ id∗id $ $ Output 𝐸′ → 𝜖
CS 335 Swarnendu Biswas
![Page 53: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/53.jpg)
Predictive Parsing
• Grammars whose predictive parsing tables contain no duplicate entries are called LL(1)
• If grammar 𝐺 is left-recursive or is ambiguous, then parsing table 𝑀will have at least one multiply-defined cell
• Some grammars cannot be transformed into LL(1)• The adjacent grammar is ambiguous
CS 335 Swarnendu Biswas
𝑆 → 𝑖𝐸𝑡𝑆𝑆′ | 𝑎𝑆′ → 𝑒𝑆 | 𝜖𝐸 → 𝑏
![Page 54: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/54.jpg)
Predictive Parsing Table
CS 335 Swarnendu Biswas
Nonterminal a b e i t $𝑆 𝑆 → 𝑎 𝑆 → 𝑖𝐸𝑡𝑆𝑆′
𝑆′ 𝑆′ → 𝜖𝑆′ → 𝑒𝑆
𝑆′ → 𝜖
𝐸 𝐸 → 𝑏 𝑇 → 𝐹𝑇′
𝑆 → 𝑖𝐸𝑡𝑆𝑆′| 𝑎𝑆′ → 𝑒𝑆 | 𝜖𝐸 → 𝑏
![Page 55: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/55.jpg)
Error Recovery in Predictive Parsing
• Error conditions• Terminal on top of the stack does not match the next input symbol
• Nonterminal 𝐴 is on top of the stack, 𝑎 is the next input symbol, and 𝑀[𝐴, 𝑎]is error
• Choices• Raise an error and quit parsing
• Print an error message, try to recover from the error, and continue with compilation
CS 335 Swarnendu Biswas
![Page 56: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/56.jpg)
Error Recovery in Predictive Parsing
• Panic mode – skip over symbols until a token in a set of synchronizing (synch) tokens appears• Add all tokens in FOLLOW(𝐴) to the synch set for 𝐴
• Add symbols in FIRST(𝐴) to the synch set for 𝐴
• Add keywords that can begin sentences
• …
CS 335 Swarnendu Biswas
![Page 57: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/57.jpg)
Predictive Parsing Table with Synchronizing Tokens
CS 335 Swarnendu Biswas
Nonterminal id + * ( ) $
𝐸 𝐸 → 𝑇𝐸′ 𝐸 → 𝑇𝐸′ synch synch
𝐸′ 𝐸′ → +𝑇𝐸′ 𝐸′ → 𝜖 𝐸′ → 𝜖
𝑇 𝑇 → 𝐹𝑇′ synch 𝑇 → 𝐹𝑇′ synch synch
𝑇′ 𝑇′ → 𝜖 𝑇′ →∗ 𝐹𝑇′ 𝑇′ → 𝜖 𝑇′ → 𝜖
𝐹 𝐹 → id synch synch 𝐹 → (𝐸) synch synch
𝐸 → 𝑇𝐸′
𝐸′ → +𝑇𝐸′ | 𝜖𝑇 → 𝐹𝑇′
𝑇′ →∗ 𝐹𝑇′ | 𝜖𝐹 → 𝐸 | id
![Page 58: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/58.jpg)
Error Recover Moves by Predictive ParserStack Input Remark
𝐸$ )id ∗ +id$ Error, skip )
𝐸$ id ∗ +id$ id is in FIRST(𝐸)
𝑇𝐸′$ id ∗ +id$
𝐹𝑇𝐸′$ id ∗ +id$
id𝑇𝐸′$ id ∗ +id$
𝑇′𝐸′$ ∗ +id$
∗ 𝐹𝑇′𝐸′$ ∗ +id$
𝐹𝑇′𝐸′$ +id$ Error, 𝑀 𝐹,+ = synch
𝑇′𝐸′$ +id$ 𝐹 has been popped
𝐸′$ +id$
CS 335 Swarnendu Biswas
![Page 59: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/59.jpg)
Error Recover Moves by Predictive ParserStack Input Remark
+𝑇𝐸′$ +id$
𝑇𝐸′$ id$
𝐹𝑇′𝐸′$ id$
id𝑇′𝐸′$ id$
𝑇′𝐸′$ $
𝐸′$ $
$ $
CS 335 Swarnendu Biswas
![Page 60: CS 335: Top-down Parsing - CSE - IIT Kanpur...CS 335: Top-down Parsing Swarnendu Biswas Semester 2019-2020-II CSE, IIT Kanpur Content influenced by many excellent references, see References](https://reader034.fdocuments.in/reader034/viewer/2022042920/5f67bdc707384906bb29ba1c/html5/thumbnails/60.jpg)
References
• A. Aho et al. Compilers: Principles, Techniques, and Tools, 2nd edition, Chapter 4.4.
• K. Cooper and L. Torczon. Engineering a Compiler, 2nd edition, Chapter 3.3.
CS 335 Swarnendu Biswas