Tomita‘s Parser Tihomira Panayotova Paolina Teneva
-
Upload
adriana-synge -
Category
Documents
-
view
49 -
download
1
description
Transcript of Tomita‘s Parser Tihomira Panayotova Paolina Teneva
![Page 1: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/1.jpg)
Tomita‘s Parser
Tihomira PanayotovaPaolina Teneva
Seminar für Sprachwissenschaft
31.01.2007
![Page 2: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/2.jpg)
A simple overview
LR(0) conflicts Tomita‘s method summarized Complications Two optimiziations A moderately ambiguous grammar Stack duplication Combinig equal states Combinig equal stack prefixes Discussion Summary
![Page 3: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/3.jpg)
LR(0) Conflicts
LR Parser:
1.handle recognizing FS automaton
2. no inadequate states
There exist grammars for which the automaton has some inadequate states
![Page 4: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/4.jpg)
LR(0) Conflicts
![Page 5: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/5.jpg)
Tomita‘s method summarized
Simple definition: – A breadth-first search over those parsing
decisions that are not solved by the LR automaton
– It gives an efficient and very effective approach to grammars for which the automaton has some inadequate states
![Page 6: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/6.jpg)
Tomita‘s method summarized
How does the parser act when it encounters an inadequate state on the top of the stack?
Step1. It duplicates the stack and splits the parse into a different process for each copy:
One copy is reserved for the REDUCE step
The other copy is reserved for the SHIFT step
Step2. Stacks that have a right-most state that does not allow a shift on the next input token are DISCARDED
![Page 7: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/7.jpg)
Tomita‘s method summarized
SHIFT step: push a new symbol and the new state onto the stack
REDUCE step: removes part of the right end of the stack and replaces it with a non-terminal; using this non-terminal as a move in the automaton, we find a new state to put on the top of the stack
Conclusion: Every time we encounter an inadequate state on the top of the stack, the duplication process is repeated untill all reduces have been treated.
![Page 8: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/8.jpg)
Complications
The repetition of the duplication process can cause a proliferation of stacks. A great number will be copied and subsequently discarded
If all stacks are discarded in Step2 => the input was in error
Grammars with loops : A->B B->A - the process may not terminate
![Page 9: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/9.jpg)
Complications
Some ideas to cope with the complications:
1. Use of look-ahead to decide which reduces can be made in Step1
2.Grammar with loops: 2.1. upon creating a stack, check if it is already there (and then ignore it) 2.2 check the grammar in advance for loops (and then reject it).
![Page 10: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/10.jpg)
Two optimizations
Combining equal states
Combibing equal stack prefixes
![Page 11: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/11.jpg)
A moderately ambiguous grammar
SS -> E #
E -> E + E
E -> d Figure 9.38 A moderately ambiguous grammar
![Page 12: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/12.jpg)
LR(0) automaton for the grammar
![Page 13: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/13.jpg)
Stack Duplication
a. 1 d+d+d# shiftb. 1 d 2 +d+d# reducec. 1 E 3 +d+d# shiftd. 1 E 3 + 4 d+d# shifte. 1 E 3 + 4 d 2 +d# reducef. 1 E 3 + 4 E 5 +d# duplicate to g1 and g2 g1. 1 E 3 + 4 E 5 +d# REDUCE; reduce to g1.1g2. 1 E 3 + 4 E 5 +d# SHIFT; shift to g1.2g1.1 1 E 3 +d# shift to h1g1.2 1 E 3 + 4 E 5 + 4 d# shift to h2h1 1 E 3 + 4 d# shift to h1.1h2 1 E 3 + 4 E 5 + 4 d 2 # reduce to h1.2h1.1 1 E 3 + 4 d 2 # reduce to ih1.2 1 E 3 + 4 E 5 + 4 E 5 # duplicate to i1 and i2 i 1 E 3 + 4 E 5 # duplicate to j1 and j2
![Page 14: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/14.jpg)
Stack Duplication
i1 1 E 3 + 4 E 5 + 4 E 5 #REDUCE, reduce to k1I2 1 E 3 + 4 E 5 + 4 E 5 #SHIFT - DISCARDEDj1 1 E 3 + 4 E 5 #REDUCE, reduce to k2j2 1 E 3 + 4 E 5 #SHIFT - DISCARDEDk1 1 E 3 + 4 E 5 # reduce to l1k2 1 E 3 # shift to l2l1 1 E 3 # shift to m1l2 1 E 3 # 6 reduce to m2m1 1 E 3 # 6 reduce to nm2 1 S ACCEPTn 1 S ACCEPT
![Page 15: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/15.jpg)
Parse trees
![Page 16: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/16.jpg)
Combining equal states
Examine the following:
Both stacks have the same state on top=>further actions on both stacks will be identical
Combine the two stacks to avoid duplicate work
![Page 17: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/17.jpg)
Combining equal states
f) 1. 1 E 3 + 4 d 2 # both
2. 1 E 3 + 4 E 5 + 4 d 2 # REDUCE to g
![Page 18: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/18.jpg)
Combining equal states
f) 1. 1 E 3 + 4 d 2 # both
2. 1 E 3 + 4 E 5 + 4 d 2 # REDUCE to g
g) 1. 1 E 3 + 4 E 5 # duplicate to
2. 1 E 3 + 4 E 5 + 4 E 5 # g’ and g ’’
![Page 19: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/19.jpg)
Combining equal states
f) 1. 1 E 3 + 4 d 2 # both
2. 1 E 3 + 4 E 5 + 4 d 2 # REDUCE to g
g) 1. 1 E 3 + 4 E 5 # duplicate to
2. 1 E 3 + 4 E 5 + 4 E 5 # g’ and g ’’
g’) 1. 1 E 3 + 4 E 5 # for REDUCE
2. 1 E 3 + 4 E 5 + 4 E 5 #
![Page 20: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/20.jpg)
Combining equal states
f) 1. 1 E 3 + 4 d 2 # both
2. 1 E 3 + 4 E 5 + 4 d 2 # REDUCE to g
g) 1. 1 E 3 + 4 E 5 # duplicate to
2. 1 E 3 + 4 E 5 + 4 E 5 # g’ and g ’’
g’) 1. 1 E 3 + 4 E 5 # for REDUCE
2. 1 E 3 + 4 E 5 + 4 E 5 #
g’’) 1. 1 E 3 + 4 E 5 # copy to h3)
2. 1 E 3 + 4 E 5 + 4 E 5 # for SHIFT
![Page 21: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/21.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
![Page 22: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/22.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
![Page 23: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/23.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
h.1 ) 1 E 5 # SHIFT
![Page 24: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/24.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
h.1 ) 1 E 5 # SHIFT
h2 ) 1 E 3 + 4 E 5 # REDUCE to h2.1) and h2.2)
![Page 25: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/25.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
h.1 ) 1 E 5 # SHIFT
h2 ) 1 E 3 + 4 E 5 # REDUCE to h2.1) and h2.2)
h2.1) 1 E 5 # SHIFT
![Page 26: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/26.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
h.1 ) 1 E 5 # SHIFT
h2 ) 1 E 3 + 4 E 5 # REDUCE to h2.1) and h2.2)
h2.1) 1 E 5 # SHIFT
h2.2) 1 E 3 + 4 E 5 # SHIFT
![Page 27: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/27.jpg)
Combining equal states
g’.1) 1 E 3 + 4 E 5 # REDUCE to h.1)
g’.2) 1 E 3 + 4 E 5 + 4 E 5 # REDUCE to h.2)
h.1 ) 1 E 5 # SHIFT
h2 ) 1 E 3 + 4 E 5 # REDUCE to h2.1) and h2.2)
h2.1) 1 E 5 # SHIFT
h2.2) 1 E 3 + 4 E 5 # SHIFT
h3) 1. 1 E 3 + 4 E 5 # SHIFT
2. 1 E 3 + 4 E 5 + 4 E 5
![Page 28: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/28.jpg)
Combining equal states
Now we have five stacks (h1, h2.1, h2.2, h3). h1) and h2.1) carry state (3) on top h2.2) and h3) carry state (5) on top
h1 ) 1 E 3 # SHIFT
h2 ) 1 E 3 + 4 E 5 # REDUCE to h2.1), copy to h2.2)
h2.1) 1 E 3 # SHIFT
h2.2) 1 E 3 + 4 E 5 # SHIFT
h3) 1. 1 E 3 + 4 E 5 # SHIFT 2. 1 E 3 + 4 E 5 + 4 E 5
![Page 29: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/29.jpg)
Combining equal states
We combine the stacks with identical states on top into two bundles h’ and h’’.
h’) h1) 1 E 3 #copy to i)
h2.1) 1 E 3
![Page 30: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/30.jpg)
Combining equal states
We combine the stacks with identical states on top into two bundles h’ and h’’.
h’) h1) 1 E 3 #copy to i)
h2.1) 1 E 3
h’’) h3) 1. 1 E 3 + 4 E 5
2. 1 E 3 + 4 E 5 + 4 E 5 #discard
h2) 1 E 3 + 4 E 5
![Page 31: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/31.jpg)
Combining equal states
i) 1) 1 E 3
3 # 6
2) 1 E 3
![Page 32: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/32.jpg)
Combining equal states
i) 1) 1 E 3
3 # 6
2) 1 E 3
i’) 1 E 3 # 6 REDUCE to j1)
![Page 33: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/33.jpg)
Combining equal states
i) 1) 1 E 3
3 # 6
2) 1 E 3
i’) 1 E 3 # 6 REDUCE to j1)
i”) 1 E 3 # 6 REDUCE to j2)
![Page 34: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/34.jpg)
Combining equal states
i) 1) 1 E 3
3 # 6
2) 1 E 3
i’) 1 E 3 # 6 REDUCE to j1)
i”) 1 E 3 # 6 REDUCE to j2)
j1) 1 S ACCEPT
![Page 35: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/35.jpg)
Combining equal states
i) 1) 1 E 3
3 # 6
2) 1 E 3
i’) 1 E 3 # 6 REDUCE to j1)
i”) 1 E 3 # 6 REDUCE to j2)
j1) 1 S ACCEPT
j2) 1 S ACCEPT
![Page 36: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/36.jpg)
Combining equal stack prefixes
When the parser makes the call for the stack
to be copied,there is no actual need to copy the entire stack!
It is enough to copy the top state suffixes
![Page 37: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/37.jpg)
Combining Equal Stack-Prefixes
If we observe the example :e) 1 E 3 + 4 E 5 +d#
![Page 38: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/38.jpg)
Combining Equal Stack-Prefixes
If we observe the example :e) 1 E 3 + 4 E 5 +d#
When we duplicate the stack we have two copies of
It and REDUCE is applied only to one of the copies and
only “so much” of the stack is copied:
![Page 39: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/39.jpg)
Combining Equal Stack-Prefixes
If we observe the example :e) 1 E 3 + 4 E 5 +d#
When we duplicate the stack we have two copies of
it and REDUCE is applied only to one of the copies and
only “so much” of the stack is copied:
e’) 1 E 3 +d# SHIFT
e’’) 1 E 3 + 4 E 5 +d# SHIFT
![Page 40: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/40.jpg)
Discussion
Table characteristics:
-the method can work with every bottom-up table
-the weaker the table, the more non-determinism will have to be resolved by breadth-first search
Time requirements:
- in theory – exponential
- in practice - linear or slightly more than linear
![Page 41: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/41.jpg)
SUMMARY
Breadth-first search over those parsing decisions that are not solved by the LR automaton
Important notions that should be memorized:
– stack duplication (inadequate states, reduce, shift, discarded)
– combining equal states– combining equal stack prefixes
![Page 42: Tomita‘s Parser Tihomira Panayotova Paolina Teneva](https://reader036.fdocuments.in/reader036/viewer/2022062507/5681329a550346895d99356e/html5/thumbnails/42.jpg)
References
Dick Grune & Ceriel Jacobs (1990). Parsing Techniques