Bottom up parsing

Bottom-Up Parsing

Bottom-up parsing is more general than top- down.

A bottom-up parser builds a derivation by working

from the input sentence back toward the start symbol S

Preferred method in practice

Also called LR parsing

L means that tokens are read left to right

R means that it constructs a rightmost derivation

Bottom-up Parsing

• Two types of bottom-up parsing:

• Operator-Precedence parsing

• LR parsing

• covers wide range of grammars.

• SLR – simple LR parser

• LR – most general LR parser

• LALR – intermediate LR parser (Lookahead LR

parser)

Bottom-Up Parsing

LR parsing reduces a string to the start symbol by inverting productions:

Str input string of terminals

repeat

Identify β in str such that A →β is a production

Replace β by A in str

until str = S (the start symbol) OR all possibilities are exhausted

Bottom-Up Parsing

int + (int) + (int)

E + (int) + (int)

E + (E) + (int)

E + (int)

E + (E)

E

E → E + ( E ) | int

A rightmost derivation

in reverse

ReductionsBottom-up parsing as the process of

"reducing" a string w to the start symbol of the grammar.

At each reduction step, a substring that matches the body of a production is replaced by the non-terminal at the head of that production.

Handle

Handle of a string: Substring that matches the

RHS of some production AND whose reduction

to the non-terminal on the LHS is a step along

the reverse of some rightmost derivation.

Handles

Formally:

Handle of a right sentential form is <A ,

location of in >

i.e. A is a handle of at the location

immediately after the end of , if:S => A =>

A certain sentential form may have many different

handles.

Right sentential forms of a non-ambiguous grammar

have one unique handle

Example

S aABe aAde aAbcde abbcde

S aABe

A Abc | b

B d

S aABe is a handle of aABe in location 1.

B d is a handle of aAde in location 3.

A Abc is a handle of aAbcde in location 2.

A b is a handle of abbcde in location 2.

Handle Pruning

A rightmost derivation in reverse can be obtained by “handle-pruning.”

abbcde

Find the handle = b at loc. 2

aAbcde

b at loc. 3 is not a handle:

aAAcde

... blocked.

Handle-pruning

The process of discovering a handle & reducing it to the appropriate left-hand side is

called handle pruning.

Handle pruning forms the basis for a bottom-up parsing method.

To construct a rightmost derivation

S = 0 1 2 … n-1 n = w

Apply the following simple algorithm

for i n to 1 by -1

Find the handle Ai i in i

Replace i with Ai to generate i-1

1. S -> 0 S1|01 indicate the handle in each of the following right-sentential forms:

1. 000111

2. 00S11

2. For the grammar S -> S S + | S S * | a indicate the handle in each of the following

right-sentential forms:

1. SSS + a * +

2. SS + a * a+

3. aaa * a + +.

3. Give bottom-up parses for the following input strings

1. The input 000111 according to the grammar of

Exercise 1

2. The input aaa * a + + according to the grammar of 2

Shift Reduce Parsing with a

StackTwo problems:

locate a handle

decide which production to use (if there are

more than two candidate productions).

Shift-reduce Parsing

A shift-reduce parser is a stack automaton with four actions

Shift — next word is shifted onto the stack

Reduce — right end of handle is at top of stack

Locate left end of handle within the stack

Pop handle off stack & push appropriate lhs

Accept — stop parsing & report success

Error — call an error reporting/recovery routine

Accept & Error are simple

Shift is just a push and a call to the scanner

Reduce takes |rhs| pops & 1 push

x - 2 * y

Stack Input Handle Action

$ id - num * id None shift

$ id - num * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Back to x - 2 * yStack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 4

$ Expr - num * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Back to x - 2 * yStack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 4

$ Expr - num * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Expr is not a handle at this point because it does not

occur at this point in the derivation.

Back to x - 2 * yStack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 3

$ Expr - num * id none shift

$ Expr - num * id none shift

$ Expr - num * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Back to x - 2 * yStack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 3

$ Expr - num * id none shift

$ Expr - num * id none shift

$ Expr - num * id 7,3 reduce 7

$ Expr - Factor * id 6,3 reduce 6

$ Expr - Term * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Back to x - 2 * yStack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 3

$ Expr - num * id none shift

$ Expr - num * id none shift

$ Expr - num * id 7,3 reduce 7

$ Expr - Factor * id 6,3 reduce 6

$ Expr - Term * id none shift

$ Expr - Term * id none shift

$ Expr - Term * id

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Back to x - 2 * y

5 shifts +

9 reduces +

1 accept

Stack Input Handle Action

$ id - num * id none shift

$ id - num * id 8,1 reduce 8

$ Factor - num * id 6,1 reduce 6

$ Term - num * id 3,1 reduce 3

$ Expr - num * id none shift

$ Expr - num * id none shift

$ Expr - num * id 7,3 reduce 7

$ Expr - Factor * id 6,3 reduce 6

$ Expr - Term * id none shift

$ Expr - Term * id none shift

$ Expr - Term * id 8,5 reduce 8

$ Expr - Term * Factor 4,5 reduce 4

$ Expr - Term 2,3 reduce 2

$ Expr 0,1 reduce 0

$ Goal none accept

1. Shift until the top of the stack is the right end of a handle

2. Find the left end of the handle and reduce

0 Goal Expr

1 Expr Expr + Term

2 | Expr - Term

3 | Term

4 Term Term * Factor

5 | Term / Factor

6 | Factor

7 Factor number

8 | id

9 | ( Expr )

Goal

<id,x>

Term

Fact.

Expr –

Expr

<id,y>

<num,2>

Fact.

Fact.Term

Term

*

Stack Input Action

$ id - num * id shift

$ id - num * id reduce 8

$ Factor - num * id reduce 6

$ Term - num * id reduce 3

$ Expr - num * id shift

$ Expr - num * id shift

$ Expr - num * id reduce 7

$ Expr - Factor * id reduce 6

$ Expr - Term * id shift

$ Expr - Term * id shift

$ Expr - Term * id reduce 8

$ Expr - Term * Factor reduce 4

$ Expr - Term reduce 2

$ Expr reduce 0

$ Goal accept

Back to x - 2 * y

Corresponding Parse Tree

Conflicts During Shift-Reduce Parsing

Conflicts“shift/reduce” or “reduce/reduce”

Example:

stmt if expr then stmt

| if expr then stmt else stmt

| other (any other statement)

Stack Input

if … then stmt else … Shift/ Reduce Conflict

We can’t tell

whether it is a

handle

LR Parsing

Bottom-up parser based on a concept called

LR(k) parsing

"L" is for left-to-right scanning of the input.

"R" for constructing a rightmost derivation in

reverse,

“k” for the number of input symbols of

lookahead that are used in making parsing

decisions.

Why LR Parsers?

For a grammar to be LR it is sufficient that a

left-to-right shift-reduce parser be able to

recognize handles of right-sentential forms

when they appear on top of the stack.

Why LR Parsers?

LR parsers can be constructed to recognize allprogramming language constructs for whichcontext-free grammars can be written.

The LR-parsing method is the most general non-back-tracking shift-reduce parsing method.

An LR parser can detect a syntactic errors.

The class of grammars that can be parsed using LRmethods is a proper superset of the class ofgrammars that can be parsed with predictive or LLmethods.

Components of LR Parser

X

Y

Z

$

A + B $

Parsing Program

Parse Table

Action Goto

outputstack

Input buffer

Techniques for Creating Parse Table

SLR: Construct parsing table for small set of grammars called SLR(1).

Easy to develop.

CLR(CANONICAL LR) : Most powerful.

Generates a large parse table.

More difficult develop.

Works for all types of CFG

May detect position of error.

LALR(LOOK AHEAD LR) :Widely used method.

Optimizes the size of parse table, by combining some states.

Information may get lost.

SLR Parsers

1. Formation of augmented grammar G’ for

the given grammar G

2. Construction of LR(0) collection of

items.

To find LR(0) collection of items Closure(I)

and Goto(I,X) have to be computed.

3. Finding first and follow of non-terminals

4. Construction of parse table.

Formation of Augmented

GrammarThe augmented grammar G’, is G with a new start

symbol S’ and an additional production S’ -> S

E->E+T|T

T->T*F|F

The augmented grammar G’ is given by

E’->E

E->E+T|T

T->T*F|F

Items and the LR(O)

AutomatonHow does a shift-reduce parser know when

to shift and when to reduce?

How does the parser know that symbol on thetop of the stack is not a handle?

An LR parser makes shift-reduce decisionsby maintaining states to keep track of all theoperations.

States represent sets of "items."

Items and the LR(O)

Automaton An LR(O) item of a grammar G is a production of G, with a

dot at some position in the body.

Production A -> XYZ yields the four items

A -> ·XYZ

A -> X·YZ

A -> XY·Z

A -> XYZ·

A ->ε generates only one item, A ->.

An item indicates how much of a production we have seen

at a given point in the parsing process.

Items and the LR(O)

Automaton The item A -> ·XYZ indicates that we hope to see a string derivable from XYZ next

on the input.

Item A -> X·YZ indicates that we have just seen on the input a string derivable from

X and that we hope next to see a string derivable from YZ.

Item A -> XY·Z indicates that we have just seen on the input a string derivable from

XY and that we hope next to see a string derivable from Z.

Item A -> XYZ· indicates that we have seen the body XY Z and that it may be time

to reduce XYZ to A

Items and the LR(O)

Automaton Collection of sets of LR(0) items, called the

canonical LR(0) collection.

Provides the basis for constructing a deterministic

finite automaton that is used to make parsing

decisions.

Automaton is called an LR(0) automaton.

Each state of the LR(0) automaton represents a

set of items in the canonical LR(0) collection.

Construction of LR(0) Items

Items are viewed as states in NFA.

Grouped to form same states.

Process of grouping together is called subset

Construction Algorithm.

Closure and goto operations have to be

computed.

Closure of Item Sets

If I is a set of items for a grammar G, then CLOSURE(I)

is the set of items constructed from I by the two rules:

Initially, add every item in I to CLOSURE(I).

If A -> α·Bγ is in CLOSURE(I) and B -> γ is a

production, then add the item B -> γ to CLOSURE(I),

if it is not already there. Apply this rule until no more

new items can be added to CLOSURE (I).

Computation of CLOSURE

E’ EE E + T | T

T T * F | F

F ( E ) | id

I0

E’ .E

E .E + T

E .T

T .T * F

T .F

F .( E )

F .id

Computation of CLOSURE

SetOfltems CLOSURE (I) {

J = I;

repeat

for ( each item A -> a·Bγ in J )

for ( each production B ->γ of G )

if (B ->.γ is not in J )

add B ->.γ to J;

until no more items are added to J on one round;

return J;

}

Kernel items : The initial item, S' ->·S, and all items whose dots are not at

the left end.

Non-kernel items : All items with their dots at the left end, except for S' ->

·S.

The Function GOTO

GOTO (I, X) is defined to be the closure of the set of all items [A -> αX.β] such that

[A -> α.Xβ] is in I.

If I is the set of two items { [E' -> E·] , [E -> E· + T] } , then

GOTO(I, +) contains the items

E -> E + ·T

T -> ·T * F

T -> ·F

F -> · (E)

F -> ·id

The Function GOTO

Grammar

S E + S | E

E num

E num .

S’ S . $

num

ES’ . S $

S .E + S

S . E

E .num

1

S E . +S

S E .

2

S E + S .

5

S E + . S

S . E + S

S . E

E . num

3

S’ S $ .

7

4

S

S

$

+

E

num

num + $ E S

1 s4 g2 g6

2 s3 SE

3 s4 g2 g5

4 Enum Enum

5 SE+S

6 s7

7 accept

S' SS L=RS RL *RL idR L

id

S' SS L=RS RL *RL idR L

L id

S L =RR L

S' S I0

I1

I2

I3

S R I4

L * RR LL idL * R

I5

S

L

*

id R

S L= RR LL *RL id

I6

=

RS L=R

R L L

L

I7

id

I3

*

*

L *R R

I8

I9

state action gotoid = * $ S L R

0 s3 s5 1 2 41 accept2 s6/r(RL)3 r(Lid) r(Lid)4 r(SR)5 s3 s5 7 86 s3 s5 7 97 r(RL) r(RL)8 r(L*R) r(L*R)9 r(SL=R)

Structure of the LR Parsing

Table1. The ACTION function takes as arguments a state i and a terminal

a (or $, the input endmarker). The value of ACTION[i, a] can have one of four forms:

a) Shift j , where j is a state. The action taken by the parser shifts input a to the stack, but uses state j to represent a .

b) Reduce A ->β. The action of the parser reduces β on the top of the stack to head A.

c) Accept. The parser accepts the input and finishes parsing;

d) Error. The parser discovers an error in its input and takes some corrective action.

2. Extend the GOTO function, defined on sets of items, to states: if GOTo [Ii , A] = Ij , then GOTO also maps a state i and a nonterminal A to state j .

Construction of SLR Parse

Table1. Construct C={I0,I1…….In} the collection sets of LR(0)

items for G’.

2. Initial state of the parser is constructed from the set of items

for [S’->S]

3. State I is constructed from Ii. The parsing actions are

determined as follows.

1. If [A->α.aβ] is in Ii and GOTO(Ii,a]=Ij, then ACTION[i,a]

is set to ‘shift j’ here ‘a’ must be a terminal.

2. If[A->α.] is in Ii, then ACTION[i,a] is set to reduce A->a

for all ‘a’ Follow(A).

3. If S’->S is in Ii, than action[i, $] is set to ‘accept’.

Construction of SLR parsing

table4. GOTO transitions are constructed for all non-terminals. If GOTO(Ii,A) =Ij, then

goto(i,A)of the parse table is set to j.

5. All other entries are error entries.

LR-Parser Configurations

Helps to have complete state o its stack and the remaining

input. A configuration of an LR parser is a pair:(s0s1

………… sm, aiai+1…………an$).

where the first component is the stack contents and the

second component is the remaining input.

Behavior of the LR Parser

LR-parsing algorithm.

INPUT: An input string w and an LR-parsing table with functions ACTION

and GOTO for a grammar G.

OUTPUT: If w is in L ( G), the reduction steps of a bottom-up parse for W ;

otherwise, an error indication.

METHOD: Initially, the parser has So on its stack, where So is the initial

state, and w$ in the input buffer.

LR-Parser Configurations