Lexical Analysis

Arial Font Family

Figure 3.1: Interactions between the lexical analyzer and the parser

Figure 3.2: Examples of tokens

Figure 3.3: Using a pair of input buffers

Figure 3.4: Sentinels at the end of each buffer

Figure 3.5: Lookahead code with sentinels

Figure 3.6: Definitions of operations on languages

Figure 3.7: Algebraic laws for regular expressions

Figure 3.8: Lex regular expressions

Figure 3.9: Filename expressions used by the shell command sh

Figure 3.10: A grammar for branching statements

Figure 3.11: Patterns for tokens of Example 3.8

Figure 3.12: Tokens, their patterns, and attribute values

Figure 3.13: Transition diagram for relop

Figure 3.14: A transition diagram for id's and keywords

Figure 3.15: Hypothetical transition diagram for the keyword then

Figure 3.16: A transition diagram for unsigned numbers

Figure 3.17: A transition diagram for whitespace

Figure 3.18: Sketch of implementation of relop transition diagram

Figure 3.19: Algorithm to compute the failure function for keyword blb2 . . . bn

Figure 3.20: The KMP algorithm tests whether string ala2 . . a, contains a single keyword bl b2 . . .

bn as a substring in O(m + n) time

Figure 3.21: Trie for keywords he, she, his, hers

Figure 3.22: Creating a lexical analyzer with Lex

Figure 3.23: Lex program for the

tokens of Fig. 3.12

Figure 3.24: A nondeterministic finite automaton

Figure 3.25: Transition table for the NFA of Fig. 3.24

Figure 3.26: NFA accepting aa* 1 bb*

Figure 3.27: Simulating a DFA

Figure 3.28: DFA accepting (aJb)*abb

Figure 3.29: NFA for Exercise 3.6.3

Figure 3.30: NFA for Exercise 3.6.4

Figure 3.31: Operations on NFA states

Figure 3.32: The subset construction

Figure 3.33: Computing E- closure(T)

Figure 3.34: NFA N for (alb)*abb

Figure 3.35: Transition table Dtran for DFA D

Figure 3.36: Result of applying the subset

construction to Fig. 3.34

Figure 3.37: Simulating an NFA

Figure 3.38: Adding a new state s, which is known not to be on newstates

Figure 3.39: Implementation of step (4) of Fig. 3.37

Figure 3.40: NFA for the union of two regular expressions

Figure 3.41: NFA for the concatenation of two regular expressions

Figure 3.42: NFA for the closure of a regular expression

Figure 3.43: Parse

tree for (alb)*abb

Figure 3.44: NFA for r3

Figure 3.45: NFA for r5

Figure 3.46: NFA for r

Figure 3.47: An NFA that has many fewer states than the smallest equivalent DFA

Figure 3.48: Initial cost and per-string-cost of various methods of recognizing the language of a regular expression

Figure 3.49: A Lex program is turned into a transition table and actions, which are used

by a finite-automaton simulator

Figure 3.50: An NFA constructed from a Lex program

Figure 3.51: NFA's for a, abb, and a*b+

Figure 3.52: Combined NFA

Figure 3.53: Sequence of sets of states entered when processing

input aaba

Figure 3.54: Transition graph for DFA handling the patterns a, abb, and a*b+

Figure 3.55: NFA recognizing the keyword IF

Figure 3.56:

Syntax tree for

(aJb)*abb#

Figure 3.57: NFA constructed by Algorithm 3.23 for (a(b)*abb#

Figure 3.58: Rules for computing nullable and firstpos

Figure 3.59:

firstpos and

lastpos for nodes in

the syntax tree for

(alb)*abb#

Figure 3.60: The function followpos

Figure 3.61: Directed graph for the function followpos

Figure 3.62: Construction of a DFA directly from a regular expression

Figure 3.63: DFA constructed from Fig. 3.57

Figure 3.64: Construction of Π new

Figure 3.65: Transition table of minimum-state DFA

Figure 3.66: Data structure for representing transition tables

Terima Kasih