Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C...
Transcript of Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C...
![Page 1: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/1.jpg)
Tutorial Lexer and Parser
Compiler Construction 1
S C I E N C E P A S S I O N T E C H N O L O G Y
u www.tugraz.at
Georg Hinteregger, Christopher Liebmann
Institute for Software Technology
Tutorial 1: Lexer & Parser
![Page 2: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/2.jpg)
Tutorial Lexer and Parser
Compiler Construction 2
Overview
ANTLR
Lexical and Syntax Analyzer
![Page 3: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/3.jpg)
Tutorial Lexer and Parser
Compiler Construction 3
Procedure
Source
Code
(.calc file)
Abstract
Syntax Tree
(AST)
Lexical and
syntactical errors
Debugging Output
(Lexer and Parser)
ANTLR Lexer
and Parser
![Page 4: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/4.jpg)
Tutorial Lexer and Parser
Compiler Construction 4
ANother Tool for Language Recognition
(ANTLR)
framework for constructing
interpreters
compilers
translators from grammatical descriptions
supports
tree construction (AST only up to version 3.5)
tree walking
translation
error recovery / reporting
![Page 5: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/5.jpg)
Tutorial Lexer and Parser
Compiler Construction 5
ANTLR - Getting started
Download
• ANTLR v3.5 (http://www.antlr3.org/download.html)
• Set classpath
Compile Example
• java org.antlr.Tool Calculator.g
• javac Calculator*.java
Run Example
• java CalculatorParser CalcExample.txt
![Page 6: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/6.jpg)
Tutorial Lexer and Parser
Compiler Construction 6
Example: Simple Calculator
Parser for a simple calculator (Calculator). It should
handle the 4 basic arithmetical operations.
ignore whitespaces.
read the input-string from a file.
Valid input
13 * 2
3.5 + 4.0
Invalid input
17 a + 3
4 + / 2
4.0 + 3
42
![Page 7: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/7.jpg)
Tutorial Lexer and Parser
Compiler Construction 7
ECLIPSE DEMO
![Page 8: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/8.jpg)
Tutorial Lexer and Parser
Compiler Construction 8
Calculator.g 1 – Lexer Rules
grammar Calculator; INT : (DIGIT DIGIT0*) | '0'; FLOAT : ((DIGIT DIGIT0*) | '0') '.' (DIGIT0)+; fragment DIGIT : '1'..'9'; fragment DIGIT0 : '0'..'9'; SIGN : '+' | '-' ; MULOP : '*' | '/' ; WHITESPACE : ( '\t' | ' ' | '\r' | '\n') { $channel =
HIDDEN; } ;
![Page 9: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/9.jpg)
Tutorial Lexer and Parser
Compiler Construction 9
Calculator.g 2 – Parser Rules
program : main_function EOF ; main_function : 'main' '(' ')' fct_body ; fct_body : '{' expression* return_stmt '}' ; expression : float_expr | int_expr; float_expr : FLOAT ( (MULOP FLOAT) | (SIGN FLOAT)) ; int_expr : INT ( (MULOP INT) | (SIGN INT)) ; return_stmt : 'return' ;
![Page 10: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/10.jpg)
Tutorial Lexer and Parser
Compiler Construction 10
Call of Lexer / Parser
public static void main(String[] args) throws Exception{ CalculatorLexer lex = new CalculatorLexer(new ANTLRFileStream(args[0])); CommonTokenStream tokens = new CommonTokenStream(lex); CalculatorParser parser = new CalculatorParser(tokens); try { parser.program(); } catch (RecognitionException e) { e.printStackTrace(); } }
![Page 11: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/11.jpg)
Tutorial Lexer and Parser
Compiler Construction 11
General Remarks
Arithmetic Rule: MUL before SIGN
Generate a structured AST
![Page 12: Tutorial 1: Lexer & Parser - ist.tugraz.at · Tutorial Lexer and Parser Compiler Construction 1 S C I E N C E P A S S I O N T E C H N O L O G Y u Georg Hinteregger, Christopher Liebmann](https://reader034.fdocuments.in/reader034/viewer/2022042221/5ec75f87236dc135c330aa38/html5/thumbnails/12.jpg)
Tutorial Lexer and Parser
Compiler Construction 12
Questions?