Language Workbenches & Syntax Definition

Course IN4308Master Computer Science

Delft University of Technology

Eelco Visserhttp://eelcovisser.org

Lecture 6

Part IIModeling Languages

Language Workbenches

Lecture 6

Language Modeling

Lectures 6-7

Transformation, Generation, Analysis

Lectures 8-10

Customization

Lecture 10

Codecustomize

Model

generate

Advanced Topics

Lectures 12-14

Economics

★ costs and benefits of developing (with) DSLs

Evolution

★ maintenance of models and languages

Portability

★ supporting multiple platforms

Internal DSLs

★ library as a language

Language composition

★ combining multiple DSLs

Language Workbenches

Model

Code

AST

core

Errorsparse check

desugar

generate

Traditional Compiler Architecture

Compiler Ingredients

Syntax definition

★ concrete syntax

★ abstract syntax

Static semantics

★ error checking

★ name resolution

★ type analysis

Model-to-model transformation

★ express constructs in core language

Code generation

★ translate core language models to implementation

Integrated Development Environments

extended requirements for language definition

Editor Services for Modern IDEs

Syntactic Editor Services

★ syntax checking

★ bracket matching

★ syntax highlighting

★ code folding

★ outline view

Semantic Editor Services

★ error checking

★ reference resolving

★ hover help

★ code completion

★ refactoring

Demonstration: NWL in Spoofax

Editor

Code

AST

core

IntegratedDevelopmentEnvironment

feedback

parse

Architectural Requirements for IDEs

Editor services are based on AST

★ services analyze structure, not text of source

Error recovery

★ continue services in presence of syntactic errors

Incremental processing

★ effort of applying analysis, transformation, generation should be proportional to change in source

Separate compilation (analysis, transformation)

★ keep track of dependencies

Holy Grail of Software Language Definition

Automatically derive efficient, scalable, incremental compiler + usable IDE from high-level, declarative language

definition

Language Definition Tools

Parser generators

★ automatically generate parsers from grammars

★ LEX/YAC, Bison, ANTLR, ...

Meta-programming tools

★ DSLs for programming with programs as data

★ TXL, ASF+SDF, Stratego, JastAdd, RascalMPL, ...

IDE generators/frameworks

★ tools for construction of IDE plugins

★ IMP, TMF

★ Synthesizer Generator, Centaur, Lrc

Language Workbenches

Integrated development environment for

★ language definition

★ application of language

Textual languages

★ ASF+SDF MetaEnvironment, XText, MontiCore, EMFText, Spoofax

Structure editors

★ MPS

Visual languages

★ MetaEdit+, DSL Tools, ...

Spoofax/IMP

The Spoofax/IMP Language Workbench

Syntax definition: SDF

★ declarative, modular syntax definition

Transformation, analysis, generation: Stratego

★ rewrite rules, strategies, dynamic rules

Editor services

★ domain-specific languages for configuration

Based on IMP framework by IBM Research

Syntax Definition

Syntax Definition

Parse Table Signature Pretty-Print Table

TransformParse Pretty-Print

entity User { name :: String pw :: Secret}def output(u : User) {

@Entityclass User { String _user; public User getUser() { return _user; }syntax definition is basis of language definition

Lexical & Context-Free Syntax

composed from

Structure of Sentences

Context-Free Grammar

grammar production

symbol

DefinitionProperty*

{Stat “;”}*“entity”

regular expressionsort

Lexical Syntax

character class

characters in lexical syntax not separated by layout

regular expression

Syntax of Data Models

RecognizingWell-Formed

Sentences

Lexical Syntax: Identifiers & Literals

follow restriction: symbol may not be followed by character from this class: BlogEntry is one ID

reserved words

Lexical Syntax: Strings

complement

escape

escape escape

Lexical Syntax: Whitespace

LAYOUT? is inserted between symbols in CF productions

normalize

Lexical Syntax: Comment

Modular Syntax Definition

Meta-Grammar

Parsing & Abstract Syntax

building structured representation of sentence

Constructor Annotations

Terms to Represent Abstract Syntax

Entity(“Blog”, [Property(“name”,SimpleType(“String”)), Property(“author”, SimpleType(“User”))])

Parsing

Parsing

Parsing

Parsing

Parsing

Parsing in Spoofax

Signatures

defining the structure of abstract syntax terms

Syntax Definition

Parse Table Signature Pretty-Print Table

TransformParse Pretty-Print

entity User { name :: String pw :: Secret}def output(u : User) {

@Entityclass User { String _user; public User getUser() { return _user; }syntax definition is basis of language definition

Signatures

declare arguments and types of constructorssignature is generated from grammar

Disambiguation

Arithmetic Expressions

Plus(Times(Var(“x”),Int(2)),Var(“z”))note: this style produces good abstract syntax

Associativity

Priority

Parsing after Disambiguation

Brackets

Syntax Definition

Parse Table Signature Pretty-Print Table

TransformParse Pretty-Print

entity User { name :: String pw :: Secret}def output(u : User) {

@Entityclass User { String _user; public User getUser() { return _user; }syntax definition is basis of language definition

Schedule

Lab this week

★ Design 1: deadline is April 1

★ Design 2: what DSL will you design?

Cases

★ Case 2: web abstractions

★ Deadline Case 2: April 8

Next

★ Lecture 7: editor services & basic code generation