May 14, 2002 Macro Languages AoPL, S'02 Macro Languages Claus Brabrand Michael I. Schwartzbach...

Post on 21-Dec-2015

215 views 0 download

Tags:

Transcript of May 14, 2002 Macro Languages AoPL, S'02 Macro Languages Claus Brabrand Michael I. Schwartzbach...

May 14, 2002 Macro Languages AoPL, S'02

Macro Languages

Claus Brabrand

Michael I. Schwartzbach

BRICS, University of Aarhus, Denmark

May 14, 2002 Macro Languages AoPL, S'02

Outline

• Introduction

• Macro survey:• Lexical macros: CPP, M4, TEX, (Dylan)

• Syntax macros: C++ templates, Scheme, JTS, MS2

• The <bigwig> macro language

• Metamorphic syntax macros

• Next week: The metafront Tool• Specificity parsing• Language transformation

May 14, 2002 Macro Languages AoPL, S'02

Introduction

May 14, 2002 Macro Languages AoPL, S'02

“Macro”

Webster’s(“macro”) =

Main Entry: 2macroPronunciation: 'ma-(")krOFunction: nounInflected Form(s): plural macrosEtymology: short for macroinstructionDate: 1959“a single computer instruction that stands for a sequence of operations”

Main Entry: 2macroPronunciation: 'ma-(")krOFunction: nounInflected Form(s): plural macrosEtymology: short for macroinstructionDate: 1959“a single computer instruction that stands for a sequence of operations”

May 14, 2002 Macro Languages AoPL, S'02

Motivation (#1)

Abstraction (language extension):

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

foreach (String s in list) { System.out.println(s); }

foreach (String s in list) { System.out.println(s); }

vs.

May 14, 2002 Macro Languages AoPL, S'02

Motivation (#2)

Genericity (uniform abstraction mechanism): • Generic abstraction mechanism for all syntactic

categories:

…whereas functions only “take” and “give” expressions:

<exp> <id> ( <exps> )<exp> <id> ( <exps> )

Principle of Abstraction:

“Any semantically meaningful syntactic class can in principle be used as the body of an abstraction”

- Robert Tennent, 1981

Principle of Abstraction:

“Any semantically meaningful syntactic class can in principle be used as the body of an abstraction”

- Robert Tennent, 1981

May 14, 2002 Macro Languages AoPL, S'02

Motivation (#3)

Consistency:

Object[] strings = list.toArray();for (int i=0; i<strings.length; i++) { String s = (String) strings[i]; System.out.println(s); }

Object[] strings = list.toArray();for (int i=0; i<strings.length; i++) { String s = (String) strings[i]; System.out.println(s); }

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

vs.

May 14, 2002 Macro Languages AoPL, S'02

Motivation (#4)

Laziness (abbreviation):

vs.

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

foreach (String s in list) { System.out.println(s); }

foreach (String s in list) { System.out.println(s); }

May 14, 2002 Macro Languages AoPL, S'02

Motivation (#5)

Encapsulation (hide complexity):

vs.

foreach (String s in list) { System.out.println(s); }

foreach (String s in list) { System.out.println(s); }

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

Iterator iterator = list.iterator();while (iterator.hasNext()) { String s = (String) iterator.next(); System.out.println(s);}

May 14, 2002 Macro Languages AoPL, S'02

Macro Survey

May 14, 2002 Macro Languages AoPL, S'02

Many Macro Languages

CPP

M4

TEX

Scheme

C++templates

MS2

...and many more

Dylan

JTS

<bigwig>

May 14, 2002 Macro Languages AoPL, S'02

Non-Alphabetical Characters

CPP

M4

TEX

Scheme

C++templates

MS2

...and many more

Dylan

JTS

<bigwig>

May 14, 2002 Macro Languages AoPL, S'02

Level of Operation

CPP

M4

TEX

Scheme

C++templates

Dylan

JTS

Lexical Syntactic

...and many more

MS2

<bigwig>

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macros

• Operate on token sequences:• MLEX: (TOKENSEQ)n TOKENSEQ

• Precede actual compilation (conceptually)– i.e. a preprocessor

• Independent of host language syntax

• Languages:• CPP: “The C Preprocessor”• M4: “The Unix Macro Preprocessor”• TEX: TEX’s macro mechanism• Dylan: Dylan’s macro mechanism (hybrid)

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):#define square(X) X * X#define square(X) X * X

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):#define square(X) X * X

square(z + 1)

#define square(X) X * X

square(z + 1)

( )z + 1square

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):#define square(X) X * X

square(z + 1) => z + 1 * z + 1

#define square(X) X * X

square(z + 1) => z + 1 * z + 1

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):

• Work-around: Explicitly add parentheses

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) (X) * (X)#define square(X) (X) * (X)

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):

• Work-around: Explicitly add parentheses

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) (X) * (X)

square(z + 1) => (z + 1)*(z + 1)

#define square(X) (X) * (X)

square(z + 1) => (z + 1)*(z + 1)

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macro Example

• Square (CPP):

• Work-around: Explicitly add parentheses

– Problem: Independent of host language syntax• Unsafe: parse errors discovered at invocation-time

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) X * X

square(z + 1) => z +(1 * z)+ 1

#define square(X) (X) * (X)

square(z + 1) => (z + 1)*(z + 1)

#define square(X) (X) * (X)

square(z + 1) => (z + 1)*(z + 1)

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

Syntactic Macros

• Operate on abstract syntax trees:• MSYN: (AST)n AST

• Integrated with host language– typed with host language nonterminals

• Languages:• C++: The C++ template mechanism• Scheme: Scheme’s define-syntax mechanism• JTS: “Jakarta Tool Suite” macro

mechanism• MS2: “Meta Syntactic Macro System”• <bigwig>: The <bigwig> macro language

May 14, 2002 Macro Languages AoPL, S'02

Syntactic Macro Example

• Square (<bigwig>):

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

May 14, 2002 Macro Languages AoPL, S'02

Syntactic Macro Example

• Square (<bigwig>):

**

exp

exp

Eexp

E

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}~

May 14, 2002 Macro Languages AoPL, S'02

Syntactic Macro Example

• Square (<bigwig>):

**

exp

exp

Eexp

E

y + 1y

+ 1

exp

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

square( )

~

May 14, 2002 Macro Languages AoPL, S'02

Syntactic Macro Example

• Square (<bigwig>):

**

exp

exp

Eexp

E

**

exp

y + 1y

+ 1 y + 1y

+ 1

exp

Eexp

Ey

+ 1y + 1

exp

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

macro <exp> square ( <exp E> ) ::= { <E> * <E>

}

square( ) =>

~

May 14, 2002 Macro Languages AoPL, S'02

Lexical Macros

CPP, M4, TEX, (Dylan)

( )z + 1 z + 1 z + 1*=>square

May 14, 2002 Macro Languages AoPL, S'02

CPP

• CPP (“The C Preprocessor”):

• Also as a stand-alone expander:• gcc –E program• cpp program

• Intercepts preprocessor directives “#”:• #define, #undef, #ifdef, #if, #include, #line, ##, …

#define square(X) (X) * (X)square(z + 1) => (z + 1)*(z + 1)

#define square(X) (X) * (X)square(z + 1) => (z + 1)*(z + 1)

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; } #define swap(X,Y) { int t=X; X=Y; Y=t; }

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’

#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)

if (a>b) swap(a,b); else b = 0;

May 14, 2002 Macro Languages AoPL, S'02

Fixed Invocation Syntax

– Problem: fixed invocation syntax• same for exp, stm, … M(x,y,z)

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) { int t=X; X=Y; Y=t; }

if (a>b) swap(a,b); else b = 0;

*** test.c:3: parse error before ‘else’*** test.c:3: parse error before ‘else’

#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)

if (a>b) swap(a,b); else b = 0;

#define swap(X,Y) do { int t=X; X=Y; Y=t; } while(0)

if (a>b) swap(a,b); else b = 0;

May 14, 2002 Macro Languages AoPL, S'02

Body Expansion

• Consider: #define A 87#define B A#undef A#define A 42

B => ???

#define A 87#define B A#undef A#define A 42

B => ???

May 14, 2002 Macro Languages AoPL, S'02

Body Expansion

• Consider:

• Eager expansion (definition-time):

#define A 87#define B A#undef A#define A 42

B => ???

#define A 87#define B A#undef A#define A 42

B => ???

B => 87B => 87

May 14, 2002 Macro Languages AoPL, S'02

Body Expansion

• Consider:

• Eager expansion (definition-time):

• Lazy expansion (invocation-time):

#define A 87#define B A#undef A#define A 42

B => ???

#define A 87#define B A#undef A#define A 42

B => ???

B => 87B => 87

B => AB => A CPP

May 14, 2002 Macro Languages AoPL, S'02

Body Expansion

• Consider:

• Eager expansion (definition-time):

• Lazy expansion (invocation-time):

#define A 87#define B A#undef A#define A 42

B => ???

#define A 87#define B A#undef A#define A 42

B => ???

B => 87B => 87

B => A => 42B => A => 42 CPP

May 14, 2002 Macro Languages AoPL, S'02

Order of Expansion

• Consider: #define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

#define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

May 14, 2002 Macro Languages AoPL, S'02

Order of Expansion

• Consider:

• Inner (aka. “AOR”, call-by-value):

#define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

#define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

one(two) => one(a,b) => *** arity error ‘one’one(two) => one(a,b) => *** arity error ‘one’

May 14, 2002 Macro Languages AoPL, S'02

Order of Expansion

• Consider:

• Inner (aka. “AOR”, call-by-value):

• Outer (aka. “NOR”, call-by-name):

#define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

#define id(X) X#define one(X) id(X)#define two a,b

one(two) => ???

one(two) => one(a,b) => *** arity error ‘one’one(two) => one(a,b) => *** arity error ‘one’

one(two) => id(two) => two => a,bone(two) => id(two) => two => a,b

May 14, 2002 Macro Languages AoPL, S'02

CPP: Order of Expansion

• “Argument prescan”:

one(two) one(two)

#define id(X) X#define one(X) id(X)#define two a,b

#define id(X) X#define one(X) id(X)#define two a,b

CPP

May 14, 2002 Macro Languages AoPL, S'02

CPP: Order of Expansion

• “Argument prescan”:

one(two) => id(a,b)one(two) => id(a,b)

#define id(X) X#define one(X) id(X)#define two a,b

#define id(X) X#define one(X) id(X)#define two a,b

CPP

May 14, 2002 Macro Languages AoPL, S'02

CPP: Order of Expansion

• “Argument prescan”:

one(two) => id(a,b) => *** arity error ‘id’one(two) => id(a,b) => *** arity error ‘id’

#define id(X) X#define one(X) id(X)#define two a,b

#define id(X) X#define one(X) id(X)#define two a,b

CPP

May 14, 2002 Macro Languages AoPL, S'02

CPP: Order of Expansion

• “Argument prescan”:

• For piecing together new macro invocations• partly from the arguments, partly from the body:

one(two) => id(a,b) => *** arity error ‘id’one(two) => id(a,b) => *** arity error ‘id’

#define succ(X) ((X) + 1)#define call7(X) X(7)

call7(succ) => succ(7) => ((7) + 1)

#define succ(X) ((X) + 1)#define call7(X) X(7)

call7(succ) => succ(7) => ((7) + 1)

#define id(X) X#define one(X) id(X)#define two a,b

#define id(X) X#define one(X) id(X)#define two a,b

CPP

May 14, 2002 Macro Languages AoPL, S'02

Recursion

• Consider: #define x 1+x

x => ???

#define x 1+x

x => ???

May 14, 2002 Macro Languages AoPL, S'02

Recursion

• Consider:

• Definition-time (static intercept-and-reject):

#define x 1+x

x => ???

#define x 1+x

x => ???

#define x 1*x *** definition-time error!#define x 1*x *** definition-time error!

May 14, 2002 Macro Languages AoPL, S'02

Recursion

• Consider:

• Definition-time (static intercept-and-reject):

• Invocation-time (non-termination):

#define x 1+x

x => ???

#define x 1+x

x => ???

#define x 1*x *** definition-time error!#define x 1*x *** definition-time error!

x => 1+x => 1+1+x => … // loop at compile-time!x => 1+x => 1+1+x => … // loop at compile-time!

May 14, 2002 Macro Languages AoPL, S'02

CPP: Recursion

• “Dynamic intercept-and-ignore”:• Keep stack of macro invocations• Ignore invocations of already invoked macros:

x => 1+xx => 1+x

int x = 2;#define x 1+x

x => ???

int x = 2;#define x 1+x

x => ???

CPP

May 14, 2002 Macro Languages AoPL, S'02

CPP: Recursion

• “Dynamic intercept-and-ignore”:• Keep stack of macro invocations• Ignore invocations of already invoked macros:

x => 1+x // intercept-and-ignore: (at runtime x 3)x => 1+x // intercept-and-ignore: (at runtime x 3)

int x = 2;#define x 1+x

x => ???

int x = 2;#define x 1+x

x => ???

CPP

May 14, 2002 Macro Languages AoPL, S'02

M4

• M4 (Unix Macro Preprocessor):• Originally called “ratfor”:

– “The Rational Fortran Preprocessor”

• Not tailored for a particular language• universal preprocessor

• 30+ built-in constructions:• macro definition, arithmetic evaluation, string

operations, file and system interfacing, …

May 14, 2002 Macro Languages AoPL, S'02

M4 Example

• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”

• Quoting:– Expansion removes one layer of quotes

• Controls expansion-time (eager by default):

define(‘square’, ‘eval($1 * $1)’)define(‘square’, ‘eval($1 * $1)’)

May 14, 2002 Macro Languages AoPL, S'02

M4 Example

• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”

• Quoting:– Expansion removes one layer of quotes

• Controls expansion-time (eager by default):

define(‘square’, ‘eval($1 * $1)’)

square(3)

define(‘square’, ‘eval($1 * $1)’)

square(3)

May 14, 2002 Macro Languages AoPL, S'02

M4 Example

• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”

• Quoting:– Expansion removes one layer of quotes

• Controls expansion-time (eager by default):

define(‘square’, ‘eval($1 * $1)’)

square(3) => eval(3 * 3)

define(‘square’, ‘eval($1 * $1)’)

square(3) => eval(3 * 3)

May 14, 2002 Macro Languages AoPL, S'02

M4 Example

• Implicit arguments (named: $1, …, $9)• Excess arguments ignored• Missing arguments default to “”

• Quoting:– Expansion removes one layer of quotes

• Controls expansion-time (eager by default):

define(‘square’, ‘eval($1 * $1)’)

square(3) => eval(3 * 3) => 9

define(‘square’, ‘eval($1 * $1)’)

square(3) => eval(3 * 3) => 9

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1]

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1]

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

(x’0, …, x’n-1)(x’0, …, x’n-1)

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

(x’0, …, x’n-1)(x’0, …, x’n-1)

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)

– Implies:

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

(x’0, …, x’n-1)(x’0, …, x’n-1)

M M x // invokes M(M), not M(M(x))M M x // invokes M(M), not M(M(x))

May 14, 2002 Macro Languages AoPL, S'02

TEX

• Flexible invocation syntax:• Designed by macro programmer• Parsing ambiguities (chooses shortest invocation)

– Implies:

• Recursion permitted:• TC compile-time language to “break the recursion”

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

\def \vector #1[#2..#3]{ // header $({#1}_{#2},\ldots,{#1}_{#3})$ // body}

\vector x’[0..n-1] => $({x’}_{0},\ldots,{x’}_{n-1})$

(x’0, …, x’n-1)(x’0, …, x’n-1)

M M x // invokes M(M), not M(M(x))M M x // invokes M(M), not M(M(x))

May 14, 2002 Macro Languages AoPL, S'02

Syntax Macros

C++ templates, Scheme, JTS, MS2

May 14, 2002 Macro Languages AoPL, S'02

C++ Templates

• Intended as a genericity mechanism• But often used as a macro language

May 14, 2002 Macro Languages AoPL, S'02

C++ Templates

• Intended as a genericity mechanism• But often used as a macro language

• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration

May 14, 2002 Macro Languages AoPL, S'02

C++ Templates

• Intended as a genericity mechanism• But often used as a macro language

• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration

• Multiple definitions:template<int X, int Y> struct M { … }template<int X> struct M { … }

template<int X, int Y> struct M { … }template<int X> struct M { … }

May 14, 2002 Macro Languages AoPL, S'02

C++ Templates

• Intended as a genericity mechanism• But often used as a macro language

• Syntax types:• Arguments: id, const, all types (e.g. int)• The result is always a declaration

• Multiple definitions:

• Constant folding:

template<int X, int Y> struct M { … }template<int X> struct M { … }

template<int X, int Y> struct M { … }template<int X> struct M { … }

(1 + 2) 3 // at compile-time(1 + 2) 3 // at compile-time

May 14, 2002 Macro Languages AoPL, S'02

However…template<int X>struct ct_pow<X,0> { static const int res = 1;};

template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};

const int z = ct_pow<5,3>::res; // c-time z 125

template<int X>struct ct_pow<X,0> { static const int res = 1;};

template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};

const int z = ct_pow<5,3>::res; // c-time z 125

May 14, 2002 Macro Languages AoPL, S'02

However…

• Constant folding + multiple definition =Turing complete (at compile-time)!

template<int X>struct ct_pow<X,0> { static const int res = 1;};

template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};

const int z = ct_pow<5,3>::res; // c-time z 125

template<int X>struct ct_pow<X,0> { static const int res = 1;};

template<int X, int Y>struct ct_pow { static const int res = X * ct_pow<X,Y-1>::res;};

const int z = ct_pow<5,3>::res; // c-time z 125

May 14, 2002 Macro Languages AoPL, S'02

Scheme

• Convenient pattern matching

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

May 14, 2002 Macro Languages AoPL, S'02

Scheme

• Convenient pattern matching

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

May 14, 2002 Macro Languages AoPL, S'02

Scheme

• Convenient pattern matching• Multiple definitions:

– Selection: first match in order listed

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

May 14, 2002 Macro Languages AoPL, S'02

Scheme

• Convenient pattern matching• Multiple definitions:

– Selection: first match in order listed

• Ellipsis list constructor: “…”– More on this later…

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b …) (if b (and …) #f)))))))))))))))))) :)

(and a (if x y z) c) => (if ...)

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

)

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) ((gen-inc x) 5)

)

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) ((gen-inc x) 5) => ((lambda (x) (+ x x)) 5)

)

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10

)

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

• With -conversion:

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10

((gen-inc x) 5) ((gen-inc x) 5) )

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

• With -conversion:

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10

((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) ((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) )

May 14, 2002 Macro Languages AoPL, S'02

Hygienic Macro Expansion

• Scheme has automatic -conversion:• Identifier renaming to avoid name capture

• let ((x 3)) (• Without -conversion

• With -conversion:

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

(define-syntax gen-inc (syntax-rules () ((gen-inc i) (lambda (x) (+ x i))))

((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10((gen-inc x) 5) => ((lambda (x) (+ x x)) 5) => 10

((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) => 8((gen-inc x) 5) => ((lambda (x’) (+ x’ x)) 5) => 8 )

May 14, 2002 Macro Languages AoPL, S'02

Compile-Time Programming

• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

May 14, 2002 Macro Languages AoPL, S'02

Compile-Time Programming

• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):

• Non-termination possible:

(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))

(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

May 14, 2002 Macro Languages AoPL, S'02

Compile-Time Programming

• Basically compile-time functions on S-exps• Lazy (inv.-time) body expansion ( evaluation):

• Non-termination possible:

(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))

(ct-loop) => // c-time loop

(define-syntax ct-loop (syntax-rules () ((ct-loop) (ct-loop))))

(ct-loop) => // c-time loop

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

(define-syntax m (syntax-rules () ((m) (if 1 2 3 4))))

(m) => // *** parse error: too many arg’s to if

May 14, 2002 Macro Languages AoPL, S'02

JTS

• JTS (“Jakarta Tool Suite”):

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

May 14, 2002 Macro Languages AoPL, S'02

JTS

• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

May 14, 2002 Macro Languages AoPL, S'02

JTS

• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”

– Fixed invocation syntax:

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

#swap(C.x, y);#swap(C.x, y);

May 14, 2002 Macro Languages AoPL, S'02

JTS

• JTS (“Jakarta Tool Suite”):• Argument types: name, exp, stm, decl, class, type• Result types: “exp”, “stm”, “mth”, “cls”, “prg”

– Fixed invocation syntax: – Safe:

• Guaranteed termination, only generate legal syntax

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

macro swap(AST_QualifiedName x, AST_QualifiedName y) local temp // explicit -conversion stm{ // body constructor: stm integer temp = x; x = y; y = temp; }stm

#swap(C.x, y);#swap(C.x, y);

May 14, 2002 Macro Languages AoPL, S'02

MS2

• MS2 (“Meta Syntactic Macro System”):• Turing complete AST programming language

– for computing C parse trees at compile-time

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}

May 14, 2002 Macro Languages AoPL, S'02

MS2

• MS2 (“Meta Syntactic Macro System”):• Turing complete AST programming language

– for computing C parse trees at compile-time

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}myenum fruit { apple, orange }; print_fruit(apple);

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));}myenum fruit { apple, orange }; print_fruit(apple);

May 14, 2002 Macro Languages AoPL, S'02

• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }

Macro Survey

May 14, 2002 Macro Languages AoPL, S'02

• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }

• 31 Properties:• { Level of operation, body expansion, order of expansion, … }

Macro Survey

May 14, 2002 Macro Languages AoPL, S'02

• 8 Languages:• { CPP, M4, TEX, Dylan, C++ Templates, Scheme, JTS, MS2 }

• 31 Properties:• { Level of operation, body expansion, order of expansion, … }

Macro Survey

May 14, 2002 Macro Languages AoPL, S'02

<bigwig>

• Declarative:• Based entirely on simple concepts: grammars and

substitution

• Safe:• Guaranteed termination• Only generate legal (and -converted) syntax

• Flexible:• All (55) nonterminals may be arg and return types• Invocation syntax design (guides parsing)

May 14, 2002 Macro Languages AoPL, S'02

On all (55) Nonterminals

• <floatconst>:macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }

May 14, 2002 Macro Languages AoPL, S'02

On all (55) Nonterminals

• <floatconst>:

• <stm>:

macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }

macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>

}

macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>

}

May 14, 2002 Macro Languages AoPL, S'02

On all (55) Nonterminals

• <floatconst>:

• <stm>:

• <regexp>:

macro <floatconst> pi ::= { 3.1415926 }macro <floatconst> pi ::= { 3.1415926 }

macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>

}

macro <stm> maybe <stm S> ::= { if (random(2)==0) <S>

}

macro <regexp> plus ( <regexp R> ) ::= { concat(star(<R>),<R>)

}

macro <regexp> plus ( <regexp R> ) ::= { concat(star(<R>),<R>)

}

May 14, 2002 Macro Languages AoPL, S'02

Example: repeat

• Invocation syntax design:• Guides the parser

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

<S>

while (<E>) <S>

}

}

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

<S>

while (<E>) <S>

}

}

May 14, 2002 Macro Languages AoPL, S'02

Example: repeat

• Invocation syntax design:• Guides the parser

• Code duplication: <S>• Worst-case: O(2n)

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

<S>

while (<E>) <S>

}

}

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

<S>

while (<E>) <S>

}

}

May 14, 2002 Macro Languages AoPL, S'02

Example: repeat

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

bool first = true;

while (first || !<E>) {

<S>

first = false;

}

}

}

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

bool first = true;

while (first || !<E>) {

<S>

first = false;

}

}

}

May 14, 2002 Macro Languages AoPL, S'02

Example: repeat

-Conversion:• first => first_87

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

bool first = true;

while (first || !<E>) {

<S>

first = false;

}

}

}

macro <stm> repeat <exp E> until ( <stm S> ) ; ::= { {

bool first = true;

while (first || !<E>) {

<S>

first = false;

}

}

}

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitions

• Macros with same name: si/si-sinon:

• Parsing:• Which macro to select?

macro <stm> si (<exp E>) <stm S> ::= { if (<E>) <S>

}

macro <stm> si (<exp E>) <stm S> sinon <stm T> ::= { if (<E>) <S> else <T>

}

macro <stm> si (<exp E>) <stm S> ::= { if (<E>) <S>

}

macro <stm> si (<exp E>) <stm S> sinon <stm T> ::= { if (<E>) <S> else <T>

}

May 14, 2002 Macro Languages AoPL, S'02

Specificity Parsing

macro <exp> select <id I> from <exp E> where <exp E2>

macro <exp> select all from <exp E> where <exp E2>

macro <exp> select <id I> from <exp E> where <exp E2>

macro <exp> select all from <exp E> where <exp E2>

May 14, 2002 Macro Languages AoPL, S'02

Specificity Parsing

• Challenge rounds:– Select most specific productions (wrt. FIRST sets)

• Resolves many ambiguities• Independent of definition-order• Overloading• Avoids keywordification• Commit no branch explosion, no backtracking

macro <exp> select <id I> from <exp E> where <exp E2>

macro <exp> select all from <exp E> where <exp E2>

macro <exp> select <id I> from <exp E> where <exp E2>

macro <exp> select all from <exp E> where <exp E2>

May 14, 2002 Macro Languages AoPL, S'02

Pretty Print and Error Reporting

• Pretty Printing:• Terminal printers:

ASCII, LA EX, HTML(+/- expansion)

T

May 14, 2002 Macro Languages AoPL, S'02

Pretty Print and Error Reporting

• Pretty Printing:• Terminal printers:

ASCII, LA EX, HTML(+/- expansion)

• Error Reporting:• stdout, HTML*** symbol errors:*** test.wig:7: Identifier ‘inf’ not declared in macro argument ‘S’ in macro invocation ‘reader’ (test.wig:7) defined in [std.wigmac:44]

*** symbol errors:*** test.wig:7: Identifier ‘inf’ not declared in macro argument ‘S’ in macro invocation ‘reader’ (test.wig:7) defined in [std.wigmac:44]

T

May 14, 2002 Macro Languages AoPL, S'02

Concurrency Stack

May 14, 2002 Macro Languages AoPL, S'02

Representation

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

May 14, 2002 Macro Languages AoPL, S'02

Representation

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

May 14, 2002 Macro Languages AoPL, S'02

Representation

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

macro <ids> MXY ( <ids Is> ) ::= { X, <Is>, Y }

A, MXY(B, C), D A, X, B, C, Y, D

• “Weaving” yields transparency!

weave

May 14, 2002 Macro Languages AoPL, S'02

Metamorphic Syntax Macros

May 14, 2002 Macro Languages AoPL, S'02

Argument Structure

• Many variants of an abstraction?

enum { zero };

enum { zero, one };

enum { zero, one, two };

enum { zero };

enum { zero, one };

enum { zero, one, two };

May 14, 2002 Macro Languages AoPL, S'02

Argument Structure

• Many variants of an abstraction?

– Syntax:• Argument structure?

– Transformation:• Specification?

enum { zero };

enum { zero, one };

enum { zero, one, two };

enum { zero };

enum { zero, one };

enum { zero, one, two };

May 14, 2002 Macro Languages AoPL, S'02

Argument Structure

• Many variants of an abstraction?

– Syntax:• Argument structure?

– Transformation:• Specification?

enum { zero };

enum { zero, one };

enum { zero, one, two };

enum { zero };

enum { zero, one };

enum { zero, one, two };

Safety?

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitionsmacro <decls> enum { <id X> }; ::= { const int <X> = 0;

}

macro <decls> enum { <id X>, <id Y> }; ::= { const int <X> = 0;

const int <Y> = 1;

}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= { const int <X> = 0;

const int <Y> = 1;

const int <Z> = 2;

}

macro <decls> enum { <id X> }; ::= { const int <X> = 0;

}

macro <decls> enum { <id X>, <id Y> }; ::= { const int <X> = 0;

const int <Y> = 1;

}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= { const int <X> = 0;

const int <Y> = 1;

const int <Z> = 2;

}

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitionsmacro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitions

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

• Corresponds togrammar extension:

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitions

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

• Corresponds togrammar extension:

Three unrelated productions

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitions

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

• Corresponds togrammar extension:

• Problems:• Only fixed (finite) extent

Three unrelated productions

May 14, 2002 Macro Languages AoPL, S'02

Multiple Definitions

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

decls enum { id } ;

enum { id , id } ;

enum { id , id , id } ;

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

macro <decls> enum { <id X> }; ::= {…}

macro <decls> enum { <id X>, <id Y> }; ::= {…}

macro <decls> enum { <id X>, <id Y>, <id Z> }; ::= {…}

• Corresponds togrammar extension:

• Problems:• Only fixed (finite) extent• Highly redundant

Three unrelated productions

May 14, 2002 Macro Languages AoPL, S'02

Lists

• Scheme:• Special list constructor: “...”

decls ( enum id * )decls ( enum id * )

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b ...) (if b (and ...) #f))))

(define-syntax and (syntax-rules () ((and) #t) ((and b) b) ((and b ...) (if b (and ...) #f))))

(enum x y z)(enum x y z)

May 14, 2002 Macro Languages AoPL, S'02

e-BNF

• MS2:• Regular expressions:

– Optionals “?”, lists “+, *”, tuples “{…}”, t-sep. lists “\+t”

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));

syntax decl myenum[] {| $$id::name { $$+\,id::ids }; |} { return (list(`[enum $name $ids;], `[$(symbolconc(“print_”,name))(arg) { switch (arg) $(map((| @id id; `{ case $id: printf(“%s”, $(pstring(id)));}), ids))}));

decls enum { id , } ;decls enum { id

, } ;

myenum fruit { apple, orange }; print_fruit(apple);myenum fruit { apple, orange }; print_fruit(apple);

May 14, 2002 Macro Languages AoPL, S'02

Grammar

• Dylan (and JSE, a Java adaptation):• Describe argument syntax via

user-defined nonterminals

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Grammar

• Dylan (and JSE, a Java adaptation):• Describe argument syntax via

user-defined nonterminals

– Unsafe:• Transformations return lexical token sequences• Return user defined abstract syntax trees?

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Metamorphisms

• <bigwig>:• Attach host nonterminals to user def’d nonterminals• Specify morphing (into host syntax) inductively

metamorph <decls> enums;

macro <decls> … <enums: Ds> … ::= { … <Ds> … }

morph <enums> … ::= { … }

metamorph <decls> enums;

macro <decls> … <enums: Ds> … ::= { … <Ds> … }

morph <enums> … ::= { … }

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Example: enum

• Ideally:

enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;

const int a = 0;const int b = 1;const int c = 2;

=>

May 14, 2002 Macro Languages AoPL, S'02

Example: enum

• Ideally:

• But requires compile-time evaluation: 0, 1, 2, …

enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;

const int a = 0;const int b = 1;const int c = 2;

=>

May 14, 2002 Macro Languages AoPL, S'02

Example: enum

• Ideally:

• But requires compile-time evaluation: 0, 1, 2, …

• Instead generate:

enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 2;

const int a = 0;const int b = 1;const int c = 2;

=>

enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;

const int e = 0;const int a = e++;const int b = e++;const int c = e++;

=>

May 14, 2002 Macro Languages AoPL, S'02

Example: enummetamorph <decls> enums;metamorph <decls> enums;

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Example: enummetamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

metamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Example: enummetamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;

<Ds>

}

metamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;

<Ds>

}

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Example: enummetamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;

<Ds>

}

morph <enums> ::= { }

metamorph <decls> enums;

macro <decls> enum { <id X> <enums: Ds> } ; ::= { int e = 0;

const int <X> = e++;

<Ds>

}

morph <enums> , <id X> <enums: Ds> ::= { const int <X> = e++;

<Ds>

}

morph <enums> ::= { }

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– States: ( , ) (T N)* sentential form T* input string

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– States: ( , ) (T N)* sentential form T* input string

– Transitions:– (t , t ) (, )– (N , ) ( , ) , if N

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– Termination function:( , ) =

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– Termination function:( , ) = < | |, || || >

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– Termination function:( , ) = < | |, || || >

– || t || = 0

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– Termination function:( , ) = < | |, || || >

– || t || = 0

– || N0 || = k+1

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

N0 N1 1 N2 2 1 … Nk k … 1N0 N1 1 N2 2 1 … Nk k … 1

longest

May 14, 2002 Macro Languages AoPL, S'02

Guaranteed Termination

• Termination proof:• Transition system + termination function

– Termination function:( , ) = < | |, || || > lexicographically

ordered– || t || = 0

– || N0 || = k+1

0 1 2 …(0) > (1) > (2) > …

0 1 2 …(0) > (1) > (2) > …

N0 N1 1 N2 2 1 … Nk k … 1N0 N1 1 N2 2 1 … Nk k … 1

longest

May 14, 2002 Macro Languages AoPL, S'02

Metamorph Wellformedness

• Check at definition time:• No left-recursion

– guarantees parser termination:

xlist xlist X

xlist xlist X

xlist X xlist

xlist X xlist

May 14, 2002 Macro Languages AoPL, S'02

Metamorph Wellformedness

• Check at definition time:• No left-recursion

– guarantees parser termination:

• Derivability– metamorphisms must derive something finite:

xlist xlist X

xlist xlist X

xlist X xlist

xlist X xlist

xlist X xlist

xlist X xlist

xlist X xlistxlist X xlist

May 14, 2002 Macro Languages AoPL, S'02

Example: switchmetamorph <stm> swb;

metamorph <stm> swb;

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

May 14, 2002 Macro Languages AoPL, S'02

Example: switchmetamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}

metamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

May 14, 2002 Macro Languages AoPL, S'02

Example: switchmetamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}

metamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

May 14, 2002 Macro Languages AoPL, S'02

Example: switchmetamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}morph <swb> case <exp E> : <stms Ss> break; ::= { if (x == <E>) { <Ss> }}

metamorph <stm> swb;

macro <stm> switch ( <exp E> ) { <swb: S> } ::= { { var x = <E>; <S> }}morph <swb> case <exp E> : <stms Ss> break; <swb: S> ::= { if (x == <E>) { <Ss> } else <S>}morph <swb> case <exp E> : <stms Ss> break; ::= { if (x == <E>) { <Ss> }}

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

stm switch ( exp ) { swb }

swb case exp : stms break; swb

case exp : stms break;

May 14, 2002 Macro Languages AoPL, S'02

Example: reserve

=>

stm reserve ( res ) stm

res id res

stm reserve ( res ) stm

res id res

reserve ( a b c ) ...;reserve ( a b c ) ...;

acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);

acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);

May 14, 2002 Macro Languages AoPL, S'02

Example: reserve

• Requires non-local transformations• e.g. “b” must generate both “acquire(b)” and

“release(b)” at different locations

=>

stm reserve ( res ) stm

res id res

stm reserve ( res ) stm

res id res

reserve ( a b c ) ...;reserve ( a b c ) ...;

acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);

acquire(a); acquire(b); acquire(c); ...; release(c); release(b);release(a);

May 14, 2002 Macro Languages AoPL, S'02

Multiple Results

• Extend metamorphisms:• Attach host nonterminals to user def’d nonterminals

metamorph <stms,stms> res;

macro <stm> … <res: As,Rs> … ::= { … <As> … <Rs> … }

morph <res> … ::= { … } { … }

metamorph <stms,stms> res;

macro <stm> … <res: As,Rs> … ::= { … <As> … <Rs> … }

morph <res> … ::= { … } { … }

stm reserve ( res ) stm

res id res

stm reserve ( res ) stm

res id res

May 14, 2002 Macro Languages AoPL, S'02

Example: reserve

metamorph <stms,stms> res;

macro <stm> reserve ( <res: As,Rs> ) <stm S> ::= { { <As> <S> <Rs> }

}

morph <res> <id X> <res: As,Rs> ::= { acquire(<X>); <As>

} {

<Rs> release(<X>);

}

morph <res> ::= { } { }

metamorph <stms,stms> res;

macro <stm> reserve ( <res: As,Rs> ) <stm S> ::= { { <As> <S> <Rs> }

}

morph <res> <id X> <res: As,Rs> ::= { acquire(<X>); <As>

} {

<Rs> release(<X>);

}

morph <res> ::= { } { }

stm reserve ( res ) stm

res id res

stm reserve ( res ) stm

res id res

May 14, 2002 Macro Languages AoPL, S'02

Example: enum (cont’d)

enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;

const int e = 0;const int a = e++;const int b = e++;const int c = e++;

=>

May 14, 2002 Macro Languages AoPL, S'02

Example: enum (cont’d)

• Suppose no initialization expressions• Instead generate:

– CPS Style:• Send initialization expression to metamorphism

enum { a, b, c };enum { a, b, c };const int e = 0;const int a = e++;const int b = e++;const int c = e++;

const int e = 0;const int a = e++;const int b = e++;const int c = e++;

=>

enum { a, b, c };enum { a, b, c };const int a = 0;const int b = 1;const int c = 1+1;

const int a = 0;const int b = 1;const int c = 1+1;

=>

May 14, 2002 Macro Languages AoPL, S'02

Metamorph Argumentsmetamorph <decls> enums(<exp K>);

macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;

<Ds>

}

morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;

<Ds>

}

morph <enums> ::= { }

metamorph <decls> enums(<exp K>);

macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;

<Ds>

}

morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;

<Ds>

}

morph <enums> ::= { }

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

May 14, 2002 Macro Languages AoPL, S'02

Metamorph Argumentsmetamorph <decls> enums(<exp K>);

macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;

<Ds>

}

morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;

<Ds>

}

morph <enums> ::= { }

metamorph <decls> enums(<exp K>);

macro <decls> enum { <id X> <enums: Ds>({1}) }; … { const int <X> = 0;

<Ds>

}

morph <enums> , <id X> <enums: Ds>({<K>+1}) ::= { const int <X> = <K>;

<Ds>

}

morph <enums> ::= { }

decls enum { id enums } ;

enums , id enums

decls enum { id enums } ;

enums , id enums

enum {…x…}; => … const int x = 1+1+1+1; …enum {…x…}; => … const int x = 1+1+1+1; …

May 14, 2002 Macro Languages AoPL, S'02

Metamorph Advantages

• Flexibility:• Tree structures• Non-local transformations (multiple results)

• Safety:• No parse errors as a conseq. of macro expansion• Guaranteed termination

• Simplicity: • Based entirely on declarative concepts:

grammars and substitution

May 14, 2002 Macro Languages AoPL, S'02

vDSL:very Domain-Specific Languagestudies

course Math101

title “Mathematics 101”

2 point fall term

exclusions

Math101 <> MathA

Math102 <> MathB

prerequisites

Math101, Math102 < Math201, Math202, Math203

Math101, CS101 < CS202

studies

course Math101

title “Mathematics 101”

2 point fall term

exclusions

Math101 <> MathA

Math102 <> MathB

prerequisites

Math101, Math102 < Math201, Math202, Math203

Math101, CS101 < CS202

May 14, 2002 Macro Languages AoPL, S'02

FIN

May 14, 2002 Macro Languages AoPL, S'02

Next Week: metafront

• Macros are just a special case usage:• A is an extension of B: m: L+ => L

• Make sure only need to write delta: = L+ \ L

metafront

x: A => B

A B

program.a program.b

transformation

input language

input program(s) output program(s)

output language