Tree Oriented Programming Jeroen Fokker. Tree oriented programming nMany problems are like: Input...

Tree Oriented Programming

Jeroen Fokker

Tree oriented programming

Many problems are like:

Inputtext

Outputtexttransform unparseprocess


Many problems are like:

Inputtext

Outputtexttransform unparseparse prettyprint

internal tree representation

Tree oriented programming tools

should facilitate: Defining trees Parsing Transforming Prettyprinting

Mainstream approach totree oriented programming

Defining

trees Parsing Transforming Prettyprinting

libraryclever hackingpreprocessorOO programming language

http://abyss.hubbe.net/gfx/covers/bks/lg/c++.jpg

Our approach totree oriented programming

Defining

trees Parsing Transforming Prettyprinting

library

librarypreprocessor

functional languageHaskell

This morning’s programme

A crash course inFunctional programming using Haskell

Defining trees in Haskell The parsing library Transforming trees

using the UU Attribute Grammar Compiler

Prettyprinting Epilogue: Research opportunities

Language evolution:Imperative & Functional

50 years ago

Now

Haskell

Part I

A crash course inFunctional programming

using Haskell

Function definition

static int fac (int n){ int count, res; res = 1; for (count=1; count<=n; count++) res *= count; return res;}

fac n = product [1..n] Haskell

fac :: Int Int

Definition forms

Function

Constant

Operator

fac :: Int Intfac n = product [1..n]

pi :: Floatpi = 3.1415926535

( !^! ) :: Int Int Intn !^! k = fac n / (fac k * fac (n-k))

Case distinction with guards

abs :: Int Int

abs x = x= -x

| x>=0| x<0

“guards”

Case distinction with patterns

day :: Int Stringday 1 = “Monday”day 2 = “Tuesday”day 3 = “Wednesday”day 4 = “Thursday”day 5 = “Friday”day 6 = “Saturday”day 7 = “Sunday”

constantas formal

parameter!

Iteration

fac :: Int Int

fac n = 1=

| n==0| n>0

recursion

n * fac (n-1)

without using standard function

product

List:a built-in data structure

List: 0 or more values of the same type

“empty list” constant

“put in front” operator

[ ]

:

Shorthand notation for lists

enumeration

range

[ 1, 3, 8, 2, 5]

[ 4 .. 9 ]

> 1 : [2, 3, 4][1, 2, 3, 4]

> 1 : [4..6][1, 4, 5, 6]

Functions on lists

sum :: [Int] Intsum [ ] = 0sum (x:xs) = x + sum xs

length :: [Int] Intlength [ ] = 0length (x:xs)= 1 + length xs

patterns recursion

Standard library of functions on lists

null

++

take

> null [ ]True

> [1,2] ++ [3,4,5][1, 2, 3, 4, 5]

> take 3 [2..10][2, 3, 4]

challenge:Define these functions, using pattern matching and recursion

Functions on lists

null [ ] = Truenull (x:xs) = False

[ ] ++ ys = ys(x:xs) ++ ys = x : (xs++ys)

take 0 xs = [ ]take n [ ] = [ ]take n (x:xs) = x : take (n-1) xs

null :: [a] Bool

(++) :: [a] [a] [a]

take :: Int [a] [a]

Polymorphic type

Type involving type variables

take :: Int [a] [a]

Why did it take10 years and5 versionsto put this in Java?

Functions as parameter

Apply a function to allelements of a list

map

> map fac [1, 2, 3, 4, 5][1, 2, 6, 24, 120]

> map sqrt [1.0, 2.0, 3.0, 4.0][1.0, 1.41421, 1.73205, 2.0]

> map even [1 .. 6][False, True, False, True, False, True]

Challenge

What is the type of map ?

What is the definition of map ?

map ::

[a] (ab) [b]

map f [ ] =map f (x:xs)=

[ ]map f xs

f x

:

Another list function: filter

Selects list elements thatfulfill a given predicate

filter :: (aBool) [a] [a]filter p [ ] =filter p (x:xs) =

[ ]x : filter p xs| p x

| True = filter p xs

> filter even [1 .. 10][2, 4, 6, 8, 10]

Higher order functions:repetitive pattern? Parameterize!

product :: [Int] Int product [ ] =product (x:xs)

=

1product xsx *

and :: [Bool] Bool and [ ] =and (x:xs)=

Trueand xsx

&&sum :: [Int] Int sum [ ] =sum (x:xs)

=

0sum xsx +

Universal list traversal: foldr

foldr :: [a] a

foldr (#) e [ ] =foldr (#) e (x:xs)=

efoldr (#) e xsx #

(aaa) a

combining function start value

foldr :: (abb) b [a] b

Partial parameterization

foldr is a generalizationof sum, product, and and ....

…thus sum, product, and andare special cases of foldr

product = foldr (*) 1and = foldr (&&) Truesum = foldr (+) 0or = foldr (||) False

Example: sorting (1/2)

insert :: a [a] [a]insert e [ ] = [ e ]insert e (x:xs)

| e x = e : x : xs| e x = x : insert e xs

Ord a

isort :: [a] [a]isort [ ] = [ ]isort (x:xs) = insert x (isort xs)

Ord a

isort = foldr insert [ ]

Example: sorting (2/2)

qsort :: [a] [a] [a]qsort [ ] = [ ]qsort (x:xs) = qsort (filter (<x) xs) ++ [x] ++ qsort (filter (x) xs)

Ord a

(Why don’t they teach itlike that in thealgorithms course?)

Infinite lists

repeat :: a [a]repeat x = x : repeat x

> repeat 3[3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3

replicate :: Int a [a]replicate n x = take n (repeat x)

> concat (replicate 5 ”IPA ” )”IPA IPA IPA IPA IPA ”

Lazy evaluation

Parameter evaluation is postponeduntil they are really needed

Also for the (:) operatorso only the part of the listthat is needed is evaluated

Generic iteration

iterate :: (aa) a [a]iterate f x = x : iterate f (f x)

> iterate (+1) 3[3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20

Convenient notations(borrowed from mathematics)

Lambda abstraction

List comprehension

\x x*x

[ x*y | x [1..10] , even x , y [1..x] ]

for creating anonymous functionsmore intuitive thanequivalent expression using map , filter & concat

Part II

Defining treesin Haskell

Binary trees

4 23

15 29103

1 6 11

5 8

18 26 34

14with internallabels

How would you do thisin Java/C++/C# etc?

The OO approach to trees

class Tree{ private Tree left, right;

private int value;

// constructorpublic Tree(Tree al, Tree ar, int av){ left = al; right=ar; value=av; }

// leafs are represented as null}

The OO approach to trees:binary trees with external labels

class Tree {// empty superclass

}class Leaf extends Tree {

int value}class Node extends Tree {

Tree left,right}

Functional approach to trees

I need a polymorphic type and constructor functions

Leaf :: a Tree aNode :: Tree a Tree a Tree a

Tree a

data Tree a= Leaf a | Node (Tree a) (Tree a)

Haskell notation:

Functions on trees

In analogy to functions on lists

we can define functions on trees

length :: [a] Intlength [ ] = 0length (x:xs)= 1 + length xs

size :: Tree a Intsize (Leaf v) = 1size (Node lef rit) = size lef + size rit

Challenge: write tree functions

elem tests element occurrence in tree

front collects all values in a list

elem :: a Tree a Boolelem x (Leaf y) = x==yelem x (Node lef rit) = elem x lef || elem x rit

front :: Tree a [a]front (Leaf y) = [ y ]front (Node lef rit) = front lef ++ front rit

Eq a

A generic tree traversal

In analogy to foldr on lists

we can define foldT on trees

foldr ::

[a] b

(abb) -- for (:)

b -- for [ ]

foldT ::

Tree a b

(ab) -- for Leaf (bbb) -- for Node

Challenge: rewrite elem and front using foldT

foldT ::

Tree a b


elem x (Leaf y) = x==yelem x (Node lef rit) = elem x lef || elem x rit

front (Leaf y) = [ y ]front (Node lef rit) = front lef ++ front rit

elem x = foldT (==x) (||)

front = foldT (\y[y]) (++)front = foldT ( :[] ) (++)

Part III

A HaskellParsing library

Approaches to parsing

Mainstream approach (imperative) Special notation for grammars Preprocessor translates grammar to C/Java/…

Our approach (functional) Library of grammar-manipulating functions

-YACC (Yet Another Compiler Compiler) -ANTLR (ANother Tool for Language Recognition)

ANTLR generates Javafrom grammarExpr : Term

( PLUS Term | MINUS Term ) *

;Term : NUMBER

| OPEN Expr CLOSE ;

public void expr (){ term (); loop1: while (true) { switch(sym) { case PLUS: match(PLUS); term (); break; case MINUS: match(MINUS); term (); break; default: break loop1; } }}public void term(){ switch(sym) { case INT: match(NUMBER); break; case LPAREN: match(OPEN); expr (); match(CLOSE); break; default: throw new ParseError(); }}

ANTLR: adding semantics

Expr

: Term ( PLUS Term| MINUS Term) *

;Term: NUMBER| OPEN Expr CLOSE ;

returns [int x=0]{ int y; }

returns [int x=0]

x= y

=y=

x=

{ x += y; }{ x –= y; }

n: { x = str2int(n.getText(); }

{ $$ += $1; }

Yacc notation:

A Haskell parsing library

Building blocks

Combinators

type Parser

symbol :: a Parser satisfy :: (aBool) Parser

() :: Parser Parser Parser() :: Parser Parser Parser

epsilon :: Parser

A Haskell parsing library

Building blocks

Combinators

symbol :: a Parser satisfy :: (aBool) Parser

() :: Parser Parser Parser() :: Parser Parser Parser

type Parser a b

symbol :: a Parser a asatisfy :: (aBool) Parser a a

() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)(®) :: (bc) Parser a b Parser a c

start :: Parser a b [a] b

epsilon :: Parserepsilon :: Parser a ()

Domainspecific Combinator Language vs. Library

New notation and semantics

Preprocessing phase

What you gotis all you get

Familiar syntax, just new functions

‘Link & go’ Extensible at will

using existing function abstraction mechnism

Expression parseropen = symbol ‘(’close = symbol ‘)’plus = symbol ‘+’minus = symbol ‘–’

expr = term (plusminus) expr term

term = number open expr close

data Tree = Leaf Int | Node Tree Op Treetype Op = Char

Leaf

Node

middle

expr, term :: Parser Char Tree

where middle (x,(y,z)) = y

Example of extensibility

Shorthand

Parameterized shorthand

New combinators

open = symbol ‘(’close = symbol ‘)’

pack :: Parser a b Parser a bpack p = open p closemiddle

many :: Parser a b Parser a [b]

The real type of ()

() :: Parser a b Parser a b Parser a b() :: Parser a b Parser a c Parser a (b,c)(®) :: (bc) Parser a b Parser a c

How to combine b

and c ?

() :: Parser a b Parser a c (bcd) Parser a d

() :: Parser a (cd) Parser a c Parser a d

pack p = open p closemiddle where middle x y z = y

Another parser example;design of a new combinator

many :: Parser a b Parser a [b]

many p = p many p epsilon

(\b bs b:bs)

(\e [ ])

many p = p many p succeed [ ]

(:)

Challenge:parser combinator design

many :: Parser a b Parser a [b]many1 :: Parser a b Parser a [b]sequence :: [ Parser a b ] Parser a [b]

EBNF * EBNF + Beyond EBNF

many1 p = p many p

(:)

sequence [ ] =sequence (p:ps) =

succeed [ ](:) p sequence

pssequence = foldr

f (succeed [])where f p r = (:) p r

More parser combinators

sequence :: [ Parser a b ] Parser a [b]choice :: [ Parser a b ] Parser a [b]listOf :: Parser a b Parser a s Parser a [b]

choice = foldr () fail

listOf p s = p

many ( s p )

separator

(\s b b)

(:)

chain :: Parser a b Parser a (bbb) Parser a b

Parser for Expressions(with precedence)

expr = chain term (symbol ‘+’)

term = chain fact (symbol ‘*’)

fact = number pack expr

(\o(:+:))

(\o(:*:))

Con

( (\o(:–:)) (symbol ‘–’))( (\o(:/:)) (symbol ‘/’))

Var name Fun name pack (listOf expr

(symbol ‘,’) )

A programmers’ reflex:Generalize!

expr = chain term ( … (:+:)…‘+’ … … (:–:)…‘–’ …)

term = chain fact ( … (:*:)…‘*’ … … (:/:)…‘/’ …)

fact = basicCases pack expr

gen ops next= chain next ( choice …ops…

)

Expression parser(many precedence levels)

expr = gen ops1 term1term1= gen ops2 term2term2= gen ops3 term3term3= gen ops4 term4term4= gen ops5 factfact = basicCases

pack expr

expr = foldr gen fact [ops5,ops4,ops3,ops2,ops1]

gen ops next= chain next ( choice …ops…

)

expr = gen ops1 term1term1= gen ops2 term2term2= gen ops3 term3term3= gen ops4 term4term4= gen ops5 fact

Library implementation

type Parser = String X

type Parser b = String b

polymorphicresult type

type Parser b = String (b, String) rest string

type Parser a b = [a] (b, [a]) polymorphic

alfabet

type Parser a b = [a] [ (b, [a]) ] list of successesfor ambiguity

Library implementation

() :: Parser a b Parser a b Parser a b

(p q) xs = p xs ++ q xs

() :: Parser a (cd) Parser a c Parser a d

(p q) xs =p xsq ys

(f,ys) [|,]

(c,zs)

( , )f c zs

() :: (bc) Parser a b Parser a c(f p) xs =

p xs(b,ys) [|]

( f b , ys )

Part IV

Techniques forTransforming trees

Data structure traversal

In analogy to foldr on lists

we can define foldT on binary trees

foldr ::

[a] b

(abb) -- for (:)

b -- for [ ]

foldT ::

Tree a b


Traversal of Expressions

data Expr= Add Expr Expr| Mul Expr Expr| Con Int

foldE ::

Expr b

(bbb) -- for Add

(bbb) -- for Mul(Int b) -- for Con

type ESem b = ( b b b , b b b , Int b )

Traversal of Expressions


foldE :: ESem b Expr b


foldE (a,m,c) = f where f (Add e1 e2) = a (f e1) (f e2) f (Mul e1 e2) = m(f e1) (f e2) f (Con n) = c n

Using and defining Semantics



evalExpr :: Expr IntevalExpr = foldE evalSem

evalSem :: ESem IntevalSem = ( (+) , (*) , id )

Syntax and Semantics

“ 3 + 4 * 5 ”

Add (Con 3) (Mul (Con 4) (Con 5))

parseExpr

evalExpr

23

= start p where p = ………

= foldE s where s = (…,…,…,…)

Multiple Semantics

“ 3 + 4 * 5 ”


23

evalExpr compileExpr

Push 3Push 4Push 5Apply (*)Apply (+)

runCode

parseExpr

:: String

:: Expr

:: Int :: Code

= foldE s where s = (…,…,…,…) s::ESem Int

= foldE s where s = (…,…,…,…) s::ESem Code

A virtual machine

What is “machine code” ?

What is an “instruction” ?

type Code = [ Instr ]

data Instr = Push Int | Apply (IntIntInt)

Compiler generates Code



evalExpr :: Expr IntevalExpr = foldE evalSem where evalSem :: ESem Int evalSem = ( (+) , (*) , id )

compExpr :: Expr CodecompExpr = foldE compSem where compSem :: ESem Code compSem = ( add , mul , con )

mul :: Code Code Codemul c1 c2 = c1 ++ c2 ++ [Apply (*)]con n = [ Push n ]

Compiler correctness

evalExpr

“ 3 + 4 * 5 ”


23

compileExpr

Push 3Push 4Push 5Apply (*)Apply (+)

runCode

parseExpr

runCode (compileExpr e)=

evalExpr e

runCode:virtual machine specification

run :: Code Stack Stackrun [ ] stack = stackrun (instr:rest) stack = exec instr stackrun rest ( )

exec :: Instr Stack Stackexec (Push x) stack = x : stackexec (Apply f) (x:y:stack) = f x y : stack

runCode :: Code IntrunCode prog = run prog [ ]hd ( )

Extending the example:variables and local def’s



data Expr= Add Expr Expr| Mul Expr Expr| Con Int| Var String| Def String Expr Expr

type ESem b = ( b b b , b b b , Int b , String b , String bbb )

evalExpr :: Expr IntevalExpr = foldE evalSem where evalSem :: ESem Int evalSem = ( add , mul , con ), var, def )

Any semanticsfor Expressionadd :: b b b add x y =

mul :: b b b mul x y =

con :: Int b con n =

var :: String b var x =

def :: String b b b def x d b =

Evaluation semanticsfor Expressionadd :: b b b add x y =





Int Int (EnvInt)Int

Int Int (EnvInt)Int

Int

Int

Int IntInt

x + y

x * y

n

Evaluation semanticsfor Expressionadd :: b b b add x y =





Int Int (EnvInt)Int

Int Int (EnvInt)Int

Int

Int

Int IntInt

x + y

x * y

n

lookup e x\e (EnvInt)

(EnvInt)

(EnvInt)

(EnvInt)

(EnvInt

Evaluation semantics for Expressionadd :: b b b add x y =





Int Int (EnvInt)Int

Int Int (EnvInt)Int

Int

Int

IntInt

x + y

x * y

n

lookup e x\e

Int

(EnvInt)

(EnvInt)

(EnvInt)

(EnvInt)

(EnvInt)

\e

\e

\e

\e

(EnvInt)(EnvInt)

(EnvInt) (EnvInt)

(EnvInt)(EnvInt)

e e

e e

b e((x,d e) : )

Extending the virtual machine

What is “machine code” ?

What is an “instruction” ?

type Code = [ Instr ]

data Instr = Push Int | Apply (IntIntInt)

data Instr = Push Int | Apply (IntIntInt) | Load Adress | Store Adress

Compilation semantics for Expressionadd :: b b b add x y =





\e

\e

\e

\e

\e

(EnvCode) (EnvCode) Env Code

Env Code

Env Code



x e ++ y e ++ [Apply (+)]

x e ++ y e ++ [Apply (*)]

[Push n]

[Load (lookup e x)]

d e++ b e((x,a) : )[Store a]++

where a = length e

Language: syntax and semantics


type ESem b = ( b b b , b b b , Int b , String b , String bb b )

compSem :: ESemcompSem = (f1, f2, f3, f4, f5) where ……

(EnvCode)

compile t = foldE compSem t [ ]

Language: syntax and semantics


type ESem b = ( b b b , b b b , Int b , String b , String bb b )

compSem :: ESemcompSem = (f1, f2, f3, f4, f5) where ……

(EnvCode)

compile t = foldE compSem t [ ]

data Expr= Add Expr Expr| Mul Expr Expr| Con Int| Var Stringdata Stat= Assign String Expr| While Expr Stat| If Expr Stat Stat | Block [Stat]

type ESem b c = ( ( b b b , b b b , Int b , String b ) , ( String b c , b c c , b c c c , [ c ] c ) )

Code(EnvCode)compSem = ((f1, f2, f3, f4), (f5, f6, f7, f8)) ……

Real-size example

data Module = ……data Class = ……data Method = ……data Stat = ……data Expr = ……data Decl = ……data Type = ……

type ESem a b c d e f = ( (…,…,…) , (…,...) , (…,…,…,…,…,…) , …

compSem :: ESem

compSem = (…dozens of functions…) ……

(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)(…… ……)

Attributesthat are passed

top-down

Attributesthat are generated

bottom-up

Tree semantics

data Expr= Add Expr Expr| Var String| …

codeSem =( \ a b \ e a e ++ b e ++ [Apply (+)], \ x \ e [Load (lookup e x)], ……

ATTR Expr inh e: Env syn c: Code

Explicit names for fields and attributes

DATA Expr= Add a: Expr b: Expr| Var x: String| …

generated by Attribute Grammar

SEM Expr| Add this.code = a.code ++ b.code ++ [Apply (+)]

a.e = this.eb.e = this.e

| Var this.code = [Load (lookup e x)]

Attribute value equationsinstead of functions

UU-AGCAttribute Grammar Compiler

Preprocessor to Haskell Takes:

Attribute grammar Attribute value definitions

Generates: datatype, fold function and Sem type Semantic function (many-tuple of

functions) Automatically inserts trival def’s

a.e = this.e

UU-AGCAttribute Grammar Compiler

Advantages: Very intuitive view on trees

no need to handle 27-tuples of functions

Still full Haskell power in attribute def’s Attribute def’s can be arranged modularly No need to write trivial attribute def’s

Disadvantages: Separate preprocessing phase

Part IV

Pretty printing


Inputtext

Outputtexttransform prettyprintparse

internal tree representation

SEM Stat| Assign this.code = …| While this.code = …| Block this.code = …

Prettyprinting is justanother tree transformation

Example:transformation from Stat to String

SEM Stat| Assign this.code = x.code ++ “=” ++ e.code ++ “;”| While this.code = “while (” ++ e.code ++ “)”++ s.code| Block this.code = “{” ++ body.code ++ “}”

ATTR Expr Stat [Stat] syn code: String

DATA Stat= Assign a: Expr b: Expr| While e: Expr s: Stat| Block body: [Stat]

SEM Stat| While s.indent = this.indent + 4

inh indent: Int

But howto handlenewlines &indentation?

A combinator libraryfor prettyprinting

Type Building block Combinators

Observer

type PPDoc

text :: String PPDoc

(>|<) :: PPDoc PPDoc PPDoc(>–<) :: PPDoc PPDoc PPDocindent :: Int PPDoc PPDoc

render :: Int PPDoc String

Epilogue

Research opportunities

Research opportunities (1/4)

Parsing library: API-compatible to naïve library, but With error-recovery etc. Optimized

Implemented using the “Attribute Grammar” way of thinking


UU - Attribute Grammar Compiler More automatical insertions Pass analysis optimisation


A real large compiler (for Haskell) 6 intermediate datatypes 5 transformations + many more

Learn about software engineering aspectsof our methodology

Reasearch opportunities (4/4)

Generate as much aspossible with preprocessors Attribute Grammar Compiler Shuffle

extract multiple views & docsfrom the same source

Rulergenerate proof ruleschecked & executable

.rul

.cag

.ag

.hs

.o

.exe

Tree Oriented Programming Jeroen Fokker. Tree oriented programming nMany problems are like: Input...

Documents

Transcript of Tree Oriented Programming Jeroen Fokker. Tree oriented programming nMany problems are like: Input...