Realizability: Extracting Programs from proofs

Types Summer School2007 Bertinoro (Italia)

4-hours Course by: Stefano Berardi Semantic of Computation Research groupComputer Science Dept. Turin Universityhttp://www.di.unito.it/~stefano

1

http://www.di.unito.it/~stefano

(look for the word “Bertinoro”)

The text of this short course on Realizability may be found in the home page of Stefano Berardi:

2

Plan of the course

Lesson 1. The Realization Interpretation. • 1.1. The Realization Interpretation. 1.2.

From Proofs to Realizers. 1.3. The Harrop Types.

Lesson 2. Program Extraction and Useless Code.

• 2.1. An example of Program Extraction. 2.2. Program Extraction and Useless code. 2.3.Useless Code Analysis.

3

Reference Text

• L. Boerio “Optimizing Programs Extracted from Proofs”. Ph. D. Thesis, C. S. Dept. Turin University, 1997.

• Available in the web page of the course:

http://www.di.unito.it/~stefano

• (look for the words: “Boerio Ph.d.”)

4

Lesson 1The Realization interpretation

5

Plan of Lesson 1

• § 1.1 The Realization Interpretation.

• § 1.2 From Proofs to Realizers.

• § 1.3 The Harrop Types.

6

§ 1.1 The Realization Interpretation• In Intuitionistic Logic, any proof D of a closed

formula of Heyting Arithmetic HA may be interpreted by some effective operation r associated to (Heyting’s Interpretation).

• We will call such an r a Realizer of . • In the simplest case, r is the proof D itself,

executed through normalization.• Yet, in order to effectively use Heyting

Interpretation, is is convenient to think of r as a separate object, a term of a typed -calculus T.

• We will define Heyting’s interpretation of HA in the typed -calculus T.

7

§ 1.1 The First Order system HA• We formalize HA as the first order multi-sorted

language for Integers and Lists of integers, with induction over Integers and over Lists.

• HA has some fixed set of recursive functions, of predicates, and of rules having atomic premises and an atomic conclusion.

• x denotes any sequences of variables of HA, each labeled with its type (Integer or List).

• The quantifiers of HA are: yT.[x,y], yT.[x,y], with T = Integers, Lists.

8

§ 1.1 Extending the language of HA

• All what we will say applies when we have in Heyting Arithmetic not just the types

T=Integers, List

• but also the types

T=any Bohm-Berarducci Data Type

of cardinality > 1

• Look at Boerio Ph.d in the Web page of the docent if you want to learn more.

9

§ 1.1 The -calculus T• We will define the realizers for HA in the

simply typed -calculus T. • The atomic types of T are the Data TypesUnit={unit}, Bool={True,False}, N={0,1,2,3,..},List={nil, cons(n,nil), cons(n,cons(m,nil)),…} • T has product types and 1, 2, <_,>, arrow

types and, , appl. T includes if, and primitive recursion recN,T, recL,T over integers and over lists.

• We abbreviate AB…C by A,B,…C. We abbreviate 1(a), 2(a) by a.1, a.2.

10

§ 1.1 Dummy constants.• For each simple type T of T, we need some

dummy element dummyT:T, the default value for such a type.

• We define dummyT:T by induction over T.1. dummyUnit = unit2. dummyBool = False3. dummyN = 04. dummyList = nil5. dummyTU = x. dummyU

6. dummyTU = <dummyT, dummyU>

11

§ 1.1 The Realization relation

• For any formula of HA and any -term r of T, we will define the Realization relation “r is a realizer of ”, abbreviated “r:”.

• If x1,…,xn are the free variables of (all integers or lists), then the realizer r may depend on the same variables.

• We distinguish, in the definition of r: for any formula , one case for each possible first symbol of :

P (any predicate letter ), , , , , 12

§ 1.1 Realizing Atomic Formulas.

• If = P(t1,…,tm), we define r:P(t1,…,tm) r=unit, and P(t1,…,tm) is true in the standard interpretation.

• Note. We chose r=unit because a proof of an atomic formula corresponds to a vacuous operation, therefore to a dummy value.

• The type Unit={unit} is the type of vacuous operations.

13

§ 1.1 Realizing Propositional Connectives

• r:12 r.1:1 and r.2:2

• r:12 r.1Bool

r.1=True r.2.1:1

r.1=False r.2.2:2 if r.1 =True, then the canonical choice for r.2.2 is a dummy constant, and conversely.

• r:12 for all s:1, r(s):2 14

§ 1.1 Realizing Quantifiers.

• r:yT.[y] for all yT,

r(y):[y]

• r:yT.[y] r.1T and r.2:[r.1]

15

§ 1.1 Realization and programs • The canonical choice for a realizer r : yT.(f(x,y)=0)

is some pair r[x]=<y[x],unit>, of a function y=y[x]:T, solving the equation f(x,y)=0 (w.r.t. the parameters in x), and some dummy realizer unit of f(x,y)=0.

• yT.(f(x,y)=0) says “there is a solution to f(x,y)=0, parametric in x”, while r finds it.

• r:yT.(f(x,y)=0) may be seen as a program whose specification is yT.(f(x,y)=0).

• We will define a Realization interpretation, turning any intuitionistic proof in HA of solvability of some equation, into a program which effectively solves such equation.

16

§ 1.1 Realization Interpretation of Formulas.

• Let be any formula of HA. Then there is some simple type || of T such that for all r: we have r of type ||.

• The definition of || is by induction over .1. |P(t1,…,tm)| = Unit for any predicate P2. |12| = |1| |2|3. |12| = Bool ( |1| |2|)4. |12| = |1| |2|5. |xT.| = T ||6. |xT.| = T ||

17

§ 1.1 Realization Interpretation of Proofs.

• Let ={1,…,n} be a set of formulas of HA with free variables in x=x1,…,xk. We may turn every natural deduction proof D: of HA, with free assumptions in , into some realizer r:, depending on free variables in x, and on some realizer variables 1:|1|, …, k:|k|.

• The definition is in the next slides, by induction on the proof D. There is one clause for each possible rule at the conclusion of D.

• For simplicity we write r for r[x1,…,xn].

18

§ 1.2 From Proofs to Realizers

• Assumptions: i : i

• Atomic rules. Assume D ends by some rule of HA with atomic premises and conclusion.

… …unit: P1(t1) … unit: Pm(tm)

----------------------------unit: P(t)

• If |- unit: P1(t1), …, |- unit: Pm(t1), then |- unit: P(t) 19

§ 1.2 Introduction of • Rules for • Introduction rules:

s1: s2: -------------------

<s1,s2>:

• If |- s1: and |- s2: then |- <s1,s2>:

20

§ 1.2 Elimination of • Elimination rules:

s: s: ---------- ----------

1(s): 2(s):

• If |- s: , then |- 1(s): and |- 2(s):

21

§ 1.2 Introduction of • Rules for . Let T=True, F=False, and _, _’,

be dummy elements of ||, ||.• Introduction rules:

r: s: ------------------------ ------------------------

<T,<r,_’>>: <F,<_,s>>:

• If |- r: then |- <T,<r,_’>>: • If |- s: then |- <F,<_,s>>:

22

§ 1.2 Elimination of • Elimination rules for . Let i=r.1 and a = r.2.1

and b = r.2.2. Assume if i=True then a:, and if i=False then b:. Set:

u = if (i=True) then s[a] else t[b]Then

: : … …

r: s[]: t[]:

---------------------------------------------- u:

• If |- r: and , :|- and , :|-, then |- u:

\ \

23

§ 1.2 Introduction of

• Rules for . Introduction rule:

:…

s[]: --------------------

.s[]:

• If , : |- s[]: , then |- .s[]:

\

24

§ 1.2 Elimination of

• Elimination rule:

r: s:-------------------------

r(s):

• If |-r: and |-s:, then |-r(s):.

25

§ 1.2 Introduction of

• Rules for : Introduction rule. Let t be any term of the language of HA.

…

r: [t]-------------------

<t,r>: xT.[x]

• If |- r:[t] for some t, then |- <t,r>: x T.[x]

26

§ 1.2 Elimination of

• Rules for : Elimination rule. Assume that i=r.1 and a=r.2 and a: [i]. Then:

, :[x] … …

r: xT.[x] t[x,]: -----------------------------------

t[i,a]:

• Provided xFV(,).• If |-r:xT.[x], ,:|-t[x,]:, and

xFV(,), then |- t[i,a]:

\

27

§ 1.2 Introduction of • Rules for : Introduction rule.

…

r[x]:[x]-----------------------

x.r[x]:xT.[x]

• Provided xFV()

• If |-r:[x] and xFV(), then |- x.r[x]:xT.[x]

28

§ 1.2 Elimination of

• Rules for : Elimination rule. Let t be any term of the language of HA.

…

f:xT.[x]---------------

f(t):[t]

• If |- f:xT.[x], then |-f(t):[t] for all t

29

§ 1.2 Induction axiom for N

Induction Axiom for the type N=Integers.

• Let T=||. We define a realizer

RecN,T: xN.([0]yN.([y][y+1])[x])

• Rec has type: N,T,(N,TT)T

• Let n:N and r:T and s:N,TT.

• We inductive define RecN,T(n,r,s):T by:

1. RecN,T(0,r,s) = r

2. RecN,T(n+1,r,s) = s(n,RecN,T(n,r,s))

30

§ 1.2 Induction axiom for L

Induction Axiom for the type L=Lists.• Let T=||. We define a realizer

RecL,T: lL.([nil]xN,mL.([m][cons(x,m)])

[l])• RecL has type: L,T,(N,L,TT)T• Let n:N and r:T and s:N,L,TT. • We inductive define RecL,T(l,r,s):T by:1. RecL,T(nil,r,s) = r2. RecL,T(cons(n,l),r,s) = s(n,l,RecL,T(l,r,s))

31

§ 1.2 A Model for Realizers

• In order to study the behavior of Realizers we first define a model for T.

• For each simple type T of T we define some set-theoretical interpretation [T]Set:

1. [Unit] = {unit}2. [Bool] = {True,False}3. [N] = {0,1,2,3,…} 4. [L] = {nil, cons(n,nil), …}5. [TU] = [T][U] (Cartesian Product)6. [TU] = {Set-theoretical functions :

[T][U]} 32

§ 1.3 The Harrop Types

• The set-theoretical interpretation of types of T may be extended to an interpretation of open terms of T.

• We write t=Setu iff the open terms t, u have the same interpretation in the Set-Theoretical Model (for all values for free variables in t,u).

• Denote the cardinality of [T] by C([T]).• For all types T: C([T])1 (because dummyT:T)• We say that H is an Harrop type (is Harrop,

for short) iff C([T])=1 (iff t=Setu for all t,u:H).33

§ 1.3 Characterizing Harrop Types

• Lemma (Criterion for Harrop types). 1. Unit is Harrop. Bool, N, L are not Harrop2. HH’ is Harrop H, H’ are Harrop3. TH is Harrop H is Harrop • Proof.1. C({unit})=1,C({True,False}),C({0,1,2,..})>1,... 2. C([HH’]) = C([H])C([H’]) = 1 iff C([H]) =

C([H’]) =1.3. C([TH]) = C([H])C([T]) = 1 iff C([H]) = 1

(because C([T])1 ).34

§ 1.3 The Harrop(.) map• A term u is Harrop iff its type is. • We may decide if u is Harrop. If u:U is Harrop, then [U]

is a singleton and u=SetdummyU.• Denote with Harrop(t) the term obtained out of t by

replacing any maximal Harrop subterm u:U of t with dummyU.

• Theorem. t=SetHarrop(t). • Proof. We defined Harrop(t) by replacing some

subterms of t with some equal terms.• The map Harrop(.) is some simplification procedure

over Realizers. What does it do?

35

§ 1.3 The Harrop(.) map

• If is any first order formula of HA, we say that is Harrop in HA iff || is Harrop in T.

• If is an Harrop formula, then all r: are replaced with dummy|| by the map Harrop(.).

• Therefore, if is Harrop, all Realizers r: correspond to some dummy operation.

• Thus, we are interested in characterizing the set of Harrop formulas , in order to learn which realizers are (necessarily) dummy.

36

§ 1.3 Harrop formulas• If =P(t), then ||=Unit is Harrop.• If =12, then ||=|1||2| is Harrop iff |1|,|2| are.• If =2, then ||=|||2| is Harrop iff |2| is

Harrop.• If =xT.2, then ||=T|2| is Harrop iff |2| is

Harrop.• If =12, then ||=(|1||2|)(|2||1|) is Harrop

iff |1|, |2| are Harrop.• If =12, xT.2, then is not Harrop, because |

|=Bool(|1||2|), T|2| have cardinality > 1 (because T has card. > 1).

37

§ 1.3 Characterizing Harrop Formulas

• Theorem (Harrop formulas). The set of Harrop formula is the smallest set including:

• all atomic formulas, and with 1, 2, also 12, xT.2, 12, and 2 (with any formula).

• Corollary. We may decide if any is Harrop.• All proofs of Harrop formula may be

neglected if we are looking for the program hidden in a proof: they generate a dummy realizer.

38

§ 1.3 Some Harrop Formulas

• Negations. All are Harrop, because =(), and is atomic, hence Harrop. Thus, any statement defined through a negation may be neglected.

• Examples are: • “x is not a square”, • “x is a prime element = no y, z are a

factorization of x”.

39

§ 1.3 More Harrop Formulas• Universal properties. All quantifications

x1,…,xn f(x1,…,xn) = g(x1,…,xn) • are Harrop. This set includes: 1. Associativity, Commutativity, all equational

identities.2. “the list l is sorted = every element in l but the

last is of the next one”3. “the lists l, m are permutations = for all integers

x, the number of times x occurs in l and m are the same”.

40

§ 1.3 About Harrop Formulas• Note. Being Harrop depends on the exact

formal definition we choose.• For instance: if we formally define

“l, m are permutations” • by

“there is some permutation sending l into m” • then the statement we obtain is not Harrop

(because of the existential quantifier in it).• From now on, we will write in green all

Harrop parts. They are totally irrelevant and we could skip them all.

41

§ 1.3 Logical Parts• Subproofs whose conclusion is an Harrop

formulas correspond to purely logical parts of the proof. They define no action, but they may be used in order to prove that the action defined by the proof is correct.

• An example. Suppose we are looking for a solution of (x2-3x+2)=0, and we find some a such that (a2+2=3a). Then using the Harrop statement

xN.(x2-3x+2)=0 (x2+2=3x)• we may prove that a solves the original equation

(x2-3x+2)=0.

42

Lesson 2

Program Extraction

and Useless Code.

43

Plan of Lesson 2

• § 2.1 An example of Program Extraction.

• § 2.2 Program Extraction and Useless Code.

• § 2.3 Useless Code Analysis.

44

2.1: An example of Program Extraction

• We extract a toy sorting algorithm out of a proof that all lists may be sorted.

• We write first the proof, then the program. We write in green all Harrop parts. They are totally irrelevant and we could skip them all.

• Assume Perm(l,m), Sorted(m) denote the Harrop statements: “l, m are permutations”, and “m is sorted”.

• Let nil=<> and x*<x1, …, xn> = <x,x1, …, xn> • Assume we already proved that ordering on N

is total: x,yN.(xy x>y).45

2.1: The Sorting statement• We are going to prove the formal statement:

= lL.mL.(Perm(l,m) Sorted(m))• A realizer sort: has type || = LLH, where

H = |Perm(l,m) Sorted(m)| is Harrop. We abbreviate L’ = LH.

• Any realizer sort: takes some list l, and returns some pair sort(l)=<m,_>:L’, where:

1. m = sorted version of l;2. “_” = some realizer of type H, corresponding

to some proof of (Perm(l,m) Sorted(m)).

46

2.1. Optimizing the realizer “sort”

• If we apply the simplification Harrop(sort), then any subterm “_:H” in sort is replaced by dummyH :H (some dummy constant).

• Replacing “_” by dummyH is computationally correct: any proof of (Perm(l,m) Sorted(m)) states something about l, m, but it performs no action on l, m.

47

2.1. Proof of Sorting by induction on Lists

• Base case: nil. We have to prove mL.Perm(nil,m) Sorted(m). We set m=nil.

• Inductive case: x*l for some xN. By ind. hyp., Perm(l,m) and Sort(m) for some mList, hence Perm(x*l, x*m).

• We first prove the Lemma : for all m, if Sorted(m), then for all x*m there is some permutation p of x*m which is sorted.

=mL.Sorted(m)xN.pL.(Perm(x*m,p) Sorted(p))

• Out of the Lemma, we conclude Perm(x*m, p), and by transitivity of Perm, Perm(x*l,p), Q.E.D..

48

2.1. Proof of the Lemma• A realizer ins: has type || = L,Unit,NL’.

• The Lemma is proved by induction over some list m such that Sorted(m).

• Base case of the Lemma: m=nil.

• We choose p=x*nil. We have to prove Perm(x*nil, x*nil), Sorted(x*nil). This is immediate.

• Inductive case of the Lemma. Let m=y*q for some q. Assume Sorted(m)Sorted(y*q). We distinguish two subcases: xy and x>y.

49

2.1. The two subcases• Subcase xy. We choose p=x*y*q (p is

obtained from x*y*q by the identity permutation). From Sorted(y*q) and xy we conclude Sorted(x*y*q) .

• Subcase x>y. We switch x and y. By the induction hypothesis on q we have Perm(x*q,r) and Sorted(r) for some r. We choose p=y*r. We have to prove Perm(x*y*q, y*r) and Sorted(y*r) under the current hypotheses (which are: Sorted(y*q) and x>y).

50

2.1. The missing subproofs• Proof obligations. We list here the proofs of

Harrop statements which would be required to complete the proof (and we will actually skip).

• Sorted(nil) and Sorted(x*nil): by def. of Sorted• Perm(nil,nil), Perm(x*y*q, y*x*q), if Perm(l,m),

then Perm(x*l,x*m), and if Perm(l,m) and Perm(m,p), then Perm(l,p): by def. of Perm.

• If Sorted(y*q) and x>y and Perm(x*q,r) and Sorted(r), then Sorted(y*r): this comes from the fact that y<z for all elements z of x*q, hence for all elements of r. This it is a little tricky to prove formally.

51

2.1. Why we do not have to fill in all subproofs

• All proofs obligations we skipped are only needed to show that sorting is done correctly: they are proofs of Harrop statements, therefore they do not influence the Sorting Algorithm in the proof.

• We do not include these proofs here.

• This kind of formal proofs can be obtained by any interactive proof system like Coq.

52

2.1: The realizer “less”• We may already precise the realizer “sort” we

get, without knowing the proofs of Harrop statements (they will eventually be removed).

• We only have to carefully apply the inductive definition of realizer, and to replace by a dummy constant all Harrop subterms.

• Let “less” be a realizer of x,yN.(xyx>y).• “less” has type N,NBool(UnitUnit), and

less(x,y)=< True,_> xyless(x,y)=<False,_> x>y

53

2.1: The realizer “sort”sort(l) = RecL,L’( l,<nil,_>, x:N,m:L.ins(m,_,x)) :L’

• ins:L,Unit,NL’ takes some sorted list m, and a dummy value in Unit. It returns some map ins(m,_):NL’, waiting for some x:N, and returning a sorted version ins(m,_,x) of x*m.

ins(m,_)= RecL,NL’(m, base, ind) :NL’

base = x.<x*nil,_> :NL’

ind(y,f) = x.if(1(less(x,y)),x*’f(y),y*’f(x))):NL’

• In the lines above we have y:N and f:NL’ and x:N. We define x*’<m,_> = <x*m,_> :L’.

54

§ 2.2 Program Extraction and Useless Code

• In the previous examples we found examples of Useless Code (in green). Large programs are often cluttered with useless code, for many reasons.

• If a program is large, most of it comes from previous programs, or was designed some time ago, and it originally performed slightly different tasks.

• Thus, there are old tasks of no more interest, producing results of no use.

• Such useless tasks require memory space and time to initialize and to update parameters of no more use.

55

§ 2.2 Useless Code is harmful

• Useless code may slow down a computation considerably.

• Removing it is one of the problem we have to face in programming.

• To detect and remove useless parts is often conceptually simple. Yet, it is really time consuming.

• Thus, useless code removal is automatized whenever it is possible.

56

§ 2.2 Useless Code and Program Extraction

• Extracting program from proofs produces a large amount of useless code.

• In the previous lesson we introduced a simple example: Harrop subterms of any Realizer.

• Harrop subterms correspond to purely logical parts of the proof: to the parts of a proof defining no action, but showing the correctness of some action performed by the proof.

57

§ 2.2 Useless Code is more than just Harrop subterms

• Logical part of a proof are of interest only if and when we want to produce a proof.

• They are of no more interest when our task is to write a program.

• Thus, Logical parts (subproofs with an Harrop conclusion) are a typical example of useless code: they were of some use in the proof, they are of no further use now in the Realizer.

• There are also many examples of useless code in program extracted from proofs which are not Harrop formulas.

58

§ 2.2 More examples of Useless Code

• We look now through some examples of useless code in Realizers which is included in no Harrop subterm.

• Such examples are written in an extension of the language for Realizers from the previous lesson: simply typed lambda calculus + all Bohm-Berarducci Data Types.

• For a formal description of this language we refer to Boerio Ph.d..

59

§ 2.2 Data Types• Data Types. The elements of a data type D are

built by finitely many constructors c1, …, cn. Each ci has type T1, …, Tki

D, with Ti either

some previously defined type, or D itself.

• Some examples of Data Types

1. true:D, false:D: D = Bool.

2. in1:A1D, …, inn:AnDD = A1+…+An

3. <_>:A1,…,AnD: D = A1… An

4. zero:D,succ:DD: D = {Integers}

5. nil:D,cons:A,DD: D = {lists on A} 60

§ 2.2 A language for Data Types

• A language for Data Types. If D has constructors c1:T1[D], …, cn:Tn[D], then we denote D by:

X.(T1[X], …, Tn[X])

(the smallest X closed under c1, …, cn)• For each data type D we add a primitive recursor

recD. Some examples are:• D=Bool. Then recD=if.• D=A1+…+An. Then recD=case.• D={Integers}, {Lists}. See the previous lesson.

61

§ 2.2 Example 0• A particular case of Useless Code: Dead

code. A subterm t is dead code if, for all possible input values for whole term, there is an evaluation strategy not evaluating t.

• A trivial example: 3 is dead code in (x.1)(3). • Detecting dead code is a non-recursive

problem.• Indeed, to decide whether b is dead code in

if(h(x)=0,a,b) we have to decide whether h(x)=0 for all x, or not, and this is a non-recursive problem.

62

§ 2.2 Example 1

• Let p be a program of input x, of type A or B or C. • Let q be a program of output such an x, but with

possible types only A, B. Compose p with q. • Consider any “case analysis” in p over the type of

the input x. There are three clauses, according if the type of x is A or B or C.

• The “C” clause is now dead code: it is never used. Such situation may happen if we extract a program from a proof having a Lemma slightly more general than required: we prove , then we apply it to (instead of ).

63

§ 2.2 Example 1

• Let for instance p:A+B+CE be defined by:

1. p = x:A+B+C. case(x,a(x),b(x),c(x))

2. q = j°f, with f:DA+B

3. j: A+BA+B+C canonical injection.

• Then c(x) is never used, because x is obtained from f, hence is not in C.

64

§ 2.2 Example 2

• The most general case of Useless code is a code never affecting the final result, even if it may be used for some inputs.

• Let h(n,a,b) = <an,bn>:AB be recursively defined by:

• <a0, b0> = <a, b> : AB• <ai+1,bi+1> = <f(ai), g(bi)> : AB• Define k(n) = (left component of h(n)) = an :A.

65

§ 2.2 Example 2

• Look to the computation of k(n):<a,b>, <f(a),g(b)>, <f(f(a)),g((b))>, ..., <fn(a),gn(b)>

• Eventually we trash gn(a) and we return an=fn(a).• g, b are dead code in k (they are never used, under

a suitable evaluation strategy).• The pairing operation <_,_> is useless code in k: it

is indeed used, however, it never affects the final result an=fn(a).

• Again, this situation may arise in a Realizer if we proved a Lemma having a goal slightly more general than required.

66

§ 2.2 Example 3

• Let length(L) be a function taking a list L, then returning the length n of L.

• Look at the computation of length(L).

• L = cons(x1,cons(x2,...cons(xn)...)), but the actual values of x1, x2,... xn do not matter.

• Eventually, we trash all xi’s, and we return the number of cons’s.

67

§ 2.2 Example 3• All elements of L are now useless code in f. • We compute the elements of L, using an

unpredictable amount of time; yet, we do not use them in order to compute length(L).

• This situation may arise in a Realizer when:• we first prove the existence of L, then, in the

rest of the proof, we only use the number of elements of L.

68

§ 2.3 Useless code Analysis• In order to perform Useless code Analysis, it is useful

to extend the language with subtyping. • We add a dummy type Unit A for typing dead

code, the constant unit:Unit, and the (sub)typing rule: if a:A, then a: Unit.

• Data types are covariant: if A’ A, B’ B, then A’B’ AB.

• The arrow is contravariant: if A’ A , B’ B, then A’B’ AB.

• We allow Data Types D with the null constructor c:Unit, representing a redundant code in D.

69

§ 2.3 The Unit type and subtyping • For each type A, we may define some trivial

term dummyA:A by induction on A.• When we have subtyping, the type Unit

represents the set of all terms, with trivial equality: all terms of type Unit are intended to be equal.

• If we quotient Unit up to its equality notion, we get only one equivalent class: Unit is a cardinality 1 type.

• As we did in the previous lesson, we define Harrop types as types of cardinality 1 in the set-theoretical model. Unit is an Harrop type.

70

§ 2.3 Harrop types and Subtyping

• Definition. A type A is strictly positive in B iff A occurs in the left-hand side of no arrow type in B, including arrows in constructor types.

• Lemma 1. A is Harrop iff any strictly positive atomic type in A is Unit.

• Corollary 1. For any term t having an Harrop type A, we have t=dummyA in the set-theoretical model.

• Corollary 2. Let t : A, and Hp(t) be obtained by replacing all subterms v having an Harrop type B by dummyB. Then t=Hp(t) in the set-theoretical model.

71

§ 2.3 Maximizing Harrop subterms

• Remark. Corollary 2 says that each subterm of Harrop type is useless, because it may be replaced by a dummy constant without altering the value of the term.

• Hence Hp(t) removes useless code in t.

• We will introduce now a redundant code analysis much more general than in the previous lesson: before applying Hp, we will retype the term t in order to maximize the subterms of t having an Harrop type.

72

§ 2.3 A first result• Theorem 1 (Removing Harrop subterms).

Assume |-t:A.

• Retype with Unit some parts of t in such a way to preserve the condition |-t:A.

• Then replace all subterms v of t now having an Harrop type B by the constant dummyB.

• The term u we obtain in this way is equal to t in the set-theoretical model.

73

§ 2.3 Some comments• Theorem 1 outlines method for removing

useless code. We have only to select some retyping of t which:

1. is correct, and2. does not change the final typing |-t:A of t. • The clause 2 is essential, because by

subtyping every term, included t, can be retyped by the Harrop type Unit. But if we retype t with Unit, then we remove it.

• This method for eliminating dead code may be found in many languages. In Coq, all types of type Prop have cardinality 1 and play the same role as Unit type in our language.

74

§ 2.3 The Main Theorem

• Main Theorem. Let t be any term of the simply typed lambda calculus with Bohm-Berarducci Data Types.

• Order the set of terms t’ we get by a retyping of t according to the set of Harrop subterms they have. This defines a partial ordering.

1. This partial ordering has a maximum, a term t’ having a maximum of Harrop subterms. Denote t’ by Fl(t).

2. Fl(t) may be computed in linear time over the type-decorated version of t.

75

§ 2.3 Using the map Fl(t)

• Fact. Fix any term t from the Examples 1-3. Then all dead code we found belongs to some Harrop subterms of Fl(t).

• Thus, all dead code in Examples 1-3 may be removed by first computing Fl(t), then Hp(Fl(t)).

• By construction, context and type of t and Hp(Fl(t)) are the same, and

t =Set Fl(t) =Set Hp(Fl(t)).

76

§ 2.3 The “Iso” map

• Some further optimization may be obtained by applying a new map, Iso(.) (not to be included here) to the term Hp(Fl(t))

• Iso(.) removes some useless code which is not dead, by repeatedly applying the isomorphisms

AU ~ UA ~ UA ~ A• Example are: Iso(.) replaces <a,u>, <u,a>,

p1(a) (if a:AU), p2(a) (if a:UA) with a.• Iso(.) is but a minor detail, yet Iso(.) greatly

improves readability. Iso(.) removes all green parts in the example 2.1.

77

§ 2.3 About Efficiency1. Fl(p) is computed using saturation over a subset

of the adjacency graph of typing assignment. Fl is an higher-order version of the Flow algorithm for compilers.

2. The type-decorated version p’ of p is, in theory, exponential in p. Thus, in theory, Fl, which is linear in the size of p’, is exponential.

3. In programming practice, however, p’ is feasible to compute. So Fl is feasible, too.

4. In 2000, N. Kobayashi found a way to get rid of exponential time.

78

§ 2.3 Up-to-date: Year 2000• N. Kobayashi (C.S., Tokyo Univ.). For languages

with let-polymorphism, Fl(t) may be found in time (n log*(n)) over the size of the untyped program t.

• We explain what is log*(n) in the next slide.• Kobayashi idea is merging the algorithm inferring

the type with the algorithm inferring useless code out of typing.

• In this way, we do not have to apply type inference to subterms with an Harrop type, because they are redundant code and they will be removed.

79

§ 2.3 What is log*(n) ?

• Log*(n) is the inverse of exp*(n) = expn(1) (the super-exponential, or a tower of n digits: 222…

1. log*(2) = log*(2) = 1 2. log*(4) = log*(22) = 23. log*(16) = log*(222) = 3

4. log*(65536) = log*(2222) = 4

5. For n 265536, we have log*(n) 5. • 265536 is a number of 19,729 digits!• O(n*log*(n)) is, in practice, O(n*5) = O(n).

80

References for the course

• L. Boerio “Optimizing Programs Extracted from Proofs”. Ph. D. Thesis, C. S. Dept. Turin University, 1997.

• Available in the web page of the course:

http://www.di.unito.it/~stefano

• (look for the words: “Boerio Ph.d.”)• S. Berardi.

"Pruning Simply Typed Lambda Terms“ Journal of Logic and Computation, Vol.6 No 5, pp663-681 1996

81

More References• “Type-bases useless code elimination

for functional programs (Position Paper)”, by S. Berardi, M. Coppo, F. Damiani and P. Giannini, PLI Workshop - SAIG'00, LNCS 1924:172-189, 2000.

• “Automatic Useless code detection and elimination for HOT functional programs”, by F. Damiani, P. Giannini, J.F.P. 10 (6): 509-559, 2000.

82