The Difficulty of Improving Upon the Standard ML Module...
Transcript of The Difficulty of Improving Upon the Standard ML Module...
The Difficulty of Improving Uponthe Standard ML Module System
Derek Dreyer
Max Planck Institute for Software Systems (MPI-SWS)Kaiserslautern and Saarbrucken, Germany
MacQueen FestChicago, USAMay 13, 2012
Balancing expressive power and conceptual simplicity
Dave MacQueen, “Reflections on Standard ML” (1993):
“The pragmatic perspective tends to lead designers to addfeatures to the language to enhance its perceived expressive poweror convenience, and hence there is a danger of excessive complexityand featurism, especially if the interaction of different features isnot carefully considered. One of the hardest tasks in languagedesign is judging what should be left out of the language, becauseevery feature has its constituency. Another difficult and criticalproblem is to judge the expressiveness/complexity tradeoff for agiven feature. This is especially tricky because the use of aparticular feature in programming may involve a high mentaloverhead for programmers even though the feature’s semanticdescription is relatively straightforward (e.g., first-classcontinuations).”
The Standard ML module system hits a sweet spot!
The modularity mechanisms of dependent typeswithout the semantic/type-theoretic complexity
Phase separation yields decidable typechecking
Intuitive treatment of linking constraintsSharing by specification instead of parameterization
Generative semantics of type abstractionType equivalence is simple to understand
Crucial for encapsulating effects properly!
. . . (among many other things). . .
Dave MacQueen has made my life very difficult. . .
Research ≈ trying to improve the state of the artI chose to try to improve on Standard ML Modules
Oy vey iz mir!Very hard to improve on SML Modules without makingmajor changes to the design/semantics of the language
But I keep on truckin’. . .. . . while thinking of Dave as the skeptical angel/devil onmy shoulder, asking: “Is your newfangled language designreally any better than SML Modules?”
Thanks, Dave!
(no sarcasm intended)
Dave MacQueen has made my life very difficult. . .
Research ≈ trying to improve the state of the artI chose to try to improve on Standard ML Modules
Oy vey iz mir!Very hard to improve on SML Modules without makingmajor changes to the design/semantics of the language
But I keep on truckin’. . .. . . while thinking of Dave as the skeptical angel/devil onmy shoulder, asking: “Is your newfangled language designreally any better than SML Modules?”
Thanks, Dave!
(no sarcasm intended)
Dave MacQueen has made my life very difficult. . .
Research ≈ trying to improve the state of the artI chose to try to improve on Standard ML Modules
Oy vey iz mir!Very hard to improve on SML Modules without makingmajor changes to the design/semantics of the language
But I keep on truckin’. . .. . . while thinking of Dave as the skeptical angel/devil onmy shoulder, asking: “Is your newfangled language designreally any better than SML Modules?”
Thanks, Dave!
(no sarcasm intended)
Dave MacQueen has made my life very difficult. . .
Research ≈ trying to improve the state of the artI chose to try to improve on Standard ML Modules
Oy vey iz mir!Very hard to improve on SML Modules without makingmajor changes to the design/semantics of the language
But I keep on truckin’. . .. . . while thinking of Dave as the skeptical angel/devil onmy shoulder, asking: “Is your newfangled language designreally any better than SML Modules?”
Thanks, Dave!
(no sarcasm intended)
Two directions for “improvement”
1 Recursive modules
2 Substructural types
Two directions for “improvement”
1 Recursive modules
2 Substructural types
The “double vision” problem [Crary et al. ’99, Dreyer ’05]
In recursive modules, there may be multiple “paths”(i.e., multiple ways of referring to) the same type
Want all such paths to be considered equal
But this is hard to achieve, leading one to seetwo paths to the same type as distinct
Example of the “double vision” problem
module rec A : sig
type t
val f : B.u -> A.t
end = ...
and B : sig
type u
val g : A.t -> B.u
end = ...
Example of the “double vision” problem
module rec A : sig
type t
val f : B.u -> A.t
end = struct
type t = int
fun f x = ... (B.g 3) ...
end
and B : sig
type u
val g : A.t -> B.u
end = ...
Approaches to handling double vision “directly”
Force recursive modules to be transparent
[Crary et al. ’99, Russo ’01]
Too restrictive
Disallow more than one path to an abstract type
Units [Flatt-Felleisen ’98, Owens-Flatt ’06]
Can’t really encode ML module system properly
Handle special cases
OCaml [Leroy ’03], Scala [Odersky-Zenger ’05]
Don’t handle itTraviata [Nakata-Garrigue ’06]
Elaboration semantics
Interpret module language into a more explicit IL:
Classify ML modules via (annotated) IL types Σ
Erasure Σ◦ maps Σ to IL type
Soundness of elaboration:
Γ `ML mod : Σ ; M =⇒ Γ `IL M : Σ◦
Γ `ML sig ; Σ =⇒ Γ `IL Σ◦ type
Different elaborations of ML modules possible:
[Harper-Stone ’00]: IL = “translucent sums”
[Rossberg et al. ’10]: IL = Fω (“F-ing modules”)
Elaboration can be harmful to your health
From my thesis, an unmitigated disaster:
9.3. ELABORATION 227
! ! modexp ! mod :! sig
1. ! ! sigexp ! sig rec : Sig
2. !,modid!mvar rec:sig rec !stat modexp ! metasigactual
3. mvar crec "# FV(Stat(Priv(metasig actual))) $ FV(Stat(Pub(metasig actual)))
4. !,mvar actual:Stat(Pub(metasig actual)) !sub
mvar actual : Stat(Pub(metasig actual)) % Stat(sig rec) ! : tstatsig coerced
5. metasig !actual := metasigactual[mvar c
rec.visible"/mvar crec]
6. metasig static := "(mvar rec).&(mvar actual: metasig !actual ).tstatsig coerced
7. ! !can Priv(metasig static) ! mod static
8. tsig rec := s(mvar cstatic.visible" : sig rec)
9. " := !;mvar static:metasig static;modid!mvar rec:maybe(tsig rec)10. " !rec unroll(mvar static).hidden ' modexp ! mod actual : tsigactual
11. ",mvar actual:tsigactual !sub mvar actual : tsigactual % tsig rec ! pmod coerced :12. mod := let mvar static=mod static in
(rec(mvar rec:tsig rec.letmvar actual=mod actual in (pmod coerced : tsig rec)): sig rec)
! ! rec (modid : sigexp) modexp ! mod :P sig rec
(9.152)
Rule 9.152: Let us consider the premises one at a time. (I suggest the reader compare the formalsteps here with the high-level description given in Section 5.4, as they correspond quite closely.)
1. Translate the declared signature sigexp to sig rec, which need not be transparent.
2. Perform static elaboration of modexp, resulting in the meta-signature metasig actual.
3. Enforce the “dynamic-on-static” restriction on modexp by checking that the recursive modulevariable mvar rec does not appear in the static part of metasig actual. References to mvar rec
may still occur in metasig actual’s value specs (i.e., its datatype specs).
4. We need to close up references to mvar rec in metasigactual by enclosing it in an rds (cf. Fig-ure 5.17). Unfortunately, we cannot just write "(mvar rec).metasigactual, because (by step 2)metasigactual expects mvar c
rec to have kind Fst(sig rec), not Fst(metasigactual). So first wecoerce metasigactual into the shape of Stat(sig rec), which produces tstatsig coerced.
5–6. Using tstatsig coerced, we can close up metasig actual with an rds and call it metasig static (inSection 5.4, this was called CSS). For now, ignore the box around metasig !
actual. The purposeof the box will be explained below when the !rec judgment is defined.
7. Construct the canonical module mod static matching metasig static. Note: this will only succeedif Fst(Priv(metasig static)) is an expandable kind. From a programming perspective, this meansthat if in modexp there appears a total functor expression whose body contains datatype
definitions, then the argument signature of that functor must not contain any transparenttype specifications. I admit this is a rather bizarre restriction, but I see no way around it.
8. mod static will eventually be bound to mvar static (in Section 5.4, this was called Static). Wecan thus selfify the declared signature sig rec with respect to mvar static.visible", in order toobtain a transparent version of the declared signature tsig rec.
plus 37 more rules just for recursive modules! §
The essential problem
Good elaboration should not be too “lossy”Should “preserve meaning” of ML program
This is true of Harper-Stone and F-ing modules
But translucent sums / System F do not accountfor recursively defined ADTs
E.g. my thesis elaboration threw away all abstractioninside a recursive module
So a more expressive IL is needed
RTG [Dreyer ’05]
A type system for “Recursive Type Generativity”System F + forward decl’s of abstract types
Distinguish creation of type name from definition
new α in e (α bound)
def α := τ in (e : τ ′) (α free)
“Ability to define α” treated as a linear resourceTracked using a simple effect system,and in later work, using a linear type-and-kind system
Curing double vision in RTG
The new’s and def’s can be inferred automatically as part ofan elaboration translation for recursive modules [Dreyer ’07].
new α, β in
module rec A : sig
type t
val f : B.u -> A.t
end = struct
type t = int
fun f x = ... (B.g 3) ...
end
and B : sig
type u
val g : A.t -> B.u
end = ...
Curing double vision in RTG
The new’s and def’s can be inferred automatically as part ofan elaboration translation for recursive modules [Dreyer ’07].
new α, β in
module rec A : sig
type t = αval f : B.u -> A.t
end = def α := int in struct
type t = int
fun f x = ... (B.g 3) ...
end
and B : sig
type u = βval g : A.t -> B.u
end = def β := ??? in ...
Greatly improved account of recursive modules
Well-formed modules: !;" ! mod : # with ! "We omit “with ! "” if ! = # (i.e., ifmod does not define any abstract types).
X :# $ "!;" ! X."s : #."s
(14)!;" ! con ! A : K
!; " ! [con ] : [[= A :K]](15)
! ! # : T !;" ! exp : #
!;" ! [exp] : [[# ]](16)
K = Tn %T !;" ! con ! A : K $ = $1, . . . , $n ! &K $ !
!; " ! [" ' con : K] : [[" : [[= ! : K]], in : [[([$]. A($) ) !($)]], out : [[([$]. !($) ) A($)]]]] with ! "(17)
!;" ! [] : [[]](18)
!;" ! mod1 : #1 with !1 " !@ !1 ";", X1 :#1 ! [" % X= mod ] : [[" : #]] with !2 "!; " ! ["1 %X1 =mod1, " % X=mod ] : [["1 :#1, " : #]] with !1, !2 "
(19)
!;" ! mod1 : #1 with !1 " ! @ !1 ";", X :#1 ! mod2 : #2 with !2 "!;" ! let X= mod1 in mod2 : #2 with !1, !2 " (20)
!;" ! sig ! *(!1 "K1).(L1;#1) !, !1 "K1, !2 &K2;", X : #1 ! mod : #2 with !2 "!; " ! &(X : sig).mod : ((!1 " K1).(L1; #1)% *(!2 "K2).(#; #2)
(21)
!;" ! P1 : ((!1 " K1).(L1; #1)% *(!2 "K2).(L2;#2) !;" ! P2 : #
! &K2 + ! ! lookup L1 in # ! ' FV(') , &(!) = # ! # - '#1
!;" ! P1(P2) : '{!2 .% !}#2 with ! " (22)
!; " ! sig ! *(! "K).(L;#) !, ! "K;", X :# !stat mod : #stat with $ "! lookup L in #stat ! ' !;", X : '# ! mod : #! with $ " ! #! - '#
!;" ! rec (X : sig)mod : #! with $ "(23)
!;" ! sig ! *(!0 "K0).(L0;#0) ! = !!, ! & K0 (L;#) = {!0 .%!}(L0;#0)
!, $ &L;" !stat mod : #stat with $ " ! lookup L in #stat ! ' FV(') , &(!) = #!!, $ & L; '" ! mod : #! with $ " ! #! - '#
!;" ! mod :> sig : # with ! " (24)
!;" ! sig ! *(! "K).(L;#) !;" ! mod : #! with $ " ! lookup L in #! ! ' ! #! - '#
!;" ! mod : sig : '# with $ "(25)
Statically well-formed modules: !; " !stat mod : # with ! "The rules defining this static judgment are precisely the same as the rules defining the regular module typing judgment (above),except with the shaded premises removed, and all occurrences of the regular module typing judgment replaced by this static judgment.
Signature matching: ! #1 - #2
! # - #(26)
#! = [[" :#]]
! #! - [[]](27)
! #!."1 - #1 ! #! - [[" :#]]
! #! - [["1 :#1, " :#]](28)
! lookup L!1 in #1 ! '1 ! #1 - '1#
!1 ! lookup L2 in '1#
!2 ! '2 ! '1#
!2 - '2#2
! ((!!1 " K!
1).(L!1;#
!1)% *(!!
2 "K!2).(L!
2;#!2) - ((!1 "K1).(L1;#1) % *(!2 "K2).(L2;#2)
(29)
Abstract type lookup: ! lookup L in # ! '
L = {!1 : K1 .% "s1, . . . , !n : Kn .% "sn} '0 = #(i $ 1..n : #."si = [[= Ai : Ki]] FV(Ai) , dom(L) + {!1, . . . , !i"1} 'i = 'i"1 / {!i .% 'i"1Ai}
! lookup L in # ! 'n(30)
Figure 6. Typing Rules for Modules
38-fold reduction in the number of typing rulesneeded to support recursive modules!
But a problem remains: Separate compilation
Greatly improved account of recursive modules
Well-formed modules: !;" ! mod : # with ! "We omit “with ! "” if ! = # (i.e., ifmod does not define any abstract types).
X :# $ "!;" ! X."s : #."s
(14)!;" ! con ! A : K
!; " ! [con ] : [[= A :K]](15)
! ! # : T !;" ! exp : #
!;" ! [exp] : [[# ]](16)
K = Tn %T !;" ! con ! A : K $ = $1, . . . , $n ! &K $ !
!; " ! [" ' con : K] : [[" : [[= ! : K]], in : [[([$]. A($) ) !($)]], out : [[([$]. !($) ) A($)]]]] with ! "(17)
!;" ! [] : [[]](18)
!;" ! mod1 : #1 with !1 " !@ !1 ";", X1 :#1 ! [" % X= mod ] : [[" : #]] with !2 "!; " ! ["1 %X1 =mod1, " % X=mod ] : [["1 :#1, " : #]] with !1, !2 "
(19)
!;" ! mod1 : #1 with !1 " ! @ !1 ";", X :#1 ! mod2 : #2 with !2 "!;" ! let X= mod1 in mod2 : #2 with !1, !2 " (20)
!;" ! sig ! *(!1 "K1).(L1;#1) !, !1 "K1, !2 &K2;", X : #1 ! mod : #2 with !2 "!; " ! &(X : sig).mod : ((!1 " K1).(L1; #1)% *(!2 "K2).(#; #2)
(21)
!;" ! P1 : ((!1 " K1).(L1; #1)% *(!2 "K2).(L2;#2) !;" ! P2 : #
! &K2 + ! ! lookup L1 in # ! ' FV(') , &(!) = # ! # - '#1
!;" ! P1(P2) : '{!2 .% !}#2 with ! " (22)
!; " ! sig ! *(! "K).(L;#) !, ! "K;", X :# !stat mod : #stat with $ "! lookup L in #stat ! ' !;", X : '# ! mod : #! with $ " ! #! - '#
!;" ! rec (X : sig)mod : #! with $ "(23)
!;" ! sig ! *(!0 "K0).(L0;#0) ! = !!, ! & K0 (L;#) = {!0 .%!}(L0;#0)
!, $ &L;" !stat mod : #stat with $ " ! lookup L in #stat ! ' FV(') , &(!) = #!!, $ & L; '" ! mod : #! with $ " ! #! - '#
!;" ! mod :> sig : # with ! " (24)
!;" ! sig ! *(! "K).(L;#) !;" ! mod : #! with $ " ! lookup L in #! ! ' ! #! - '#
!;" ! mod : sig : '# with $ "(25)
Statically well-formed modules: !; " !stat mod : # with ! "The rules defining this static judgment are precisely the same as the rules defining the regular module typing judgment (above),except with the shaded premises removed, and all occurrences of the regular module typing judgment replaced by this static judgment.
Signature matching: ! #1 - #2
! # - #(26)
#! = [[" :#]]
! #! - [[]](27)
! #!."1 - #1 ! #! - [[" :#]]
! #! - [["1 :#1, " :#]](28)
! lookup L!1 in #1 ! '1 ! #1 - '1#
!1 ! lookup L2 in '1#
!2 ! '2 ! '1#
!2 - '2#2
! ((!!1 " K!
1).(L!1;#
!1)% *(!!
2 "K!2).(L!
2;#!2) - ((!1 "K1).(L1;#1) % *(!2 "K2).(L2;#2)
(29)
Abstract type lookup: ! lookup L in # ! '
L = {!1 : K1 .% "s1, . . . , !n : Kn .% "sn} '0 = #(i $ 1..n : #."si = [[= Ai : Ki]] FV(Ai) , dom(L) + {!1, . . . , !i"1} 'i = 'i"1 / {!i .% 'i"1Ai}
! lookup L in # ! 'n(30)
Figure 6. Typing Rules for Modules
38-fold reduction in the number of typing rulesneeded to support recursive modules!
But a problem remains: Separate compilation
The trouble with separate compilation
ML’s separate compilation mechanism isthe functor.
functor Sep A (X : SIG B) :> SIG A = ...
Problem: SIG B depends on type components of A,which are not in scope.
Not obvious how to generalize functors to work inthe recursive case.
The trouble with separate compilation
ML’s separate compilation mechanism isthe functor.
functor Sep A (X : SIG B) :> SIG A = ...
Problem: SIG B depends on type components of A,which are not in scope.
Not obvious how to generalize functors to work inthe recursive case.
Symptom of a larger problem. . .
Structure formation (struct)
Signature formation (sig)
Signature inheritance (include)
Transparent type specifications (type t = typ)
Opaque type specifications (type t)
Value specifications (val v : typ)
Signature refinement (where type / with type)
Sharing constraints (sharing type)
Signature bindings (signature)
Functor abstraction (functor)
Functor application ( ( ) )
Transparent signature ascription ( : )
Opaque signature ascription ( :> )
Local definitions (let / local)
Recursive structures (struct rec)
Recursively dependent signatures (sig rec)
Question
Why are modules composed using parameterizationwhile signatures are composed using fibration?
Symptom of a larger problem. . .
Structure formation (struct)
Signature formation (sig)
Signature inheritance (include)
Transparent type specifications (type t = typ)
Opaque type specifications (type t)
Value specifications (val v : typ)
Signature refinement (where type / with type)
Sharing constraints (sharing type)
Signature bindings (signature)
Functor abstraction (functor)
Functor application ( ( ) )
Transparent signature ascription ( : )
Opaque signature ascription ( :> )
Local definitions (let / local)
Recursive structures (struct rec)
Recursively dependent signatures (sig rec)
Question
Why are modules composed using parameterizationwhile signatures are composed using fibration?
MixML (Dreyer-Rossberg ’08)
Key idea: Compose everything by fibration!
MixML modules synthesize ML’s structure andsignature languages into one.
Consequences:
ML structures and signatures are endpoints on aspectrum of MixML modules.
Signatures and structures (and mixtures of both)are composed using the exact same constructs.
Both structures and signaturesare just a particular mode of use of modules!
MixML (Dreyer-Rossberg ’08)
Key idea: Compose everything by fibration!
MixML modules synthesize ML’s structure andsignature languages into one.
Consequences:
ML structures and signatures are endpoints on aspectrum of MixML modules.
Signatures and structures (and mixtures of both)are composed using the exact same constructs.
Both structures and signaturesare just a particular mode of use of modules!
The MixML module language
mod ::= X (variable)| {} (empty)| [exp] | [: typ] (term)| [typ] | [: kind ] (type)| {` = mod} | mod .` (namespaces)| (X = mod1) with mod2 (linking)| (X = mod1) seals mod2 (sealing)| [mod ] | new mod (units)
Some useful derived forms
Structure formation (struct)
Signature formation (sig)
Signature inheritance (include)
Transparent type specifications (type t = typ)
Opaque type specifications (type t)
Value specifications (val v : typ)
Signature refinement (where type / with type)
Sharing constraints (sharing type)
Signature bindings (signature)
Functor abstraction (functor)
Functor application ( ( ) )
Transparent signature ascription ( : )
Opaque signature ascription ( :> )
Local definitions (let / local)
Recursive structures (struct rec)
Recursively dependent signatures (sig rec)
A qualified success
MixML could provide a clean core for anext-generation ML-style module system
Syntactically minimal and useful for understandingvanilla SML modules (even ignoring recursion)
But semantics is still quite complex:
Mixin’ Up the ML Module System A:23
Modules: Γ; R;β � mod : Σ
X : |Σ| ∈ Γ
Γ; {||}; ∅ � X : |Σ| (VAR)Γ; {||}; ∅ � {} : {||} (EMP)
� A ⇑ knd
Γ; [[= A]]; ∅ � [:knd] : [[= A]](ITYP)
Γ � typ ❀ A
Γ; {||}; ∅ � [typ] : [[= A]](ETYP)
Γ � typ ❀ A � A ⇑ type
Γ; {||}; ∅ � [:typ] : [[A]]−(IVAL)
Γ � exp : A � A ⇑ type
Γ; {||}; ∅ � [exp] : [[A]]+(EVAL)
Γ; R;β � mod : Σ
Γ; {|� : R|};β � {�=mod} : {|� :Σ|}(STR)
Γ; {|� : R|};β � mod : {|� :Σ, �� : |Σ�||}Γ; R;β � mod .� : Σ
(DOT)
� L1 locates α1 R1 # Σ2 Γ; R � R1 � L1;β1 � mod1 : Σ1
� L2 locates α2 R2 # Σ1 Γ, X : |Σ1|; R � R2 � L2;β2 �stat mod2 : Σ�2
� (L1;Σ1) � (L2;Σ�2) ❀ δ Γ, X : |δΣ1|; R � R2 � δL2;β2 � mod2 : Σ2
α1,α2 fresh � δΣ1 + Σ2 ⇒ Σ
Γ; R � R1 � R2;β1,β2 � (X =mod1) with mod2 : Σ(LINK)
� L1 locates α1 Γ; L1;β1 � mod1 : Σ1
� L2 locates α2 Γ, X : |Σ1|; L2;β2 �stat mod2 : Σ�2
� (L1;Σ1) � (L2;Σ�2) ❀ δ δΓ, X : |δΣ1|; δL2;β2 � mod2 : Σ2
β2,α2 fresh � δΣ1 + Σ2 ⇒ |Σ|Γ; {||};β1,α1 � (X =mod1) seals mod2 : |Σ1|
(SEAL)
Γ � mod : ΦΓ; {||}; ∅ � [mod] : [[Φ]]+
(EUN)
Γ � mod : [[∀α. ∃β. (L;Σ)]]+ dom(δ) = {α,β}Γ; δL; δβ � new mod : δΣ
(NEW)
Complete Modules: Γ � mod : Σ
Γ; {||};β � mod : |Σ| β fresh β �∈ fv(Σ)
Γ � mod : |Σ| (COMPL)
Units: Γ � mod : Φ
Γ; L;β � mod : Σ � L locates α α,β fresh
Γ � mod : ∀α. ∃β. (L;Σ)(UNIT)
Fig. 5. Typing Rules for MixML
ACM Transactions on Programming Languages and Systems, Vol. V, No. N, Article A, Publication date: January YYYY.
Actively working with my student Scott Kilpatrick onsimplifying it further
A qualified success
MixML could provide a clean core for anext-generation ML-style module system
Syntactically minimal and useful for understandingvanilla SML modules (even ignoring recursion)
But semantics is still quite complex:
Mixin’ Up the ML Module System A:23
Modules: Γ; R;β � mod : Σ
X : |Σ| ∈ Γ
Γ; {||}; ∅ � X : |Σ| (VAR)Γ; {||}; ∅ � {} : {||} (EMP)
� A ⇑ knd
Γ; [[= A]]; ∅ � [:knd] : [[= A]](ITYP)
Γ � typ ❀ A
Γ; {||}; ∅ � [typ] : [[= A]](ETYP)
Γ � typ ❀ A � A ⇑ type
Γ; {||}; ∅ � [:typ] : [[A]]−(IVAL)
Γ � exp : A � A ⇑ type
Γ; {||}; ∅ � [exp] : [[A]]+(EVAL)
Γ; R;β � mod : Σ
Γ; {|� : R|};β � {�=mod} : {|� :Σ|}(STR)
Γ; {|� : R|};β � mod : {|� :Σ, �� : |Σ�||}Γ; R;β � mod .� : Σ
(DOT)
� L1 locates α1 R1 # Σ2 Γ; R � R1 � L1;β1 � mod1 : Σ1
� L2 locates α2 R2 # Σ1 Γ, X : |Σ1|; R � R2 � L2;β2 �stat mod2 : Σ�2
� (L1;Σ1) � (L2;Σ�2) ❀ δ Γ, X : |δΣ1|; R � R2 � δL2;β2 � mod2 : Σ2
α1,α2 fresh � δΣ1 + Σ2 ⇒ Σ
Γ; R � R1 � R2;β1,β2 � (X =mod1) with mod2 : Σ(LINK)
� L1 locates α1 Γ; L1;β1 � mod1 : Σ1
� L2 locates α2 Γ, X : |Σ1|; L2;β2 �stat mod2 : Σ�2
� (L1;Σ1) � (L2;Σ�2) ❀ δ δΓ, X : |δΣ1|; δL2;β2 � mod2 : Σ2
β2,α2 fresh � δΣ1 + Σ2 ⇒ |Σ|Γ; {||};β1,α1 � (X =mod1) seals mod2 : |Σ1|
(SEAL)
Γ � mod : ΦΓ; {||}; ∅ � [mod] : [[Φ]]+
(EUN)
Γ � mod : [[∀α. ∃β. (L;Σ)]]+ dom(δ) = {α,β}Γ; δL; δβ � new mod : δΣ
(NEW)
Complete Modules: Γ � mod : Σ
Γ; {||};β � mod : |Σ| β fresh β �∈ fv(Σ)
Γ � mod : |Σ| (COMPL)
Units: Γ � mod : Φ
Γ; L;β � mod : Σ � L locates α α,β fresh
Γ � mod : ∀α. ∃β. (L;Σ)(UNIT)
Fig. 5. Typing Rules for MixML
ACM Transactions on Programming Languages and Systems, Vol. V, No. N, Article A, Publication date: January YYYY.
Actively working with my student Scott Kilpatrick onsimplifying it further
Two directions for “improvement”
1 Recursive modules
2 Substructural types
Major theme in types research
Enriching type systems with information aboutpurity and effects
Want types (e.g., τ1 → τ2) to “mean more”Monads useful for isolating effectful computations
Want to use types for compositional verification ofhigher-order imperative programs
Substructural type systems have been developed to animpressive extent, e.g., Hoare Type Theory (HTT)
Richly expressive types are not always a clear win
Rich types can be too informativeHard to hide local uses of state (e.g., memoization)that are not externally observable
As Bob Harper likes to say, “Monads seem like a greatidea on Monday, Wednesday and Friday. . . ”
“Poor” old SML types are great at hiding stateGenerative semantics plays a crucial role here
Using SML generativity to hide local state
signature SYMBOL = sigtype tval eq : t * t -> boolval insert : string -> tval lookup : t -> string
endfunctor Symbol () :> SYMBOL = structtype t = intval size = ref 0val table = ref nilfun eq (x,y) = x = yfun insert str = (
size := !size + 1;table := str :: !table;!size
)fun lookup n =
List.nth (!table, !size - n)end
Figure 1. Generativity Example
given string to the table and returns a fresh symbol mapped to it;and a function lookup, which looks up a given symbol in the tableand returns the corresponding string.The functor Symbol implements the symbol type t as an integer
index into a (mutable) list of strings. When applied, Symbol createsa fresh table (represented as a pointer to an empty list) and amutable counter size (representing the size of the table). Theimplementations of the various functions are straightforward, andthe body of the functor is sealed with the signature SYMBOL, thushiding access to the local state (table and size).The call to List.nth in the lookup function might in general
raise a Subscript exception if the input n were an arbitrary inte-ger. However, we “know” that this cannot happen because lookupis exported with argument type t, and the only values of type t thata client could possibly have gotten hold of are the values returnedby insert, i.e., integers that are between 1 and the current size oftable. Therefore, the implementation of the lookup function neednot bother handling the Subscript exception.This kind of reasoning is commonplace in modules that encap-
sulate local state. But what justifies it? Intuitively, the answer istype generativity. Each instantiation of the Symbol functor createsa fresh symbol type t, which represents the type of symbols that arevalid in its own table (but not any other). Were Symbol not gen-erative, each application of the Symbol functor would produce amodule with distinct local state but the same symbol type. It wouldthen be easy to induce a Subscript error by accidentally passinga value of one table’s symbol type to another’s lookup function.1While this intuition about the importance of generativity is very
appealing, it is also completely informal. The goal of this paper isto develop a formal framework for reasoning about the interactionof generative type abstraction and mutable state.In the case of an example like the Symbol functor, we will
be able to show that the implementation of Symbol shown inFigure 1 is contextually equivalent to one whose lookup functionis replaced by:
fun lookup n =if n > 0 andalso n <= !size
andalso !size = length(!table)then List.nth (!table, !size - n)else "Hell freezes over"
1 This is the case, for example, in OCaml, which only supports applicative(i.e., non-generative) functors [15].
In other words, there is no observable difference between the orig-inal Symbol functor and one that dynamically checks the variousinvariants we claim to “know.” Hence, the checks are unnecessary.This kind of result can be understood as an instance of represen-
tation independence, albeit a somewhat degenerate one in that theADTs in question share the same type representation. As with mostsuch results, the proof hinges on the construction of an appropriaterelational interpretation of the abstract type t, which serves to im-pose an invariant on the possible values of type t. In this case, wewish to assert that for a particular structure S defined by Symbol(),the only values of type S.t are integers between 1 and the currentsize of S’s table. This will allow us to prove that any range checkon the argument to S’s lookup function is superfluous.The problem is that the relational interpretation we wish to
assign to S.t depends on the current values stored in S’s localstate. In effect, as S’s insert function is called repeatedly overtime, its table grows larger, and the relational interpretation ofS.t must grow accordingly to include more and more integers.Thus, what we need is an account of state-dependent representationindependence, in which the relational interpretations of abstracttypes are permitted to grow over time, in a manner that is tightlycoupled with changes to some local state.
1.2 OverviewIn this paper, we present a novel method for proving state-dependentrepresentation independence results. Our method extends previ-ous work by Ahmed on syntactic step-indexed logical relationsfor recursive and quantified types [1]. We extend her techniquewith support for reasoning about local state, and demonstrate itseffectiveness on a variety of small but representative examples.Although our primary focus is on proving representation indepen-dence for ADTs that exhibit an interaction of existentials and state,our method also handles several well-known simply-typed exam-ples from the literature on local state (such as Pitts and Stark’s“awkward” example [21]) that have proven difficult for previouslogical-relations-based methods to handle.In order to reason about local state, we build into our logical
relation a notion of possible worlds. While several aspects of ourpossible worlds are derived from and inspired by prior work, otheraspects are quite novel:1. We enrich our possible worlds with populations and laws,which allow us to evolve the relational interpretation of anabstract type over time in a controlled, state-dependent fashion.For instance, we can use a population to grow a set of val-ues (e.g., the integers between 1 and some n), together with alaw that explains what the current population implies about thecurrent machine state (e.g., that the symbol table has size n).
2. Second, our method provides the ability to reason locally aboutreferences to higher-order values. While ours is not the firstmethod to handle higher-order state, our approach is novel andarguably simpler than previous accounts. It depends criticallyon step-indexing in order to avoid a circularity in the construc-tion of possible worlds.The remainder of the paper is structured as follows. In Sec-
tion 2, we present Fµ!, our language under consideration, whichis essentially System F extended with general recursive types andgeneral ML-style references. In Section 3, we explain at a high levelhow our method works and what is novel about it. In Section 4, wepresent the details of our logical relation and prove it sound (but notcomplete) with respect to contextual equivalence of Fµ! programs.In Section 5, we show how to use our method to prove a num-ber of interesting contextual equivalences. Finally, in Section 6, weconclude with a thorough comparison to related work, as well asdirections for future work.
2
Key observation
SML generativity (which can be modeled using ∃’s)enables hiding of monotonic state
State that undergoes “irreversible” changes
Across a series of papers (POPL’09, ICFP’10, POPL’12),Ahmed, Birkedal, Hur, Neis, Rossberg, Vafeiadis and Ihave explored the semantic underpinnings of this
But what about hiding of non-monotonic state?What does that even mean?
We need a story about hiding in a substructural setting!
Key observation
SML generativity (which can be modeled using ∃’s)enables hiding of monotonic state
State that undergoes “irreversible” changes
Across a series of papers (POPL’09, ICFP’10, POPL’12),Ahmed, Birkedal, Hur, Neis, Rossberg, Vafeiadis and Ihave explored the semantic underpinnings of this
But what about hiding of non-monotonic state?What does that even mean?
We need a story about hiding in a substructural setting!
Example of non-monotonic state
Specification of realistic memory allocator:Internal invariant A depends on set of allocated locations
∃A : LocSet→ Type.
init cap : A(∅)⊗ malloc : !∀L : LocSet. A(L) (
∃X : Loc. ptr X ⊗ cap X 1⊗ A(L ] {X})⊗ free : !∀L : LocSet. ∀X : Loc.
ptr X ⊗ cap X 1⊗ A(L ] {X}) ( A(L)
Problem: Interface pollution for clientsA client must thread the “capability” A(L) through itsinterface to guard against interference from other clients
Superficially substructural types(Submitted – Joint work with Krishnaswami, Turon, Garg)
We propose a new sharing rule:Enables A(L) to be split into “fictionally disjoint” pieces,so clients can be oblivious to one another’s existence
split : ∀L1, L2 : LocSet. A(L1 ] L2) ( A(L1)⊗ A(L2)join : ∀L1, L2 : LocSet. A(L1)⊗ A(L2) ( A(L1 ] L2)
This can be done for any commutative monoid!Each ADT can pick whatever monoid it wants,so long as all its operations are frame-preserving
Builds on Birkedal et al.’s work on separation logic andDreyer et al.’s Kripke logical relations for SML
ML-style “hidden state” supported as a mode of use
But several problems remain. . .
Dealing with unsafe reentrancyCurrently, we use locks to forbid reentrancy,but this will not scale well to concurrency
Possible idea: synthesize with Pottier’s anti-frame rule
Scaling to full-blown dependent typesWe have shown soundness of the sharing rule in thecontext of a hybrid of Dependent ML and L3
But we want to scale to an HTT-like language
Doing strictly better than SML Modules is hard!
Fond memories of Chicago. . .
Thanks, Dave. . .
. . . for being such a tough act to follow!
Thanks, Dave. . .
. . . for being such a tough act to follow!