Function-Passing Style -...

38
Heather Miller @heathercmiller Function-Passing Style Typed, Distributed Functional Programming Philipp Haller @philippkhaller EPFL KTH Royal Institute of Technology Scala Days 2015, San Francisco, CA, USA. March 17, 2015

Transcript of Function-Passing Style -...

Page 1: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Heather Miller@heathercmiller

Function-Passing StyleTyped, Distributed Functional Programming

Philipp Haller@philippkhaller

EPFL KTH Royal Institute of Technology

Scala Days 2015, San Francisco, CA, USA. March 17, 2015

Page 2: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

…been working onLanguage support for distributed system builders.

SerializationThat happens mostly at compile time, so it’s performant.

Type classes to allow users to serialize to different formats (binary, JSON, etc)

1.

Page 3: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

…been working onLanguage support for distributed system builders.

SporesPortable (serializable) closures.Type constraints to restrict what they capture

2.

Page 4: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

In this talk…A programming model.

Builds on the basis of serializable functionsto provide a substrate that distributed systems can be built upon

Page 5: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

In this talk…A programming model.The result…the model greatly simplifies the design and implementation of mechanisms for:

Fault-toleranceIn-memory cachingDebugging(i.e., pushing types into

more layers of the stack)

IN A CLEAN & FUNCTIONAL WAY.(STATELESS!)

Page 6: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Note:Currently a research project. Thus, all aspects of it are under development + publication in the works.

(Thanks, )

Page 7: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

A DUAL WHICH NICELY

COMPLEMENTS ACTORS!

Inversion of the actor model.Can be thought of as a dual to actors.

FUNDAMENTALIDEA:

Page 8: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Um,HOW?

Page 9: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Actors…Encapsulate state and behavior.

Actors exchange data/commands through asynchronous messaging.

Are stationary.

Page 10: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-passing…Stateless. Built on persistent data structures.

Functions are exchanged through asynchronous messaging.

Keep the data stationary.

Of note:This is a model for programming with data and not a new model of concurrent processes like actors. !Instead, we provide a new means of working with distributed data in a functional way.

Page 11: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

2

A Note on Distributed Computing

Jim Waldo, Geoff Wyant, Ann Wollrath, and Sam Kendall

Sun Microsystems Laboratories2550 Garcia Avenue

Mountain View, CA 94043

1 Introduction

Much of the current work in distributed, object-orientedsystems is based on the assumption that objects form a sin-gle ontological class. This class consists of all entities thatcan be fully described by the specification of the set ofinterfaces supported by the object and the semantics of theoperations in those interfaces. The class includes objectsthat share a single address space, objects that are in sepa-rate address spaces on the same machine, and objects thatare in separate address spaces on different machines (with,perhaps, different architectures). On the view that allobjects are essentially the same kind of entity, these differ-ences in relative location are merely an aspect of theimplementation of the object. Indeed, the location of anobject may change over time, as an object migrates fromone machine to another or the implementation of theobject changes.

It is the thesis of this note that this unified view of objectsis mistaken. There are fundamental differences betweenthe interactions of distributed objects and the interactionsof non-distributed objects. Further, work in distributedobject-oriented systems that is based on a model thatignores or denies these differences is doomed to failure,and could easily lead to an industry-wide rejection of thenotion of distributed object-based systems.

1.1 Terminology

In what follows, we will talk about local and distributedcomputing. By local computing (local object invocation,etc.), we mean programs that are confined to a singleaddress space. In contrast, we will use the term distributedcomputing (remote object invocation, etc.) to refer to pro-grams that make calls to other address spaces, possibly onanother machine. In the case of distributed computing,nothing is known about the recipient of the call (other thanthat it supports a particular interface). For example, theclient of such a distributed object does not know the hard-ware architecture on which the recipient of the call is run-ning, or the language in which the recipient wasimplemented.

Given the above characterizations of “local” and “distrib-uted” computing, the categories are not exhaustive. Thereis a middle ground, in which calls are made from oneaddress space to another but in which some characteristicsof the called object are known. An important class of thissort consists of calls from one address space to another onthe same machine; we will discuss these later in the paper.

Page 12: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

2

A Note on Distributed Computing

Jim Waldo, Geoff Wyant, Ann Wollrath, and Sam Kendall

Sun Microsystems Laboratories2550 Garcia Avenue

Mountain View, CA 94043

1 Introduction

Much of the current work in distributed, object-orientedsystems is based on the assumption that objects form a sin-gle ontological class. This class consists of all entities thatcan be fully described by the specification of the set ofinterfaces supported by the object and the semantics of theoperations in those interfaces. The class includes objectsthat share a single address space, objects that are in sepa-rate address spaces on the same machine, and objects thatare in separate address spaces on different machines (with,perhaps, different architectures). On the view that allobjects are essentially the same kind of entity, these differ-ences in relative location are merely an aspect of theimplementation of the object. Indeed, the location of anobject may change over time, as an object migrates fromone machine to another or the implementation of theobject changes.

It is the thesis of this note that this unified view of objectsis mistaken. There are fundamental differences betweenthe interactions of distributed objects and the interactionsof non-distributed objects. Further, work in distributedobject-oriented systems that is based on a model thatignores or denies these differences is doomed to failure,and could easily lead to an industry-wide rejection of thenotion of distributed object-based systems.

1.1 Terminology

In what follows, we will talk about local and distributedcomputing. By local computing (local object invocation,etc.), we mean programs that are confined to a singleaddress space. In contrast, we will use the term distributedcomputing (remote object invocation, etc.) to refer to pro-grams that make calls to other address spaces, possibly onanother machine. In the case of distributed computing,nothing is known about the recipient of the call (other thanthat it supports a particular interface). For example, theclient of such a distributed object does not know the hard-ware architecture on which the recipient of the call is run-ning, or the language in which the recipient wasimplemented.

Given the above characterizations of “local” and “distrib-uted” computing, the categories are not exhaustive. Thereis a middle ground, in which calls are made from oneaddress space to another but in which some characteristicsof the called object are known. An important class of thissort consists of calls from one address space to another onthe same machine; we will discuss these later in the paper.

A better approach is to accept that there are irreconcilable differences between local and distributed computing, and to be conscious of those differences at all stages of the design and implementation of distributed applications. Rather than trying to merge local and remote objects, engineers need to be constantly reminded of the differences between the two, and know when it is appropriate to use each kind of object.

“”

Differences in latency, memory access, partial failure, and concurrency make merging of the computational models of local and distributed computing both unwise to attempt and unable to succeed.

“”

Page 13: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

So,WHAT DOESIT LOOKLIKE?

Page 14: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Stationary, immutable data.1.Portable functions – move the functionality to the data.2.

Two concepts:Silos

Spores

(for a lack of a better name)

(illustrated)

Page 15: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Two concepts:

Function-Passing(illustrated)Model

the

Portable functions – move the functionality to the data.2.

Spores

(for a lack of a better name)

Stationary, immutable data.1. Silos

Page 16: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

(illustrated)

Page 17: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Silo SiloRef

MasterWorker

Silos don’t have toall live on the same node( )

Page 18: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Silo[T]

(illustrated)

T

WHAT ARE THEY?Silos.

SiloRef[T]

(The workhorse.)The handle to a Silo.

Takes two spores: framework logic (combinator), e.g. mapuser/application-provided argument function

Defers application of fn to silo, returns SiloRef with info for later materialization of silo.

def  apply(s1:  Spore,  s2:  Spore):  SiloRef[T]

LAZY!

def  applydef  senddef  flatMap

Page 19: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Silo[T]

(illustrated)

T

WHAT ARE THEY?Silos.

SiloRef[T]

The handle to a Silo.

def  apply(s1:  Spore,  s2:  Spore):  SiloRef[T]

Sends info for function application and silo materialization to remote node

EAGER!

Asynchronous/nonblocking data transfer to local machine (via Future)

def  send():  Future[T]

def  applydef  senddef  flatMap

Page 20: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Silo[T]

(illustrated)

T

WHAT ARE THEY?Silos.

SiloRef[T]

The handle to a Silo.

def  apply(s1:  Spore,  s2:  Spore):  SiloRef[T]def  send():  Future[T]

def  applydef  senddef  flatMap

def  flatMap[S](spore:  Spore[T,  SiloRef[S]]):  SiloRef[S]

Sometimes you need to move data. !!

send spore to silo of the receiver SiloRef (this)

result of applying spore is a SiloRef whose contents should be made available in a new silo on the same node as this SiloRef

Page 21: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Silo[T]

(illustrated)

T

WHAT ARE THEY?Silos.

SiloRef[T]

The handle to a Silo.

def  apply(s1:  Spore,  s2:  Spore):  SiloRef[T]def  send():  Future[T]

def  applydef  senddef  flatMap

def  flatMap[S](spore:  Spore[T,  SiloRef[S]]):  SiloRef[S]

Sometimes you need to move data. !!

send spore to silo of the receiver SiloRef (this)

result of applying spore is a SiloRef whose contents should be made available in a new silo on the same node as this SiloRef

SiloRef of the new silo is returned

Note: needs to transfer data if the SiloRef that's the result of the spore is not on the same node already

Page 22: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Silo[T]

Machine 1 Machine 2

SiloRef[T]

(illustrated)

λT

SiloRef[S]

S

Silo[S]

) )T⇒S

Page 23: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-Passing

Silo

Modelthe

Silo

Silo

Machine 1

SiloSilo

Silo

Machine 2

(illustrated)

Page 24: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Stationary, immutable data.1.

Two concepts:Silos

(for a lack of a better name)

(illustrated)

Portable functions – move the functionality to the data.2.

Spores

Page 25: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

What do spores look like?Basic usage:val  s  =  spore  {      val  h  =  helper      (x:  Int)  =>  {          val  result  =  x  +  "  "  +  h.toString          println("The  result  is:  "  +  result)      }  }

THE BODY OF A SPORE CONSISTS OF 2 PARTS

2 a closure

a sequence of local value (val) declarations only (the “spore header”), and1

http://docs.scala-lang.org/sips/pending/spores.html

Page 26: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Spore

http://docs.scala-lang.org/sips/pending/spores.html

1. All captured variables are declared in the spore header, or using capture

2. The initializers of captured variables are executed once, upon creation of the spore

3. References to captured variables do not change during the spore’s execution

vsclosures( )A Guarantees...

Page 27: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Spores&

http://docs.scala-lang.org/sips/pending/spores.html

ClosuresEvaluation semantics:Remove the spore marker, and the code behaves as before

spores & closures are related:

You can write a full function literal and pass it to something that expects a spore. (Of course, only if the function literal satisfies the spore rules.)

Page 28: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

(illustrated)

Spores.Benefits:

environment (captured variables) is declared explicitly, and fixed at spore creation time.

can statically ensure that everything captured is serializable

6SRUHV� $ 7\SH�%DVHG )RXQGDWLRQ IRU &ORVXUHV LQ WKH

$JH RI &RQFXUUHQF\ DQG 'LVWULEXWLRQ

+HDWKHU 0LOOHU� 3KLOLSS +DOOHU1 � DQG 0DUWLQ 2GHUVN\

(3)/ DQG 7\SHVDIH� ,QF�1

Ƈ�������Ŝ������ř ������Ŝ�������

ƈɒ����Ŝ�� DQG �������Ŝ������ɒ��������Ŝ���

1

$EVWUDFW� )XQFWLRQDO SURJUDPPLQJ �)3� LV UHJXODUO\ WRXWHG DV WKH ZD\ IRUZDUG

IRU EULQJLQJ SDUDOOHO� FRQFXUUHQW� DQG GLVWULEXWHG SURJUDPPLQJ WR WKH PDLQVWUHDP�

7KH SRSXODULW\ RI WKH UDWLRQDOH EHKLQG WKLV YLHZSRLQW KDV

HYHQ OHG WR D QXPEHU RI

REMHFW�RULHQWHG �22� SURJUDPPLQJ ODQJXDJHV RXWVL

GH WKH 6PDOOWDON WUDGLWLRQ DGRSW�

LQJ IXQFWLRQDO IHDWXUHVVXFK DV ODPEGDV DQG WKHUHE\ IXQFWLRQ FORVXUHV� +RZHYHU

GHVSLWH WKLV HVWDEOLVKHG YLHZSRLQW RI )3 DV DQ HQDEOHU� UHOLDEO\ GLVWULEXWLQJ IXQF�

WLRQ FORVXUHV RYHU D QHWZRUN� RU XVLQJ WKHP LQ FRQFXUUHQW HQYLURQ

PHQWV QRQHWKH�

OHVV UHPDLQV D FKDOOHQJH DFURVV )3 DQG 22 ODQJXDJHV� 7KLV SD

SHU WDNHV D VWHS WR�

ZDUGV PRUH SULQFLSOHG GLVWULEXWHG DQG FRQFXUUHQW SURJUDPPLQJ E\ LQWURGXFLQJ D

QHZ FORVXUH�OLNH DEVWUDFWLRQ DQG W\SH V\VWHP

� FDOOHG VSRUHV� WKDW FDQ JXDUDQWHH FOR�

VXUHV WR EH VHULDOL]DEOH� WKUHDG�VDIH� RU HYHQ KDYH FXVWRP XVHU�GHILQHG SURSHUWLHV�

&UXFLDOO\� RXU V\VWHP LV EDVHG RQ WKH SULQFLSOH RI HQ

FRGLQJ W\SH LQIRUPDWLRQ FRU�

UHVSRQGLQJ WR FDSWXUHG YDULDEOHV LQ WKH W\SH RI D VSRUH� :H SURYH RXU W\SH V\

VWHP

VRXQG� LPSOHPHQW RXU DSSURDFK IRU 6FDOD� HYDOXDWHLWV SUDFWLFDOLW\ WKURXJK D VPDOO

HPSLULFDO VWXG\� DQG VKRZ WKH SRZHU RI WKHVH JXDUDQWHHV WKURXJKD FDVH DQDO\VLV

RI UHDO�ZRUOG GLVWULEXWHG DQG FRQFXUUHQW IUDPHZRUNV WKDW WKLV VDIH IRXQGDWLRQ IRU

FORVXUHV IDFLOLWDWHV�

.H\ZRUGV� FORVXUHV� IXQFWLRQV� GLVWULEXWHG SURJUDPPLQJ� FRQFXUUHQW SU

RJUDP�

PLQJ� W\SH V\VWHPV

� ,QWURGXFWLRQ

:LWK WKH JURZLQJ WUHQG WRZDUGV FORXG FR

PSXWLQJ� PRELOH DSSOLFDWLRQV�DQG ELJ GDWD� GLV�

WULEXWHG SURJUDPPLQJ KDV HQWHUHG WKH PDLQVWUHDP� 3RSXODU SDUDGLJPV LQ VRIWZDUH HQJL�

QHHULQJ VXFK DV VRIWZDUH DV D VHUYLFH �

6DD6�� 5(67IXO VHUYLFHV� RU WKH ULVH RI

D PXOWLWXGH

RI V\VWHPV IRU ELJ GDWD SURFHVVLQJ DQG LQWHUDFWLY

H DQDO\WLFV� HYLGHQFH WKLV WUHQG�

0HDQZKLOH� DW WKH VDPH WLPH� IXQFWLRQDO SURJU

DPPLQJ KDV EHHQ XQGHQLDEO\ JDLQLQJ

WUDFWLRQ LQ UHFHQW \HDUV� DV LV HYLGHQFHG E\ WKH RQJRLQJ WUHQG RI WUDGLWLRQDOO\ REMHFW�

RULHQWHG RU LPSHUDWLYH ODQJXDJHVEHLQJ H[WHQGHG ZLWK IXQFWLRQDO IHD

WXUHV� VXFK DV ODPE�

GDV LQ -DYD � >�@� &���� >�@� DQG 9LVXDO %DVLF � >��@� WKH SHUFHLYHGLPSRUWDQFH RI IXQF�

WLRQDO SURJUDPPLQJ LQ JHQHUDO HPSLULFDO VWXGLHV RQ VRIWZDUH GHYHORSHUV >��@� DQG WKH

SRSXODULW\ RI IXQFWLRQDO SURJUDPPLQJ PDVVLYHO\ RQOLQH RSH

Q FRXUVHV �022&V� >��@�

2QH UHDVRQ IRU WKH ULVH LQ SRSXODULW\ RI IXQFWLRQDO SURJUDPPLQJ ODQJXDJHV DQG IHD�

WXUHV ZLWKLQ REMHFW�RULHQWHG FRPPXQLWLHV LV WKH EDVLF SKLORVRSK\ RI WUDQVIRUPLQJ LP�

PXWDEOH GDWD E\ DSSO\LQJ ILUVW�FODVV IXQFWLRQV� DQG WKH REVHUYDWLRQ WKDW WKLV IXQFWLRQDO

VW\OH VLPSOLILHV UHDVRQLQJ DERXW GDWD LQ SDUDOOHO�

FRQFXUUHQW� DQG GLVWULEXWHG FRGH� $ SRS�

XODU DQG ZHOO�XQGHUVWRRG H[DPSOH RI WKLV VW\OH RI

SURJUDPPLQJ IRU ZKLFK PDQ\ SRSXODU

ECOOP’14

http://docs.scala-lang.org/sips/pending/spores.html& A SIP:

Page 29: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

PLEASE. EXAMPLE,

Ok,

Page 30: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

map f

.apply

map (_*2)

SiloRef[List[Int]]

Function-PassingModel

the

(illustrated)

reduce (_+_)

SiloRef[Int]SiloRef[List[Int]]SiloRef[List[Int]]

Distributed List with operations map and reduce.EXAMPLE:

(This is what would be happening under the hood) (Spores)

.apply .send().apply

Page 31: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

(illustrated)Distributed List with operations map and reduce.EXAMPLE:

(This is what would be happening under the hood)

map f

.apply

map (_*2)

SiloRef[List[Int]]

reduce (_+_)

SiloRef[Int]SiloRef[List[Int]]SiloRef[List[Int]]

.apply .send()

Page 32: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

map f

.apply

map (_*2)

SiloRef[List[Int]]

reduce (_+_)

SiloRef[Int]SiloRef[List[Int]]SiloRef[List[Int]]

.apply .send()

Machine 1

Silo[List[Int]]

Machine 2

List[Int]

λ

Silo[Int]

Int

Silo[List[Int]]

List[Int]

Silo[List[Int]]

List[Int]Int

Page 33: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

HOW DOESOk,

FAULT TOLERANCE?

THISHELPWITH

Page 34: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Data in silos easily

The persistent data structure is based on the chain of operations to derive the data of each silo.

Since the lineage is composed of spores, it’s serialized. This means it can be persisted or transferred to other machine.

Thus, traversing the silo data structures yields the complete lineage of a silo.

Silos and SiloRefs relate to each other by means of a persistent data structure

reconstructed:

Page 35: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

SUMMARY,

All operations, including operations provided by system builders, are spores – so, serializable!

Data (Silos) managed using persistent data structure.

Taken together:

in

A lot simpler to build mechanisms for fault tolerance!

Page 36: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Function-PassingModel

the

Paper draft:

http://infoscience.epfl.ch/record/205822

Page 37: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

[email protected]

Looking for feedback,demanding use cases, contributors.

Get involved!

Thank you!

Page 38: Function-Passing Style - Lightbenddownloads.typesafe.com/.../ScalaDaysSF2015/T4_Miller_Function-Pa… · Function-Passing Style Typed, Distributed Functional Programming Philipp Haller

Can’t I just…

However, spores + silos additionally provides a lot of benefits that actors + spores alone do not provide out-of-the-box:

Granted, that’s already quite powerful. :-)

Send spores within messages between actors/processes?

Benefits:deferred evaluation (laziness) enables optimizations to reduce intermediate results.

statelessness + lineages simplifies the implementation of mechanisms for fault tolerance for certain applications (think dist. collections)