Scala Days NYC 2016

Scala

The Road Ahead

Martin Odersky

Scala Days 2016, NYC

2015 was on the quiet side•  Maturing tools: 2.11.x, IDEs, sbt

•  Steady growth

indeed.com jobs google trends

In 2016, things are heating up again

•  The Scala Center

•  Scala 2.12 release

•  Rethinking the Scala libraries

•  New target platforms

•  DOT and dotty

A New Steward for Scala-  Undertake projects that benefit all of the Scala

community.-  Industrial support by Lightbend, Goldman Sachs,

Nitro, IBM, 47 Degrees, Verizon.-  More in Heather Miller’s Keynote tomorrow.

Scala 2.12Optimized for Java 8

•  Uses Java 8’s lambdas and default methods

•  for shorter code and faster execution speed.

Projected release date: mid 2016.

In case you are still on Java 6/7, Scala 2.11 will be around for a while.

33 features in release notes

2.12 Data33 features in release notes

336 PRs closed (40 more to go)

65 committers177 LukasRytz76 JasonZaugg70 AdriaanMoors33 A.P.Marki25 FelixMulder17 SimonOchsenreither14 JanekBogucki14 RexKerr12 SébastienDoeraene

And, finally:•  Programming in Scala

3rd edition is out.

•  Updated for 2.12.

•  Book signing Tuesday lunch 12.20-13.20 atthe Artima booth.

Beyond 2.12Scala 2.13 will focus on the librariesPlans to revamp collections

•  Even simpler to use

• More in line with Spark usage

•  Better lazy collections (views)

Current Status:•  Strawman proposals under study.

• Would like to have more of them,

Beyond 2.12Scala 2.13 will focus on the libraries

Better modularization. We are thinking about a split:

Scalacore

Scalastdlib Scalaplatform

Your input and help is vital for this!

Scala.js 0.6.9•  “native” anonymus classes

•  js.TupleN

•  JUnit support

•  Faster code generation

COMING SOON

And then there is the

DOTWe finally have a proven foundation for Scala!

The DOT calculus talks about a minimal language subset, chosen so that

• we can make and prove formal statements about it

• we can encode much of the rest of the language in it.

This concludes an 8 year effort!

It opens the door to do language work with much better confidence than before.

DOT Terms•  Translated to Scala notation, the language

covered by DOT is:Value v = (x:T)=>t Function

new{x:T=>d} Object

Definitiond = defa=t Method definition typeA=T Type

Term t = v Value x Variable t1(t2) Application t.a Selection {valx=t1;t2} Local definition.

DOT TypesThe Types covered by DOT are:

Type T=Any Top type Nothing Bottom type x.A Selection (x:T1)=>T2 Function {defa:T} Method declaration {typeT>:T1<:T2} Type declaration T1&T2 Intersection {x=>T} Recursion

Type SoundnessThe following property was shown with a mechanized proof:

If a term t has type T,and evaluation of t terminates:

the result will be a value v of type T.

Why is this Important?It gives us a technique to reason about correctness of other language features.

dottydotty is working name for our new Scala compiler.

•  Builds on DOT in its internal data structures.

• Generics get expressed as type members.

•  Supports an evolution of the Scala programming language.

•  A first developer preview is around the corner

•  Targeted at contributors and experimenters.

dotty – Technical DataA bit more than half the size of the current Scala compiler, nsc.

•  dotty: 45 KLoc

•  nsc: 75 KLoc

•  About twice the speed of nsc.•  should improve significantly in the future.

dotty Architecture

dotty Frontend

dotty Transforms

GenBCode

Classfiles

Simplified AST

AST

Scala Sources

nsc

Scala Sources

TASTY Pickled

nsc Transforms

Simplified AST

AST

Evolving The LanguageMy Goal:

Make Scala “best” programming language I know to how make.

There is not agreement on what is “best”, so all we can really do is try to find a local optimum.

But finding that optimum is what drives me.

Essential ElementsConceptually we play with:

•  Functions

• Classes and objects,

•  Strict evaluation.

•  Local type inference

•  Implicits.

Many languages are currently moving in that space

Goals

1.  Deepen the synthesis of functional programming and modular programming.

2.  Improve the connection of Scala with its theoretical foundations.

3.  Improve the guarantees of type system.

4.  But stay simple and approachable.

A Whirlwind Tour Of Dotty

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers

General Type Projection

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


defrun(){...}Will be rewritten automatically to defrun():Unit={...}

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


classFooextendsDelayedInit

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


(the reflection based kind)defm(...)=macroimpl(...)

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


classCextends{valx=e}withDUse trait parameters instead

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


C[U]forSome{typeU}

Wildcards C[_]still supported.

Dropped Features

DelayedInit

Macros

Existential Types

Procedure Syntax

Early Initializers


T#X

- Was shown to be unsound for general types T. - Projection C#X from class types C still available.

Implemented New Features

Multiversal Equality

Intersection TypesUnion types

Trait parameters

Function arity adaptation

Named type parameters

@static methods and fieldsnon-blocking lazy vals




Trait parameters



T&U - replaces TwithU - is commutative





Trait parametersNamed type parameters

T|U avoids huge lubs





Trait parameters



pairs.map((x,y)=>x+y)

instead of

pairs.map{case(x,y)=>x+y}





Trait parameters




traitT(x:Int){...}




Trait parameters




objectO{@staticvalx=...@staticdeff()=...}




Trait parameters




lazyvalx=...// thread-local@volatilelazyvalx-...// thread-safe, // avoids dead-locks




Trait parameters




type-safe==,!=




Trait parameters



traitMap[typeKey,typeValue]Map[Key=Int]

allows partial type parameterization: - specify some types, - leave others to be inferred





Trait parameters



Motivation

•  Better foundations•  Easier, safer to use•  More orthogonal


Trait or Class?

Classes and traits now have largely the same capabilities.

Rule of thumb:•  When it's fully defined, make it a class

•  When it's abstract, make it a trait

Abstract classes are retained mainly for Java interop and for optimization.

Improvements in Detail•  Type System

•  Strongly influenced by DOT

•  Better integration of type refinements

•  Type Inference•  Subtyping constraint

solver

•  Inference is simpler to specify

•  Implicit Search•  Faster search

algorithm.

•  Better behaved for contravariant types.

•  Value Classes• Nested value classes

•  Arrays of value classes.

Who’s working on all this?•  So far, mostly team at EPFL, including

• Dmitry Petrashko @darkdimius

• Guillaume Martres @smarter

•  Vladimir Nikolayev @vladimirnik

•  Felix Mulder @felixmulder

•  Scala team at Lightbend helps with infrastructure, reviews and suggestions.

•  Your contribution would be very much appreciated!

•  Without it, nothing much will happen.

Advances in Tooling•  SBT integration

•  Basic integration exists

• We are working on incremental compilation.

•  REPL• With syntax highlighting!

•  IDEs•  Jetbrains working on

IntelliJ plugin.

•  Doc generation• Dynamic hyperlinks for

cross-linking libraries.

•  Linker•  A whole-program

analyzer/opimizer• Uses TASTY for

serialization• Makes specialization

cheaper and more robust

And the Future?

Planned In Future Releases

scala.meta

Generic Programming

Better records

Implicit Function Types

null safety

Effect System


scala.meta

Generic Programming

Better records


null safety

Effect Systeminlinedefm(inlinex:Int,y:Float):Float=meta{...}-inline for inlining, meta for meta-programming.-run by an interpreter (no reflection)-meta uses quasi quotes for matching and construction-blackbox and annotation macros

Why the change?-Simpler-Fewer implementation dependencies -Safer, since interpretation allows sandboxing-Restrict syntactic freedom, since no whitebox macros.


scala.meta

Generic Programming

Better records


null safety

Effect SystemtypeCtxS=implicitContext=>Sdeff(x:T):CtxS={...implicitly[Context]...}f(e)//Contextargumentispassedimplicitly

Why the change?-Small as it looks, this is huge-Allows abstraction over implicit parameters-Eliminates boilerplate


scala.meta

Generic Programming

Better records


null safety

Effect System

Effects are implicit capabilities

Two function arrows:

A=>B (impure)A->B (pure)

Why the change?-Effect checking is very much in demand.-I believe we can do better than monads to achieve it.-Implicits are a natural fit.


scala.meta

Generic Programming

Better records


null safety

Effect System

Model nullable types as a union types

T?=T|Null-Types don’t have null by default-Values coming from Java get a “?”. E.g. System.out:PrintStream?-nulldereferencing is an effect.-System.out.println()is ok in impure code.

Why the change?-nullposes unsoundness problems.-Null-safety is conceptually easy once you have union types and effects.


scala.meta

Generic Programming

Better records


null safety

Effect System

“Scrap Your Boilerplate”-Product-of-sum interpretation of case class hierarchies-Tuples are HLists, but implemented more efficiently

(S,T,U)=(S,(T,(U,Unit)))-Tuple22/Product22/Function22will be history.

Why the change?-Abstract over arity.-Define behavior for ADTs in a generic way


scala.meta

Generic Programming

Better records


null safety

Effect System

-Analogue of tuples, but with labels.-Should integrate with generic programming-Should be implementable by a HashMap instead of by reflection.

Why the change?

-Existing structural types are not very useful.-Need something more flexible, and hopefully simpler.

What About Guard Rails?•  Scala’s premise: Trust devs to do the right thing.

•  But what if they don’t?

•  And can we even agree what the right thing is?

First Advice: Read this

ContainmentProblem: Even if our code base uses the principle of least power, we do not know whether the libraries that we depend on do the same.

E.g. library X might define an implicit conversion

implicitdefi2s(x:Int):String

(patterns like this are more common in practice than you’d like to think)

So instead of least power you get maximal surprise.

A Modest ProposalFor a conversion

implicitdefc(x:A):B

make it a style error if the conversion -  is not defined in one of the packages containing A or B.

-  is itself public (visible outside its package).

When using a library defining such conversions, warn on the client side as well.

Flexibility: Blessing or Curse?Scala has great syntactic flexibility.

Sometimes this can be a burden.

Example: Should I writexs.map(f) or xsmapf?

What aboutxs.map(f).filter(p) vs xsmapffilterp

x.min(y) vs xminy ?

Another Modest Proposal•  Add an annotation @infixthat indicates that an

operator is supposed to be used infix.@infixdefmin(y:Int):Int

•  Make it a style error if an operator is used in the wrong mode.xminy // ok

xsmapf // warning: Infix usage of normal method map

Symbolic Operators•  If the operator is symbolic, require in addition an

alphanumeric alias:

@infix(“append”)def+=(elem:T)

•  Make it a style error if the alias is missing

à  One knows how to pronounce the operator

à  It’s easier to search for it, e.g. “Scala append operator” as a search term.

•  To be worked out: What are the precise rules for style errors?

Thank You

Scala Days NYC 2016

Software

Transcript of Scala Days NYC 2016