Scala - where are are - where we plan to go
description
Transcript of Scala - where are are - where we plan to go
Scala- where are are
- where we plan to go
Martin Odersky
Welcome to ScalaDays!
• 155 attendees, coming from all corners of the planet.• We are packed. Sadly, many more people could not be
admitted for lack of space.• This shows the drive and energy of this community!• The Scala team is looking forward to talk with you and
learn from you during the next two days.• We are particularly glad to have you here at EPFL, the
place where Scala was born.
History (incomplete)
2001 Scala design begins, as a more practicalsuccessor to Funnel.
2003 First experimental release, first coursetaught with it.
2004 Scalable Component Abstractions paper.2005 Scala 2.0, written in Scala.2006 Industrial adoption starts.2007 First Lift release.2008 First Scala LiftOff, Twitter adopts Scala.2009 Big increase in adoption, IDEs mature.2010 First ScalaDays.
The main idea behind Scala
Scala is a Unifier
Agile, with lightweight syntax
Object-Oriented Scala Functional
Safe and performant, with strong static typing
The Philosophy behind Scala
Put productivity and creativity back in the hands of developers.
“Joy is underrated as a metric for a language’s potential success in a development organization.” a3lx@twitter
- address professional developers
- trust them & value their expertise
- (don’t tell them how they should do their job)
Some users in industry
Open Source Ecosystem
Many great projects, including:
lift (David P. & friends)akka (Jonas B.)kestrel, cachet, queroulous (Twitter)migrations (Sony)sbt (Mark H.)ScalaTest (Bill V. & friends)specs, ScalaCheck, ScalaModules, ScalaQL, ....
( the list goes on, many more tools & projects presented at this conference )
Now: 2.8
First release candidate Scala 2.8 RC1 is out!
This is primarily a consolidating release• Some new language features• More complete libraries• Redesigns of a few features that did not
work out well.• Big improvements in tooling.• Big improvements in stability.• Laying the groundwork for the future.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Named and Default Parameters
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
def newTable(size: Int = 100, load: Float = 0.5f) =
new HashMap[String, String]{
override val initialSize = size
override val loadFactor = (load * 1000).toInt
}
}
newTable(50, 0.75f)
newTable()
newTable(load = 0.5f)
newTable(size = 20, load = 0.5f)
Defaults can be arbitrary expressions, not just constants.Default evaluation is dynamic.
Copy OperationScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Problem: Case classes are great for pattern matching,but how do you transform data?
Often need to update (functionally) some specific field(s)of a date structure defined by case classes.
Solution: Selective copy method.
Q: How to define “copy”?
case class Tree[+T](elem: T, left: Tree[T], right: Tree[T])
def incElem(t: Tree[Int]) = t copy (elem = t.elem + 1)
Defining “copy”Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Problem: A class with n parameters allows 2n possible combinations of parameters to copy.
Default parameters to the rescue.
case class Tree[+T](elem: T, left: Tree[T], right: Tree[T]) {
// generated automatically:
def copy[U](elem: U = this.elem,
left: Tree[U] = this.left,
right: Tree[U] = this.right) =
Branch(elem, left, right)
}
Nested Annotations
• Scala 2.7 largely followed Java annotation syntax, but left out some of the more convoluted constructions. Nested annotations were not possible.
• Scala 2.8 throws out (deprecates) all special annotation syntax.
• Instead uses class constructors, which can have named parameters.
• This greatly simplifies the syntax, and allows all annotations to be expressed, including nested ones.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
@JoinTable( joinColumns = Array(new JoinColumn(name = "a name“)) var test: String = _
@specializedProblem: Generics cause a performance hit because they
require a uniform representation: everything needs to be boxed.
Example:
Scala 2.8
named and default parameters
copy operation
nested annotations
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
package objects
safer package nesting
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
trait Function1[-T, +U] { def apply(x: T): U }
def app(x: Int, fn: Int => Int) = fn(x)
app(7, x => x * x)Translatedby scalac 2.7 to: trait Function1 {
def apply(x: Object): Object }
class anonfun extends Function1 { def apply(x: Object): Object = Int.box(apply(Int.unbox(x))) def apply(x: Int): Int = x * x}
def app(x: Int, fn: Function1) = Int.unbox(fn(Int.box(x)))
app(7, new anonfun)
If Function1 is @specialized ...Scala 2.8
named and default parameters
copy operation
nested annotations
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
package objects
safer package nesting
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
trait Function1[@specialized -T, @specialized +U] { def apply(x: T): U }
def app(x: Int, fn: Int => Int) = fn(x)app(7, x => x * x)
... scalac 2.8 generates instead:
trait Function1 { def apply(x: Object): Object def apply$$II(x: Int): Int = Int.unbox(apply(Int.box(x))) // specializations for other primitive types}class anonfun extends (Int => Int) { def apply(x: Object): Object = Int.box(applyIntInt(Int.unbox(x))) override def apply$$II(x: Int): Int = x * x}
def app(x: Int, fn: Function1) = fn.apply$II(x)
app(7, anonfun)
Scala CollectionsScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Collection Properties:object-oriented
generic: List[T], Map[K, V]
optionally persistent, e.g. collection.immutable.Seq
higher-order, with methods such as foreach, map, filter.
Uniform return type principle: Operations return collections of the same type (constructor) as their left operand.
scala> val ys = List(1, 2, 3)ys: List[Int] = List(1, 2, 3)
scala> val xs: Seq[Int] = ysxs: Seq[Int] = List(1, 2, 3)
scala> xs map (_ + 1)res0: Seq[Int] = List(2, 3, 4)
scala> ys map (_ + 1)res1: List[Int] = List(2, 3, 4)
Old Collection StructureScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
trait Iterable[A] { def filter(p: A => Boolean): Iterable[A]
= ... def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Iterable[B] = ...}
trait Seq[A] extends Iterable[A] { def filter(p: A => Boolean): Seq[A] = ... override def partition(p: A => Boolean) = (filter(p(_)), filter(!p(_))) def map[B](f: A => B): Seq[B] = ...}
Types force duplication!
New Collection API
• Avoids type & code duplication through– uniform framework of traversers and builders.
– supported by: implicits, higher-kinded types
• Uniform package organization reflects mutability:– scala.collection– scala.collection.mutable– scala.collection.immutable
• Richer and more regular set of collection operations.
• New collection implementations: vectors and hash tries.
• More info: “Fighting bit rot with types”, FSTTCS09
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Vectors and Hash Tries
• Trees with branch factor of 32.• Persistent data structures with very efficient sequential
and random access.• Invented by Phil Bagwell, then adopted in Clojure.• New: Persistent prepend/append/update in constant
amortized time.• Next: Fast splits and joins for parallel transformations.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Package Objects
Motivation:• The new collection design reshuffled a lot of
classes.• For instance:
scala.List scala.collection.immutable.List
• How can we keep the old aliases around?• How can we migrate old to new?• Easy: put
type List = scala.collection.immutable.Listval List = scala.collection.immutable.List
into the scala package.• But how to we put a type alias and a val definition
into a package?
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Package ObjectsScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Problem: Everything has to live inside an object or class.=> No top-level value definitions, type aliases, implicits.
Package objects make packages more like objects.
They also let you define your own ``Predefs’’.
package object scala {
type List[+T] = scala.collection.immutable.List
val List = scala.collection.immutable.List
...
}
Safer Package Nesting
• Java knows only absolute packages• But Scala’s packages nest• This can lead to an impedance mismatch.• For instance:
package net.liftwebjava.lang.System.print(“hello”)
• This used to fail if net contains a subpackage java.• New rules: Intermediate packages only visible in
multiple package clauses.• To access net.java as java, you’d need to write
package netpackage liftweb...
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
New Treatment of ArraysScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
• Arrays were “magical” up to 2.8– sometimes primitive– sometimes wrapped– automatic boxing/unboxing conversions based
on static type.
• This could be confusing and slow.• 2.8 removes the magic.• Arrays are always Java arrays• Two implicit conversions make sure that
– arrays are compatible with sequences.– arrays support all sequence operations
• Strings are treated similarly to arrays.
Manifests
• Question: How to create an Array[T] where T is a type parameter?
• No magic allowed!• Solution: supply the run-time type as a type
descriptor called a Manifest or ClassManifest:
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
def evens[T: ClassManifest](xs: Vector[T]): Array[T] = { val arr = new Array[T]((xs.length + 1) / 2) for (i <- 0 until xs.length by 2) arr(i / 2) = xs(i) arr}
def evens[T](xs: Vector[T]) (implicit ev: ClassManifest[T]): Array[T]
The first line is equivalent to:
Better ImplicitsBetter Type Inference
• Type inference for implicits has been refined.
• Previously: Infer type first, search implicit for inferred type afterwards.
• Now: Implicit search can drive type inference.
• This makes implicits more powerful.• Other advance: Type constructors of
higher-kinded types can now be inferred.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Delimited Continuations
• Continuations capture the return stack as a data structure.
• Delimited continuations capture only up to some enclosing scope.
• Lost of advanced control structures can be built on them, e.g:– coroutines– event-based I/O– reactive programming– lightweight actors
• Scala 2.8 supports delimited continuations in a compiler plugin.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
delimited continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Better IDEs
• The Scala plugins for Eclipse, IntelliJ, Netbeans all have matured a lot.
• The Eclipse and Netbeans plugins are now based on completely redesigned compiler interfaces for command completion, semantic highlighting and incremental building.
• If you have discounted IDEs before for Scala, maybe now is the time to try again!
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Docs for Advanced Types
Problem: Advanced statically typed APIs pose challenges for presentation and documentation.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Simpler Type Views
• How to explaindef map[B, To](f: A => B)
(implicit cbf: CanBuildFrom[Coll, B, To]): To
to a beginner?• Key observation: We can approximate the type of map.• For everyone but the most expert user
def map[B](f: A => B): Traversable[B] // in class Traversabledef map[B](f: A => B): Seq[B] // in class Seq, etc
is detailed enough.• These types are correct, they are just not as general as
the type that’s actually implemented.• These are documented in the new ScalaDoc as use
cases.
Scala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
New ScalaDocScala 2.8
named and default parameters
copy operation
nested annotations
package objects
safer package nesting
@specialized
new collection API
vectors
hash tries
new treatment of arrays and strings
manifests
better implicits
better type inference
continuations
packrat parsing
faster actors
improved Swing support
better IDEs
interactive compiler API
new scaladoc
better REPL
Why 2.8, not 3.0?
• Scala 2.8 is numerically a point release, but feels like a major release.
• So why did we not name it Scala 3.0?
• 2.8 has been 18 months in the making, and acquired more changes on the
way.
• Originally, we just wanted to rewrite collections.
• That took longer than expected.
• Meanwhile, a lot more additions and changes got included.
• But 2.8 was already announced, and referred to in books and other
documentation.
• In light of this, we judged it too confusing to change the version number in
mid-flight.
And now...
...what’s next?
Next: Scala on .NET
• Work started again to have a native Scala compiler on .NET with Visual Studio integration.
• Funded my Microsoft.• Stay tuned...
Next: Reliability and Contracts
• Lost of things happening around contracts and verification.
• Peter Müller: Scala verifier• Viktor Kuncak: Scala code synthesis from constraints• Our group: effects• Common project: ProgLab.NET
Next: Concurrency
• Actor encapsulation– How to ensure that actors do not share memory?– Solved by capability-based uniqueness type system.– Implemented in compiler plugin.– Paper in ECOOP 2010.
• Software Transactional Memory– CCSTM work in this conference
• Open question: What’s a good combination of actors and transactions?
Next: Parallelism
• PPP Challenge: How to make use of multi-cores, GPUs, clusters, clouds, to achieve better performance for normal applications?
• First step in this direction: Parallel collections.– will be integrated in the new collection API.
– based on persistent data structures with fast splits and joins.
• Longer-term vision: Embedded parallel DSLs.
• See keynote by Kunle Olokutun tomorrow.
Next: The Green House
Great work on Scala is springing up everywhere, no longer focussed just at EPFL.
Q: How can we integrate contributions?Tension: Want to get lots of exciting
new stuff contributed, yet maintain a stable and reliable distribution.
Planned Solution: three-tiered:Incubator Hosts collaborative Scala
projectsGreen house A “cutting edge” version of
the distribution with experimental features.
Trunk The stable distribution.
Thanks
• To the Scala team at EPFL:Lukas Rytz, Hubert Plociniczak, Iulian Dragos, Miguel Garcia, Gilles Dubochet, Philipp Haller, Aleksandar Prokopec, Antonio Cunei, Tiark Rompf, Donna Malayeri, Phil Bagwell, Adriaan Moors, Ingo Maier.
• To our external committers and frequent contributors:Paul Phillips, Miles Sabin, Ilya Sergey, Caoyuan Deng, James Matlik, Frank Teubler, Kevin Wright, Manohar Jonnalagedda, Pedro Furlanetto, Johannes Rudolph, Jason Zaugg, Seth Tisue, Ismael Juma, Mark Harrah, Colin Howe, Mirko Stocker, Spiros Tzavellas, Matt Russell, David Taylor
• To the community at large for your support, suggestions, encouragements.