Migrating to Scala 2.13Julien Richard-Foy, Scala Center

Stefan Zeiger, Lightbend

Scala 2.13

• Simplifying the collections• Compiler performance• Modularizing the standard library• User-friendliness

https://github.com/scala/scala-dev/issues/324

Scala 2.13 Roadmap

• Scala 2.13.0-M4 to be released shortly• First milestone with the new collections library

• Final milestone release: M5• Minor API changes, bug fixes and performance improvements

Current Status

• Project that works on Scala 2.12• Targeting Scala 2.13.0-M4 or higher• sbt build (for examples shown)

Starting Point

Prepare for migration

• Remove deprecated calls• Many deprecated APIs were removed in 2.13 (e.g. JavaConversions)• A clean build without deprecations on 2.12 makes the migration easier

• build.sbt:scalacOptions in Compile += "-deprecation"

Clean up on 2.12: Deprecations

• Compiler flags that modify the language are being removed in 2.13• Only -Xsource from now on• Compile without the following flags:

• -Yno-adapted-args• -Xstrict-inference• -Xfull-lubs• -Yoverride-objects• -Yoverride-vars• -Yinfer-argument-types• -Yvirtpatmat

• See https://github.com/scala/scala/pull/6505

Clean up on 2.12: Compiler flags

• Argument adaptation cannot be turned off selectively any more• -Yno-adapted-args removed

• Dotty has more restrictive adaptation• Scala 2.14 will align with Dotty rules• Automatic eta-expansion of zero-arg methods already disabled in 2.13

under -Xsource:2.14 (see https://github.com/scala/scala/pull/6475)

Clean up on 2.12: Argument adaptation / Auto-tupling

• scala-library-all has been removed• Depend on the required modules individually

• Parallel collections have been moved into a separate module• Add a dependency to scala-parallel-collections on 2.13 and see

https://github.com/scala/scala-parallel-collections/issues/22 for cross-building

• scala-xml is no longer a transitive dependency of scala-compiler• Depend on it directly if necessary

Clean up on 2.12: Modules

Build on 2.13

crossScalaVersions := Seq("2.13.0-M4-pre-20d3c21", "2.12.6")

scalaVersion := crossScalaVersions.value.head

Cross-building: Basic setup

scalacOptions ++=(CrossVersion.partialVersion(scalaVersion.value) match {case Some((2, n)) if n >= 13 => Seq("-Xsource:2.14")case _ => Seq("-Yno-adapted-args")

})

Cross-building: Compiler options

• Expect some breakage due to the new collections library• Singleton types and minor type inference changes can influence types and

implicit resolution in corner cases• Other small changes in source compatibility (see https://github.com/scala/scala-

dev/issues/470)• Try -Xsource:2.12

Build on 2.13

• The new collections library is mostly source compatible• Many old methods and types are deprecated

• e.g. Traversable, TraversableOnce, Stream, ...• Some features could not be added back in a 2.12-compatible way• Instead scala-collection-compat provides the 2.13 syntax on 2.11 & 2.12• https://github.com/scala/scala-collection-compat/

scala-collection-compat

• build.sbt:libraryDependencies +="org.scala-lang" %% "scala-collection-compat" % "0.1"

• Old Scala code:xs.to[List]

• New Scala code:import scala.collection.compat._xs.to(List)

scala-collection-compat: Example

Not yet available

New collections

• This is not the topic of this talk, but in short:

• Simpler user-facing API (no CanBuildFrom)• Correct operation implementations for non-strict collections• Simpler internal hierarchy (no Gen… types)

Goals of the collections redesign

• scala.Seq is now scala.collection.immutable.Seq• Same for IndexedSeq• Consistent with Set and Map• Decide on a case-by-case basis which one to use• Use s.c.Seq or s.c.i.Seq explicitly when cross-building

Major collection incompatibilities: Immutable Seq

• Varargs use scala.Seq, so also immutable now• Wrapped Java varargs pretend the array is immutable

• Using the new s.c.i.ArraySeq• You can do the same with ArraySeq.unsafeWrapArray• A deprecated implicit conversion makes a copy of the array

Major collection incompatibilities: Immutable Seq

method copyArrayToImmutableIndexedSeq in class LowPriorityImplicits2 is deprecated (since 2.13.0): Implicit conversions from Array to immutable.IndexedSeq are implemented

by copying; Use the more efficient non-copying ArraySeq.unsafeWrapArray or an explicit toIndexedSeq call

• CanBuildFrom replaced by BuildFrom• BuildFrom requires an instance of the source collection:

trait BuildFrom[-From, -A, +C] {def fromSpecificIterable(from: From)(it: Iterable[A]): Cdef newBuilder(from: From): Builder[A, C]

}

• Used in methods like Future.sequence• Collection methods like flatMap no longer need it

• Overloaded to produce different results depending on source collection type

Major collection incompatibilities: CanBuildFrom

• Factory allows target type-driven building (like CanBuild in 2.12)• Used in Iterable.to

• Like BuildFrom but does not use a source collection:trait Factory[-A, +C] {def fromSpecific(it: IterableOnce[A]): Cdef newBuilder: Builder[A, C]

}

• General rule: Use BuildFrom to rebuild with the best matching type of an existing source collection, otherwise use Factory

Major collection incompatibilities: CanBuildFrom

• Without CanBuildFrom, there is no breakOut• Use an Iterator: xs.iterator.map(…).to(Vector)

• Methods with BuildFrom or Factory can be called with a companion object instead (via implicit conversion to BuildFrom / Factory):• Old:

val xs: List[Future[Int]] = …Future.sequence(xs)(breakOut, implicitly):Future[Vector[Int]]

• New:val xs: List[Future[Int]] = …Future.sequence(xs)(Vector, implicitly)

Major collection incompatibilities: breakOut

Doesn't actually work in 2.12

• Views in the new collections are reified Iterator operations• View vs Iterator is similar to Stream vs Spliterator in Java 8

collections• Views don't remember source collection types

• Use an explicit to… operation instead of force to build the desired type• mapValues and filterKeys on Map now return MapView

• It was always a lazy View-like object but pretended to be a Map• Add .toMap if necessary

Major collection incompatibilities: Views

• Only very simple collection implementations are source compatible• General rule: Use two different source files in different source directories to

cross-build• Type hierarchy simplified• Less boilerplate• All methods have alphabetic names, symbolic operators are aliases

• E.g. override concat instead of ++• Overloading instead of CanBuildFrom

Major collection incompatibilities: Custom collections

crossScalaVersions := Seq("2.13.0-M4-pre-20d3c21", "2.12.6")

scalaVersion := crossScalaVersions.value.head

// Add src/main/scala-2.13+ for Scala 2.13 and newer// and src/main/scala-2.12- for Scala versions older than 2.13unmanagedSourceDirectories in Compile += {

val sourceDir = (sourceDirectory in Compile).valueCrossVersion.partialVersion(scalaVersion.value) match {

case Some((2, n)) if n >= 13 => sourceDir / "scala-2.13+"case _ => sourceDir / "scala-2.12-"

}}

Cross-building: Source directories

sbt already gives you:• scala• scala-2.12• scala-2.13.0-M4-pre-20d3c21

• See https://github.com/scala/collection-strawman/wiki/FAQ for a more comprehensive list

Major collection incompatibilities

Automated migration

2.12 2.13Scalafix

manual work

Migrating an application Rules under development –more in M5

• plugins.sbt:addSbtPlugin("ch.epfl.scala" % "sbt-scalafix" % "0.5.7")

• From your shell:$ sbt> scalafixEnable> scalafix github:scala/scala-collection-compat/NewCollections

• build.sbt:scalaVersion := "2.13.0"

• And then:> ;reload ;compile

• Documentation: https://github.com/scala/scala-collection-compat

Running the migration rulesStep 1: Add sbt-scalafix to

your project

Step 2: Run the migration rule

Step 3: Update the scalaVersion

• Renamings• Stream(1, 2, 3).append(Stream(4, 5, 6))• LazyList(1, 2, 3).lazyAppendedAll(LazyList(4, 5, 6))

• Expression rewritings• xs.copyToBuffer(b)• b ++= xs

• The more complex the expression, the harder to implement the rewrite rule

Scope of the migration rulesHas a lazy head

• Custom collection implementations• Advanced usage of CanBuildFrom• scala.Seq usage

• Comprehensive list of supported rewrites:https://github.com/scala/collection-strawman/wiki/FAQ

Not in the scope of the migration

2.112.12

2.112.122.13

Scalafix

scala-collection-compat

manual work

Migrating a library Rules not yet implemented

• Draft of "The Architecture of Scala Collections" for 2.13: https://github.com/scala/docs.scala-lang/pull/960

• Demo project for cross-building:https://github.com/szeiger/new-collections-demo

• FAQ: https://github.com/scala/collection-strawman/wiki/FAQ• Previous talks on the new collections:• The new collections library for Scala 2.13 and Dotty• The next() collections

• @julienrf• @StefanZeiger

Links