Post on 01-Jan-2017
1
ModernizingScalaTheJourneytoJava8
JasonZaugg
2
3
OverviewScala2.12TraitEncodinginvokedynamicLambdaEncoding
4 .1
Scala2.12
4 .2
Scala2.12DropJava6/7supportExploitnewfeaturesinJVMImproveJavainteropNewcodegeneratorand-optimiseimplementation
NonGoals:newlanguagefeatures,majorlibrarychanges
4 .3
JavaSupport/TargetMatrixJava6 Java8 Java9
Scala2.11 ST S
Scala2.12.0 ST
Scala2.12.N ST S
4 .4
CodeGeneratorpreviously:AST->IR->optimizedIR->bytecodenow:AST->bytecode->optimizedbytecodeAOToptimization(ifenabled):
standardlocaloptimizations:jumpchain,dce,nullness,push-pop,redundantstoreetc.Scalaawarebox/unboxelimination(tuples,options)inlining(w/ScalaawareCHA)
4 .5
InteropScalalambdasnowfulfilanyfunctionalinterface(FI)ScalatraitswithconcretemethodscanbeimplementedbyJavaclasses
hence,FunctionNarenowFIs
4 .6
ConsumeJavaFunctionalAPIsUsefunctionalAPIsfromJavalibraries,e.g.j.u.streamscala> import java.util.stream._import java.util.stream._
scala> Stream.of("a", "b", "c").map[String](_.toUpperCase).findFirst.getres0: String = A
4 .7
ConsumeJavaFunctionalAPIsUsesitevarianceinStreamAPIdoesn'tplaynicelywithinference.scala> Stream.of("a", "b", "c").map(_.toUpperCase).findFirst.getres8: ?0 = A
Retrofitdeclarationsitevariance?scala> :powerscala> val List(i, o) = symbolOf[Function[_, _]].typeParamsi: $r.intp.global.Symbol = type To: $r.intp.global.Symbol = type R
scala> i.setFlag(Flags.CONTRAVARIANT); o.setFlag(Flags.COVARIANT)res6: o.type = type R
scala> Stream.of("a", "b", "c").map(_.toUpperCase).findFirst.getres8: String = A
4 .8
ExposeScalafunctionalAPIsJavalambdascanfulfilscala.FunctionNjshell> new scala.Some<String>("42").map(x -> x + "!!!").get()$1 ==> "42!!!"
4 .9
SmallerJARslambdaloversprofitthemostYMMV
5 .1
EncodingTraitswithoutthebloat
5 .2
TraitsCheatSheetclassesmayextendoneclass+manytraitstraitsmaycontainanythingthataclassdoes:
initialization,fields,methodbodiesnoconstructorparams(yet)
5 .3
2.11traitsTrait:
interfaceTincludingabstractaccessorsforfields
classT$classwithmethodbodies
Subclass:
implementinterfacemethodsbyforwardingtoimplclassfields+concreteaccessors
5 .4
2.11traitstrait T { def foo = 42; var v = 1 }class C extends T
Compileto:public interface T { int foo(); int v(); void v_$eq(int); }class T$class { static int foo(T $this) { return 42; } static void $init$(T) { T.v_eq(1); }}class C implements T { C() { T.$init$(this); } private int v; int v() { return v; } void v_eq(int v) { this.v = v } int foo() { return T$class.foo(this); }}
5 .5
2.12traits(naive)fieldencodingunchangedleavetraitmethodbodiesintheinterfacetraitconstructorasastaticmethodomitforwardersinclasses
5 .6
2.12traits(naive)trait T { def foo = 42 }class C extends T
Compileto:interface T { default int foo() { return 42; } }class C implements T
5 .7
Challenge:runtimedispatchclass C { def foo = "C" }trait T extends C { override def foo = "T" }class D extends Tnew D().foo // prints?
5 .8
Challenge:runtimedispatchJVMandScaladisagreeondispatchhereWeneedtoadda"traitforwarder"toDclass D extends T {/*synthetic*/ def foo = super[T].foo}
onlyaddincaseofdisagreement
5 .9
TraitforwarderstraitforwardersneedtonamearbitraryancestortraitJVMonlyallowsnamedparentsAddredundantparents?
5 .10
theparentofmyparentismyparentAddallancestorstoparentinterfacelistWhatcouldpossiblygowrong?!
5 .11
theparentofmyparentismyparentStartuptimeballoonedHierarchyVisitorinvm/classfile/defaultMethods.cppuncooperative
5 .12
Challenge:supercallswhataboutexplicitsupercalls?supercallsinclassesfollowJava'srestrictionsbutintraits,supercallscantargetanyancestor
5 .13
Challenge:supercallsSimplecase:trait T { def foo = 42 }trait U extends T { override def foo = -super[T].foo }class C extends U
Compileto:interface T { default int foo() { return 42; }}interface U extends T { default int foo() { return super.foo(); }}class C implements U
5 .14
superdispatchScalasuperrequiresstaticdispatchinvokespecialdispatchesvirtuallyLatentbuginScala2.11SD-143
5 .15
class A { def m = 1}
class B extends A { override def m = 2}
trait T extends A
class C extends B with T { override def m = super[T].m}
5 .16
class A { def m = 1}
class B extends A { override def m = 2}
trait T extends A
class C extends B with T { override def m = invokespecial T.m}
5 .17
class A { def m = 1}
class B extends A { override def m = 2}
trait T extends A
class C extends B with T { override def m = invokeplease A.m // }
5 .18
class A { def m = 1}
class B extends A { override def m = 2}
trait T extends A
class C extends B with T { override def m = invokeactually B.m // }
5 .19
super-dilemmaIndytotherescue?
bootstrapcouldunreflectSpecialwouldrequireelevatedpermissions
Or,addstaticaccessorsforalltraitmethods
wecanthenuseinvokestaticasbefore
can'ttargetJavadefaultsorclassmethods
introducelanguagerestrictionforsuchcases
5 .20
Cleanerstacks2.11:scala> trait T { def foo = throw null }; new T {}.foojava.lang.NullPointerException at T$class.foo(<console>:11) at $anon$1.foo(<console>:13) ... 32 elided
2.12:scala> trait T { def foo = throw null }; new T {}.foojava.lang.NullPointerException at T.foo(<console>:11) ... 30 elided
5 .21
TraitsRecapwe'veshedtraitimplclassesandmosttraitforwardersshallower,lesscruftycallstacksnon-privatemethodsstillrequirestaticaccessorsreconsidertheindy-supersolutiondownthetrack
6 .1
arriving(fashionablylate)at
theindyparty
6 .2
@PolymorphicSignaturescala> import java.lang.invoke._, MethodType._scala> val lookup = MethodHandles.lookup(); import lookup._
scala> findVirtual(classOf[Object], "hashCode", methodType(classOf[Int]))res0: java.lang.invoke.MethodHandle = MethodHandle(Object)int
scala> val hc: Int = res0.invoke(new Object)hc: Int = 1251285265
supportaddedin2.11series
6 .3
SymbolLiterals'foo // shorthand for scala.Symbol("foo")def x = f('blah)x eq q
Weusedtocacheinsyntheticstaticfieldprivatestaticsnotsupportedininterfacespublicise?indify!
6 .4
Structuralcalls(c: {def close(): Unit}) => c.close()
Compiledtoreflectivecalls+reflectioncacheagain,indifythecacheintothecallsitefuturework:dynalink
first,needtoteachitaboutInt.+etc
6 .5
indyliberationscalacbackendemitsindyfornewASTshape
notspecialcasedtousecasesabovemacros:transformapplication=>arbitraryAST(undocumented,experimental)librarylevelsupportforindy
6 .6
ScalamacroprimerExperimentalfeatureofScala2.{10,11,12}Clientcodelooks,feels,typecheckslikeanmethodcallMacroauthorwritesapairofmethods:
adeclarationwiththetypesignatureanimplthattransformstheAST
stdlibmacros:stringformatting,quasiquotesquasiquotemacroshorthandsASTde-/con-struction
6 .7
Example:linktimepatterncompilation
6 .8
patterns:ClientCode// pattern is compiled and cached when call site is linkedcompilePattern("foo.bar").matcher("foo!bar").matches
// bonus static error: Unclosed character classcompilePattern("abc[..")
6 .9
patterns:Macroclass impl(val c: Context) extends IndySupport { import ...
val bootstrapSym = typeOf[test.Bootstrap].companion .member(TermName("bootstrap")) val invokedType = NullaryMethodType(typeOf[Pattern])
def compilePattern(s: Tree): Tree = s match { case l@Literal(Constant(s: String)) => checkPattern(l.pos, s) Indy(bootstrapSym, List(l), invokedType) case _ => q"_root_.java.util.regex.Pattern.compile($s)" }}def compilePattern(s: String): Pattern = macro impl.compilePattern
6 .10
patterns:BootstrapCallSite bootstrap(Lookup lookup, String invokedName, MethodType invokedType, String value) { return new ConstantCallSite( MethodHandles.constant(Pattern.class, Pattern.compile(value)));}
6 .11
indyliberationSeeJohnRose'sideas:Scala'smacrosystemcanhelpexploretheseideas
mlvm-dev/2016-July/006661.html
6 .12
Example:Java8/9shimsCouldweusethistohelppeoplecrosstargetJava8/9?Link-timechoiceofUnsafevsVarHandle
6 .13
shims:ClientCodeclass Foo { var blah = 42 }
import Macro._
val f = new Foo()compareAndSetInt(classOf[Foo], "blah", f, -1, 0)assert(!compareAndSetInt(classOf[Foo], "blah", f, -1, 0))assert(compareAndSetInt(classOf[Foo], "blah", f, 42, 0))assert(f.blah == 0)
6 .14
autostaging? def foo(x: String) = { partiallyEvaluate { List("1", "2", "3").contains(x)) } }
Idea:macropartiallyEvaluatecouldmechanicallyturnthisinto:def foo(x: String) = { invokedynamic[LinkTimeInterpreter, List("1", "2", "3")].contains(x)}
6 .15
autostaging?LotsofrelatedresearchinScalaworld,although
"LightweightModularStaging",Rompf,Odersky"Yin-Yang:ConcealingtheDeepEmbeddingofDSLs",Jovanovicetal
integrationwithlinktimeevaldeservesmoreresearch
7 .1
indyλ
7 .2
Lambdas:BeforeandafterEssentiallythesameasJava7=>8
7 .3
Challenge:LMFassumptionsCan'tdefinetoStringUnboxingofnullsinScalais0,notNPEScalaValueClassboxingvoidreturntypeneedstobeboxed
7 .4
Challenge:LMFassumptionsscala> (() => ()).toStringres1: String = [function0]
scala> ((x: Int) => x).asInstanceOf[Any => Int].apply(null)res2: Int = 0
scala> ((() => ()) : (() => Any)).apply().getClassres3: Class[_] = class scala.runtime.BoxedUnit
7 .5
Challenge:LMFassumptionsSolution:
livewiththenewtoStringselectivelyuseadaptermethodsatthelambdatarget
7 .6
Challenge:ScalaSpecializationSpecializedtraitsleavethewrongmethodabstractShouldchangespecializationtofixthisbutitisacomplexbeast,andwe'vestartedoutwithintermediateFIstoflipthemethods.
7 .7
Challenge:Serializationjavactreatsserializablefuncinterfacesastheexceptionscalalambdasaretypicallyserializablejavac-style$deserializeLambda$wouldgrowlarge
hardlimitonnumberoflambdas(JDK-8008413)useageneric insteadlambdadeserializer
7 .8
Challenge:SerializationSolution:genericlambdadeserializer
bodyof$deserializeLambda$isanindycallsiteforacachepasseslookupalongtogenericcodethatcallsLMFmanuallytradeoff:can'tfailfastforsubtleserializationincompatibilties
7 .9
Challenge:lockscopinglazyvalsinlambdastranslateintothis.synchronizedthisusedtobetheanonclassnowitisthelambdahost,whichcontainsthelambdabodyinasyntheticmethod
7 .10
Challenge:unwantedcapturelocalmethodsinlambdastranslatetoprivatemethods......intheanonclass(before)...inthelambdahostclass(now)lambdanowretainsareferencetothelambdahostsolution:transitiveanalysistomakeliftedmethodsstatic
8
What'snext?2.12.0-RC1endofthemonthDeliverJava9supportincrementallyInvestinbenchmarking,performanceofcompiler/library
9
Thanks!Questions?