Stateof

Managed RuntimesAttila Szegedi

@asz

THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT DIRECTION. FOR INFORMATION PURPOSES ONLY, AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING PURCHASING DECISION. THE DEVELOPMENT, RELEASE, AND TIMING OF ANY FEATURES OR FUNCTIONALITY DESCRIBED FOR ORACLE'S PRODUCTS REMAINS AT THE SOLE DISCRETION OF ORACLE.

What is avirtual machine?

A virtual machine is…

• a software… (not hardware)

• …designed as runtime environment for other programs… (open ended)

• …that conform to a particular machine specification…

• …either existing hardware specification…

• …or entirely synthetic specification.

This is a virtual machine

This is a virtual machine

• a software…

• …designed as runtime environment for other programs… (operating systems, mostly)

• …that conform to a particular machine specification…

• …in this case, a particular combination of x86 CPU and other goodies (graphics, audio, IO).

This is a virtual machine

• This is not the kind of a virtual machine I’m talking about today.

What we’ll survey today

• Java Virtual Machine

• Parrot

• Some applications of Low Level Virtual Machine

• Rubinius

• PyPy

There’s lots of ways to target programs today

• Directly for hardware.

• Compiled for a virtual machine.

• Have it interpreted.

• These are not mutually exclusive.

What are benefits of managed runtimes?

• Automatic memory management

• Object model

• Code transformations

• ...

Java Virtual Machine

• Probably the best known VM out there.

• Started out as a VM for executing programs in Java.

• Today it has a vibrant ecosystem with myriad languages around it.

Java Virtual Machine

• Defined as a stack-based machine.

• Built-in concept of classes, objects, and fixed-size arrays.

• Classes have fixed field layout, with strongly typed field types.

• Single inheritance for classes.

JVM as target for dynamic languages

• Fixed-layout classes make it harder.

• People end up with hashtables etc.

• Leading to poor interop with Java.

• Your dynamic object doesn’t look like a Java object to Java program.

JVM as target for dynamic languages

• No multiple dispatch.

• Wait, doesn’t Java have overloaded methods?

• It does, but they’re resolved by the Java compiler, not by the VM.

• Every language has to figure it out themselves.

• No pluggable dispatch.

JVM as target for dynamic languages

• Where it excels:

• Threading support

• Memory management

• JIT compiler

JVM as target for dynamic languages

• How to fix the deficiencies for dynamic languages?

• Step 1: INVOKEDYNAMIC

• Step 2: high-level operation resolver framework

• This is what I’m doing by day.

Invokedynamic lets you be abstract

Duck duck = ...;

duck.quack();

INVOKEINTERFACE quack(Duck)V

Invokedynamic lets you be abstract

Object duck = ...;

duck.quack(); // won’t work in Java!

INVOKEDYNAMIC call:quack(Object)Object

Invokedynamic lets you be abstract

INVOKEINTERFACE quack(Duck)V

vs.

INVOKEDYNAMIC call:quack(Object)Object

• Doesn’t prescribe types

• Doesn’t prescribe dispatch

Invokedynamic lets you be abstract

Duck duck = ...;

Voice voice = duck.voice;

INVOKEINTERFACE getVoice(Duck)Voice

Invokedynamic lets you be abstract

Object duck = ...;

Object voice = duck.voice;

INVOKEDYNAMIC get:voice(Object)Object

Invokedynamic lets you be abstract

INVOKEINTERFACE getVoice(Duck)Voice

vs.

INVOKEDYNAMIC get:voice(Object)Object

• “get” can map to a field getter, array element getter, or a property method implementation.

• Okay, but what is mapping it?

Bootstrap methodspublic static CallSite bootstrap( String operation, MethodType signature)

• The JVM will invoke it with bootstrap(“get:voice”, MethodType(Object.class, Object.class))

• We can return a “CallSite” - basically a slot with a target function pointer.

• Target can be set multiple times.

Putting it togetherpublic class Ducks { public static class ShyDuck { public void quack() { System.out.println("...quack?"); } }

public static class LoudDuck { public void quack() { System.out.println("QUACK!"); } }

public static class ConfusedDuck { public void quack() { System.out.println("Woof!"); } }}

Putting it together

var ducks = [ new org.szegedi.geekout2103.Ducks.ShyDuck(), new org.szegedi.geekout2103.Ducks.LoudDuck(), new org.szegedi.geekout2103.Ducks.ConfusedDuck()]

for each(duck in ducks) { duck.quack()}

Putting it togetherALOAD 2INVOKEDYNAMIC dyn:getProp:duck(Ljava/lang/Object;)Ljava/lang/Object; [ jdk/nashorn/internal/runtime/linker/Bootstrap.bootstrap((Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;I)Ljava/lang/invoke/CallSite;)]DUPINVOKEDYNAMIC dyn:getMethod:quack(Ljava/lang/Object;)Ljava/lang/Object; [ jdk/nashorn/internal/runtime/linker/Bootstrap.bootstrap((Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;I)Ljava/lang/invoke/CallSite;)]SWAPINVOKEDYNAMIC dyn:call(Ljava/lang/Object;Ljava/lang/Object;)Ljava/lang/Object; [ jdk/nashorn/internal/runtime/linker/Bootstrap.bootstrap((Ljava/lang/invoke/MethodHandles$Lookup;Ljava/lang/String;Ljava/lang/invoke/MethodType;I)Ljava/lang/invoke/CallSite;)]

How’s JVM faring so far

• Java 8 will ship Nashorn, a JavaScript-on-the-JVM

• completely INVOKEDYNAMIC based

• in-house dogfooding of INVOKEDYNAMIC

• A lot of other languages, both using Indy and not, already target the JVM. I hear some are here at this conference too :-)

• My Dynalink project (on GitHub) helps you reduce the pain of providing high level operations for your language.

Parrot virtual machine

• Started out as a VM for Perl 6.

• Now it is language agnostic.

• Specifically designed with dynamic languages in mind.

Parrot virtual machine

• Objects are “polymorphic containers” with changing layout.

• Register based (not stack based as JVM).

• Continuations as primary expression of control flow.

• Built-in opcodes invokecc, yield, tailcall, capture_lex, newclosure…

Parrot virtual machine

• Cross-level language interop is basically built-in

• getprop, setprop opcodes!

• opcodes for loading languages and registering compilers!

• Much higher level bytecode operations than JVM.

• Basically starts out with Indy+Metaobject Protocols built in.

Parrot virtual machine

• Basic input formats:

• Parrot Intermediate Representation (PIR)

• Parrot Abstract Syntax Tree (PAST)

Parrot virtual machine• PIR is human readable assembly-level language

.sub factorial # Get input parameter. .param int n

# return (n > 1 ? n * factorial(n - 1) : 1) .local int result

if n > 1 goto recurse result = 1 goto return

recurse: $I0 = n - 1 result = factorial($I0) result *= n

return: .return (result) .end

LLVM

• It’s not actually a VM.

• Used to stand for “Low-Level Virtual Machine”.

• Today, LLVM is the name of the project, and it’s not considered an acronym.

• It’s a compiler toolchain project.

LLVM• Register based intermediate representation

define i32 @add1(i32 %a, i32 %b) {entry: %tmp1 = add i32 %a, %b ret i32 %tmp1} define i32 @add2<i32 %a, i32 %b) {entry: %tmp1 = icmp eq i32 %a, 0 br i1 %tmp1, label %done, label %recurse recurse: %tmp2 = sub i32 %a, 1 %tmp3 = add i32 %b, 1 %tmp4 = call i32 @add2(i32 %tmp2, i32 %tmp3) ret i32 %tmp4 done: ret i32 %b

LLVM

C

Haskell

Ruby

LUA

Python

Clang

GHC

Rubinius

LLVM-LUA

PyPy

LLVMOptimizer

x86

PPC

ARM

JS

LLVM

C

Haskell

Ruby

LUA

Python

Clang

GHC

Rubinius

LLVM-LUA

PyPy

LLVMOptimizer

x86

PPC

ARM

JS

C ClangLLVM

Optimizer JS

Emscripten

• LLVM backend for JavaScript

• Anything that compiles to LLVM IR can be translated to JavaScript.

• Audio, graphics, IO libraries mapped to HTML5 constructs.

Under the hood{xb(6155784,-1);C[1571410]=1;C[1571411]=16;A[3142824]=0;C[1571413]=32;C[1571414]=641;C[1571415]=481;C[1571416]=31;C[1571417]=1;C[1571418]=1;C[1571419]=38;xc(6550284,0,0,0,93,0,0,6285640,4);xc(6569992,0,0,0,94,0,0,6285640,4)})},{c:(function(){xb(6155764,-1);Fc(6322920,1e4,0);Dba();Eba();Nc(5479372,1);C[1369843]=5970032;C[1369845]=5288420;C[1369851]=0;C[1369852]=0;C[1369846]=0;C[1369847]=0;C[1369848]=0;C[1369849]=0;y[5479400]=0;ub(780,5479372);for(var b=5472972;!(ve(b+84|0),b=b+100|0,5479372==(b|0));){}for(b=5448396;!(ve(b+32|0),b=b+48|0,5472972==(b|0));){}})},{c:(function(){xb(6155744,-1);we(6219428,21);ye(6219440,17,7,-1);ye(6219452,12,7,-1);ze(6219464,1797,183);ye(6219476,13,7,-1);ye(6219488,14,7,-1);Ae(6219500);we(6219512,21);ye(6219524,10,7,0);ze(6219536,2049,1798);Be(6219548);Ce(6219560,1);Fba(6219572,1);ye(6219584,30,7,1);Ae(6219596);ye(6219608,1,7,2);Ce(6219620,0);Ae(6219632);we(6219644,21);ye(6219656,6,7,3);Be(6219668);Ce(6219680,0);ze(6219692,1812,1813);ye(6219704,6,7,4);Be(6219716);Ce(6219728,0);ze(6219740,-30664,0);ye(6219752,16,7,-1);Ae(6219764);xc(6434916,1,170,212,60,0,1,6219428,29);we(6221152,21);ye(6221164,17,9,-1);ye(6221176,12,9,0);ze(6221188,2347,183);ye(6221200,15,9,-1);ye(6221212,13,9,-1);ye(6221224,14,9,-1);Ae(6221236);ye(6221248,1,9,-1);ye(6221260,2,9,-1);Gba(6221272,2,2,2,2);ye(6221284,27,255,1);Hba(6221296,254,86);Be(6221308);Gba(6221320,1,1,1,1);Ce(6221332,1);Ae(6221344);Ae(6221356);ye(6221368,1,9,2);Hba(6221380,260,2);Ce(6221392,0);Ae(6221404);we(6221416,21);ye(6221428,134,9,3);Be(6221440);Ce(6221452,0);ze(6221464,180,2350);ye(6221476,134,9,4);Be(6221488);Ce(6221500,0);ze(6221512,1812,1813);ye(6221524,16,9,-1);Ae(6221536);xc(6446848,1,260,120,49,0,2,6221152,33);we(6187416,21);ye(6187428,17,13,-1);ye(6187440,12,13,-1);ze(6187452,2341,183);ye(6187464,13,13,-1);ye(6187476,14,13,-1);Ae(6187488);we(6187500,21);we(6187512,23);we(6187524,21);ye(6187536,6,13,0);ze(6187548,179,176);ye(6187560,18,13,1);ze(6187572,-30664,177);ye(6187584,1,13,-1);Ce(6187596,0);Ae(6187608);Ae(6187620);ye(6187632,1,13,2);ze(6187644,0,2346);Ce(6187656,1);Fba(6187668,3);Ae(6187680);Ae(6187692);we(6187704,23);ye(6187716,30,13,3);ye(6187728,16,13,-1);Ae(6187740);Ae(6187752);xc(6313948,1,428,190,34,0,2,6187416,29);we(6197828,21);ye(6197840,17,13,-1);ye(6197852,12,13,0);ze(6197864,1805,183);ye(6197876,13,13,-1);ye(6197888,14,13,-1);Ae(6197900);we(6197912,21);we(6197924,23);ye(6197936,1,13,2);Ce(6197948,10);Hba(6197960,200,90);Fba(6197972,3);Ae(6197984);we(6197996,21);ye(6198008,6,13,1);ze(6198020,1814,1815);ye(6198032,1,13,-1);Be(6198044);Ce(6198056,0);Ae(6198068);Ae(6198080);Ae(6198092);we(6198104,23);ye(6198116,30,13,3);ye(6198128,16,13,-1);Ae(6198140);Ae(6198152);xc(6366212,1,300,210,71,0,0,6197828,28)})},{c:(function(){Iba();xc(6523164,2,0,0,61,0,2,6262368,140)})},{c:(function(){xb(6155704,-1);C[1566261]=27;C[1566262]=255;A[3132526]=0;C[1566264]=31;C[1566265]=1;C[1566266]=1;xc(6533728,0,0,0,0,0,0,6265044,2);Jba()})},{c:(function(){xb(6155664,-1);De(6285544,0);Ee(6285556,17,14,-1);Ee(6285568,12,14,-1);Fe(6285580,1816,183);Ee(6285592,15,14,-1);Ie(6285604);Ee(6285616,1,14,0);Ie(6285628);xc(6550252,1,0,0,15,0,0,6285544,8);De(6382688,0);Ee(6382700,17,14,-1);Ee(6382712,12,14,-1);Fe(6382724,1913,183);Ie(6382736);Ee(6382748,1,14,-1);Kba(6382760,4,2,4);Ee(6382772,8,14,-1);Fe(6382784,1914,0);Ee(6382796,8,14,-1);Fe(6382808,1915,0);Ee(6382820,11,14,-1);Lba(6382832,0,5,1,5);Ee(6382844,0,255,0);Ie(6382856);Ee(6382868,8,14,1);Fe(6382880,-30641,0);Ee(6382892,8,14,-1);Fe(6382904,1916,0);Ie(6382916);xc(6587772,2,0,0,87,0,0,6382688,20);Ee(6294292,1,14,0);Je(6294304,200,32);Ie(6294316);xc(6547548,0,0,0,10,0,0,6294292,3);De(6232508,0);Ee(6232520,17,14,-1);Ee(6232532,12,14,0);Fe(6232544,-30636,0);Ie(6232556);Ee(6232568,1,14,-1);Ee(6232580,19,14,1);Je(6232592,256,12);Mba(6232604);Lba(6232616,2,2,2,2);Ie(6232628);De(6232640,1);Ee(6232652,6,14,2);Je(6232664,87,12);Mba(6232676);Fe(6232688,192,0);Ee(6232700,6,14,3);Je(6232712,86,12);Mba(6232724);Fe(6232736,193,0);Ee(6232748,6,14,4);Je(6232760,87,12);Mba(6232772);Fe(6232784,194,0);Ie(6232796);xc(6517512,2,0,0,11,0,0,6232508,25);De(6381092,0);Ee(6381104,17,4,-1);Ee(6381116,12,4,0);Fe(6381128,-30664,0);Ie(6381140);Ee(6381152,1,4,-1);Kba(6381164,8,15,8);Ee(6381176,9,4,1);Je(6381188,200,12);De(6381200,1);Kba(6381212,20,29,20);Ee(6381224,134,3,2);Je(6381236,71,12);Fe(6381248,1178,0);Ee(6381260,134,3,3);Je(6381272,71,12);Fe(6381284,1177,0);Ie(6381296);Ie(6381308);xc(6586432,2,0,0,12,0,6,6381092,19)})},{c:(function(){Ke(6387132);C[1596783]=5978576;C[1596783]=5987384;Le(6387132,5279752,0,3);if(m){var b=M([]);$6$1=v;Me(6387132);m?(M([0]),$9$1=v,ad()):(0==C[T>>2]&&(C[T>>2]=b),a(b))}C[1596783]=5985060;ub(7702,6387132)})},{c:(function(){Nba();xc(6269464,1,0,0,86,0,2,6177916,59);Oba();xc(6519984,1,0,0,85,0,2,6235988,156)})},{c:(function(){Ke(6449624);C[1612406]=5978576;C[1612406]=5987412;Le(6449624,5308336,0,1);if(m){var b=M([]);$6$1=v;Me(6449624);m?

asm.js

• Mozilla embraced Emscripten’s code generator output as a JS subset:

• arithmetic operations only

• loads and stores into a single ArrayBuffer

• calls to functions that only take numers and return number.

asm.js• Force an expression to be double: “+x”

• Force an expression to be int: “x | 0”

function diagonal(x, y) { x = +x; // x has type double y = +y; // y has type double return +sqrt(square(x) + square(y));}

function add1(x) { x = x|0; // x : int return (x+1)|0;}

asm.js

• It’s a statically typed language that only looks like JavaScript.

• No strings

• No allocation

• No calling of other JS libraries

Other LLVM based runtimes

• Let’s take a brief look at some single-language runtimes.

• PyPy is Python runtime using LLVM.

• Has sandboxing, good memory management

• Has stackless mode (ever played EVE Online?)

• Itself largely written in Python!

• Toolchain available as separate project RPython.

• Rpython is a statically compilable subset of Python

• Most of PyPy written in it.

• There’s now Topaz: a Ruby on top of PyPy!

• How crazy is that?!

• A word on “stackless”

• Execution doesn’t use C stack

• Can have million of micro-threads executing as coroutines efficiently on few CPUs

• Very useful to program MMOs.

• Similar to PyPy, only for Ruby:

• Big part of implementation written in Ruby itself

• Relies on LLVM to efficiently compile to Ruby

• Similar memory management etc. characteristics

• Similar to JVM too:

• intermediate format is stack machine with bytecode

• has high level bytecodes for yield, pushing blocks etc.

• Object representation:

• Has immediates for fixnums, symbols, booleans

• This can be huge deal performance-wise

• JVM-like objects (header, class pointer, fields)

• Generational GC, similar layout to JVM:

• Nursery, Young, Mature generations.

Self-hosting• Recurring theme is to write as much of the

runtime as possible in the runtime’s target language itself.

• More accessible for language users.

• Reduced bootstrapping core.

• Benefits from its own optimizations.

• Of course, old news in Lisp world.

Maxine VM• Speaking of self hosted VMs…

• Oracle’s open-sourced Java VM written in Java.

• Pluggable GC and JIT-compiler.

• “Maxine Inspector” for debugging/visualizing running VM state.

• Great tool for learning and research of VM technologies.

• Great IDE support (Eclipse, NetBeans, IntelliJ)

Maxine inspector

JavaScript runtimes

• Probably biggest innovation hotbed right now.

• Google’s V8, Mozilla’s *Monkey, Apple’s Nitro

• … as well as Oracle’s Nashorn.

JavaScript is ideal

• JavaScript is ideal language to get paid to work on it.

• Extreme dynamism ensures there’s no single correct optimization approach.

• Got to try them all!

JS optimization strategies

• Type-specializing code generators

• Optimistic typing with on-stack code rewrite

• Allocation-site object shaping

• Static analysis (as far as it can go)

• Profiling-driven recompilation.

• Many more.

Summary• I love JVM. JVM is great. Lots of innovation going

on.

• There is life outside the JVM.

• I’m very enthusiastic for Parrot, PyPy, and Rubinius.

• Diversity of JS runtimes and their strategies for efficient execution is also great to watch and participate in.

• … especially if it means bringing back Transport Tycoon.

Summary

• As a general observation, commercial JVM technologies still have best memory management, observability/serviceability, and threading support.

• Not saying others won’t catch up, but HotSpot/JRockit/Azul/J9 etc. have a big lead in engineering effort poured into them.

• So, if you stay with JVM, you’re also in a very good place.

Image Credits

Machine: http://www.flickr.com/photos/38712296@N07/5264174576/

All used images are Creative Commons licensed, and used according to their terms of license.