Inline Caching in JavaScriptCore · “Inline Cache” indexing type ﬂags cell state structure...

Inline Caching in JavaScriptCore

Filip PizloApple Inc.

webkit.org

https://svn.webkit.org/repository/webkit/trunk

JavaScriptCore.framework

Safari

Agenda

• JavaScript execution strategy

• Focus on property access inline caches - Simple inline caching

- Inline Caching Optimizations

- Lots of perf data

JavaScript execution strategy

Four Tiers

LLInt (interpreter)

Baseline (template JIT)

DFG (less optimizing JIT)

FTL (optimizing JIT)

latency throughput

Profiling

LLInt (interpreter)




Speculation and OSR

LLInt (interpreter)




Parser

ParserAST

Parser

Bytecompiler

AST

Parser

Bytecompiler

AST

unlinked bytecode

Parser

Bytecompiler

Generatorification

AST

unlinked bytecode

Parser

Bytecompiler

Generatorification

Bytecode Linker

AST

unlinked bytecode

Parser

Bytecompiler

Generatorification

Bytecode Linker

AST

unlinked bytecode

bytecode

Parser

Bytecompiler

Generatorification

Bytecode Linker

LLInt

AST

unlinked bytecode

bytecode

Parser

Bytecompiler

Generatorification

Bytecode Linker

LLInt Bytecode Template JIT

AST

unlinked bytecode

bytecode

Parser

Bytecompiler

Generatorification

Bytecode Linker


AST

unlinked bytecode

bytecodeDFG

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG

DFG Optimizer

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG

DFG Optimizer

DFG Backend

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG

DFG Optimizer

DFG Backend

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

DFG Optimizer

DFG Backend

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

DFG Optimizer

DFG Backend

Extended DFG Optimizer

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

FTL-to-B3 lowering

DFG Optimizer

DFG Backend


Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

FTL-to-B3 lowering

DFG Optimizer

DFG Backend


B3 Optimizer

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

FTL-to-B3 lowering

DFG Optimizer

DFG Backend


B3 Optimizer

Instruction Selection

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

FTL-to-B3 lowering

DFG Optimizer

DFG Backend


B3 Optimizer


Air Optimizer

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

FTL-to-B3 lowering

DFG Optimizer

DFG Backend


B3 Optimizer


Air Optimizer

Air Backend

Parser

Bytecompiler

Generatorification

Bytecode Linker


DFG Bytecode Parser

DFG Bytecode Parser

AST

unlinked bytecode

bytecodeDFG FTL

Inline Caching

Inline Caching• Performance

• Why?

• Simple inline caching- in interpreter

- in JIT

- prototype

• Advanced topics- Polymorphic Access JIT

- Inlining inline caches

- Type inference

Performance

Benchmark Summary Number of Samples in my experiments

JetStream 1.1 ES5 benchmarks(real and synthetic) 9

ARES-6 1.0.1 ES6 benchmarks(real and synthetic) 24

Speedometer 2.0DOM framework

benchmarks(real)

30

Source: browserbench.org

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5×

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5×

Baseline = no property inline caching

{x: 1, y: 2}

{x: 42, y: 3}

{x: -5, y: 7}

{x: 1, y: 2}

{x: 42, y: 3}

{x: -5, y: 7}

var x = o.x;

{x: 1, y: 2}

{x: 42, y: 3}

{x: -5, y: 7}

o.x = x;

Hashtable

{x: 1, y: 2}{ hashtable

object

x: 1

y: 1

Hashtable

• Pointer chasing is slow


object

x: 1

y: 1

Hashtable


• Hash codes take time to compute


object

x: 1

y: 1

Hashtable


• Hash codes take time to compute

• Lots of instructions, hard to inline


object

x: 1

y: 1

{x: 1, y: 2}

{x: 42, y: 3}

{x: -5, y: 7}

{1, 2}

{42, 3}

{-5, 7}

{x, y}

{1, 2}

{42, 3}

{-5, 7}

{x, y}

structure

{1, 2}

{42, 3}

{-5, 7}

{x, y}

structure

‣ hashtable

{1, 2}

{42, 3}

{-5, 7}

{x, y}

structure

‣ hashtable

‣ maps name to offset

{1, 2}

{42, 3}

{-5, 7}

{x, y}

structure

‣ hashtable

‣ maps name to offset

‣ hash-consed

JSC Object Model

indexingtypeflags

cell state

structure ID butterfly pointer inline slot 0 inline slot 1

public length

vector lengthout of line slot 0 array slot 0

…

……

JSC Object Model

indexingtypeflags

cell state


public length


…

……

64 bits

64 bits

64 bits 64 bits 64 bits

64 bits 64 bits

JSC Object Model

indexingtypeflags

cell state


public length


…

……

JSC Object Model

indexingtypeflags

cell state


public length


…

……

statically configurable

JSC Object Model

indexingtypeflags

cell state


public length


…

……

statically configurable

dynamically configurable

Fast JSObject

indexingtypeflags

cell state

structure ID: 42 null 0xffff000000000005

var o = {f: 5, g: 6};

0xffff000000000006

Fast JSObject

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Fast JSObject

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Structure

Fast JSObject

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

Fast JSObject

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

Fast JSObject

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

g: inline(1)

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

g: inline(1)

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

g: inline(1)

var v = o.f;

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

g: inline(1)

var v = o.f;

if (o->structureID == 42) v = o->inlineStorage[0]else v = slowGet(o, “f”)

“Inline Cache”

indexingtypeflags

cell state


var o = {f: 5, g: 6};

0xffff000000000006

Structure Table

Property Table

Structure

f: inline(0)

g: inline(1)

var v = o.f;

if (o->structureID == 42) v = o->inlineStorage[0]else v = slowGet(o, “f”)

Interpreter Inline Cache

get_by_id <result>, <base>, <properyName>


get_by_id <result>, <base>, <properyName>, <cachedStructureID>, <cachedOffset>


get_by_id loc42, loc43, “g”, 0, 0



indexingtypeflags

cell state

structure ID: 42 null

f:0xffff000000000005

g:0xffff000000000006

JIT Inline Cache

0x46f8c30b9b0: mov 0x30(%rbp), %rax0x46f8c30b9b4: test %rax, %r150x46f8c30b9b7: jnz 0x46f8c30ba2c0x46f8c30b9bd: jmp 0x46f8c30ba2c0x46f8c30b9c2: o16 nop %cs:0x200(%rax,%rax)0x46f8c30b9d1: nop (%rax)0x46f8c30b9d4: mov %rax, -0x38(%rbp)

JIT Inline Cache

0x46f8c30b9b0: mov 0x30(%rbp), %rax0x46f8c30b9b4: test %rax, %r150x46f8c30b9b7: jnz 0x46f8c30ba2c0x46f8c30b9bd: cmp $0x125, (%rax)0x46f8c30b9c3: jnz 0x46f8c30ba2c0x46f8c30b9c9: mov 0x18(%rax), %rax0x46f8c30b9cd: nop 0x200(%rax)0x46f8c30b9d4: mov %rax, -0x38(%rbp)

“Self” IC speed-up

JetStream

ARES-6

Speedometer 2

1× 1.1× 1.2× 1.3× 1.4× 1.5× 1.6× 1.7× 1.8×

1.19×

1.66×

1.49×


JetStream

ARES-6

Speedometer 2

1× 1.1× 1.2× 1.3× 1.4× 1.5× 1.6× 1.7× 1.8×

1.19×

1.66×

1.49× p = 10-23


JetStream

ARES-6

Speedometer 2

1× 1.1× 1.2× 1.3× 1.4× 1.5× 1.6× 1.7× 1.8×

1.19×

1.66×

1.49× p = 10-23

p = 10-57


JetStream

ARES-6

Speedometer 2

1× 1.1× 1.2× 1.3× 1.4× 1.5× 1.6× 1.7× 1.8×

1.19×

1.66×

1.49× p = 10-23

p = 10-57

p = 10-46

Simple Inline Caching

• Interpreter edits instruction operands.

• JIT reserves a nop sled and patches it.

Prototypes

Prototypesfunction Foo(f){ this.f = f;}

Foo.prototype.getF = function(){ return this.f;}



getF: λ

Foo.prototype



var o = new Foo(42);{f: 42}

getF: λ

Foo.prototype



var o = new Foo(42);

var tmp = o.getF();

{f: 42}

getF: λ

Foo.prototype

Prototypesclass Foo { constructor(f) { this.f = f; }

getF() { return this.f; }}

var o = new Foo(42);

var tmp = o.getF();

{f: 42}

getF: λ

Foo.prototype

Prototype Inline Caching

• Goals:

• o.getF should be fast

• o.getF() should be inlineable

• o.newField = value should be fast

Prototype Inline Caching

• Mono proto

• Transition watchpoint

• Replacement watchpoint

Mono Proto

{1, 2}

{42, 3}

{-5, 7}

{x, y}

prototype

global object

Mono Proto

{1, 2}

{42, 3}

{-5, 7}

{x, y}

prototype

global object

structure

Proto Chain

object

Proto Chain

object

structure 42

Proto Chain

object

structure 42

prototype P1

Proto Chain

object

structure 42

prototype P1

structure 43

Proto Chain

object

structure 42

prototype P1

structure 43

prototype P2

Proto Chain

object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Proto Chain

object

structure 42

prototype P1

structure 43

prototype P2

structure 44

// tmp = o.getFif (o->structureID == 42 && P1->structureID == 43 && P2->structureID == 44) return P2->inlineStorage[…]else slowGet(o, “getF”)

Proto Chain

object

structure 42

prototype P1

structure 43

prototype P2

structure 44// o.newField = value

if (o->structureID == 42 && P1->structureID == 43 && P2->structureID == 44) o->inlineStorage[…] = value o->structureID = 100;else slowPut(o, “newField”, value)

{}

Transition Table

Transition Hash Consing

Structure {}

{x}

Transition Table


Structure {}

Transition Table

x Structure {x}

{x, y}


Structure {}

Transition Table

x Structure {x}

Structure {x, y}

Transition Table

y

Transition Table

Optimized Code

return P2->inlineStorage[…]else slowGet(o, “…”)

object

structure 42

prototype P1

structure 43

prototype P2

structure 44

if (o->structureID == 42 && P1->structureID == 43 && P2->structureID == 44)

Optimized Code


object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table


Optimized Code


object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table


Optimized Code


object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

Transition Table


Optimized Code


if (o->structureID == 42)

object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

Transition Table

Optimized Code



object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

Transition Table

P1.thingy = “boom”;

Transition Table

Optimized Code



object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

structure 666

thingy


Transition Table

object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

structure 666

thingy


Watchpoint

class Watchpoint {public: virtual void fire() = 0;};

Transition Table

Optimized Code



object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

Transition Table

Optimized Code



object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

Replacement Map

field

Transition Table

Optimized Code


object

structure 42

prototype P1

structure 43

prototype P2

structure 44

Transition Table

Transition Table

return constantelse slowGet(o, “…”)

Replacement Map

field


• tmp = o.f // self

• o.f = tmp // self, existing

• o.f = tmp // self, new

• tmp = o.f // prototype


• tmp = o.f // self

• o.f = tmp // self, existing

• o.f = tmp // self, new

• tmp = o.f // prototype}only need one branch

JIT Monomorphic IC speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×

1.49× Self+ Proto+ New Property+ Transition Watch+ Replacement Watch


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9

p = 0.06


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9

p = 0.06

p = 0.009


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9

p = 0.06

p = 0.009

p = 0.0001


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9

p = 0.06

p = 0.009

p = 0.0001

p = 10-12


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4×

1.30×

3.52×

2.09×

1.29×

3.13×

2.05×

1.26×

3.05×

1.97×

1.25×

2.82×

1.87×

1.19×

1.66×


p = 10-18

p = 10-48

p = 10-13

p = 10-22

p = 10-9

p = 10-15

p = 10-9

p = 0.06

p = 0.009

p = 0.0001

p = 10-12

p = 0.09

Advanced Topics

• Polymorphic Inline Caches

• Inlining Inline Caches

• Type Inference

Polymorphic Inline Caches

{f: 64, g:75}

{f: 64, g:75}

{f: 98, g:23}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 54, g:75, h:389}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 54, g:75, h:389}

{f: 6, g:18, h:83}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 54, g:75, h:389}

{f: 6, g:18, h:83}

{g: 37, f: 30}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 54, g:75, h:389}

{f: 6, g:18, h:83}

{g: 37, f: 30}

{g: 835, f: 0}

{f: 64, g:75}

{f: 98, g:23}

{f: 7, g:32}

{f: 342, g:5}

{f: 54, g:75, h:389}

{f: 6, g:18, h:83}

{g: 37, f: 30}

{g: 835, f: 0}

{f, g}

{f, g, h}

{g, f}

var tmp = o.f;

var tmp = o.f;

{f: 64, g:75}

S1: {f, g}

var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389}

S1: {f, g}

S2: {f, g, h}

var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389} {g: 37, f: 30}

S1: {f, g}

S2: {f, g, h}

S3: {g, f}

var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389} {g: 37, f: 30}

S1: {f, g}

S2: {f, g, h}

S3: {g, f}

if (o->structureID == S1 || o->structureID == S2) o->inlineStorage[0]else if (o->structureID == S3) o->inlineStorage[1]else slowGet(o, “f”)

function foo(o) { return o.f;}

noInline(foo);

for (var i = 0; i < 10000; ++i) { foo({f: 1, g: 2}); foo({f: 1, g: 2, h:3}); foo({g: 2, f: 1});}

0x456923529be: cmp $0x129, (%rax)0x456923529c4: jnz 0x45692352a2d0x456923529ca: mov 0x10(%rax), %rax0x456923529ce: nop 0x200(%rax)

S1: {f, g}

function foo(o) { return o.f;}

noInline(foo);

for (var i = 0; i < 10000; ++i) { foo({f: 1, g: 2}); foo({f: 1, g: 2, h:3}); foo({g: 2, f: 1});}

0x45692352be0: mov (%rax), %esi0x45692352be2: cmp $0x129, %esi0x45692352be8: jnz 0x45692352bf70x45692352bee: mov 0x10(%rax), %rax0x45692352bf2: jmp 0x456923529d50x45692352bf7: cmp $0x126, %esi0x45692352bfd: jnz 0x45692352c0c0x45692352c03: mov 0x10(%rax), %rax0x45692352c07: jmp 0x456923529d50x45692352c0c: cmp $0x12b, %esi0x45692352c12: jnz 0x45692352a2d0x45692352c18: mov 0x18(%rax), %rax0x45692352c1c: jmp 0x456923529d50x45692352c21: jmp 0x45692352a2d

S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}

S2: {f, g, h}

S3: {g, f}

done


S1: {f, g}

S2: {f, g, h}

S3: {g, f}

done

done


S1: {f, g}

S2: {f, g, h}

S3: {g, f}

done

done

slow


S1: {f, g}

S2: {f, g, h}

S3: {g, f}

done

done

done

slow

Polymorphic Access JIT• Self

• Prototype

• Transition (new property)

• Getters

• Setters

• Custom accessors

• Snippets (DOM JIT)

JIT Polymorphic IC speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×

2.09× Monomorphic IC+ Simple Poly IC+ Misses+ Accessors & Snippets


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7

p = 0.05


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7

p = 0.05

p = 0.7


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7

p = 0.05

p = 0.7

p = 10-10


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7

p = 0.05

p = 0.7

p = 10-10

p = 10-21


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5×

1.37×

4.68×

2.28×

1.32×

3.99×

2.14×

1.31×

3.92×

2.13×

1.30×

3.52×


p = 0.003

p = 10-14

p = 0.4

p = 0.7

p = 0.05

p = 0.7

p = 10-10

p = 10-21

p = 0.004

Inlining Inline Caches

Four Tiers

LLInt (interpreter)




latency throughput

Four Tiers

LLInt (interpreter)




latency throughput

profiling

Four Tiers

LLInt (interpreter)




latency throughput

profiling

speculation and OSR

LLInt (interpreter)




get_by_idjmp Lslow jmp Lslow jmp Lslow

tmp = o.fS1:

{f, g}

LLInt (interpreter)




jmp Lslow jmp Lslow jmp Lslow

tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)




cmp S1, (%rax)jnz Lslowmov 10(%rax), %rax

jmp Lslow jmp Lslow

tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)






jmp Lslow

tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)







tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)

get_by_id

tmp = o.fS1:

{f, g}

LLInt (interpreter)

tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)


jmp Lslow

tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)



tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)




tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

LLInt (interpreter)




tmp = o.f

get_by_id…, S1, 0

S1: {f, g}

CheckStructure(@o, S1)GetByOffset(@o, “f”, 0)

LLInt (interpreter)





tmp = o.f

get_by_id…, S1, 0

S1: {f, g}


LLInt (interpreter)

get_by_id

tmp = o.ftmp2 = o.g

S1: {f, g}

get_by_id

LLInt (interpreter)

tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

get_by_id

LLInt (interpreter)

tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

get_by_id…, S1, 1

LLInt (interpreter)


jmp Lslow

tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

jmp Lslowget_by_id…, S1, 1

LLInt (interpreter)



tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

jmp Lslowget_by_id…, S1, 1

LLInt (interpreter)



tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

get_by_id…, S1, 1


LLInt (interpreter)




tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}

get_by_id…, S1, 1


LLInt (interpreter)




tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}


get_by_id…, S1, 1

CheckStructure(@o, S1)GetByOffset(@o, “g”, 1)


LLInt (interpreter)





tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}


get_by_id…, S1, 1



LLInt (interpreter)





tmp = o.ftmp2 = o.g

get_by_id…, S1, 0

S1: {f, g}



get_by_id…, S1, 1



Branch(Equal( @structure, $S1))

Load(@object, offset = 0x10) Call(slow path)

… things …



… more things …

Inline Cache Control Flow



… things …



… more things …

not redundant!

Inline Cache Control Flow



… things …



… more things …


OSR exit

Load(@object, offset = 0x10)

… things …



… more things …

OSR exit

Inline Cache Control Flow Inlined with OSR exits



… things …



… more things …


OSR exit


… things …


… more things …

Inline Cache Control Flow Inlined with OSR exits

Benefits of Inlining

• Common subexpression elimination

• Abstract interpreter models the structure

• Object allocation sinking

• Many other optimizations

IC Monomorphic Inlining Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.37×

4.68×

2.28×Polymorphic IC+ Mono IC Inlining


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.37×

4.68×


p = 10-14


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.37×

4.68×


p = 10-14

p = 10-21


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.37×

4.68×


p = 10-14

p = 10-21

p = 0.02

IC Inlining and Watchpoints

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×

2.24×IC Inlining w/o WatchpointsIC inlining w/ Transition WatchpointsIC Inlining w/ Transition & Replacement


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8

p = 10-9


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8

p = 10-9

p = 0.0003


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8

p = 10-9

p = 0.0003

p = 10-13


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8

p = 10-9

p = 0.0003

p = 10-13

p = 10-26


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.39×

5.67×

2.49×

1.36×

4.54×

2.33×

1.32×

4.33×


p = 10-8

p = 10-9

p = 0.0003

p = 10-13

p = 10-26

p = 0.02

Minimorphic IC Inlining

var tmp = o.f;


var tmp = o.f;

{f: 64, g:75}

S1: {f, g}


var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389}

S1: {f, g}

S2: {f, g, h}


var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389}

S1: {f, g}

S2: {f, g, h}

CheckStructure(@o, [S1, S2])GetByOffset(@o, “f”, 0)

Polymorphic IC Inlining

var tmp = o.f;


var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389} {g: 37, f: 30}

S1: {f, g}

S2: {f, g, h}

S3: {g, f}


var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389} {g: 37, f: 30}

S1: {f, g}

S2: {f, g, h}

S3: {g, f}

MultiGetByOffset(@o, “f”, [S1, S2] => 0, [S3] => 1)DFG IR:


var tmp = o.f;

{f: 64, g:75} {f: 54, g:75, h:389} {g: 37, f: 30}

S1: {f, g}

S2: {f, g, h}

S3: {g, f}

MultiGetByOffset(@o, “f”, [S1, S2] => 0, [S3] => 1)

if (o->structureID == S1 || o->structureID == S2) result = o->inlineStorage[0]else result = o->inlineStorage[1]

DFG IR:

B3 IR:

IC Polymorphic Inlining Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×

2.49× Mono IC Inlining+ Mini IC Inlining+ Poly IC Inlining


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×

2.49× Mono IC Inlining+ Mini IC Inlining+ Poly IC Inliningp = 0.02


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×


p = 0.5


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×


p = 0.5

p = 0.6


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×


p = 0.5

p = 0.6

p = 0.8


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×


p = 0.5

p = 0.6

p = 0.8

p = 0.6


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7×

1.38×

5.76×

2.52×

1.40×

5.72×

2.52×

1.39×

5.67×


p = 0.5

p = 0.6

p = 0.8

p = 0.6

p = 0.3

function foo(o) { return o.f; }


S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}

S2: {f, g, h}

S3: {g, f}

function bar(p) { return foo(p.g); }


S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}


S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}


…—> foo

<- foo

jmp Lslow


S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}


…—> foo

<- foo



S1: {f, g}

S2: {f, g, h}

S3: {g, f}


S1: {f, g}


…—> foo

<- foo



…—> foo CheckStructure GetByOffset<- foo

IC Polyvariant Inlining Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.14×

2.53×

1.38×

5.76×

2.52×Polymorphic IC Inlining+ Polyvariant IC Inlining


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.14×

2.53×

1.38×

5.76×


p = 0.4


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.14×

2.53×

1.38×

5.76×


p = 0.4

p = 0.0002


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.14×

2.53×

1.38×

5.76×


p = 0.4

p = 0.0002

p = 0.9

MultiGetByOffset(@o, “f”, [S1, S2] => 0, [S3] => 1)MultiGetByOffset(@o, “g”, [S1, S2] => 1, [S3] => 0)DFG IR:

MultiGetByOffset(@o, “f”, [S1, S2] => 0, [S3] => 1)MultiGetByOffset(@o, “g”, [S1, S2] => 1, [S3] => 0)DFG IR:

B3 IR:


Branch(Equal( @structure, $S2))Load(@object,

offset = 0x10)Branch(Equal( @structure, $S3))


OSR exit





OSR exit

then

elsethen

then

then

then

then

else

else

elseelse

else





OSR exit





OSR exit

then

elsethen

then

then

then

then

else

else

elseelse

else




Load(@object, offset = 0x18)Branch(Equal( @structure, $S1))

OSR exit





OSR exit

then

elsethen

then

then

then

then

else

else

elseelse

elsethenelse





Load(@object, offset = 0x18)Branch(Equal( @structure, $S1))

OSR exit




OSR exit

then

elsethen

then

then

then

then

else

else

elseelse

else

then

else





OSR exit




OSR exit

then

elsethen

then

then

then

else

else

elseelse





OSR exit




OSR exit

then

elsethen

then

then

else

else

else





OSR exit



then

elsethen

then

else

else



offset = 0x10)Load(@object, offset = 0x18)


Load(@object, offset = 0x18)Load(@object, offset = 0x10)

OSR exit

then

elsethen

then

else

else

B3 IR before:






OSR exit






OSR exit

then

elsethen

then

then

then

then

else

else

else

else

else

B3 IR before:






OSR exit






OSR exit

then

elsethen

then

then

then

then

else

else

else

else

else

B3 IR after:






OSR exit

then

elsethen

then

else

else

duplicateTails(procedure); foldPathConstants(procedure);

Taildup Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.23×

2.57×

1.39×

6.14×

2.53×Poly Poly IC Inlining+ Taildup

Taildup Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.23×

2.57×

1.39×

6.14×


p = 0.03

Taildup Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.23×

2.57×

1.39×

6.14×


p = 0.03

p = 0.5

Taildup Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.39×

6.23×

2.57×

1.39×

6.14×


p = 0.03

p = 0.5

p = 0.6

o.f.g

a: CheckStructure(@o, S1)b: GetByOffset(@o)c: CheckStructure(@b, S2)d: GetByOffset(@c)

Property Type Inference


{f: , …)


{f: , …)

{g: , …)


{f: , …)

{g: , …)S1:

{f, …}


{f: , …)

{g: , …)S1:

{f, …}

S2: {g, …}


{f: , …)

{g: , …)S1:

{f, …}

S2: {g, …}Inferred Type

Table


{f: , …)

{g: , …)S1:

{f, …}


Table

f: S2


• Stores to f check type{f: , …)

{g: , …)S1:

{f, …}


Table

f: S2


• Stores to f check type

• Loads don’t have to

{f: , …)

{g: , …)S1:

{f, …}


Table

f: S2


• Stores to f check type

• Loads don’t have to

• S1’s inferred type table watches S2’s transitions

{f: , …)

{g: , …)S1:

{f, …}


Table

f: S2

Inferred Type “Speed-up”

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×

2.57×Poly Poly IC Inlining + Taildup+ Inferred Type


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×


!!!


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×


!!!

!!!


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×


!!!

!!!

p = 0.03


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×


!!!

!!!

p = 0.03

p = 0.08


JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.04×

2.60×

1.39×

6.23×


!!!

!!!

p = 0.03

p = 0.08

p = 0.02

function foo(){ class Helper { … } var h = new Helper(); …}


P1

S1


P1

S1 P2

S2


P1

S1 P2

S2 P3

S3


P1

S1 P2

S2 P3

S3 P4

S4


P1

S1 P2

S2 P3

S3 P4

S4 P5

S5


P1

S1 P2

S2 P3

S3 P4

S4 P5

S5 P6

S6

{1, 2}

{42, 3}{-5, 7}

{x, y}

proto

global object

Mono Proto

{1, 2}

{42, 3}{-5, 7}

{x, y}global object

Poly Proto

proto1

proto2

Poly Proto Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.27×

2.61×

1.37×

6.04×

2.60×Poly Poly + Taildup + Inferred Type+ Poly Proto

Poly Proto Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.27×

2.61×

1.37×

6.04×


p = 0.8

Poly Proto Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.27×

2.61×

1.37×

6.04×


p = 0.8

p = 0.02

Poly Proto Speed-up

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5× 7× 7.5×

1.37×

6.27×

2.61×

1.37×

6.04×


p = 0.8

p = 0.02

p = 0.7

How I Ran These Experiments

• WebKit trunk revision 233406

• Safari trunk hash dd8296dbaac7afa5ed9a699aa261033ea5f5577c

• macOS Internal SDK

• make release (not a root, no shared cache)

• Patch, scripts, and raw data at webkit.org/b/187414

• MacBookPro11,5 2.8GHz 16GB RAM 1TB SSD

http://webkit.org/b/187414

JetStream

ARES-6

Speedometer 2

1× 1.5× 2× 2.5× 3× 3.5× 4× 4.5× 5× 5.5× 6× 6.5×1.37×

6.27×

2.605×

Self+ Proto+ New Property+ Transition Watch+ Replacement Watch+ Polymorphic+ Miss+ Accessors+ Monomorphic Inlining+ Minimorphic Inlining+ Polymorphic Inlining+ Polyvariant Inlining+ Taildup+ Inferred Types+ Poly Proto

Inline Caching in JavaScriptCore · “Inline Cache” indexing type ﬂags cell state structure...

Documents

Transcript of Inline Caching in JavaScriptCore · “Inline Cache” indexing type ﬂags cell state structure...