Post on 22-Jan-2018
AUTOMATIC CONSTRUCTION OF
INLINING HEURISTICS USING
MACHINE LEARNING
(S. KULKARNI, J. CAVAZOS, C. WIMMER,
D. SIMON, 2013)
NATALLIE BAIKEVICH
@LU-A-JALLA
PAPERS WE LOVE: ZÜRICH
WHY THIS PAPER?
Research papers in the .NET source (post
by Matt Warren, @matthewwarren) :
http://mattwarren.org/2016/12/12/Research-
papers-in-the-.NET-source/
Inspired by Java papers:
https://lowlevelbits.org/java-papers/
MOTIVATION
Inlining can improve performance
A LOT
Things can go REALLY WRONG
The only thing worse than no
optimization is a harmful
optimization
INLINING:
FUNCTION CALL -> BODY
+ eliminates the overhead of a call
+ broadens the scope for other
optimizations
- increases register pressure
- increases memory footprint
- slower compilation
FACTORS
calling context of the method
target platform*
other optimizations
effect of previous inlining decisions
*Java HotSpot & Maxine VM C1X compiler: static,
hard-coded heuristics, same for all platforms
APPROACH AND
IMPLEMENTATION
DEFAULT HEURISTICS AND NEAT
FEATURES
Caller and Callee
- Simple instructions count
- Method calls
- Conditional branches
- Unconditional branches
- Memory load/store
- New objects
- Other instructions
- Size (all instructions)
Calling Context
- In loop or not
- Depth of recursive call
- Depth of non-recursice
call
- Current graph size
- Synchronized or not
HotSpot Only
- Loop depth
- Block weight
SE
NS
IT
IV
IT
Y (S
PE
CJV
M98)
SE
AR
CH
S
PA
CE
(S
PE
CJV
M98)
METHODS
Traditional supervised
learning: requires
labeled training set
Each decision affects
next: difficult to study
the effect
Unsupervised
learning: generate
solution, measure
effectiveness
METRIC: average
speedup
Large space of possible solutions -> ML
INLINING HEURISTIC
(HOTSPOT)
if (calleeSize > ALWAYS_INLINE_SIZE)
if (calleeSize > CALLER_MAX_SIZE)
return NO;
if (inlineDepth > MAX_INLINE_DEPTH)
return NO;if (callWarmth > CALL_WARMTH_THRESHOLD*)
return NO;// Passed all tests so we inline
return YES;
* # of call invocation + est. profit from
inlining + est. amount of work
NEAT: NEURO-EVOLUTION OF
AUGMENTING TOPOLOGIES
Randomly generate an initial generation
of neural networks
Evaluate the performance of each NN
Use the best NNs to produce next
generation
Repeat
Interpretation?
CREATING
HEURISTICS
Input: Features
Multi-layer NN
Output: Number
between 0 and 1
(>0.5 for inline)
RESULTS
BENCHMARKING INLINERS
DECISION TREE FOR
READABLE HEURISTICS
The best NEAT NN -> training dataset
Construct decision tree
Prune
Average performance gain
NEAT 11% over default heuristic
Decision Tree 10%
Probability of
execution
WH
AT
IS
U
SE
FU
L?
Close to size
threshold
BUT
many memory
operations
=> INLINE
Factors
depend on
other factors
BENCHMARKS: ALGORITHMS
BENCHMARKS: FEATURES
INLINING:
GOOD OR EVIL?
MANUAL INLINING EXAMPLES IN F#
OUTSMARTING COMPILERS:
TO BE OR NOT TO BE?
Let’s take .NET world as an example:
JIT-inlining: automatic + manual hints
(e.g. MethodImplAttribute)
Manual inlining (e.g. inline keyword in
F#)
What could possibly go
wrong?
CONCLUSIONS
Inlining is important
NEAT and Decision Trees allow to
construct good heuristics
Average performance improvement
over Java HotSpot VM and Maxine
VM - 11%
Decision Trees for readable
heuristics
REFERENCES: PAPERS
Automatic construction of inlining heuristics using Machine Learning: https://www.eecis.udel.edu/~skulkarn/papers/cgo-2013.pdf
Evolving Neural Networks through Augmenting Topologies: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf
Meta optimization: improving compiler heuristics with Machine Learning:http://groups.csail.mit.edu/commit/papers/03/metaopt-pldi.pdf
Adaptive online context-sensitive inlining:http://www.research.ibm.com/people/d/dgrove/papers/cgo03.pdf
REFERENCES: .NET & JAVA
Research papers in the .NET
source:http://mattwarren.org/2016/12/12/Research-
papers-in-the-.NET-source/
CoreCLR
inlining:https://github.com/dotnet/coreclr/blob/maste
r/src/jit/inline.cpp
CoreCLR
intrinsics:https://github.com/dotnet/coreclr/blob/mas
ter/src/inc/corinfo.h#L908-L965
Java papers: https://lowlevelbits.org/java-papers/
Scala Benchmarking project: http://scalabench.org/