Stuff I Learned About Performance
56
Stuff I’ve learnt about software performance @mikeb2701 learned
-
Upload
michael-barker -
Category
Software
-
view
279 -
download
3
description
Slides from Auckland JVM meetup.
Transcript of Stuff I Learned About Performance
Stuff I’ve learnt about software performance
@mikeb2701
learned
• What is a financial exchange?
• How to lie with numbers
• Design is (almost) everything
• Everything else is about making trade-offs
TL;DR
1. Low Latency
2. ???
3. Profit
Lies, Damed Lies and Performance Numbers
320µs @ 10,000 tx/s
Customer
FixGateway
LoadBalancer
ExecutionVenue
Record Latency
Parse/Encode FIX
Order MatchingReplicationJournalling
320µs @ 10,000 tx/s
320µs @ 10,000 tx/s
http://www.infoq.com/presentations/latency-pitfalls
What did I learn
• Detail Matters
• Don’t oversimplify the behaviour of your system
If we can really understand the problem, the answer will come out of it, because the answer is
not separate from the problem. !
- Jiddu Krishnamurti
Customer
FixGateway
LoadBalancer
ExecutionVenue
Record Latency
Parse/Encode FIX
Order MatchingReplicationJournalling
Instruction
ExecutionReport
Execution Venue
OrderBook
Order
Execution
Execution Venueinstrument: Carrots
CreateOrderBook<Admin Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
instrument: Carrots
CreateOrderBook<Admin Instruction>
id : ABCinstrument: Carrotsprice : 110quantity : 5side : BID
PlaceOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
id : ABCinstrument: Carrotsprice : 110quantity : 5side : BID
PlaceOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 0cancelled : 0
Order<Entity>
id : ABCinstrument: Carrotsprice : 110quantity : 5side : BID
PlaceOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 0cancelled : 0
Order<Entity>id : ABC
Accept<ExecutionReport>
instrument: Carrotsprice : 110quantity : 5
LiquidityAdded<ExecutionReport>
Execution Venue
instrument: Carrots
OrderBook<Entity>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 0cancelled : 0
Order<Entity>
originalId: ABC
CancelOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
originalId: ABC
CancelOrder<Instruction>
id : ABCinstrument: Carrotsprice : 110quantity : 5
Cancelled<ExecutionReport>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 0cancelled : 5
Order<Entity>
Execution Venue
instrument: Carrots
OrderBook<Entity>
originalId: ABC
CancelOrder<Instruction>
id : ABCinstrument: Carrotsprice : 110quantity : 5
Cancelled<ExecutionReport>
Execution Venue
instrument: Carrots
OrderBook<Entity>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 0cancelled : 0
Order<Entity>
id : DEFinstrument: Carrotsprice : 110quantity : 5side : ASK
PlaceOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
id : DEFinstrument: Carrotsprice : 110quantity : 5side : ASK
PlaceOrder<Instruction>
id : DEF
Accept<ExecutionReport>
price : 110quantity : 5aggressive: DEFpassive : ABC
Trade<ExecutionReport>
orderId : 1id : ABCprice : 110quantity : 5side : BIDfilled : 5cancelled : 0
Order<Entity>
Execution Venue
instrument: Carrots
OrderBook<Entity>
id : DEFinstrument: Carrotsprice : 110quantity : 5side : ASK
PlaceOrder<Instruction>
id : DEF
Accept<ExecutionReport>
price : 110quantity : 5aggressive: DEFpassive : ABC
Trade<ExecutionReport>
Execution Venue
instrument: Carrots
OrderBook<Entity>
id : ABCinstrument: Bananasprice : 110quantity : 5side : BID
PlaceOrder<Instruction>
Execution Venue
instrument: Carrots
OrderBook<Entity>
id : ABCinstrument: Bananasprice : 110quantity : 5side : BID
PlaceOrder<Instruction>
id : ABC
Reject<ExecutionReport>
Interesting Properties
• Stable working set
• Fits in memory
• Doesn’t not grow monotonically with time
• All decisions are made from local information
Event-Sourced Service
Journalling
Replication
BusinessLogicMessage Message
http://www.infoq.com/presentations/Event-Sourced-Architectures-for-High-Availability
What did I learn• Enough understanding of the problem and
solutions will appear
• Pay attention to:
• Entity Life-cycles
• Number of entities - in real world scenarios
• Try to approach the problem without bias
If you can’t talk about the downside and the trade offs then you’re just a fan person waiving a coffee mug around. That type of behaviour
really doesn’t benefit anyone. !
- Some guy on the Internet (charmalloc)
Source: http://allthingshadoop.com/2013/12/06/technology-decisions-are-about-trade-offs-and-solving-problems/
Customer
FixGateway
LoadBalancer
ExecutionVenue
Market DataService
10K msg/sec
160K msg/sec
Design for trade-offs
TIMESTAMP,BID_PRICE_1,BID_QTY_1,ASK_PRICE_1,ASK_QTY_1 1405890300827,1.35263,50,1.35296,50 1405890300908,1.35263,50,1.35296,49.9 1405890300909,1.35263,50,1.35296,47.9 1405890301090,1.35263,48.2,1.35296,47.9 1405890301193,1.35263,48.2,1.35296,46.1 1405890301194,1.35263,48.2,1.35296,44.6 1405890301280,1.35263,48.2,1.35296,42.6 1405890301571,1.35263,48.1,1.35296,42.6 1405890303037,1.35263,47.1,1.35296,42.6
List<String[]> lines = ! parseAsList("data.csv");!for (String[] line : lines)!{! double bid = parseDouble(line[1]);! double ask = parseDouble(line[3]);! ! // Stuff...!}!
Iterable<String[]> lines = ! parseAsIterable("data.csv");!for (String[] line : lines)!{! double bid = parseDouble(line[1]);! double ask = parseDouble(line[3]);! ! // Stuff...!}!
public interface CsvObservable!{! default void onError(Exception e)! {! }!! default void onComplete()! {! }!! void onEvent(! int row, int column, ! CharSequence value, boolean endOfLine);!}
parse("data.csv", ! (row, column, value, endOfLine) ->!{! switch (column)! {! case 1:! bid = parseDouble(value);! case 3:! ask = parseDouble(value);! }!! // Stuff...!});
Ops/sec GC Count GC Time (ms)
Iterable 8.20 104 154
Observable 16.75 0 0
public class CsvObservableDelegate ! implements CsvObservable<CharSequence>!{! private CsvObservable<String> delegate;!! public CsvObservableDelegate(CsvObservable<String> delegate)! {! this.delegate = delegate;! }!! public void onEvent(! int row, int column, ! CharSequence value, boolean endOfLine)! {! delegate.onEvent(! row, column, value.toString(), endOfLine);! }!! // Other delegate methods!}!
Garbage CollectionStallin’ your app since 1959
mean 95% 99% 99.99%
JDK6 1.386 1.82 5.75 15.72
JDK7 1.44 1.79 11.75 20.66
Solutions
• Don’t Generate Any • Use Zing (Azul Systems)
Why Use Java? (why not C/C++/..)
Source: https://software.intel.com/en-us/articles/introduction-to-intel-advanced-vector-extensions
public class Doubles !{ ! public static void sum(! double[] sum, ! double[] addendA, ! double[] addendB)! {! for (int i = 0; i < 0 + sum.length; i++)! {! sum[i] = addendA[i] + addendB[i];! }! }!}!
#include <immintrin.h>!!void sum(double* c, double* a, double* b, int len)!{! __m256d rA_AVX, rB_AVX, rC_AVX;!! for (int i = 0; i < len; i += 4)! {! rA_AVX = _mm256_load_pd(&a[i]);! rB_AVX = _mm256_load_pd(&b[i]);! rC_AVX = _mm256_add_pd(rA_AVX, rB_AVX);! _mm256_store_pd(&c[i], rC_AVX);! }!}!
More Java vs. C++
https://github.com/real-logic/simple-binary-encoding
What did I learn?• Details matter (performance measurement) • Let the problem define the solution
• Design without bias • Understand the costs/benefits of trade-offs
• Design to let others make trade-offs • Some trade-offs are not technical
Q&A
Links• Measuring Latency - http://goo.gl/6bllUP
• Event Sourcing - http://goo.gl/VcZI7l
• LMAX Architecture - http://goo.gl/UI5z0T
• Source Code
• http://github.com/mikeb01/qconsf2014
