Why Java Works (Well) for Low-Latency Trading Systems · 5 Things People Think They Know about Java...

© 2013 Azul Systems 1 1

Why Java Works (Well) for

Low-Latency Trading

Systems

Howard Green

Azul Systems Inc.


Who we are

© 2013 Azul Systems 3 3 3 3

Azul Systems: Deep Java DNA

• Award-Winning Leader in Java Technology

• Founded in 2002, privately held; based in Sunnyvale, CA

• Trusted partner to the financial sector for over ten years

• Azul CTO Gil Tene on the JCP Executive Committee

• Unique technology and IP in Java runtimes and performance ─ Over 40 patents issued, 15 additional filed

• Deep low-latency Java expertise

• Strong financial partner & ISV ecosystem


Azul: Proven in Capital Markets

Java use cases:

• HFT and Algo Trading

Platforms

• FX, Fixed Income

• Real-time Risk

• Hedge Funds (CEP)

• DMA, Matching

Engines, FIX engines,

Messaging, BI, & more

• Low-latency Java is

here – and improving


So…

Why do people use Java for

low latency apps?

Java in the low latency world?


Are they crazy?


Are they crazy?

• No.

• There are good, easy to articulate reasons:

• Projected lifetime cost

• Developer productivity

• Time-to-product, Time-to-market, ...

• Leverage, ecosystem, ability to hire


Why use Java in Algo Trading?

• One customer’s perspective:

“Our strategies have a shelf life”

We have to keep developing and deploying new ones

Only one out of N is actually productive

Profitability therefore depends on ability to successfully deploy new strategies, and on the cost of doing so

Our developers seem to be able to produce 2x-3x as much when using a Java environment as they would with C/C++ ...


Some Low-Latency

Java Examples

© 2013 Azul Systems 10 10 10 10

FX Example: LMAX Exchange

• London-based FX exchange

• Primarily Open Source infrastructure

• 3 ms average latency; 100% Uptime, 500 usec internal latency

• LMAX Disruptor: 6 million TPS in 2011

“LMAX Exchange are already the fastest venue for FX and

our deep technological collaboration with Azul will help to

ensure we continue to deliver even faster execution for our

clients in the future."

-- Edward McDaid, CIO, LMAX Exchange

© 2013 Azul Systems 11 11 11 11

Algo Example: OptionsCity

• Java-Based (Some Scala) ─ Quoting and execution

─ Options pricing and modeling

─ Algo trading

─ Integrated risk management

• Co-location next to matching engines at key exchanges

• Under 100 µsec round trip


What’s Driving Java LL

Performance?

© 2013 Azul Systems 13 13 13 13

5 Things People Think They Know about Java

1. Java is inherently slower

2. My team of C and C++ developers can write faster code

3. Java isn't for low-latency applications

4. Garbage Collection makes Java impractical for Trading

5. Java heap sizes over 10 GB are asking for trouble

© 2013 Azul Systems 14 14 14 14

Is Java Slow?

No • A good programmer will get roughly the same speed

from both Java and C++

• A bad programmer won’t get you fast code on either

• The 50%‘ile and 90%‘ile are typically excellent...

© 2013 Azul Systems 15 15 15 15

Tiered Compilation

• Today’s C1 and C2 compilers generate high-quality code

• C1: “Fast and Dumb” – 5X-10X interpreter speed ─ Smaller code

─ Lower-quality

• C2: Slower and Smarter – 30-50% faster than C1 ─ Bigger (aggressive/ more inlining)

─ Much higher peak throughput

• Tiered (default) behavior – Both are used ─ C1 for fast startup

─ C2 for peak performance

• Ensure sufficient iterations for optimization

© 2013 Azul Systems 16 16 16 16

Memory Management

• Garbage Collectors are getting better

• CMS can be very good in tuned applications

• Some Jrockit features being added to Oracle HotSpot

• Traditional GC target peaks in low milliseconds

• Object allocation much faster than malloc

• Can drive peak latencies far lower

• With aggressive JVM tuning and optimized application

• Tens of microseconds peak latency is the rough current state of the industry

© 2013 Azul Systems 17 17 17 17

Thread handling

• Concurrency has been key to Java since its inception

• Moore’s law / human capabilities

• Accelerates developer productivity

• Asynchronous development simplified

• Very fast, low overhead

• Originally depended upon operating system

• Major advantage of Java – the JVM does more

© 2013 Azul Systems 18 18 18 18

“Mechanical Sympathy”

• cf: Martin Thompson’s blog

• Modern Intel-architecture servers have lots of cores

• …And very good cache designs

• Cache miss – “lost opportunity for 500 instructions”

• Linux optimizations can help tremendously

• Power management can generate more jitter than GC does

• Designers that understand how to keep key data in cache…

• Locking threads to particular sockets (or cores)

• Designing for few (1) writers and many readers

© 2013 Azul Systems 19 19 19 19

Better Open Source/ More Profiling Tools

• Use of Open Source throughout the financial sector has become widespread

• Driven by replatforming onto Linux

• Good frameworks and other OSS initiatives help jumpstart development

• Java profiling tools are extremely robust ─ AppDynamics

─ New Relic

─ Jprofiler

─ VisualVM

─ Many, many more

© 2013 Azul Systems 20 20 20 20

Human Capital is Key

• There’s a worldwide war for talent

• “The New Kingmakers” – James Governor of RedMonk

• You need to provide the best tools and most interesting challenges for your developers

• And you’d rather use your best and most creative people to build new functionality and enable new offerings, instead of tuning glitchy legacy systems

• Java helps keep your best people engaged in building your business


Any issues for Java in

Trading Systems?

© 2013 Azul Systems 22 22 22 22

• Example: some Java-based systems look like this:

Traditional JVM: Typical case is fine; worst case more than 20x too high

Java in Today’s Electronic Markets

© 2013 Azul Systems 23 23 23 23

Is “jitter” even the right word for this?

99%‘ile is ~60 µsec

Max is ~30,000%

higher than “typical”

© 2013 Azul Systems 24 24 24 24

What do actual L-L developers do?

• Today…

• They use “Java” instead of Java

• They write “in the Java syntax”

• They avoid allocation as much as possible

• E.g. They build their own object pools for everything

• They write all the code they use (no 3rd party libs)

• They train developers for their local discipline

• In short: They revert to many of the practices that hurt productivity. They lose out on much of Java.


<And a very brief commercial>

Summing Up…

© 2013 Azul Systems 27 27 27 27 Side-by-side – Unmodified Low latency App - Drawn to scale…

Standard JVM

(pure newgen) Zing JVM

© 2013 Azul Systems 28 28 28 28

The Zing JVM eliminates the SLA issues and performance glitches people spend weeks (or even months) trying to solve

Zing delivers lower averages, eliminates latency spikes

30-50 sec peaks in highly tuned applications

•

Java for Today’s Trading Systems

© 2013 Azul Systems 29 29 29 29

In one slide…

Real Java is finally viable for low-latency applications

Garbage Collection is no longer a dominant issue

…Even for outliers

No need to code in special ways any more

You can finally use “real” Java for everything

You can finally 3rd party libraries without worries

You can finally use as much memory as you want

You can finally use regular (good) programmers

©2013 Azul Systems, Inc. 30

Azulsystems.com/solutions/onWallSt

Azulsystems.com/solutions/low-latency

Azulsystems.com/solutions/big-data

www.azulsystems.com/resources

LinkedIn: /company/azul-systems

Twitter: @azulsystems

FaceBook: AzulSystemsInc

Email: [email protected]

Phone: +1 650 230 6616

Got Questions? Just Ask Us

For more information:

© 2013 Azul Systems 31 31 31 31

Everybody knows Java tuning can be tough

java -Xmx12g -XX:MaxPermSize=64M -XX:PermSize=32M

-XX:MaxNewSize=2g -XX:NewSize=1g -XX:SurvivorRatio=128

-XX:+UseParNewGC -XX:+UseConcMarkSweepGC

-XX:MaxTenuringThreshold=0 -XX:CMSInitiatingOccupancyFraction=60

-XX:+CMSParallelRemarkEnabled -XX:+UseCMSInitiatingOccupancyOnly

-XX:ParallelGCThreads=12 -XX:LargePageSizeInBytes=256m

java –Xms8g –Xmx8g –Xmn2g -XX:PermSize=64M -XX:MaxPermSize=256M

-XX:-OmitStackTraceInFastThrow -XX:SurvivorRatio=2

-XX:-UseAdaptiveSizePolicy -XX:+UseConcMarkSweepGC

-XX:+CMSConcurrentMTEnabled -XX:+CMSParallelRemarkEnabled

-XX:+CMSParallelSurvivorRemarkEnabled

-XX:CMSMaxAbortablePrecleanTime=10000

-XX:+UseCMSInitiatingOccupancyOnly

-XX:CMSInitiatingOccupancyFraction=63 -XX:+UseParNewGC

–Xnoclassgc

Two real-world Java tuning settings. And they contradict each other.

© 2013 Azul Systems 32 32 32 32

…Except with Zing

java –Xmx40g

Just pick a good starting point and let Zing tune itself

Now you can focus more on new capabilities

And keep your best talent developing instead of tuning

Why Java Works (Well) for Low-Latency Trading Systems · 5 Things People Think They Know about Java...

Documents

Transcript of Why Java Works (Well) for Low-Latency Trading Systems · 5 Things People Think They Know about Java...