Virtual Queues as a Trade Processing Pattern
description
Transcript of Virtual Queues as a Trade Processing Pattern
![Page 1: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/1.jpg)
Virtual Queues as a Trade Processing Pattern
Uri Cohen @uri1803 | github.com/uric
Head of Product @ GigaSpaces
![Page 2: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/2.jpg)
Event Processing at Massive Scale Approaches to Concurrency
Uri Cohen @uri1803 | github.com/uric
Head of Product @ GigaSpaces
![Page 3: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/3.jpg)
This is What It Used to Be Like
![Page 4: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/4.jpg)
That’s What
It’s Like Now
![Page 5: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/5.jpg)
Some Numbers
15 Billion Trades / Day on
NYSE alone
http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html
![Page 6: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/6.jpg)
Some Numbers
That’s
641K Trades / Second
http://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html
![Page 7: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/7.jpg)
Some Numbers
12 Billion Shares change
hands every day
http://www.bloomberg.com/news/2012-01-23/stock-trading-is-lowest-in-u-s-since-2008.html
![Page 8: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/8.jpg)
Some Numbers
$4 Million The cost of 1
millisecond of latency to a broker
http://www.tabbgroup.com/PublicationDetail.aspx?PublicationID=346
![Page 9: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/9.jpg)
The Problem
Massive stream of events
Time is money, literally
Can’t lose a single message
Fairness is a must
![Page 10: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/10.jpg)
Order Book -
Simplistic Example
Buy Sell 50, $12 60, $10
60, $11 100, $11
30, $10 30, $12
![Page 11: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/11.jpg)
Buy Sell 50, $12 60, $10
60, $11 100, $11
30, $10 30, $12
Price: $10
Order Book -
Simplistic Example
![Page 12: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/12.jpg)
Buy Sell 60, $11 10, $10
30, $10 100, $11
30, $12
Price: $10
Order Book -
Simplistic Example
![Page 13: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/13.jpg)
Buy Sell 60, $11 10, $10
30, $10 100, $11
30, $12
Price: $10
Order Book -
Simplistic Example
![Page 14: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/14.jpg)
Buy Sell 50, $11 100, $11
30, $10 30, $12
Price: $10
Order Book -
Simplistic Example
![Page 15: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/15.jpg)
Buy Sell 50, $11 100, $11
30, $10 30, $12
Price: $11
Order Book -
Simplistic Example
![Page 16: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/16.jpg)
Buy Sell 30, $10 50, $11
30, $12
Price: $11
Order Book -
Simplistic Example
![Page 17: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/17.jpg)
What it Really Means
Low latencyIn memory, GC tuning
Scalability Multi-coreMulti-node
OrderingBy price, order time
Exclusivity
Resiliency
![Page 18: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/18.jpg)
Trading is Just One Use Case
All things FCFS, with a limited stock
Flight booking
Betting
Online Auctions
Cloud Spot Instances
eCommerce
![Page 19: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/19.jpg)
Let’s Talk Solutions
![Page 20: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/20.jpg)
Queue (SEDA/Actor Style)
Not Validated Validated Processed
Validator Processor
![Page 21: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/21.jpg)
Queue (SEDA /Actor Style)
The Good: Ordered (Is it fair?)Multi-threaded
The Bad: Not very scalable
LockingContext
switchingTransient
![Page 22: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/22.jpg)
The Cost of Locking
Method Time in msec
Single Thread 300
Single Thread w/ Lock 10.000
2 Threads w/ Lock 224.000
Single Thread w/ CAS 5.700
2 Threads w/ CAS 30.000
Single Thread w/ Volatile Write 4.700
http://disruptor.googlecode.com/files/Disruptor-1.0.pdf
![Page 23: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/23.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
60
50 Buy Sell 100
![Page 24: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/24.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
50
60
Buy Sell 100
![Page 25: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/25.jpg)
Queue (Lack of) Fairness
Consumer Thread 1
Consumer Thread 2
Buy Sell 100
60 50
Can you tell which order will be executed 1st?
![Page 26: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/26.jpg)
Single-Threaded Queue
Validator Processor
![Page 27: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/27.jpg)
Single- Threaded
Queue
The Good: Fast, very fast
No contentionNo context
switchesAlways fair
The Bad: Multi-core?
Not fit for Intense compute & I/O
Need to be async.Transient
![Page 28: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/28.jpg)
Single- Threaded
Queue
They do it…
![Page 29: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/29.jpg)
Disruptor (LMAX)
![Page 30: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/30.jpg)
Segmented Queue
Symbol=A-H Symbol=I-S Symbol=T-Z
Validator Processor Processor
Processor thread pool per segment
![Page 31: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/31.jpg)
Segmented Queue - Optimization Single Processor thread pool, pick random segment
Symbol=A-H Symbol=I-S Symbol=T-Z
Processor
![Page 32: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/32.jpg)
Segmented Queue
The Good: Scalable
But segments can get hot
Minimizes contention
The Bad: Not trivial to
implementStill unfair
Is total ordering needed?
Transient
![Page 33: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/33.jpg)
What about
Fairness?
Exclusivity is Key Process one message for each segment at the same time
No exclusivity across segments
![Page 34: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/34.jpg)
Implicit ExclusivitySingle processor thread per segment
Symbol=A-H Symbol=I-S Symbol=T-Z
Processor Processor Processor
![Page 35: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/35.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1
Segment 2
Segment 3
![Page 36: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/36.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1 X
Segment 2
Segment 3
![Page 37: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/37.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1 X
Segment 2
Segment 3 X
![Page 38: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/38.jpg)
Explicit ExclusivityShared thread pool, mark segments under processing (CAS)
Segment 1 Segment 2 Segment 3
Processor
Segment 1
Segment 2
Segment 3 X
![Page 39: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/39.jpg)
Explicit Exclusivity
Num. of segments is keyToo few: little concurrencyToo many: wasting memory
![Page 40: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/40.jpg)
Dynamic Segmentation Segments are created and removed as needed
Processor
![Page 41: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/41.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
“GOOG”
![Page 42: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/42.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
“GOOG”
![Page 43: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/43.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
GOOG
AAPL
“AAPL”
![Page 44: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/44.jpg)
Dynamic Segmentation Segments are created and removed as needed
“GOOG”
Processor
GOOG X
AAPL
AMZN
“AAPL” “AMZN”
![Page 45: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/45.jpg)
Dynamic Segmentation
Segments created as needed
Randomize on segments until available one found
Fast, scalable, fair
We call it “FIFO groups” or “Virtual Queues”
![Page 46: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/46.jpg)
It Can (and Does) Get Much
More Complex
Memory state can get corrupt on errors
It’s not always as simple as “pop off the queue”
limits, priorities, circuit breakers, etc.
Resiliency is always a pain
![Page 47: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/47.jpg)
A Bit about Usability
What you don’t want to do
Implement data structuresHandle concurrencyHandle HA Handle transactions
![Page 48: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/48.jpg)
A Bit about Usability
What you want to control
Event flow Grouping attribute (e.g
symbol)Event handlers
![Page 49: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/49.jpg)
Data Grid as a
Foundation
Transactional
Highly available
Supports complex matching
![Page 50: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/50.jpg)
How We Thought
of It
![Page 51: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/51.jpg)
How We Thought
of It
![Page 52: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/52.jpg)
How We Thought
of It
![Page 53: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/53.jpg)
How We Thought
of It
![Page 54: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/54.jpg)
How We Thought
of It
![Page 55: Virtual Queues as a Trade Processing Pattern](https://reader036.fdocuments.in/reader036/viewer/2022062310/5681649c550346895dd67f25/html5/thumbnails/55.jpg)
Thank You!
References: http://martinfowler.com/articles/lmax.htmlhttp://www.nytimes.com/2011/08/27/business/as-trade-volumes-soar-exchanges-cash-in.html http://disruptor.googlecode.com/files/Disruptor-1.0.pdfhttp://www.gigaspaces.com/wiki/display/XAP9/FIFO+Grouping