FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(•...
Transcript of FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(•...
![Page 1: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/1.jpg)
FaBRiQ: Fast, Balanced and Reliable Queue
Iman Sadooghi
![Page 2: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/2.jpg)
Outline
• Ka;a • SQS • Fabriq
– Mo?va?on – Design – features – Communica?on cost analysis
• Performance evalua?on
![Page 3: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/3.jpg)
Ka;a
![Page 4: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/4.jpg)
Mo?va?on • Log Data
– Orders of Magnitude larger than the actual data – Facebook 5TB daily – China Mobile 5-‐8 TB daily
• Many types of events • user ac?vity events: impression, search, ads, etc • opera?onal events: call stack, service metrics, etc
• High volume: billions of events per day • Both online and offline use case
• repor?ng, batch analysis • security, news feeds, performance dashboard
![Page 5: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/5.jpg)
Solu?on!
• Tradi?onal Messaging Systems – JMS
• Acknowledge aTer msg consump?on • Weak distributed support • No batching
– IBM WebSphere MQ • Provides transac?onal support!
– Ac?veMQ • Assuming msgs should be consumed real quick • No offline support • No focus on throughput
![Page 6: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/6.jpg)
Log Aggregators
• Collect data and load into DWH or Hadoop – Facebook Scribe
• Periodically dumps bunch to HDFS – Cloudera Flume
• Uses push approach – Yahoo Data Highway
• Problem – All for offline data consump?on – No online consump?on support
![Page 7: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/7.jpg)
Message queues
• Ac?veMQ
• TIBCO
Log aggregators
• Flume
• Scribe
• Low throughput
• Secondary indexes
• Tuned for low latency
• Focus on HDFS
• Push model • No rewindable consump?on
KAFKA
![Page 8: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/8.jpg)
Ka;a
• Collect and deliver high volumes of large data • Low latency • Scalable
![Page 9: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/9.jpg)
Design • Problem: point to point pipelines!!
OracleOracle
Oracle User Tracking
HadoopLog
SearchMonitoring
Data
Warehous
e
Social
Graph
Rec.
EngineSearch Email
VoldemortVoldemort
Voldemort
EspressoEspresso
EspressoLogs
Operational
Metrics
Production Services
...Security
![Page 10: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/10.jpg)
Design
OracleOracle
Oracle User Tracking
HadoopLog
Search
Monitorin
g
Data
Warehous
e
Social
Graph
Rec
EngineSearch Email
VoldemortVoldemort
Voldemort
EspressoEspresso
EspressoLogs
Operational
Metrics
Production Services
...Security
Data Pipeline
![Page 11: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/11.jpg)
Pub-‐Sub
Producer Consumer
Producer
Consumer
Topic 1
Topic 2
Topic 3
subscribe publish(topic, msg)
Publish subscribe system
msg
msg
![Page 12: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/12.jpg)
Design
• Producer, Consumer • Broker • Topic, Par??on
• Load balance
![Page 13: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/13.jpg)
Efficiency
• No message IDs – Logical offset
• Consumer consumes msgs from a Par??on sequen?ally (star?ng on an offset)
• Single par??on in a topic – Only used by single consumer
• No Master – Consumers and brokers coordinate via ZooKeeper
![Page 14: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/14.jpg)
Stateless Broker
• Broker doesn’t keep track of consump?on • Each consumer maintains its own state • Message dele?on driven by reten?on policy, not by tracking consump?on – rewindable consumer
![Page 15: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/15.jpg)
ZooKeeper
• Create a path • Set value to path • Read value on path • Delete path • Get no?fica?ons on a path • Provides replica?on
![Page 16: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/16.jpg)
Auto Consumer Load Balance
• brokers and consumers register in zookeeper • consumers listen to broker and consumer changes • each change triggers consumer rebalancing
![Page 17: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/17.jpg)
Guarantees
• At least once delivery • In order delivery, inside a single par??on • No guarantee on order from diff par??ons • No support for duplicated messages • Persistence
– If broker goes down • msgs temp unavailable
– If broker disk damaged • Msgs lost permanently
![Page 18: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/18.jpg)
Usage in LinkedIn
![Page 19: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/19.jpg)
Hadoop Data Load for Ka;a
Live data center Offline data center
Hadoop Hadoop Dev Hadoop
Frontend Frontend Real ?me consumers
Ka;a Ka;a Ka;a Ka;a Ka;a Ka;a
Hadoop Hadoop PROD Hadoop
![Page 20: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/20.jpg)
Performance Evalua?on § 2 Linux boxes
• 16 2.0 GHz cores
• 6 7200 rpm SATA drive RAID 10
• 24GB memory
• 1Gb network link
§ 200 byte messages
§ 10 million msgs in total
§ Batch size: § 1: 50K msgs/sec
§ 50: 400K msgs/sec
![Page 21: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/21.jpg)
Producer performance
![Page 22: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/22.jpg)
Consumer performance
![Page 23: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/23.jpg)
Scalability
101
190
293
381
0
50
100
150
200
250
300
350
400
1 broker 2 brokers 3 brokers 4 brokers
Throughp
ut in M
B/s
(10 topics, broker flush interval 100K)
![Page 24: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/24.jpg)
SQS
• Amazon Simple Queue Service (SQS) – Distributed message delivery queue
• Highly scalable • Messages sent and read simultaneously • Reliable
– Guarantees message delivery » At least once delivery
![Page 25: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/25.jpg)
Outline
• Ka;a • SQS • Fabriq
– Mo?va?on – Design – features – Communica?on cost analysis
• Performance evalua?on
![Page 26: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/26.jpg)
Mo?va?on • More than 2.5 exabytes of data is generated every day – more than %70 of it is unstructured
• not possible for the tradi?onal data processing systems to handle needs in Big Data processing. – There is a need to reinvent the wheel instead of using the tradi?onal systems
• Tradi?onal data processing middleware being replaced: – SQL databases by No-‐SQL datastores – file system by key-‐value storage systems
![Page 27: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/27.jpg)
Mo?va?on • A Distributed message queue • useful in various data movement and communica?on scenarios
• monitoring, • workflow applica?ons, • big data analy?cs, • log processing
• Companies started using queues: – Linkedin, Facebook, Cloudera and Yahoo have developed similar queuing
solu?ons • to handle gathering and processing of terabytes of log data on their servers
– Ka;a feeds hundreds of gigabytes of data into Hadoop clusters and other servers every day
• Queues can play an important role in Many Task Compu?ng (MTC) and High Performance Compu?ng (HPC) – handle data movement on HPC and MTC workloads in larger scales without
adding significant overhead to the execu?on process – CloudKon
![Page 28: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/28.jpg)
Challenges
• Tradi?onal queue services – usually have centralized architecture – cannot scale well to handle today’s big data requirements
– Providing transac?onal support – Providing consump?on acknowledgement – Persistence: Many are in memory queues. – Delivery guarantee!
![Page 29: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/29.jpg)
Introducing FaBRiQ
• FaBRiQ (Fast, Balanced and Reliable Queue): – a persistent message queue that aims to achieve high throughput and
– low latency – while keeping the near perfect load balance and high u?liza?on on large scales
– Uses ZHT as its building block • Communica?ons • Storing data
![Page 30: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/30.jpg)
Design
• Distribu?ng queues among servers
![Page 31: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/31.jpg)
FaBRiQ Server
![Page 32: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/32.jpg)
Opera?ons
• CreateQueue • Push • Pop • Remove
![Page 33: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/33.jpg)
Push
![Page 34: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/34.jpg)
pop
• Keep the latency low • steps to follow
– Local Access – Random server – Last known server – Metadata list owner
• Redirect to a message keeper
![Page 35: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/35.jpg)
Pop
![Page 36: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/36.jpg)
Load Balancing
• Load Balancing – Using a uniformly distributed hash func?on.
![Page 37: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/37.jpg)
Features
• Order of messages • Message delivery guarantee • Persistence • consistency and fault tolerance • Mul?threading
![Page 38: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/38.jpg)
Communica?on cost analysis
• Push: 1 hop • Pop: 0 – 3 hops
![Page 39: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/39.jpg)
Comparison Feature Fabriq Ka@a SQS
Persistence Yes Yes Yes
Delivery Guarantee Exactly Once At least Once At least Once
Message Order Inside Node Inside Node -‐
ReplicaJon Customizable Mirroring 3x
Shared Pool Yes No Yes
Batching No (Future work) Yes Yes
![Page 40: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/40.jpg)
Performance Evalua?on
• FaBRiQ, Ka;a, SQS – Latency – Throughput
• Measuring: – Push / produce – Pop / consume
![Page 41: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/41.jpg)
Latency (short messages)
![Page 42: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/42.jpg)
Latency (CDF)
![Page 43: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/43.jpg)
Throughput (short messages)
![Page 44: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/44.jpg)
Throughput (large messages)
![Page 45: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/45.jpg)
Duplicate messages Scale -‐ #msgs Ka@a SQS
1 -‐ 1000 0 0
2 -‐2000 0 1
4 -‐ 4000 0 5
8 -‐ 8000 1 5
16 -‐ 16000 2 8-‐11
32 -‐ 32000 4 8-‐15
64 -‐ 64000 4 20-‐25
![Page 46: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/46.jpg)
Future work
• Batching
![Page 47: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/47.jpg)
Introduc?on
• Large Scale Task Execu?on – Run on distributed resources
– Workloads • Tasks
– More in number – Shorter in length
– Requirements for high performance • Concurrency • Load Balance • System U?liza?on
![Page 48: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/48.jpg)
State-‐of-‐the-‐art job schedulers
− Centralized Master/Slaves architecture − Scalability issues at petascale and beyond − Single point of failure − Example: SLURM, CONDOR, Falkon
![Page 49: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/49.jpg)
State-‐of-‐the-‐art job schedulers
− Distributed Architectures − Hierarchical
− several dispatchers in a tree-‐based topology − Example: Distributed Falkon, Dremel
− Fully distributed − each compu?ng node maintains its own job execu?on − Example: Sparrow, Omega, MATRIX
§ Common challenges − Complex Design and Implementa?on − Load balancing − System u?liza?on
![Page 50: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/50.jpg)
Idea: Scheduling with Message Queues
• Idea: leverage Distributed Message Queues! – Mapping between Job Schedulers and Message Queues
![Page 51: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/51.jpg)
Amazon AWS Cloud • Amazon EC2
• IaaS Cloud Service • Launch VMs and access remotely
• Ability to launch more than 1000 instances
• Amazon Simple Queue Service (SQS) • Distributed message delivery queue
• Highly scalable • Messages sent and read simultaneously • Reliable
• Guarantees message delivery • At least once delivery
![Page 52: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/52.jpg)
Proposed Work • Use SQS as a task delivery component / task pool • Decouple Clients and Workers • Pushing vs. Pulling approach
• Pushing • Local/global manager node needs to predict/decide
• About the address of worker nodes • Underlying network topology
• Pulling • No need to know about workers • Workers decide for themselves
• Load balancing • System Utilization
![Page 53: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/53.jpg)
CloudKon Architecture • MTC
• General format, running MTC tasks • Benefits:
• Dynamic workforce • Non-blocking task submission
• HPC
• Running HPC jobs with multiple tasks
![Page 54: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/54.jpg)
Task consistency • SQS only guarantees at least once delivery • Some workloads require exactly once execution of tasks! • Use DynamoDB to verify • Use conditional write
Ø Write if the task does not exist Ø Throw exception if exists
• Atomic operation
• Using a single operation, the checking is done • Minimize the communication overhead
![Page 55: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/55.jpg)
Communication Cost • Communication overhead is high on Cloud
• Need to minimize the communication
• Message batching • Bundle tasks together to send
• Number of communications • Minimum possible number
![Page 56: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/56.jpg)
Throughput (MTC)
• 1 to 1024 instances,16K to 16.38M tasks • MATRIX and Sparrow crashing on 256 instances
• Too many sockets open on TCP connection
![Page 57: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/57.jpg)
Latency
• Stable latency on different scales
• Ranging from 90 ms to 104 ms
![Page 58: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/58.jpg)
Efficiency – Homogenous Tasks
• MATRIX achieved better efficiency on shorter tasks • Efficiency of MATRIX drops lower than CloudKon on 64 instances • CloudKon is stable and scalable
![Page 59: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/59.jpg)
Efficiency – Heterogenous Tasks
• CloudKon is more stable and scalable compared to the other two. • Efficiency of MATRIX and Sparrow drop lower than CloudKon on 64 instances
• Trace of a real MTC workload • 2.07M tasks at the largest scale • 1 milliseconds to 1 second tasks
![Page 60: FaBRiQ: Fast,(Balanced(and(Reliable(Queue(iraicu/teaching/CS554-S15/lecture07-Fabriq.pdfOutline(• Kaa • SQS • Fabriq(– Mo?vaon(– Design – features(– Communicaon(costanalysis(•](https://reader034.fdocuments.in/reader034/viewer/2022042016/5e749433d721d00bcc1f20e5/html5/thumbnails/60.jpg)
Conclusion
• Design and implement simple yet effective distributed task execution framework • Using cloud services like SQS, DynamoDB
• Run on Public Cloud environment as an alternate resource • Optimum usage of cloud resources
• Outperforming other state of the art systems on larger scales • Sparrow 2013 • MATRIX 2013