Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream...

27
ELF Efficient Lightweight Fast Stream Processing at Scale* Liting Hu Karsten SchwanHrishikesh Amur Xin Chen Georgia Institute of Technology *Funded in part with support from the Intel Cloud ISTC

Transcript of Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream...

Page 1: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

ELF Efficient Lightweight Fast Stream Processing at Scale* Liting Hu Karsten Schwan Hrishikesh Amur

Xin Chen Georgia Institute of Technology

*Funded  in  part  with  support  from  the  Intel  Cloud  ISTC  

Page 2: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Outline

Motivation

ELF Design

Evaluation

Page 3: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

#TopK

Micro-promotions

Product-bundling

Sales-prediction ... … …

Coupons Recommendations

Predictions

Applications

Page 4: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

In

Filter @item

Sort #clicks

Out

Filter @buys

Out

Query Angry

Birds5?

⋈Yes/No?

Updates every half hour

Updates per min

Updates every x milliseconds

Batch model

Stream model

User query

Concurrent, diverse jobs with varied delays

Page 5: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Performance •  High throughput •  Low latency

Flexible •  Diverse jobs •  Cross-job coordination •  Runtime job change

Scalable •  Hundreds of concurrent jobs with shared inputs

Batch job

Stream job

Query job

ELF

System Needs

Page 6: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

C  

Web server

Log collection systems e.g., Flume, Kafka

Storage

N jobs

M

Existing Solutions

Page 7: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Outline

Motivation

ELF Design

Evaluation

Page 8: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

ELF runs directly on the web server tier

Page 9: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Master

5

hash(‘app2’) =d462ba

hash(‘app1’) =d502fb hash(`app3’)

=2080c3

Join Join

Master

Master

Join

Each organized into a peer-to-peer overlay

Many Masters/Many Workers

Page 10: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

ELF Software Architecture

Page 11: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

CBT-based agent `pre-reduces’ local key-value pairs

SRT globally `shuffles’ and `reduces’ key-value pairs

ELF APIs for programmers to implement diverse jobs

ELF Design

Page 12: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Sample ELF Query

Page 13: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

in-memory data structure

<(a,1)(b,1)(a,1)(b,1)(b,1)>

<(a,2)(b,3)>

(1) ̀ insert ‘to fill

(2) ̀ flush’ to fetch the pre-reduced result

(3)‘empty’ to empty the states

Compressed Buffer Tree (CBT)

Page 14: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

CBT-based agent `pre-reduces’ local key-value pairs

Stateful, asynchronous, and synchronous execution

SRT globally `shuffles’ and `reduces’ key-value pairs

ELF APIs for programmers to implement diverse jobs

ELF Design

Page 15: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

CBT-based agent `pre-reduces’ local key-value pairs

Stateful, asynchronous, and synchronous execution

SRT globally `shuffles’ and `reduces’ key-value pairs

ELF APIs for programmers to implement diverse jobs

ELF Design

Page 16: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

(a,2) (b,3)

(a,5)

(b,2) (c,2)

(c,1) (d,7)

(a,7) (b,3)

(d,7) (c,3) (b,2)

(a,7) (d,7) (b,5) (c,3)

hash(`micro-promotion’) =d462ba

(a,<b>) (b,<a>)

(a,<c>)

(c,<f>) (d,<e>)

(c,1) (d,7)

(a,<b,c>) (d,<e,g,h>)

(a,<b,c>) (b,<a>) (c,<f>) (d,<e,g,h>)

hash(`product-bundling’) =2080c3

(b,<a>) (c,<f>)

Per-job dataflow

webserver

Page 17: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

hash(`micro-promotion’) =d462ba

hash(`product-bundling’) =2080c3

Synchronizing CBT to be emptied Last iteration’s result New job function or parameter

Per-job dataflow

Page 18: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

hash(`micro-promotion’) =d462ba

hash(`product-bundling’) =2080c3

Limited to O(Log(N)) hops

hash(`Sale-predication’) =43bc3c

Cross-job Coordination

Page 19: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

CBT-based agent `pre-reduces’ local key-value pairs

Stateful, asynchronous, and synchronous execution

SRT globally `shuffles’ and `reduces’ key-value pairs

Per-job dataflow, and cross-job coordination

ELF APIs for programmers to implement diverse jobs

ELF Design

Page 20: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

CBT-based agent `pre-reduces’ local key-value pairs

Stateful, asynchronous, and synchronous execution

SRT globally `shuffles’ and `reduces’ key-value pairs

Per-job dataflow, and cross-job coordination

ELF APIs for programmers to implement diverse jobs

ELF Design

Page 21: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Using ELF

Page 22: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Outline

Motivation

ELF design

Evaluation

Page 23: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Latency Replay Twitter’s stream 1280 agents 50 events/second each

60 × 12-core 2.66 GHz AMD Opteron 48 GB RAM per server Gigabit Ethernet

ü  ELF outperforms for large windows due to the efficient CBT flush

Page 24: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

New Functionality 10s millisecond for multicast messages 100s millisecond to update functions

Page 25: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Scalability 60 × 12-core 2.66 GHz AMD Opteron 48 GB RAM per server Gigabit Ethernet 1000 agents

ü  ELF scales well up to thousands of concurrent jobs

Page 26: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Conclusions and Future Work ELF introduces a `many masters/many workers’ framework: • To achieve good performance: low latency and

high throughput •  scalability: for diverse, concurrent jobs, and •  new functionality: job flexibility feedback and

coordination. Future work: understand the limitations of Elf; Explore more complex web-tier processing; look at task isolation.

Page 27: Efficient Lightweight Fast Stream Processing at Scale*...Efficient Lightweight Fast Stream Processing at Scale* ... ELF runs directly on the web server tier . Master 5 hash(‘app

Thank you!

Liting Hu [email protected]