Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 1

EventWave: Programming Model and Runtime Support

forTightly-Coupled Elastic Cloud Applications

Wei-Chiu Chuang, Bo Sang, Sunghwan Yoo, Rui Gu, Charles Killian, Milind Kulkarni

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 2

Motivationclients

server

world

building building

room

room

room

room

TimeResponse time

# clients

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 3

MotivationScale up

TimeResponsetime

clients

server

# clients

Elasticity is hard

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 4

Objectives

A programming model which supports:

Stateful computation

Simple sequential semanticsTransparent elasticity

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 5

Related Work

Data Flow stateless

Live MigrationLive Migration of Virtual Machines

[Clark et. al. NSDI ‘05]

Zephyr [Elmore et. al. SIGMOD ‘11]

Does not change scale:“split”/”merge” state

Scalable programming model Orleans [Bykov et. al. SoCC ‘11]

Transactional, reconcile conflicts

MapReduce [Dean et. al. OSDI ‘04]

Dryad [Isard et. al. EuroSys ‘07]

CIEL [Murray et. al. NSDI ‘11]

No Stateful Computation

No transparent elasticity

No simple sequential semantics

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 7

Event Driven Systems

123

Event 1 commitsEvent 2

commitsEvent 3

commits

Typical event driven systems are not scalable.

Event queueclient

server

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 8

Context

world

building building

room

room

room

room

Scalability comes from parallelism Partition program state into `contexts` An event accesses one or more contexts Events accessing disjoint contexts can run in parallel

world

building

room

hallwayContexts enable implicit parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 9

EventWave

Event 1 commits

Event 2 commits

Event 3 commits

Event 2 finishes

Event 1 finishes

Event 3 finishes

Context 1

Context 2

Context 3

123

Enforce sequential orderingEvent 2 can not commit until Event

1 commits

Stateful

Sequential semantics

Parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Access Multiple Contexts

A player can move from one room to another Remove it from source room Insert it into destination room

10

world

building building

room

room

room

room

Room 1 Room 2

Alice BobBob

Player list

An event may access multiple contexts

Bob

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13

Access Multiple Contexts

Must ensure Sequential semantics parallelism

Event 1 finishes

Event 2 commits12

10

To be scalable, events can not access contexts arbitrarily

Event 2 can’t start before event 1 finishes

Context 1

Context 2

Context 3

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 12

Hierarchical Contexts

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Contexts are not completely independent The world has many buildings

A building has many rooms

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 13

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Must access contexts from top to bottom

The hierarchical access enables parallelism

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 14

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Move a player from room 1 to room 2

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 15

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Allow the next event to access Building<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 16

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Enter Room<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 17

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Remove playerRelease

exclusive access

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 18

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Enter Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 19

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Insert player

Event finishes, releasing all contexts

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 20

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 21

Wave of Events

world

Building<1> Building<2>

Room<1> Room<2> Room<1> Room<2>

Event commits, releasing snapshot

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 22

Scale more by executing events across multiple nodes Map contexts Head node

Distributed Execution

Head nodeworld

Building<1> Building<2>

Room<1>

Room<2>

Room<3>

Room<4>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 23

Distributed Execution

Client Logical Node #1

Client Logical Node #2

Logical Node: a set of physical nodesServer Logical Node

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 24

Distributed Execution

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 25

Elasticity

world

Building<1> Building<2>

Room<1>

Room<2>

Room<1>

Room<2>

Request nodes from cloud scheduler

Update context mapping

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 26

Elasticity

world

Building<1> Building<2>

Room<1>

Room<2>

Room<1>

Room<2>

Transfer contexts to the new node

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 28

Evaluation

Does it scale?Microbechmarks

ScalabilityWhat is the cost of migration? Microbechmarks

Migration latencyCase studyMulti-player game server

In the paperKey-value store

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 29

Microbenchmark-Scalability

SetupOne logical node, fixed context mappingEC2 Small Instances

1 vCPU, 1.7GB RAM, 160 GB local diskDistribute 160 contexts to physical nodesMeasuresThroughput

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 30

Microbenchmark-ScalabilityTakeaway:

Throughput grows w.r.t. # of nodes

P: workload

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 31

Microbenchmark-Migration Latency

Setup2 x 8-core 2.0 GHz Xeon, 8GB RAM1Gb Ethernet connectionScale does not change

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 32

Microbenchmark-Migration Latency

MeasureThroughput of events

The migration event commits

Finished events must wait for migration event

Migrate a 100MB context

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 33

Multi-player Game Server

SetupServer logical node

1 x Extra Large Instance (head) 64 x Small Instances

Client logical nodes 128 clients on 16 EC2 Small Instances

MeasureLatency

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 34

Multi-player Game Server

Server contexts spread to 64 physical nodes

Synthetic workload

Server contexts merge to 1 physical nodes

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 35

Conclusion

Elasticity is crucial for cloud applications.Our programming model enables transparent

elasticity for tightly-coupled applicationsCase studies show EventWave is efficient

http://www.macesystems.org

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 36

Backups

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 37

Language Construct

state_variables{ Hallway hw; vector<Room> rooms; }

context Hallway{ int x;}context Room<int>{ int y;}

Declare implicit

parallelism

Mace [Killian et. al. PLDI ‘07]

HallwayRoom[0]Room[1]…

Hallway

Room<0>

Room<1>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 38

Event Handler

upcall deliver(Message m){ }

upcall [Room<m.roomID>] deliver(Message m){ }

AnnotationSpecify what context to access

Message(roomID = 2)

Context Room<2>

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 39

Key-value store

Setup2 x 8-core 2.0 GHz Xeon, 8GB RAM1Gb Ethernet connection MeasureLatency

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 40

Key-value store

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 41

Microbenchmark-Migration Latency

Setup2 x 8-core 2.0 GHz Xeon, 8GB RAM1Gb Ethernet connectionScale does not change

Context

Chuang, Sang, Yoo, Gu, Killian and Kulkarni, “EventWave” SoCC ‘13 42

Context Migration

Head

Old node

New node

Copy context state

Replicate context state

Event 1 goes to the old node

Event 3 goes to the new node

1 M 3

Update context-node mapping