Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and...
-
Upload
debra-anthony -
Category
Documents
-
view
217 -
download
0
Transcript of Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and...
![Page 1: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/1.jpg)
Computer Science
Cataclysm: Policing Extreme Overloads in Internet
Applications
Bhuvan Urgaonkar and Prashant Shenoy
University of Massachusetts
![Page 2: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/2.jpg)
Computer Science
Motivation
Internet applications used in a variety of domains Online banking, online brokerage,
online music store, e-commerce
Internet usage continues to grow rapidly Broadband deployment is
accelerating
Outages of Internet applications more common
“Site not responding”“connection timed out”
![Page 3: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/3.jpg)
Computer Science
Internet Application Outages
Down for 30 minutes
Average download time ~ 260 sec
Periodic outages over 4 days
Cause: Too many users leading to overload
Holiday Shopping Season 2000:
9/11: site inaccessible for brief periods
![Page 4: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/4.jpg)
Computer Science
Internet Data Centers Internet applications run
on data centers Server farms
Provide computational and storage resources
Applications share data center resources
Problem: How can the platform handle extreme overloads seen by applications?
![Page 5: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/5.jpg)
Computer Science
Handling Extreme Overloads
Existing work is based on three approaches Request policing [Kanodia00, Li00, Verma03, Welsh03, …] Dynamic capacity provisioning [Chase01, Ranjan04] Degrade performance of admitted requests [Abdelzaher99]
Shortcomings of existing work: Does not attempt to integrate these three approaches Does not address scalability of the policer!
• The policer itself may become the bottleneck during overloads
![Page 6: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/6.jpg)
Computer Science
Our Contribution: Cataclysm
Comprehensive approach Novel policer that can scale during overloads Dynamic provisioning for both application and policer SLA-based performance adaptation
Implementation and evaluation on a Linux cluster
Focus of this talk: design of the policer
![Page 7: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/7.jpg)
Computer Science
Talk Outline
Motivation Internet data center model Request policing Cataclysm Server Platform Experimental results Summary
![Page 8: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/8.jpg)
Computer Science
Data Center Model
Dedicated hosting: each application runs on a subset of servers in the data center Subsets are mutually exclusive: no server sharing Data center hosts multiple applications
Free server pool: unused servers
Retail Web site streaming
![Page 9: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/9.jpg)
Computer Science
Internet Application Model
Internet applications replicated on multiple servers E.g., clustered HTTP
Each application employs a sentry Load balancing and request policing
One or more request classes Service-level agreement
Specifies certain guaranteed request admission rate per class Specifies allowed degradation in response time with arrival rate
requests
http
load balancing sentry
droppedrequests
![Page 10: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/10.jpg)
Computer Science
Talk Outline
Motivation Internet data center model Request policing Cataclysm Server Platform Experimental results Summary
![Page 11: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/11.jpg)
Computer Science
Policer: Design Goals
Class-based differentiation Each class should sustain its guaranteed admission rate
Revenue maximization Challenging due to online nature of the problem
• An admitted request may cause a more important request arriving later to be dropped
Approach: Preferential admission to higher class requests
Scalability The policer should remain operational even under extremely
high arrival rates
![Page 12: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/12.jpg)
Computer Science
Overview of Policer Design
Cataclysm policer has three components Request classifier and per-class leaky buckets Class-specific queues Admission control
Classifier
Leaky buckets
Class gold
Class silver
Class bronze
Class-specific queues
Admission control
dgold
dsilver
dbronze
dropped
admitted
![Page 13: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/13.jpg)
Computer Science
Class-based Differentiation
Classifier
Leaky buckets
Class gold
Class silver
Class bronze
Class-specific queues
dgold
dsilver
dbronze
Each incoming request undergoes classification Per-class leaky buckets used to ensure that rates
guaranteed in SLA are admitted
Admission control
dropped
admitted
![Page 14: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/14.jpg)
Computer Science
Revenue Maximization
Classifier
Leaky buckets
Class gold
Class silver
Class bronze
Class-specific queues
dgold
dsilver
dbronze
Idea: Add different delays in processing of requests of different classes More important requests processed more frequently Methodology to compute delay values in online manner
Bounds probability of a request denying admission to a more important request
Admission control
dropped
admitted
![Page 15: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/15.jpg)
Computer Science
Admission Control
Classifier
Leaky buckets
Class gold
Class silver
Class bronze
Class-specific queues
dgold
dsilver
dbronze
Admission control
Goal: Ensure that an admitted request meets its response time target Measurement-based admission control algorithm Use information about current load on servers and estimated size of
new request to make decision
dropped
admitted
![Page 16: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/16.jpg)
Computer Science
Scalability of Admission Control
Idea #1: Reduce the per-request admission control cost Admission control on every request may be expensive
Bursty arrivals during overloads => batches get formed Delays for class-based differentiation => batches get formed
Admission control test that operates on batches instead of requests
Idea #2: Sacrifice accuracy for computational overhead When batch-based processing becomes prohibitive
Threshold-based scheme• E.g., Admit all Gold requests, drop all Silver and Bronze requests
• Thresholds chosen based on observed arrival rates and service times
+ Extremely efficient
- Wrong threshold => bad response times or fewer requests admitted
![Page 17: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/17.jpg)
Computer Science
Scaling Even Further …
Protocol processing overheads will saturate sentry resources at extremely high arrival rates Indiscriminate dropping of requests will occur
• Important requests may be turned away without even undergoing the admission control test
• Loss in revenue! Sentry should still be able to process each arriving request!
Idea: Dynamic capacity provisioning for sentry Pull in an additional sentry if CPU utilization of existing
sentries exceeds a threshold (e.g., 90%) Round-robin DNS to load balance among sentries
![Page 18: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/18.jpg)
Computer Science
Talk Outline
Motivation Internet data center model Request policing Cataclysm Server Platform Experimental results Summary
![Page 19: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/19.jpg)
Computer Science
Cataclysm Server Platform
Prototype data center 20 Pentium servers Gigabit switches Linux-based platform
Sentry implemented in Layer-7 switch Linux module ktcpvs
Replicated Web server applications using Apache Dynamic content using PHP
![Page 20: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/20.jpg)
Computer Science
Class-based Differentiation
Arrival rate
0
50
100
150
200
250
0 100 200 300 400 500
Time (sec)
Arriv
al ra
te (r
eq/s
)
Gold
Silver
Bronze
Fraction admitted
0
0.2
0.4
0.6
0.8
1
0 100 200 300 400 500
Time (sec)
Fra
cti
on
ad
mit
ted
Gold
Silver
Bronze
Three classes of requests: Gold, Silver, Bronze Policer successful in providing preferential admission
to important requests
![Page 21: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/21.jpg)
Computer Science
Threshold-based: Higher Scalability
Scalability
0
20
40
60
80
100
0 5000 10000 15000 20000
Arrival rate (req/s)
CP
U u
til
(%)
Batch
Threshold
Threshold-based processing allows the policer to handle upto 4 times higher arrival rate Single sentry can handle about 19000 req/s
![Page 22: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/22.jpg)
Computer Science
Threshold-based: Loss of Accuracy
Admission rate
0
50
100
150
200
250
0 100 200 300 400 500
Time (sec)
Adm
issi
on r
ate
(req
/s)
Gold
Silver
Bronze
95th resp time
0
1000
2000
3000
4000
5000
0 100 200 300 400 500
Time (sec)
95th
res
p tim
e (m
sec)
Gold
Silver
Bronze
Higher scalability comes at a loss in accuracy of admission control
Occasional violations of response time targets
![Page 23: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/23.jpg)
Computer Science
Sentry Provisioning
CPU util
0
20
40
60
80
100
0 100 200 300 400 500 600
Time (sec)
CP
U u
til
(%)
CPU util
Arrical rate
0
10000
20000
30000
40000
50000
0 100 200 300 400 500 600
Time (sec)
Arr
ival
rat
e (r
eq/s
)
Total arrival
Arival at sentry 1
![Page 24: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/24.jpg)
Computer Science
Summary
Cataclysm: a comprehensive overload management technique consisting of Request policing Dynamic capacity provisioning SLA-based performance adaptation
Cataclysm achieves the following Class-based differentiation Revenue maximization Ability to scale to extreme overloads
More information: http://lass.cs.umass.edu
![Page 25: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/25.jpg)
Computer Science
Policing and Provisioning
Application 1
0
500
1000
1500
2000
2500
3000
0 1000 2000 3000 4000 5000
Time (sec)
95th
resp
tim
e (m
sec) 95th resp time
Target resptime
Application 1
0
500
1000
1500
2000
0 1000 2000 3000 4000 5000
Time (sec)
Rat
e (r
eq/s
)
Arrival rate
Admission rate
![Page 26: Computer Science Cataclysm: Policing Extreme Overloads in Internet Applications Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts.](https://reader030.fdocuments.in/reader030/viewer/2022032705/56649db55503460f94aa626e/html5/thumbnails/26.jpg)
Computer Science
Policing and Provisioning
Application 2
0
200
400
600
800
1000
1200
1400
0 1000 2000 3000 4000 5000
Time (sec)
Rat
e (r
eq/s
)
Arrival rate
Admission rate
Application 2
0
500
1000
1500
2000
2500
3000
0 1000 2000 3000 4000 5000
Time (sec)
95th
resp
tim
e (m
sec) 95th resp time
Target resptime