Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety...
-
date post
15-Jan-2016 -
Category
Documents
-
view
217 -
download
0
Transcript of Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety...
![Page 1: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/1.jpg)
Tians Scheduling: Using Partial Processing in Best-Effort Applications
Yuxiong He*, Sameh Elnikety*, Hongyang Sun+
*Microsoft Research+ Nanyang Technological University
![Page 2: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/2.jpg)
Background
• Interactive services are an important part of data center workload
– Example: web search, web server, VOD server, etc.– SLA
» Example: web search can require 99% results returned within 150ms.
• Server utilization for interactive services is embarrassingly low
– Today high server utilization kills responsiveness and result quality.
![Page 3: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/3.jpg)
ExampleM/M/1 Queue• Mean service time = 15ms• Deadline = 100ms• Quality = 1 if it is fully
serviced within deadline; 0 otherwise
• Average quality ≥ 0.99• What is max system
utilization?
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.950000000000001
0.955000000000001
0.960000000000001
0.965000000000001
0.970000000000001
0.975000000000001
0.980000000000001
0.985000000000001
0.990000000000001
0.995000000000001
1
Normalized Arrival Rate
Qua
lity
FIFO Server:Norm. arrival rate = 0.3CPU utilization = 30%
![Page 4: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/4.jpg)
Example
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.950000000000001
0.955000000000001
0.960000000000001
0.965000000000001
0.970000000000001
0.975000000000001
0.980000000000001
0.985000000000001
0.990000000000001
0.995000000000001
1
Normalized Arrival Rate
Qual
ity
FIFO Server:Arrival rate = 0.3CPU utilization = 30%
?
M/M/1 Queue, FIFO Scheduling• Mean service time = 15ms• Deadline = 100ms• Quality = 1 if it is fully
serviced within deadline; 0 otherwise
• Average quality ≥ 0.99• What is max system
utilization?
![Page 5: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/5.jpg)
MotivationM/M/1 Queue, FIFO Scheduling• Mean service time = 15ms• Deadline = 100ms• Quality = 1 if it is fully serviced
within deadline; 0 otherwise• Average quality ≥ 0.99• What is max system utilization?
Goal* Sustain higher load while meeting
SLA. * Reduce hardware, energy and
operational cost.
FIFO Server:Arrival rate = 0.3CPU utilization = 30%
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10.950000000000001
0.955000000000001
0.960000000000001
0.965000000000001
0.970000000000001
0.975000000000001
0.980000000000001
0.985000000000001
0.990000000000001
0.995000000000001
1
Normalized Arrival Rate
Qual
ity
![Page 6: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/6.jpg)
Characteristics of Interactive Services• Deadline with each request• Partial results
– Find the best available result within a predefined response time
• Quality function– Response quality improves with processing time progressive– Exhibits diminishing return concave
![Page 7: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/7.jpg)
Traditional System Tians System
Strict request model
Processing time
Qua
lity
flexible model with partial results
Processing time
Qua
lity
+ Tians scheduler• Decide processing time of requests• Maximize overall response quality while meeting their deadline
Bx
Ax
A’xB’
x
![Page 8: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/8.jpg)
Example3 Jobs, deadline 100
J1 J2 J3 Quality
Service demand 90 20 100
FIFO 90 10 / 0.98 + 0 + 0 = 0.98
FIFO + partial results 90 10 / 0.98 + 0.27 + 0 = 1.25
Tians scheduling + partial results 40 20 40 0.74 + 0.47 + 0.74 = 1.91
0 10 20 30 40 50 60 70 80 90 1000
0.10.20.30.40.50.60.70.80.9
1
processing time
Quality
Benefits of partial results
Benefits of scheduling
![Page 9: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/9.jpg)
Contribution• Propose Tians scheduling for interactive services, embracing
partial results.
• Present three scheduling algorithms– Tians-Optimal (offline)
» Prove its optimality– Tians-Clairvoyant (online clairvoyant)– Tians-NonClairvoyant (online nonclairvoyant)
• Evaluate Tians scheduling algorithms using simulation– Improve response quality
![Page 10: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/10.jpg)
Outline• Introduction• Scheduling model• Optimal offline algorithm• Online algorithms• Simulation results• Concluding Remarks
![Page 11: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/11.jpg)
Scheduling Model• A set of requests• Each request has
– Arrival time– Deadline – Service demand
• Quality function – maps the processing time the request receives to a quality
value
• Scheduler – assigns request processing time– maximize total quality of all requests
![Page 12: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/12.jpg)
Optimal Offline Algorithm: Tians-Optimal
Three Intuitions1. When SOEP is feasible, SOEP is optimal.2. An optimal schedule is composed by SOEP blocks.3. Jobs in the busiest interval define a SOEP block.
![Page 13: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/13.jpg)
Intuition 1When SOEP is feasible, SOEP is optimal.
SOEP (service-oriented equi-partitioning) policy– Each request gets equal share of processing time unless it
requires less.» Small requests get what they demand» Large requests share equal processing time
Example: 3 Jobs, deadline 100ms
J1 J2 J3 Quality
Service demand 90 20 100
SOEP 40 20 40 1.91
![Page 14: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/14.jpg)
Intuition 2• SOEP is not always feasible because of deadlines• SOEP not feasible divide into sub-blocks
• Inside each sub-block, SOEP is feasible
• An optimal schedule is composed by SOEP blocks.
• How to identify the SOEP blocks?
SOEP block1 SOEP block2 SOEP block3
![Page 15: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/15.jpg)
Intuition 3Jobs in the busiest interval define a SOEP block.• Inside the block, assign processing time using SOEP
![Page 16: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/16.jpg)
Intuition 3Jobs in the busiest interval define a SOEP block.• Inside the block, assign processing time using SOEP• Remove the block and repeat the process on remaining requests.
![Page 17: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/17.jpg)
Tians-Optimal• Divide and conquer algorithm:
• THEOREM: Tians-Optimal produces an optimal schedule to maximize the total quality of requests.
Tians-Optimal([i,j]) {find the busiest block [k,h]
apply SOEP on requests in block [k,h]
Tians-Optimal ([i,k-1])Tians-Optimal ([h+1, j])
}
![Page 18: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/18.jpg)
Online Algorithms
Tians-Clairvoyant• Know service demand of released requests• Apply Tians-Optimal on the set of ready requests.
Tians-Nonclairvoyant• Do not know service demand of any requests• Apply Tians-Clairvoyant using mean service demand of requests
Key features of Tians scheduling:• Share processing time equally among requests• Prevent long requests from starving short ones• Easy to implement in real systems : FIFO, no preemption
![Page 19: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/19.jpg)
Simulation Results
• Application– Web Search Engine
» Search and rank matching documentss
• Implement five algorithms– Three Tians algorithms– FIFO : no partial results– FIFO-Partial : support partial results
![Page 20: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/20.jpg)
Web Search Engine• Setup
– Service demand exponential distribution with mean = 26ms
– Poisson arrival– Quality function– SLA
» Deadline 150ms» Average quality ≥0.99
![Page 21: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/21.jpg)
Simulation Result
FIFOArrival rate 0.15Utilization 15%
FIFOFIFO-Partial
Tians-Noncl
FIFO-PartialArrival rate 0.6Utilization 60%
Tians-Noncl.Arrival rate 0.78Utilization 78%
![Page 22: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/22.jpg)
Simulation Result
FIFOArrival rate 0.15Utilization 15%
FIFOFIFO-Partial
FIFO-PartialArrival rate 0.6Utilization 60%
Tians-Noncl.Arrival rate 0.78Utilization 78%
Gain from partial results
Tians-Noncl
![Page 23: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/23.jpg)
Simulation Result
FIFOArrival rate 0.15Utilization 15%
FIFOFIFO-Partial
Tians
FIFO-PartialArrival rate 0.6Utilization 60%
Tians-Noncl.Arrival rate 0.78Utilization 78%
Gain from better scheduling
![Page 24: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/24.jpg)
Simulation Result
FIFOArrival rate 0.15Utilization 15%
FIFOFIFO-Partial
FIFO-PartialArrival rate 0.6Utilization 60%
Tians-Noncl.Arrival rate 0.78Utilization 78%
420%Tians-Noncl• Tians sustains more than 400% load.
• Save 80% servers.
![Page 25: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/25.jpg)
Simulation Result
0.15
0.60
0.78
0.90
0.97
FIFOTians-Noncl
FIFO-Partial
![Page 26: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/26.jpg)
Simulation Result
0.15
0.60
0.78
0.90
0.97
Gain from knowing service demand
![Page 27: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/27.jpg)
Simulation Result
0.15
0.60
0.78
0.90
0.97
Gain from knowing future
![Page 28: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/28.jpg)
More Experiments• VOD Server
– Tians manages the upstream bandwidth– Tians-Clairvoyant streams to 40% more clients than FIFO.
• Variance Reduction– Partial results that produces smooth quality function– Share processing time equally among requests
![Page 29: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/29.jpg)
Conclusions
• Tians– Partial results + enhanced scheduling
• Scheduling– Share processing time equally among requests– Prevent long requests from starving short ones
• Simulation results– Improve response quality– To achieve the same QoS, Tians supports much higher
system utilization than traditional server.
![Page 30: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/30.jpg)
Future Work
• Applying Tians in large-scale systems
![Page 31: Tians Scheduling: Using Partial Processing in Best-Effort Applications Yuxiong He *, Sameh Elnikety *, Hongyang Sun + * Microsoft Research + Nanyang Technological.](https://reader035.fdocuments.in/reader035/viewer/2022062804/56649d485503460f94a230c8/html5/thumbnails/31.jpg)
Thank you!
Questions?