Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase...
-
date post
22-Dec-2015 -
Category
Documents
-
view
216 -
download
2
Transcript of Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase...
![Page 1: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/1.jpg)
Balancing Risk and Reward in a Balancing Risk and Reward in a Market-based Task ServiceMarket-based Task Service
David Irwin, Laura Grit,
Jeff Chase
Department of Computer ScienceDuke University
![Page 2: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/2.jpg)
Resource Management in the LargeResource Management in the Large
Grids enable resource sharing• Each user has ability to use more resources
• Requires global coordination of resource sharing
Current technology: private grids (Virtual Organizations)
Next generation: public Grid• Larger scale of resources and participants
• Dynamic collection suppliers and consumers
• Varying supply and demand
Market-based approaches are attractive• Decentralized resource management
• Independent actors acting on self-interest produce desired global outcomes
e.g. Spawn, Mariposa, G-commerce framework, Nimrod-G
• Increasingly important as we move to larger grids
![Page 3: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/3.jpg)
Example: Market-Based Task ServiceExample: Market-Based Task Service
Tasks are batch computation jobs• Self-contained units of work
• Execute anywhere
• Consume known resources
Characteristics of a market-based task service• Tasks deliver value when they complete
• Negotiation between customers and task service sitesValue (price) and quality of service (completion time)
• Form contracts for task executionBreach of contract implies a penalty
• Consumers look for the best deal; sites maximize their profits
![Page 4: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/4.jpg)
Customer
Task Service Sites
Bid (value, service demand)
Accept (completion time, price)
Accept (contract)
Bid (value, service demand)Reject
Bid (value, service demand)
Reject
Accept (completion time, price)
Market FrameworkMarket Framework
![Page 5: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/5.jpg)
Goals and Non-goalsGoals and Non-goals
Goals
• Define profit-maximizing heuristics for acceptance (admission control) and scheduling for task service sites
Which tasks should a site accept? When? For how much?
• Financial metaphor: balance risk and reward subject to user bids that trade off price and quality of service
Non-goals
• Other pieces for a fully functioning economyHow is currency supplied and replenished?
How to make payments and enforce contracts?
How to propagate price signals to buyers?
What incentive mechanisms will induce truthful user bids?
![Page 6: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/6.jpg)
OutlineOutline
Overview• Motivation and Goals/Non-goals
• Task Services
Background• Specifying user bids
• Problem Statement
Heuristics
• Methodology
• Present Value and Opportunity Cost
• Negotiation and Admission Control
Conclusions
![Page 7: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/7.jpg)
Specifying User Bids and ContractsSpecifying User Bids and Contracts
Negotiation establishes agreement on price and service quality
Use value functions giving an explicit mapping of service quality to value
• Need a representation that is simple, rich, and tractable
• Millennium: linearly decaying value functions [Chun02]
Delayed tasks decay at constant rate of decayi (urgency)
Extend functions to include penalties
• May specify an optional bound on a penalty
Penalties expire at time expirei
![Page 8: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/8.jpg)
Time
Val
ue
RuntimeMaximum Value
Decay at constant rate decayi
Penalty
Example Value FunctionExample Value Function
![Page 9: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/9.jpg)
Problem StatementProblem Statement
Based on user bids we must decide…• Admission control: which tasks to commit to?
• Scheduling: when to run a task?
Schedule accepted tasks to maximize value
• How much to charge for tasks?
Price to service quality tradeoff specified in value functions
Problem extends classical value-based scheduling problems• Total Weighted Tardiness and Total Weighted Completion Time
NP-hard for off-line instances; problem is difficult
• Examine on-line instances of the problem: need heuristics
• Site can negotiate for higher value or reject some tasks
![Page 10: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/10.jpg)
Server Scheduling HeuristicsServer Scheduling Heuristics
Discounting future gains
• Bias schedule for shorter tasks
• Realizing gains quickly may be more important than value
Accounting for opportunity cost
• Bias towards high urgency tasks
• Account for losses in other tasks from a scheduling decision
Admission control
• High valued tasks earn value for the system
• High urgency tasks constrain the future task mix
![Page 11: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/11.jpg)
Experimental MethodologyExperimental Methodology
Develop heuristics to maximize value and opportunity cost• Heuristics have multiple components
Evaluate in on-line open market setting• Schedule varying task mixes over emulated batch task engine
• Evaluate components in isolation and combination
• Compare with Millennium FirstPrice policy
Generated workloads to drive task sites
• Explore different areas of the parameter space
Focus on relative value and sensitivity analysis
![Page 12: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/12.jpg)
Workload ConsiderationsWorkload Considerations
Workload characteristics • Arrival and cost distributions representative of real batch workloads as
characterized by previous studies
Exponential inter-arrival times and durations [Downey99]
• Previous studies give little guidance on how users value their jobs
Distribution of value and urgency similar to Millennium study [Chun02]
Adapt bimodal distributions for value and urgency
Characterize by skew ratios: ratio of high/low means
Magnitude of results dependent on workload characteristics• Results are conservative: look at stable markets
![Page 13: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/13.jpg)
Discounting Future GainsDiscounting Future Gains
Account for risk of deferring future gains• Example: Given two tasks with same unit gain and urgency it is
preferable to run shorter task firstShorter tasks carry lower risk of preempting newly arriving tasks
• Risk-averse scheduler may choose to run lower-yield task if it can realize gains quickly
Approach based on notion of present value common in finance• PVi = yieldi / (1 + (discount_rate * RPTi))
• PVi represents investment value
Earning simple interest at discount_rate for RPTi
• Higher discount_rate results in more risk-averse system
Present value heuristic (PV) selects jobs in order of discounted unit gain• PVi/RPTi
![Page 14: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/14.jpg)
Improvement vs. Discount RateImprovement vs. Discount Rate
-1
0
1
2
3
4
5
6
7
8
9
0.001 0.01 0.1 1 10
Impr
ovem
ent o
ver F
irstP
rice
(%)
Discount Rate (%)
Value Skew Ratio=9Value Skew Ratio=4
Value Skew Ratio=2.15Value Skew Ratio=1.5
Value Skew Ratio=1FirstPrice
![Page 15: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/15.jpg)
Opportunity CostOpportunity Cost
Extend heuristic to consider opportunity cost• Losses occurring from choosing task i instead of task j, causing task j
to decay in value
• Opportunity cost depends only on the urgency of competing tasks
Opportunity Cost:
• Bounded penalties requires O(n2) to compute least cost task
• We can simplify with unbounded penalties
Takes O(log n) to compute least cost task
• Equivalent to Shortest Weighted Processing Time First (SWPT)
),(*;0
j
ijj
ijj expireRPTMINdecaycost
![Page 16: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/16.jpg)
Balancing Gains and Opportunity CostBalancing Gains and Opportunity Cost
Risky to defer gains on basis of opportunity cost alone• FirstReward metric combines task gains with opportunity
cost
rewardi = ((α)*PVi – (1-α)*costi)/RPTi
• α controls degree to which system considers expected gains
With α=1 and discount_rate = 0 rewardi reduces to
FirstPrice
With α=0 rewardi reduces to a variant of SWPT
![Page 17: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/17.jpg)
Bounded: Improvement vs. Risk/Reward WeightBounded: Improvement vs. Risk/Reward Weight
2
3
4
5
6
7
8
9
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Impr
ovem
ent o
ver F
irstP
rice
(%)
Risk versus Reward weight (Alpha)
Decay Skew Ratio=5Decay Skew Ratio=7Decay Skew Ratio=3
It is useful to consider value: high α biases against low-valued jobs, which tend to reach their bounds faster
![Page 18: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/18.jpg)
Unbounded: Improvement vs. Risk/Reward WeightUnbounded: Improvement vs. Risk/Reward Weight
0
10
20
30
40
50
60
70
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Impr
ovem
ent o
ver F
irstP
rice
(%)
Risk versus Reward Weight (Alpha)
Decay Skew Ratio=7Decay Skew Ratio=5Decay Skew Ratio=3
Little benefit in considering gains with unbounded penalties
![Page 19: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/19.jpg)
Negotiation and Admission ControlNegotiation and Admission Control
Each site may accept or reject a task• Accepted tasks negotiate to establish a price and expected completion
time
Admission control procedure• Integrate task into current schedule according to heuristic
• Determine expected yield for task if completed
• Apply acceptance heuristic to determine acceptance
• If task is profitable then accept the bid and issue a server bid to client
• If client accepts the contract then execute the task
Later arrivals could delay task beyond its expected completion time
![Page 20: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/20.jpg)
Admission Control HeuristicAdmission Control Heuristic
Acceptance Heuristic
• Consider potential reward and constraining future task mix
• Urgent tasks incur more risk
Heuristic based on task’s slack
• Slack is the amount of additional delay that the task can incur before its reward falls below some yield threshold
Slacki = (PVi – costi)/decayi
• Policy rejects tasks whose slack falls below some slack threshold
Slack captures the risk of accepting tasks as determined by its decay rate and position in the schedule
![Page 21: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/21.jpg)
Improvement vs. Admission ThresholdImprovement vs. Admission Threshold
50
100
150
200
250
300
350
400
450
500
550
-200 -100 0 100 200 300 400 500 600 700
Impr
ovem
ent o
ver N
o A
dmis
sion
Con
trol
(%)
Admission Control Threshold
Load=2Load=1.33Load=0.89Load=0.67Load=0.50
![Page 22: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/22.jpg)
ConclusionsConclusions
Develop heuristics for market based task scheduling and admission control
Bids capture both user value and urgency
• Approach based on a financial metaphor
• Cost and risk often more important than gains
Heuristics that consider gains are effective in some cases
Contributions• Detail different areas of scheduling risk
• Explore parameter space for a general scheduling heuristic
• Show how value-based schedulers can drive server bidding and admission control in a computational economy
![Page 23: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/23.jpg)
QuestionsQuestions
![Page 24: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/24.jpg)
Simplifying System AssumptionsSimplifying System Assumptions
System setting
• Homogeneous processors
• Preemption enabled; suspended tasks resumed on any processor
• System never schedules a job with less than its full resource request
• Predicted service demand is accurate
• No interference due to network, memory, or storage contention
![Page 25: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/25.jpg)
Future WorkFuture Work
Further study of market dynamics
Task services participate in a service market
• User utility functions may be used in an underlying resource market
How do services operate in a commodities market?
• Services must buy and sell resources
![Page 26: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/26.jpg)
Yield Rate vs. Load FactorYield Rate vs. Load Factor
0
100
200
300
400
500
600
0.5 1 1.5 2 2.5 3 3.5 4 4.5
Ave
rage
Yie
ld R
ate
Load Factor
FirstReward, Alpha=0FirstReward, Alpha=0.2FirstReward, Alpha=0.4FirstReward, Alpha=0.6FirstReward, Alpha=0.8
FirstReward, Alpha=1FirstPrice w/o Admission Control
![Page 27: Balancing Risk and Reward in a Market-based Task Service David Irwin, Laura Grit, Jeff Chase Department of Computer Science Duke University.](https://reader035.fdocuments.in/reader035/viewer/2022062516/56649d785503460f94a5a327/html5/thumbnails/27.jpg)
Service MarketsService Markets
Consider task service as part of a Service Market
• Sites sell a service
Service quality and/or price varies according to system conditions
Sites distributed across the Grid
• Service abstracts the physical resources Client bids based on meaningful performance
measures (i.e. response time)
• Clients negotiate contracts that incorporate measures of service quality and assurance as well as price
Clients pay more for better service
Services may incur a penalty if contracts are not honored