Managing Server Energy and Managing Server Energy and Operational CostsOperational Costs

Chen, Das, Qin, Sivasubramaniam, Chen, Das, Qin, Sivasubramaniam, Wang, GautamWang, Gautam(Penn State)(Penn State)

Sigmetrics 2005Sigmetrics 2005

IntroductionIntroduction

Usual argumentsUsual arguments– Variable workloadsVariable workloads– High operational cost even at low workloadsHigh operational cost even at low workloads– Power consumption a serious concernPower consumption a serious concern

WORK NOT IN CONTEXT OF WORK NOT IN CONTEXT OF VIRTUALIZATIONVIRTUALIZATION

……IntroductionIntroduction

New points:New points:– Server on/off and DVS has been used for Server on/off and DVS has been used for

power management BUT no consideration for power management BUT no consideration for impact of on/off on reliability of hardwareimpact of on/off on reliability of hardware

– Authors claim no previous paper considered Authors claim no previous paper considered SLA violations – only energy optimization SLA violations – only energy optimization (maybe true in 2005!)(maybe true in 2005!)

……IntroductionIntroduction

Three new approaches for server Three new approaches for server provisioning with both DVS and server provisioning with both DVS and server on/off, with response time SLAs considered, on/off, with response time SLAs considered, and with cost of server on/off:and with cost of server on/off:– First queuing model basedFirst queuing model based– Second feedback control theory basedSecond feedback control theory based– Third hybrid of above twoThird hybrid of above two

System modelSystem model

Cluster with identical servers (hosts)Cluster with identical servers (hosts) Any host can run any applicationAny host can run any application Applications run on multiple hostApplications run on multiple host Assuming web front end each HTTP request Assuming web front end each HTTP request

is routed to one of the web serversis routed to one of the web servers– Service time is related to the request parameter Service time is related to the request parameter

(e.g. file size)(e.g. file size) We will control the number of servers We will control the number of servers

allocated to a given applicationallocated to a given application

……System modelSystem model

SLA used: average response timeSLA used: average response time Goal: uses minimum energy required to Goal: uses minimum energy required to

meet SLAmeet SLA Two options for power managementTwo options for power management

– Server on/off: costs in terms of (1) time and Server on/off: costs in terms of (1) time and energy required to boot up (2) wear and tear of energy required to boot up (2) wear and tear of components (esp disk)components (esp disk)

– Dynamic voltage-frequency scaling of CPUDynamic voltage-frequency scaling of CPU

……System modelSystem model

Assume Assume ll frequency levels: frequency levels: ff11, f, f22,…f,…fll Assume “centralized” DVS control:Assume “centralized” DVS control:

– Single DVS setting for all hosts running one Single DVS setting for all hosts running one applicationapplication

Assumption: Only one application on one Assumption: Only one application on one serverserver

Solution required for two steps: (1) how Solution required for two steps: (1) how many servers per app (2) Freq setting for many servers per app (2) Freq setting for servers of each appservers of each app

……Problem FormulationProblem Formulation

MM identical servers identical servers NN different applications different applications Each appl Each appl i i is allocated is allocated mmii servers at any servers at any

time time Max/min frequency for each server:Max/min frequency for each server:

– ffmaxmax, f, fminmin

……Problem Formulation Problem Formulation (Power Model)(Power Model)

Dynamic power consumption of CPU Dynamic power consumption of CPU operating at frequency operating at frequency ff is proportional to is proportional to VV22*f*f

Further, V is proportional to fFurther, V is proportional to f Thus, model of CPU operating at frequency Thus, model of CPU operating at frequency

f is:f is:– PPfixedfixed + P + Pff*f*f33

Energy consumption of the cluster at Energy consumption of the cluster at frequency f over time t:frequency f over time t:

……Problem FormulationProblem Formulation

MM and and ff are controlled periodically with are controlled periodically with interval interval tt

Over Over ZZ such intervals of time such intervals of time tt, energy , energy consumption is:consumption is:

Where Where mmii(z)(z) is the number of servers is the number of servers

allocated to application allocated to application ii during duration during duration zz

……Problem FormulationProblem Formulation

KK$$ is the per unit electricity cost is the per unit electricity cost

Total cost:Total cost: BB00 is the cost per server turn-on cycle. Then is the cost per server turn-on cycle. Then

total cost:total cost:

……Problem formulationProblem formulation

BB00 itself has two components: power itself has two components: power

required to turn on, and “MTBF” impact cost required to turn on, and “MTBF” impact cost due to the turn ondue to the turn on

PPmaxmax is power consumed at max frequency, is power consumed at max frequency,

TTrebootreboot is time taken for rebooting, is time taken for rebooting, CCrr is MTBF is MTBF

costcost (MTBF = Mean Time Between Failures)(MTBF = Mean Time Between Failures)

……Problem FormulationProblem Formulation

Objective Function to Minimize, under Objective Function to Minimize, under constraintsconstraints

……ConstraintsConstraints

Solutions.Solutions.First: Queuing TheoreticFirst: Queuing Theoretic

Assumptions:Assumptions: Number of servers is managed every time Number of servers is managed every time TT Frequency level per application controlled Frequency level per application controlled

every time every time tt TT is an integer multiple of is an integer multiple of t, T= S*tt, T= S*t Optimization period: Optimization period: UU intervals of time intervals of time TT

– Each with Each with SS intervals of time intervals of time tt

PredictionPrediction

Need the following parameters for each Need the following parameters for each application:application:– Mean arrival rate (Mean arrival rate ())– Squared coeff of interarrival times CSquared coeff of interarrival times Caa

22

– Squared coeff of service times CSquared coeff of service times Css22

– Mean file size in bytes: Mean file size in bytes: S-ARMA model used for prediction of arrival S-ARMA model used for prediction of arrival

parameters (seasonal autoregressive moving parameters (seasonal autoregressive moving average)average)

““Winter’s” smoothing method for file size Winter’s” smoothing method for file size parametersparameters

Queueing AnalysisQueueing Analysis

Model the application on each server as a Model the application on each server as a G/G/1 queue. Use Bolch approximationG/G/1 queue. Use Bolch approximation

Optimization ProblemOptimization Problem

Heuristic Solution Heuristic Solution

First assumeFirst assume t=T t=T and consider only one interval at and consider only one interval at a time. Then objective is to minimize a time. Then objective is to minimize

Start solution for interval u by finding for all i, the Start solution for interval u by finding for all i, the

minumum number of servers minumum number of servers mmii(u)(u) required for the required for the constraint constraint WWii <= <= WWi,SLAi,SLA using highest frequencies using highest frequencies

Do this for each intervalDo this for each interval

……Heuristic SolutionHeuristic Solution

Then try to reduce Then try to reduce ffii(u)(u) and increase and increase mmii(u)(u), if , if

that reduces cost.that reduces cost. Select applications in decreasing order of Select applications in decreasing order of

ffii(u)(u)

– i.e. select app with highest frequency, first.i.e. select app with highest frequency, first.

……Heuristic SolutionHeuristic Solution

Now consider all intervals (still assume Now consider all intervals (still assume t=Tt=T)) Start with “upper bound” solution of previous Start with “upper bound” solution of previous

round (upper bound because intervals were round (upper bound because intervals were considered separately)considered separately)

……Heuristic solutionHeuristic solution

Again consider one interval at a time (from Again consider one interval at a time (from 11stst to last), but the total obj function to last), but the total obj function

In each interval, start from apps with highest In each interval, start from apps with highest frequency and go in decreasing orderfrequency and go in decreasing order

……Heuristic solutionHeuristic solution

For each app, compare number of servers in this For each app, compare number of servers in this interval with previous, and “level off” the number of interval with previous, and “level off” the number of servers, if soln improvesservers, if soln improves– Search greedily where number of servers are closer to Search greedily where number of servers are closer to

number in previous intvlnumber in previous intvl– Then, tune frequencies to get a feasible solution Then, tune frequencies to get a feasible solution

……Heuristic solutionHeuristic solution

So far, t =T. So far, t =T. For cases where T > t, use average arrival and For cases where T > t, use average arrival and

files size estimates for interval T and use above files size estimates for interval T and use above solutionsolution

Then given the Then given the mmii(u)(u) from this step, tune from this step, tune ffii(u,s)(u,s) for for each small interval each small interval ss

Control Theoretic ApproachControl Theoretic Approach

Set up a feedback control formulation that finds an Set up a feedback control formulation that finds an “aggregate frequency” for each app “aggregate frequency” for each app ii – Objective: to meet response time SLAObjective: to meet response time SLA

Solve the problem of allocating optimal number of Solve the problem of allocating optimal number of servers which achieve this aggregate frequencyservers which achieve this aggregate frequency– Objective: minimize power costObjective: minimize power cost

……Control theoretic formulationControl theoretic formulation

FFii(u,s) =m(u,s) =mii(u)* f(u)* fii(u,s)(u,s): aggregate frequency: aggregate frequency

– Indexed as Indexed as FFii(k)(k), where k = 1,2,..U*S, where k = 1,2,..U*S

Implicit assumption: response time from Implicit assumption: response time from multiple servers with total capacity F is the multiple servers with total capacity F is the same as that from single server with capacity Fsame as that from single server with capacity F

Objective function:Objective function:

RF and RW,i are relative weights

LQR methodLQR method

Modify decision variables to formulate problem as Modify decision variables to formulate problem as well-known Linear Quadratic Regulator:well-known Linear Quadratic Regulator:

……Control theoretic approachControl theoretic approach

is the “control gain” calculated by is the “control gain” calculated by using standard LQR methodsusing standard LQR methods

After above, use Integral Controller After above, use Integral Controller

Server AllocationServer Allocation

Given Given FFii(u,s)(u,s) we need to determine we need to determine mmii(u)(u)

Define Define m(u)m(u) = =– Total number of servers required for all appsTotal number of servers required for all apps

Define Define F(u)F(u) = =– Total frequency required for all appsTotal frequency required for all apps

Algorithm allocates Algorithm allocates mmii(u)(u) in proportion to its in proportion to its

aggregate frequencyaggregate frequency

……server allocationserver allocation

•Server frequency min,max is :•Therefore, number of servers is bounded between:

where

and

Online server allocation algorithmOnline server allocation algorithm

•Start with Equation 11, ignore the server on/off costs•The remaining expression quantifies the tradeoff between the cost of a new server vs cost of higher frequency•Differentiate this w.r.t. m(u) and find minima– get number of servers that minimize the cost

… … Online server allocation algorithmOnline server allocation algorithm

This m*(u) will be a minimum – adding even This m*(u) will be a minimum – adding even one more server will increase the cost, one more server will increase the cost, without even considering the cost of server without even considering the cost of server on (if any)on (if any)

Now consider two cases:Now consider two cases:– m(u-1) is greater than m*(u) or m(u-1) is greater than m*(u) or – Else turn on additional servers with the following Else turn on additional servers with the following

algo…algo…

Online server allocation algoOnline server allocation algo

……online server allocationonline server allocation

Where D denotes rounding of to available discrete frequency levels