Intelligent Power Budgeting in Data Centers

Arka Bhattacharya, David Culler (UCB)Aman Kansal, Sriram Sankar, Sriram Govindan (Microsoft)Latency-aware and performance-preserving Power CappingWhat do I mean by power capping ?Restrict server power consumption to a specific power budget , through manipulation of load or scaling of processor frequency.Summary of the talk :Data Centers need power capping.Any power capping technique should beFast ; andEnsure graceful degradation of performanceRelated work has proposed power capping either through frequency scaling, or processor utilization cappingIn an open system, using any one of these knobs might lead to a cascading failureHence, to maintain a stable system one needs to Maintain desired power level through admission control,Implement a frequency scaling governor for safety

Data Center Cost AnalysisJames Hamiltons 2010 figures for a 50k server, 8 MW facility.Power Distribution and Cooling is close to ~20% of total data center budget

- James Hamilton4Why do power capping ?To under-provision UPS batteries/GeneratorsAccording to previous figure, annual cost of power distribution and cooling equipment is > $7m for about 50k serversCurrent UPS provisions are mostly based on worst-case faceplate or spec-power ratings.

1. When utility power is lost and the UPS system provides power to the systems, power capping will be invoked to reduce the total load. The reduced load will allow for smaller, lower cost batteries to provide the necessary ride-through. Once the generators come online and are providing power to the facility and the UPS system is no longer providing power to the systems, power capping will be disabled.2. In this scenario, Power capping would be used to guarantee that the server does not exceed the allocated power. This circuit protection enables protecting the electric circuits and prevents overloading.3. The quantity of servers in the datacenter is roughly determined by taking the available power and dividing it by the maximum server power. The available power is calculated by taking the raw power and subtracting losses such as those lost in transformers and distribution, and subtracting the power required for other purposes such as UPS re-charge. Budgeting for UPS re-charge power prevents overloading electrical circuits when the utility power returns after an outage that resulted in the servers running on UPS. The power allocated to re-charging the batteries is overhead that is used only a few times each year. To re-claim this power, the servers are power capped when the UPS is in a charge state. The power saved by capping the servers is used to re-charge the batteries. The UPS power budget is set to zero watts, leading to higher available power and more total servers in the datacenter.

5PDF of power consumption of a colo containing an online application

CURRENT UPS PROVISIONINGPeak power HeadroomAggressive UPS provisioningOther reasons for doing power cappingEnsure circuit protectionRe-claim UPS re-charge budgetShave off data center peak power usage (for data centers paying peak-pricing rates)Differentiating service among data center appsReact to change in power supply from utility1. When utility power is lost and the UPS system provides power to the systems, power capping will be invoked to reduce the total load. The reduced load will allow for smaller, lower cost batteries to provide the necessary ride-through. Once the generators come online and are providing power to the facility and the UPS system is no longer providing power to the systems, power capping will be disabled.2. In this scenario, Power capping would be used to guarantee that the server does not exceed the allocated power. This circuit protection enables protecting the electric circuits and prevents overloading.3. The quantity of servers in the datacenter is roughly determined by taking the available power and dividing it by the maximum server power. The available power is calculated by taking the raw power and subtracting losses such as those lost in transformers and distribution, and subtracting the power required for other purposes such as UPS re-charge. Budgeting for UPS re-charge power prevents overloading electrical circuits when the utility power returns after an outage that resulted in the servers running on UPS. The power allocated to re-charging the batteries is overhead that is used only a few times each year. To re-claim this power, the servers are power capped when the UPS is in a charge state. The power saved by capping the servers is used to re-charge the batteries. The UPS power budget is set to zero watts, leading to higher available power and more total servers in the datacenter.

7Ramp Rate of power spikesTimePower consumption95th percentilePower Spike = PowerConsumption(S2)-PowerConsumtion(S1)S1S2Ramp Rate = PowerSpike per sampling periodPower spikes in case of under-provisioned UPSSampling rate = 30secFor circuit protection : Latency Analysis of power-capping methodsPrior work in feedback based power-capping e.gPower Budgeting for virtualized data centers Lim et.al (ATC,2011) Coordinated power control and performance management for virtualized server clusters Wang et. Al (IEEE TOPDS,2010)Ship: Scalable hierarchical power control for large-scale data centers (PACT, 2009)Dynamic Voltage Scaling in multitier web servers with end-to-end delay control Horvath et.al (IEEE Trans. Comput. 2007)SHIP runs linpack11Worst-case power rise in ServersPower rise in an Intel Xeon L5520 ServerFastest observed power rise (from min to max): 100msPower rise in an Intel Xeon L5640 ServerFastest observed power rise (from min to max) : 200msSo what is the worst case eventuality that we have to be prepared for with regard to server power spikes ?These experiment conducted was max out all the cores at the same time. Just compute the square of a random floating point number.There is some effect for which power spikes as soon it is ramped up , it maybe because of some fan which is switching on at that moment,Or just the increase in the speed of the fan !12Methods to decrease server powerDVFS (Dynamic Voltage and Frequency Scaling) :Reduces the frequency and voltage the processor runs at.

Processor utilization capping :Imposes a certain number of idle cycles on the CPU , while running at the same frequency

Admissions control : Reduces the amount of network traffic that the server serves.

Time Line of eventsCentral controller gives actuation commandCommand reaches destination serverCommand received by DaemonFunction Call returnsSettings changed in hardwarePower decreasestime12345This is a time line of every place where latency creeps into the control loop.The central controller gives the actuation command, which is then received by the network stack of the destination server.The network stack gives the command packet to the power capping agent resident on the server, which then send the actuatin command to the OS , which changes the hardware setting. After the hardware setting is changed, one needs to wait for some time before the power value reduces and settles to a lower value.14Central controller gives actuation commandCommand reaches destination serverCommand received by agent Function call returnsSettings changed in hardwarePower decreasestime< 1ms< 1ms~20ms 2secFrequency scalingProcessor utilization CappingAdmissions controlOne-step actuation< 1 sec~2 sec>2 sec*2-level feedback controller with 5 iterations each> 10sec>10sec>10sec* Still to be measured accuratelyShows the delays once again in the same chart.15If UPS capacity < Peak Power of IT equipment , one needs to be implement non-feedback governor (based on DVFS/proc capping/hardware capping)Take-away 1:Why do we need Network Admissions Control ?In an open system , if a frequency scaled server is stuck with more work than it can handle ,Server latency goes up ( because of filled queues )Requests getting dropped are retried by TCP stack of clients.The entire load on the system keeps increasing = > cascading failure

In a closed system, An implicit admissions control takes place because new requests are not issued until old requests are served.Latency increases, but does not dive into cascading failure.Experiment Setup to check frequency scaling effects on closed and open systemsOpen System => a Xeon X5550 server running wikipedia benchmark on apache web server , on linux

Closed System => StockTrader Application on Windows Server2008, Xeon L5520 .

In both systems , load was generated by 3-5 external servers.Open loop :Effect of frequency scaling (1)Open Loop :Effect of frequency scaling (2)Open Loop :Effect of frequency scaling (3)Frequency scaling : demarcating stable and unstable regions for each frequencyUnstable open loop system due to frequency scalingFrequency scaling applied hereOpen system power capping

Power reduction requiredAdmission control requiredIn an open system, while doing power capping one must perform admissions control to maintain stability.Take-away 2:Capping a closed online applicationExperiment : Generate constant load for the server. Lower the processor cap gradually and observe effects on latency Relation between power, response rate and latencyAdmissions Control effectivenessIn an closed system, doing admissions control along with frequency scaling leads to almost same throughput but with better latencyTake-away 3:Admissions control Assumptions : known relation between network traffic (T) and power consumption (P).

Problem Statement : Reduce traffic of an application from current state T1 to T2 , such that power goes from current state P1 to P2.Challenges : Traffic changes every instant.A request from a user may spawn multiple flows.How to do it in an app-agnostic way ?

Admissions Control Tradeoffs at each layerDoing Admissions Control at Layer 2 :Layer 2:Pros : Simple implementation . Cons : All connections get hurt equally.

Admissions control continuedLayer 3 : Pros : cuts off entire requests, spanning across multiple flows.Easy to configure in a firewall.Does not need app-level complianceCon : Coarse admissions control due to NATs.

Admissions control continuedLayer 4 : Pro : Can do finer grained admissions control than IP.Cons : A webpage may be served over multiple flows , and different flows of the same request might get different service.

Layer 7: Pro : Has most insight into app working . Can do fine-grained admissions controlCon : data center needs app-compliance / load balancer compliance .Future workEvaluate tradeoffs between doing Network Admissions Control at different layers.Devise and implement algorithms to do admissions control at various layers.Thank you