Thermal Aware Server Provisioning (TASP) and Workload Distribution (TAWD) for Internet Data Centers (IDCs)

Zahra Abbasi, Georgios Varsamopoulos and Sandeep Gupta

Impact Laboratory

School of Computing, Informatics and Decision Systems Engineering

Arizona State University

http://impact.asu.edu/

Funded in parts by NSF CNS grants#0834797 and #0855277, and by Intel Corp

Introduction-Motivation The magnitude of data center

energy consumption

Internet users’ growth in the world from 2000-2009: 400% [http://www.internetworldstats.com/stats.htm]

Data center energy consumption grew 20-30% annually in 2006 and 2007 [ Uptime Institute research]

Addressing energy saving for internet Data Center Thermal awareness to improve

energy consumption

2

Historical energy use Future energy use projection- current efficiency trend

Projected Electricity Use of data centers\, 2007 to 2011

Typical data center energy end use

[Source: EPA]

[Source: Department of energy]

The BlueTool project

Design and on-site assessment services

BlueWeb: On-line data and simulation services

Online services:• Measurement archive• Profile archive• Model archive• Energy Calculator

authorized userBackend data andservice access

Research:• Model development• Scheduling testing• Design methodology development• Alternative, eco-friendly cooling technologies

Researchers at ASU

Consulting services:• Energy and efficiency

assessment• Design and online

solutions• Expert advising

BlueCenter:Experimental testebd

BlueSense: on-site monitoring

http://impact.asu.edu/BlueTool/

Talk outline Why thermal awareness for data centers? Opportunities for energy saving in IDCs TASP and TAWD Modeling IDCs:

Software: two tier architecture Hardware: performance and power consumption

Heuristics for TASP and TAWD Simulation model and results Experimental validation of TASP and TAWD

4

Typical layout of a data center Rack outlet temperature Tout

Rack inlet temperature Tin

Computing Room Air conditioner (CRAC) supply temperature Tsup Tout

Tin

Tsup

Redline temperature:Tred of Tin

Fah

renh

eit

[Source: Uptime Institute research ]

Heat recirculation

Possible ways to save energy in IDCs Resizing the active server set

Dynamically changing number of active servers according to the long traffic fluctuation (couple of hours) [Chase et al. SOSP ’01] , [Chen et al. NSDI ’08]

DVFS Adapting CPU speed with respect to incoming workload during fine time slots

[Ranganathan et al. ISCA ’06]

Virtualization Consolidating applications in a few physical machines with respect to their

performance requirement [Kusic et al. CCJ ’09]

Thermal awareness Considering servers’ thermally impact on the cooling system. Servers’ thermal

impact is tightly related to the heat recirculation.

6

Why thermal awareness? PUE (Power Usage Effectiveness):

A metric to measure data center energy efficiency

Large value for PUE is indication of large value for cooling energy

Current state of PUE

7

Total Facility Power {cooling, IT equip., lighting, other}

PUE=IT equipment Power

Scenario PUE

Current Trends 1.9

Improved Operations

1.7

Best Practices 1.3

State-of-the-Art 1.2

EPA estimated value of PUE for 2011(2007 report)

Improving cooling energy by thermal aware task placement [Moore et al. ATEC’ 05], [Tang et al. T-PDS ’08]

Task placement determines IDCs’ thermal profile Servers thermally interfere with each other by

recirculated heat Heat recirculation is uneven and creates hot

spots CRAC must supply sufficient cooling to keep hot

spots under the redline temperature Thermal aware task placement can reduce heat

recirculation and hot spots and improve cooling efficiency

8

Thermal aware task placement

9

[Source: HP]

Heat removedCoP= Work required to remove the heat

Co

effi

cie

nt

of

Pe

rfo

rma

nce

Tsup: CRAC Supply Temperature

= + ×

Tin Tsup D P

inlettemperatures

supplied airtemperatures

heat distribution powervector

Cooling = Pcomputing /CoP(Tsup)=Pcomputing /CoP(Tred – maxi(DiP))

N1 AC

N3N2

d21 d31

d11

d12 d13

Tsup TAC, inTin

Recirculation

Servers thermally interfere with each other by recirculated heat

Linear model for the heat recirculation [Tang et al. T-PDS ’08]

CRAC ‘s CoP

Directly affected by task placement

Measuring thermal efficiency: LRH Thermal efficiency: least contribution on the heat recirculation

LRH: A metric of thermal efficiency of a server [Tang et al. T-PDS ’08]

10

Based on a two-layer rank calculation Rank the servers as recipients of heat

recirculation Rank the servers as contributors of heat

recirculationLRH weight of S =

Σrecipients recipient value amount of heat from S to recipient

LRH rank of Server B is worse than AB

The direction and amount of heat recirculation

A

Example: LRH ranking of servers A and B

Measuring thermal efficiency: LRH Thermal efficiency: least contribution on the heat recirculation

LRH: A metric of thermal efficiency of a server [Tang et al. T-PDS ’08]

11

Based on a two-layer rank calculation Rank the servers as recipients of heat

recirculation Rank the servers as contributors of heat

recirculationLRH weight of S =

Σrecipients recipient value amount of heat from S to recipient

Low Medium High

Server A: Low low low medium

Server B: Medium medium medium high

Server C: high medium high high

Contribution on

the heat recirculation

Incoming heat to the recipients of heat recirculation

(Low means better LRH rank)

Example: LRH ranking of servers A, B and C with respect to their heat recipients

TASP and TAWD TASP

Saving energy by choosing active server set according to

servers’ computing power efficiency(Joules/MIPS) AND thermal efficiency (e.g. LRH)

Doing TASP in long time intervals (couple of hours) called

epochs TAWD

Saving more energy by skewing workload toward thermally

efficient and computing power efficient servers in fine time slots Constraints

Maintain performance [response time] Prevent redlining of servers

12

13

Server 1 Server 2 Server 3 Server N

Load Dispatcher

……

TASP Tier (Epochs)

{λi} TAWD Tier

(Slots)

λ request/sec

On/Off Control

Traffic flow

Parameters

Control data

Server N-1

Two-Tier architecture for IDCs

0 100 200 300 400 500 600 700 800 900 1000550

600

650

700

750

800

850

900

950

1000

Num

ber

of

request

arr

ivals

every

5 s

econds

Time index (every half an hour) over one month

Peak arrival rate

Time index (every 5 second)

Nu

mb

er

of

req

ue

sts

HTTP requests over time, 1998 FIFA World Cup

Server 1 Server 3 Server N-1

λ1=0 λ2 λ3=0 λNλN-1=0

Heat recirculation contribution

Computing capabilities of machines

Computing power efficiency

∑λi= λ

TASP and TAWD: Problem statements TASP:

Input: Data center server set S with N servers, epoch lengths (T), history of arrival rate

Find: Active server set: Ŝ S , where, | Ŝ|=n ≤ N⊆ Objective: Minimizing total energy Constraint: Performance requirements [i.e. response time]

TAWD: Input: Active server set Ŝ , L fine time slots of length t (T=Lt),

history of arrival rate Find: For time slot m (1 ≤ m ≤ L) the workload distribution among

active servers : λim   s∀ i Ŝ ∈ Objective: Minimizing total energy Constraint: Performance requirements [i.e. response time]

14

TASP analytical formulation : Prerequisites

15

Observed relationship b/w CPU utilization, arrival rate and turnaround time

Model:

Energy consumption modeling Assumption: Dynamic CRAC temperature setting Etotal = Ecomputing + Ecooling (=Ecomputing/CoP(Tsup))

Performance modeling

This

0 500 1000 1500 2000 2500 3000 35000

10

20

30

40

50

60

70

80

90

100

Throughput, Request Per Second (rps)

Serv

er C

PU u

tiliza

tion

(%)

0 500 1000 1500 2000 2500 3000 3500400

500

600

700

800

900

1000

Turn

arou

nd T

ime

( s

)CPU Utilization(%)Turn-around time( sec)

λmax

uthresh

Sscu iiithreshi ,max

dual-CPU dual-core E7520- chipset “Sossaman” Xeon LV systems

Arrival rate (Requests/Sec)

TASP analytical formulation: Prerequisites Power Consumption Modeling

Linear relationship between power consumption and utilization

Workload Prediction

Request_ArrivalPeak = Request_arrivalAvg. + Request_arrivalStd. dev

16

ωIdle power

power

ω + αMaximum power

Utilization0 1

Active server set size

overestimation factor (>1)Kalman Filtering

iiicompi up

Λ peak

Formulating TASP: Optimization problem Unknown variable

How many servers are required? Which servers among all servers should be chosen as active server set?

Objective: Minimizing total energy consumption:

Constraints: Meet the capacity requirement:

x is a binary vector:

17

Defining a binary vector as the variable. Each element determines if a server should be chosen or not. x: 1 0 0 1 0 1 …..

peaki

threshi

N

ii c

ux

1

tuxuxDTCoP iiiiiiiii

i

red)(

))(((

11

max

Computing powerHeat recirculation

Nixi ...1},1,0{

Heuristic approaches for TASP MinMax Approximation Numerical approx.:SQP (Seq. Quad. Prog.) Ŝ determined by discretization real solution High time complexity (QP: O(n5))

sLRH (scaled LRH) Ranking servers based on heat recirculation and

computing performance:

Ŝ = High ranking server sufficient for peak request arrival

CP-sLRH (Computing Power efficiency and sLRH) servers are first ranked according to their

computing power efficiency (J/MIPS) and then according to sLRH

Ŝ determined similar to sLRH

18

Series1Series2

Series1

Series2

Heat recirculation

Computing power efficiency: Series1>Series2

Series1

Series2

Least recirculated heat (LRHi)

Computing efficiency (MIPS)

sLRHi=

Heuristic solution for TAWD Ranking servers based on CP-sLRH Giving the maximum affordable workload to the highest ranking servers

Skewing workloads toward thermal efficient server rather than performance oriented Load Balancing (LB)

Solution for TAWD

19

10 20 30 40 500

20

40

60

80

100

Total uAverage

=30%

uThreshold

=60%

Servers are ranked according to heat recirculation

Util

iza

tion

10 20 30 40 500

20

40

60

80

100

uAverage

=30%

uThreshold

=60%

Servers are ranked according to heat recirculation

Util

iza

tion

Utilization in LB Utilization in TAWD

Evaluation Baseline algorithms

TASP versus CPSP (Computing Power efficient Server Provisioning) TAWD versus LB

Simulation Heterogeneous DC Model ASU HPCI (PUE>1.3)

Heat recirculation using CFD 50 computing nodes (1000 cores)

Experimental validation Using carton boxed Sossaman

systems

20

ASU HPCI data center

Validating thermal awareness TASP Validation: NoSP: All servers are equally utilized (25%)

TASP(sLRH): Two thermal efficient servers are utilized 50% and the other two machines are turned off

CPSP (Thermally oblivious): Two non thermal efficient ate utilized 50% and the other two are turned off

TAWD Validation LB: All servers are equally utilized such that their

utilization fluctuate over fine time slots (30 second) TAWD: Workload is skewed toward thermal efficient

servers in fine time slots, such that the total workload in any moment equals to LB scenario

21

Simulation: Workload model

SPECweb2009 benchmark (e-commerce ) suite Synthesizing SPECweb2009 + FIFA World CUP 1998

22

HTTP requests over time, 1998 FIFA World Cup

Request arrival rate over time of SPECweb2009 epoch-level peaks are obtained from the 1998 FIFA World Cup traces

23

Simulation results: TASP versus CPSP

Energy saving with respect to CPSP for different TASP schemes over different values for

Simulation results- TASP-TAWD versus TASP-LB

24

Energy saving of TAWD with respect to LB.

Conclusion Extra energy saving by choice of servers

TASP approaches have nothing to do with QoS violations

TASP MiniMax approach yields to the maximum energy saving

CP-sLRH, the low complex heuristic approach, can be used for large sale data center

More energy saving by combining TASP with TAWD TAWD improved cooling energy by 3%

25

Future Works

BlueTool Project (Ongoing project) http://impact.asu.edu/BlueTool/wiki/index.php/Main_Page

Enhancing Thermal Modeling Considering the dynamic behavior of cooling

systems

Virtualization, internet multi-tier applications

26

THANKSQuestions?

27

References [Moore et al. ATEC ’05] J. Moore, J. Chase, P. Ranganathan, and R. Sharma, “Making

scheduling "cool": temperature-aware workload placement in data centers,” in ATEC ’05: Proceedings of the annual conference on USENIX Annual Technical Conference.

[Tang et al. T-PDS ’08] Q. Tang, S. K. S. Gupta, and G. Varsamopoulos, “Energy-ecient thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach,” IEEE Trans. Parallel Distrib. Syst., vol. 19, no. 11, pp. 1458–1472, 2008.

[Chase et al. SOSP ’01] J. Chase, D. Anderson, P. Thakar, A. Vahdat, and R. Doyle, “Managing energy and server resources in hosting centers,” in SOSP ’01: Proceedings of the eighteenth ACM symposium on Operating systems principles. New York, NY, USA: ACM, 2001, pp. 103–116.

[Chen et al. NSDI ’08] Y. Chen, A. Das, W. Qin, A. Sivasubramaniam, Q. Wang, and N. Gautam, “Managing server energy and operational costs in hosting centers,” SIGMETRICS Performance Evaluation Review, vol. 33, no. 1, pp. 303–314, 2005.

[Ranganathan et al. ISCA ’06] P. Ranganathan, P. Leech, D. Irwin, and J. Chase, “Ensemble-level power management for dense blade servers,”. ISCA ’06. 33rd International Symposium in Computer Architecture, 2006, pp. 66–77.

[Kusic et al. CCJ ’09] D. Kusic, J. O. Kephart, J. E. Hanson, N. Kandasamy, and G. Jiang, “Power and performance management of virtualized computing environments via lookahead control,” Cluster Computing, vol. 12, pp. 1–15, 2009.

28

LRH weight

Experimental setup to validate TASP and TAWD

Heat recirculation Coefficient

LRHA=0.69P2

LRHB=0.36P2

LRHC=0LRHD=0

A B

CD

Experimental Results TASP Validation:

NoSP: All servers are equally utilized (25%)

TASP(sLRH): Two thermal efficient servers are utilized 50% and the other two machines are turned off

CPSP (Thermally oblivious): Two non thermal efficient ate utilized 50% and the other two are turned off

TAWD Validation LB: All servers are equally utilized such that their

utilization fluctuate over fine time slots (30 second) TAWD: Workload is skewed toward thermal efficient

servers in fine time slots, such that the total workload in any moment equals to LB scenario

30

System model and assumptions Heterogeneous Data Center

Different computing efficiency (MIPS) Different computing power efficiency (Joules/MIPS) The solutions can be applied for Homogenous data centers

Heat recirculation in the data center room Computing Racks are organized in hot aisle and cold isle Heat recirculation among computing nodes Different thermal efficiency

31

Energy consumption model of Data Center

32

Coefficient of Performance(source: HP)

= + ×

N 1 A C

R ecircu la t io n

T su p T in T o u t T A C in

N 2 N 3

1 2 1 3

2 13 1

1 1

Tin Tsup D P

inlettemperatures

supplied airtemperatures

heat distribution

powervector

Computing power Cooling

power

+Etotal=

[1] J. Moore, J. Chase, P. Ranganathan, and R. Sharma, “Making scheduling "cool": temperature-aware workload placement in data centers,” in ATEC ’05: Proceedings of the annual conference on USENIX Annual Technical conference. Berkeley, CA, USA: USENIX Association, 2005, pp. 5–5.[2] Q. Tang, S. K. S. Gupta, and G. Varsamopoulos, “Energy-efficient thermal-aware task scheduling for homogeneous high-performance computing data centers: A cyber-physical approach,” IEEE Trans. Parallel Distrib. Syst., vol. 19, no. 11, pp. 1458–1472, 2008.

tPcomp tTCoP

pcomp

)( sup = tpDTCoP

pcompii

i

red

comp

))(( maxtPcomp +

Improving cooling energy by minimizing the maximum of servers’ inlet temperature

Perquisites of the analytical formulation for TASP Energy consumption modeling

Computing Power consumption modeling Linear model with respect to utilization

Performance modeling (Response time) Posing a cap for the CPU utilization:

Workload modeling Kalman filtering to predict average traffic,

33

= tpDTCoP

pcompii

i

red

comp

))(( maxtPcomp +Computing

power Cooling power

iiicompi up

i

threshi

i c

umax

avgprev

avg

prevavgpeak

)(

Idle power

Workload arrival during fine time slots

Formulating TASP: Optimization problem Unknown variable

How many servers are required? Which servers among all servers should be chosen as active server set?

Objective: Minimizing total energy consumption:

Constraint: Meet the capacity requirement: x is a binary vector:

34

Defining a binary vector as the variable. Each element determines if a server should be chosen or not. x: 1 0 0 1 0 1 …..

peak

i

threshi

N

ii c

ux

1

Nixi ...1},1,0{

tuxuxDTCoP iiiiiiiii

i

red)(

))(((

11

max

Computing power

Heat recirculation

Formulating TAWD

Unknown Variable: Finding the workload distribution weights of :

Objective: Minimizing total energy consumption during a slot

Constraints: Performance Constraint :

Capacity Constraint:

Solutions: Using heuristic approaches (CP-sLRH) Ranking servers based on CP-sLRH Giving the maximum affordable workload to the highest ranking servers

35

TccDTCoP iiiiiiiii

i

red)(

))(((

11

max

,1ˆ

Ss

i

i

}{ ib

Ssuuc ithreshiiii

ˆ,

Evaluation

Baseline Algorithms TASP with respect to CPSP

TAWD with respect to LB

Evaluation methods Experiments (Small scale) Simulation

36

Series1

Series2

Simulation results-The performance of various TASP approaches

Saving energy over time: MinMax and CP-LRH always surpasses CPSP, sLRH may perform worse than CPSP when active server set becomes large

37

Energy consumption of thermal aware server provisioning scenario over time (intervals in epochs). MinMax always do better than CPSP.

Request arrival rate over time of SPECweb2009 where epoch-level peaks are obtained from the 1998 FIFA World Cup traces

The More overestimation The less energy saving

The more overestimation the less QoS violations

The smaller active server size the larger saving

Energy saving with respect to CPSP for different TASP schemes over ϒ. Note that higher utilization yields higher savings.

38

Simulation results -Energy saving with respect to the overestimation factor (ϒ)

CPU utilization violations with respect to ϒ over time. Violations for ϒ=1 are much higher than for

the rest values.

Energy saving of MiniMax over different number

of active server size for ϒ =6.

Simulation results -The performance of various TASP approaches

39

Total energy consumption with respect to server provisioning scenario. The energy-saving percentages are with respect to CPSP.

Simulation resultsPerformance of TAWD Saving energy through

skewing workload toward thermal efficient servers

Average data center utilization of each server (over one week), as sorted with respect to LRH. The effects of TAWD’s load skewing on the utilization are obvious.

40

System model and assumptions Virtualized Data Center

All systems are capable of running any web application

Internet traffic Short transaction-based traffic Short and long term variation

41

- Software assumptions

http://www.internetworldstats.com/stats.htm

TASP Algorithm

42

TAWD Algorithm

43

How effective are TASP and TAWD? Simulation setup according to the physical layout of ASU

HPCI data centre and the combination of SPECweb traffic profile and FIFA World CUP 1998 web trace

Evaluating TASP compared to CPSP(Computing Power based Server Provisioning) Saving energy from 4.5% to 8.4% with respect to TASP

scenario and overestimation of active server set size

Evaluating TAWD compared to LB(Load Balancing) Saving 1% more energy by combination of TASP and TAWD

44