How Can EDA Help Solve Challenges in Data Center Energy Efficiency?

Ayse K. Coskun

Electrical and Computer Engineering Department

Boston University

http://people.bu.edu/acoskun

http://www.bu.edu/peaclab/

June 5, 2016

Energy Efficiency of Computing

2

Emerging applications in big data, cyberphysicalsystems, internet of things, cloud, etc.:• Growing

performance/Watt demand

Source: J. Koomey (Stanford, LBNL), 2011

• Energy-related costs are among the largest contributors to the total cost of ownership in data centers

Source: International Data Corporation (IDC)

3

• Data centers consume ~3-4% of US electricity (2011)

• IT is estimated to be responsible of 10% of world energy use (2013)

• Cutting 40% of server room energy waste could save businesses $3B annually (2013)

Energy Efficiency of Computing

Power Grid & Market• Power supply = demand ? ( => blackouts )

• Renewable energy sources: intermittent

• Lack of reliable, large-scale, economical energy storage solutions

• Independent System Operator (ISO): • Demand Response: • Peak Shaving, Capacity Reserves (new)

• Credits provided to the participant who modulates its power consumption dynamically 4

• Electricity: >3% of the overall consumption in the US[1]

• Power capping /management techniques • Enable flexibility in power consumption

• Workload flexibility

5

Demand Side –Data Centers

Benefits of Participation• Help solve unstable renewable energy problem

• Provide additional reserves to accommodate other less flexible uses of electricity

• Achieve significant monetary savings

Data centers offer a unique opportunity for providing power capacity reserves.

[1]: J. Koomey. Growth in Data Center Electricity Use 2005 to 2010.Oakland, CA: Analytics Press. August, 1, 2010.

6

Data Centers in the Smart Grid

7

Regulation Service (RS) Reserves

Bidding: ( P , R)Price Settling:Get contract

ISO: RS signal

Data Center Regulation

Pcap(t) = + z(t)R

Error:

ε(t) needs to be small: ε(t) > threshold => lose license

Costs: • ΠE and ΠR : market clearing

prices • Credits are reduced based on

statistics of ε(t)

P

e(t) =Preal (t)-Pcap(t)

R

RP RE

Typical PJM 150sec ramp rate (F) and 300sec ramp rate (S) regulation signal trajectories

Credit Earned

• Server States:

• Active: Pserver = Pdyn + Pstatic

• Pdyn can be modulated by DVFS or CPU resource limits

• Pdyn = k * RIPS

• Idle: Pserver= Pstatic

• Sleep: Pserver= Psleep

• Constant low power, but resuming from sleep has time delay (tres) and energy cost (Eloss)

• Servicing Model:

8

Data Center Model

Queue

Server 1

*

Allocation

FIFOJob arrival(Homogeneous jobs)

Server 2

Server N

Server i

……

……

Each server: 1 job at a time

[ICCAD’13, ASPDAC’14, IGCC’14]

Server State Transition Rules [Gandhi IGCC12]:

• A server that has been in idle > ttout (timeout threshold):

goes to sleep;

• When a new job arrives:

select the server with the smallest current tidle(t) to activate;

• When we need to force servers to sleep:

select the servers with the largest current tidle(t) to put to sleep.

tidle(t): the time that a server has been in the idle state at time t.

9

Dynamic Power Control Policy

[ICCAD’13, ASPDAC’14, IGCC’14]

• Goal: Reduce energy consumption under QoS constraints

• Put all active servers at their maximal throughput (to reduce waste from idle power)

• Determine the minimal number of servers required at each time t, based on:

• the current length of queue

• the overall QoS performance till t

• SLA:

• If additional servers are required: wake them up

• Otherwise: apply server state transition rules for spare servers 10

QoS-feedback

Pserver, j = k j *RIPS j +Pstatic

Nmin =h(S(t)+F(t)+Q(t))-SSLA(t)-FSLA(t)

d

(d,h), d =Treal /Tmin

[ICCAD’13, ASPDAC’14, IGCC’14]

• Case 1: Preal(t) < Pcap(t)

1. Active servers with Pserver < Pmax: Pserver Pmax;

2. Existing waiting jobs and idle servers: activate idle servers Pmax;

3. Sleeping servers: resume using server state transition rules.

Do the above three steps in order until Preal(t) = Pcap(t).

• Case 2: Preal(t) > Pcap(t)

1. Active servers with Pserver < Pmax: Pserver -> Pmin;

2. Active servers with Pserver = Pmax: Pserver -> Pmin;

3. Idle servers: suspend using server state transition rules.

Do the above three steps in order until Preal(t) = Pcap(t).

11

Regulation Service (RS)

Pmin set for guaranteeing QoS.

[ICCAD’13, ASPDAC’14, IGCC’14]

Server provisioning for a cluster[ASPDAC’14, IGCC’14]

Goals: • Minimize tracking error• Reduce #transitions• Reduce idle energy waste

0 0.5 1 1.5 2 2.50

0.2

0.4

0.6

0.8

1

Power Tracking Error

Pro

ba

bili

ty

Distribution of Power Tracking Error

single server

data center

0 5 10 15 20 250

0.1

0.2

0.3

0.4

0.5

Servicing Time Degradation

Pro

ba

bili

ty

Distribution of Servicing Time Degradation

single server

data center

Regulation Reserves (R) /Avg. Power ( ): • Single Server: 29.7%• 100-server Data

Center: 56.8%

12

e(t) =Preal (t)-Pcap(t)

R

minu(t )ÎU (x(t ))

J(x(t),u(t)) =a1 Preal (t)-Pcap(t) +a2Ntran(t)-a3Nsleep(t)-a4Npeak (t)

Tracking Error Transition Energy Waste Static Energy Waste

13

Results[ICCAD’13, ASPDAC’14, IGCC’14]

Data Center Power Management

14

Racks

Multi-core server

Multi-level load queues

Data Center

Pcooling

Pcomputing

Sensor feedback

Data center cooling control

Workload arrivals

ISOs

Load forecasting & bidding in the energy market

Optimal control &

allocation of power caps

Regulation requests

Performance,

power &

temperature

models

• Server power capping

• Efficient consolidation

• Cooling control

Power Capping on Multicore Systems

Large Scale Computing System (several racks)

Server(multicore processors)

Individual Server Power Cap

Allocated by a budgeting policyMaintain target power consumptionTypically performed with DVFS

Goal: Adaptively control individual server to maximize performance within power cap

15

Parallel Workloads Increasingly Prevalent

Core 1

Core 3

Core 2

Core 4

Thread 1

Thread 3

Thread 2

Thread 4

Active

Low-Power State

Power Capping on Multicore Systems

Large Scale Computing System (several racks)

Server(multicore processors)

Individual Server Power Cap

Allocated by a budgeting policyMaintain target power consumptionTypically performed with DVFS

Goal: Adaptively control individual server to maximize performance within power cap

16

Parallel Workloads Increasingly Prevalent

Core 1

Core 3

Core 2

Core 4

Thread 1

Thread 3

Thread 2

Thread 4

Active

Low-Power State

Optimal DVFS + Thread Packing(Pack & Cap)

blackscholes canneal fluidanmiate swaptions

17

[ICCAD’11, MICRO’11, IEEE Micro’12]

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

1.60 GHz1 core

2.00 GHz1 core

2.67 GHz1 core

1.60 GHz2 cores

2.00 GHz2 cores

2.67 GHz2 cores

Φ(x1)

Power Cap = 120 W

X XX XX

X XX

X X

X X

XXX

Φ(x2)

Φ(x3)

Φ(x4)

Φ(x5)

Φ(x6)

18

[ICCAD’11, MICRO’11, IEEE Micro’12]

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

110 W50 s

120 W45 s

130 W40 s

115 W30 s

125 W25 s

135 W20 s

105 W50 s

115 W45 s

125 W40 s

110 W30 s

120 W25 s

130 W20 s

1.60 GHz1 core

2.00 GHz1 core

2.67 GHz1 core

1.60 GHz2 cores

2.00 GHz2 cores

2.67 GHz2 cores

Φ(x1)

Power Cap = 120 W

X XX XX

X XX

X X

X X

XXX

Φ(x2)

Φ(x3)

Φ(x4)

Φ(x5)

Φ(x6)

Sensor and counter inputs

Model LearningOptimal Setting

Calculation

Model Parameters

ϕ(x)

w

y

Statistical classifier to determine most relevant metrics (Logistic regression, L1 regularization, …)

C. ControllerModel Query Controller

Model Lookup

Server Node optimal settings

ϕ(x)

w

y

Runtime Operation

Optimal DVFS + Thread Packing(Pack & Cap)

Adherence to Power Caps

Without a power meter:

• 0 W margin –

82% adherence

• 5 W margin –

96% adherence

• 10 W margin –

99+% adherence

Feedback-control using a power meter improves tracking ability .

19

[ICCAD’11, MICRO’11, IEEE Micro’12]

Temperature vs. Leakage Power

20

[Zapater, DATE’13], [Zapater, Trans. PDS’14]

• Multi-threading becoming more common (media processing, scientific apps, financial computing, etc.)

• Resource allocation per application / per VM

a key factor in power control & efficient consolidation

21

Virtualization Layer Virtualization Layer

vCPU vCPU vCPU vCPU vCPU vCPU

VM

vCPU vCPU vCPU

VM

vCPU vCPU vCPU vCPU

VM

vCPU vCPU vCPU vCPU vCPU vCPU

Efficient Consolidation[Hankendi, IGCC’13, ISLPED’13]

• Cloud resources for HPC (among other traditional uses of the cloud)

Predicting Performance Scalability

22

[Hankendi, IGCC’13, ISLPED’13]

Estimate the CPU demand of VM:• CPU demand=RUN%+READY%• 97% accuracy for estimating the CPU

demand of the applications• Without requiring offline training

Adaptive Power Capping:vCap

23

VM Monitor

Hypervisor

vCap

Power readings

Estimate CPU demands

Power Cap

Compute Rcap

QoS Req.-Check QoS

and Pcap

violations

-Set CPU

limits

Climit(VMn)

[Hankendi, IGCC’13, ISLPED’13]

Budgeting Computational vs. Cooling Power

24

𝑇𝑚𝑎𝑥

𝑃𝑐𝑜𝑚𝑝𝑢𝑡𝑒𝑇𝑐𝑟𝑎𝑐 , 𝑃𝑐𝑜𝑜𝑙𝑖𝑛𝑔

𝑃𝑠𝑒𝑟𝑣𝑒𝑟0

𝑃𝑠𝑒𝑟𝑣𝑒𝑟1

𝑃𝑠𝑒𝑟𝑣𝑒𝑟2

[Tuncer, ICCD’14]

CoolBudget

25

Offline system

characterization

Wait for

workload

change

Model workloadBudget 𝑃𝑐𝑜𝑚𝑝𝑢𝑡𝑒

for a given 𝑇𝑐𝑟𝑎𝑐

Change 𝑇𝑐𝑟𝑎𝑐

Best

𝑇𝑐𝑟𝑎𝑐found?

Set 𝑇𝑐𝑟𝑎𝑐 and

server powers

Optimization

No

Yes

[ICCD’14]

Computing Energy Efficiency – Take-Aways

1. Complexity of the systems and the physical phenomena

performance power

thermal hot spots and gradients

cooling cost

Performance Reliability

Leakage

energy cost

26

27

1. Complexity of the systems and the physical phenomena1. Complexity of the

2. Time-varying and diverse application behavior

Computing Energy Efficiency – Take-Aways

28

1. Complexity of the systems and the physical phenomena1. Complexity of the

2. Time-varying and diverse application behavior

3. Changes in how costis assessed

– e.g., integration with

“provider-side” programs

ISO

Data Center

Computing Energy Efficiency – Take-Aways

Key Aspects of Data Center Efficiency Research

• Intra- & cross-layer optimization

• Application-awareness in all the layers

• Runtime learning and optimization capabilities

• Interactions of physical phenomena

• Strong interdisciplinary aspect: novel materials, architectures, energy markets, etc.

29

Current graduate students:

Hao Chen, Fulya Kaplan, Tiansheng Zhang, Ozan Tuncer,

Onur Sahin, Emre Ates, Onur Zungur, Yijia Zhang

Collaborators:

D. Atienza & Y. Leblebici @ EPFL,

J. Ayala @ UCM, J. M. Moya @ UPM,

C. Isci, S. Duri @ IBM TJ Watson,

T. Brunschwiler @IBM Zurich,

L. Benini @ ETHZ/U. of Bologna,

M. Caramanis, M. Herbordt, A. Joshi, J. Klamkin and Y. Paschalidis @ BU,

K. Gross & K. Vaidyanathan @ Oracle,

V. Leung, and A. Rodrigues @ Sandia Labs

S. Reda @ Brown University,

D. Tullsen @ UCSD.

Postdoctoral researcher: Dr. Ata Turk

Alumni: Dr. Jie Meng, Dr. Can Hankendi, Nathaniel Michener, Ann Lane, Katsu Kawakami, John Furst, Samuel Howes, Jon Bell, Benjamin Havey, Ryan Mullen

30

RecentFunding:

Performance and Energy Aware Computing Laboratoryhttp://www.bu.edu/peaclab

Many masters students, especially: Dan Rossell, Charlie De Vivero

Visitors: Dr. Marina Zapater, Dr. Andrea Bartolini