1

Design and Analysis of an Energy Agile Cluster Computing System

Andrew Krioukov, Prashanth Mohan, Stephen Dawson-Haggerty, Sara Alspaugh, David Culler, Randy Katz

2

Grid Evolution

SUPPLIES

LOADS

mostly dispatchable

renewable, variable, intermittent, greatly non-dispatchable

oblivious, stochastic, mostly non-power proportional

reactive, mostly power proportional

TODAY IDEAL FUTURE

oblivious, flat

OLD GRID

non-renewable, reactive, dispatchable

3

Grid Evolution

SUPPLIES

LOADS

mostly dispatchable

renewable, variable, intermittent, greatly non-dispatchable

oblivious, stochastic, mostly non-power proportional

reactive, mostly power proportional

TODAY IDEAL FUTURE

oblivious, flat

OLD GRID

non-renewable, reactive, dispatchable

4

Grid Evolution

SUPPLIES

LOADS

mostly dispatchable

renewable, variable, intermittent, greatly non-dispatchable

oblivious, stochastic, mostly non-power proportional

power proportional, reactive, grid-aware

TODAY IDEAL FUTURE

oblivious, flat

OLD GRID

non-renewable, reactive, dispatchable

5

Grid

Internet

SUPPLIES:provide power

communicate renewable availability, price

LOADS:adapt demand

communicate forecast

electricity

information

Pieces Needed

?

6

Non-dispatchable, variable supply

Power proportional, grid-aware loads

NREL Western Wind and Solar Integration Study Datasethttp://wind.nrel.gov/Web_nrel/

Pacheco wind farm

Scientific computing cluster

Figure of merit: amount of wind used.How do we get here?

Renewable Integration

7

PO

WER

TIME

oblivious, flat load

dispatchable supply

power proportionality

grid-awareness

8

Data Center Loads

IT Equip-ment 61%

Cooling30%

Power Cond.8%

Light1%

data center consumption dominated by IT load

IT load driven by workload

need power proportionality

need load shaping mechanism

Server Idle:Peak

HP ProLiant DL160

63.5%

Apple XServe 3.1 51.8%

IBM System x3450

51.6%

Dell PowerEdge 2950

57.9%

Pelley, et. al, Understanding and Abstracting Total Data Center Power, 2009Barroso et. al. The Case for Energy-Proportional Computing, 2007

SPECpower Results http://www.spec.org/power_ssj2008/results/power_ssj2008.html

5,000 servers at Google

average 30% utilization

IT equipment is not power proportional

pow

er

(W)

utilization

9

Power Proportionality

Spinning Reserve

10

Architecture

11

Outline

• Motivation• Enabling technology• Methodology• Algorithms• Evaluation

12

Renewable Energy Component

13

Formulation

We assume the wind farm is sized for the data center.

Option 1: grid blend (open system)

Wind

Other

Requires assuming load is negligible fraction of grid – not realistic

Option 2: dedicated wind farm (closed system)

Fit load to specific wind farm

http://www.greenhousedata.com/

14

Wind

Wind power over 48 hours from a wind farm in Monterrey County, California.

Variation in wind power for month long intervals at multiple wind farms.

15

Workload Component

16

Workloads

Torque jobs

Num

Jobs

Batch: Less latency sensitive, longer jobse.g., analytics, scientific computing

Request

Rate

Wikipedia traffic

Interactive: Latency sensitive, generally short jobse.g., web app server, email server, etc.

23

Slack

slack = max run time – job duration

24

Cluster: NERSC FranklinAverage duration: 98 minAverage slack: 68 min

Cluster: EECS PSIAverage duration: 55 minAverage slack: 17 hours

Slack in Real Systems

25

Grid-Aware Batch Scheduling

• example goal: shape load to match wind availability

• method: exploit temporal slack

Pacheco wind farm

Scientific computing cluster

26

Greedy Algorithm

B(t) = power budget for next 10 min

Sort jobs by slack

Schedule all jobs with no remaining slack

Schedule other eligible jobs in least-remaining-slack order until B(t) is exceeded

27

Run-immediately, grid-oblivious scheduler

Greedy, grid-aware scheduler

Grid-aware scheduling increases wind energy use.

Correspondingly, reduces grid dependence.

29

Reduction in grid dependence is robust to choice of wind farm.

30

31

As slack increases, grid dependence diminishes.

PSI Franklin

32

Grid-aware scheduling is equivalent to 5 hours worth of data center-sized batteries.

33

Grid-aware scheduling is equivalent to 5 hours worth of data center-sized batteries.

4

34

Summary

• Power proportionality and grid-aware scheduling

• Energy savings, renewable integration, grid stability

reduce grid dependence by halfequivalent to 5 hours of batteries

• Next stepsslack in other systems...?

QUESTIONS?THE END

35