The green data center. - IBM WWW Page · The green data center. Page 3 Highlights While creating a...

Optimizing I T

May 2007

The green data center.

More than social responsibility: a foundation for growth,

economic gain and operating stability


Contents

2 Introduction

4 Challenges facing CIOs

7 Transitioning to a green

data center

10 Whose job is it?

10 Reducing cooling requirements

11 Increasing facilities

system efficiency

13 Reducing power consumption

with innovative technologies

17 Do you have a game plan?

18 Conclusion

Introduction


Highlights

While creating a green data center

can be a complex undertaking, there

are many solutions and techniques

available to support the transition.


Highlights

High-density rack-mounted servers

can increase hot spots and tax cool-

ing systems, making it difficult for

aging data centers to keep up with

today’s demands.

Challenges facing CIOs


Highlights

Cooling and electrical costs rep-

resent up to 44 percent of a data

center’s total cost of ownership,

although some companies are

finding that they can’t buy extra

electricity at any price.


Highlights

Energy costs are rising, supply is

limited, the data center infrastruc-

ture is being taxed, and its ability to

meet business demands is at stake.


Highlights

Transitioning to a green data center

Thermal load management

Data center energy efficiency opportunities

Integration maximizes power savings

Product design and workload/energymanagementComponent

power efficiency

Rack design

Power management Power and workload

management

Cap

Virtualization

Measure Adjust

to workload

Liquid heat

removal

Airflow

management

Variable cooling

delivery

Rack/room layout

Facilities system

improvements

Data center Network

Optimize the delivery of power and cooling to the data center (reduce waste)

Reduce systems power losses through utilization of more efficient equipment

Integrate cooling delivery with equipment demand

The technologies and strategies

for improving data center energy

efficiency span the data center

ecosystem.


Highlights

A best practices assessment and

energy audit make it possible to

pinpoint areas of high energy use,

while establishing a baseline for

further planning.


Highlights

Opportunities to improve energy

efficiency can range from major

infrastructure upgrade projects

to a number of simple and

inexpensive measures.


Highlights

Facilities and IT departments need

to collaborate—sometimes with the

addition of outside help—in finding

ways to meet environmental and

energy challenges.

Whose job is it?

Reducing cooling requirements


Highlights

Increasing facilities system efficiency

Organizing data centers into

thermal zones can eliminate

hot spots that challenge cooling

systems and enhance system

reliability by helping to avoid

heat-related hardware failures.


Highlights

New chiller systems, thermal

storage systems and air delivery

systems can help reduce both

energy requirements and costs.


Highlights

Reducing power consumption with innovative technologies

With IT equipment becoming more

energy efficient and greener all the

time, replacing older IT equipment

with newer models can reduce over-

all power and cooling requirements.


Highlights

Virtualization

A server that’s only 15 percent

utilized may cost as much to run

as a server that’s fully utilized,

making virtualization an energy

efficient and cost-effective option.


Highlights

Just as server virtualization reduces

the number of servers needed,

storage virtualization reduces

the number of spindles required.


Highlights

Power management in IT systems

New power management technology

makes it possible to meter actual

power usage and cap the amount

of power used by a single server or

group of servers.

Going green at IBM

Like many companies, IBM has found that

supporting environment-friendly initiatives

can be a smart business move. A significant

area of focus is reducing a company’s carbon

footprint, or the amount of carbon dioxide

(CO2) emissions a company is directly or

indirectly responsible for producing. Power

consumption is considered an indirect

contributor to a company’s carbon footprint

because power companies produce CO2

emissions in the generation of electricity.

“While some assume that cutting CO2 emis-

sions costs businesses money, we have

found just the opposite,” said Wayne Balta,

vice president, Corporate Environmental

Affairs and Product Safety. “Energy effi-

ciency solutions have saved IBM an annual

average of US$15.8 million and achieved an

enviable 4.9 percent annual average energy

savings rate against its annual energy usage

since 1998. In CO2 emission avoidance,

that’s equal to removing 51,600 cars, each

traveling 10,000 miles annually.”

Eco-friendly disposal

Do you have a game plan?



HighlightsConclusion

For more information

ibm.com/cio

CIOs’ ongoing efforts to think green

will help keep their companies oper-

ating in the black.

Contributors


© Copyright IBM Corporation 2007

IBM Global Services

Route 100

Somers, NY 10589

U.S.A.

Produced in the United States of America

05-07

All Rights Reserved

IBM and the IBM logo are trademarks or registered

trademarks of International Business Machines

Corporation in the United States, other countries,

or both.

Other company, product, or service names

may be trademarks or service marks of others.

References in this publication to IBM products

or services do not imply that IBM intends to

make them available in all countries in which

IBM operates.

1 Kenneth G. Brill, “Data Center Energy Efficiency

and Productivity,” The Uptime Institute. 2007.

2 Jonathan G. Koomey Ph.D. Staff Scientist,

Lawrence Berkeley National Laboratory and Con-

sulting Professor, Stanford University, “Estimating

total power consumption by servers in the U.S.

and the world,” February 15, 2007.

3 Alex Barrett, “For PG&E customers, it pays to

virtualize,” SearchServerVirtualization.com, Octo-

ber 26, 2006. http://searchservervirtualization.

techtarget.com/originalContent/0,289142,sid94_

gci1226458,00.html.

GTW01270-USEN-00

Systems and Technology Group Lab Services

Data Center Services Offering

Data Center Thermal Analysis and OptimizationOptimize data center thermal profile to eliminate hot spots and reduce

energy consumption.

Offering Highlights

p" Analyzes existing or future

data center thermal

capacity, installed IT

equipment load, and data

center features using state-

of-the-art tools including

Computational Fluid

Dynamics (CFD) modeling

software.

p" Optimizes existing or future

IT equipment layout.

p" Evaluation completed by

expert IBM Thermal

Engineers.

p" Attractively priced offering

Increasing Stress on Infrastructure

In the previous 5 years the

performance of servers and storage

has increased exponentially. Along

with this performance growth has

come a significant increase in the

power dissipated by the server

and storage hardware. This has

caused a significant strain on the

data center infrastructure that was

built for hardware power levels much

less than what is being delivered

today. Placement of IT equipment, air

handlers, and other related equipment

is critical to efficient use of available

space. Lead time for facility upgrades

and capital planning requirements

mandate comprehensive planning.

Realizing the Vision

A key step to effective deployment

of IT equipment is an evaluation of

the current and future thermal profile

of the data center. IBM thermal

analysis services enable clients

to provide the best ventilation and

cooling for their IT hardware by

making the necessary infrastructure

modifications or layout changes.

Offering Description and Value

This offering can be divided into

two phases depending on client

needs. Phase 1 will focus on today’s

existing data center with analysis

and recommendations intended for

the existing situation. Phase 2 will

focus on the future data center with its

expected IT equipment. The analysis

between the two phases is similar,

while the inputs and results for each

phase could be rather different.

The Project

This rapid engagement to evaluate

current and future data centers is

typically scheduled to complete

in an elapsed time frame of 4

weeks. This may include up to

5 days of on-site collaboration

between IBM and the client.

http://www.ibm.com/us/

Typical Project Tasks

1. Project scope/resources/schedule

agreed upon

2. On-site evaluation and consultation

3. Analyze and optimize with CFD

models

4. Final meeting with evaluation report

and recommendations for data center

layout

For More Information

To find out more about IBM Data Center

Thermal Analysis and Optimization

and other related products and

services, contact IBM Data Center

Services at [email protected] .


IBM Corporation

Route 100

Somers, NY 10589

U.S.A.


1-07

All Rights Reserved

References in this publication to IBM products or

services do not imply that IBM intends to make them

available in every country in which IBM operates.

Consult your local IBM business contact for

information on the products, features, and services

available in your area.

IBM and the IBM logo are trademarks of

International Business Machines Corporation in the

United States, other countries or both.

Other company, product or service names may be

trademarks or service marks of others.

All statements regarding IBM future direction or

intent are subject to change or withdrawal without

notice and represent goals and objectives only.

The IBM home page on the Internet can be found at

ibm.com


mailto:[email protected]

http://www.ibm.com


IBM IT Systems Energy Efficiency Assessment

Reducing the environmental demand of IT

Offering Highlights

The “IBM IT Systems Energy

Efficiency Assessment” is a

rapid assessment that:

p Investigates and analyzes a

client’s IT power, cooling

and space utilization

p Can develop a solution that

reduces power, cooling, and

data center space required

by IT

p Considers capacities or

constraints of client’s

facilities

p Outlines strategy to help

achieve client’s goals

p Typically completes in 2-3

weeks

p Leverages IBM’s STG

Lab — experts in the

optimization of systems

power, thermal, and space.

p" Is attractively priced

Achieve More with Less Power

Many data centers have hit or are

approaching their maximum power

limits, and are expansion constrained.

By utilizing proven IT Optimization

methods, this assessment will

provide steps that can dramatically

improve the ability of a datacenter to

deliver workload and performance,

while using less power, cooling

resources and data center space.

This improvement in IT efficiency

can help clients grow within their

existing datacenter constraints.

This assessment and strategy is

very attractive when compared

with other expensive and long

lead time options for relocating

or building a new data center.

The assessment is performed by teams

of IBM professionals with deep skills

in IT optimization methods combined

with professionals with deep skills

in IT and facilities power, cooling

resources, and data center space.


The assessment analyzes existing

IT systems and provides guidance

and strategies to achieve required

IT workload and performance with

less power, cooling resources, and

data center space requirements.


This often allows the client to grow

within the power constraints of

existing datacenter, and avoid or

delay the expense of datacenter

expansion or relocation.

The reduction in power, cooling

resources, and data center space

required to achieve computational

workloads provides for reduced

energy operating expenses.

Some utilities provide attractive

incentives for projects that

reduce power consumption.

The Project

This assessment rapidly

analyzes clients IT systems and

facilities parameters, and then

formulates strategy to reduce

power consumed by IT.


1. Agree upon scope / resources / on

site-site schedule

2. Baseline collection of IT Systems

information / survey estimations

3. Outline possible technical solutions

4. Develop recommendations

5. Deliver assessment report

Project Prerequisites

This assessment requires client

IT Executive and client Facilities

Executive sponsorships.


To find out more about the IBM

IT Systems Energy Efficiency

Assessment and other related

products and services , contact IBM

Data Center Services at datacntr@

us.ibm.com.


IBM Corporation

Route 100

Somers, NY 10589

U.S.A.


8-07

All Rights Reserved
















ibm.com




http://www.ibm.com


Data Center Services Offering

Server and Storage Power/Cooling Trends and

Data Center Best Practices

Building your IT investment knowing infrastructure best practices were employed

Offering Highlights

p" Review power/cooling

trends of the current IBM

product line; provide future

power/cooling trends (under

non-disclosure agreement)

and show how they impact

the data center

p" Provide state-of-the-art

data center designs

p" Provide potential

ventilation schemes used in

cooling high density racks

p" Provide comparison of the

various data center liquid

cooled enhancements

p" Review data center power,

thermal, and I/O best

practices

p" Completed in a single day

seminar and discussion

Increasing Stress on Infrastructure

In the last 5 years the performance of

servers and storage has increased

exponentially. Along with this

performance growth has come a

significant increase in the power

dissipated by the server and storage

hardware. This has caused a

significant strain on the infrastructure

of data centers that were built for

hardware power levels much less than

what are being shipped today. The

much higher rack power levels have

caused customers to spread out their

server products in order to cool them in

the current facilities, using up valuable

and expensive raised floor space.


This offering helps our clients

understand the current and future

power, cooling and I/O demands that

IT equipment will place on their existing

or planned data center infrastructure.

IBM Lab Services Data Center

Services Best Practices education

provides a clear and concise roadmap

of power, cooling, and I/O trends for

our Server/Storage product line and its

design impacts on the data center, as

well as allowing the client to develop

a robust growth plan for IBM’s next

generation Server/Storage hardware.


The overall approach involves

working with the client to understand

the impacts of Server/Storage

power, thermal, and I/O trends on

their current data centers, as well

as those planned for the future. The

consultant will provide the client with

the knowledge to architect a unique

and balanced plan to successfully

optimize IT resources and help reach

business goals with less risk, reduced

Total Cost of Ownership, greater

efficiency, and keeping the power

levels of the IT equipment within the

power constraints of the data center.

The Project

This client/consultant engagement

is completed at the client’s site as a

single day seminar and discussion.

The consultant will educate the client

as to the power and cooling trends of

the IBM product line and then show

how the data center best practices

can help to handle current and

future situations in the data center.


1. Provide current IBM product power/

cooling trends across all brands

2. Provide power/cooling trends for

future IBM products (requires non-

disclosure agreement)

3. Show ventilation schemes for cooling

clusters of high power racks and

provide TCO of these schemes

4. Provide comparison studies on liquid

spot cooling solutions


To find out more about IBM Server

and Storage Power/Cooling Trends &

Data Center Best Practices education

and other related products and

services, contact IBM Data Center

Services at [email protected].


IBM Corporation

Route 100

Somers, NY 10589

U.S.A.


1-07

All Rights Reserved
















ibm.com



Identifying opportunities for optimizing airflow

IBM IT Facilities Assessment, Design and

Construction Services – optimized airflow

assessment for cabling

Highlights

p Offers guidance for increasing

data center availability

p Proposes a cabling design that

can help improve data center

airflow for optimized cooling

p Recommends steps to facilitate

operational efficiency through

simplified change management

p Suggests changes that may

lower the operating cost of

managing a data center

Understanding the impact of increased

cabling in data centers

Data center infrastructures have

reached a tipping point. Many major

changes have occurred over the past

five years that impact data center

operations. For one, high-density

servers contain more computing

capacity per square foot than ever

before. While individual servers use

less space than traditional servers,

the increased number of servers per

square foot actually increases the

demand for data center connectivity.

As a result, the higher density of serv-

ers creates denser cabling networks.

As important as cabling is, however,

the expansion of the data center’s

cabling infrastructure in many cases

can be an ad hoc activity. This can

lead to unknown cable routing; undoc-

umented cabling; airflow blockages;

unpredictable results from moves,

additions and changes; and unknown

risks with every underfloor activity.

An unstructured cabling infrastructure

can result in several issues for business

processes and IT systems, including:

2

IBM IT Facilities Assessment, Design

and Construction Services – optimized

airflow assessment for cabling provides

a comprehensive evaluation to help

you measure your current cabling infra-

structure, identify where your cabling

infrastructure issues are and determine

which improvements to implement.

The optimized airflow assessment for

cabling service includes the following

physical inspections:

Increasing the availability of your

data center

The IT department spends consider-

able time and effort supporting data

center availability. But availability comes

from more than the data center hard-

ware. Backup hardware and mirroring

concepts may not pay off if the cabling—

often hidden under the raised floor—is

not designed with the same quality as

the rest of the IT environment. An inade-

quately designed cabling environment, for

example, may not provide the redundancy

you need to support prompt problem

determination and error recovery.

IBM IT Facilities Assessment, Design

and Construction Services – optimized

airflow assessment for cabling provides

a comprehensive, fact-based analysis

that prioritizes tactical plans across your

data center to improve system avail-

ability through optimizing the underfloor

airflow. Underfloor cabling systems are

often blocked with dense tangles of

cables. These blockages prevent cool-

ing systems from doing their jobs. When

the system overheats, it can go down.

Optimizing your cabling infrastructure

airflow can protect the system from over-

heating, which can result in improved

system availability and a better service

level from your data center.

Offering recommendations to optimize

data center airflow for improved cooling

Many IT managers are unaware of the

jumble of cables hiding in their under-

floor plenum. With unstructured cabling

systems, each jumper cable connects

one machine port directly to another to

form a link. With the huge number and

diversity of fiberoptic cables in today’s

data centers, an underfloor cabling

system can soon get out of control and

lead to inefficiencies. Cabling blockages

can cause inefficient airflow and can

ultimately lead to wasteful cooling and

increased energy demands.

Another cause of hotspots within data

centers is bypass airflow. Unsealed

cable openings and open rack spaces

can allow hot air within the data center

to access the cabinets, leading to addi-

tional cooling and energy requirements.

During the optimized airflow assess-

ment for cabling engagement, IBM

informs you of the latest technology avail-

able for a structured cabling system

as well as the potential benefits. Not only

can an efficient cabling system lead to

3

increased data center availability, it can

lower cooling and energy requirements

as well. With the optimized airflow assess-

ment for cabling service, IBM helps you

get the facts to understand your current

cabling setup and what the ideal cabling

solution is for your environment.

The optimized airflow assessment for

cabling service offers advice about:

Planning for upgrades and changes

After IBM assesses your current cabling

infrastructure and explains the potential

benefits that a structured cabling system

can provide, we can help you determine

what you can do to make changes and

upgrades to your data center easier and

more affordable.

You may be able to achieve signifi-

cant benefits by adopting a structured

cabling system—principally through

the reduction in the number of fiber-

optic cables under your raised floor.

In a typical underfloor environment,

the fiberoptic trunk cables connect

the machine ports to the back of patch

panels that are located in a series of

cabinets or racks that make up the

CPL. The fronts of the patch panels

contain individual ports that, in a struc-

tured cabling environment, are directly

connected to the machine ports. In a

structured environment, connections

between two machine ports can be

made more quickly and easily by run-

ning a short jumper cable to connect the

two patch panel ports, decreasing the

number of cables in the plenum. Fewer

cables can simplify documenting the

location of cables. And better docu-

mentation can make it easier to trace

a fiberoptic link when you are problem

solving or planning for future growth.

And in a structured cabling system,

unlike a discrete jumper cable solution,

none of the change activity happens

near the active equipment. Future equip-

ment additions and changes can be

made in the same manner and should

not be affected by the amount of equip-

ment already installed. During the life

of your data center, you’re likely to add

and upgrade storage devices, switches,

directors and even servers. The opti-

mized airflow assessment for cabling

service can help you understand how

a structured fiberoptic cabling system

can enable you to plan and implement

these critical changes faster and with

reduced risk.

Recommending modifications to lower

data center operating costs

IBM Facilities Assessment, Design

and Construction – optimized airflow

assessment for cabling provides a

comprehensive evaluation of the physi-

cal infrastructure that supports your IT

equipment to identify potential energy

cost savings. The assessment can help

you understand the potential financial

payback that may accompany efficiency

improvements. It can provide you with

a framework to make future data center

cabling infrastructure decisions. And,

ultimately, this assessment may help

you redirect existing cooling to addi-

tional IT equipment.

Recommendations to improve cabling

efficiency are tailored to your facility

and may include recommendations for

upgrades or replacements of inefficient

cabling infrastructure or the addition

of new technologies. In IBM’s experi-

ence, cooling systems typically present

the greatest opportunity for efficiency

improvements, followed by electrical

and building systems. Your assessment

report will also give you a comprehensive

checklist of information gathered during

the assessment as well as recommenda-

tions and conclusions so you can make

improvements to your cabling system.

Why IBM?

IBM is a leading global provider of

data center design, construction, relo-

cation and optimization services. We

have built more than 30 million square

feet of raised-floor data centers

for clients worldwide. We currently

manage more than 100 IBM data

centers of six million square feet.

Our structured methodology, intel-

lectual capital and global reach

position us to deliver superior, com-

prehensive data center solutions.

IBM has a strong ecosystem of data

center technology alliances, includ-

ing strategic relationships with major

power, cooling and cabling equip-

ment vendors that give us exceptional

insight into industry trends and the

data center’s changing energy needs.

And unlike other data center solu-

tions providers, IBM manufactures a

full suite of storage and server prod-

ucts and has developed a structured

enterprise fiber cabling solution using

fiberoptic components known as the

IBM fiber transport system. All these

factors give us a deep understanding

of the infrastructure support needs of

your IT equipment.

Beyond the optimized airflow assess-

ment for cabling service, IBM also

offers services to implement data

center efficiency improvements.

We can help ensure that your data

centers and facilities support your

business needs — and that they are

capable of scaling to support busi-

ness growth. IBM services can cover

virtually all aspects of establishing and

running data centers, from assess-

ment and strategy to construction and

management—virtually anywhere in

the world.


To learn more about IBM IT Facilities

Assessment, Design and Construction

Services – optimized airflow assess-

ment for cabling, contact your IBM

representative or visit:

ibm.com/services


IBM Global Services

Route 100

Somers, N.Y. 10589

U.S.A.


05-07

All Rights Reserved

IBM and the IBM logo are trademarks or registered

trademarks of International Business Machines

Corporation in the United States, other countries,

or both.

Other company, product and service names may

be trademarks or service marks of others.



available in all countries in which IBM operates.

GTD01332-USEN-00

IBM Data Center Services

Building or upgrading your data center,

while helping to minimize cost, risk and delay

When you need to expand your data center beyond its current

walls, renovate its infrastructure or build a completely new data

center on a new site, IBM Data Center Services can help—with

everything from preliminary consulting to a total turnkey solution.

IBM’s 50 years of experience in data-center construction can

provide just the solution you need—whether big or small. You

can have IBM analyze your existing data center and project its

growth—based on your business plan, computing environments

and IBM’s unique understanding of future computing technologies.

You can get a complete recommendation with all drawings and

specifications. The entire project can be turned over to IBM, using

our experienced project managers and contractors, IBM Data

Center Services can build your data center in a cost effective and

timely manner with our qualified and experienced data center

professionals.

The power of experience

If yours is like most organizations, you won’t be facing another

major data center construction project for a long time—if ever

again—so it has to be built right. Take advantage of the exper-

tise and experience of IBM Site Planning Services personnel to

handle both facilities and computing concerns. Because they

specialize in the fit-up, renovation, moving and expansion of

data-processing environments like yours, these professionals

can provide their services economically, with low risk and minor

impact to your ongoing operations. Working with IBM means

that you have a single point of contact for both facilities and

computer expertise.

The power of relationships

IBM offers a wide range of services at every level through its

nationwide organization and network of qualified subcontrac-

tors — a network that includes architects, engineers, contractors

and suppliers. With proven expertise in data-center construction

and operations, IBM Data Center Services can handle practi-

cally any job in a timely, cost-effective manner. You get access to

IBM technical assistance for planning and management, along

with qualified, locally-based contractors and trades people. For

example, if you need more air-conditioning capacity, you can be

confident that you are going to get the right unit, installed cor-

rectly by people who have experience handling the job.

The value of IBM

IBM’s expertise and commitment to excellence helps you feel

confident that your data center can meet all technological and

environmental requirements for today and tomorrow. IBM guar-

antees the work of its contractors against defects in materials

and workmanship for a full year. As a result, bringing IBM into

your data-center construction project means that you can focus

on your real business—providing effective business solutions

to the users of your computing system.

The result: world-class quality in a data center built to last and designed for growth.

IBM Data Center Services: planning, design and construction

IBM can plan, design, build and test your data center, as well as

train your staff on how to maintain the new environment.

Planning

IBM builds a five-year growth model for your system based on

your company’s business plan, your current and future data-pro-

cessing needs and likely technological evolution. Based on this

model, IBM provides a full evaluation report that includes electri-

cal, mechanical and space requirements, along with equipment

layout drawings, cost estimates and a proposed construction

schedule. The optimal predesign solution described in this report

gives you step-by-step directions for proceeding.

Design

The evaluation report provided by IBM is so thorough that it can

be used to generate a design for a data center. And that’s exactly

what IBM does—helping to ensure that all report topics, such

as power, cooling, fire protection and other relevant topics, are

addressed and resolved in the final design. Also, IBM can work

with your architect, if you have one, to help ensure that unique

data-center requirements, such as underfloor airflow for ventila-

tion, are met.

Construction

IBM can act as your project manager through all phases of con-

struction. You get a turnkey solution no matter how complex the

job—and it’s guaranteed. Planning and design are available as

separate packages, or you can use IBM for the entire project.

Beyond the data center

Along with IBM Data Center Services, IBM can provide IBM and

non-IBM equipment maintenance; data-center relocation ser-

vices; application, data and hardware migration support; systems

integration; network implementation, analysis and management;

business-recovery services and more. IBM can help you manage

virtually any aspect of your information-systems environment you

choose. All so you can better manage what matters most—your

competitive edge.

You get a turnkey solution no matter how complex the job—and it’s guaranteed.


To learn more about IBM Data Center Services, or any other IBM

services, visit:

ibm.com/services/datacenter

IBM Data Center Services at a glance

Planning services

• Computer hardware planning

− Verify and assess existing facility (including hardware

configuration, personnel workflow and adjacency requirements)

− Provide electrical and mechanical load analysis

− Develop hardware layout plans

• Architecture and space planning

− Identify needs

− Recommend guidelines for design

− Develop workflow and adjacency diagrams

− Develop planning report (including modeling of growth

expectations, establishing general cost estimates and

preliminary schedules, and technically evaluating physical

needs)

− Develop contract options

• Mechanical engineering

− Evaluate existing systems capacity and condition

− Recommend upgrade equipment, sizing and location

(including air conditioning, cleansing and distribution, water

chillers, and heat rejection)

− Recommend fire detection and suppression equipment

− Recommend uninterruptible power supply and battery

equipment

• Electrical engineering

− Evaluate existing electrical components

− Recommend upgraded equipment, sizing and location

− Evaluate power distribution units (PDUs)

− Evaluate lighting and other areas for data center and

support rooms

− Address safety, grounding and noise attenuation issues

− Evaluate existing uninterruptible power supply systems and

recommend any necessary upgrades

Consulting services

• Design consulting

− Review data-center plans developed by other vendors

− Recommend design objectives, parameters and guidelines

− Review designs to help ensure compliance with requirements

− Review progress submissions for working drawings

− Make recommendations to help ensure design compliance

• Bidding

− Evaluate and select bidders

− Prepare and issue competitive bidding documents

− Receive and analyze competitive proposals

− Arrange bidding meetings

• Construction consulting (available only with design consulting

services)

− Attend project progress meetings

− Review to help ensure construction meets specifications

− Perform on-site visits for progress and compliance

− Recommend actions to resolve discrepancies

− Review contractor’s monthly invoicing and final payment

− Help ensure compliance with all contractual obligations

− Regularly report on project status

Bringing IBM into your data-center construction project means that you can

focus on your real business—providing effective business solutions to the

users of your computing system.

Site selection services

• Land evaluation

− Provide a comprehensive study of sites, including character-

istics and utilities

− Evaluate and rank multiple sites based on location, environ-

mental limitations, available utilities and aesthetics

• Land analysis

− Provide an in-depth study of site conditions (surface and

subsurface), characteristics, and suitability

• Building evaluation

− Analyze existing building for suitability

Turnkey services

• Design

− Select and direct the design team

− Integrate architectural and engineering features, such as site

usage, design concepts, and plans and specifications

− Establish final characteristics of facility

• Construction (available only with turnkey services design)

− Handle general-contractor responsibilities, including

providing a single point of contact, awarding construction

and labor contracts, and purchasing materials and environ-

mental equipment

− Provide overall project management, including reviewing

and approving submissions by subcontractors; securing

all guarantees, instructions, certificates of inspection, record

drawings and so on; conducting post-construction inspec-

tion and testing, and final inspection; and familiarizing

you with the new facility

© IBM Corporation 2006

IBM Corporation

Department EU6

122 Westchester Avenue

White Plains, NY 10604

U.S.A.


09-06

All Rights Reserved

IBM and the IBM logo are trademarks of International

Business Machines Corporation in the United States,

other countries or both.

Other company, product and service names may be


GUIDELINES FOR ENERGY-EFFICIENT

DATACENTERS

FEBRUARY 16, 2007

ABSTRACT

In this paper, The Green Grid™ provides a framework

for improving the energy eficiency of both new and

existing datacenters. The nature of datacenter energy

consumption is reviewed and best practices are

suggested that can signiicantly impact operational

eficiencies.

ABOUT THE GREEN GRID™

The Green Grid is a non-proit trade organization of IT

professionals formed to address the issues of power

and cooling in datacenters. The Green Grid seeks

to deine best practices for optimizing the eficient

consumption of power at the IT equipment and facility

levels, as well as the manner in which cooling is

delivered at these levels. The association is funded by

four levels of membership, and activities are driven by

end-user needs. The Green Grid does not endorse any

vendor-speciic products or solutions, but will seek

to provide industry-wide recommendations on best

practices, metrics, and technologies that will improve

overall datacenter energy eficiencies.

white paper 1

INTRODUCTION

Power availability is one of the most important

challenges facing datacenters today. In the past,

datacenter loor space has always loomed as the

primary issue. Now, more and more datacenters

run out of power availability before they run out of

loor space. In addition, cooling requirements for

dense servers are driving power demand and taxing

the normal datacenter operational procedures.

Operations are not properly “tuned” to accommodate

the new energy-hungry environment. This paper

illustrates existing electrical consumption patterns

and suggests various strategies for reducing

consumption. Energy improvements can be made

from both an equipment-planning perspective and

an operational-practices perspective for both IT and

physical infrastructure (power, cooling, rack, security,

ire suppression, and monitoring) devices.

ENERGY COSTS AND CONSUMPTION

For years, electrical power usage was not considered

a key design criteria for datacenters. Nor was

electrical consumption effectively managed as an

expense. In fact, many datacenter managers are

unaware of what their monthly energy bill is. This

is true despite the fact that the electrical energy

costs over the life of a datacenter may exceed the

costs of the electrical power system including the

uninterruptible power supplies (UPS), or even exceed

the cost of the IT equipment itself.

The reasons for this situation are as follows:

• Electrical bills are sent out long after charges

are incurred. No clear link exists between

particular decisions, like the installation of a

new zone of equipment in the data center or

operational practices and the increased cost of

the electricity. In fact, electrical bills are viewed

as an inevitable event that most people don’t

consider trying to inluence.

• Tools for modeling the electrical costs of

datacenters are not widely available and are

not commonly used during datacenter design.

• Billed electrical costs are often not within

the responsibility or budget of the datacenter

operating group.

• The electrical bill for the datacenter may be

included within a larger electrical bill and may

not be available separately.

• Decision-makers are not given suficient

information during planning and purchasing

decisions regarding the energy cost

consequences.

If the datacenter were 100% eficient, all power

supplied would reach the IT loads. This would

represent Power Usage Effectiveness (PUE) of 1.0.

PUE is further discussed in the Green Grid white

paper entitled “Green Grid Metrics.” In the real world,

electrical energy is consumed by devices in a number

white paper 2

of ways before it even reaches the IT loads. Practical

requirements such as keeping IT equipment properly

housed, powered, cooled, and protected is one

example of how energy consumption is sidetracked,

or rendered less eficient (see Figure 1).

Note that all energy consumed by the datacenter in

Figure 1 ends up as waste heat, which is rejected

outdoors into the atmosphere. This diagram is based

on a typical datacenter with 2N power and N+1

cooling equipment, operating at approximately 30%

of rated capacity.1

System design issues that commonly reduce the

eficiency of datacenters include:

• Power distribution units and/or transformers

operating well below their full load capacities.

• Air conditioners forced to consume extra power

to drive air at high pressures over long distances.

• Cooling pumps which have their low rate

automatically adjusted by valves (which

dramatically reduces the pump eficiency).

• N+1 or 2N redundant designs, which result

in underutilization of components.

• The tradition of oversizing a UPS to avoid

operating near its capacity limit.

• The decreased eficiency of UPS equipment

when run at low loads.

• Under-loor blockages that contribute to

ineficiency by forcing cooling devices to

work harder to accommodate existing load

heat removal requirements. (This can lead

to temperature differences and high-heat

load areas might receive inadequate cooling).

BEST PRACTICES

Right-sizing the physical infrastructure system to the

load, using eficient physical infrastructure devices,

and designing an energy-eficient system are all

techniques to help reduce energy costs. A successful

strategy for addressing the datacenter energy

management challenge requires a multi-pronged

approach that should be enforced throughout the

lifecycle of the datacenter. The following categories

of practices serve as cornerstones for implementing

an energy-eficient strategy: engineering, deployment,

operations, and organization.

white paper 3

Chiller

Humidifier

CRAC

IT Equipment Indoor

Data

Center

Heat

Electrical

Power

IN

Waste

Heat

OUT

PDU

UPS

Switchgear/generator

Lighting

33%

3%

9%

30%

5%

18%

1%

1%

Figure 1: Where Does It Go?

SYSTEM DESIGN

In datacenters, system design has a much greater

effect on the electrical consumption than does

the eficiency of individual devices. In fact, two

datacenters comprised of the same devices may have

considerably different electrical bills. For this reason,

system design is even more important than the

selection of power and cooling devices in determining

the eficiency of a datacenter.

FLOOR LAYOUT

Floor layout has a signiicant effect on the eficiency

of the air conditioning system. Ideal arrangements

involve hot-aisle/cold-aisle conigurations with

suitable air conditioner locations. The primary design

goal of this loor layout approach is cool air and warm

air segregation.

PROPER CONFIGURATION OF SERVER SOFTWARE

When coniguring servers, many datacenter managers

are not careful about how they conigure the power-

related software. Power-economizer modes should

always be selected to ensure more eficient operation

of the server.

LOCATION OF VENTED FLOOR TILES

In an average datacenter, many vented tiles are

either placed in incorrect locations or an insuficient

or excessive number of vented tiles is installed. By

using Computational Fluid Dynamics (CFD) in the

datacenter environment, the designer can optimize

datacenter cool air low by “tuning” loor tiles by

varying locations and by regulating the percent of

vents that are open at any given time or can optimize

CRAC (Computer Room Air Conditioning) unit

locations. Some vendors offer cooling optimization

services and have demonstrated over 25% energy

savings in real-world applications.2

RIGHTSIZED PHYSICAL INFRASTRUCTURE

COMPONENTS

Of all of the techniques available to users, rightsizing

the physical infrastructure system to the load has

the most impact on physical infrastructure electrical

consumption. There are ixed losses in the power

and cooling systems that are present whether

the IT load is present or not, and these losses

are proportional to the overall power rating of the

system. In installations that have light IT loads, the

ixed losses of the physical infrastructure equipment

commonly exceeds the IT load. Whenever the physical

system is oversized, the ixed losses become a larger

percentage of the total electrical bill.

Rightsizing has the potential to eliminate up to 50%

of the electrical bill in real-world installations. The

compelling economic advantage of rightsizing is a key

reason why the industry is moving toward modular,

scalable, physical infrastructure solutions. The very

nature of the modular, scalable infrastructure implies

that new physical infrastructure equipment is added

only when additional IT loads are added.

The ability to predict future power and cooling

loads is also key in managing an energy-eficient

datacenter. The American Society of Heating,

Refrigeration & Air Conditioning Engineers (ASHRAE)

offers a series of design guides that provide help in

this task. The additional work performed up front to

accurately predict the datacenter power and cooling

load will pay for itself in both reduced capital and

operational expense.

ENGINEERING FOR EFFICIENCY

white paper 4

INSTALLATION OF MORE EFFICIENT

POWER EQUIPMENT

New best-in-class UPS systems have 70% less energy

loss than legacy UPS at typical loads.3 Average

“light” load eficiency is the key parameter, not the

full load eficiency. In addition UPS losses must be

cooled, doubling system energy costs.

CLOSELY COUPLED COOLING

Due to increasing density in the datacenter, a

trend towards closely coupled cooling solutions has

developed. Close-coupling targets speciic areas

where cooling is needed (such as an individual row,

rack, or server) as opposed to a large open space

(such as the datacenter room). In addition, close-

coupling can result in shorter air paths that require

less fan power. Close-coupled heat removal minimizes

and almost eliminates the mixing of cool and hot air,

since the airlow is completely contained in the row

or rack.

Traditional datacenters that move to high-density

server implementations without close-coupled cooling

typically attempt to modify existing infrastructure

through additional construction. Those modiications

for high density rarely improve the eficiency of the

datacenter. However, new datacenter designs, where

the focus is on matching the room airlow to server

airlow and on preventing the mixing of cool and warm

air, can be quite eficient.4

Most of today’s existing datacenters attempt to cool

equipment by looding the air supply with as much

cool air as possible. The cool air produced by CRAC

units mixes with the heat produced by the load.

This system makes it dificult, if not impossible, to

target speciic heat sources within the datacenter.

The closely coupled approach greatly increases the

eficiency of the cool air distribution and hot air

removal systems. Due to the close coupling of CRAC

units to the load, all of the capacity can be delivered

to the load up to power densities on the order of

25 kW, or approximately 4X the practical density

capacity of room-oriented architecture.5

VIRTUALIZATION

Virtualization consolidates existing and expected

future workloads. This reduces the number of physical

servers required, thereby reducing loor space,

cooling, and capital costs. It also increases the

utilization of servers to improve energy eficiency.

Furthermore, virtualization can also serve as an

effective means for placing additional compute

capability into production.

INSTALLATION OF ENERGY-EFFICIENT LIGHTING

Other facility savings can be realized through

devices such as timers or motion-activated lighting.

Lighting power produces heat which, in turn, must

be cooled—doubling the cost. The beneit of energy-

eficient lighting is larger on low-density or partly

illed datacenters.

DEPLOYING FOR EFFICIENCY

white paper 5

INSTALLATION OF BLANKING PANELS IN RACKS

Airlow dynamics can be improved by utilizing

blanking panels on racks. The panels are an

inexpensive way to decrease server inlet temperature

while increasing the CRAC return air temperature—

thereby reducing energy consumption.

PLUMBING FOR CLOSELY COUPLED COOLING

Deployment of rack and server (chip-level) cooling

systems in closely coupled cooling solutions requires

delivery of facility water (chiller or condenser

water) to the racks in question. A variety of options

are available for delivering the water to these

racks, including hard plumbing and soft or lexible

plumbing. Delivery of water away from the periphery

of the datacenter into the heart of the datacenter

may be cause for concern for some datacenter

operators. These concerns can be allayed by

deploying sound engineering practices. The following

best practices are suggested when preparing a

datacenter for the deployment of closely coupled

cooling solutions:

• Insulate plumbing to prevent condensation (if

water has to be below the facility’s dew point).

• Ensure piping is easily accessible for service

and repairs, and to minimize disruption to

existing datacenter infrastructure (power,

communications, HVAC, etc.).

• Include stub-outs with shut-offs from periphery

plumbing at necessary intervals to allow isolation

of each rack row and each rack.

• Run plumbing in a direction parallel to that of the

CRAC air low to minimize air-low blockage, or

place plumbing in loor recesses (where possible)

to prevent air-low blockage (i.e., air damming).

• Provide leak-containment features around water

line components, such as drip pans, pipe wraps,

and gravity drains.

• Utilize home-run lexible piping to minimize the

number of pipe joints (and thus, the risk of leaks)

near critical components.

• Isolate plumbing from electrical wiring (place

plumbing in loor recesses, where possible, below

the elevation of power cables and components).

• Employ leak-detection systems and reaction

plans to minimize or eliminate impact of leaks

on datacenter operations.

DEVELOPMENT OF NEW SERVER

REPLACEMENT POLICIES

Server consolidation, if properly executed, can

also contribute to the overall high eficiency of the

datacenter. Below are examples of how to leverage

eficiency during server consolidation:

• Use a two-way server or a single-processor dual-

core server to replace two or more old servers.

• Replace an old server with a blade based on a

low-voltage or mid-voltage processor.

• Replace a dual-processor server with a single,

dual-core processor.

• Use a two-way dual-core server in place of a four-

way server.

white paper 6

UTILIZATION OF AIR CONDITIONING

ECONOMIZER MODES

Many air conditioners offer economizer options. This

can offer substantial energy savings, depending on

geographic location. Although some datacenters

have air conditioners with economizer modes, the

economizer operation is often disabled.

COORDINATION OF AIR CONDITIONERS

Many datacenters have multiple air conditioners that

actually ight each other. One may actually heat while

another cools and one may dehumidify while another

humidiies. The result is gross waste that may require

a professional assessment to diagnose.

OPERATING FOR EFFICIENCY

white paper 7

ALIGNMENT OF STAFF

To properly engineer the migration from a traditional

energy-consuming datacenter to a modern energy-

conserving datacenter requires an organizational

alignment that facilitates such a migration. Figure 2

illustrates an IT organizational structure that

integrates the expertise of personnel who understand

both IT systems and physical infrastructure systems.

The new organizational wrinkle involves the

integration of an IT facilities arm to the rest of

the IT organization.

This organizational alignment presents several

advantages. For years, IT and facilities departments

have operated as separate entities and evolved

separate cultures and even separate languages.

As a result, most datacenter design/build or

upgrade projects are painful, lengthy, and costly.

This new IT facilities group is a separate group

from the traditional “building” facilities group. The

IT facilities group acts as a liaison between IT and

the facilities building group, but is under the direct

control of IT.

The IT facilities group addresses datacenter issues

speciic to hardware planning, electrical deployment,

heat removal, and physical datacenter monitoring.

This organizational alignment allows a datacenter

team to rapidly deploy an energy-eficient datacenter

upgrade policy that addresses both IT systems and

physical infrastructure systems.

ORGANIZING FOR EFFICIENCY

white paper 8

IT Department

Hardware SoftwareVoice

and Data

IT

Facilities

Figure 2: Aligning for Energy Efficiency6

These strategies are effective for new datacenters,

and some can be deployed immediately or over time

in existing datacenters (see Figure 3 for printable

checklist). Simple no-cost decisions made in the

design and operation of a new datacenter can result

in savings of 20–50% of the electrical bill, and, if

deploying a systematic approach, up to 90% of the

electrical bill can be avoided.

For more information concerning Green Grid activities,

go to www.thegreengrid.org

CONCLUSION

white paper 9

Efficiency

Best Practice

Itemized datacenter electric

bill in hand

Optimization of datacenter

design

Optimization of data

equipment floor layout

Proper location of vented

floor tiles

Rightsizing of UPS

Installation of “green” power

equipment

Installation of a close-coupled

cooling architecture

Deployment of server

virtualization

Installation of energy-efficient

lighting

Installation of blanking panels

Installation of efficient

plumbing

Efficient server consolidation

practices

Utilization of air conditioner

economizer modes

Coordination of air

conditioners

Proper configuration of

server software

Proper alignment of

datacenter staff

Check-off

Box

Date

Executed

Figure 3: Datacenter Energy Efficiency Checklist

REFERENCES

1 Rasmussen, N., “Electrical Eficiency Modeling of Data Centers,” White Paper #113, APC, (2005) http://www.apcmedia.com/salestools/NRAN-66CK3D_R1_EN.pdf

2 Belady, C., “How to Minimize Data Center Utility Bills”, E-Business News, Hewlett-Packard, (September 5, 2006) HYPERLINK “http://www.line56.com/articles/

default.asp?ArticleID=7881” http://www.line56.com/articles/default.asp?ArticleID=7881

3 Rasmussen, N., “Implementing Energy Eficient Data Center,” White Paper #114, APC, (2006) HYPERLINK “http://www.apcmedia.com/salestools/NRAN-6LXSHX_

RO_EN.pdf” http://www.apcmedia.com/salestools/NRAN-6LXSHX_RO_EN.pdf

4 Patterson, M.K., Costello, D., Grimm P, Loefler, M., “Data center TCO; a comparison of high-density and low-density spaces” THERMES 2007, Santa Fe, NM (January

2007)

5 Dunlap, K., Rasmussen, N., “The Advantages of Row and Rack Oriented Cooling Architectures for Data Centers,” White Paper #130, APC, (2006) HYPERLINK “http://

www.apcmedia.com/salestools/VAVR-6J5VYJ_RO_EN.pdf” http://www.apcmedia.com/salestools/VAVR-6J5VYJ_RO_EN.pdf

6 Marcoux, P., MBA, APC (2006)

©2007 The Green Grid. All rights reserved.

Executive Summary

The United States (U.S.) Environmental Protection Agency (EPA) developed this report in response to the request from Congress stated in Public Law 109-431. This report assesses current trends in energy use and energy costs of data centers and servers in the U.S. and outlines existing and emerging opportunities for improved energy efficiency. It provides particular information on the costs of data centers and servers to the federal government and opportunities for reducing those costs through improved efficiency. It also makes recommendations for pursuing these energy-efficiency opportunities broadly across the country through the use of information and incentive-based programs.

Background

As our economy shifts from paper-based to digital information management, data centers — facilities that primarily contain electronic equipment used for data processing, data storage, and communications networking — have become common and essential to the functioning of business, communications, academic, and governmental systems. Data centers are found in nearly every sector of the economy: financial services, media, high-tech, universities, government institutions, and many others use and operate data centers to aid business processes, information management, and communications functions.

The U.S. data center industry is in the midst of a major growth period stimulated by increasing demand for data processing and storage. This demand is driven by several factors, including but not limited to:

the increased use of electronic transactions in financial services, such as on-line banking and electronic trading,

the growing use of internet communication and entertainment,

the shift to electronic medical records for healthcare,

the growth in global commerce and services, and

the adoption of satellite navigation and electronic shipment tracking in transportation.

Other important trends contributing to data center growth in the government sector include:

use of the internet to publish government information,

government regulations requiring digital records retention,

enhanced disaster recovery requirements,

emergency, health and safety services,

information security and national security,

digital provision of government services (e.g., e-filing of taxes and USPS on-line tracking), and

high performance scientific computing.

During the past five years, increasing demand for computer resources has led to significant growth in the number of data center servers, along with an estimated doubling

1

Public Law 109-431

SECTION 1. STUDY.

Not later than 180 days after the date of enactment of this Act, the Administrator of the Environmental Protection Agency, through the Energy Star program, shall transmit to the Congress the results of a study analyzing the rapid growth and energy consumption of computer data centers by the Federal Government and private enterprise. The study shall include--

(1) an overview of the growth trends associated with data centers and the utilization of servers in the Federal Government and private sector; (2) analysis of the industry migration to the use of energy efficient microchips and servers designed to provide energy efficient computing and reduce the costs associated with constructing, operating, and maintaining large and medium scale data centers; (3) analysis of the potential cost savings to the Federal Government, large institutional data center operators, private enterprise, and consumers available through the adoption of energy efficient data centers and servers; (4) analysis of the potential cost savings and benefits to the energy supply chain through the adoption of energy efficient data centers and servers, including reduced demand, enhanced capacity, and reduced strain on existing grid infrastructure, and consideration of secondary benefits, including potential impact of related advantages associated with substantial domestic energy savings; (5) analysis of the potential impacts of energy efficiency on product performance, including computing functionality, reliability, speed, and features, and overall cost; (6) analysis of the potential cost savings and benefits to the energy supply chain through the use of stationary fuel cells for backup power and distributed generation; (7) an overview of current government incentives offered for energy efficient products and services and consideration of similar incentives to encourage the adoption of energy efficient data centers and servers; (8) recommendations regarding potential incentives and voluntary programs that could be used to advance the adoption of energy efficient data centers and computing; and (9) a meaningful opportunity for interested stakeholders, including affected industry stakeholders and energy efficiency advocates, to provide comments, data, and other information on the scope, contents, and conclusions of the study.

SEC. 2. SENSE OF CONGRESS.

It is the sense of Congress that it is in the best interest of the U.S. for purchasers of computer servers to give high priority to energy efficiency as a factor in determining best value and performance for purchases of computer servers.

in the energy used by these servers and the power and cooling infrastructure that supports them. This increase in energy use has a number of important implications, including:

increased energy costs for business and government,

increased emissions, including greenhouse gases, from electricity generation

increased strain on the existing power grid to meet the increased electricity demand, and

increased capital costs for expansion of data center capacity and construction of new data centers.

For these reasons, there has been mounting interest in opportunities for energy efficiency in this sector. To its credit, the information technology (IT) industry is actively

2

investigating and developing solutions, such as power-managed servers and adaptive cooling.

The direct energy use of IT and infrastructure equipment is not, however, the only way that data centers affect energy use. The data processing and communication services provided by data centers can also lead to indirect reductions in energy use in the broader economy, which can exceed the incremental data center energy expenditures in some cases.1 For instance, e-commerce and telecommuting can reduce both freight and passenger transportation energy use. Nonetheless, even though IT equipment may improve energy efficiency in the economy as a whole, pursuit of energy efficiency opportunities in data centers remains important because of the potential for rapid growth in direct energy use in this sector and the resulting impact on both the power grid and U.S. industries.

Role of EPA EPA has a more than 15-year history of advancing energy efficiency in IT equipment as well as commercial buildings, beginning with the first ENERGY STAR specifications for computers established in 1992 and the Green Lights program established in 1991. Through the ENERGY STAR program, EPA now qualifies a wide array of IT products, including personal computers, imaging equipment, printers, and monitors. EPA has made particular strides in addressing standby energy and power management for these products, demonstrating that it is possible to encourage rapid development and adoption of energy-efficient technologies and practices. The energy savings from efficiency improvements in these products are currently in the billions of dollars per year (US EPA 2006). EPA has also developed an innovative commercial building rating system that helps owners and managers assess the energy performance of their buildings and target efficiency improvements.

In January 2006, EPA convened the first national conference dedicated to examining energy savings opportunities for enterprise servers and data centers. Representatives from the utility, financial services, healthcare, internet, and manufacturing sectors attended the conference (http://www.energystar.gov/datacenters). EPA is now working on the first priority identified in that conference, the development of objective measurements of server energy performance, on which future efficiency criteria would be based.

To develop this report, EPA convened a study team led by researchers from the Lawrence Berkeley National Laboratory. The study team offered stakeholders multiple opportunities to give input to and review this report, including:

conducting preliminary calls with key stakeholders to help plan the study;

holding a public workshop on February 16, 2007 (attended by approximately 130 people) to solicit input on the topic of energy efficiency in servers and data centers;

1 The magnitude of indirect energy reductions attributable to IT equipment is uncertain; one of this report’s recommendations is that research should be conducted to better understand this effect.

3

following up on workshop attendees’ offers of assistance, to gather and refine information for the study;

posting on the ENERGY STAR web site an open call for interested parties to submit information, as well as a list of data needs;

posting on the ENERGY STAR web site a public review draft of this report; and

incorporating into the final version of this report comments on the public review draft from more than 50 organizations and individuals.

Energy Use in Data Centers Through 2011

The energy used by the nation’s servers and data centers is significant. It is estimated that this sector consumed about 61 billion kilowatt-hours (kWh) in 2006 (1.5 percent of total U.S. electricity consumption) for a total electricity cost of about $4.5 billion. This estimated level of electricity consumption is more than the electricity consumed by the nation’s color televisions and similar to the amount of electricity consumed by approximately 5.8 million average U.S. households (or about five percent of the total U.S. housing stock). Federal servers and data centers alone account for approximately 6 billion kWh (10 percent) of this electricity use, for a total electricity cost of about $450 million annually.

The energy use of the nation’s servers and data centers in 2006 is estimated to be more than double the electricity that was consumed for this purpose in 2000. One type of server, the volume server, was responsible for the majority (68 percent) of the electricity consumed by IT equipment in data centers in 2006. The energy used by this type of server more than doubled from 2000 to 2006, which was the largest increase among different types of servers. The power and cooling infrastructure that supports IT equipment in data centers also uses significant energy, accounting for 50 percent of the total consumption of data centers. Among the different types of data centers, more than one-third (38 percent) of electricity use is attributable to the nation’s largest (i.e., enterprise-class) and most rapidly growing data centers.

These energy consumption estimates were derived using a bottom-up estimation method based on the best publicly available data for servers and data centers. The estimation was performed as follows:

- estimated the U.S. installed base of servers, external disk drives, and network ports in data centers each year (based on industry estimates of shipments and stock turnover);

- multiplied by an estimated annual energy consumption per server, disk drive, or network port; and

- multiplied the sum of energy use for servers, storage, and networking equipment by an overhead factor to account for the energy use of power and cooling infrastructure in data centers.

This method was also used to develop five-year projections for future energy use. A five-year time horizon was chosen for the scenarios because this is the period for which equipment shipment forecasts were available, and a period for which change in the rapidly evolving IT sector can be reasonably forecasted. Two baseline scenarios were analyzed to estimate expected energy use in the absence of expanded energy-efficiency

4

efforts. The “current efficiency trends” scenario projected the current energy use trajectory of U.S. servers and data centers based on recently observed efficiency trends for IT equipment and site infrastructure systems. The “historical trends” scenario did not reflect these current energy efficiency trends but simply extrapolated observed 2000 to 2006 energy-use trends into the future. The historical trends scenario projected the energy use of U.S. servers and data centers if no energy-efficiency improvements were made, and therefore indicates the energy savings associated with efficiency trends that are already under way.

Under current efficiency trends, national energy consumption by servers and data centers could nearly double again in another five years (i.e., by 2011) to more than 100 billion kWh (Figure ES-1), representing a $7.4 billion annual electricity cost. The peak load on the power grid from these servers and data centers is currently estimated to be approximately 7 gigawatts (GW), equivalent to the output of about 15 baseload power plants. If current trends continue, this demand would rise to 12 GW by 2011, which would require an additional 10 power plants.

These forecasts indicate that unless energy efficiency is improved beyond current trends, the federal government’s electricity cost for servers and data centers could be nearly $740 million annually by 2011, with a peak load of approximately 1.2 GW.

These estimates of data center energy use should be considered approximate because limited data are available on current data center energy use, and there is significant uncertainty about the effects of future technology trends, such as server consolidation and developments in network and storage technologies. However, these estimates and projections illustrate the magnitude of energy use in data centers and the need for effective energy-efficiency strategies. Energy consumption monitoring and reporting may be needed to both improve these estimates and inform future policy initiatives.

Energy-Efficiency Opportunities in Servers and Data Centers There is significant potential for energy-efficiency improvements in data centers. Although some improvements in energy efficiency are expected if current trends continue, many technologies are either commercially available or will soon be available that could further improve the energy efficiency of microprocessors, servers, storage devices, network equipment, and infrastructure systems. For instance, existing technologies and design strategies have been shown to reduce the energy use of a typical server by 25 percent or more. Even with existing IT equipment, implementing best energy-management practices in existing data centers and consolidating applications from many servers to one server could reduce current data center energy usage by around 20 percent. Energy-efficiency strategies could be implemented in ways that do not compromise data center availability, performance or network security, which are essential for these strategies to be accepted by the market. To develop a better understanding of energy-efficiency opportunities that would accelerate adoption of energy-efficient technologies beyond current trends, three energy-efficiency scenarios were explored:

5

The “improved operation” scenario includes energy-efficiency improvements beyond current trends that are essentially operational in nature and require little or no capital investment. This scenario represents the “low-hanging fruit” that can be harvested simply by operating the existing capital stock more efficiently.

The “best practice” scenario represents the efficiency gains that can be obtained through the more widespread adoption of the practices and technologies used in the most energy-efficient facilities in operation today.

The “state-of-the-art” scenario identifies the maximum energy-efficiency savings that could be achieved using available technologies. This scenario assumes that U.S. servers and data centers will be operated at maximum possible energy efficiency using only the most efficient technologies and best management practices available today.

Details of the key energy-efficiency assumptions used in this analysis are shown in Table ES-1. These assumptions represent only a subset of the energy-efficiency strategies that could be employed in practice; it is not a comprehensive list of all energy-efficiency opportunities available in U.S. data centers.

Table ES-1. Summary of Assumptions for Analysis of Alternative Efficiency

Scenarios

Data Center Subsystem

Scenario IT Equipment Site Infrastructure (Power and Cooling)

Continue current trends for server consolidation

30% improvement in infrastructure energy efficiency from improved airflow management

Eliminate unused servers (e.g., legacy applications)

Improved operation

Adopt “energy-efficient” servers to modest level

Enable power management on 100% of applicable servers

Assume modest decline in energy use of enterprise storage equipment

Bestpractice

All measures in “Improved operation” scenario, plus:

Consolidate servers to moderate extent

Aggressively adopt “energy-efficient” servers

Assume moderate storage consolidation

Up to 70% improvement in infrastructure energy efficiency from all measures in “Improved operation” scenario, plus:

improved transformers and uninterruptible power supplies

improved efficiency chillers, fans, and pumps

free cooling

State-of-the- art

All measures in “Best practice” scenario, plus:

Aggressively consolidate servers

Up to 80% improvement in infrastructure energy efficiency, due to all measures in “Best practice”

6

scenario, plus: Aggressively consolidate storage

direct liquid cooling Enable power management at data center

level of applications, servers, and combined heat and power equipment for networking and storage

Note: These measures should be considered illustrative of efficiency opportunities in a typical data center. Some measures may only be applicable in new or expansion data centers or may be infeasible for a given data center because of local constraints. Selection of efficiency measures for a particular facility should be based on a site-specific review.

Because the best practice and state-of-the-art scenarios imply significant changes to data centers that may only be feasible to implement during major facility renovations, it was assumed in these scenarios that the site infrastructure measures requiring new capital investments would apply to only 50 percent of the current stock of data centers. For IT equipment, it was assumed that the entire existing stock turns over within the five-year forecast period.

These scenarios, based on the assumptions outlined above, illustrate significant potential for efficient technologies and practices to improve the energy efficiency of servers and data centers by 2011:

The state-of-the-art scenario could reduce electricity use by up to 55 percent compared to current efficiency trends, representing the maximum technical potential.

The best practice scenario could reduce electricity use by up to 45 percent compared to current trends, with efficiency gains that could be realized using today’s technologies.

The improved operational management scenario offers potential electricity savings of more than 20 percent relative to current trends, representing low-cost energy efficiency opportunities.

7

Figure ES-1. Comparison of Projected Electricity Use, All Scenarios, 2007 to 2011

These scenarios show annual savings in 2011 of approximately 23 to 74 billion kWh compared to current efficiency trends, which reduces the peak load from data centers by the equivalent of up to 15 new power plants and reduces annual electricity costs by $1.6 billion to $5.1 billion. The projected savings in electricity use correspond to reductions in nationwide carbon dioxide (CO2) emissions of 15 to 47 million metric tons (MMT) in 2011. The best practice scenario shows that electricity use in servers and data centers can be reduced below its 2006 level during the next five years rather than almost doubling, which would be the result if current efficiency trends continue.

Based on the assumption that the federal sector accounts for about 10 percent of electricity use and electricity costs attributable to servers and data centers, the annual savings in electricity costs in 2011 to the federal government range from $160 million (for the improved operation scenario) to $510 million (for the state-of-the-art scenario).

Table ES-2. Annual Savings in 2011 by Scenario (Compared to

Current Efficiency Trends)

Scenario Electricity Electricity Carbon dioxide

consumption cost emissions

savings savings avoided

(billion kWh) ($billion 2005) (MMTCO2)

Improved operation 23 1.6 15Best practice 60 4.1 38State-of-the-art 74 5.1 47

8

These efficiency gains appear to be achievable without compromising product or data center performance. Because energy efficiency is a secondary attribute of the equipment used in data centers, changes that would compromise performance will generally not be implemented. In other words, data center designers and managers will first ensure that primary needs – performance and availability – are satisfied and will only then differentiate among products and practices based on energy efficiency. In some situations, improved energy efficiency increases performance and availability. For instance, better cooling distribution in data centers can eliminate hotspots and thereby prevent equipment faults. Finally, it is important to note that the energy-efficiency improvements addressed in this report reduce the costs of excess energy use and excessive power and cooling infrastructure.

The analysis in this report includes consideration of use of fuel cells and other distributed generation (DG) technologies in data centers. DG resources can reduce data center energy costs, particularly when used in combined heat and power (CHP) systems, which use waste heat to provide cooling. CHP systems can produce attractive paybacks and are well suited to the steady power and cooling loads of data centers. Clean DG also has the environmental benefits of reduced criteria pollutants and greenhouse gas emissions. Fuel cell DG systems offer many attractive qualities, such as DC power output, for use in data centers. But fuel cells, as a new-market entrant, have a premium price over more traditional DG systems. So while DG systems based on traditional gas turbine or engine technologies can be considered cost effective without incentives, fuel cells, in many cases, will need financial incentives to be cost effective. Finally, DG systems, particularly fuel cells, do not have a long track record in high power quality, high availability applications such as data centers Given the high cost of outages for these types of facilities, more demonstration and conclusive information about system availability are needed before most facility designers and operators would likely be willing to adopt DG and CHP technologies.

Incentives and Voluntary Programs to Promote Energy Efficiency

To realize the potential benefits from greater energy efficiency in the nation’s data centers, a number of market barriers need to be addressed. The adoption of energy-efficient technologies and practices is often impeded by barriers such as higher first cost, lack of knowledge, institutional failures, and perverse incentives, and these issues apply equally to data centers. The barriers that prevent data centers from adopting changes that offer very reasonable paybacks are typically not technological but organizational. Three barriers of particular importance in data centers are:

Lack of efficiency definitions: It is difficult to define energy efficiency for a complex system such as a data center or a server. “Energy efficient” is usually defined based on the delivery of the same or better service output with less energy input, but for servers and data centers service output is difficult to measure and varies among applications. Data center operators need standard definitions of productivity in order to purchase energy-efficient equipment, operate it in an optimal way, and design and operate the buildings to house it.

9

Split incentives: In many data centers, those responsible for purchasing and operating the IT equipment are not the same people that are responsible for the power and cooling infrastructure, who in turn typically pay the utility bills. This leads to a split incentive, in which those who are most able to control the energy use of the IT equipment (and therefore the data center) have little incentive to do so.

Risk aversion: With the increasing importance of digital information, data centers are critical to businesses and government operations. Thus, data center operators are particularly averse to making changes that might increase the risk of down time. Energy efficiency is perceived as a change that, although attractive in principle, is of uncertain value and therefore may not be worth the risk.

These barriers are not unique to data centers but may be more pronounced in this sector.There is a long history of incentive and informational programs to address barriers like these in other sectors – e.g., government agencies, public and private utilities. Although there are few current programs that specifically target data centers, existing energy policies and programs that promote high efficiency buildings and equipment – such as product labeling programs, commercial building technical assistance programs, financial incentives, and government procurement – may be applicable to data centers. These programs include:

Product labeling: Labels identify products that meet certain specifications for performance, including high energy performance, based on standard methods for measuring energy efficiency. These labels can make customers aware of the energy costs associated with their purchasing decisions and encourage consumer acceptance and recognition of high-efficiency products. The performance specifications that underlie the labels form clear purchasing guidelines. This in turn encourages manufacturers to make increasing numbers of efficient products.

Commercial building technical assistance: The growth of data centers is a relatively recent phenomenon, so best practices for design and operation are only recently being developed. Technical assistance programs provide information to facility designers and operators to help them effectively incorporate energy efficiency in the design and operation phases for their facilities. Newer practices in this area include establishment of whole-building energy performance benchmarking. Technical assistance can be provided by government agencies, electric utilities, professional organizations, and industry groups.

Financial incentives: Electric utilities and governments often offer financial incentives to encourage investments in energy-efficiency measures. Financial incentives help buy down the additional cost of more efficient products when initial product costs are higher than for less-efficient products, help compensate for the increased effort needed to learn about and locate energy-efficient equipment, draw attention to technologies, and legitimize these technologies in the eyes of consumers. The most active utility in the data center sector is Pacific Gas and Electric Company, which offers incentives for server consolidation, among other strategies.

10

Government procurement: Federal, state, and local governments spend tens of billions of dollars annually on energy-consuming products, which means that there are thousands of opportunities to reduce government energy use through the purchase of energy-efficient products. Government procurement programs help raise awareness of new-to-market energy-efficient products, increase comfort levels as efficient products are put into use, and reduce costs of manufacture through economies of scale. The federal government is required by law to purchase energy-efficient products unless these products are proven to be not cost-effective. The government has developed energy performance specifications for more than 70 types of products.

EPA has begun addressing the energy performance of equipment in data centers by supporting development of energy-performance metrics for servers. In addition, governments and utilities are exploring program mechanisms for promoting improved efficiency.

Recommendations

A mix of programs and incentives is necessary to achieve a significant portion of the potential savings identified in the energy-efficiency scenarios above. Improvements are both possible and necessary at the level of the whole facility (system level) and at the level of individual components. Although it is not possible to optimize data center components without considering the system as a whole, it is also true that efficient components are important for achieving an efficient facility (for instance, efficient servers generate less waste heat which reduces the burden on the cooling system). Nevertheless, the greatest efficiency improvements will likely result from a comprehensive approach, given that there are opportunities for improvement in many areas of the IT equipment and infrastructure systems.

Based on a review of a range of incentives and voluntary programs that have been used in other sectors, and considering the unique aspects of the server and data center market, a number of recommendations can be made to pursue improved energy efficiency in the near term. These recommendations include:

Standardized performance measurement for data centers — Data center operators need standard metrics to assess and report the energy performance of their facilities. The federal government and industry should work together to develop an objective, credible energy performance rating system for data centers, initially addressing the infrastructure portion but extending, when possible, to include a companion metric for the productivity and work output of IT equipment. These metrics should account for differences in data centers in areas such as computing output and availability requirements.

Federal leadership — The federal government can act as a model in encouraging improved data center efficiency. The government should commit to: publicly reporting the energy performance of its data centers once standardized metrics are available, conducting energy efficiency assessments in all its data centers within two

11

to three years, and implementing all cost-effective operational improvements. Additionally, the Architect of the Capitol should implement the server-related recommendations from the Greening of the Capitol report (Beard 2007).

Private-sector challenge — The federal government should issue a challenge to private-sector chief executive officers (CEOs) to conduct DOE Save Energy Now energy-efficiency assessments, implement improvements, and report energy performance of their data centers. These assessments require protocols and tools that should be jointly developed by government and industry.

Information on best practices — Objective, credible information is needed about the performance of new technologies and about best practices as well as the effect of both on data center availability. This information will help raise awareness of energy-efficiency issues in this sector and reduce the perceived risk of energy-efficiency improvements in data centers. The government should partner with industry to develop and publish information on field demonstrations and case studies of best practices. This information should be disseminated as part of a campaign to make data center managers aware of the benefits of energy efficiency in addressing power and cooling constraints in data centers.

Standardized performance measurement for data center equipment —Purchasers of data center equipment, such as servers, storage, network equipment, and uninterruptible power supplies (UPSs), need objective, credible energy performance information if they are to purchase efficient products.

o The federal government should work with industry to develop objective, credible energy performance metrics for this equipment.

o Using these metrics, the government should also investigate whether development of ENERGY STAR specifications for these product categories would be an effective strategy to complement the whole-facility approaches outlined above.

o If and when ENERGY STAR specifications are developed, federal procurement specifications that build on ENERGY STAR should be implemented.

Research and development—The federal government, in collaboration with industry, universities, electric utilities, and other stakeholders, should initiate a comprehensive research and development (R&D) program to develop technologies and practices for data center energy efficiency. Specific research needs are identified in Chapter 8 (R&D recommendations) of this report, covering the following topics: computing software, IT hardware, power conversion, heat removal, controls and management, and cross-cutting activities.

Public/private partnership for energy efficiency in data centers—The federal government should engage stakeholders to formulate a common initiative (including public policies and private-sector actions) to promote energy efficiency in data centers to continue the dialog that this report initiates. Logical next steps would

12

include defining priorities for the various strategies outlined in this report, developing timelines, defining roles for the various stakeholders, and identifying gaps and issues that require further assessment.

In addition to these near-term actions, several other actions can also play an important role in saving energy used by servers and data centers:

Federal Government:

- Develop a procurement specification for the energy performance of outsourced data centers.

- Work with industry to develop better tools, such as life-cycle risk models and total cost of ownership models that incorporate energy costs, for management of energy in data centers.

- Separately meter all federally owned data centers with significant energy use. - Charge data center tenants for energy consumption of IT equipment in

government-owned data centers. - Partner with electric utilities, universities, and the data center industry to develop

one or more neutral, “real-world” testing and demonstration centers (“National Center for Data-Center Best Practices”) to verify new technologies for reducing energy consumption in data centers.

- Help organize a technology procurement program to bring to market energy-efficient products for data centers.

- Partner with training organizations to develop education and training information and curricula about energy efficiency in data centers.

- Target data centers for efficiency upgrades using energy services performance contracts (ESPCs) and utility energy service contracts (UESCs).

- Provide technical assistance for demonstration projects of energy efficiency in data centers.

- Conduct demonstration and education projects for fuel cells and other clean, efficient DG technologies used for CHP in data centers.

- Develop a procurement specification to improve the efficiency of high-performance computing facilities.

State and Local Governments:

- Consider requiring separate utility meters on large data centers, either through utility regulation or building codes.

- Consider offering financial incentives for clean, efficient technologies used for CHP in high-availability installations (data centers, telecom facilities, etc.).

Electric Utilities:

- Consider offering incentives for energy-efficient data center facilities and equipment, based on the metrics described above.

- Consider partnering with the federal government to develop a neutral, “real-world” testing and demonstration center to verify new technologies for reducing energy consumption in data centers.

13

- Consider partnering with the federal government to develop a technology procurement program for efficient products.

- Consider offering education and training resources as a component of energy-efficiency programs for data centers.

- Consider offering financial incentives for clean, efficient DG and CHP in data centers.

Data Center Industry:

- Consider partnering with the federal government to develop an objective, credible energy-performance rating system for data centers.

- Consider partnering with the federal government to develop improved tools, such as “energy aware” total cost of ownership models and life-cycle risk models, for management of energy in data centers.

- Consider partnering with the federal government to develop a neutral, “real-world” testing and demonstration center to verify new technologies for reducing energy consumption in data centers.

Conclusions

This report helps define a vision for achieving energy efficiency in U.S. data centers. Although the growing energy use of servers and data centers makes this a challenging goal, there are large opportunities for savings. These savings will not be easy to achieve, given the barriers outlined in this report, but there are many policies available to overcome the barriers. Realizing these efficiency gains will take coordination and collaboration among many stakeholders: the government, the IT industry, data center operators, electric utilities, and others. The outlook for efficiency gains is encouraging, though, because industry is very engaged with these issues and is working with customers who are demanding solutions to the growing energy use in data centers. Federal initiatives should build on these efforts and partner in ways that develop objective, credible information, benchmarks, metrics, and industry standards. Finally, as a significant operator of data centers itself, the federal government can help facilitate change by changing the way it designs and operates its own facilities.

14

The green data center. - IBM WWW Page · The green data center. Page 3 Highlights While creating a...

Documents

Transcript of The green data center. - IBM WWW Page · The green data center. Page 3 Highlights While creating a...