+ Conservation starts with utility providers

+ Saving power at the server

+ The debate of AC vs. DC power

+ Replacing batteries with flywheels

+ Measuring efficiency progress

c

CHAPTER 3

BACKUP POWER,DISTRIBUTIONAND CONVERSIONSTargeted steps to lower data center powerconsumption and strengthen the bottom lineBY MARK FONTECCHIO AND STEPHEN J. BIGELOW

Data CenterFacilitiesDesign for ITE-book

can’t even plug in more IT equipment.But why should data center man-agers care if they aren’t currentlyhurting for power capacity? A short-term benefit of controlling power con-sumption is a lower bottom line onyour company’s electric bill. In thelong term, it can forestall new datacenter construction projects, whichcould cost tens or hundreds of mil-lions of dollars. Those are financialbenefits that company executivesfavor—and could prompt salarybumps or promotions.Finally, while utility companies

and the government areeager to reward good ener-gy behavior now, it won’t belong before power and car-bon regulations are in placeto punish bad behavior.

CONSERVATION STARTSWITH UTILITY PROVIDERS

The first step in conserving powerin a new data center begins with theutility providers. Because powerusage has become a dominant factorin data center operations, begin dis-cussions with local utility providersearly when you’re planning to builda new facility.If a company’s data center is largeenough, it may be able to negotiatewith the utility provider to help payfor a new electrical substation. Thecompany might also agree to limitelectricity use when the power gridis straining.

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 2

CHAPTER 3 Backup power,distribution andconversionsFeeding the data center’s energy needswithout breaking the bank is possible—if you knowa few tricks.

Power consumption has become the reference ofchoice when describing the size of a data center. Beforethe rising costs of power caught the industry’s eye, datacenter managers described their facilities in terms ofsquare footage. Now they talk about how many megawattsthe data center consumes. In fact, some data centers stillhave plenty of physical room, but can’t expand. The powercapacity is tapped to the point that data center managers

Some utility providers have incen-tive programs that refund companiesfor making their data centers moreenergy efficient. A leader in this fieldis San Francisco-based Pacific Gas

and Electric (PG&E). The utility com-pany has 25 different energy-efficien-cy incentive programs, including onefor deploying server virtualization andanother that offers discounts to com-panies that install air-side or water-side economizers.The online marketplace eBay hastaken advantage of utility providerrebates. Olivier Sanche, former seniordirector of eBay’s data center servic-es, said the company did a simpletech refresh, installed solar panels onthe roof of its San Jose, Calif., datacenter and purchased economizers tohelp reduce energy consumption andsave money.“The incentives help us do the rightthing,” said Sanche. “Would we bedoing some of these things withoutthe incentives? Probably, yes. But this[incentives] helps the projects hap-

pen quicker.”Not all utility providers have theseincentive programs in place, so it’simportant to check with eachprovider individually. Rebate moneycomes from the rate base, meaningthat rate payers fund the program,said Chris Johnston, national criticalfacilities chief engineer for New York-based data center design firm SyskaHennessy. In essence, smart datacenter users benefit financially fromother “not-so-smart” users.

SAVING POWER AT THE SERVERIT equipment is the largest energyconsumer within the data center—accounting for about half of the en-ergy used, which makes it the bestplace to start when trying to saveenergy. A simple tech refresh can goa long way toward saving energy inthe data center.Equipment manufacturers are cre-ating new generations of processors,memory and power supplies that aremore energy efficient than previousgenerations. This is mainly becauseof pressure from within the industryas power consumption has growninto a major data center issue.The Environmental ProtectionAgency (EPA) now has an Energy Starrating for data center servers, includ-ing 1U, 2U and blade servers. Serversthat meet Energy Star requirementsare recognized for their energy effi-ciency and power-saving features.In June 2010, the EPA began issuingEnergy Star ratings for large data cen-

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 3

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

IT equipment isthe largest energyconsumer withinthe data center—accounting forabout half of theenergy used.

FIVEWAYS TO SAVE POWERIN THE DATA CENTERBuy energy-efficient servers. Newer servers tend to be more energyefficient; a simple tech refresh can actually save money. The EnvironmentalProtection Agency (EPA) lists servers that qualify under its Energy Starenergy-efficient server standard.

Optimize server efficiency. Deploying virtualization increases server use,which can help reduce overall power consumption. Turning on power-savingserver features, such as a sleep mode, can also save money.

Improve data center cooling infrastructures. Data center coolingaccounts for about 37% of a center's power consumption. Making smart cool-ing choices and planning efficient airflow designs will help cap energy use.

Use flywheel power supplies. Flywheel uninterrupted power supply (UPS)units rely on kinetic energy instead of huge strings of batteries. Because ofthis, flywheel UPSes can provide temporary backup power to IT equipmentand save money.

Measure, measure, measure. Although you won’t see direct savings simplyfrommeasuring your energy use, knowing where you stand in regard topower consumption gives you a starting point for any efficiency evaluation,and provides a basis for making informed decisions. �

ters. The Standard Performance Eval-uation Corp., an industry benchmark-ing organization based inWarrenton,Va., also publishes several power-related metrics. The first, calledSPECpower ssj2008, compares aserver’s Java performance to itspower consumption. The SPECvirt_sc2010 benchmark evaluates serverperformance in a virtual data center.Taking advantage of these free toolscan help determine which servers arebest from a power perspective.

Server virtualization is one ofthe most common ways to reducepower costs in a data center. Once aserver is virtualized, it’s possible toconsolidate multiple virtual work-loads onto one physical server, allow-ing the same number of workloads tooperate on fewer physical machines.This results in vastly improved serveruse and considerable power and cool-ing savings, and is a key element topower conservation strategies. Bycomparison, non-virtualized servers

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 4

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

often run at low usage rates, some-times in the single digits.The mantra with data center serversused to be “one app, one server.” Vir-tualization software, however, haschallenged this, allowing IT managersto stack several virtual machines(VMs) on top of a single physical box.This means that IT managers can getrid of older servers or delay the pur-chase of additional servers.

Server power supplies have alsoreceived a lot of attention as a way tosave energy. The 80 Plus initiative, anelectric utility-funded program basedin Portland, Ore., certifies server powersupplies that are at least 80% effi-cient. This means that 20% or less ofthe energy coming from the server’ssupply turns into wasted heat. Olderservers could have power supplieswith efficiency levels as low as 60%.Finally, more data center adminis-trators are implementing the power-down features available in modernservers, which often are alreadyembedded into operating systemsand just need to be activated. WhenIT loads are low, power-down fea-tures put unused servers into sleepmode, much like desktop computersgo into standby mode. When ITneeds spike, servers turn on again.If you can adjust all of these servercomponents to be more energy effi-cient, the benefits can add up. In2007, the EPA reported to Congressthat a data center server incorporat-ing energy efficiencies throughout allof its components could consume25% less energy in the data center.

Today, we’re seeing more energy-efficient server components andpower-saving features appear inthe latest server models.Cooling equipment is the secondlargest energy consumer in the datacenter. Cooling consumes up 37% ofa data center’s power envelope, notedan Emerson Network Power report.Carefully raising the temperature inthe data center, avoiding bypass andre-circulated air and using variable-speed cooling devices are just a fewways to save money when controllingyour data center’s climate..

THE DEBATE OF AC VS. DC POWERAccording to the SearchDataCen-ter.com 2011 Data Center Decisionssurvey, more than 27% of respon-dents have implemented direct cur-rent (DC) power or plan to do so inthe next year. Still, there are questionsabout whether DC power is more effi-cient than alternating current (AC).The AC versus DC dispute datesback to the 19th century, when DCpower advocate Thomas Edison

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 5

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

Power supplies onolder servers could have

efficiency levelsas poor as 60%.

Optimize your cash flow and efficiency with our scalable, flexible, and adaptable InfraStruxure

Because right-sized data center infrastructure is good business strategy

1

2

3

4 5

Cooling. Rack-, row-, and room-based cool-ing options, including new overhead InRow™ cooling units, for greater efficiency

Management. End-to-end monitoring and management software for greater efficiency and availability

Physical Security. A single-seat view for monitoring and surveillance across the facility.

Power. Modular power distribution and parallelling capabilities on UPS for loads from 10 kW to 2 MW

Rack Systems. Any-IT vendor-compatible rack enclosures and accessories for high densities

1

2

3

4

5

The only right-sized data center architecture engineered as a system

Introducing Next Generation InfraStruxure Whether you just acquired a new company or must increase its ever-expanding customer or inventory database capacity, you’re most likely facing pressing demands on your company’s IT infrastructure. Your existing data center infrastructure may not be able to handle these up-to-the-minute changes. That’s where APC by Schneider Electric™ steps in with its proven high-performance, scalable, data center infrastructure. As the industry’s one-of-a-kind, truly modular, adaptable, and “on-demand” data center system, only InfraStruxure™ ensures that your data center can adapt effectively, efficiently, and, perhaps most important, quickly, to business changes. InfraStruxure data centers mean business We say that InfraStruxure data centers mean business. But what does that mean to you? The answer is simple. A data center means business when it is always available, 24/7/365, and performs at the highest level at all times, is able to grow at the breakneck speed of business, lets you add capacity without waiting on logistical delays (e.g., work orders), enables IT and facilities to keep pace with the business in a synchronized way, continues to achieve greater and greater energy efficiency—from planning through operations, is able to grow with the business itself, and supports—instead of hinders—business. The triple promise of InfraStruxure deployment InfraStruxure fulfills our triple promise of superior quality, which ensures highest availability; speed, which ensures easy and quick alignment of IT to business needs; and cost savings based on energy efficiency. What better way to “mean business” than to enable quality, speed, and cost savings—simultaneously?

Discover which physical infrastructure management tools you need to operate your data center… download White Paper #104, “Classification of Data Center Operations Technology (OT) Management Tools,” today!Visit www.apc.com/promo Key Code b542v • Call 888-289-APCC x9809 • Fax 401-788-2797

©2011 Schneider Electric. All Rights Reserved. Schneider Electric, APC, InRow, and InfraStruxure are trademarks owned by Schneider Electric Industries SAS or its affiliated companies.e-mail: esupport@apc.com • 132 Fairgrounds Road, West Kingston, RI 02892 USA • 998-5038_US

Classification of Data Center Operations Technology (OT) Management Tools

Contents 1

2

7

7

9

10

> Executive summary

THE DATA CENTER POWERTRAINTRANSFERRING POWER FROM the utility company to IT equipment involves severalsteps all data center staff should know. From the utility grid, the power goesthrough switchgear to the automatic transfer switch (ATS). In a power emer-gency, the ATS changes loads from the utility to fuel-based generators. Thesegenerators keep the data center running until the main power returnsor the data center backs up to a disaster recovery site.The wait for obtaining a 2 MW diesel generator—the most popular for data

centers—can often be longer than a year. Uninterruptible power supply (UPS)units act as immediate backup power sources for the IT load. If the utilitypower feed is on when a current goes through the UPS input, it powers ITequipment. If the power goes out, the load is transferred to the UPS until thegenerators can get up and running—usually a few seconds.Traditionally, UPS units have large cabinets of batteries. The amount of time

the UPS unit can handle the load depends on the size of the load and howmanybatteries are available for backup. Some data centers now also use flywheels inplace of or in conjunction with batteries to run UPS units.The next stop in the data center power train for the IT load is the power dis-

tribution units. These units are typically installed in server cabinets and act aspower strips for servers, storage and networking gear. While cheaper modelsdo indeed resemble power strips, more costly ones can provide power con-sumption data for either the entire unit or individual receptacles back to datacenter staff via aWeb browser. �

squared off against AC power backersGeorgeWestinghouse and NikolaTesla. AC power won largely becauseit could be transferredmore easily overlong distances. The battle has sincebeen dubbed the “war of currents.”In 2006, the Lawrence BerkeleyNational Laboratory performed ademonstration project in California tocompare the use of AC power versusDC power in the data center. The lab-oratory claimed data centers couldsee up to 20% savings using DC

power. How is this possible? Thedifference between the two currentsis in the conversions.In a typical AC-powered data cen-ter, electricity comes from the utilityas AC power at either 480 V or 600V. By the time it reaches the server’scomponents, it’s usually running at 12V of DC power. In between the mod-erate-voltage AC and low-voltage DClevels, there are multiple points wherethe power is converted back and forthfrom AC to DC power.

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 7

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

Proponents of DC power claim thateach of those conversions introducelosses that affect efficiency. In a DC-powered data center, there is just oneconversion at the beginning—AC toDC. But this has its drawbacks.DC-powered IT equipment can bemore difficult to find on the market.Transferring AC power over long dis-tances is easier; doing so with DCpower requires much thicker wires.This is why AC power eventually wonthe “war of currents.”There is an ongoing study thatquestions whether DC power pres-ents any savings at all. The GreenGrid, a Beaverton, Ore.-based non-profit focused on data center efficien-cy, released a white paper on differentdata center power distribution config-urations. In part, the study concludedthat there were no major differencesbetween AC- and DC-powered datacenters.“When considering efficiency, thereis no single AC or DC configurationthat provides superior efficiency at allloads or in all situations,” the reportstated. “The efficiency differencesamong the contemporary implemen-tations are relatively minor. All com-ponents and configurations haveundergone great efficiency improve-ments in the last decade and will likelycontinue to improve.”

REPLACING BATTERIESWITH FLYWHEELSSome data centers have looked atusing flywheels in place of or in con-

junction with batteries to run theiruninterruptible power supply (UPS)units. Generating electricity using aflywheel is not new; the technologyhas been around for decades.The main power source for the datacenter jumpstarts the flywheel andstarts it spinning. This builds kineticenergy based on the mass of the fly-wheel and the speed at which itrotates, which can be as fast as54,000 rotations per minute. Whenthe power goes out—even if it’s justfor a second or two—the flywheelreleases its built-up kinetic energyinto the data center until primarypower returns or a backup generatorstarts.The chief criticism of using fly-wheels is that they don’t providepower to the data center for as longas batteries can. While flywheels canprovide ride-through backup powerfor about 15 seconds, batteries canprovide power for at least 15 minutes,depending on the length of the bat-tery string.

MEASURING PROGRESSWhile there are several establishedmethods to improve data center ener-gy efficiency, none of themmatterunless you can properly measureimprovements. Being able to provethat a data center is more efficientand how that efficiency equates tolower energy bills will help justifyfuture efficiency projects.The Green Grid created the basicdata center energy efficiency metric

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 8

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

EPA DRAFTS DATA CENTERFACILITY SPECS

THE FEDERAL GOVERNMENT is taking a closer look at data centers and their comput-ing equipment. Interest began in 2007 when Lawrence Berkeley National Labo-ratory scientist Jonathan Koomey released a report on global data center energyconsumption. Koomey found that data centers consume about 1.2% of all elec-tricity in the U.S. The Environmental Protection Agency (EPA) became inter-ested in finding ways to make data centers more efficient. After developing anEnergy Star specification for servers, the EPA has developed a loose specifica-tion for measuring the energy efficiency of the actual data center facility.Generally, the Energy Star spec leans heavily on power usage effectiveness

(PUE), a metric that compares the power entering a facility to the power thatservers consume. The closer these numbers are to one another, the more en-ergy efficient the data center is. But some experts question the validity of de-vising an Energy Star specification around PUE alone since the metric doesn’taccount for server functions.If IT equipment sits idle and remains unused, the data center wastes en-

ergy—no matter how efficient the PUE says it is. It’s likely that the EPA willeventually incorporate some measure of productivity as well as a way to meas-ure energy efficiency into this spec.Experts note the potential for different data center categories, but would

prefer that those categories are defined by the type of work the data centerperforms—not necessarily what industry they are in. But it’s going to take sometime to hash it out; further development and refinement of the data centerEnergy Star spec is ongoing. �

called power usage effectiveness(PUE). The metric compares theamount of power that enters a facilityto the power that goes to IT equip-ment. Dividing the former by the lat-ter yields a ratio that improves as itgets closer to 1.According to The Uptime Institute,a data center consulting group head-quartered in Santa Fe, N.M., the aver-

age PUE as of May of 2011 was 1.8.Therefore, for every 1.8W entering adata center, only 1 W is used to powerIT equipment. The remaining 0.8Ware used to cool the facility and arelost in wasted heat through transfersbetween equipment.Data centers can also use the PUE’sreciprocal—data center infrastructureefficiency (DCIE)—which is expressed

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 9

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 10

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

as a percentage that shows improve-ment as it nears 100%. Measure-ments for this number should betaken in two locations. Energy useshould be measured at or near thefacility’s utility meter.

If the data center is in a mixed-use facility or office building, take ameasurement only at the meter that’spowering the data center. If it is noton a separate utility meter, estimatethe amount of power that the non-data center portion of the buildingconsumes and remove it from theequation.IT equipment load should be meas-ured after power conversion, switch-ing and conditioning is completed.According to The Green Grid, themost useful measurement point is atthe output of the power distributionunits (PDUs), but it can also be takenat the output of the UPS. If you takethe measurement at the output of theUPS, your PUE should be lower, andtherefore less accurate, than if youtake it at the output of the PDUs,

because there are fewer power con-versions. This measurement shouldrepresent the total power delivered tothe server racks in the data center.It’s important to keep the PUE met-ric in perspective. Some data centercolocation companies release project-ed PUE numbers; however, The GreenGrid and other data center expertsrecommend keeping PUE numbers in-ternal since they’re not all-encompass-ing. PUE doesn’t take into accounthow productive IT equipment is.

STANDARDS ANDLEED CERTIFICATIONAfter years of research, the EPA hasdeveloped an Energy Star specifica-tion for data center facilities, whichleans heavily on PUE. Data centerscan also obtain a Leadership in Ener-gy and Environmental Design (LEED)certification for energy efficiency.The U.S. Green Building Councildesigned the LEED certification forany building, but data centers canalso participate. Some data centersseek LEED certification for both theenergy efficiency aspects as well asits possible public relations benefits.The certification, however, does haveits drawbacks since it doesn’t focusspecifically on data centers, which aremore power hungry than traditionaloffice environments.Despite that, companies such asDigital Realty Trust, a large data cen-ter realty company, have obtainedLEED certification for their datacenters.

It's important tokeep the PUEmetricin perspective. TheGreen Grid and otherexperts recommendkeeping PUEnumbers internal.

“LEED is a very holistic view ofwhat happens to a building,” said JimSmith, Digital’s VP of engineering.“Is LEED 100% appropriate for datacenters? Yes. Is it the best metric todecide whether a data center is goodor bad? Probably not.”So what is the future of data centerenergy efficiency? The industry iscontinuing to define and refine energyefficiency in data center design. And,over time, data center managers willadopt the goals of energy efficiencyas best practices—whether they wantto or not.Many large corporations now havecompany-wide policies for controllingenergy consumption, and data cen-ters are a large slice of that pie. Thefederal government is moving towardan energy policy that would rewardcompanies and people who saveenergy and limit carbon dioxide emis-sions as well as punish those whodon’t. Getting ahead of that curve willbring job security for data centermanagers. �

ABOUT THE AUTHORS

Mark Fontecchio is currently site editor forSearchOracle.com. Before heading SearchOra-cle.com, Mark spent four years reporting and writ-ing about data centers and servers at SearchData-Center.com.

Stephen J. Bigelow, senior technology writer, hasmore than 20 years of technical writing experiencein the PC and technology industry. He has writtenhundreds of articles and more than 15 featurebooks on computer troubleshooting. Contact himat sbigelow@techtarget.com.

REVISED: Data Center Facilities IT Handbook, 2009.

DATA CENTER FACILITIES DESIGN FOR IT • SEARCHDATACENTER.COM 11

CONSERVATION

STARTS WITH

UTILITY PROVIDERS

SAVING POWER

AT THE SERVER

THE DEBATE OF

AC VS. DC POWER

USING FLYWHEELS

INSTEAD OF

BATTERIES

MEASURING

PROGRESS

CHAPTER 3

SENIOR TECHNOLOGY EDITOR

Stephen J. Bigelow

SITE EDITOR

Nicole Harding

SENIOR MANAGING EDITORS

Michelle BoisvertLauren Horwitz

MANAGING EDITOR

Christine Casatelli

ASSOCIATE MANAGING EDITORS

Jeannette BeltranEugene DemaitreMartha Moore

DIRECTOR OF ONLINE DESIGN

Linda Koury

EDITORIAL DIRECTOR

Cathleen Gagne

PUBLISHER

Marc Laplante

TechTarget275 Grove Street

Newton, MA 02466www.techtarget.com

RESOURCES FROM OUR SPONSOR

See ad page 6

• APC Trade Off Tools

• Interactive InfraStruXure Tour

• APC.com

About APC by Schneider Electric:APC by Schneider Electric, a global leader in critical power and cooling services, provides industryleading product, software and systems for home, office, data center and factory floor applications.APC delivers pioneering, energy efficient solutions for critical technology and industrialapplications.