Implementing the Poughkeepsie Green Data Center

© 2009 IBM Corporation

Implementing the Poughkeepsie Green Data Center -- showcasing a dynamic infrastructure


AgendaWhy Energy Efficiency for Data Centers ?The Legacy Data CenterThe TransformationThe ToolsOptions and DecisionsEnergy Efficient TechnologiesBest Practices and Energy Management TechniquesInstrumentation ResultsNext StepsConclusion


My Summer Vacation 2009


Grinnell Glacier, Glacier National Park, 1940 vs. 2004

Source: http://www.livescience.com/php/multimedia/imagegallery/igviewer.php?imgid=626&gid=42&index=0


Science Panel Calls Global Warming ‘Unequivocal’2007


Where We WereComputer room air conditioner’s

(CRAC) high temperature alarm going off daily and no true hot/cold aisles

The raised floor was organized as a showroom, not optimized

Perforated tiles in the wrong location Rack arrangements allowed for hot/cold air to intermix Poor cable management

10 yr. old raised floor, approx 2,600 sq ft. in 2 rooms, supporting 9 organizations


How We Got StartedUnderstand what and where

the real problems were, not the symptoms

Identify options that will correct the problems and select the one that will give us the most flexibility for future enhancements, at an affordable price

Instrument, Interconnect, and Monitor Thermals, Power, and workload, where necessary

Step 1, MMT Study


“BEFORE” MMT scan at z=3.5 feet


Options and DecisionsExpand raised floor by 700 sq. ft.

Increase operational costs

Fixes the symptoms in the Showroom, but not the actual cooling problemsReplace Computer Room Air Conditioner in Showroom

Would require 3-4 week outage, holding up all mainframe benchmarks, Proof of Concepts and ongoing work

Transform existing Showroom to use new cooling technologiesOrganize the Showroom into 2 zones, air cooled and water cooled

Would allow for a transformation rather than new construction

Would use newer, lower operation cost cooling technologies

Would fix the base problems, not just the symptoms


Energy Efficient TechnologiesComputational Fluid Dynamics Modeling showed a water cooled zone and an air cooled zone would provide greater flexibility at a lower operational costNew Cooling Distribution Unit (CDU) would

Use less power to cool more equipmentLower the workload on the existing CRACFree up over 600 sq.ft. in the back room for more equipment

New Rear Door Heat Exchanger units (RDHx) would take the heat directly away from the racks


Water Cooled Zone and Best PracticesBest practices in rack layout and organization– No gaps between racks– No gaps within the rack– Proper cable management– Pillows stopping air flow in the back of

the racks– Underfloor cabling in the hot aisle

where possibleCDU manifold can handle 10 RDHx’s but we only used 7, allowing for 3 more racks to be added with no additional cooling required


The Poughkeepsie Railroad Bridge


Air Cooled ZoneLayout defined by CFD Modeling

Higher velocity perforated tiles than in water cooled zone

No gaps between frames where possible and a flexible baffle to fill empty space between frames


Energy Management TechniquesCapacity Planning

Virtualization and Consolidation–18 servers 2 Power Systems

–2 mainframes 1

Provisioning

Cloud Computing


Instrumentation

Thermal sensors SynapSensePower consumption Director/Active Energy ManagementNetCool Omnibus for Event managementIBM Tivoli Monitoring (ITM), Tivoli Enterprise Portal to integrate all monitored data into a single dashboardPower Utilization Efficiency (PUE) dynamic monitoring


Results

The Power Usage Effectiveness (PUE) was improved while increasing the IT load by 39 kilowatts (kW) in the same space footprint. IT power increased by 74%.IT power density increased by 37.5 watts/sq ft. The average rack inlet temp was reduced by 6.6 degrees Fahrenheit.The Computer Room Air Conditioner (CRAC) air heat load increased by 14.1 kW. 27.2 kW out of 39 kW were removed in select racks using RDHx units.

The return on investment (ROI) is less than a year.


Maslow’s Hierarchy of Needs

http://www.abraham-maslow.com/


Energy Efficiency Maturity Model

Ado

ptio

n St

ages

InitialEnergy is seen as a commodity and has no interaction with infrastructure choices. • No monitoring of energy consumption• Oversized or undersized cooling and powering infrastructures lead to PUE > 2.5, power outages and hot spots

Energy Monitoring & ManagementEnergy is not measured/monitored which leads to major cooling, powering problems• Monitoring of the energy consumption at the IT level.• IT resources can measure and report their own energy consumption.• Introduce server virtualization and probes and sensors.

Workload and Power OptimizationAssets management system is implemented but resources not well optimized• Real time PUE monitoring.• Introduce storage and network virtualization, free cooling

Dynamic Data CenterScalable and flexible Data Center – Energy Producer• Fully redundant cooling and powering infrastructure• Exploitation of a virtualized environment with application migration. Workload management.• District Heating

IT and Data Center Asset ManagementAll the Data Center assets (IT and Non-IT) are not managed.• Monitoring of the energy consumption at the IT and Non-IT level.• Non-IT resources can measure and report their own energy consumption.

Stage 1

Stage 2

Stage 3

Stage 4

Stage 5

Source: IBM PSSC WW Energy Efficiency Leadership Center


Next StepsInstrumentation Updates–Tivoli Business Service Manager (TBSM)

– Business related dashboard, solution oriented– A level above Infrastructure monitoring– Integrates NetCool and IBM Tivoli Monitoring (ITM) information

–Tivoli Maximo Asset Manager for Energy Optimization (MEO)– Thermal view with drilldown; Integrate MMT with MEO

–Tivoli Service Automation Manager (TSAM)– Dynamic provisioning; Cloud computing demo of self service portal

–Migration of Tivoli Data Warehouse onto a separate server, eliminating multiple TDW’s and creating a single one supporting multiple solutions

–Upgrade of Omnibus with new version that integrates into TBSM


Next Steps

Storage Consolidation–4 storage systems consolidated onto one DS8300 for savings in

space and energy

–Highlighting of virtualization capability

COLD ALARMS


What can you do in your data center ?

Implement thermal analysis tools to determine where your problems lie.Address your problems with techniques such as hot and cold aislemanagement, air circulation management, addition of baffles, perforated tile adjustments and the use of water cooling.Consider replacing lights and adding rear door heat exchangers to your systems. Consider energy management techniques such as virtualization andconsolidation.Utilize measuring and monitoring technology to track the overall health of the data center and to ensure that changes made within the infrastructure are achieving the desired results.


Conclusions

By implementing the Poughkeepsie Green Data Center, IBM was able to prepare for continued growth while adding cooling capacity.

Through the use of high efficiency water cooling technologies, data center best practices, and energy monitoring and management techniques, the 100 kW load required is now being satisfied in the existing data center area with more headroom for future growth.

Additionally, this upgrade was accomplished without any interruption to mission critical workload.

Results are significant.


References

[BRAD09] J. Brady, I. Chavis, M. Finlayson, M. Harkins, J. Mullin, J. Peet, S. Qualters, R. Samper, E. Stahl, “Implementing the Poughkeepsie Green Datacenter--showcasing a dynamic infrastructure ,” (2009). http://www.redbooks.ibm.com/redpapers/pdfs/redp4534.pdf

[EBBE08] M. Ebbers, A. Galea, M. Schaefer, M. Khiem, “The Green Data Center: Steps for the Journey,” (2008).http://www.redbooks.ibm.com/redpapers/pdfs/redp4413.pdf

The contributors to this paper and co-authors of the original Redpaper are John Brady, Ira Chavis, Matthew Finlayson, Michael Harkins, John P. Mullin, Julie Peet, Sheryl Qualters, Rodrigo Samper, Elisabeth Stahl.


IBM, the IBM logo, and ibm.com are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. These and other IBM trademarked terms are marked on their first occurrence in this information with the appropriate symbol (® or ™), indicating US registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at http://www.ibm.com/legal/copytrade.shtml The following terms are trademarks of the International Business Machines Corporation in the United States, other countries, or both:AIX®AS/400®BladeCenter®Cool Blue™DS4000®DS8000®Dynamic Infrastructure®Enterprise Storage Server®eServer™IBM Systems Director Active EnergyManager™IBM®Netcool®Netfinity®Power Systems™POWER6™PR/SM™Redbooks®Redbooks (logo) ®RS/6000®S/390®System i®System p®System Storage™System x®System z10™System z9®System z®Tivoli®TotalStorage®WebSphere®XIV®z/OS®z/VM®z9®The following terms are trademarks of other companies:Red Hat, RHCT, and the Shadowman logo are trademarks or registered trademarks of Red Hat, Inc. in the U.S. and other countries.VMware, the VMware "boxes" logo and design are registered trademarks or trademarks of VMware, Inc. in the United States and/or other jurisdictions.Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both.Intel, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both.Linux is a trademark of Linus Torvalds in the United States, other countries, or both.Other company, product, or service names may be trademarks or service marks of others.

Implementing the Poughkeepsie Green Data Center

Documents

Transcript of Implementing the Poughkeepsie Green Data Center