1 Data Centers for the 21st Century Working with Industry to Improve Energy Efficiency in Data Centers Dale Sartor, P.E. LBNL Applications Team June 7, 2007 2 Crying Uncle! 3 LBNL Computer Systems Power 4 Why Data Centers 5 Highly energy-intensive and rapidly growing Consume 10 to 100 times more energy per square foot than a typical office building A single rack of servers can be 20 kW $17k per year per rack (at $.10/kWh) Hundreds of racks per center can have significant impact on electricity supply and distribution Used about 45 billion kWh in 2005about 1.2% of all retail U.S. electricity sales. At current growth, power requirements could double in less than 10 years. Why Data Centers 6 Typical Data Center Energy End Use Server Load /Computing Operations Cooling Equipment Power Conversions & Distribution 100 Units 33 Units Delivered 35 Units Where Does it Go? Data Center Energy Use 7 Save 20 billion kWh per year by 2015 Worth $2 billion and roughly equal to annual electricity use in 1.8 million American homes Potentially defer need to build 2,300 MW of new generating capacity and avoid 3.4 million metric tons of carbon emissions (like taking 675,000 cars off the road) Extend life and capacity of existing data center infrastructures Potential Benefits of Improved Data Center Energy Efficiency 8 Courtesy of Michael Patterson, Intel Corporation Overall power use in Data Centers 9 Computational Energy Efficiency While my presentation focuses on the data center infrastructure, significant opportunities are available to increase computational energy efficiencies: Improve Utilization of Resources: Consolidation and virtualization Grid computing Enable and improve power management (dynamic loading) Improve software efficiency (and software instructions to hardware) Any reduction in IT equipment energy use has a corresponding savings in infrastructure Potential first cost savings are often missed Very challenging to set standards 10 Computational Energy Efficiency Standards ENERGY STAR for Servers EPA will release strawman proposal this year EPA considering power supply efficiency & system energy efficiency performance Need support from industry stakeholders EPA also interested in other IT equipment -- storage, networking, etc. Efforts to Develop Server Performance Benchmark: SPEC No metric available to compare server energy efficiency SPEC Committee developing energy efficiency benchmark Working prototype developed, more info on progress: 11 With funding from PG&E and CEC, LBNL conducted benchmark studies of 22 data centers: Found wide variation in performance Identified best practices CEC and PG&E continue R&D and demonstrations New DOE program will expand current knowledge base Efficient Data Centers - Building Knowledge Base 12 Benchmarking How do I Stack Up? 13 Energy Intensity Growth Growing but not a good performance metric 14 Data Center Cooling and Power Conversion Performance Varies Typical PracticeBetter Practice Server Load /Computing Operations Cooling & Power Conversions Server Load /Computing Operations Cooling & Power Conversions Infrastructure Efficiency 15 Best Practices: Lessons from Benchmarking Total Data Center Power/ IT power 16 The ratio of IT equipment to the total power is an indicator of relative overall efficiency. Examination of individual systems and components in the centers that performed well helped to identify best practices: Air management Right-sizing Central plant optimization Efficient air handling Free cooling Humidity control Liquid cooling Improving power chain UPSs and equipment power supplies On-site generation Design and M&O processes Using benchmark results to find best practices 17 Enforce hot aisle/cold aisle arrangement Eliminate bypasses and short circuits Reduce air flow restrictions Proper floor tile arrangement Proper locations of air handlers Optimize Air Management 18 Data Center HVAC often under-loaded Ultimate load uncertain Design for efficient part-load operation modularity variable-speed fans, pumps, compressors Upsize fixed elements (pipes, ducts) Upsize cooling towers Right-Size the Design 19 Have one (vs. distributed cooling) Medium temperature chilled water Aggressive temperature resets Primary-only CHW with variable flow Thermal storage Monitor plant efficiency Optimize the Central Plant 20 Fewer, larger fans and motors VAV easier Central controls eliminate fighting Outside-air economizers easier Design for Efficient Central Air Handling 21 Outside-Air Economizers Can be very effective (24/7 load) Controversial re: contamination Must consider humidity Water-side Economizers No contamination question Can be in series with chiller Use Free Cooling 22 Eliminate inadvertent dehumidification Computer load is sensible only Medium-temperature chilled water Humidity control at make-up air handler only Use ASHRAE allowable RH and temperature Eliminate equipment fighting Coordinated controls on distributed AHUs Improve Humidity Control 23 Water is 3500x more effective than air on a volume basis Cooling distribution is more energy efficient Water-cooled racks available now; liquid-cooled computers are coming Heat rejection at a higher temperature Chiller plant more efficient Water-side economizer more effective Use Liquid Cooling of Racks and Computers 24 25 Improving the Power Chain Increase distribution voltage DC distribution Improve equipment power supplies Improve UPS 26 Power supplies in IT equipment generate much of the heat. Highly efficient supplies can reduce IT equipment load by 15% or more. UPS efficiency also varies a lot. (Do they need the same environment as the IT equipment or can their conditions be relaxed?) Specify Efficient Power Supplies and UPSs 27 Can use waste heat for cooling sorption cycles typically required for cost effectiveness Swaps role with utility for back-up Air-quality issues Sell-back options complex controls required Consider On-Site Generation 28 Get IT and Facilities people to work together Use life-cycle total cost of ownership analysis Document design intent Introduce energy optimization early Benchmark existing facilities Re-commission as a regular part of maintenance Improve Design and Operations Processes 29 Top best practices identified through benchmarking 30 Design Guides were developed based upon the observed best practices Guides are available through PG&E and LBNL websites Self benchmarking protocol also available Design guidelines are available 31 A research roadmap was developed for the California Energy Commission and outlined key areas for energy efficiency research, development, and demonstration What is in the R&D Pipeline 32 Air management improvements (PG&E) Dramatic savings in fan energy Improved cooling efficiency and output Significant reduction in hot spots Outside air economizers (PG&E) Field test of air quality Contamination and humidity control concerns generally unfounded DC powering (CEC-PIER) Facility level savings 10 to 20% LBNL Data Center Demonstrations 33 Energy efficiency can slow expected growth in electricity use: Current trends will lead to ~10% reduction Simple management improvements can reduce consumption by an additional 20% Best practices can lead to ~50% reduction Impact of Energy Efficiency 34 Comparison of Projected Electricity Use, All Scenarios, 2007 to 2011 Annual Energy Use (Billion kWh/year) Baseline Business as usual Current trends Improved operational management Best practice State of the art U.S. Opportunity Potential 35 CEC is providing the technical foundation to improve data centers PG&E is the leading U.S. utility providing support to data centers DOE is initiating a national program to provide assistance in identifying and evaluating savings opportunities Where can I get help? 36 Program Objectives Provide systems approach Build tools, expertise, and strategies Raise awareness and recognize industry leaders Program Strategies (2007) Build on successful Save Energy Now model With industry input, develop appropriate tools, training, and qualified experts Implement assessment program (solicitation in Fall) Screen for industrial demonstrations Federal government procurement specifications DOE Data Center Program www1.eere.energy.gov/industry/saveenergynow/partnering_data_centers.html 37 Sponsors: California Energy Commission (CEC)U.S. Department of Energy (DOE)U.S. Environmental Protection AgencyPacific Gas and Electric Company (PG&E)Stakeholders: Industry Organizations e.g., Green Grid, ASHRAE, AFCOM, 7x24, SVLG Equipment suppliers Research organizations Consultants Sponsors and Stakeholders 38Good starting point for those seeking efficiency measures Best Practices Case Studies Design Guidance Self-benchmarking Guide Benchmark data Other Reports (demonstrations) Web-based Resources 39 Dale Sartor, P.E. Lawrence Berkeley National Laboratory Applications Team MS University of California Berkeley, CA (510) Contact Information: