Special Studies on Power & Cooling Practices, Requirements, Issues and Concerns for Data Centers...
-
Upload
julianna-horton -
Category
Documents
-
view
216 -
download
1
Transcript of Special Studies on Power & Cooling Practices, Requirements, Issues and Concerns for Data Centers...
Special Studies on Power & Cooling Practices, Requirements, Issues and Concerns for Data Centers
Roger A Panton, Avetec Executive Director DICE
HPC User Forum Sept [email protected]
Background: The Growing Issue
Power & cooling demand is a world-wide major concern Exponential growth in HPC system size and performance
have increased energy requirements Estimates are that the energy use has increased nearly
100% for data centers over the last six years If data center energy usage continues at the current rate,
the nation will need to build two large power plants per year to meet demand
Energy prices have risen substantially Aging data center infrastructure is an issue In this “perfect storm” the challenge is how to increase
energy efficiency without compromising performance improvement
Two Related Studies on Power & Cooling
Study #1: Power & Cooling Practices and Planning at HPC Data Centers Status: Completed late Spring 2009 REPORT AVAILABLE TO ANYONE WHO IS INTERESTED!
WWW.DICEPROGRAM.ORG Study #2: Power & Cooling Solutions for Enterprise
and HPC Data Centers Status: Complete report by end of September 2009
Both studies are being conducted via a new Avetec-IDC partnership
Phase 1 Study:
Power & Cooling Practices and Planning at
HPC Data Centers
Background and Objective
Avetec was tasked to evaluate power and cooling planning, constraints, requirements and how HPC data centers are coping Avetec contracted with IDC to survey HPC data centers
and vendors to assess: Current power and cooling situation Planning in place to address requirements Solutions currently being used Forecasted solutions over the next three to five
years Respondents included HPC data centers and vendors
including HPC systems, equipment and facilities
Survey Sample and Methodology
Survey includes 41 respondents: 28 HPC data centers 13 vendors of HPC products and services Response rate was approximately 40% of sites contacted
Respondents were from the US, Europe and Asia HPC data centers were selected from the “Top500” list Centers selected fell between number 50 and 250
Interviews were conducted by phone or in person Respondents also had the option to complete the survey
on their own
Sample Respondent Mix
HPC Data Centers
HPC System and Service Vendors
F I G U R E 1
Mix Of User Si tes Surveyed
Academic (42.9%)
Government (28.6%)
Industrial (28.6%)
N = 28
Source: IDC 2009
Government 28.5%
Industry 28.6%
Academic 42.9%
F I G U R E 2
Mix Of Vendor Sites Surveyed
Vendor of data center power and cooling, products
and solutions (30.8%)
Provider of data center
construction/services (15.4%)
Vendor of HPC systems (53.8%)
N = 13
Source: IDC 2009
Vendor Power & Cooling 30.8%
Vendor HPC Systems
53.8%
Construction / Service
15.4%
Study #1Highlights
Current HPC Data Center View
Over 96% considered “green” design important Majority of sites expect power and cooling to
impact future HPC center planning Majority of respondents have studied or
implemented “greener” operations Most centers have used software models to
analyze heat flow and/or power consumption Approximately half of the centers paid for power
and cooling out of their budgets
General Findings
HPC data centers’ average per site: Available floor space over 26,000 ft2
Used floor space about 17,000 ft2 (63% of available space) Cooling capacity 22.7 million BTUs or 1,839 tons Annual power consumption 6.356 MW
HPC data center cost Annual power cost was $2.9 million or $456 per KW Ten sites provided the percentage of their budget spent
on power—average was 23% Two-thirds of the sites had budget for power and cooling
upgrades Average amount is $6.87 million
Challenges and Expansion Constraints
Majority of centers are starting to consider power and cooling efficiency equal to or more important than HPC computing performance
Power and cooling issues are becoming the biggest barriers to expansion and upgrades
Most HPC vendors are starting to see power and cooling as a brake on performance
Power and cooling costs are becoming a key factor in upgrade decisions
Current Approaches Power and cooling are becoming a key factor in
upgrade decisions Majority of centers have accomplished an air flow
analysis to improve air cooling efficiency Majority of centers have analyzed power consumption
Use of chilled water for cooling is increasing The power and cooling issues are being discussed
across the HPC community Other approaches include:
Hot/cold aisle containment Re-directing/reusing excess heat Facility modifications/equipment refurbishment Raising temperatures
DRILLING DEEPER INTO THIS IN STUDY #2
Future Solutions and Technologies
Approximately two-thirds of centers plan to expand or build new data centers
About half of the data centers have or are planning to distribute HPC resources
Liquid cooling is most frequently cited future alternative being considered (centers & vendors)
Other potential solutions included combination of air/water cooling In-row cooling Use outside air ambient
DRILLING DEEPER INTO THIS IN STUDY #2
Specific Questions
Belief in Game Changing Technologies
Question: Do you anticipate any changing cooling technologies that will significantly change the way HPC systems and data center are cooled in the next 5 years?
Response % Government Industry Academia All Sites
Yes 62.5% 50.0% 8.3% 35.7%
No 25.0% 50.0% 58.3% 46.4%
Not certain 12.5% 0.0% 33.3% 17.9%
Total 100% 100% 100% 100%
Government respondents more optimistic about game changing
technologies
Question: How do your power and cooling costs divide among your HPC compute, storage, and visualization sub-systems?
Power and Cooling by Sub-systems
Response % Government Industry Academia All Sites
% Compute 92.6% 81.7% 90.1% 89.7%
% Storage 4.9% 18.3% 8.2% 8.6%
% Visualization 2.1% 0.0% 1.1% 1.3%
Don’t know or not sure 0.4% 0.0% 0.6% 0.4%
Storage costs greater among non-government sites
Expected Changes by Sub-systems
Question: In which of the three basic HPC data center sub-systems defined do you expect power and cooling to grow the most rapidly in the next five years?
Response % Government Industry Academia All Sites
% Compute 87.5% 25.0% 75.0% 64.3%
% Storage 12.5% 50.0% 0.0% 17.9%
% Visualization 0.0% 12.5% 0.0% 3.6%
Don’t know or not sure 0.0% 12.5% 25.0% 14.3%
Total 100% 100% 100% 100%
Government & academic centers forecast most growth in compute whereas industry sees most growth in storage
Guidance for Users
Improving a site's power and cooling profile is a multi-dimensional process of stepwise refinement that begin at any time
The most efficient and progressive HPC data centers surveyed are already using metrics to measure their performance in power and cooling
A handful of power and cooling efficiency drivers will govern how HPC data centers take to greater efficiency
HPC system upgrades will stimulate demand for detailed efficiency analysis services
Guidance for Vendors
The recession will elevate the importance of the "green" design features of HPC systems, especially in Europe
The next round of HPC system upgrades will stimulate demand for HPC data center efficiency analysis services
Future multi-Pflop HPC data centers will require as much power as a small to mid-sized city and stimulate greater public policy concerns
Public PolicyImplications
from Study #1
Public Policy Implications for Discussion
Should the community take a proactive position through collaborative discussions and then recommend a set of Public Policies?
To start the discussions should: The Federal Government establish a timeframe and fund the
following research areas: Invest to maintain the current performance growth in
HPC? Invest in new cooling technologies to improve
efficiencies? Invest in lower power, higher performance processor(s)? Invest in new material research for chips?
HPC data centers will have to become more accountable for power and cooling consumption
Phase 2 Study:
Study of Power & Cooling of
Enterprise Data Centers
Study #2 Objectives
"Potential Solutions to Address and Improve Power and Cooling for HPC Servers” Study to explore data center power and cooling
technologies: Practices, and future plans/solutions Exploring energy efficiency management tools—being used
today and emerging solutions. The study will focus on new products and solutions
Near term horizon, that can reasonably expect to be able within the next 1 to 2 years.
Study will include surveys of providers of solutions and HPC vendors, in addition it will include a "testing" of how likely each solution will actually become available in the market in the near term.
Study #2 Objective (cont’d)
The study intends to leverage the recent IDC/Avetec study Identified the key strengths and shortcomings of
current cooling approaches Identify those that are under research by suppliers
As an important side benefit, the study will also document user pain points and concerns about future HPC data center cooling solutions.
Potential Impact and Outcomes
Create a policy document that will assist US government and manufacturers to develop new products and perhaps laws to enhance overarching computer and communications facilities in conjunction with improvements in power and cooling demands.
Compare and contrast solutions and approaches among the communities. Approaches that work across communities have strong potential to be leveraged for economies-of-scale that can lead to lower costs.
Recommend a set of test tools (hardware and software) and a methodology to assist centers in self-assessment.
Future Solutions and Technologies
Potential technology solutions Advanced power management in hardware Dynamic network power management New data storage technologies Hardened equipment Novel computing architectures
Game-Changing Technologies All-optical networks Superconducting components Nano-electronic circuitry Eliminate voltage conversion steps Liquid cooling of hardware components Transition to DC operations
Conclusion and Recommendations
Data Centers: Proactively work with vendors, researchers and the HPC
community to identify and pursue breakthrough power and cooling technologies.
If not currently monitoring power and cooling consumption and cost, implement an approach and methodology to measure and track consumption.
Identify and implement methods to reduce consumption. Install new air conditioning and power systems that reduce
power consumption. Establish comprehensive performance metrics to track and
report results of new initiatives. Investigate enhancing cooling methodologies through the
use of liquid technology.
Conclusions and Recommendations
Vendors: Develop technologies that will drive down power and
cooling demands by using technologies that dynamically power up processors on demand.
Technology vendors need to design their systems requirements to address end-to-end data center infrastructure in order to develop the most efficient center possible.