Ultra-Efficient HPC Data Centre - Gary Bernstein, McGill University
-
Upload
cybera-inc -
Category
Technology
-
view
1.308 -
download
0
Transcript of Ultra-Efficient HPC Data Centre - Gary Bernstein, McGill University
10/8/10 1
Natural Low Energy Cooling Conceptual Design
Ultra Efficient HPC Data Center
Project Funding
Canada's Advanced Research and Innovation Network (CANARIE)
Canada California Strategic Innovation Partnership (CCSIP) • ISTP Canada
• University of California
• McGill University
Site Selection
Three candidate locations in Quebec • McDonald Campus of McGill University in St. Anne de Bellevue
• Campus of the Institut de recherche d’Hydro-Quebec (IREQ) in Varennes
• IREQ campus in Shawinigan
McDonald Campus of McGill University selected as site for project
All enjoy Cold climate
Renewable energy resource (hydroelectric)
Inexpensive electricity
The System Approach: An Overview
Goals: Most Efficient Class One Data Center
Climate Evaluation
Define Loads and How to Best Serve Them • Water cooled equipment
• Medium temperature chilled water (65F, 75F)
Optimize Heat Rejection for Climate and Loads • Evaporative free cooling – Primary cooling
• Seasonal ice storage – Top up cooling
Backup Approach
Results
Goals
Provide ASHRAE TC9.9 Class 1 Datacenter
No compressor based cooling
Lower construction cost
Lower operating cost
Best efficiency
Environmentally friendly
Construction materials
Recycle heat, water
$0
$1,000,000
$2,000,000
$3,000,000
$4,000,000
$5,000,000
$6,000,000
$7,000,000
$8,000,000
$9,000,000
$10,000,000
San Diego (1.35 PUE) at $0.09/kWh
Montreal (1.06 PUE) at $0.05/kWh
$5 Million/yr Annual Savings Target
Proposed Annual Electrical Costs Comparison
Aerial Perspective – “Farm” at McDonald Campus
Stormwater Detension
Office, Shipping/ Receiving
20,000 SF Phase 1 8 MW IT Load
Electrical Infrastructure
Cooling Ice Pond
Cooling Towers and Mechanical Infrastructure
VA Hospital
Fuel Tanks
Climate: Free Cooling Analysis with 65F CHWS
0.000
0.002
0.004
0.006
0.008
0.010
0.012
0.014
0.016
0.018
0.020
0.022
0.024
0.026
0.028
0.030
-30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Hum
idity
Rat
io (l
bs H
2O p
er lb
s dr
y ai
r)
Dry Bulb Temperature (F)
Data Source: Government of Canada - National Climate Data & Information Archive
Data Set: WMO #71627, Montreal/Pierre Elliott Trudeau Airport, Typical Year
Elevation: 118 feet
Air Pressure: 14.633224 psia
Full Free Cooling
7446 hrs/yr
Partial Free Cooling 1234 hrs/yr
Auxillary Cooling 80 hrs/yr
Climate: Free Cooling Analysis With 75F CHWS
0.000
0.002
0.004
0.006
0.008
0.010
0.012
0.014
0.016
0.018
0.020
0.022
0.024
0.026
0.028
0.030
-30 -25 -20 -15 -10 -5 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Hum
idity
Rat
io (l
bs H
2O p
er lb
s dr
y ai
r)
Dry Bulb Temperature (F)
Data Source: Government of Canada - National Climate Data & Information Archive
Data Set: Montreal/Pierre Elliott Trudeau Intl Airport, Typical Year
Elevation: 118 feet
Air Pressure: 14.633224 psia
Full Free Cooling
8646 hrs/yr
Partial Free Cooling 114 hrs/yr
Auxillary Cooling 0 hrs/yr
Load Strategies
Climate analysis shows higher temperature chilled water offers many more hours of free cooling
Highly concentrated heat loads • A single high density rack
can put off as much waste heat as a VW Beetle (40kW)
• Air exiting racks typically exceeds 90F
Load Heat Collection Strategies
Higher temperature Chilled Water Supply (CHWS) offers many more hours of free cooling: Design to use 75F and 65F CHWS • Direct water based cooling most efficient
• Hot aisle / cold aisle for minority of load
Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling
Primary Cooling Strategies: Medium Temp. Cooling Water and Free Cooling
Design to cool with 65F/18C and 75F/24C water • 90% of 65F load served with cooling tower provided free cooling;
99.3% of 75F load
• 590,000 ton-hrs (2,100 MWh) Top up Cooling Required
Supplemental Cooling: Seasonal Ice Storage Slush Pond System
Fill in winter with plowed snow collection
Melt water cools data center
Slush Pond
Paved collection basin, 75,000 ft3 (2,100 m3) Drive-in slope on one side for plow loading
Lightweight, waterproof insulating cover or roof to protect from warm rains
Extensive drain system to collect meltwater
Berms for sides, or dig into ground
Sundsvall, Sweden, Snow Storage - Empty
Sundsvall, Sweden, Snow Storage - Full
Snow dump overruns in Montreal, 2008-09
Slush Pond System – Pumping and Filtration
Mature waste water handling technology Remove gravel, wood, grit from melt water
Remove oils and road chemicals prior to release as required
Filter
Select heat exchangers for highly corrosive fluid Maintain complete separation between pond water and
building loop water
Integrate settling tank to also serve as emergency storage
Key approaches
Keep storage pond simple Leverage local snow removal program if possible
Collect snow dumpage fees?
Provide appropriate maintenance Provide for pile grooming, drain clearing, filter cleaning, etc
Use in lieu of chillers to save cost Consider emergency chiller rental for backup
Design properly Simple concept but careful design required
Office Approaches
Much lower load
Design for comfort and optimal use of medium temperature water
Backup – Do Not Invest in Chillers 'Just in Case'!
Pay for it only
when (if) you
ever need it
Design for
portable
air-cooled
chillers
to connect in
an emergency
Results
Transformer Loss; 0.5%
UPS Loss; 0.6%
PDU Loss; 1.0%
Data Center Lights; 0.2%
Racks; 94.0%
Fans; 1.5%
CRAH Fans; 0.0%
Humidifier; 0.0%
CHW Plant; 2.1%
McGill-USCD HPC Data Center PUE Itemization
Power Usage Effectiveness (PUE) = Total Energy / Rack Energy = 1.06
Results
Supply Temperatures
Hours of Free Cooling / year PUE
Annual Energy Use Mechanical Cooling Needed Water Usage
Air Cooled
Water Cooled Energy Cost
( $0.058/kWh) Hours per Year
Additional Load at Extreme Weather (wetbulb = 68.7°F)
Evaporation + Carry Over
°C °F °C °F hrs/yr % of yr MWh/yr $ tons gallons
23.9 75.0 23.9 75.0 8,532 97% 1.06 74,567 $4,325,000 228 0 30,100,000