Post on 22-Feb-2016
description
Effect of Rack Server Population on Temperatures in Data Centers
CEETHERM Data Center LaboratoryG.W. Woodruff School of Mechanical Engineering
Georgia Institute of TechnologyAtlanta, GA 30332-0405
Yogendra.Joshi@me.gatech.edu404-385-2810
Rajat Ghosh, Vikneshan Sundaralingam, Yogendra Joshi
Collaborators: Pramod kumar, Vaibhav Arghode, Steven Isaacs.
Ghosh, Joshi: ITherm 2012 2
Outline
• Problem Statement.• Methodology
– Experiments.– Computational fluid dynamics (CFD) analysis.
• Results.• Conclusions.
May 30-June 1
Ghosh, Joshi: ITherm 2012 3
Problem Statements• Characterize rack-level air temperatures for
a full-capacity server rack.
• Estimate effect of rack server population on air temperatures in a data center.
• Estimate effect of server location on its CPU temperature and average fan speed.
May 30-June 1
Ghosh, Joshi: ITherm 2012 4
Experimental Setup
May 30-June 1
• Measurement Uncertainty:– Temperature:
– Flowrate:
– Length:
• A data center with 10 server racks arranged in a 5x2 architecture.
• Raised floor plenum supply and overhead plenum return.
• Alternating cold-aisle/ hot-aisle.• CRAC-1 is only active CRAC
–
00.2 .C
1 .mm
320 0.00944 / .CFM m s ( )
9300 .CFM
Ghosh, Joshi: ITherm 2012 5
Test Rack
May 30-June 1
• Server rack (consists of 42 1-U server and headnode) with heterogeneous heat load ranging from 240 W to 0 W.
Ghosh, Joshi: ITherm 2012 6
Containment System
May 30-June 1
• Isolate airflow into the test rack.
Cooling Air from
Plenum
Test RackHot Aisle Containment
Cold Aisle Containment
Perforated Tile
Hot Exhaust Air
Exhaust Plane
Ghosh, Joshi: ITherm 2012 7
Thermocouple Grid
May 30-June 1
600x
z
x
y
Steel Frame
Thermocouple
Tube
• Grid: 21 T-type copper-constantan thermocouples made from 28 gauge (0.9 mm diameter) wire.
• Response time: 28 ms• x-axis: Parallel to rack width.• y-axis: Parallel to tiles.• z-axis: parallel to rack height.
Ghosh, Joshi: ITherm 2012 8
Experimental Procedure1. Deploy the rack-level containment.2. Vary the server population in the test rack
(N=42, 32, 22,12) and measure air temperatures in cold and hot aisles.
3. Vary the location of a server stack and measure temperatures of CPUs and speeds of fans inside servers.
May 30-June 1
Ghosh, Joshi: ITherm 2012 9
Server Population as Parameter
May 30-June 1
Ghosh, Joshi: ITherm 2012 10
Varying Position of a Server Stack
May 30-June 1
Ghosh, Joshi: ITherm 2012 11
CFD Simulation
May 30-June 1
• x-dimension: 2.46 m• y-dimension: 0.60 m• Z-dimension: 1.95 m• 1-U server: 1m x 0.6 m x0.4 m• Server fan:
• Tile: Velocity inlet with 0.8 m/s to match 639 CFM (0.3 m3/s) supply
• Exhaust: Pressure outlet• Server inlet and outlet: Porous jump• Grid number: 1.4 millions for grid-
independent solution
3
2
( ) 0.187
6.76 38.48315.4
p v v
v v
Ghosh, Joshi: ITherm 2012 12
Transient CRAC Supply Air Temperature
May 30-June 1
• CRAC-1 has return air temperature control.
• Variable supply air temperature– Mean=12.2 0C. Std. Dev.=0.9 0C.
Ghosh, Joshi: ITherm 2012 13
Cold Aisle Temperature Variation
May 30-June 1
• With height in the cold aisle, average temperature varies irregularly.
• N=42
Ghosh, Joshi: ITherm 2012 14
Hot Aisle Temperature Variation
May 30-June 1
• With height in the hot aisle, average temperature varies irregularly.
• N=42
Ghosh, Joshi: ITherm 2012 15
CFD-predicted Airflow
May 30-June 1
• Recirculation in the airflow explains irregular pattern of air temperatures in the cold and hot aisles.
• N=42
Ghosh, Joshi: ITherm 2012 16
Temperature Difference Variation
May 30-June 1
• Temperature difference increases with height.
• N=42
Ghosh, Joshi: ITherm 2012 17
Average Temperature in Exhaust Plane
May 30-June 1
• Average temperature in the exhaust plane increases with server population.
Ghosh, Joshi: ITherm 2012 18
Effect of Server Population on Temperature Difference
May 30-June 1
• For all heights, temperature difference increases with server number.
Ghosh, Joshi: ITherm 2012 19
Effect of Containment
May 30-June 1
13 13.5 14 14.5 15 15.5 16 16.5 170
304.8
609.6
914
1219.2
1524
1828.8
2000
Average Temperature in Cold Aisle (0C)
Hei
ght a
long
the
Rac
k (m
m)
Without ContainmentWith Containment
• The containment system reduces average temperature in the cold aisle
- Blocks hot air recirculating from other parts of the room.
Ghosh, Joshi: ITherm 2012 20
Effect of Server Location
May 30-June 1
• Keeping server stack near the highest possible location is more energy-efficient practice in this case- Lowest CPU temperature.- Lowest average server fan speed.
7800 8000 8200 8400 8600 8800 9000 92000
44
89
133
178
222
267
311
356
400
445
489
533
600
Average Server Fan Speed (rpm)Lo
cal H
eigh
t alo
ng th
e Se
rver
Sta
ck (m
m)
TopMiddleBottom
18 20 22 24 26 28 30 320
44
89
133
178
222
267
311
356
400
445
489
533
600
Average CPU Temperature (0C)
Loca
l Hei
ght a
long
the
Serv
er S
tack
(mm
)
TopMiddleBottom
Ghosh, Joshi: ITherm 2012 21
Conclusion• Rack-level temperature field (average temperatures in cold and
hot aisles; and average temperature difference between cold and hot aisles) is characterized for a full capacity server rack (N=42). – Air recirculation thorough the void affects convective temperature field.
• Rack server population has a significant impact on air temperatures– Temperature difference across the rack increases with the server
population in the test rack.
• Recommended best practice for filling out a server stack in an empty stack– Sever stack should be placed at the highest possible location.
May 30-June 1
Ghosh, Joshi: ITherm 2012 22
Acknowledgement
The authors acknowledge support for this work from IBM Corporation, with Dr. Hendrik Hamann as the Technical Monitor. Acknowledgements are also due to the United States Department of Energy as the source of primary funds. Additional support from the National Science Foundation award CRI 0958514 enabled the acquisition of some of the test equipment utilized.
May 30-June 1
Ghosh, Joshi: ITherm 2012 23
Questions?
May 30-June 1