John DailyBill Best

Nate Slinin

Project Concept

• Our focus is centered around addressing the growing demands placed on the cooling infrastructure in a modern data center.

Facing the Truth

• Computational efficiencies over the last 40 years have doubled every 1.6 years providing a smooth logarithmic growth which is derived from Moore's Law

• However power use doubled between 2000 and 2005 when demand for computational power was a fraction of today’s requirements.

Current Method

• Centered around the Computer Room Air Handler.

Image: http://www.phoenixnap.com/mechanical-distribution.php

Server Fans CRAH External Heat Exchanger

Traditional Method Using CRAH system

Immersive Liquid Cooling System

External Heat ExchangerILC Exchanger

Benefits over Forced Convection

• Liquid has higher thermal conductivity than air.

• Better electrical insulator.• Its high heat capacity decreases risk of

hardware failure in case of cooling failures

Key ILC Design Challenges

• Non-Conductive Fluid

Non-Conductive Fluid

• The effectiveness of heat transfer and heat dissipation of the fluid will be determined by its thermal conductivity and the cooling requirements of the most critical portion of each server, the CPU

• To measure the amount of heat production we can transmit through the fluid we set the thermal source as the chip junction temperature, and each exchange point is modeled as resistance leading to the ambient liquid temperature. Since thermal resistance is additive we can add the various junctions together to get the total resistance and determine the ambient temperature of the fluid in its passive state. This is given by the equation R= ∆T/ Pth –Rsource; where R is the maximum thermal resistance, ∆T is the change in temperature (Tj-TL) and Pth is the thermal power in watts.

Key ILC Design Challenges

• Non-Conductive Fluid• Cooling Configuration

Cooling Configuration

• Replacing traditional data center 19inch CEA-310-E racks with a enclosure that will house standard 19 servers in 1U, 2U or 4U configurations.

• Heat exchange plates will be placed in between each server.

• Our rack equivalent enclosures will be placed horizontally as opposed to standing vertically and server will be inserted from the top at waist level.

Key ILC Design Challenges

• Non-Conductive Fluid• Cooling Configuration• Control System

PID CONTROLLER SERVER

-+T_D T_A

error

Control System Model

MICROCONTROLLER WATER PUMP

PID CONTROLLER

PID Equation

• PID = K_P*e+K_I*∫edt+K_D*(de/dt)

Constants

• K_p = 0.15• K_i = 0.05• K_d = 0.05

time T_D T_O e K_P K_P*e int(e) K_I K_I*int(e) deriv(e) K_D K_D*deriv(e) PID T_S Cooling

0 30.00 30.00 0.00 0.15 0.00 0.00 0.05 0.00 0.00 0.05 0.00 0.00 40.00 0.00

1 30.00 40.00 -10.00 0.15 -1.50 -10.00 0.05 -0.50 -10.00 0.05 -0.50 -2.50 35.00 2.50

2 30.00 32.50 -2.50 0.15 -0.38 -12.50 0.05 -0.63 7.50 0.05 0.38 -0.63 31.00 0.63

3 30.00 30.38 -0.38 0.15 -0.06 -12.88 0.05 -0.64 2.13 0.05 0.11 -0.59 33.00 0.59

4 30.00 32.41 -2.41 0.15 -0.36 -15.28 0.05 -0.76 -2.03 0.05 -0.10 -1.23 39.00 1.23

5 30.00 37.77 -7.77 0.15 -1.17 -23.05 0.05 -1.15 -5.37 0.05 -0.27 -2.59 31.00 2.59

6 30.00 28.41 1.59 0.15 0.24 -21.47 0.05 -1.07 9.36 0.05 0.47 -0.37 35.00 0.37

7 30.00 34.63 -4.63 0.15 -0.69 -26.10 0.05 -1.31 -6.22 0.05 -0.31 -2.31 31.00 2.31

8 30.00 28.69 1.31 0.15 0.20 -24.79 0.05 -1.24 5.94 0.05 0.30 -0.75 40.00 0.75

9 30.00 39.25 -9.25 0.15 -1.39 -34.04 0.05 -1.70 -10.57 0.05 -0.53 -3.62 40.00 3.62

10 30.00 36.38 -6.38 0.15 -0.96 -40.43 0.05 -2.02 2.87 0.05 0.14 -2.83 32.00 2.83

11 30.00 29.17 0.83 0.15 0.13 -39.59 0.05 -1.98 7.22 0.05 0.36 -1.49 39.00 1.49

12 30.00 37.51 -7.51 0.15 -1.13 -47.10 0.05 -2.35 -8.34 0.05 -0.42 -3.90 30.00 3.90

13 30.00 26.10 3.90 0.15 0.58 -43.20 0.05 -2.16 11.40 0.05 0.57 -1.01 33.00 1.01

14 30.00 31.99 -1.99 0.15 -0.30 -45.19 0.05 -2.26 -5.89 0.05 -0.29 -2.85 36.00 2.85

15 30.00 33.15 -3.15 0.15 -0.47 -48.34 0.05 -2.42 -1.15 0.05 -0.06 -2.95 32.00 2.95

16 30.00 29.05 0.95 0.15 0.14 -47.39 0.05 -2.37 4.09 0.05 0.20 -2.02 32.00 2.02

17 30.00 29.98 0.02 0.15 0.00 -47.37 0.05 -2.37 -0.92 0.05 -0.05 -2.41 34.00 2.41

18 30.00 31.59 -1.59 0.15 -0.24 -48.96 0.05 -2.45 -1.61 0.05 -0.08 -2.77 38.00 2.77

19 30.00 35.23 -5.23 0.15 -0.78 -54.19 0.05 -2.71 -3.64 0.05 -0.18 -3.68 33.00 3.68

20 30.00 29.32 0.68 0.15 0.10 -53.52 0.05 -2.68 5.91 0.05 0.30 -2.28 40.00 2.28

21 30.00 37.72 -7.72 0.15 -1.16 -61.24 0.05 -3.06 -8.40 0.05 -0.42 -4.64 32.00 4.64

22 30.00 27.36 2.64 0.15 0.40 -58.60 0.05 -2.93 10.36 0.05 0.52 -2.02 34.00 2.02

23 30.00 31.98 -1.98 0.15 -0.30 -60.58 0.05 -3.03 -4.62 0.05 -0.23 -3.56 40.00 3.56

24 30.00 36.44 -6.44 0.15 -0.97 -67.02 0.05 -3.35 -4.46 0.05 -0.22 -4.54 38.00 4.54

25 30.00 33.46 -3.46 0.15 -0.52 -70.48 0.05 -3.52 2.98 0.05 0.15 -3.89 38.00 3.89

26 30.00 34.11 -4.11 0.15 -0.62 -74.59 0.05 -3.73 -0.65 0.05 -0.03 -4.38 39.00 4.38

27 30.00 34.62 -4.62 0.15 -0.69 -79.21 0.05 -3.96 -0.52 0.05 -0.03 -4.68 31.00 4.68

28 30.00 26.32 3.68 0.15 0.55 -75.53 0.05 -3.78 8.30 0.05 0.42 -2.81 38.00 2.81

29 30.00 35.19 -5.19 0.15 -0.78 -80.72 0.05 -4.04 -8.87 0.05 -0.44 -5.26 36.00 5.26

30 30.00 30.74 -0.74 0.15 -0.11 -81.46 0.05 -4.07 4.45 0.05 0.22 -3.96 33.00 3.96

31 30.00 29.04 0.96 0.15 0.14 -80.50 0.05 -4.03 1.70 0.05 0.09 -3.80 31.00 3.80

32 30.00 27.20 2.80 0.15 0.42 -77.71 0.05 -3.89 1.83 0.05 0.09 -3.37 35.00 3.37

33 30.00 31.63 -1.63 0.15 -0.24 -79.33 0.05 -3.97 -4.42 0.05 -0.22 -4.43 33.00 4.43

34 30.00 28.57 1.43 0.15 0.21 -77.90 0.05 -3.90 3.06 0.05 0.15 -3.53 37.00 3.53

35 30.00 33.47 -3.47 0.15 -0.52 -81.37 0.05 -4.07 -4.90 0.05 -0.25 -4.83 39.00 4.83

36 30.00 34.17 -4.17 0.15 -0.62 -85.54 0.05 -4.28 -0.69 0.05 -0.03 -4.94 30.00 4.94

37 30.00 25.06 4.94 0.15 0.74 -80.60 0.05 -4.03 9.10 0.05 0.46 -2.83 33.00 2.83

38 30.00 30.17 -0.17 0.15 -0.02 -80.77 0.05 -4.04 -5.10 0.05 -0.26 -4.32 35.00 4.32

39 30.00 30.68 -0.68 0.15 -0.10 -81.45 0.05 -4.07 -0.52 0.05 -0.03 -4.20 37.00 4.20

40 30.00 32.80 -2.80 0.15 -0.42 -84.25 0.05 -4.21 -2.12 0.05 -0.11 -4.74 40.00 4.74

41 30.00 35.26 -5.26 0.15 -0.79 -89.51 0.05 -4.48 -2.46 0.05 -0.12 -5.39 40.00 5.39

42 30.00 34.61 -4.61 0.15 -0.69 -94.12 0.05 -4.71 0.65 0.05 0.03 -5.37 39.00 5.37

43 30.00 33.63 -3.63 0.15 -0.55 -97.76 0.05 -4.89 0.98 0.05 0.05 -5.38 31.00 5.38

44 30.00 25.62 4.38 0.15 0.66 -93.37 0.05 -4.67 8.02 0.05 0.40 -3.61 35.00 3.61

45 30.00 31.39 -1.39 0.15 -0.21 -94.76 0.05 -4.74 -5.77 0.05 -0.29 -5.24 33.00 5.24

46 30.00 27.76 2.24 0.15 0.34 -92.53 0.05 -4.63 3.63 0.05 0.18 -4.11 38.00 4.11

47 30.00 33.89 -3.89 0.15 -0.58 -96.42 0.05 -4.82 -6.13 0.05 -0.31 -5.71 35.00 5.71

48 30.00 29.29 0.71 0.15 0.11 -95.71 0.05 -4.79 4.60 0.05 0.23 -4.45 30.00 4.45

49 30.00 25.55 4.45 0.15 0.67 -91.26 0.05 -4.56 3.74 0.05 0.19 -3.71 38.00 3.71

50 30.00 34.29 -4.29 0.15 -0.64 -95.55 0.05 -4.78 -8.74 0.05 -0.44 -5.86 37.00 5.86

Ave 31.87 Total 172.31

Max 40.00 Ave 3.38

Max 5.86

1 2 3 4 5 6 7 8 9 10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00

40.00

45.00

T_ActualCoolingT_ServerPolynomial (T_Server)

The Future

• Concurrent Systems

Image: http://eetd.lbl.gov/hightech/img/datacenter.gif

Concurrent Systems

• Chip manufactures are pushing new CPU packages with multiple processors. GPU's are becoming increasingly like CPU's and may have helped fuel the popularity of distributed computing, Map Reduce, Software as a service models and other concurrent systems in the marketplace.

The Future

• Concurrent Systems• Sustainability

Image: http://sustainabilityninja.com

Sustainability

• By reducing the number of exchanges the increase in heat transfer can be passed to an external environment can include active sources currently used today such as a water chiller or passive sources such as a large body of water or geothermal heat pump.

The Future

• Concurrent Systems• Sustainability• Flexibility and Challenges

Image: http://www.oraone.nl/datacenter

Flexibility and Challenges

• Greater utilization of current datacenter space and the ability to retrofit our system into current outdated water-chilled datacenters.

• Current industry stigma against change. The IT industry doesn’t like radically different approaches which are viewed as risky. The project only becomes feasible when ROI is high enough to offset risk.