[ASME ASME 2013 International Technical Conference and Exhibition on Packaging and Integration of...

IMPROVING THE THERMAL PERFORMANCE OF A FORCED CONVECTION AIR COOLED SOLUTION – PART 1: MODIFICATION OF HEAT SINK ASSEMBLY

Saeed Ghalambor University of Texas at Arlington

Arlington, TX, USA

John Edward Fernandes University of Texas at Arlington

Arlington, TX, USA

Dereje Agonafer University of Texas at Arlington

Arlington, TX, USA

Veerendra Mulay Facebook Incorporation Menlo Park, CA, USA

ABSTRACT Forced convection air cooling using heat sinks is one of the

most prevalent methods in thermal management of microelectronic devices. Improving the performance of such a solution may involve minimizing the external thermal resistance (Rext) of the package. For a given heat sink design, this can be achieved by reducing the thermal interface material (TIM) thickness through promotion of a uniform interfacial pressure distribution between the device and heat sink. In this study, a dual-CPU rackmount server is considered and modifications to the heat sink assembly such as backplate thickness and bolting configuration are investigated to achieve the aforementioned improvements. A full-scale, simplified model of the motherboard is deployed in ANSYS Mechanical, with emphasis on non-linear contact analysis and torque analysis of spring screws, to determine the optimal design of the heat sink assembly. It is observed that improved interfacial contact and pressure distribution is achieved by increasing the number of screws (loading points) and positioning them as close to the contact area as possible. The numerical model is validated by comparison with experimental measurements within reasonable accuracy. Based on the results of numerical analysis, the heat sink assembly is modified and improvement over the base configuration is experimentally quantified through interfacial pressure measurement. The effect of improved interfacial contact on thermal performance of the solution is discussed.

Keywords: Server, back plate, interfacial pressure, TIM performance, mechanical analysis

INTRODUCTION The data center industry has experienced significant growth over the past decade with the introduction and explosion of online banking, cloud computing, internet entertainment and social networking services. Such are the implications that, it has been reported, the national energy usage by data centers more than doubled between 2000 and 2005 and it was projected that consumption would continue to rise over the course of the following five years [1]. By 2010 it was reported that data centers accounted for around 2% (between 1.7% and 2.2%) of the total national electricity consumption [2]. With a sizeable portion (around 30%) of typical data center power consumption attributed to cooling [3], which is categorized as a parasitic load, it has become imperative that energy savings and efficiencies be pursued in these components at various levels within the data center facility. This paper focuses on the module-level cooling in conventional rackmount servers. By reducing thermal resistance of the TIM through improved interfacial contact, thermal performance of the air cooling solution will be enhanced thereby reducing its power consumption.

A typical module-level cooling solution for an air cooled server consists of a passive heat sink and TIM II (between case and sink, henceforth referred to as TIM). The external thermal resistance of the microelectronics package would consist of a combination of TIM and heat sink resistances. For a given heat sink design, the external resistance of the package would vary with the TIM only, whose thermal resistance (RTIM) is given by [4],

Proceedings of the ASME 2013 International Technical Conference and Exhibition on Packaging and Integration of Electronic and Photonic Microsystems

InterPACK2013 July 16-18, 2013, Burlingame, CA, USA

IPACK2013-73108

1 Copyright © 2013 by ASME

Downloaded From: http://proceedings.asmedigitalcollection.asme.org/ on 04/09/2014 Terms of Use: http://asme.org/terms

(1)

Rc is the contact resistance between the TIM and bonding surfaces, kTIM is the thermal conductivity and BLT is the bond line thickness of the TIM. Both these terms may be reduced by ensuring a uniform, high-pressure contact between the heat sink base and the CPU integrated heat spreader (IHS). This is because the BLT of a TIM reduces with increasing pressure and uniform application causes the two metal surfaces to come into contact in greater proportion reducing the contact resistance. Thus, promoting a uniform pressure distribution at the interface improves performance of the microelectronics package by reducing the external resistance.

Fig. 1: Intel-based Open Compute server [5, 6]

Current system: In this study, we investigate the module level cooling solution installed in an Intel-based Open Compute server [5-6], as shown in Fig. 1. Installed in the server chassis are four axial fans, with pulse width modulation (PWM) capability, which cool all components on the motherboard (PCB). An on-board control algorithm modulates the fan speed as a function of the CPU die temperature, ensuring that adequate cooling is provided. Thus, reducing the junction temperature of the device will result in reduction of both fan speeds and power consumption. Ensuring a uniform pressure distribution at the interface of the CPU and heat sink, thereby reducing the thermal resistance of the TIM and overall cooling solution, will assist in obtaining this objective.

The maximum (perfect) contact area at the top surface of the IHS is around 1.7 in2. Figure 2 shows the pressure distribution, obtained by employing pressure measurement films and processing software [7-8], at the interface of the CPU and heat sink. For the baseline configuration, pressed area for CPU0 and CPU1 is 0.633 in2 (37.24%) and 0.667 in2 (39.24%) respectively. With physical contact concentrated at the leading and trailing edges of the interface, we can conclude that the thermal performance of the solution will not be ideal. Bowing of the backplate, which holds the heat sink against the CPU and socket assembly, is observed and a deflection of around 7 mils

(0.007 in) is recorded at the center of its longitudinal axis (or along the transverse axis). Lack of contact at the center of the thermal interface can be attributed to this deflection in the entire module-level assembly. Thus, in order to promote a uniform pressure distribution at the thermal interface, deflection under loading should be minimized. Through application of finite element analysis (FEA), the following two approaches are studied to achieve the same:

i. Design change A: Increase the backplate thickness to make the assembly more rigid or,

ii. Design change B: Modify the heat sink to increase the number of heat sink bolts and reduce the distance between these loading points and the center of the interface.

The following section will outline the modeling methodology employed to study these two design changes.

Fig. 2: Pressure distribution at the thermal interfaces for the original configuration

MODELING METHODOLOGY Before the proposed design changes can be studied, a

robust numerical model must be generated, and validated for accuracy with experimental results. This section will outline creation of the simplified model, integration of proposed design changes and numerical modeling employed to study structural behavior of the complete module-level assembly under loading.

Table 1: Material assignments

Component Material Motherboard/PCB FR-4 [9]

Backplate Structural Steel CPU Copper Alloy

Heat Sink Aluminum Alloy Fasteners Structural Steel

Simplified model: PCB installed in the server, as shown in Fig. 3(a), has multiple components mounted on it in addition to the CPU and heat sink assembly. However, these parts are peripheral to the analysis being conducted and are excluded from the simplified model (solid model created in PTC Creo [10]) depicted in Fig. 3(b), which contains only the



motherboard and module-level assemblies. This assembly consists of the backplate, socket, CPU, independent loading mechanism (ILM), heat sink and fasteners (spring screws installed in the heat sink). The lever mechanism of the ILM rigidly holds the CPU against the socket while ensuring electrical connectivity between the socket pins and CPU. An immeasurable amount of penetration is permitted during the aforementioned assembly beyond which both socket and CPU move in unison during installation of the heat sink. In addition, the plastic casing of the socket prohibits excessive penetration of the CPU into the socket to prevent overloading and bending of pins. Therefore, for the purpose of this study, the CPU, socket and ILM can be modeled as a merged part (henceforth referred to as the CPU). Holes in the motherboard are used to define ‘fixed support’ constraints to account for standoffs that hold the PCB in place within the server chassis. Table 1 lists material assignments for each component in the simplified model. A detailed tabulation of properties of aforementioned material assignments can be found in Table A1 in the appendix. Apart from the PCB, which is assigned material properties as defined in [9]; remaining components rely on the in-built database in ANSYS for material allocation. Effort has been made to ensure that properties assigned to the PCB material are as representative as possible. However, it should be noted that this assumption will directly influence the accuracy of numerical predictions, especially the deflection.

Numerical Modeling: Structural analysis of the simplified model is conducted in ANSYS Mechanical [11], with emphasis on non-linear contact analysis and torque analysis of spring screws, to simulate the nature and magnitude of deflection in the motherboard and interfacial pressure distribution between the heat sink and CPU IHS. The first task is to determine the contact settings. Contact between the bolts and nuts are always bonded. However, contact pairs that run along the length of the bolt should be ‘frictionless’ to allow the two components to slide in relation to one another. The rest of the contact areas are

the interface between the heat sink and IHS, ILM fixtures with motherboard, and back plates to the motherboard; all of which are categorized as ‘frictional’ contacts.

The primary algorithm governing frictional contact analysis in ANSYS is the augmented Lagrangian. This algorithm contains an extra term for contact traction as opposed to the pure Lagrangian scheme. It will allow for higher penetration and movement between the contact areas and usually requires less iterations for the solution to converge.

In this study, more emphasis is placed on making node-to-node contacts that pass FEA criteria and result in highest accuracy. The objective is to eliminate cross penetration into coarsely discretized contact surfaces as well as local penetration which violates the compatibility condition. Node-to-node CONTA178 and surface-to-surface CONTA171 elements were employed as applicable. The CONTA171 is the most versatile contact element in the ANSYS program. It is compatible with both lower order and higher-order elements and supports large deformations with significant amount of sliding and friction.

In the model, the only external forces acting on the assembly is the torque applied on the bolts to sustain the heat sink and ILM to the motherboard through the back plate. To simulate the effect of the torque on the screws, bolt pretension is applied where applicable. A torque of 12.8 kgf-cm magnitude is required for driving the heat sink screws and a 10.4 kgf-cm equivalent is required for installing the ILM per manufacturer’s specification [12]. Using the following relationship one can determine the load pretension on each screw [13],

(2)

T is the bolt installation torque, μ is the torque coefficient, F is bolt preload and D is the bolt nominal size. Once the axial

Fig. 3: (a) Actual server motherboard [5] and (b) simplified model



loads are determined, the pretensions to the cylindrical face of the bolts are applied by creating pretension elements (PRETS179) along the direction of the cylindrical axis of the bolts.

Proceeding Models: The original simplified model is modified for analysis of the proposed design changes. In order to study the effect of backplate thickness on deflection and interfacial contact, this design variable is parameterized in the CAD model before importing into the software. A design of experiments (DoE) is setup with thickness varying from the baseline value of 0.092 in to 0.35 in.

Studies have been conducted to correlate interfacial contact between two mating surfaces with number of loading points and the torque applied at each location [14, 15]. It has been reported that interfacial contact improves with an increase in number of bolts and applied torque, however, contact at the center will be achieved only through the application of a direct compressive force at this location. In order to implement this concept in a practical manner, we increased the number of loading points on the heat sink to four and located them to coincide with four corresponding points at the ILM. Thus, a single bolt at each corner of the backplate can be employed to fasten both the ILM and heat sink as shown in Fig. 4. In addition, the backplate thickness is increased based on results of the first approach.

Fig. 4: Modified ILM configuration

RESULTS AND DISCUSSION In order to create a robust model that can be employed to

study effects of the proposed modifications, results from FEA of the baseline configuration need to agree with observations of experimental measurements within acceptable accuracy. Plots of contact pressure and magnitude of total deformation at the backplates from numerical modeling are compared with pressure film and backplate bowing measurements. Note that pressure film testing is conducted on three different

motherboards to check for repeatability of measurements. For subsequent design changes, only average values of interfacial contact are reported.

Baseline study: A grid independence study is conducted (see Fig. A5 in appendix) and Fig. 5 shows the contact pressure distributions at the two CPUs reported from numerical analysis of the simplified model. The pressure distributions are similar to the film measurements (Fig. 2) with interfacial contact concentrated at the leading and trailing edges of the IHS. In addition, as seen in Fig. A1, maximum total deformation of the backplates is reported to be around 4.55 mils (0.00455 in) which is in good agreement to experimental measurements that place this value at 7 mils (0.007 in). Considering the aggressive simplification of the model for numerical analysis and possible inherent deformation or bowing that may be present in the actual backplates and motherboard before loading, the reported results of numerical simulation can be qualified to be reasonably accurate. The simplified model is now validated and can be modified to study effect of the proposed modifications on interfacial contact and bowing of the module-level assembly.

Fig. 5: Numerical prediction of interfacial pressure distribution for the baseline configuration

Fig. 6: Variation of backplate deflection with thickness



Design change A: Results from the parametric study correlating backplate thickness with corresponding total deformation is shown in Fig. 6. As expected, deformation decreases with an increase in thickness due to increased rigidity of the entire assembly. It is expected that beyond a thickness of 0.3 in, decrease in bowing is not substantial enough to warrant the increase in material required. In addition, no substantial improvement in interfacial contact is observed, irrespective of the thickness utilized. The gap between the motherboard and chassis is 0.25 in and this represents the maximum backplate depth that can be accommodated within the server for normal operation. The corresponding bowing is predicted to be around 2 mils (0.002 in). Two backplates of specified thickness (0.25 in) are fabricated to quantify improvement in pressed area that may be available taking into account surface non-uniformity that is inherent to both CPU and heat sink. As shown in Fig. 7, film measurements report noticeable improvement in interfacial contact, 0.867 in2 (51%) for CPU0 and 0.867 in2 (51%) for CPU1, as compared to the baseline configuration.

Fig. 7: Pressure distribution at the thermal interfaces for assemblies accommodating 0.25 inch thick backplates

Design change B: To reap the benefit of reduced deformation in addition to greater number of loading points located closer to the contact area, backplates utilized in the ‘ILM configuration’ are modeled with a thickness of 0.25 in. Both ILM and heat sink rely on the same bolts for retention as this represents the closest loading points that are practically feasible, given the current motherboard design. Torque applied is lowered to 4 kgf-cm to account for the additive loading at each corner of ILM and heat sink under which the system (server) is still fully functional. The simplified model is modified to account for the aforementioned changes and is shown in Fig. 4. A grid independence study is conducted (see Fig. A6) and contact pressure distributions predicted by the numerical model are shown in Fig. 8(a). Due to relocation of loads to the four corners of the CPU, the pressure is now concentrated at all four corners of the IHS. Total deformation of the backplates is appreciably reduced to around 0.0086 mils (0.0000086 in), as seen in Fig. A3. Since the numerical model does not account for non-uniformity present at the interfacing surfaces of the heat sink and CPU, the proposed modifications to both heat sink and backplate are physically carried out to enable measurement with pressure films. Smaller backplates are fabricated with four countersunk holes for introduction of 2 in long machine screws that facilitate retention of ILM against the motherboard as well as heat sink against the CPU. The ILM is also modified to permit passage of the screws through the retention PEM nuts to corresponding holes in the heat sink. Here, torque is applied to four 6-32 hex nuts that serve to hold the body in place.

Figure 8(b) shows interfacial contact measured by pressure films for the proposed ‘ILM configuration’. Contrary to the results of numerical analysis, pressed areas are significantly increased as compared to the baseline assembly. The measured values are 1.233 in2 (72.53%) for CPU0 and 1.333 in2 (78.41%) for CPU1. Difference between the numerical prediction and physical measurement can be attributed to the deflection being

Fig. 8: Interfacial pressure distribution for the ILM configuration from (a) numerical analysis, and (b) experimental measurement



inadequate to move the non-uniform surfaces away from contact. Pressure film measurements also depict more uniform pressure distributions at the thermal interfaces as compared to both the original configuration and design change A. Based on equation 1 and variation of BLT with contact pressure, improvement in TIM performance is expected through application of the ‘ILM configuration’ for heat sink assembly. As a result, reduction in both CPU operating temperatures and fan operation (power consumption) may be evident. Quantifying these improvements in system-level performance will be carried out in the accompanying study, to be reported in Part 2.

Noticeable improvement in interfacial contact between design points A and B (both have a backplate thickness of 0.25 in) begs the question if such an improvement can be observed if the ‘ILM configuration’ utilizes a backplate of equal thickness as the baseline case (0.092 in). To accommodate this measurement, the original backplate is modified to permit passage of 2 in long machine screws for retention of both ILM and heat sink. Results of pressure film measurements are shown in Fig. 10. Reported pressed areas for CPU0 and CPU1 are 1.133 in2 (66.65%) and 1.167 in2 (68.65%) respectively. In comparison with the baseline configuration, it is observed that better contact is achieved with four closely-spaced loading points than with a thicker backplate. However, to maximize interfacial contact, a combination of these two modifications needs to be employed as seen in Fig. 8.

Fig. 9: Interfacial pressure distribution for the ILM configuration utilizing the original backplates

Feasibility of proposed designs: While the suggested modifications make sense from an investigative point-of-view, it is important to comment on the implications of implementing these changes from production and application standpoints. Table 2 lists the weight of each backplate employed in this study. The 0.25 in units are significantly heavier in comparison to the original design. This will increase cost of both manufacturing (material) and shipment of servers. However, in reference to Fig. 9, application of the ‘ILM configuration’ may not require a backplate that is much thicker than the original

unit to seek improved TIM performance. In addition, backplates employed in the ‘ILM configuration’ are smaller and lighter (see table 2) than ones utilized in the baseline (0.25 in) configuration. Thus, greater thickness can be accommodated for the proposed configuration in a feasible manner.

Table 2. Weights of backplates

Backplate Design Weight (g) Baseline (0.092 in) 62.7

163.3 143.2

Thicker backplate (0.25 in) ILM configuration (0.25 in)

CONCLUSIONS Modification of the module-level assembly to improve

contact and promote uniform pressure distribution at the thermal interface between CPU and heat sink has been conducted. FEA was successfully employed to analyze the system under consideration and the resulting simplified model was validated with experimental measurements within reasonable accuracy. Two design changes to the assembly were proposed to reduce bowing of the backplates and achieve the aforementioned overlying objectives. Increasing the backplate thickness reduced total deformation of the backplates but produced marginal improvement in interfacial contact. Increasing the number of loading points at the heat sink to coincide with corresponding locations at the ILM in addition to maximum thickness of the backplates caused a significant reduction in deflection of the assembly. Since the simplified model does not account for non-uniformity in the contacting surfaces, interfacial pressure was concentrated at all four edges of the IHS. However, physical measurements of the ‘ILM configuration’ show a vast improvement in pressed area in addition to a more uniform pressure distribution at the interface. A concise summary of results has been listed in table 3. Quantifying improvements in system-level performance through application of the ‘ILM configuration’ will be carried out in Part 2 of this study.

Table 3. Summary of results

Design (Backplate Thickness, in)

Deformation of Backplate (x10-3 in)

Interfacial Contact (%)

CPU0 CPU1 Baseline (0.092) 4.55 37.24 39.24 Thicker backplate (0.25) 2 51.00 51.00 ILM configuration (0.092) N/A 66.65 68.65 ILM configuration (0.25) 0.0086 72.53 78.41

ACKNOWLEDGMENTS The authors would like to acknowledge the assistance of

Richard Eiland, Bharath Nagendran, Dianne Narzinski, Jonathan Dow, Shishir Dhungana, Krishna Sedhai and Deepak Koirala during the course of this study.



REFERENCES [1] US EPA, ‘Report to Congress on Server and Data Center Energy Efficiency, Public Law 109-431’. Prepared for the US Environmental Protection Agency, ENERGY STAR Program, by Lawrence Berkeley National Laboratory LBNL-363E, 2007. [2] Koomey J.G., ‘Growth in Data Center Electricity Use 2005 to 2010’. A report by Analytics Press, completed at the request of The New York Times, 2011. [3] Pelley S., Meisner D., Wenisch T.F. and VanGilder J.W., ‘Understanding and abstracting total data center power’, in WEED: Workshop on Energy Efficient Design, 2009. [4] Prasher R., ‘Surface Chemistry and Characteristic Based Model for the Thermal Contact Resistance of Fluidic Interstitial Thermal Interface Materials’, Journal of Heat Transfer, v123, pp. 969-975, 2001. [5] Li H. and Michael A., ‘Intel Motherboard Hardware v1.0’, Open Compute Project. http://www.opencompute.org/projects/intel-motherboard/ [6] Frachtenberg, E., Heydari, A., Li, H., Michael, A., Na, J., Nisbet, A. and Sarti, P., ‘High-Efficiency Server Design’, Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis, 2011. [7] Fujifilm Prescale pressure measurement film. http://www.fujifilm.com/products/prescale/ [8] Fujifilm, Pressure distribution mapping system FPD-8010E. http://www.fujifilm.com/products/prescale/fpd8010e/ [9] Zhang B., Liu P., Ding H. and Cao W., ‘Modeling of board-level package by Finite Element Analysis and laser interferometer measurements’, Microelectronics Reliability, v50(7), pp. 1021-1027, 2010. [10] Creo Elements/Pro by Parametric Technology Corporation. http://creo.ptc.com/ [11] ANSYS Mechanical 13. http://www.ansys.com/Products/Simulation+Technology/Structural+Mechanics/ANSYS+Mechanical [12] Intel® Xeon® Processor 5500/5600 Series: Thermal/Mechanical Design Guide, March 2010. [13] Juvinall, R.C. and Marshek K.M., Fundamentals of Machine Component Design, John Wiley and Sons, New York, 1983. [14] Yeh C.L., Wen C.Y., Chen Y.F., Yeh S.H. and Wu C.H., ‘An experimental investigation of thermal contact conductance across bolted joints’, Experimental Thermal and Fluid Science, v25, pp. 349-357, 2001. [15] Gould H.H. and Mikic B.B., ‘Areas of contact and pressure distribution in bolted joints’, final technical report prepared for George C. Marshall Space Flight Center, June 1970.

APPENDIX

Fig. A1: Total backplate deformation for original design

Fig. A2: Total system deformation for original design

Fig. A3: Total backplate deformation for design change B



Fig. A4: Total system deformation for design change B

Fig. A5: Grid independence study for baseline configuration – solution becomes relatively grid independent at around 120,000 cells

Fig A6: Grid independence study for ‘Design Change B’ with retention nut torque of 8 kgf-cm – solution becomes relatively grid independent at around 110,000 cells

Units (Pa) FR-4 Copper Alloy Structural Steel Aluminum Alloy Young’s Modulus X direction 1.69E10 1.1E11 2E11 7.1E10 Young’s Modulus Y direction 1.69E10 1.1E11 2E11 7.1E10 Young’s Modulus Z direction 7.4E9 1.1E11 2E11 7.1E10

Poisson’s Ratio XY 0.11 0.34 0.3 0.33 Poisson’s Ratio YZ 0.39 0.34 0.3 0.33 Poisson’s Ratio XZ 0.39 0.34 0.3 0.33 Shear Modulus XY 7.6E9 4.1E10 7.6923E10 2.6692E10 Shear Modulus YZ 3.3E9 4.1E10 7.6923E10 2.6692E10 Shear Modulus XZ 3.3E9 4.1E10 7.6923E10 2.6692E10

Tensile Yield Strength 2.5E8 2.8E8 2.5E8 2.8E8 Tensile Ultimate Strength 4.6E8 4.3E8 4.6E8 3.1E8

Table A1. Properties of materials employed in current study



[ASME ASME 2013 International Technical Conference and Exhibition on Packaging and Integration of...

Documents

Transcript of [ASME ASME 2013 International Technical Conference and Exhibition on Packaging and Integration of...