Embedded Cooling: A New Thermal Packaging Paradigm
Avram Bar-Cohen
DARPA/MTO
United States
EPTC 2012
14th Electronics Packaging Technology Conference
5-7 December 2012
Resorts World Sentosa, Singapore
Distribution Statement A, Approved For Public Release, Distribution Unlimited
2
Presentation Roadmap
Brief History of Thermal Packaging
1946 – present: 5 eras – 2 Paradigms
2000+ :Nanoelectronics Triple Threat (power-hot spots-3D)
Moore’s Law: End or Just Hiatus
DARPA Thermal Packaging Programs
Embedded Cooling Paradigm
Limitations of Remote Cooling
Near-Junction Thermal Transport
Intrachip/Interchip Microfluidic Cooling
Distribution Statement A, Approved For Public Release, Distribution Unlimited
http://ei.cs.vt.edu/~history/ENIAC.Richey.HTML
Electronic Numerical Integrator and Computer ENIAC (1946)
3 Distribution Statement A, Approved For Public Release, Distribution Unlimited
Eras of Thermal Packaging
HVAC Era: 1945-1975 • ENIAC, IBM Mainframes • Telephone switching equipment • Vacuum tubes and early solid-state
transistors • Goal: Remove heated air from room/rack/cabinet
Rack Cooling Era: 1975-1985 • DIP’s and SMT’s on PCB’s • PCB’s in Card Cages • Goal: Maximize natural and forced
convection cooling in racks
IBM Mark I Mainframe (1950's)
Distribution Statement A, Approved For Public Release, Distribution Unlimited
IBM 360 Card Cage (1982)
Eras of Thermal Packaging
Liquid/Refrigerant Cooling Era:1980-1990 • Maturation of bipolar devices: ~5W Chips,
~300W Multi Chip Modules • Honeywell, IBM, CDC, Hitachi, NEX,
Fujitsu,….mainframes/supers • Goal: Gain control over the local
“coldplate,” “cold bar” temperature
Enhanced Air Cooling Era: 1985-2000
• Thermally-engineered heat sinks for CMOS microprocessors
• Miniaturized servers create Data Center cooling challenge
• Goals: • Reduce “case-to-air” resistance
for chip package • Improve Data Center thermal
management
5
Fujitusu - 8CPU 470x580x80
1600W Airflow~3.5 m/s
IBM 3081
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Chip Power/Heat Flux Trends (iNEMI)
6
0
100
200
300
400
500
600
2005 2006 2007 2008 2009 2010 2011 2012 2013
Year
Maxim
um
Ch
ip P
ow
er
(W)
0
50
100
150
200
250
300
350
2000 2002 2004 2006 2008 2010 2012 2014 2016
Year
Po
wer (
W), H
eat F
lux (
W/c
m2)
0
0.5
1
1.5
2
2.5
3
3.5
Ch
ip A
rea (
cm
2)
Max. Steady-State Chip Pow er
(W)
Max. Chip Heat Flux (W/cm2)
Chip Area (cm2)
Servers
Automotive
0
50
100
150
200
2000 2005 2010 2015 2020
Year
Max C
hip
Po
wer
(W)
2002
2004
2006Desk Top PC’s
Distribution Statement A, Approved For Public Release, Distribution Unlimited
0
50
100
150
200
2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020
0
50
100
150
200
2000 2002 2004 2006 2008 2010 2012 2014 2016 2018 2020
2011
S.M. Sri-Jayantha, G. McVicker, K. Bernstein, J.U. Knickerbocker IBM Journal,Res & Dev, 2008, Vol 52, No 6 q” hot spot = 500W/cm2, 2x2mm; q” avg = 50W/cm2, 40x40mm
Thermal Packaging Challenge Microprocessor Hot Spots (IBM, 2008)
7 Distribution Statement A, Approved For Public Release, Distribution Unlimited
Heat Flux Challenge
8
0 1000 2000 3000 4000 5000 600010
-2
10-1
100
101
102
103
104
H
ea
t F
lux(W
/cm
2)
Temperature (K)
On Sun’s surface
Solar Flux On Earth’s surface
Rocket motor case
Reentry from earth orbit
Nuclear blast
Ballistic entry
Chip Hot Spots
Distribution Statement A, Approved For Public Release, Distribution Unlimited
(Dereje Agonafer and Bahgat Sammakia, InterPACK’05)
3D Packaging Configurations
9 Distribution Statement A, Approved For Public Release, Distribution Unlimited
10
Heterogeneous Chip Stacks
Source: M. Swaminathan, Keynote Presentation ,ITherm 2012.
Distribution Statement A, Approved For Public Release, Distribution Unlimited
11
Next Semiconductor Revolution – 3D Integration
Source: M. Swaminathan, Keynote Presentation ,ITherm 2012.
Distribution Statement A, Approved For Public Release, Distribution Unlimited
12
Heat Density Challenge
Chip Stack
Human brain
Electric stove
IBM TCM Module
Cray-3 Module
Light-water nuclear reactor
Liquid Metal nuclear reactor
Mercury Vapor lamp
Halogen bulb
Home light bulb
SX-3 Module
Distribution Statement A, Approved For Public Release, Distribution Unlimited
High Power
Thermal Packaging “Triple Threat”
heat spreader
chip carrier
heat sink
heat sink
Hot Spots
13
Nanoelectronics Era 2000 - :
• GHz-level CMOS with features below 100 nanometers
• Power dissipation increasing, distinct on-chip “hot spots” on silicon/compound semiconductors
• Emergence of homogeneous/heterogeneous “chip stacks” denying access to back of chip for “thermal solution”
Thermal Management Goals: Remove large flux Reduce/eliminate on-chip “hot spots” Extract high heat density
3-Dimensional
Distribution Statement A, Approved For Public Release, Distribution Unlimited
14
Moore’s Law: Driver for the Industry
Source: M. Swaminathan, Keynote Presentation ,ITherm 2012.
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Intel I7
Source: Intel
The Future of Computing Performance, Game Over or Next Level National Academies Press, 2010
Thermal Management Plateau – Performance Plateau
Mic
ropro
cess
or
Pow
er
dis
sipation (
W)
Source: Ellsworth, IBM ‘11
Source: Fuller and Millett, 2010
15
“The growth in the performance of computing systems will become limited by their power and thermal requirements within the next decade.”
Source: Ellsworth, ITHERM 2008
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Concurrency only path left to processor designers – National Academy of Science report
16
Micro Processor Technology Landscape
CONCURRENCY • More but slower
cores. • Potentially more
performance. • Potentially more
power efficiency.
Year of Introduction
Mic
ropro
cess
or
pow
er
(watt
s)
Power limited
Transistor counts continue to increase
Meanwhile:
Tra
nsi
stor
count
Year of Introduction
2011 NRC/CSTB Study: “The Future of Computing
Performance”
Mic
ropro
cess
or
clock
fre
quency
(M
Hz)
Clock frequency stalled
Year of Introduction
Year of Introduction
SPECin
t2000 s
core
norm
aliz
ed t
o 1
985=
1
Single-threaded performance stalled
Distribution Statement A, Approved For Public Release, Distribution Unlimited
From Charles Holland - MTO
Single-threaded performance stalled!
18
End of Moore’s Law or Just Hiatus
Moore’s Law progression stalled
by voltage and feature size limits
𝑓 ∝ 𝑃𝑑𝑒𝑛𝑠𝑖𝑡𝑦
𝑓 ∝ ℎ𝑒𝑎𝑡
Ng = CMOS gates/unit area Cload = capacitive load/CMOS gate V = supply voltage f = clock frequency
Dennard’s Equation:
𝑓 =𝑃𝑑𝑒𝑛𝑠𝑖𝑡𝑦
𝑁𝑔𝐶𝑙𝑜𝑎𝑑𝑉2
2011 NRC/CSTB Study: “The Future of
Computing Performance”
1985 1990 1995 2000 2005 2010 2015 2020 10
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Enhanced cooling can restore chip
frequency progression
19
DARPA Thermal Management Programs Timeline
Heat Removal by Thermo-Integrated Circuits (HERETIC)
• 1998: DARPA PM Towe • 2001: DARPA PM Radack • 2002: HERETIC ends
Micro Cryo Coolers (MCC)
• 2006-2011: DARPA PM Dennis Polla
Technologies for Heat Removal in Electronics at the Device Scale (THREADS) • 2005-2006: DARPA PM Rosker • 2009-present: DARPA PM Albrecht
ACM
MACE NTI
TGP
98 00 02 04 06 08 10 12
TMT
MCC
THREADS
THREADS
NJTT
Thermal Management Technologies (TMT) • 2007-2010: DARPA PM Kenny • 2010-present: DARPA PM Bar-Cohen
14
ICECool
16
Intrachip Embedded Cooling (ICECool)
• 2012 - 2015: DARPA PM Bar-Cohen • ICECool will explore novel, disruptive, chip/package
level – embedded - thermal technologies in Si and non-Si electronics
Distribution Statement A, Approved For Public Release, Distribution Unlimited
20
Thermal Management Technologies (TMT) Program Goals (2008-2013)
• Leverage significant recent
advances in nanostructured
materials, active structures and
integrated manufacturing
• Enhance performance of DoD
systems through manipulation,
transport and rejection of waste
heat
Task Areas :
Microtechnologies for Air-Cooled Exchangers (MACE): Active surfaces and jets for enhanced heat sinks
Thermal Ground Plane (TGP): Nanostructured wicks and cases for 2-phase vapor chambers
Nano-Thermal Interfaces (NTI): Engineered, reworkable nanostructures for low resistivity TIMs
Active Cooling Modules (ACM): High COP cooler using novel TE materials and refrigeration concepts
Goal: To deliver transformative thermal management technology that will reduce
or remove thermal limitations on DoD platforms
Heat
Distribution Statement A, Approved For Public Release, Distribution Unlimited
21
Air-Cooled Heat Sink Approaches and Performance
MACE 1
MACE 2
MACE 5
MACE 3
MACE 4
Aavid
Wakefield
Radian
Alpha Novatech
1
10
100
1 10 100 1000
Th
erm
al R
esis
tivit
y (
K·c
m2/W
)
Base Area (cm2)
Current
COTS
Motor
Blower
Spiral
Fin-Diffuser
Active
Reeds
Motor
Blower
Spiral
Fin-Diffuser
Active
Reeds
Raytheon: Micro/Macro Fin;
Synthetic jets
MIT: 3D vapor chamber; fan/fin
integration
Honeywell: Jet-driven entrainment
Thermacore: 3D vapor chamber,
vibrating elements
United Technologies Research Center:
Integrated blower/heat sink with optimized fin
geometry
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Motor
Blower
Spiral
Fin-Diffuser
Active
Reeds
Motor
Blower
Spiral
Fin-Diffuser
Active
Reeds
Active Heat Transfer Enhancement
0
50
100
150
200
1.5 2.0 2.5 3.0Inp
ut P
ow
er
[mW
]
Rth [C/W]
Integrated Blower & Fin-Diffuser
Integrated Design Concept
Improved performance at
decreased volume
Full-scale Phase I demo
Single-channel enhancement test bed
Results Summary
Key Design Elements
Cool air
supplyHeated
ejected
air
Reed Enhanced
Baseline
20% 75 LPM
60 LPM
• United Technologies Research Center
• Georgia Institute of Technology
• Hamilton Sundstrand
MACE Recent Progress
5
9
13
17
21
0 5 10 15 20
Ne
t P
ow
er
De
ns
ity
(k
W/m
3)
Cooling Power (W)
Baseline (Natural Convection)
MACE At design point (10 W), Phase II MACE will allow power density to be increased by 60%
Source: UTRC Source: UTRC
Source: UTRC Source: UTRC
Source: UTRC
Source: UTRC
Distribution Statement A, Approved For Public Release, Distribution Unlimited 22
23
Two-Phase Spreaders: Approaches and Performance
U Colorado: flexible/conformal case with Cu nanomesh wick
UC Berkeley: two phase flow with coherent porous
silicon wick
UC Santa Barbara: large scale titanium TGP
UCLA: metallic powder + biporous wick/posts
Teledyne: CNT/Si wick structures
GE: nanostructured super hydrophobic/philic wick
Northrop Grumman: SiC oscillating heat
pipe
Raytheon: Patterned CNTs on Cu wick
Distribution Statement A, Approved For Public Release, Distribution Unlimited
24
Thermal Interface Materials (TIMs): Approaches and Performance
General Electric: Copper Nanosprings
Teledyne: Laminated Graphite and Solder
Georgia Tech: Well-Aligned Open-End Carbon Nanotubes
Raytheon: Double-sided Multi-walled Carbon Nanotubes
coppercopper
siliconsilicon 5 m
coppercopper
siliconsilicon 5 m
RCu-CNT = 0.9 0.5 mm2K/W
RCNT layer1 < 0.1 mm2K/W
RCNT-CNT = 2.1 0.4 mm2K/W
RCNT layer2 < 0.1 mm2K/W
RSi-CNT = 0.8 0.5 mm2K/W
Intra-Interface
Thermal Resistances
Commercial TIM data courtesy of: D. Altman (Raytheon) Y. Zhao (Teledyne)
Distribution Statement A, Approved For Public Release, Distribution Unlimited
NTI Recent Progress: General Electric Global Research
Compliance scales with turns for larger area bonding
Patterned GLAD, 3 turns, diameter=250nm, 550 nm pitch
Worlds 1st Cu GLAD Nanosprings
R GLAD Process
Tungsten TIM base
GLAD fabrication by Micralyne (Alberta, CA)
Source: GE
Source: GE
Source: GE
* GERC team leapfrogs to Phase 3 goals, with
GLAD fabrication of copper nano springs * More than 50% of samples providing thermal resistivities below Phase 3 goal of 0.01 cm2-K/W
Distribution Statement A, Approved For Public Release, Distribution Unlimited 25
Program Final Goal
NTI Recent Progress: Teledyne Scientific
Teledyne – graphite/solder laminate
Thermal performance at 30 Psi
Thermal performance at 58 Psi
NTI Prototypes Heat flux
range
Total thermal
resistivity
Number Thickness
(m)
W/cm2 cm2 C/W
1 150 80-120 0.023
2 150 80-120 0.016
3 150 90 0.020
4 200 90 0.012
Average 0.018
NTI Prototypes Heat flux
range
Total thermal
resistivity
Number Thickness
(m)
W/cm2 cm2 C/W
1 150 80-120 0.023
2 150 80-120 0.016
3 150 90 0.020
4 200 90 0.012
Average 0.018
Microscopic Image
200 µm
NTI prototypes
Comparison with SOAs
Novel TIM – laminated layers of aligned graphite films and solder
• Thermal resistivity ~ 0.02 cm2 C/W on 150-200 µm thick prototypes at 30 Psi
• Thermal resistivity ~ 0.01 cm2 C/W on 125 µm thick prototypes at 58 Psi
Source: Teledyne Source: Teledyne
Source: Teledyne
Source: Teledyne 26 Distribution Statement A, Approved For Public Release, Distribution Unlimited
28
Remote cooling paradigm: Heat rejection to a remote fluid involving thermal conduction and spreading in substrates across multiple material interfaces with associated thermal parasitics
Limitations of Remote Cooling H
eat
• Accounts for a large fraction of SWaP-C of advanced high power electronics, lasers, and computer systems
• Stymies attempts to port advanced systems
to small form-factor applications
• Frustrates attempts to reach SWaP-C
targets for electronic systems
Limitations:
• Incapable of effectively limiting the device “junction” temperature rise
• Can not selectively target the thermally-critical devices
• Can not extract heat efficiently from 3D package
Distribution Statement A, Approved For Public Release, Distribution Unlimited
30
High Performance Computing – IBM Supercomputers
IBM ASCI Purple Supercomputer
• Built in 2005 for Lawrence Livermore National Laboratory
• Peak compute performance = 100 Tflops • 100% air-cooled
IBM Power 775
• Built in 2011 • Peak compute performance = 93 TFlops • Integral cold plate: 100% water-cooled
Metric Remote Cooling (ASCI Purple)
Integral Cooling (Power 775)
Decrease
Weight 178,715 kg 3,410 kg 52x
Volume 1800 m3 6 m3 300x
System Power 3400 kW 175 kW 20x
Source: IBM
Source: IBM
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Thermal Management Drives Computational Efficiency
31
In 2011 Japanese K-computer ~ 1 PetaFLOP/kW Rise in MFLOPS/W parallels rise in chip q” (W/cm2) • 1995-2005, CMOS q” from 1 W/cm2 to 10W/cm2 HPC
from ~1MFLOP/W to 200MFLOP/W (Blue Gene1).
Higher heat flux cooling can be used to improve HPC performance and energy efficiency
Power Consumption - Computation
Power Consumption - Communication
Distribution Statement A, Approved For Public Release, Distribution Unlimited
32
ICECool Technologies Achieve kW-level Chip Heat Dissipation
Technology Goals
• GaN MMIC PAs with 10x output power
• Microprocessors with up to 10x frequency
Areas of Focus
• Integrated Microfluidics
• Thermal Substrates and Interconnects
• Thermal Co-Design
Hea
t D
issi
pat
ed (W
)
SOA: Remote Cooling • Heat removed far from chip
• Low ∆T for heat transfer to ambient
• Limits power dissipation
• Contributes to high SWaP
ICECool: Intrachip Cooling • Heat is removed at the chip
• High ∆T for heat transfer to ambient
• Overcome SOA component thermal limits
• Reduces SWaP
Manifold Microcooler
On-Chip TECs
Evaporative Microfluidics
Source: UMD
Source: Nextreme
Serizawa and Feng (2001)
High k materials for vias/interconnects
Balandin (1999) Source: Wikipedia
Distribution Statement A, Approved For Public Release, Distribution Unlimited
33
Active Liquid Cooling • Eliminate impact on device
electrical properties due to time varying dielectric constant of liquid
High Thermal Conductivity Over-layer for Heat Removal from Topside of Devices
• High thermal conductivity in deposited material • Conformal coverage with no gaps
Embedded Thermal Vias • Micro-machined vias within ~1 micron
of junction • High thermal conductivity conformal fill
materials • Low coupling resistance for junction-to-
thermal via, thermal via-to-heat sink
Anisotropic Heat Transport • Efficient nanoscale phonon channel • Long LO phonon lifetime (3ps) • Extremely low electrical contact
resistance
High Thermal Conductivity Substrates • Integrate lattice-mismatched heat
spreaders • Eliminate thermal interface resistance • Match coefficient of thermal expansion
of electronic material
Substrate
Drain
Gate
Source ~ 1mm thickness
Thermal Management Technologies (TMT) Near-Junction Thermal Transport (NJTT)
Vision: Provide
localized thermal
management within
the device substrate
to increase Output
Power from WBG PA’s
by >3x
Distribution Statement A, Approved For Public Release, Distribution Unlimited
34
I. Passive “Thermally-Informed” Design
• Distribute functional tasks to reflect temperature distribution
• Avoid creating hot spots
II. Active Thermal Co-Design
• Functional blocks/paths and thermal elements placed in most favorable locations
• Functional blocks remapped to accommodate temperature effects
III. Fully-Integrated Thermal Co-Design
• Create passive/active thermal interconnect network
• Include local joule heating and temperature sensitivity of functionality
• optimize layout for energy consumption and functional performance
Hierarchy of Thermal-Electrical Co-Design
Source: B. Shi, A. Srivastava and A. Bar-Cohen, “Hybrid 3D-IC Cooling System Using Micro-Fluidic Cooling and Thermal TSVs “, To Appear ISVLSI, Aug 2012
Distribution Statement A, Approved For Public Release, Distribution Unlimited
35
ICECool Technologies and Challenges
Integrated Microfluidics & Thermal Substrates/Interconnects Thermal Co-Design
Evaporative Microfluidic Cooling
Thermal Vias and Integrated TECs
Microchannels and Microvalves Design of ICECool
Active Chips
Evaporative Microchannel Flow
Serizawa and Feng (2001)
Thin-film Thermoelectrics
Integrated high k vias
Challenges:
• Highly conductive thermal vias
• k > 1000 W/mK
• n > 1000 temp cycles
• TECs for hot spot cooling
• ΔT < 5 0C rise
• CoP > 2.5
• n > 1000 temp cycles
Challenges:
• > 90% vapor in exiting flow
• ΔT < 5 0C across chip
• CoP > 30 (Coefficient of Performance)
• Pressure drop < 10% Psat
SiC Microchannels
MEMS Valves
Challenges:
• Walls, channels in SiC and diamond
• < 50 micron thick
• >10:1 aspect ratio
• Microvalves:
• 10% to 90% control of maximum flow
• n > 1000 temp cycles
ICECool Design Schematic
Challenges:
• Evaluate impact of ICECool techniques on device performance
• Optimize placement and efficiency of thermal and electronic/RF features
• Achieve 10x in MMIC and microprocessor performance
CoP is defined as the ratio of heat removed to the power required to deliver the cooling.
Balandin (1999)
Source: Nextreme
Source: Texas Tech
Carter et al(2009)
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Towards a New Thermal Packaging Paradigm
36
Challenges:
• Complete the Inward Migration of Thermal Packaging
• Extract heat directly from device, chip, and package
• Place thermal management on an equal footing with functional design and power delivery
Benefits:
• Allow electronic systems to reach material, electrical, optical limits
• Reduce SWaP-C for comparable performance
• Lead the way to integrated, intelligent system co-design
Enabling Technologies:
• Microfluidics – convective and evaporative
• Thermal interconnects – active/passive
• Microfabrication – channeling, hermeticity
• Thermal Co-Design
Distribution Statement A, Approved For Public Release, Distribution Unlimited
www.darpa.mil
37 Distribution Statement A, Approved For Public Release, Distribution Unlimited
http://www.nasm.si.edu/research/dsh/artifacts/GC-CDC3800.htm http://www.cisl.ucar.edu/computers/gallery/cdc/7600.jsp
CDC Refrigerated Computers 7600 + Cyber 201/203 1971-1983
38
• Pioneering refrigeration cooled mainframe
• Refrigerant channels under large PCB’s, later Water cooled PCB’s
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Water-Cooled Mainframe/Super Computers
39
Honeywell SLIC ~ 1980
NEX SX-3 ~1990
Distribution Statement A, Approved For Public Release, Distribution Unlimited
Top Related