7/26/2019 c Mosaic 2010

1/20

3D Stacked Architectures with

Interlayer Cooling - CMOSAIC

Prof. John R. Thome, LTCM-EPFL, Project Coordinator

Prof. Yusuf Leblebici, LSM-EPFL

Prof. Dimos Poulikakos, LTNT-ETHZ

Prof. Wendelin Stark, FML-ETHZ

Prof. David Atienza Alonso, ESL-EPFL

Dr. Bruno Michel, IBM Zrich Research Laboratory

7/26/2019 c Mosaic 2010

2/20

CMOSAIC: Technological Aims

CMOSAIC aims to make an importantcontribution to the development of the

first 3D computer chip with a

functionality per unit volume that nearly

parallels the functional density of ahuman brain.

A 3D computer chip with integratedcooling system is expected to:

-Overcome the limits of air cooling

-Compress ~1012nanometer sizedfunctional units (1 Tera)

into one cubic centimeter

0 2 4 6 8

Wire Length (mm)

100

1000

10000

100000

1000000

WireCountYield 10-100 fold higher connectivity

-Cut energy and CO2

emissions drastically

2

7/26/2019 c Mosaic 2010

3/20

CMOSAIC: Project Objectives

3

7/26/2019 c Mosaic 2010

4/20

Imagine the world of the future: Energy consumption by datacenters and mainframe

computers is increasing at a rate of ~15% annually.

Total consumption is currently about 2% of total electricalconsumption in USA.

Worlds largest supercomputer currently consumes about14MWe.

New electrical production is increasing at a rate of about 1%

annually.

In 2090, allelectrical energy in USA will be consumed byone supercomputer!

Solution: 3D-ICs will consume much less energy!!!

3D-ICs: Solution to 2D-IC Energy Crisis?

4

7/26/2019 c Mosaic 2010

5/20

TSV Electrical Interconnects for 3D Chips Stacks

MSc Student: Lihui Yang, Ph.D. Students: Fengda Sun, Michael Zervas

(Top view after via etching)

Polymer passivation on the TSVside walls

No need for photolithography on

the CMOS metal

Difficulty of uniform

filling for deep trenches

(Top view after via etching)

Inorganic dielectric passivation on theTSV side walls

Passivation on the metal has to be

removed

Difficulty of conformal

deposition for high aspectration vias.

Annular Through-Silicon-Via Circular Through-Silicon-Via

Annular ring etching

and polymer filling

100umDRIE

20um

KOH

DRIE and KOH via

etching

Development of process technologies for annular and circular TSVs

Fabrication of test wafers for TSV characterization (daisy chain)

Wafer thickness: 200-500um , Via Diameter: 40-100um

5

7/26/2019 c Mosaic 2010

6/20

Fabrication of the Test Vehicles for Single- and

Two-Phase Liquid Cooling of 3D Chip Stacks

Composed of a background heater, hotspot heaters, temperaturesensors and backside microchannels on each layer.

M.Sc. Students: Will iam Cesar, Gozde Toral, Ph.D. Student: Yuksel Temiz

Basic Process Steps

Metal evaporation

and lift-offSi

SiO2

PAD

Oxide

sputtering

Back-side DRIE

for micro-channels

Front-side DRIE

for liquid inlet/outletopenings

A

A

A-A (Side view)

CHANNELOPENING

OPENING

Front-side

Back-sideComplete test vehicle mounted on a

printed circuit board with fluid manifolds

and connections. (IBM-ZLR / EPFL-LSM) Pin model with periodic fluid flow 6

7/26/2019 c Mosaic 2010

7/20

Fabrication of the Test Vehicles for Two-Phase

Cooling of 3D Chip Stacks

Ph.D. Student: Yuksel Temiz

EPFL-LTCM / EPFL-LSM

Sealed test device for two-

phase micro-channel coolingexperiments with front-side

hotspot heaters, completely

designed and manufactured at

EPFL (LSM / LTCM).

Channel dimension: 50umwidth, 100um depth

7

7/26/2019 c Mosaic 2010

8/20

2D Multi-Microchannel Flow Boiling Experiment

Test section

(1) Silicon multi-microchannel evaporator with

channel sizes from 100 to 50 microns or even

less (50 to 100 parallel channels).

(2) Silicon cover plate with inlet and outlet slitsfor flow stability and copper manifold.

(3) Edge connector for electrical connections.

(4) Test element with 1cm2 heater.

(5) Transient IR temperature measurements.

Goals

Create first experimental database for very

small multi-microchannel evaporators.

Carry out high-speed 2D flow visualization

using high speed IR and video cameras. Advance our prediction methods.

Channels

Inlet slit Outlet slit

1 cm

5

1 2 3 4

4

Ph.D. Student: Sylwia Szczukiewicz

8

7/26/2019 c Mosaic 2010

9/20

Two-Phase Cooling of 3D Stacked Processors

Test section

(1) Stacking solder bump technology as

interconnections.(50 um diameter, 100 um pitch)

(2) Through Silicon Via technology electrical

conduction

(3) Interlayer cooling solder bumps as pin fins

(4) 2D backside coolercombined with

interlayer

avoid CHF(5) 2D results to characterize cold plate copper

microchannels680 um high, 100 um wide, 72

um fins with R134a fluid

Goals

3D technology packaging solder bumps

dimensions, shape & distribution

TSVs LSM EPFL combination

3D stack tests with low pressure refrigerants

h, pressure drop, CHF Cooling capabilities

3 4

Ph.D. Student:Yassir Madhour (LTCM at IBM)

9

Ti St i I P i f E ti

7/26/2019 c Mosaic 2010

10/20

Time Strip Image Processing of Evaporating

Flow in Microchannels: Fluid Mechanisms

Time strip (a) made from channel video sequence (b) showing nucleation in an

evaporating film rewetting a dried out region, forG= 100kgs1m2, q= 26 Wcm2

andx= 70% in 200 micron wide channel (Int. J. Heat Mass Transfer, 2010)

Post-Doc: Dr. Borhani, LTCM (Vice-Coordinator of CMOSAIC)

10

7/26/2019 c Mosaic 2010

11/20

Numerical Simulation of Microscale

Two-Phase Flows

Development

(1) Two-Phase 3D finite element mesh.

(2) Continuum surface tension model (Brackbill

1992).

(3) Comparison of surface representations. The righthand figure shows points over the surface, leading to

reducted mass conservations erros.

(4) Test case: surface tension bringing the perturbed

geometric shape to its smallest surface.

Goals

To create a 3D code using state-of-art

techniques such as Arbitrary Lagrangian

Eulerian and DNS.

Predict two-phase flow in complex geometries. Design tool for micro evaporators.

3 4

Ph.D. Student: Gustavo Rabello dos Anjos

(2)

(3)

(1)

(4)

11

E Effi i t M lti i bl C t l f 3D IC

7/26/2019 c Mosaic 2010

12/20

Joint flow rate and task migrationFixed flow rate

Reactive:Scheduler sends the new job to the

coolest core for balancing temperatureProactive:

Temperatureprediction Thermalbalancing among the cores based on the

forecastVariable flow rate

T 80C Increment flow rate setting

T < 80C Decrement flow rate setting

95%hotspot

reduction

ThermalRepistory

UtilizationRepistory

Controller L1

L3L2

Task migrator

CompressorActuator

Inference Engine

For Prediction

Environment sensing

Multivariable Takagi-Sugeno flow rate and DVFSFuzzy controller

Discrete (limited) VF settings and flow rate values.

Implies minimal Fuzzy rule base.

Reactive or Proactive implementations.

Output is limited between minimal and maximum

actuating values.Adaptability of Fuzzy control.

Possibility of added task migration triggering signal.

Future Work:

Energy Efficient Multivariable Control for 3D-ICs

With Liquid and Two-Phase Cooling

Ph.D. Student: Mohamed M. Sabry

12

F t C t T i t Th l M d l (CTTM)

7/26/2019 c Mosaic 2010

13/20

Finite Difference Approximation for a

microchannel Thermal Cell

RqTTAucx

TTTk

t

TRc SSyyavp

ki

iiiip

12

,

2

11 2

0.18 0.2 0.22 0.24 0.26 0.2835

35.5

36

36.5

37

37.5

38

38.5

39

Time (s)

TemperatureoC

Proposed Model

CFD Simulation

Microchannel and the silicon surrounding it are

divided into thermal cells

a FD approximation-based RC circuit is created-

similar to conventional compact modeling

Voltage controlled current sources model

convectional transport of heat downstream

Correlation-based Nusselt numbers are used to

model heat transfer from Silicon walls into thecenter of the fluid

Compact: very small problem size for large diesTransient: gives temperature transients

Fast: speed-ups of up to 950x compared to CFD

R

x

zy

S1

S2

Future Work:Parallelize the model on CPU/GPU

Incorporate two-phase flows

Fast Compact Transient Thermal Model (CTTM)

for Microchannels

Ph.D. Student: Arvind Sridhar

13

7/26/2019 c Mosaic 2010

14/20

Superhydrophobic Surfaces

Approach

(1) Creation of a nanostructure (metal oxide)

(2) Surface functionalization of the created

structure (fluorosiloxane, lauric acid, )

Goals

Production of a highly hydrophobic surface

to reduce the pressure drop in microchannelused for water cool ing system

3 4

Ph.D. Student: Michael Rossier

Constraints

(1) Stable over a long time period

(2) Simple manufacturing in microchannel

Water droplet on a functionalized

nanostructure

Nanostructure (cobalt oxide) 14

7/26/2019 c Mosaic 2010

15/20

T. Brunschwiler,

IBM

Single cavity test vehicle for

microfluidics experiments

Micro-channels Pin-fins

Preliminary results

Planned: Measure and evaluate Post-transition velocity and vorticity

Resolve transient flow

Pressure drop correlation

Thermal measurements

Hot processors

Cooling structures

3D chip stack

Water flow

Pin-fin

Pin-finSEM image of pin-fins

Pre-transition

-particleimage

velocimetry

Measured pressure

drop

Transition in gradient

at high flow rates

Single Phase Water Cooling - Experiments

15

Ph.D. Student: Adrian Renfer

7/26/2019 c Mosaic 2010

16/20

3D chip-stack modeled as porous medium with thermal

non-equilibrium for computation efficacy and local hot

spot capturing

Junction and hot-spot temperatures

for different layers validated against

experiments

Alfieri et al., Journal of Heat Transfer, (2010), submitted.

Transient modeling and parametric study of cooling performance underway

Top

Middle

Bottom

Lines: Junction temperature

Filled symbols: Simulations

Open symbols: Measurements

Single Phase Water Cooling - Modeling

16

Ph.D. Student: Fabio Alfieri

Water/Silica Interface with Surface Functionalized

7/26/2019 c Mosaic 2010

17/20

heat sink

heat source

heat currentheat current

water SAMs silica SAMs water

Molecular dynamics simulation

A temperature gradient across the simulation domain is established by imposing1D heat flux.

The interfacial thermal conductance is obtained for both silica/SAMs andSAMs/water interfaces.

The results show that using SAMs can double the water/silica interfacialconductance.

Hu et al., App. Phys. Lett., (2009), 95, 151903; J. Heat Transfer, (2010), submitted.

2nm

Water/Silica Interface with Surface Functionalized

by Self-Assembled Monolayers (SAMs)

17

Ph.D. Student: Ming Hu

7/26/2019 c Mosaic 2010

18/20

Parameter range of interest:

Heat flux: hot-spot 250W/cm2,background 50W/cm2

Through-silicon-via (TSV) pitch: 50

200m

Fluid channel dimensions:

limited by TSV fabrication 25-100mPower map TSV floorplan

Design input parameters:

Interconnect compatible

heat transfer structures

Two-phase dielectric refrigerant cooling:

solder ball stack possible

Interlayer cooling architecture:

Single-phase water cooling:

cavity with elecrtical insulation

3D-IC System Specifications Defined

Thomas Brunschwiler and Stephan Paredes

IBM Zurich Research Laboratory | 15-Jun-16 IBM Research 2010 18

7/26/2019 c Mosaic 2010

19/20

Modeling concept validated with experimental results (+/- 10% accuracy)

Extracted parameters can be used in transient compact thermal models directly

Single-Phase Liquid Multi-Scale Modeling Validated

Thomas Brunschwiler and Stephan Paredes

IBM Zurich Research Laboratory | 15-Jun-16 IBM Research 2010 19

7/26/2019 c Mosaic 2010

20/20

IBM Zurich Research Laboratory | 15-Jun-16 IBM Research 2010 20

Eutectic thin film solder bonding of pin fins Pyramid chip stack with 4 cavities and 3 tiers, utilizing wire-bonding for IO

Stack alignment and bonding, using fl ip-chip bonding toolModification to reflow under reducing atmosphere

Bonding Technology to Build Thermal Prototypes

Thomas Brunschwiler and Stephan Paredes