IC Technology What Will the Next Node Offer Us? · 8/17/2019 · 2D baseline system Off-chip DRAM...

IC Technology –What Will the Next

Node Offer Us?

H.-S. Philip WongVice President, Corporate Research, TSMC

Willard R. & Inez Kerr Bell Professor, Stanford University

MOORE’S LAWTransistors per microprocessor

Source: Karl Rupp. 40 Years of Microprocessor Trend Data.

1971 1980 1990 2000 2010 2017

MOORE’S LAW DENSITY AND COST PER FUNCTION

Source: G. Moore, Electronics, 1965

1105104103102101

Number of Components per Integrated Circuit

tive M

ring C

ponent

MOORE’S LAW IS WELL AND ALIVE

DENSITY: A NECESSARY ATTRIBUTE

1970 1975 1980 1985 1990 1995 2000 2005 2010 2015 2020

Standard cell inverter

High density SRAM

Logic gates

Transistor density (microprocessors)

IMAGINE: TRANSISTOR PERFORMANCE W/O DENSITY

• Not enough memory

• No multi-core chips

• No accelerators

• Wire delay slows big chips.

IMAGINE: TRANSISTOR PERFORMANCE W/O DENSITY

TECHNOLOGY LEADERSHIP

N7World’s first 7 nmParticipated in all the products on 7 nm

N7Best performance

Highest density

Extensive EUV layers

Design ecosystem ready

In risk production

N5 ( )P

THE ELEPHANT

IN THE ROOM

m 10-1 m

Bacteria

2 μmStrand of hair

0.1 mmTennis ball

10 cmVirus

50 nmCarbon

nanotube

1.2 nm

FinFET

Water molecule

0.28 nm

-+ Hydrogen atom

0.1 nm

10-2 m 10-3 m 10-4 m 10-5 m 10-6 m 10-7 m 10-8 m 10-9 m 10-10 m

CONTINUOUS BENEFITS NODE AFTER NODE

MOORE’S LAW – A HISTORY OF INNOVATIONS

Dennard scaling

Strained Si, high-k / metal gate

FinFET / DTCO

MULTIPLE ROADS LEAD TO ROME

Innovations

INTEGRATING CHIPS INTO SYSTEMS

It may prove to be more economical to build

large systems out of smaller functions, which

are separately packaged and interconnected.

The availability of large functions, combined

with functional design and construction,

should allow the manufacturer of large

systems to design and construct a

considerable variety of equipment both

rapidly and economically.

Source: G. Moore, Electronics, 1965

CoWoS® SYSTEM INTEGRATION

Source: 2013 TSMC Technology Symposium

TSMC CoWoS® fully assembled test chip

1 SoC + 2 DRAMs

CoWoS® SYSTEM INTEGRATION

2500 mm2 interposer:

2 processors (600 mm2)

+ 8 HBM DRAM

Integrated Si/Package Area, Reticle

SYSTEM INTEGRATION TECHNOLOGIESI/O

Package Size

rposer

GP100(Courtesy of Nvidia)

7V580THeterogeneous Integration

(Courtesy of Xilinx)

7V2000THomogeneous Integration

(Courtesy of Xilinx)

XCVU440(Courtesy of Xilinx)

GV100(Courtesy of Nvidia)

CHIPLETS INTEGRATION REDUCES SYSTEM COST PER FUNCTION

PC / Internet Mobile AI / 5GMini-ComputerTransistor Radio

SEMICONDUCTOR TECHNOLOGY EVOLVES

DRIVEN BY CHANGING APPLICATION LANDSCAPE

Invention of point-contact transistor

Transistor Scaling Principle

Intel 4004

Invention of IC

Pentium CPU

1995Flash Memory

1984Mobile phone

iPhone

FinFET

7nm FinFET

GPU (21B Transistors)

5nm CMOS

2050 and beyond

Memory Compute

Deep Learning Accelerators

Intel performance counter monitors 2 CPUs, 8-cores/ CPU + 128GB DRAM

DATA MOVEMENT HITS THE MEMORY WALL

ABUNDANT-DATA APPLICATIONS: ENERGY MEASUREMENTS

Source: S. Mitra (Stanford)

…ResNet-152

AlexNet

Language Model

(LSTM)

Network(application)

Type(LSTM/ CNN)

Training/ Inference

Model SizeMemory Usage

(GBytes)

ResNet(vision)

CNNTraining

120 MBytes21*

Inference 0.12

LanguageModel(NLP)

LSTMTraining

2.5 GBytes40*

Inference 2.5

* Training memory usage: Batch size 64, word size 64-bit, memory can increase with greater batch sizes, footprint of activations, weights, errors and gradients.

Source: M. Lee, W. Hwang, Prof. S. Mitra (Stanford), M. Aly (NTU, Singapore), Y. Wang, K. Akarvardar (TSMC)

DEEP NEURAL NETWORKS

REQUIRE LARGE MEMORY CAPACITY

ON-CHIP SRAM CAPACITY:

NEVER ENOUGH

Launch Year

2006 201820122009 2015

Intel Xeon X5355 NVIDIA Tesla K40

NVIDIA Tesla V100

Intel Xeon E7-8890 v4

3.8 Gbytes

1.4 nm node

Source: W. Hwang, Prof. S. Mitra (Stanford)

CAN WE PUT LOTS OF MEMORY ON-CHIP?

WHAT KINDS OF MEMORY, FOR WHICH APPLICATION?

Source: “Inside Volta” , Nvidia GPU Tech. Conf. , May 10, 2017.

Heterogeneous Integration:GPU + High Bandwidth Memory (HBM2)

CoWoS Module

Superior processing power that equals to 100 CPUs

>300 B transistors

SUPER AI ACCELERATOR ENABLED BY CoWoS®

HBM2HBM2

COMPUTE-MEMORY INTEGRATION

Off-Chip DRAM

Si Logic Die

Printed Circuit Board

Limited I/O Connectivity

2D System (traditional baseline)

Source: W. Hwang, W. Wan, Y. Malviya, H. Li, M. Lee, M. Aly, H.-S. P. Wong, S. Mitra. Work in progress 2017 – 2019 w/ TSMC

2.5D System HBM-Type DRAM

Si Logic Die

Si InterposerMicron Scale Connectivity

HBM-Type DRAM

Si Logic Die

TSV + µBump Connectivity(Micron Scale)

3D TSV System

N3XT SystemHigh Density On-Chip Nonvolatile Memory

Dense ILV Connectivity(Nanometer Scale)

Si Logic Die

Energy Efficient Logic(Thin Device Layers)

High Speed On-ChipNonvolatile Memory

Energy Efficient MemoryAccess Transistors

Nonvolatile Memory Cells

Bottom Electrode

Top Electrode

oxide isolation

switching region

phase change material

PCMPhase change

memory

filament

oxygen ion

Top Electrode

Bottom Electrode

metaloxide

oxygen vacancy

RRAMResistive

switching random access memory

filament

Bottom Electrode

solid electrolyte

Active Top Electrode

metal atoms

CBRAMConductive

bridge random access memory

STT-MRAMSpin torque

transfer magnetic random access

memory

FERAMFerro-electric

random access memory

Ferroelectric layer

Interface Layer

top gate

Source: H.-S. P. Wong, S. Salahuddin, Nature Nanotech (2015)

“NEW” MEMORIES FOR COMPUTE-MEMORY INTEGRATION

Soft Magnet

Pinned Magnet

tunnel barrier (oxide)

current

Random access, non-volatile, no erase before write, on-chip integration

Source: Stanford/NTU: M. Aly, S. Mitra, TSMC: Yih (Eric) Wang, K. Akarvardar, 2019

2D baselinesystem

AcceleratorCores

SRAM on-chip memory

NEW MEMORY: HIGH-BANDWIDTH,

HIGH-CAPACITY, ON-CHIP

2D baselinesystem

Off-chip DRAM (LPDDR3)• Capacity: 4 GBytes• Latency: 50 ns• BW: 12 GBytes/s• Read/write energy: 17 pJ/bit

AcceleratorCores

SRAM on-chip memory

2D baselinesystem

AcceleratorCores

SRAM on-chip memory

New system

AcceleratorCores

SRAM on-chip memory

Off-chip DRAM (LPDDR3)• Capacity: (4 GBytes minus New Mem. Cap.)• Latency: 50 ns • BW: 12 GBytes/s• Read/write energy: 17 pJ/bit

2D baselinesystem

AcceleratorCores

SRAM on-chip memory

New system

High Bandwidth, High Capacity both critical

AcceleratorCores

SRAM on-chip memory

Off-chip DRAM (LPDDR3)• Capacity: (4 GBytes minus New Mem. Cap.)• Latency: 50 ns • BW: 12 GBytes/s• Read/write energy: 17 pJ/bit

On-chip New memory• Capacity: sweep (up to 4 GBytes)• Latency: sweep (down to 3ns)• BW: sweep (up to 128 GBytes/s)• Read/write energy: 5 pJ/bit

5 ns memory access latency, 5 pJ/bit access energy

EDP benefits

NEW MEMORY ESSENTIAL REQUIREMENTON-CHIP CAPACITY MUST EXCEED DATA SIZE

Language model (LSTM)

2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

ytes/s

4.2x1.3x

1.3x 2.9x - 3.6x

64 GBytes/s

Capacity1 MByte 4 GByte

Capacity

50x2.1x – 8x

2.1x – 8x1

5x –

yte100 GBytes/s

1 MByte 4 GByte

2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

ytes/s

Capacity (MBytes)

1024 2048 3072 4096

ytes/s

Capacity (MBytes)

1024 2048 3072 4096

NEW MEMORY ESSENTIAL REQUIREMENTON-CHIP CAPACITY MUST EXCEED DATA SIZE

EDP benefits

5 ns memory access latency, 5 pJ/bit access energy

5 pJ/bit access energy

EDP benefits

NEW MEMORY ESSENTIAL REQUIREMENTHIGH BANDWIDTH MORE CRITICAL THAN LATENCY

2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

ytes/s

4.2x 4.1x

1.1x - 3x

64 GBytes/s

Latency (ns)3 50

Latency (ns)

50x 20x – 35x

1.1x – 20x

100 GBytes/s

2.5 GByte data size

ResNet-152 (CNN)

120 MByte data size

EDP benefits

5 pJ/bit access energy

ytes/s

Latency (ns)

5 10 40 80

ytes/s

Latency (ns)

5 10 40 80

NEW MEMORY ESSENTIAL REQUIREMENTHIGH BANDWIDTH MORE CRITICAL THAN LATENCY

Energy ✕ Execution Time

525X320X

159X63X

Lang.Model (LSTM) AlexNet (CNN) Captioning (LSTM) ResNet152 (CNN) VGG19 (CNN)

enefits

N3XT Benefits: relative to 2D Baseline System (28nm silicon CMOS, LPDDR3)Inference: 16-bit data, batch size of 1

Workload: Inference on ML Accelerator

N3XT: UP TO ~2,000X

ENERGY EFFICIENCY BENEFITS

Source: Stanford/NTU: M. Aly, T. Wu, A. Bartolo, H.-S. P. Wong, S. Mitra et. al., Proc. IEEE 2019

N3XT SYSTEM

High Density On-Chip Nonvolatile Memory

Si Logic Die

N3XT SYSTEM

High Density On-Chip Nonvolatile Memory

Si Logic Die

1D carbon nanotube (CNT)

2D TMD (MoS2, WSe2, WS2…)

Source: S.-K. Su, … L.-J. Li (TSMC), Nature Nanotech., 2019.

Photo credit: B. Radisavljevic et al., Nature Nanotech., p. 147, 2011

NANOMETER-THIN TRANSISTOR CHANNEL

< 1 nm

10,000

0 1 2 3 4

Channel thickness (nm)

WSe2 Si

Filled: electronOpen: hole

Photo credit: User Mstroeck on en.wikipedia

2D LAYERED MATERIALS (WS2, WSe2)

MoS2-e0.5

0.110 100 1000

Effective m

Mobility (cm2/V-s)

Source: C.-C. Cheng et al. (TSMC), Symp. VLSI Tech. 2019

WSe2-h ION (μA/μm)

20 nm 10 nm

Classical FET

5 nm Gate Length

SHORT-CHANNEL

CARBON NANOTUBE TRANSISTORS

10 nm Gate Length

Source: C. Qiu,…L-M. Peng (PKU), Science, 2017

VDS = -0.4V VDS = 0.4V

SS = 70 mV/Dec 70 mV/Dec

-1.0 -0.5 0.0 0.5

Vgs (V)

-1.0 -0.5 0.0 0.5

Vgs –Vt (V)

VDS = -0.1 V

SS = 73 mV/Dec

Lg = 5 nm

CARBON NANOTUBE COMPUTER

Source: M. Shulaker,… H.-S. P. Wong, S. Mitra (Stanford), Nature, 2013

instruction fetch

arithmetic block

data fetch

write-back

Kbit 6T SRAM (6144 CNFETs)

CARBON NANOTUBE FET CMOS SRAM

Source: P. Kanhaiya,… M. Shulaker (MIT), Symp. VLSI Tech., 2019

MEMORY INTEGRATION

ON LOGIC PLATFORM

Better transistor alone

Transistors integrated with memory in 3D

MEMORY INTEGRATION

ON LOGIC PLATFORM

SYSTEM INTEGRATION

A CONTINUUM FROM FAR BACK-END TO FRONT-END

Source: IMEC

Interposer Chip-on-waferWafer-on-wafer

Monolithic 3D

SOCIETAL NEEDS FOR ADVANCED

TECHNOLOGY IS INSATIABLE

ADVANCED TECHNOLOGY

− A KEY DIFFERENTIATOR

Continuous transistor & memory advances

Memory logic integration

System integration with high connectivity

A CALL TO ACTION: EARLY ENGAGEMENT

SYSTEM ↔ TECHNOLOGY

ACADEMIA ↔ INDUSTRY RESEARCH

End of Talk

Questions?

Continuous transistor & memory advances

Memory logic integration

System integration with high connectivity

COMMITTED TO PROVIDING THE MOST

ADVANCED TECHNOLOGIES

IC Technology What Will the Next Node Offer Us? · 8/17/2019 · 2D baseline system Off-chip DRAM...

Documents

Transcript of IC Technology What Will the Next Node Offer Us? · 8/17/2019 · 2D baseline system Off-chip DRAM...

ProactiveDRAM: A DRAM-initiated Retention Management ......DRAM cell DRAM row sense amp DRAM bank subarray wordline bitline Fig. 2. DRAM internal bank architecture. banks. A rank can

Co-Architecting Controllers and DRAM to Enhance DRAM ...camelab.org/uploads/Main/Co-Architecting Controllers and DRAM to... · Co-Architecting Controllers and DRAM to Enhance DRAM

TPS51116 Complete DDR, DDR2, DDR3, DDR3L, LPDDR3 … · TPS51116 Complete DDR, DDR2, DDR3, DDR3L, LPDDR3 and DDR4 Power Solution Synchronous Buck Controller, 3-A LDO, Buffered Reference

ARCHITECTURAL TECHNIQUES TO ENHANCE DRAM …research.ece.cmu.edu/safari/thesis/ykim_defense_slides.pdfCO-DESIGN: CPU & DRAM CPU Memory Management DRAM Controller DRAM Arch & Interface

DRAM Lecture2

Micron DRAM Products Overview - nxp.com · August 27, 2013 ©2012 Micron Technology, Inc. | 7 Type LPDDR(1) LPDDR2 LPDDR3 DDR2 DDR3/DDR3L DDR4 Die Density Up to 2Gb Up to 8Gb Up to

Mobile LPDDR3 SDRAM - Pine64files.pine64.org/doc/rock64/SPECTEK_178B_32GB_V91M_MOBILE_LPDDR3.pdf · 32Gb: 178-Ball, 4x8Gb Mobile LPDDR3 SDRAM Features ® ‡Products and specifications

LPDDR3/4-ECC DRAM for High-reliability - IoT, Automotive ... · PDF fileTitle: LPDDR3/4-ECC DRAM for High-reliability - IoT, Automotive and Control System Applications Author: Wolfgang

Gather-Scatter DRAM: In-DRAM Address ranslationT …people.inf.ethz.ch/omutlu/pub/GSDRAM-gather-scatter-dram...Gather-Scatter DRAM: In-DRAM Address ranslationT to Improve the Spatial

Gather-Scatter DRAM - Carnegie Mellon Universityusers.ece.cmu.edu/~omutlu/pub/GSDRAM-gather-scatter-dram... · 2015. 12. 30. · Gather-Scatter DRAM In-DRAM Address Translation to

Solar-DRAM: Reducing DRAM Access Latency by Exploiting …...reads to DRAM cache lines containing “weak” or “strong” cells, and 2) writes to all of DRAM. We evaluate Solar-DRAM

Design considerations for LPDDR4/3 PHY and controller … dual mode concept LPDDR3 mode (single channel, 32 byte access) MC CMD queue, Bus Fabric x16 /x32 LPDDR3 x32 DRAM 16 DQ 16

D9030DDRC DDR3 and LPDDR3 Compliance Test Application …

World’s First LPDDR3 Enabling for Mobile …...World’s First LPDDR3 Enabling for Mobile Application Processors System Contents Introduction Problem Statements at Early mobile platform

Gaining Insight into DDR3/4 and LPDDR3/4 - keysight.com · Gaining Insight into DDR3/4 and LPDDR3/4 Alan Gosselin Senior Field Applications Engineer Digital Verification Tools April

8Gb LPDDR3 (x32) 16Gb LPDDR3 (x32) - linux-sunxi · 2017-03-10 · ORDERING INFORMATION 8Gb,16Gb: 178-Ball, LPDDR3 SDRAM Part Number Memory Combination Operation Voltage Density Speed

CM3288 Hardware User Manual · 2020. 11. 5. · PMU POWER RK3288 DRAM SPI2 MCP2515 Download 1GB€RJ45 CODEC ES8388 HP MIC WIFI/BT MDP€OUT Camera€1 OV13850 13MP LPDDR3 2GB Camera€2

Memory Scaling : A Systems Architecture PerspectiveSolution 1: Tolerate DRAM ! Overcome DRAM shortcomings with " System-DRAM co-design " Novel DRAM architectures, interface, functions

216b 16gb 2c0f Mobile Lpddr3

event.asus.com.cnevent.asus.com.cn/HK/2018/ASUS_PC_1805.pdfFHD IPS TOUCH 16GB LPDDR3 512GB PCIE G3x4 SSD HD UX370UA-C4196T Core i5-8250U 8GB LPDDR3 256GB M.2 SSD ...