1

The Intimate Integration of The Intimate Integration of The Intimate Integration of The Intimate Integration of The Intimate Integration of The Intimate Integration of The Intimate Integration of The Intimate Integration of

Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for Photonics and Electronics for

Computing and Switching Computing and Switching Computing and Switching Computing and Switching Computing and Switching Computing and Switching Computing and Switching Computing and Switching

SystemsSystemsSystemsSystemsSystemsSystemsSystemsSystems

A. V. Krishnamoorthy

Acknowledgements:

- My colleagues at:- Bell Laboratories- AraLight- Sun Microsytems

2

Outline

• Applications> Architectures that challenge electrical interconnects

> Parallel optical interconnects in the marketplace

• Technology

> “Intimate” integration of lasers, detectors, and VLSI electronics

> Progress and Performance

• First Product: High-density Transceivers

> Challenges

> Performance & Reliability

• Interconnects to the chip: optoelectronic switching

> Architecture

> System Integration

3

10,000km

1000km

100km

10km

1km

100m

10m

1m

10cm

SM, CWDM

SM or MM, Serial or Parallel

SM, DWDM

Multi-mode, Parallel

Metro, access, cross-campus

Across central office, data centers

To the box

To the chip/package

$10,000

$3,000

$1,000

$300

$100

$30

$10

$3

1980 1985 1990 1995 2000 2005 2010 2015

Trans-oceanic

Cross-country

SM, DWDM or MM, Parallel

1Mbps 10Mbps 100Mbps 1Gbps 10Gbps 100GbpsLink Distance

Transceiver C

ost (p

er G

bps)

Year of Introduction

Bandwidth per fiber

Penetration of optics into communications

4

Journal of Parallel & Dist. Processing, Vol. 41, pp. 109-114, 1996

Data “Firehoses” stress the interconnect sub-system

• Where do data “firehoses” exist in systems today?

• What types of data firehose are difficult to implement with electrical interconnects?

5

External data firehose

> Gather Data Directly from Sensors– e.g., digitized radar, fusion of multiple data sources, memories,

> Combine Multiple Smaller “Tributary” Streams – Telecom switching systems

– Datacom switching (10m -to-5km Fiber “home-runs”)

Terabit/s Terabit/s

Terabit/s Terabit/s

– Distribute “Tributaries” throughout the system (e.g. multistage switching networks)

6

> Replicate Input Data for Parallel Internal Distribution and Processing

– e.g., matrix-vector multiplication with fixed (or infrequently changing) matrix

– switching (with fanout - crossbar)

– matrix inversion

– artificial neural networks

– clock distribution

Terabit/s

O[N] O[N2] O[N]

Internal firehose

7

> Compare input data to internal database with fast repetitive processing

– recirculating internal fixed or slowly varying database

– modest input and output data rates

– incoming data matched to contents of recirculating data

– e.g.,content-based search, information retrieval, data “mining”

Terabit/s

Recirculating firehose

8

D. A. B. Miller and H. Ozaktas, J. Par. Dist. Comp., Vol. 41, 1997

Electrical signaling favors small aspect ratios

9

(Increasing System Aspect Ratio)

System architecture

• Chip-to-Chip

• MCM-to-MCM

• Board-to-Board

• Frame-to-Frame

• Cabinet-to-Cabinet

• Chip-to-Chip

•• BoardBoard--toto--ChipChip

•• FrameFrame--toto--ChipChip

•• CabinetCabinet--toto--ChipChip

Conventional interconnect hierarchy is designed to minimize aspect ratio

Certain applications (e.g switching) are naturally characterized by large aspect ratio

(Optical interconnects in use today)

10

processors

or boards

duplex optical

fiber ribbon links,

probably 32 wide

Point-to-point fully-connected system

16-node System

11

processors

or boards

Optoelectronic/VLSI

switching chip

duplex optical

fiber ribbon links,

probably 32 wide

fiber bundle

Switched interconnection system

16-node System

12

Backplane CDR SwitchingParallel FabricOptics

Line Card Switch Card

600m

Optical

interconnect

Network

Processor

Transceiver Overhead Transceiver Backplane

SerDes Processing SerDes Parallel

CDR Framer/Mapper CDR Optics

Client Side Serial/ParallelOptics (VSR)

Optics “in” the box today

Line Card Applications

Current Products: Smart Transceivers

Future Products: “Optics to the chip”

Switch Card Applications

13

2-Dimensional ArrayIntegration Technology Platform

DISCRETEvia Wire Bonding

(Traditional Vendors)

VLSI Communications and Switching Chips

Potential Benefits to Direct Integration:

• Higher speed interconnect (capable of >40 Gbps)

�Lower power consumption

�Smaller form factor

�Better performance – jitter, crosstalk, EMI

�More reliable/higher yield process than wire bonding

�Only Proven method to integrate III-V materials with Silicon VLSI circuits

Photonics integrated with CMOS

14

Silicon

Monolithic

GaAs devices

on Si Wafer

Si transistor,

process GaAs

GaAs + Silicon

GaAs

Monolithic

InP

Monolithic

Monolithic

Epitaxial

Lift Off

Epoxy/Polyimide

Bonding

Flip Chip

Bonding

Superstrate

Bonding

Hybrid

III - V EPI

Si Electronics

Fusion Bonding

Opto-electronic integration choices

15

OptoOptoOptoOpto----electronic integration examples electronic integration examples electronic integration examples electronic integration examples

Small size of integration – can’t manufacture with a full wafer scale integration

Process optical and electronic devices separately; integrate by heating to 400 deg.

UCSB, Agilent Labs,

Bell Labs

Fusion Bonding

III-V substrate removal process. Must remove heat through Silicon chip

Flip Chip Bonding to Silicon circuits, then substrate removal

Plessey, GEC Marconi, Aralight, Teraconnect, CSU, ASU, Vixel

Flip Chip Bonding

Peregrine Semiconductor

NTT, CSU

Honeywell, Sanders, Martin Marrieta

Georgia Tech, UCSB,

ASU

Several Startups

MIT , Bell Labs,

European Union Research Teams, Bell Labs,

Univ. Rochester, UCLA, UCSD, MIT, Intel, Cornell, Columbia, Luxtera, Kotura, IBM, HP, Sun

R&D Teams

Many processing steps after attachment of GaAs VCSEL to Silicon; yield; limited ability to optimize VCSEL characteristics

Flip Chip bond GaAs epitaxial layers to silicon circuits using polyimide or epoxy, then remove substrate, finish processing VCSEL mesa, then process contacts

Epoxy/Polyimide Bonding

Thermal characteristics. Custom silicon foundry. Flip-chip Bonding to transparent Sapphire substrate containing circuits (SOS process)

Flip Chip Bonding to Silicon on Sapphire

Two bonding operations. Glass slide induces stress. Need vias through GaAs substrate – high parasitics (advantage – use top emitters)

How to manipulate very thin membranes. Need large (high capacitance) bonding areas

Very low yield. Not demonstrated in lasers.

yield and uniformity

LEDs demonstrated, not demonstrated in lasers

Si Foundry must accept GaAs impurities into its lines.

High temperature processing of GaAs devices

CMOS compatibility of devices, process integration, and electronics integration

Challenges

Bond photonic device to glass slide through which light emits after flip chip

Superstrate Bonding

Remove active membrane of photonic device and bond to circuit chip

Epitaxial Lift Off

HBT – PIN detectorInP Monolithic

FET-PIN detector (field effect transistor + PIN detector)GaAs Monolithic

Deposit GaAs photonic devices on Si VLSI wafer. Push wafer through Si Foundry. Or make Si first and then go through GaAs fab.

GaAs + Si

Optical emitters, detectors, modulators, and WDM components in SOI silicon

Silicon SOI

DescriptionProcess

16

Anti-Reflection coating

epoxy

n GaAs +

silicon

i MQW

GaAs substrate

p AlGaAs Stop Etch

Optical ChipOptical Chip

Electronic ChipElectronic Chip

Micro-bump

Example: Flip-chip photonics-on-silicon integration

Before Bonding

After Bonding &

Substrate Removal

K. W. Goossen et al., IEEE PTL, Vol. 5, 1993

17

A. Thermal Compression Bonding

1) low bond temperature

2) Smaller CTE effects

3) Increased choice of materials

4) No reflow solder steps (flux high, temperature, self alignment)

5) lead free bumps : Arbitrary bond materials

B. 10µm bump diameter

1) Lower capacitance and inductance

2) parasitic > 80 gHz

3) can contact individual device geometries

4) no limit to pitches > 10 µm

5) use of multiple dummy bumps –local thermal management

A unique flip-chip technology

18

Multiple flip-chip attachments are possible

144µm

• Flip chip bonding followed by substrate removal

• Multiple operations enable interleaved arrays

• Laser and photodetectors can be separately optimized

19

2-Dimensional Array Technology Platform

� Micro-bump technology with 4X reduction over conventional C4

• Higher speed interconnect (capable of >40 Gbps)

� due to inherently lower electrical parasitics

�RC time constants <10 femtosec

• Lower power consumption

� due to removal of wire-bond pads and reduction of off-chip parasitics

• Smaller form factor

� single optoelectronic die versus multiple dies wire-bonded to each other

� Integration of 2-D array of lasers versus single row

� Integration of additional electronic functionality into optoelectronic die

• Better performance – jitter, crosstalk, EMI

� removal of inductive wire-bond from integration (no antennae pick up)

• More reliable/higher yield process than wire bonding

� single step wafer-level integration versus individual die-level wire-bond

• Potential to use guided-wave or free-space optical communication

VCSELs and Photodetectors

directly attached to

arbitrary VLSI circuits

Advantages of integration

20

Micro-bump flip-chip roadmap

1985 1990 1995 2000 2005 2010 2015

10-1

100

101

Si ScalingLine width ( µ m )

Time (year)

100

101

102

Aralight Bump

Bump Diameter ( µ m )

Bump Diameter

C4 process

Si LineWidth

Linewidth

(microns)

Bump Diameter (m

icrons)

• Can efficiently contact VCSELs, modulators, and PiNs

• Only interconnect solution that follows Silicon VLSI trends

21

Manufacturing platform

Wafer-Level

Photolithographic

Micro-bump Bonding

III-V Wafer

Silicon Wafer

(diced)

Results in

a Wafer-Level Single-Step Single Chip and/or Array:

opto-electronic integration,

optical alignment and packaging,

and testing

which enables

a lower cost manufacturing platform

for

single channel, linear array, and

2-D array optoelectronics

22

Foundry 1

Used 0.8x0.8 cm die eachcontaining 16 2x2 mm chipshaving 10x20 diode array

Delivered 160 chips to varioususers in research community

Foundry 2

Used 1x1 cm die eachcontaining 21 2x2 mm chipshaving 10x20 diode array,and 1 4x4 mm chiphaving 25x48 diode array

Delivered 110 chips to varioususers in research community

Bell labs flip-chip OE-VLSI mini-foundries

23

Reticle from 1st foundry

- 0.8um CMOS

- 6” wafer

- 3200 devices

24

Reticle from 2nd foundry prior to bonding

- 0.5um CMOS

- 8” wafer

- >6000 devices

Partial reticle shown

25

Removal of Substrate Exposes Dicing Lanes

Batch fabrication for manufacturing

26

Mixed-signal VLSI chip with >1000 optical I/O

27

Forward bias illumination of 1024 modulator diodes

Device yield can be over 99.9%

28

• 0.8µm CMOS

• 2mm x 2mm Die

• 11,500 transistors

• 1200Kbit/cm2

0

1 1 1

00 0

1

10ns

tread = 6.2ns =>160MHz

2.5ns

twrite=8ns

Process compatible with memory circuits

29

DEPOSITED MIRROR

P-CONTACT

N-CONTACT

N-LAYER

P+- LAYER

High-speed dual-intra cavity contact design

STOP-ETCH LAYER

L. Chirovsky et al., IEEE PTL, Vol. 11, 1999

30

III-V Substrate

N Layer

Bottom Mirror

ARcoating

insulation

insulation

metal 1

metal 2metal 3

insulation

n-wellp-substrategate oxide

field oxide

p+ p+n+ n+

EpoxyMetalization

SolderContact

Top Mirror

polysilicon gate

P LayerActive Layer

hυυυυ

• 256 VCSELs integrated with 0.5um CMOS chip

• Coplanar intra-cavity contacts w/ dielectric mirror

• 980nm VCSELs with through-substrate emission

980nm Emission through Substrate

Early Work 1996-’98: 16x16 VCSEL array

bonded to CMOS

31

0.0 2.0G 4.0G 6.0G 8.0G 10.0G 12.0G 14.0G 16.0G-12

-9

-6

-3

0

3

6

5mA

Response (dB)

Frequency (GHz)

11.1GHz

16x16 flip-chip bonded 850nm VCSEL array

32

First product: ARL-36 optical backplane

interconnect modules

Product Features

• 36 parallel channels per module

• Total capacity 120 Gbps per module

• Data rate 155 Mbps to 3.3 Gbps per channel

• Designed for multimode fiber ribbon, 850 nm

• Transmission distance at least 300m

• Pigtail connectorized MTO/MTP options

• System level monitoring tools

• Single connector option reduces fiber-congestion

• Field-pluggable electrical interface

• Integrated fiber management

240 Gbit/s O/E TX, RX Pair240 Gbit/s O/E TX, RX Pair

C. Cook et al., IEEE JSTQE, Vol. 9, 2003

33

ARL-36 Noise Floor

34

-24 -23 -22 -21 -20 -19 -18 -17

6

-18.0 dbm-18.4 dbm

Next Near Neighbors

Near Neighbors

No Neighbors

BER =10-12

3

Cross Talk Penalty

4 Neighbors

2.5Gbs, 223 Word

-LOG (BER)

Attenuated Optical Power (dbm)

ARL-36 Rx Cross-Talk @ 2.5Gbit/s

Attenuated Optical Power (dBm)

Bit Error Rate 10-4

10-5

10-6

10-7

10-8

10-9

10-10

10-3

35

2.5Gb/s 2 23 –1 PRBS 400m Fiber

ARL-36 Fiber Link: Bathtub Curve

1.00E-14

1.00E-13

1.00E-12

1.00E-11

1.00E-10

1.00E-09

1.00E-08

1.00E-07

1.00E-06

1.00E-05

1.00E-04

1.00E-03

1.00E-02

1.00E-01

1.00E+00

-250 -200 -150 -100 -50 0 50 100 150 200

Delay (ps)

BER

Eye opening: 0.7UI @ BER=10 -12

36

ARLARLARLARL----36 Link Eye Diagrams (36 channels)36 Link Eye Diagrams (36 channels)36 Link Eye Diagrams (36 channels)36 Link Eye Diagrams (36 channels)

Channel 1 Channel 2 Channel 3 Channel 4

Channel 5 Channel 6 Channel 7 Channel 8

Channel 9 Channel 10 Channel 11 Channel 12

Channel 13 Channel 14 Channel 15 Channel 16

Channel 17 Channel 18 Channel 19 Channel 20

Channel 21 Channel 22 Channel 23 Channel 24

Channel 25 Channel 26 Channel 27 Channel 28

Channel 29 Channel 30 Channel 31 Channel 32

Channel 33 Channel 34 Channel 35 Channel 36

• 2.5Gb/s

• 223–1 PRBS

• OC48 Mask

37

Switching system reference design

ARL-36 TX ARL-36 RX

Vitesse switch board

AraLightDaughtercard

38

• For Top Emitters 10 µm VCSELs have thermal impedance of 2300 C/W

• Measured thermal impedance of 10 µm aperture VCSELs when flip chip bonded to be 1000ºC/W

• Enables higher output powers and avoids thermal roll over

• Improved high frequency response above 10 Gbps

• Lower junction temperatures at a given drive current => improved reliability

Potentials for VCSELs-on-Silicon

39

1-channel on 36 channels on

Difference = 6.3 C

(ASIC power 100mW) (ASIC power 3.3W)

Bottom-emitting VCSELs on ASICs at 3.3Gbps

40

• “Optics-to-the-Switch”

• Architecture

• Dual integration

• Optical interconnect packaging

• Opto-mechanical packaging

R&D Challenges for OptoElectronic-VLSI Switching

41

Line

Interface

Optoelectronic-VLSICMOS Switch

GbE

PHY

GbE

PHYSRI

12

34

16

GbE

PHYSRI GbE

PHYSRI GbE

PHYSRI

GbE

PHY

x16

x16

Fast, Fast, MemorylessMemoryless Switch FabricSwitch Fabric

TARGET: 256 channels, <10WattsTARGET: 256 channels, <10WattsLine

Interface

SwitchInterface

Sch

Scheduler

Line

Interface

OC-x

Input

Optoelectronic CMOS Crosspoint Switch

42

P1

Parallel Optics Transmitter

SwitchOutput

K Bit-Sliced Crossbars

P2

PN

1K

N x NCrossbar

Bit1

BitK

Bit1

BitK

Multimode Fiber Ribbons OE-VLSISwitch

Implementation: a bit-sliced switch

Parallel Optics Receiver

- Functions as a 16x16(x16) crossbar

- 256 optical inputs, 256 optical outputs

- 64 switch control lines (4 per channel)

43

Control

Processor

.

.

.

32 x 32 x 1

CrossBar

Switch

32 x 32 x 1

CrossBar

Switch

N x N

CrossBar

Switch

.

.

.

• Non-blocking

• Scheduling/Arbitration

• Out-of-Band Control

• Fast switching (per packet)

• Asynchronous

• Bit-rate Transparent

• Format Independent

• 2-R or 3-R modes

•••

12

3

K

KTransmitters

KTransmitters

KReceivers

KReceivers

1

N

1

N

Switch-on-a-chip architecture

44

Fiber Bundle Front View (facing bundle)

• hexagonal closepack

• multimode 50micron-core fiber

• terminated to MTP connectors on other end

497 512

1 16

Fiber bundle

45

System

512 fibers (256 in, 256 out) terminated into 64 8-

fiber MTP connectors

46

10,000km

1000km

100km

10km

1km

100m

10m

1m

10cm

SM, CWDM

SM or MM, Serial or Parallel

SM, DWDM

Multi-mode, Parallel

Metro, access, cross-campus

Across central office, data centers

To the box

To the chip/package

$10,000

$3,000

$1,000

$300

$100

$30

$10

$3

1980 1985 1990 1995 2000 2005 2010 2015

Trans-oceanic

Cross-country

SM, DWDM or MM, Parallel

1Mbps 10Mbps 100Mbps 1Gbps 10Gbps 100GbpsLink Distance

Transceiver C

ost (p

er G

bps)

Year of Introduction

Bandwidth per fiber

Penetration of optics into communications

?

47

Jack Cunningham ………………….. VCSELs and link

Helen Kim …………………………Circuits and testing

Keith Goossen …………………….. Devices and integration

William Jan ……………………... Processing

Chris Cook ……………………... Optomechanics & packaging

….. And many others ….

Acknowledgements