Meeting Demands For Camera and Sensor Interfaces in IoT and

Automotive Applications

Hezi Saar Synopsys, Inc.

Agenda •  Implementation of MIPI interfaces in mobile

applications and beyond •  Advantages of implementing MIPI interfaces

•  MIPI CSI-2, MIPI DSI, MIPI D-PHY, MIPI I3C

•  SoC design considerations •  Summary

MIPI Display & Camera Interfaces Mobile Applications

• MIPI CSI-2 and MIPI DSI protocols enable application-optimized SoCs primarily used in mobile electronics

•  Use of multiple cameras and displays for beyond traditional mobile applications

MIPI Interfaces in New Applications Automotive, IoT/Wearables, Virtual/Augmented Reality

MIPI Camera & Display Interfaces in IoT

Linux/Rich OS

Ext Flash Memory

Controller LPDDR

eMMC ARC HS Processor

HMI

MIPI DSI

GPU

Communications

Bluetooth Smart

SDIO

USB

Vision

MIPI CSI-2 Vision Processor

On-Chip Memories

ROM SRAM

Sensor & Control Subsystem

Processor

ADC

MIPI I3C / I2C

SPI

Security

Secure Core

Private Key

Public Key Accelerator

True Random Number Generator

Crypto Accelerators

Logic Libraries

•  MIPI DSI, MIPI CSI-2, MIPI I3C •  LPDDR, eMMC, ADC, BT, USB •  Embedded Vision •  Security •  Sensor Fusion

Point-of-Sale IC

Advanced Driver Assistance Systems (ADAS) A Rapidly Evolving Technology

•  Passive Driver Assistance Systems •  Back-up camera system

•  Distance alert system

•  Active Driver Assistance Systems •  Back-up camera w/ ID & breaking

•  Collision avoidance

Example: Surround View Using MIPI CSI-2 Image Sensors & MIPI DSI Display

Rear Camera

Display

CAN Interface

MPU Proprietary, LVDS or Ethernet Switch

DRAM Flash Memory

Power Supply Front Camera

Module

Left Camera Module

Right Camera Module

Rear Camera Module

Other Camera Module

Vbat

LVDS or Ethernet

Link

MIPI CSI-2 Image Sensors

MIPI DSI Display

Front Camera

Left Camera

Right Camera

Other Camera Module

Typical Automotive SoC High-EndADAS Infotainment

•  MIPIDSI,MIPICSI2,MIPII3C,MIPISoundWire•  LPDDR4,EthernetAVB,HDMI,PCIe,SATA,ADC

•  EmbeddedVision•  Security•  SensorFusion•  RequiresFuncEonalSafety

•  MIPIDSI,MIPICSI2,MIPII3C,MIPISoundWire•  USB,LPDDR4,EthernetAVB,HDMI,PCIe,SATA,

ADC•  eMMC->UFS

•  Real-EmeMulEmedia•  Security•  SensorFusion

I3C

I3C

I3CMIPID-PHY

MIPID-PHY

Augmented, Virtual, Mixed Reality Enterprise Use-Cases

Virtual Reality System Teardown HTC Vive

Source:iFixit

Examples of New Use-Cases MIPI Display, Camera & Sensor Interfaces

Microsoft HoloLens Processing Unit

•  Processor packaged together with 1GB LPDDR3 and Intel Atom x86 processor

•  Uses 5 camera interfaces, depth and motion sensor for image identification and processing, recognizing gestures

(*)Source:EETimesanalysis(*)Source:MicrosoR

ADVANTAGES OF THE MIPI SPECIFICATIONS

MIPI CSI-2, MIPI DSI, MIPI I3C

MIPI CSI-2 Over MIPI D-PHY

CSI-2Device

D-PHY

CSI-2Host

D-PHY

Clk+

Clk-

L0+

L0-

Clk+

Clk-

L0+

L0-

L1+

L1-

L1+

L1-

FrameBuffer

CSI-2TransmiYer

PacketBuilder

LaneDistribuEon

CCISlaveSCL

SDA

SCL

SDA

CCIMaster

CSI-2Packet

CSI-2Packet

D-PHYHSBurst

D-PHYHSBurst

CSI-2Receiver

PacketDecoder

LaneMerger

CSI-2Packet

CSI-2Packet

FrameBuffer

CRCDT WC ECC

Payloadbytesize DataCRCprocessing

ECCprotecEngtheheader

DataFormatDefiniEon

VirtualChannelIdenEficaEon

PacketBuilder

MIPI CSI-2 Packets

Virtual Channels

Virtual Channel 0 – Line 0 Virtual Channel 0 – Line 1 Virtual Channel 0 – Line 2 Virtual Channel 0 – Line 3 Virtual Channel 0 – Line 4

Virtual Channel 0 – Line N

Virtual Channel 1 – Line 0 Virtual Channel 1 – Line 1 Virtual Channel 1 – Line 2 Virtual Channel 1 – Line 3 Virtual Channel 1 – Line 4

Virtual Channel 1 – Line M

DSISPacke

t

DSISPacke

t

CRCWC ECC

Payloadbytesize DataCRCprocessing

ECCprotecEngtheheader

DataFormatdefiniEon

VirtualChannelIdenEficaEon

PacketBuilder–LONGPacket

MIPI DSI over MIPI D-PHY Example: DSI Video Example - DPI

D-PHY

D

-PH

Y

Clk+

Clk-

L0+

L0-

Clk+

Clk-

L0+

L0-

L1+

L1-

L1+

L1-

DSITransmiYer

PacketBuilder

LaneDistribuEon

DSIReceiver

PacketDecoder

LaneMerger

VSS HSE

DATA0 ECC

ECCProtecEngtheShortPacket

DataFormatDefiniEon

VirtualChannelIdenEficaEon

PacketBuilder–SHORTPacket

DATA1

D-PHYHS

Burst

D-PHYHS

Burst

DSILPacket

ValidImageLine

DSISPacke

t

VSS

DSISPacke

t

HSED-PHYHS

Burst

D-PHYHS

BurstD-PHYHS

Burst

D-PHYHS

Burst

DSILPacket

ValidImageLine

MIPI D-PHY Architecture The Popular Physical Layer Used for MIPI CSI-2 & MIPI DSI Protocols

•  Synchronous Forwarded DDR clock link architecture

•  One clock and multiple data lanes configuration

•  Static/dynamic de-skew supported through calibration

•  Calibration hand-shake not supported

•  No encoding overhead

TwoDataLaneConfigura0on

•  Low-power and high-speed modes

•  Primarily targeting camera and display

•  Spread spectrum clocking supported for EMI/EMC considerations

•  Large eco-system, proven in millions of phones, cars

Benefits and Evolution of MIPI D-PHY •  Higher data rate enables

ultra-high-definition cameras and displays

•  Easier adaption of newer technologies

•  Backward compatible •  Reliable with sufficient

margins •  New specs augment

existing ecosystem •  Growing market

applications and segments •  Longer channel length

MIPI I3C Specification Overview Types of Devices in an I3C System

• MIPI I3C main master • MIPI I3C secondary master(s) • MIPI I3C slave(s) •  I2C slave(s)

Examples of MIPI I3C Use-Cases

SensorHub SensorSubsystem

ImageSensors

MIPI I3C Sensor Hub Use Case •  The I3C bus has a

secondary master that acts as a hub

•  Takes ownership of the I3C bus and communicates to the sensors directly

•  As soon as the secondary master has the relevant sensor data available on its I3C bus, it can communicate to the main master, which propagates the data to the CPU

MIPI CSI-2 Use of MIPI I3C for Control & Always-ON Imaging

•  Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency

•  Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C

MIPI CSI-2 Use of MIPI I3C for Control & Always-ON Imaging

•  Supports advancements in imaging for new applications: Health, Convenience, Security, Lifestyle, Efficiency

•  Camera Controller Interface (CCI) and Always-ON advancement considerations using I2C and future MIPI I3C

SOC DESIGN CONSIDERATIONS

MIPI CSI-2, MIPI DSI & MIPI D-PHY Technical Considerations

• Bandwidth required

• D-PHY version/speed, number of ports and lanes

• Determine optimal interfaces to Tx/Rx image for memory storage or image

• Configuration & interrupts

• Memory usage

Target Multiple Applications with Flexible SoC Implementation

MIPICSI-2Host

Controller

MIPICSI-2Host

Controller

MIPICSI-2Host

Controller

MIPICSI-2Host

Controller

2LaneMIPID-PHY

2LaneMIPID-PHY

2LaneMIPID-PHY

2LaneMIPID-PHY

Applica7onA Applica7onB

OR

Form Factor Reference Channels Variety of Applications

MIPI CSI-2, MIPI DSI, MIPI I3C & MIPI D-PHY IP •  Complete, Production-Proven, Interoperable

Summary •  MIPI CSI-2, DSI, MIPI D-PHY and MIPI I3C

•  Leveraged for mobile and beyond – IoT, automotive and AR/VR

•  Synopsys provides complete camera, display and sensor interface IP

•  Enables new set of applications in automotive, AR/VR, IoT markets •  Lowers integration risk for application processors,

bridge ICs & multimedia co-processors

•  Future proof IP supporting variety of speeds, proven in silicon •  Reduces cost and power for multiple instantiations

•  Testability features enable low cost manufacturing

Industry’sfirstMIPII3CDemo

VisittheSynopsysBoothtoSeeOurDemoToday!

Thank You!

Synopsys’broadpordolioofDesignWareMIPIIPsoluEonssynopsys.com/mipi