eFPGA for AI and IoT Applications · 2018. 4. 13. · IoT Technology Stack-Up – Overview Vision...

eFPGA for AI and IoT ApplicationsTim Saxe

CTO

213/04/2018Copyright © 2018 QuickLogic, Inc. All rights reserved.

IoT Technology Stack-Up – Overview

Vision Sound Motion etc

Transistor-level (process and foundry)

Gate-level (ASIC, eFPGA, memory)

H/W accelerators/off-load engines

SoC (QUIK, NXP, ST, QCOM, etc)

Basic S/W (HAL, drivers, etc)

S/WEmbedded

CloudS/W

CloudS/W

CloudS/W

CloudS/WS/W

EmbeddedS/WEmbedded

S/WEmbedded

AI Applications

Tech

nolo

gy

FPGA IP (Data Control & Transform)

Pre-Fab(Time consuming and expensive to change)

Post-Fab(Fast cycle and relatively inexpensive to change)


AI is like a power tool – theoretically it can do anything

Build a house – much easier with tool

Put together a sofa – easier with tool


If you can connect it to the real world




If you can connect it to the real world



eFPGACan bridge the gap


And exactly what size problem are you trying to solve?

Data center problemIoT problem

eFPGACan help here, too

eFPGACan help here


Deep Learning Image Classification Network

AI Chip

rgb

3 planes32 x 32

128 planes32 x 32

convolve

128 planes32 x 32

convolve

128 planes16 x 16maxpool

256 planes16 x 16

convolve

256 planes16 x 16 convolve

256 planes8 x 8

maxpool

512 planes8 x 8

convolve

512 planes8 x 8

convolve

512 planes4 x 4

maxpool

1024x1 10x1Fully

connected

Requires 128KB scratch memory and 6MB coefficient memory



AI Chip

rgb

3 planes32 x 32

128 planes32 x 32

convolve

128 planes32 x 32

convolve


256 planes16 x 16

convolve


256 planes8 x 8

maxpool

512 planes8 x 8

convolve

512 planes8 x 8

convolve

512 planes4 x 4

maxpool

1024x1 10x1Fully

connected


And somehow the image miraculously appears at the start

CameraI/F


AI SoCeFPGA


rgb

3 planes32 x 32

128 planes32 x 32

convolve

128 planes32 x 32

convolve


256 planes16 x 16

convolve


256 planes8 x 8

maxpool

512 planes8 x 8

convolve

512 planes8 x 8

convolve

512 planes4 x 4

maxpool

1024x1 10x1Fully

connected

Camera I/F IP


eFPGA can manage the sensor and format the image


Edge Friendly Deep Learning Image Classification Network

Requires 8KB scratch memory and 150KB coefficient memoryFits a small eFPGA with scratch memory

eFPGA

Neural Net IP

rgb

3 planes32 x 32

64 planes32 x 32

convolve

64 planes32 x 32

convolve


64 planes16 x 16

convolve


64 planes8 x 8

maxpool

128 planes8 x 8

convolve

128 planes8 x 8

convolve

128 planes4 x 4

maxpool

256x1 2x1Fully

connected

Camera I/FIP


Radial Basis Function Image Classification Network

AI Chip

rgb

3 planes32 x 32

128 planes32 x 32

convolve

128 planes32 x 32

convolve


256 planes16 x 16

convolve


256 planes8 x 8

maxpool

512 planes8 x 8

convolve

512 planes8 x 8

convolve

512 planes4 x 4

maxpool

1024x1 10x1Fully

connected

DNN requires 128KB scratch memory and 6MB coefficient memory

CameraI/F

Feature Extraction Classification

RBF IP(65KB)

eFPGA

Camera I/F IP

Feature Extraction IP


eFPGA provides sensor flexibility

AI SoC

eFPGA

Camera I/F IP AI EngineROI/Subsample IP

AI SoC

eFPGA

Sensor I/F IP AI EngineScale & Segmet IP

AI SoC

eFPGA

Microphone I/F IP AI EngineMFCC IP


eFPGA provides real-time control

AI SoC

eFPGA

Camera I/F IPAI Engine

ROI/Subsample IP

Model Update IPServo Control IP


Conclusions

Big Iron AI uses powerful eFPGAs to further accelerate powerful data center CPUs

IoT AI uses eFPGAs as part of an SoC to bring the benefits of hardware: Real-time operation Low-power operation

into the post-fab environment

IoT AI uses eFPGAs as part of an SoC to: Manage sensors Preprocess and format data Provide real-time control based on AI outputs

eFPGA for AI and IoT Applications · 2018. 4. 13. · IoT Technology Stack-Up – Overview Vision...

Documents

Transcript of eFPGA for AI and IoT Applications · 2018. 4. 13. · IoT Technology Stack-Up – Overview Vision...