TechShanghai2016 - DSP Enhanced IoT Applications and Development Platform

DSP Enhanced IoT Applications and Development Platform

http://www.ceva-dsp.com/














2 CEVA Proprietary Information

Not All IoT Devices Are Created Equal …

Connected

While IoT must be connected, it is not always smart !

Smart Vs.

smart home security device


IoT Layers – Connected vs. Smart Example: a truly smart home controller would …

Input Processing Smart Analysis Actions

Speech Speech recognition

Voice command Speaker verification

Turn on/off lights, window shades, appliances … Allow access to devices per user profiles

Sound Sound sensing

Is music playing ? Which music ? Alert noises ? (e.g. breaking glass)

Change light-bulb color per music type Call security company; notify owner

Vision Face recognition

Who is in the house ? Resident or guest ?

Set mood (color, music) per person or group Send message to owner about guests presence

Motion Motion detection

People moving in home ? Device being moved ?

Learn habits of tenants Alert about anomalies

Connectivity SW PHY Is Wi-Fi available ? Yes – use in-home Wi-Fi No – use LTE/MTC (preferred for security events)

DSP enables the creation of smart IoT devices !


DSP Powered Use-cases and Solutions


Smart IoT Platform – DSP Enabled

Connectivity

Sensing

CPU OS

Intelligence

DSP Based SW PHY Wi-Fi, Bluetooth, ZigBee, LTE (CAT0/ MTC/NB-LTE), PLC

DSP based intelligence Speech Recognition, Face Recognition, Depth mapping, Analytics, Learning

DSP based sensing Motion sensing, Sound sensing, Camera, Health

CEVA IoT solutions for Connectivity + Sensing + Intelligence


Challenges in IoT Connectivity

ANT+

Blu

etoo

th

LTE

DEC

T U

LE

NFC

PLC

Wi-F

i

ZigB

ee Z-

Wav

e O

ther

s

Wearables ● ● ● ● ● ●

Smart Devices ● ● ● ● ● ●

Smart Homes ● ● ● ● ● ● ● ● ●

Cars ● ● ● ●

Smart Cities ● ● ● ● ●

Industrial Internet ● ● ●

Multiple and constantly evolving communication standards

CEVA’s DSP based SW PHY allows multi-standard connectivity platform


CEVA-Dragonfly M2M Platform ► CEVA-Dragonfly is an integrated HW/SW Platform to speed up and lower

risk of SoC development for multi-mode M2M connectivity endpoints ► Includes all required M2M modem HW IPs and SW components to let you focus on

application integration

► Target applications include wearalone, wearables, smart grid, surveillance systems, asset tracking, remote monitoring systems, connected cars and smart utilities

► Supports every M2M standard including LTE MTC Cat-1, Cat-0 and Cat-M standards, Wi-Fi 802.11n/ah, PLC, 802.15.4g, ZigBee, GNSS, LTE-OTDOA, NB-IoT, LoRa, SIGFOX


► Based on CEVA-TeakLite-4 DSP + PHY and MAC hardware ► Enables most power efficient design ► Scalable for other connectivity standards ► TeakLite-4 DSP available for additional

functionality such as audio, voice, sensing

► Down to 500K gates ► Complete solution: DSP, PHY, MAC, HW, SW

► Less than 30mW for 1Mbps 802.11n 1X1 in 55nm

CEVA-WiFi Platform

Lower MAC

RF Interface

Wi-Fi Protocol Stack

PHY & MAC Hardware

Modem

MAC HW

Bus Interface

Upper MAC

AHB

CEVA-TeakLite-4

WiFi Direct

Application


► CEVA-Bluetooth ► Classic Bluetooth (2.1+EDR, 3.0) ► Low Energy Bluetooth (4.2) Single/Dual Mode ► CEVA-Bluetooth BB HW integrated with processor ► CEVA-Bluetooth Controller SW stack ► Customizable RF interface for 3rd party Radio

► Single Mode ► Reduced HW/SW footprint for low power & cost

► Dual Mode ► Addition of low energy protocol HW/SW to Classic BT

► Bluetooth 5.0 is just around the corner ► Audio over BLE, IPv6 over BLE, extended range and

much more

CEVA-Bluetooth Platform

Classic 4.n Link Controller

Classic 4.n Link Manager

Host Controller Interface

Hardware Abstraction Layer

RF Interface

Bluetooth Software Stack

HCI

Baseband Hardware

RF Interface

Classic 4.n HW Link Controller

Bus Interface

LE - Link Controller

LE HW Link Controller

Host Software and Profiles

AHB


Always-On Multiple Sensing

Low cost, low power sensors are common

However, low cost/power generally means noisier sensors

So where do DSPs fit in sensing? Need a lot of signal-cleaning (filtering, smoothing, calibrating, etc’) in order to extract meaningful data DSP requirements further increase with introduction of mics and biometric sensors Intelligent context awareness

Sensing in IoT is based on various and growing list of sensors IoT Sensors Evolution

GPS GNSS GNSS

BT (beacon)

Accelerometer Accelerometer Accelerometer

Magnetometer Magnetometer

Barometer

Context

Motion

Location

Gyroscope

Microphone(s)

Ultrasonic Image/ Gesture CMOS Sensor

Pow

er C

onsu

mpt

ion

& C

ost

2010… 2015…

$

Power consumption is key criteria: need <2mA for the complete always-on use case

Temperature

Barometer

Fingerprint

Heart Rate

Biometric

CMOS Sensor

Light Light Light

Temperature


Application Processor (AP)

Accelero. sensor

Gyroscope sensor

Compass sensor

Pressure sensor

Motion algorithms Sensor hub DSP

I2C

I2C/SPI

Android Applications Application Framework

Sensor HAL + sensor fusion algorithms Kernel

Application Processor (AP)

Accelero. sensor

Gyroscope sensor

Compass sensor

Pressure sensor

I2C

Android Applications Application Framework

Sensor HAL Kernel

Before Sensor fusion running on AP High power consumption (120 to 190mA) Effort to maintain sensor fusion in Android user space Motion sensor vendor dependent Repeated software porting effort for new motion sensors

Now Dedicated DSP to run sensor fusion SW Low power consumption (~1mA) for Always-on 10-axis fusion algorithm sensor independent Real time DSP environment for real time algorithms Host OS independent + Host offloading

Mandates offloading to a dedicated processor for low power !

Always-On Sensor-Fusion


Always-On Voice Activation

TL410 Voice activation Wake-Up

Indication

CPU/OS

Speech Recognition SW - Voice Trigger - Voice Command - Speaker verification Keyword, e.g. “OK Baidu”

CEVA-TL410 enables < 20uW DSP power consumption at 28nm for always-on voice trigger and command


DSP-Based Sensing

Sensor fusion Contextual awareness Motion gestures Indoor navigation

Voice trigger

Ultrasonic gestures

Face trigger

BLE

Multiple Sensing Technologies Using a Single DSP

DataMemory

MSS

TeakLite-4

ProgramMemoryDMA

(4 Ch.)

4 x TIMER(WDT)

Master

32 x GPIO(I2C) ICUPMU

SlaveTDM R/T(I2S)

D P

APB

AHB

Gesture/face/voice Trigger Indication

AP

Interrupt

MEMS Mic

MEMS Sensors

MIPI

Bluetooth Baseband Radio

Less than 150uW @ 28nm HPM for Always-on Sensor Fusion + Voice Trigger + Face Trigger + BLE


IoT Layers – Local Intelligence vs. Cloud DSP powered local intelligence enables:

Camera/microphone/other sensors raw data does not need to be sent to the cloud, only processed meta-data is being sent Increased privacy

Reduced data bandwidth, transfer overhead and processing latency to/from cloud lower on-device power + lower cost of cloud service

Efficient processing for scene analysis (sound/vision) with lower power than GP CPU/GPU Lower power consumption, longer battery life

Increased security by using multiple connectivity standards Using SW PHY allows switching from Wi-Fi to LTE for tamper-proof security actions

Local intelligence is key for smart IoT devices !


DSP-Based Audio Analytics Valuable Sound Classification & Analysis

Mobile, Wearable, Smart Home and Robot applications

Voice recognition and speaker identification Speaker separation through beamforming Environment sensing (e.g. cinema, train) Emotion detection

Security applications can alert upon: Glass breaking Baby crying Keyword (virtual red button) Aggression

Extensive computational power enables a standalone (cloudless) audio analysis Efficient processing of large amounts of data

Handcrafted data cache and DMA 128-bit data bandwidth Dual load/store units

Enables intensive noise reduction required for accurate analysis Lowest-power on its category

CEVA-TL421 as an Audio Analyzer

Advanced contextual awareness includes intelligent sound analysis and requires a powerful programmable audio/voice processor


DSP-Based Vision Analytics Computer Vision & Video Analytics

Markets: Mobile, Wearable, Smart Home, Smart Cities, Security & Surveillance

Dedicated computer vision engine for complex analytics applications

4th generation computer vision DSP Up to 1.2GHz @ 28nm HPM Combines fixed and floating-point math Up to 4096-bit vector processing per cycle

Efficient processing of large amounts of data Innovative DMA mechanism capable of transpose, crop and checkered data on the fly 512-bit bandwidth

Fully equipped with SW framework, library and tools for easy SW porting and optimization

CEVA-XM4 Computer Vision Engine

Example Applications Human & object recognition Face recognition, gesture recognition Tracking based on feature and pattern matching Emotion detection Image and video enhancement for complex outdoor conditions

Growing need for sophisticated digital analysis on Camera calls for dedicated CV programmable processors


CEVA IoT Platform Enablers

DSP enables smart IoT devices !

CEVA-XM4

Computer Vision, Video Analytics

Edge Processing

Shared Memory

Interconnect

LCD

CEVA-TL421 Audio Analytics

Edge Processing

I2C I2S Ext Mem PMU

L1

UART MIPI Boot ROM

CODEC

sensors

L1 L1 L1

LTE Cat-0 PHY + MAC

CEVA-TL410 Sensor

Processing, Always-on

Radio Radio* Radio

L1 L1

Bluetooth PHY + MAC

Wi-Fi PHY + MAC

CPU / MCU L1 L1

DSP Intelligence DSP Sensing DSP based connectivity (SW PHY)


CEVA ‘Smart & Connected’ IoT Development Platform


Intelligence

The ‘Smart & Connected’ (IoT) Strategy

*Source: IDC

Total IoT units expected to grow at 18% CAGR to reach 28 billion units by 2020*

Wi-Fi, Bluetooth, LTE & Narrowband LTE (NB-LTE)

Camera : Face recognition , AR, VR ,ADAS Microphone/sensors: Always on, Noise

suppression, sensor hub

1. 2.

Connectivity CPU


‘Smart & Connected’ Development Platform

Availability: Now, directly from CEVA


CEVA-TeakLite-4 Processor The “Brains” of the Platform

High-performance Low-Power 32-bit DSP, Scalable, Extendible

Optimized for audio, voice and general control/DSP applications

CEVA’s fourth generation TeakLite architecture Leverages > 100 design wins, > 4.5 billion devices in the market, > 150 audio/voice SW modules, > 30 active ecosystem partners

Low die size and cost 90K gates in its smallest form

Designed for low power TeakLite-4 development chip made in collaboration with Brite and fabricated at SMIC 55nm LP, running at 500MHz


CEVA-TeakLite DSP Family

TeakLite-II

CEVA-TL3211

CEVA-TL3210 TeakLite-II Architecture

(16-bit)

TeakLite-III Architecture

(32-bit)

TeakLite-4 Architecture Framework (32-bit)

Feat

ures

& P

erfo

rman

ce

CEVA-TL411 CEVA-TL410

CEVA-TL421 CEVA-TL420

Future TL4 Cores

Available Roadmap All TeakLite cores are

assembly level compatible

Deeply Embedded

Cache Enabled

Dual 16x16 Single 32x32

Quad 16x16 Dual 32x32

Multiple configurations

for optimal application

needs


Complete TeakLite-4 Subsystem Included in TeakLite-4 Silicon

Fully integrated subsystem, comprised of main system peripherals, seamlessly connected to TeakLite-4 DSP

Complete with drivers and reference design

Shorter TTM and reduced risk for customer SoC

DataMemory

MSS

TeakLite-4

ProgramMemoryDMA

(4 Ch.)

4 x TIMER(WDT, PDM)

Master

32 x GPIO(I2C) ICUPMU

TDM R/T(I2S)Slave

D P

APB

DSP CoreSubsystem

AHB


More than 150 Audio/Voice SW Modules Always-on &

NUI RTOS Voice Noise Reduction

& Echo Cancellation

Audio Post-processing

Dolby HD-Audio DTS HD-Audio

Sensory TrulyHandsFree; Sensory User Defined Trigger; Sensory Speaker Verification; Rubidium Voice Trigger; Malaspina Labs Voice Trigger; Cywee Motion Sensing; Visidon Face Detection; IVT BT Stack

FreeRTOS; Express Logic ThreadX; ENEA OSEck; Quadros RTXC; uT-Kernel; CMX-RTX; Mentor Graphics Nucleus

G.723; G.729; G.728; G.729.1; G.711; G.722; G.726; G.727; G.168; G.161; iLBC; AMR-NB; HR; FR; EFR; AMR-WB; EVRC; EVRC-B; QCELP; SILK (32-bit); Opus; AMBE Vocoders; EVS

NXP SW Life Vibes Voice Experience; Alango Voice Comm. Package; Dimagic Flexbeam; Cypher Voice Isolation; Malaspina Labs Noise Reduction; Waves MAXX Voice; Waves MAXX Speech

MP3; MP3Pro; Ogg Vorbis; FLAC; MPEG4 AAC LC; HE AAC V1; HE AAC V2; HE-AAC V2 5.1 Ch; MPEG4 AAC BSAC; WMA; RealAudio; SBC; CELT; DRA

DTS/SRS TruMedia HD; DTS/SRS StudioSound 3D; DTS Neo:6; Dolby ProLogic IIx; Dolby Mobile 3+; Dolby DS1; Dirac Speaker Correction; AM3D Zirene; AM3D Sound/3D; Waves MAXX Audio; Arkamys MobiSound; Qsound MicroQ

Dolby TrueHD; Dolby Digital Plus; Dolby Digital decoder (AC3); Dolby Digital encoder (DDCE); Dolby MS10; Dolby MS11; Dolby Volume

Master Audio; High Resolution; Low Bit Rate; Extended Surround (ES); DTS 96/24; DTS Digital Surround; DTS Transcoder; DTS M6; DTS M8

Major cost reduction and time-to-market advantage


TeakLite-4 DSP Library

Bit-accurate C (Visual Studio) and TeakLite-4 assembly functions for easy integration

Supports both 16-bit and 32-bit precision

Supports all TeakLite-4 cores and configurations (audio, voice, compatibility)

Total of 74 functions including full source code Fully integrated with SDT - NO ADDITIONAL COSTS INVOLVED!

Autocorrelation Cross-correlation Convolution (Block FIR) Block LMS Filter Delayed LMS Filter Decimation Interpolation Symmetric FIR Single Sample FIR Complex IIR (Biquad)

Filter Functions

Division Square Root Inverse Square Root Log Power Cosine Sine Tangent Arctangent

Math Functions

Bit-Reverse Permutation Complex FFT Real FFT Complex Inverse FFT Real Inverse FFT

Fourier Transformations

Addition Subtraction Dot Product Maximum / Minimum Multiplication Shift

Vector Operations


Profiler – Function Graph

Profiler – Cache Performance

Profiler – Function Info

Profiler – Code Coverage

IDE / Debugger View

CEVA-ToolBox™ Software Development Environment

Highly focused on two main objectives:

Performance of generated code, especially cycle count and code size

User experience, with emphasize on ease of programming, user interface and automation tools

Build Optimizer


Android Connectivity & CPU Offloading CEVA AMF (Android Multimedia Framework)

AMF is an Android based software framework for offloading audio/voice tasks to the DSP

CEVA AMF defines a HW and SW interface between the CPU running Android and single or multiple CEVA DSPs

Multimedia workloads (e.g. voice trigger, sound sensing, audio playback) are abstracted on the CPU and executed on the DSP with better performance and reduced power > 5-9x lower power for audio/voice applications

Multimedia tasks may be “tunneled” for saving data transfer overhead to and from the CPU

CEVA’s AMF Android support: > KitKat, Lollipop – now > Marshmallow – in 2016

CPU

CEVA Link Driver CEVA Link Driver

DSP/RTOS

The CEVA AMF (Android Multimedia Framework) Seamlessly Bridges the Gap Between Android OS and CEVA Platforms !


CEVA ‘Smart & Connected’ Dev Platform

Analog Mic 2

Analog Mic 1

Line out

Line in

Digital mics

Ethernet

USB/UART RF I/F

PCIe

User switches and dip-switches

Power

Color LCD

Arduino shield connectors & GPIOs

TeakLite-4 Silicon (DSP, DMA, TDM, I2S, I2C, ICU, Timers) RTOS

JTAG port

ARM Cortex-A9 x 2 Linux OS User area FPGA

+


CEVA ‘Smart & Connected’ Dev Platform Real-time Power Measurement and Tuning

► Platform includes the ability to measure power separately for:

1. TeakLite-4 DSP 2. On-chip memory 3. On-chip peripherals (DMA, TDM …)

► On-board real-time power measurement allows the user to modify code and observe power differences application power tuning and optimization !


CEVA ‘Smart & Connected’ Dev Platform

Availability: Now, directly from CEVA


Summary

Connectivity/Sensing/Intelligence IP (HW/SW) + Dev. Platform = DSP Enhanced IoT Applications

CEVA-XM4

Computer Vision, Video Analytics

Edge Processing

Shared Memory

Interconnect

LCD

CEVA-TL421 Audio Analytics

Edge Processing

I2C I2S Ext Mem PMU

L1

UART MIPI Boot ROM

CODEC

sensors

L1 L1 L1

LTE Cat-0 PHY + MAC

CEVA-TL410 Sensor

Processing, Always-on

Radio Radio* Radio

L1 L1

Bluetooth PHY + MAC

Wi-Fi PHY + MAC

CPU / MCU L1 L1

DSP Intelligence DSP Sensing DSP based connectivity (SW PHY)

+

THANK YOU















TechShanghai2016 - DSP Enhanced IoT Applications and Development Platform

Engineering

Transcript of TechShanghai2016 - DSP Enhanced IoT Applications and Development Platform