Opportunities for ML Analytics at the Sensor Endpoint

Chris Rogers, CEO SensiML Corporation

MAKING SENSOR DATA SENSIBLE

IoT Smart DevicesHow Many Qualify as Truly Smart?

The Majority of IoT Endpoint Devices…

What’s missing is useful adaptable algorithms embedded in the device

• Incorporate sensors

• Connected but dumb

• Defer analytics elsewhere

• Network constrained

• Not real-time

• Stream unfiltered sensitive data

• Static algorithms

So What’s the Big Deal With Having Dumb IoT Sensors?

Conventional “Dumb” Sensor IoT Network

Simple Sensor- Temperature Sensors- Limit Switches- Counters

Complex Sensor- Cameras, imaging sensors- Audio, microphones- Motion, accelerometers, IMUs- Vibration, piezo sensors- Passive IR- Current, voltage, electrodes- RF signals

Key Challenges:

• Bandwidth

• Power

• Latency

• Security

Conventional IoT Sensor Network: Bandwidth / Power

Raw Payload Motion Vibration Audio Video

Sample Rate 1 kHz 5 kHz 20 kHz 30 Hz

Resolution 16 bit 16 bit 16 bit 24 bit

Channels 9 (x,y,z) 3 (x,y,z) 2 (stereo) 4 MPixel

Throughput Req’d

140 kbps 234 kbps 625 kbps 2.9 Gbps

Network Throughput

LPWAN(LoRA)

LTE IoT(Cat-M1 R13)

ZigBee BLE 4.2(BT Smart)

BT 5.0 WLAN (802.11ac)

Payload < 8 kbps < 375 kbps < 250 kbps <21 kbps* < 1.4 Mbps

< 125 kbps** 200Mbps

***

*** 802.11ac @ 40Mhz channel, 1x1 (embedded STA)

** BT5 2x long range operation

* BLE4.2 7.5ms CI, 20 byte MTU

Conventional IoT Sensor Network: Latency

"In cases where sensors generate a lot of telemetry, but only sporadic data that's actionable, you want to discern the signal from the noise without overwhelming the ingestion processes at the core …you don't want a 100 millisecond loop to the internet and back."

- Jeffrey Hammond, Forrester Research Analyst

Conventional IoT Sensor Network: Security

AmazonAVS Server

Local Wake WordEvent Detection

“Alexa”…ALLAudio

Specific Query

ALL AudioAmazon

AVS Server

The Role for Sensors in Intelligent IoT Networks

Our Own Brains: A Distributed Processing Architecture

As Applied to IoT Analytics Processing…

A High-Performance Distributed IoT Network Application

Cloud Analytics(offline data mining,business intelligence)

Edge ANN/CNN(vision, spatial and image processing)

Cloud NLP(speech recognition, automated assistant)

Endpoint Rich Sensing ML(audio processing,motion and vibration classification)

Local Critical Insight ML(network failure independent,mission critical feedback/control)

Endpoint Real-Time ML(machine control, robotics)

Sensed Property Acquired SignalPhysical World

Smart Sensors: Combine Rich Signals with Expert Insight…

Machine vibration

Conventional Sensor

… to provide local inferencing of meaningful events

Event Detection Expert Training Meaningful Insight

CauseEvent

Extruder #3

Excess Vibration

Flange Bearing Fail

Obstruction

OK

Smart Sensor

Signal Conditioning Communication

Filtering, down-sampling,averaging, etc.

Packetization, data compression,

error correction

Feature Engineering Classification

A Distributed Smart Sensor Endpoint IoT Network

Key Challenges:

•Processing

• Learning

•Data Loss

Smart Sensing: Processing Limitations

Algorithm Suitability to IoT Endpoint Device Processors

Hand-coded

Classic numerical methods(i.e. regression, heuristics,sorting, linear programming)

Efficient Execution

Costly Development

Inflexible / Static

Rules Based

Expert systems(Knowledge base collection of rules)

Code/Rules Separation

Brittle Logic

Inefficient code

AI / Deep Learning

ANN / CNN(Neuron arrays trained by backpropagation)

Overall Performance

HW Requirements

Large Training Datasets

Machine Learning

Classification(i.e. SVM, kNN, random forest, clustering)

Efficient Execution

Adaptive Learning

Training Intervention

Smart Sensing: On-Device Learning Challenge

• If sensor nodes “see” only their own input data…they can learn only from what they are exposed

• Cloud capable of seeing data from ALL sensors…but then training is centralized not distributed

Dilemma:

Smart Sensing: A Flexible Learning ArchitectureLevel 1: Algorithm Tuning and Personalization• Local model reconfiguration and parameter tuning• Improved classifier performance over time• No cloud required

Example: Tailoring generic model to a specific user or device

Level 2: Neuron Remapping• Same event triggers and features, new classifier configuration• Cloud invoked for redefinition of classifier stage (harnessing training data from all available sensors)• On-the-fly model change initiated by cloud

Example: Learn a new gesture or activity

Level 3: Algorithm Reprogramming• All new event triggers, features, and classifier• Full algorithm reconstruction via the cloud (harnessing training data from all available sensors)• Over-the-air sensor firmware update

Example: Provide an entirely new application

Throw away this stuff!?!? I never know when I might find a use for it!

A Cloud Centric ‘Big Data’ Analyst

Smart Sensing: Data Retention Limitations

Mitigations to the Need for Data Hoarding

Classifier Driven Sampling – Capture and store sampled anomalous raw data

Parameterized Sampling – Reduction of raw data to feature vectors

Localized Model Personalization – Per device customization of algorithm parameters (e.g. classifier weight factors)

Cloud Analytics(offline data mining,business intelligence)

Edge ANN/CNN(vision, spatial and image processing)

Cloud NLP(speech recognition, automated assistant)

Endpoint Rich Sensing ML(audio processing,motion and vibration classification)

Local Critical Insight ML(network failure independent,mission critical feedback/control)

Endpoint Real-Time ML(machine control, robotics)

The Role of Sensor Endpoints in IoT Analytics Processing

Endpoint: Localized Sensory Insight

Cloud: Broad Contextual Insight

Q & A