2015 SEMI MEMS Forum-06-Sensing and Sensibility for IoX in Smart Living and Healthcare6-IEK-20150902

8/20/2019 2015 SEMI MEMS Forum-06-Sensing and Sensibility for IoX in Smart Living and Healthcare6-IEK-20150902

1/40

Sensing and Sensibility for IoXin Smart Living and Healthcare

Stephen Su

General Director

ITRI IEK

9/2/2015


2/40

Outline

Smart Living Sensors Everywhere

Sensing and Sensibility Use Cases

Taiwan Sensor Industry Status

Conclusion

2


3/40

Smart Living(B2C)

IoT is Driving Both Smart Livingand Productivity

• Smart Living: B2C focus; Data analytics as latest thrust

• Productivity: B2B focus; Industry 4.0 as latest thrust

IT 1.0

Digital

IT 2.0

Internet

IT 3.0

Data

Industry 1.0

Capital

Intensive

Industry 2.0

Technology

Intensive

Industry 3.0

Innovation

Intensive

Industry 4.0

Cyber

Intensive

Productivity(B2B)

Internet of Things (IoT)

Cloud

Big Data

User Insight

e-Commerce

Cyber

PhysicalSystem Machining

3D Printing

Social Network

Internet

Source: Michael Porter(2014/11); ITRI/IEK(2015/05)

3


4/40

Future: Manufacturing byMass Customization

Right/Left Shoes with Different Sizes

Source:http://dblback.com/nike-now-3d-printing-shoes-money/; ITRI/IEK(2015/05)

3D printed shoes debut at 2014 SuperBowl

• Lattice structure to reduce weight

whi le maintaining strength andstructure

Future:• Feet sizes measured at home• Shoes custom made & delivered

to home

4

Sensors:

• G-Sensor, gyroscope,force sensor,piezoelectric sensors,energy harvester tomeasure gait, posture,weight and supply power

http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/http://dblback.com/nike-now-3d-printing-shoes-money/


5/40

Smart Living Sensors are Everywhere

Smart Living

SmartPlug

Gas Detector

SmartMeter

Smart App liance

Thermostat

Smart Lighti ng

In Home Display

Smoke Detector

IP Cam

SmartLocks

MotionSensor

CurrentSensor

Light/Image/IR Sensor

BioSensor

MagneticSensor

EnvironmentSensor

Smart ClothSmart Shoes

Smart Blood glucose meterSmartFeeding bottle

DronesVR HMD

Gesture RecognitionGame Console

Smart Map

Smart Energy Smart Security

Smart HealthcareSmart Entertainment

Acoustic

Sensor

Robotics

Source: ITRI/IEK (2015/09)

5

• Four smart applications: Energy, Security, Entertainment, Healthcare

• Ubiquitous sensors are designed to meet users’ requirements


6/40

Four Major Sensing Technologiesfor Applications of Smart Living

Smart LivingSensors

Optical

Electrochemical

G-Sensor

Magnetic Sensor

Gyroscope

Temperature SensorPressure Sensor

Humidity Sensor

Gas Sensor

MEMSMicrophone

Acoust ic

Ultrasonic

Sensor

Others

3D Image Sensor

Mechanical & Magnetic

Ambient Light Sensor

Proximity Sensor

IR Sensor

RGB/Color Sensor

UV SensorEnergy Harvester Others

Others

Bio Sensor

Flow Sensor

Others

Source: ITRI/IEK (2015/09) 6

• Four technologies cover all sensing types: Contact, contactless,environmental, etc.

Motion, Geomagnetic, Contactless Sensing (People to People, People to Machine)

Environmental Sensing (People to Machine, Machine to Machine)


7/40

Smart Energy: Electrochemical Sensors ReduceEnergy Consumption, Increase Energy Efficiency

Smart Meter/Plug

Smart Lighting

HVAC/Thermostat/Appliance

Smart System Event Reaction PerformanceSensor

Current SensorVoltage Sensor

Hall Sensor

Ambient LightSensor

Temperature,Humidity,Pressure, Flow,CO/CO2 sensors

*Abnormal Powerconsumption

*Device shut down

*Plenty of sunshine

*Turn of f ordimming the light

*Abnormal AirQuality

*Open the windowsand heat exchanger

EnergySaving

EnergyEfficiency

Multi-SensingSensor HubS/W Algorithm

Sensor Network

Machine to Machine

Source: ITRI/IEK (2015/09)7

• Electrochemical sensors critical to keep environmental green and comfort

− Temperature, humidi ty, pressure, flow, CO/CO2 and other sensors

− Detect abnormal situations to quickly activate the emergency measures


8/40

Smart Security: Optical SensorsDetect Abnormal Intrusion

Image/IRSensor

Presence/Occupancy

Sensor

Fire SmokeSensor

Thief Intrusion

Smoke

Alarm, Electronic fence tostart, Automatic watering

Smart Systemand Sensor

Event Reaction Performance

Security

Safety

Multi-Sensing, Sensor Hub, S/W Algorithm, Sensor Network, Machine to Machine


• Optical sensors such as image/IR sensors to detect abnormalsituations and automatically, real-time confirm/alarm site security


9/40

Smart Entertainment: Image and AcousticSensors Realize the NUI Control


Virtual Reality

Face Recognition

Voice Recognition

Gesture Recognit ionGaming

Google Earth

4K Video Zoom in/Zoom Out

IMU, Magnetic . Accuracy,

Proximity Sensor

Image/IR Sensor,MEMSMicrophone

Amb ient Ligh t Senso r.Temperature Sensor,3DSensor

Sensing and Event Interaction Performance

Convenience

Entertainment

Multi-Sensing, Sensor Hub, S/W Algorithm, Sensor Network, Machine to Machine

9

• Image and acoustic sensors cri tical for b iometrics applications such facerecognition, voice recognition, and gesture recogni tion


10/40

Smart Healthcare: Sensors to MeetFitness and Wellness Requirements


Multi-Sensing, Sensor Hub, S/W Algor ithm, Sensor Network, Machine to Machine

MEMSMicrophone

PH Sensor IMU Sensor Piezo Sensor

MotionSensor

EEG Sensor Piezo/PVDF

Sensor Flex Sensor

sweat detectionavoid heat stroke

Muscle endurancedetection

Walking pos turedetectionHeart rate detection

Parkinson's disease

detection

Mindfulness training Abnormal

Breath detection

Abnormal Body

posture detection

10

• Use bio sensors to measure human’s vital signs such as heart rate, sweat,body temperature, blood pressure, breath…

• For the old people, EEG sensors to detect brain waves and mental statefor mindfulness training


11/40

From Healthcare to Medical: Sensors willDetect Illness in the Near Future


Multi-Sensing, Sensor Hub, S/W Algor ithm, Sensor Network, Machine to Machine

MEMSMicrophone

Gas Sensor MicroprobePressureSensor

Microfluidicchip

DNA Chip Biologicalactuator

Chemical sensingElectrodes, Strain gauge,

G-Sensor, Humidity Sensor

PM2.5 detectionGlucosedetection

IntracranialPressure DetectionHearing Aid

Toxic substancedetection Genetic Testing insulin injectionIntraocular pressure

measurement

11

• Bio sensors could not only detect human’s vital signs, but also detectintracranial pressure, intraocular pressure to sense symptom ofglaucoma and stroke

• Also detect illness via DNA chip, microfluidic chip, biological actuator,gas sensor, etc. in the near fu ture

https://www.google.com.tw/url?q=http://www.cnet.com/news/introducing-a-flu-vaccine-you-give-yourself/&sa=U&ei=dnWMU4nvL4OklQXkyYDYDg&ved=0CDEQ9QEwAjgU&usg=AFQjCNH8sfAUBNv84vP0KXQOcLwe-83SsA


12/40

Smart Living Applications Drive MarketOpportunities of Multi-Sensing

Mechanical&

Magnetic

Optical

Acoust ic

Electro-chemical

G-SensorGyroscope

Force Touch Sensor

Piezoelectric Sensor

Magnetic SensorMotion Combo

Sensor

RGB,UV Sensor3D Image Sensor

Radar, LIDAR Sensor

Miniature Spectrometer

ALL in One Light Senso r

IR Sensor

3D+IR Combo Sensor

Ultrasonic Sensor

Current Sensor

Temperature/Flow Sensor

Humidity/Pressure Sensor

Gas SensorEnvironment Combo

Sensor

MEMS Microphone

Sensor Appl icationFunction

EntertainmentEnergy HealthcareSecurity

PCG patchSonic

EnvironmentSense

Speechrecognition

VoiceCommand

Ambient &Proximity

Sense

Presence&Occupancy

Sense

Heart RateMeter,

TemperatureSense,

Perishablefood Sense

GestureRecognition

Equipment Micro VibrationSense

MuscleEndurance

Measurement,Pedometer,

Ant i-fall

HumanInterfaceControl

Air quali tySense, PowerConsumptionMeasurement

Smoke, Gas,CO/CO2Sense

PH, Sweat,Perishable

food, Disease,Body Fat,

Breath Sense

PiezoelectricShoesControl



13/40

Multi-Sensing + Processing + Transmission= IoT Sensor Module

Combo Sensor Sensor Hub Sensor Network

µC

Multi-Sensing Processing Transmission

MotionSensor

Acoust icSensor

OpticalSensor

EnvironmentalSensor

+

S/W

+

+

+

Energy Security Healthcare Entertainment

Sensors +S/W

µC) RF

Power Source

WirelessSensorModule

M2MSolution


• After multi-sensing, data acquired/processed by sensor hub and integratedvia algorithm for customized IoT applications before transmission/execute


14/40

2015-2019 Worldwide Sensor MarketWill Reach ~10% CAGR


14

Worldwide Sensor Market Forecast(in M$)

U S $ M

R e v e n u e s

0

5,000

10,000

15,000

20,000

25,000

30,000

35,000

40,000

45,000

2015(e) 2016(f) 2017(f) 2018(f) 2019(f)

Smart Security Smart Entertainment Smart Energy

Smart Healthcare Others

24,395

27,703

30,971

34,677

39,001

• Sensor market for Smart Security and Smart Entertainment relativelymatured, compare to Smart Energy and Smart Healthcare due to

deployment of smoke sensors, CMOS Image sensors, motion sensors, etc.• Smart Energy and Smart Healthcare Sensor market will growth faster


15/40

Outline




Conclusion

15


16/40

16

People/Needs critical to IoT SuccessIoT + People to Services = Internet of Minds (IoM)

Internet of Minds (IoM)

ThingsPeople

People toPeople(P2P)

Machine toMachine (P2P)

People toMachine

(P2M)

DataThings

Services

People toServices

(P2S)

Source: ITRI/IEK(2015/05)16


17/40

Use Case: Smart Healthcare with VRand Behavior Analysis

• Real Avatar – Virtual healthcare consultant

Build trust by sensing/imitating body language Body language different by cul ture

Source: Walter Greenleaf ；Stanford mediaX ITRI Theme Day（2014/06）

17


18/40

Use Case: Smart Entertainment with VRand Behavior Analysis

• Oculus VR Sensor Appl ication Motion Sensor for sensing/imitating body language Magnetic Sensor for Position sensing Near-infrared CMOS sensor for tracing the posit ion and direction of User’s head

In the Movie

In the Game

Interact with movie content

Everybody can become a superhero

Source: Oculus VR; ITRI/IEK(2015/08) 18


19/40

Use Case: Stanford Uses VR forFootball Quarterback Training

Source: Stanford; Jeremy Bailenson (2015/04)

19

Stanford Professor Jeremy Bailenson

• Training benefits: Decision making, time saving, space

saving, health protection, etc.

U C D C t


20/40

Use Case: Dropcam CreatesNew Smart Security Service Model

Sensor Node Residential and Remote Control Service Model

IP Cam

ImageSensor,

IR

Sensor,Mic,

Algori thm…

Model 1

Model 2

Model 3

Sensing Sensibility

• Cloud Storage(7 days video)：$9.95/Monthly；$99/Year

• Cloud Storage( 1 Month v ideo)：$29.95/Monthly；

$295/Year

• Add an IP Cam：Fee discount of50%

Source: Dropcam (2015/09)

Different from other competitors, focusing on IP Cam content cloud storage andcreate various paid Service or applications for cloud video capture and sharing

Use HW/SW integration to extend applications for tour ism, education, business

20

U C S t H lth d S it

http://www.google.com.tw/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCK7B9fSgwccCFQMtpgodbqYMDQ&url=http://appleinsider.com/articles/13/10/10/dropcam-pro-takes-quick-easy-wi-fi-based-home-video-surveillance-to-next-level&ei=48_aVe7DD4PamAXuzLJo&psig=AFQjCNGX1DBARyKmRxnABOdUU3H1TKcOJw&ust=1440489823729737


21/40

Use Case: Smart Healthcare and Securityfor Kid Care + Pet Care

Motion Sensors, Image/IR Sensors, Microphone, Environmental Sensors …+ Algorithm

S en s or N o d e

R

e s i d en t i al &

R em o t e C o

n t r ol

S er v i c eM o d el

S e

n s i b i l i t y

S en s i n g

• GPS + GLONASS• Monthly Fee：$4.95

TRACTIVE• pet wearable tractive：monitor position• Motion/activity Detection• Daily goal Remote：Communication, Training

• Kiband： App Unlock,Motion remind


Use wearable device sensors to realize remote monitoring and extendthe service model of monthly fee

21

C O O S


22/40

Use Case: OMRON Smart HealthcareService Model: Web UI+APP+Cloud Service

Sensor Node Gateway Cloud Service

Web UISmart Phone APP

Environmental Situation Sensing

Vital Information Sensing

Alerts and Advices

Sensing and Sensibil itySource: OMRON (2015/09)

Use sensors to confi rm comfort and security of people and environment

22

U B h b ild O S/W Pl tf


23/40

Thermostat Motion Tag Ambient Tag 洗衣機洗碗機

One API for different Brand Appliance

SensorNode

Gateway&Sensor Network

Open S/W Platform

Use case: Bosch builds Open S/W Platformto extend Smart Energy Service

IoT Cloud Service SmartEnergy

S e

n s i n g

S en s i b i l i t y

MEMS Sensor NodeLight Pressure

Rotation

Tilt Gravity

Acceleration

Movement

Humidity

Source: Bosch (2015/09)

In addition to sensors,Open S/W Platform and

API Play an Important roleto Enable equipmentinteroperability

23


24/40

Outline




Conclusion

24

T i IC D i H B


25/40

G-

SensorGyro

Magnetic

/Hall

Sensor

6Axis

IMU

Pressure

Sensor ALS

Proximity

Sensor

3 in 1

Light

Sensor

Heart

Rate

Sensor

CIS/3D

Sensor

RGB/

UV/

IR

Sens

or

Si-

Micro

phone

Ultraso

nic

Sensor

Environmental

Sensor

NA NA

Inertial & Magnetic Optical Acoust ic Electrochemical

3C Smart Living & IoT

Taiwan IC Design Houses Beganto Invest in Sensor Program

Source: ITRI/IEK (2015/09)25

Taiwan IC Design House began to build Sensor-related product solut ion andleverage Past customers and channel.

In the future, Taiwan’s sensor will shift to wearable device and IoT applicationgradually from 3C application

Taiwan Sensors Vertical Value Chain Become


26/40

Taiwan Sensors Vertical Value Chain BecomeIncreasingly Competit ive

IoT Sensor Module Design

SiP/WLP

Packaging & TestingFoundry

Surface BulkSensorSensor

HubSensor

Network

Sensor R&D

SensorDevelopment

…

…

Taiwan Sensors Value Chain

Smart Living Application Downstream System/SI Supplier

Wearable Device Embedded System System Integration


Taiwan Sensor ’s Value chain become solid compare to past, and suppor ted by domesticdownstream supplier gradually

S S S


27/40

Now Single Sensor with Single FunctionFacing Fierce Price Competit ion

G-Sensor

Gyroscope

MagneticSensor

0

0.5

1

1.5

2

2.5

3

2011 2012 2013 2014 2015

G-Sensor Gyroscope Magnetic Sensor

A S P

： $


Taiwan Sensor ’s Vendors focus on single sensor development for Low-end

Tablet PC and Handsets mostly in the past Single Sensor(Single Function) is Facing fierce price competition so far

F t M S S/W I t ll i


28/40

Future: More Sensors, S/W, Intell igence

6 Axis G-Sensor & Gyroscope

ForceTouchSensor

MEMSMicro Phone

Optical Pulse

Sensor

Ambient

Light Sensor

Temperature Sensor

Humidity Sensor

Pressure Sensor

Gas Sensor

IR Sensor

Flow Sensor

Others

Environmental Sensing Device

Wearable Device

Multi Sensing & Processing Always Connected

Economic Benefits Behavior Judgement

• Multi Sensor• vital signs, environment

awareness• Signal Integration

• Machine to Machine• Human to Machine• Human to Human

• S/W & Algor ithm operation

• Big Data Analysis

IoT Sensor Module

Sensors ,MCU RF, Power Source

Software API

• App Creat ion• Sensor based Service

Innovation


Future Sensor characteristic：Multi-Sensing & Processing , Modularize,

Always on/connected, BehaviorJudgement, S/W orientation…

Sensor based IoT Platform and servicewill become more popular

IoT Sensor Module: From Cost Driven


29/40

IoT Sensor Module: From Cost-DrivenMarket to Value-Driven Market

IoT SensorTotal Solution

Single SensorSmart Sensor

Module

Low cost , MassProduction

Major Player Solution Pin toPin Compatible, Commodity Big Data

Economics

IoT, CloudComputing

Gray Market Tablet PC, FeaturePhone, Low end Smart Phone

Energy, Securi ty,Healthcare, Entertainment…Cost Driven Value Driven

Single Sensor Low Price, Low Margin IoT Sensor Module Customized, High Price, High Value

TechnologySupply Chain

EfficiencyEnhancement

Sensor, Algori thm,MCU, RF…

eterogeneous

Integration

A S P

$


In order to avoid fierce price competition ，Sensor provider should develop more smartsensor module for customized IoT market, From cost-dr iven market to value-driven market

ITRI’s Comprehensive Sensor Technologies


30/40

ITRI s Comprehensive Sensor Technologies

MEMS Design

Packaging &Testing

Software &

Appl ications

Patent Analysis Circuit Design

Process & Materials


Broad R&D capabil it ies: From materials to MEMS/circui t design, fromprocess to packaging/testing, from IP to S/W & applications

ITRI’ S P tf li S t d


31/40

ITRI’s Sensors Portfolio Supportedby Cross-Disciplinary Laboratories

Electrochemical Sensors

• Temperature Sensor

• Humidity Sensor

• Flow Sensor• Gas Sensor

Electrochemical


ITRI Solutions for Smart Living/Healthcare


32/40

ITRI Solutions for Smart Living/Healthcare

NPNS Sensor ECG Patch Asthmadetector

WearableNPWT

UWB BioSensor

Cloud FitnessTrainer system

ITRI Healthcare Appl ication Sensor Technology and Device

WearableContactlessRadar Sensor

Soft patchaffixed to thebody betweenthe chest andabdomen

CaptureBreath soundsignal fromthe neck tobreathe

Integrate theVacuum pump&Specialpatch

ContactlessSensor todetect Breath

IntegrateWearable Device& Camera MotionSensingTechnology

Technology Transfer & Co-operation

HOLUX, IMAGER-37, NTUH, Taipei Chang Gung Memorial Hospital…


ITRI Smart Living Application Sensor and Device

Wearable Alcohol Detector Home Energy Management System

ITRI is aggressive to transfer sensor technology to industry and co-operate with ICTcompanies and various Hospital

32

O tli


33/40

Outline




Conclusion

33

C l i


34/40

Conclusion

• Ubiquitous sensors wil l accelerate the rapid growthof IoT and Smart Living Market

• Sensing must be augmented with sensibility to haveunique user value creation

• Sensor will change from cost driven to value drivenand become customized wireless sensor module tomeet various IoT application requirements

• S/W wil l become more and more important eventually l inking the big data analysis to create aninnovative service and business model

34

Appendix


35/40

Appendix

35

User Case: Nest Creates New Business Model


36/40

User Case: Nest Creates New Business Model

of Big Data Economics


Sensor Node Gateway Big Data EconomicsSensing Sensibility

Sensors + S/W

SensorHub(µC)

SensorNetwork(RF)

Power Source

Residential/RemoteControl

Cloud ComputingService

Energy saving mode

Control

Database of User

Electrical behaviorNew Business Modelof Energy Service

Nest collects environmental data , analysis these big data, and learnuser’s behavior, then build new service model and data economics

36

User Case: Qualcomm Life Smart


37/40

User Case: Qualcomm Life SmartHealthcare: End-to-End Connectivity

Sensing Sensibility

Data Measurement Data CollectionData Storage&

Transmission

Healthcare

Service

Equipment Suppliers Equipment Suppliers Telecom Operator Service Provider

Portable DeviceSensor

Health MeasurementDevice

Smart Medical aids

Hub *Remote PhysiologicalMeasurement*Health Educatio n Services*Video Consulting Service*Pharmaceutical Securit y

Services*Living Resource Referral*Warning Security Adviso ry

Source: Qualcomm Life；ITRI/IEK (2015/09)

Qualcomm Life Service try to linkHealthcare Porfessionals,Patients

and Payers

37

LG Smart Home Sensing Value: Create

http://www.google.com.tw/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0CAcQjRxqFQoTCNzQ4Km7wccCFcallAodZC8M3w&url=http://www.medicalautomation.org/2012/01/qualcomms-branch-for-wireless-medical-devices/&ei=levaVZyyF8bL0gTk3rD4DQ&psig=AFQjCNHsqgS-LQbjezxGsHv0tc3F6kpZig&ust=1440496651925550


38/40

LG Smart Home Sensing Value: CreateDiagnosis, Control & Maintenance Service

Sensor Node Residential and Remote Control Service Platform

NFC Tag, Sensor… Home Chat APPSmart diagnosis, Energy

Saving Control,Maintenance

Sensing Sensibility

Source: LG；ITRI/IEK (2015/09)

LG use lots of sensors in Home appliance, not only detect the situation of device operation,but also build Diagnosis, Control and Maintenance Service model

38

Use case: SmartThings Realize Smart


39/40

Use case: SmartThings Realize SmartEnergy and Security Open Platform

SensorNode

OpenS/W

Platform

Hub

ServiceModel

SmartPower Outlet

Smart SenseMoisture

Smart SenseMotion

Smart SensePresence

Smart SenseMulti-Sensor

Smart Sense Hub

Partnership System/OS interoperability

...

Premium Service/APP

S en s i b i l i t y

S en s i n g

Source: SmartThings；ITRI/IEK (2015/09)

SmartThihgs also bui ld an Open S/W Platform, app and intend to cooperate with di fferentH/W & OS Key suppliers to achieve the goal of interoperabili ty of Smart Home Appliances

39


40/40

http://ieknet.iek.org.tw/

Given the rapid pace of change in cutting-edge technology and industry development， the timeliness andcomprehensiveness of the information included in this presentation cannot be guaranteed by ITRI. Users of thispresentation shall bear full liability for any injury or loss that may be sustained as a result. The Copyright of this presentationbelongs to ITRI and none of this presentation， either in part or in whole， in any form， may be reproduced， publicly

i d difi d di ib d d b h i h i i f ITRI

Thank you

Stephen Su

General Director

[email protected]

+886-3-5912548

Meng Hsien Hsieh

Researcher

[email protected]

+886-3-5913233

2015 SEMI MEMS Forum-06-Sensing and Sensibility for IoX in Smart Living and Healthcare6-IEK-20150902

Documents

Transcript of 2015 SEMI MEMS Forum-06-Sensing and Sensibility for IoX in Smart Living and Healthcare6-IEK-20150902