Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015]...

37
Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016 Never Stand Still Faculty of Engineering Computer Science and Engineering Click to edit Presents Name Never Stand Still Faculty of Engineering Computer Science and Engineering Battery-free Internet of Things Making the most of energy harvesting Mahbub Hassan School of Computer Science and Engineering University of New South Wales, Sydney, Australia

Transcript of Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015]...

Page 1: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Never Stand Still Faculty of Engineering Computer Science and Engineering

Click to edit Present’s Name

Never Stand Still Faculty of Engineering Computer Science and Engineering

Battery-free Internet of Things

Making the most of energy harvesting

Mahbub Hassan School of Computer Science and Engineering

University of New South Wales, Sydney, Australia

Page 2: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Welcome to Internet of Things

Welcome to a Smart World

Page 3: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Home

Page 4: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Industry

Page 5: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Farm

Page 6: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Health

Page 7: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Welcome to batteries?

Page 8: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Outline

•  Motivation for energy harvesting •  Part 1 - Energy harvesting IoTs •  Part 2 - Context detection from energy harvesting •  Conclusion and future directions

Page 9: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Typical Energy Harvesting IoT

Page 10: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Energy harvesting sources Temperature sensor using solar EH

Wireless EEG using thermoelectric EH

Vibr

atio

n R

F (T

V si

gnal

)

Page 11: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

More EH IoT products (from enocean – industry standard)

Occupancy Sensor (solar powered) Door/Window Sensor (solar powered)

Self-powered wireless switch (pressure powered)

Page 12: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

High power consumption for Wearable IoTs •  Continuous activity and context monitoring --- killer app •  Continuous motion sensing à high power consumption •  EH wearable IoTs more challenging

Page 13: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Powering wearable IoT from energy harvesting convert motion energy to electrical energy

Piezoelectric Energy Harvester

Sensors

MCU

Radio

Harvested Power

Page 14: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Scarcity of harvested energy

•  Humans generate a small amount of kinetic energy, but •  Accelerometer sampling is power consuming

Powering perpetual activity detection only using the harvested kinetic energy is a challenging problem

Page 15: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Outline

•  Motivation for energy harvesting •  Part 1 - Energy harvesting IoTs

•  Part 2 - Context detection from energy harvesting •  Conclusion and future directions

Page 16: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Our Idea – KEH as a sensor for context detection

KEH

walking

running

Harvested Power

•  Harvested energy is influenced by human activity, so activity should be detectable

•  Power saving potentials: unlike accelerometer, KEH does not consume power

•  KEH proxying as a motion sensor may have other benefits (simpler, smaller form factor for wearables)

Page 17: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Piezoelectric Energy Harvester (vibration-based)

Mide.com

Page 18: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Data Logger

Page 19: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Data Logger

Page 20: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #1 [IC2015] Human activity recognition – KEH Data Collection

Page 21: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Time Series

Walking

Running

Standing

accelerometer KEH

Page 22: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Does KEH contain information for activity recognition In

form

atio

n G

ain

Features

Page 23: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Activity Recognition Accuracy – 5 activities (W, R, S, SU, SD)

Classifier Activity Recognition Accuracy (%)

Accelerometer-based

KEH-based

Hand Waist Hand Waist

K-nearest neighbour 95.01 98.70 81.13 87.01

Decision Trees 87.91 91.02 79.74 79.86

Multilayer Perceptron

88.25 96.39 78.29 85.52 Low

er th

an a

ccel

erom

eter

, bu

t not

too

bad

Page 24: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #2 [iThings2015] Step counting

4 su

bjec

ts, s

trai

ght a

s w

ell

as tu

rnin

g pa

ths,

pea

k de

tect

ion

algo

rithm

, 570

st

eps,

96%

acc

urac

y

Page 25: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #3 [BODYNET2015]

Calorie burning estimation •  10 subjects, 4 male, 6 female: waist mounted •  Anthropometric data: Age (26-35 years, mean = 29, s.d.

= 3.06), Weight (58-91 Kg, mean = 69.3 s.d. = 10.21), Height (154-185 cm, mean = 168.5, s.d. = 9.98)

•  Two activities: walking and running •  Linear regression model to estimate calorie burning from

energy harvesting samples and anthropometric data •  Leave-one-out cross validation (1 for testing, 9 for

training)

Page 26: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Calorie estimation results - running

Close to accelerometer

Page 27: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #4 [PerCom-WIP2016]

Transport mode detection

Train Car Bus Walking

Running

Page 28: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Pause periods of train (pauses are removed)

Page 29: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

2-layer classification

Peak analysis

Mean analysis

Summary of Trace Data

Page 30: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Results

93% 77% 88%

Bus and Car are confused, but not train

Page 31: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #5 [WOWMOM2016]

Hotword detection

3cm

Quiet Room Conditions

Hotword: “OK Google” Non-hotwards: “Good morning”, “how are you”, “fine, thank you” 8 subjects: 4 m, 4 f 60 instances (30 hotword 30 non-hotword) per subject

Page 32: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Speaker orientation

Page 33: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Page 34: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Results

Flat Vertical

Speaker Independent

78% 62%

Speaker Dependent

85% 73%

Page 35: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Conclusion

•  Vibration energy harvesting can detect a wide range of human contexts à power saving opportunity

•  Further research is required to improve context detection accuracy and reduce system power consumption

Page 36: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

References 1.  [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE

Internet Computing, 9(5), 2015 2.  [iThings2015] Step detection from power generation pattern in energy-

harvesting wearable devices,” IEEE iThings 2015

3.  [BODYNET2015] Estimating Calorie Expenditure from Output Voltage of Piezoelectric Energy Harvester - an Experimental Feasibility Study" BODYNETS 2015

4.  [PerCom_WIP2016] Transportation Mode Detection using Kinetic Energy Harvesting Wearables, PerCom Work-in-Progress, 2016

5.  [WOWMOM2016] Feasibility and Accuracy of Hotword Detection using Vibration Energy Harvester, WOWMOM 2016 (accepted).

Page 37: Battery-free Internet of Thingsmahbub/PDF_Publications/IOT_DSP_KL_2016.pdf · 1. [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE Internet Computing, 9(5),

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Questions?