Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Never Stand Still Faculty of Engineering Computer Science and Engineering

Click to edit Present’s Name

Never Stand Still Faculty of Engineering Computer Science and Engineering

Battery-free Internet of Things

Making the most of energy harvesting

Mahbub Hassan School of Computer Science and Engineering

University of New South Wales, Sydney, Australia

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Welcome to Internet of Things

Welcome to a Smart World

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Home

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Industry

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Farm

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Smart Health

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Welcome to batteries?

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Outline

•  Motivation for energy harvesting •  Part 1 - Energy harvesting IoTs •  Part 2 - Context detection from energy harvesting •  Conclusion and future directions

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Typical Energy Harvesting IoT

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Energy harvesting sources Temperature sensor using solar EH

Wireless EEG using thermoelectric EH

Vibr

atio

n R

F (T

V si

gnal

)

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

More EH IoT products (from enocean – industry standard)

Occupancy Sensor (solar powered) Door/Window Sensor (solar powered)

Self-powered wireless switch (pressure powered)

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

High power consumption for Wearable IoTs •  Continuous activity and context monitoring --- killer app •  Continuous motion sensing à high power consumption •  EH wearable IoTs more challenging

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Powering wearable IoT from energy harvesting convert motion energy to electrical energy

Piezoelectric Energy Harvester

Sensors

MCU

Radio

Harvested Power

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Scarcity of harvested energy

•  Humans generate a small amount of kinetic energy, but •  Accelerometer sampling is power consuming

Powering perpetual activity detection only using the harvested kinetic energy is a challenging problem

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Outline

•  Motivation for energy harvesting •  Part 1 - Energy harvesting IoTs

•  Part 2 - Context detection from energy harvesting •  Conclusion and future directions

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Our Idea – KEH as a sensor for context detection

KEH

walking

running

Harvested Power

•  Harvested energy is influenced by human activity, so activity should be detectable

•  Power saving potentials: unlike accelerometer, KEH does not consume power

•  KEH proxying as a motion sensor may have other benefits (simpler, smaller form factor for wearables)

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Piezoelectric Energy Harvester (vibration-based)

Mide.com

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Data Logger

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Data Logger

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #1 [IC2015] Human activity recognition – KEH Data Collection

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

KEH Time Series

Walking

Running

Standing

accelerometer KEH

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Does KEH contain information for activity recognition In

form

atio

n G

ain

Features

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Activity Recognition Accuracy – 5 activities (W, R, S, SU, SD)

Classifier Activity Recognition Accuracy (%)

Accelerometer-based

KEH-based

Hand Waist Hand Waist

K-nearest neighbour 95.01 98.70 81.13 87.01

Decision Trees 87.91 91.02 79.74 79.86

Multilayer Perceptron

88.25 96.39 78.29 85.52 Low

er th

an a

ccel

erom

eter

, bu

t not

too

bad

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #2 [iThings2015] Step counting

4 su

bjec

ts, s

trai

ght a

s w

ell

as tu

rnin

g pa

ths,

pea

k de

tect

ion

algo

rithm

, 570

st

eps,

96%

acc

urac

y

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #3 [BODYNET2015]

Calorie burning estimation •  10 subjects, 4 male, 6 female: waist mounted •  Anthropometric data: Age (26-35 years, mean = 29, s.d.

= 3.06), Weight (58-91 Kg, mean = 69.3 s.d. = 10.21), Height (154-185 cm, mean = 168.5, s.d. = 9.98)

•  Two activities: walking and running •  Linear regression model to estimate calorie burning from

energy harvesting samples and anthropometric data •  Leave-one-out cross validation (1 for testing, 9 for

training)

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Calorie estimation results - running

Close to accelerometer

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #4 [PerCom-WIP2016]

Transport mode detection

Train Car Bus Walking

Running

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Pause periods of train (pauses are removed)

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

2-layer classification

Peak analysis

Mean analysis

Summary of Trace Data

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Results

93% 77% 88%

Bus and Car are confused, but not train

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Exp #5 [WOWMOM2016]

Hotword detection

3cm

Quiet Room Conditions

Hotword: “OK Google” Non-hotwards: “Good morning”, “how are you”, “fine, thank you” 8 subjects: 4 m, 4 f 60 instances (30 hotword 30 non-hotword) per subject

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Speaker orientation

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Results

Flat Vertical

Speaker Independent

78% 62%

Speaker Dependent

85% 73%

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Conclusion

•  Vibration energy harvesting can detect a wide range of human contexts à power saving opportunity

•  Further research is required to improve context detection accuracy and reduce system power consumption

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

References 1.  [IC2015] Energy-Harvesting Wearables for Activity-Aware Services," IEEE

Internet Computing, 9(5), 2015 2.  [iThings2015] Step detection from power generation pattern in energy-

harvesting wearable devices,” IEEE iThings 2015

3.  [BODYNET2015] Estimating Calorie Expenditure from Output Voltage of Piezoelectric Energy Harvester - an Experimental Feasibility Study" BODYNETS 2015

4.  [PerCom_WIP2016] Transportation Mode Detection using Kinetic Energy Harvesting Wearables, PerCom Work-in-Progress, 2016

5.  [WOWMOM2016] Feasibility and Accuracy of Hotword Detection using Vibration Energy Harvester, WOWMOM 2016 (accepted).

Mahbub Hassan IEEE Distinguished Lecture, Malaysia, 26 May 2016

Questions?