Mobile Sensing

Huber Flores huber@ut.ee

Tartu, Estonia, 2014

http://math.ut.ee/~huber/

Outline

• What is mobile sensing?

• Mobile sensing technologies

• Development of smartphone sensing apps with Android

– Gesture and activity detection

• Next generation mobile sensing

– Event-based sensing through notifications

Huber Flores 2

MOBILE SENSING

Huber Flores 3

What is mobile sensing?

• Individual

– Gesture

– Activity

– Location

– Proximity

• Group

– Smart mobs

Huber Flores 4

Source: http://clout-project.eu/

MOBILE SENSING TECHNOLOGIES

Huber Flores 5

Integrated sensors

Huber Flores 6

Compass

Gesture

Orientation Positioning

Acceleration

Integrated sensors: Accelerometer

Huber Flores 7

Source: Flores, H, et al. "Zompopo: mobile calendar prediction based on human activities recognition using the accelerometer and cloud services." IEEE NGMAST 2011.

Integrated sensors: Magnetic field

Huber Flores 8

Integrated sensors: Gyroscope

Huber Flores 9

Integrated sensors: Barometer

Huber Flores 10

Source: Muralidharan, Kartik, et al. "Barometric phone sensors: more hype than hope!." Proceedings of the 15th Workshop on Mobile Computing Systems and Applications. ACM, 2014.

Non-integrated sensors

Huber Flores 11

Non-integrated sensors

Huber Flores 12

Microcontroller: Arduino Mega ADK

Non-integrated sensors

Huber Flores 13

Wireless shield

Sensor shield

Non-integrated sensors

Huber Flores 14

Wireless module Usb to serial converter

Non-integrated sensors

Huber Flores 15

DEVELOPMENT OF SMARTPHONE SENSING APPS

Huber Flores 16

Mobile Sensing: Android

Huber Flores 17

Mobile sensing: Android

• Gesture API

Huber Flores 18

Mobile sensing: Android

• Start your Android emulator with Google APIs

• Download source code – https://github.com/huberflores/AndroidSensorSimula

tor.git

Huber Flores 19

DEMO

Huber Flores 20

Exercise

• Try gesture API

Huber Flores 21

What about other sensors?

Huber Flores 22

Mobile sensing: Android

• Physical device

• Android emulator?

• Sensor simulator?

Huber Flores 23

Mobile sensing: Android

• Sensor simulator tool – Download and extract

• http://code.google.com/p/openintents/wiki/SensorSimulator (SensorSimulator 2.0-rc1.zip)

• Install app in Android emulator – $ adb # command in /android-sdk/platform-tools/

– $ adb install SensorSimulatorSettings-2.0-rc1.apk

• Execute sensor simulator – $ java –jar sensorsimulator-2.0-rc1.jar

– Set IP address in Android emulator

– Alternatively, this repository contains everything you need (https://github.com/huberflores/AndroidSensorSimulator.git)

Huber Flores 24

Mobile sensing: Android

Huber Flores 25

Mobile sensing: Android

Huber Flores 26

Mobile sensing: Android

Huber Flores 27

DEMO

Huber Flores 28

Exercise

• Try and install sensor simulator

• Run accelerometer app

Huber Flores 29

Mobile sensing: Android + Arduino

Huber Flores 30

Mobile sensing: Android + Arduino

• Hardware requirements

– Accessory mode

• Usb cable

– Communication modules

• Wifi

• Bluetooth

Huber Flores 31

Mobile sensing: Android + Arduino

• Software requirements (Development)

– Arduino IDE

• Sketch (C, C++)

– Hterm console

• Communication testing

Huber Flores 32

Mobile sensing: Android + Arduino

Huber Flores 33

• Arduino side

Mobile sensing: Android + Arduino

• Android side

– Add permissions to AndroidManifest.xml

– Implement data streaming communication

Huber Flores 34

DEMO

Huber Flores 35

NEXT GENERATION MOBILE SENSING

Huber Flores 36

Next generation mobile sensing

Huber Flores 37

Notification services

• Apple Push Notification Service (APNS)

• Microsoft Push Notification Service (MPNS)

• Google Cloud Messaging (GCM)

• Message Queuing Telemetry Transport (MQTT)

• Extensible Messaging and Presence Protocol (XMPP)

Huber Flores 38

Notification services

Huber Flores 39

Source: Flores, H, et al. "Mobile cloud messaging supported by XMPP primitives." Proceeding of the fourth MobiSys workshop on Mobile cloud computing and services. ACM, 2013.

Notification services

• Push gateway proxy

– Broker

– Subscribers

• Remote executions

• Polling (e.g. HTTP)

• Heartbeats

– Highly integrated and optimized algorithms

• Low resource consumption

Huber Flores 40

Notification services

• Download source code – https://github.com/huberflores/GCMClient.git

• SENDER_ID = 66267873XXXX

• SERVER_URL = http://xxx.xxx.xxx.xxx:8080/GCM-

mavenized-server

• Server code (in case you want to try at home)

– https://github.com/huberflores/GCMServer.git

Huber Flores 41

DEMO

Huber Flores 42

Exercise

• Implement a notification client in Android

Huber Flores 43

Next generation mobile sensing

• Participatory sensing

• Community sensing

• Mobile crowdsourcing

Huber Flores 44

QUESTIONS?

Huber Flores 45

References

• Suggested readings – Srirama, S; Flores, H; and et al. “Zompopo: mobile

calendar prediction based on human activities recognition using the accelerometer and cloud services“, NGMAST 2011, Oxford, UK, September 14-16.

– Arduino tutorial https://courses.cs.ut.ee/2013/MAD/fall/uploads/Main/ArduinoTutorialWithExample.zip

– Source code https://github.com/huberflores?tab=repositories

Huber Flores 46

Home assignment 2

• We already explored the utilization of micromechanical sensors within the mobile applications – We will focus in the accelerometer

• You can use sensor emulator or a physical device for this homework – If you don’t have a device, we can provide you one, but

request it with anticipation

• Sensor emulator demonstration was provided along with some code for sensor simulator

• Sensor working code for a real device was also provided – It can be utilized as core if you find it usable, but you can

always code yours

Huber Flores 47

Home assignment 2

Huber Flores 48

Home assignment 2

Huber Flores 49

Home assignment 2

• Object moves from left to right (and vice-versa) based on the orientation of the device

• A touch event makes the object to jump over an obstacle

Huber Flores 50