Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series

1

Thesis Defense for the degree of Doctor of Philosophy Electrical and Computer Engineering Carnegie Mellon University

Jiang Zhu [email protected]

Thesis Committee

Prof. Joy Zhang, Chair Prof. Jason Hong

Prof. Patrick Tague Dr. Fabio Maino, Cisco Research

2

•  Data collected from physical and soft sensors •  Identify various behavioral factors from sensor streams •  Build behavioral models to generate quantifiable metrics •  Apply these models to a mobile security application: SenSec

Study the fundamental scientific problem

modeling a mobile user’s behavior

heterogeneous sensory time-series

of from

4

North America

Global

0

50

100

150

200

250

300

2011 2011-2016

15.5 58.3

33.6

256.7 Tablets

North America

Global

0

500

1000

1500

2000

2500

2011

2011-2016

138 253

698

2027 Smartphones

•  Cisco Visual Networking Index VNI 2012, Cisco Systems Inc. 2012

Smartphone Tablets 3-82-4 2016

5

0%

10%

20%

30%

40%

50%

60%

Mobile Device Loss or theft

Strategy One Survey conducted among a U.S. sample of 3017 adults age 18 years older in September 21-28, 2010, with an oversample in the top 20 cities (based on population).

•  “The 329 organizations polled had collectively lost more than 86,000 devices … with average cost of lost data at $49,246 per device, worth $2.1 billion or $6.4 million per organization.

"The Billion Dollar Lost-Laptop Study," conducted by Intel Corporation and the Ponemon Institute, analyzed the scope and circumstances of missing laptop PCs.

6

Reliable Non-intrusive

Password

Application

Usability

A major source of security vulnerabilities. Easy to guess, reuse, forgotten, shared

Different applications may

have different sensitivities

Authentication too-often or sometimes too loose

7

Passwords Normal passwords are not strong enough: usually meaningful words that can be remembered Stringent strong password can be annoying

Most users do not use the password-aid tools (Hong et al. 2009)

Fingerprint? Iris recognition? Face recognition? Voice recognition?

Password for the DHS E-file: Contain from 8 to 16 characters Contain at least 2 of the following 3 characters: uppercase alphabetic, lowercase alphabetic, numeric Contain at least 1 special character (e.g., @, #, $, %, & *, +, =) Begin and end with an alphabetic character Not contain spaces Not contain all or part of your UserID Not use 2 identical characters consecutively Not be a recently used password

8

•  Provides a way to passively authenticate while using common, sensitive applications and services.

•  Allows for rapid detection of unauthorized users Block their access as quickly as possible.

•  Uses a variety of sensors available on common smartphones

8

9

•  Derived from

•  Behavioral: the way a human subject behaves

•  Biometrics: technologies and methods that measure and analyzes biological characteristics of the human body

•  Finger prints, eye retina, voice patterns

•  Behaviometrics: Measurable behavior to Recognize or to Verify •  Identity of a human subject, or

•  Subject’s certain behaviors

Behavioral Biometrics Behaviometrics

10

•  Mobile devices come with embedded sensors •  Accelerometers, gyroscope, magnetometer •  GPS receiver •  WiFi, Bluetooth, NFC •  Microphone, camera, •  Temperature, light sensor •  “Clock” and “Calendar”

•  Connect with other sensors •  EEG, EMG, GSR

•  Mobile devices are connected with the Internet •  Upload sensor data to the cloud •  Viewing information computing on the server side

•  Users carry the device almost at all time. •  My phone “knows” where I am, what I am doing and my future

activities.

11

•  Motion Metrics

•  Location Metrics

•  Interaction Metrics

•  System Metrics

•  Accelerometer •  activity, motion, hand trembling, driving

style •  sleeping pattern •  inferred activity level, steps made per

day, estimated calorie burned •  Motion sensors, WiFi, Bluetooth

•  accurate indoor position and trace. •  GPS

•  outdoor location, geo-trace, commuting pattern

•  Microphone, camera: •  From background noise: activity, type

of location. •  From voice: stress level, emotion •  Video/audio: additional contexts

•  Keyboard, touches, slides •  Specific tasks, user interactions, …

12

•  Monitor and track user behavior on smartphones using various on-device sensors

•  Convert sensory traces and other context information to Behaviometrics

•  Build statistical models with these features and use them for calculation of Certainty Scores as security measure

•  Trigger various secondary Authentication Schemes when certain application is launched or certain system function is invoked.

13

Raw Data Preprocessing

Modeling

Metrics

Evaluation

Metrics

Metrics Ground Truth

Metrics

Metrics

+

14

Heterogonous Sensor Data

Feature Extraction Behavioral Text. Frequency Rep.

n-gram Skipped n-gram Helix, NN,LR,

DT, RF, SVM…

Motion Metrics

Accuracy TPR & FPR DR & FRR

Location Metrics

Interaction Metrics

Sim. Attacks Ctrl. Exp.

Auth. Records

MobiSens Framework

+

16

•  Human behavior/activities share some common properties with natural languages

•  Meanings are composed from meanings of building blocks •  Exists an underlying structure (grammar) •  Expressed as a sequence (time-series)

•  Apply rich sets of Statistical NLPs to mobile sensory data

3

3.5

4

4.5

5

5.5

6

0 20 40 60 80 100 120 140 160 180 200

log

(fre

q)

Rank of words by frequency

Zipf’s Law

17

•  Is this play Shakespeare’s work?

•  Comparing the play to Shakespeare’s known library of works

•  Track words and phases patterns in the data

•  Calculate the probability the unknown U given all the known Shakespeare’s work {S}

•  Compare with a threshold θ •  Authentic work (a=1) •  Fake, Forgery or Plagiarism (a=0)

a = sign[P (U |{S}) > �]

18

Quantization Clustering

19

•  Convert feature vector series to label streams – dimension reduction

•  Step window with assigned length

A1 A2 A1 A4

G2 G5 G2 G2

W2 W1 W2

P1 P3 P6 P1

A2 G2G5 W1 P1P3 A1A4 G2 W1W2 P1

20

Applications

Modeling

PreprocessingSensing

Feature Construction

Behavior Text Generation

N-gramModel

Classification

Threshold

Skipped N-gramModel

Helix

DT

EvaluationROC Accuracy

PrecisionRecall UX

Prediction RecognitionSVMDT SVM

Ground Truth

Anom

aly D

etection

...

21

Quantization

Risk Analysis Tree

Clustering

Activity Recognition

<

Application Sensitivity

Application Access Control

Certainty of Risk

Sensor Fusion and Segmentation

Application Access Control

22

Inference

ModelingPreprocessingSensing

FeatureConstruction

Behavior TextGeneration

N-gramModel

Cla

ssifie

rB

inary

Cla

ss

ifierThreshold

UserAuthentication

UserClassification

•  SenSec collects sensor data • Motion sensors • GPS and WiFi Scanning • In-use applications and their traffic patterns

•  SenSec modulebuild user behavior models •  Unsupervised Activity Segmentation and model the sequence using Language model •  Building Risk Analysis Tree (DT) to detect anomaly •  Combine above to estimate risk (online): certainty score

•  Application Access Control Module activate authentication based on the score and a customizable threshold.

23

•  Accelerometer •  Used to summarize acceleration stream •  Calculated separately for each dimension [x,y,z,m] •  Meta features: Total Time, Window Size

•  GPS: location string from Google Map API and mobility path

•  WiFi: SSIDs, RSSIs and path

•  Applications: Bitmap of well-known applications

•  Application Traffic Pattern: TCP UDP traffic pattern vectors: [ remote host, port, rate ]

24

•  Offline data collection (for training and testing) Pick up the device from a desk Unlock the device using the right slide pattern Invoke Email app from the "Home Screen” Some typing on the soft keyboard Lock the device by pressing the "Power" button Put the device back on the desk

26

•  71.3% True-Positive Rate with 13.1% False Positive with only 2-3 days of training data

27

•  Various Behaviometric applications be framed as a classification problem

•  Prerequisite: a learned Behaviometric model •  Input: a given observation of user’s behavior, •  Output: some decisions based on the observation and the model

•  Behavioral text representation may lead to huge features space •  Algorithms to handle large feature space •  Smart feature set construction to limit size of the feature space

•  Identification and anomaly detection work better if the users are performing the same activity

•  Activity recognition before identification/detection

•  High FPR •  Incorporate other factors and metrics •  UX improvements

29

•  The language model in general builds a single model for all types of activities.

•  Often the way people perform a certain activity is enough to distinguish them(as in PoC).

•  Models to identify between the diff. of how people perform the same, or same class of activities.

0

5

10

15

20

0 20 40 60 80 100 120 140

acceleration

time

Accelerometer X-axis

standingwalkingrunning

Activity Class

Extraction

AC-1 Model

AC-2 Model

AC-n Model

…

30

4.3 MODELING PERSONAL FACTORS FROM MOTION SENSOR DATA 50

0

5

10

15

20

0 20 40 60 80 100 120 140

acceleration

time



0

100

200

300

400

500

600

700

0 1 2 3 4 5 6 7

amplitude

frequency



0

5

10

15

20

0 20 40 60 80 100 120 140

acceleration

time

Accelerometer Y-axis


0

200

400

600

800

1000

1200

1400

0 1 2 3 4 5 6 7

amplitude

frequency



0

5

10

15

20

0 20 40 60 80 100 120 140

acceleration

time

Accelerometer Z-axis


0

100

200

300

400

500

600

0 1 2 3 4 5 6 7

amplitude

frequency

Accelerometer Z-axis


Figure 4.7: Accelerometer readings for standing, walking and running along with the discretefourier transformation

31

•  Extract the activity level of motion time series data by examining magnitude of the data and frequency composition via DFT

•  Activity Level =

•  Activity Class can be arbitrary segments of this function which has range [0, 1)

0

5

10

15

20

0 20 40 60 80 100 120 140

acce

lera

tion

time



0

200

400

600

800

1000

1200

1400

0 1 2 3 4 5 6 7

ampl

itud

e

frequency



32

•  t-SNE to map feature space into 2-D plane

•  First tested activity recognition method on ambulatory behavior with 381 samples of standing, walking, and running. Correctly classifies 359/381 samples, giving an accuracy of 94.23%

•  For user identification we ranked the features and used the top features to create models using classical ML algorithms

33

•  We approach the authentication problem by building a Multivariate Gaussian distribution for each activity class

•  We fit the parameters, mean and standard deviation, for each feature for a training set for which we define to contain only non-anomalous data

•  The calculate the probability of some unknown data being generated by the model with

34

•  Each range is an activity class with lower ranges representing activities with low magnitude or low frequency and higher ranges representing activities with high magnitude and high frequency.

•  110GB dataset of accelerometer data from 25 users.

35

•  Using only motion data can lead to a very high false positive rate

•  Combine motion with location factor to mitigate

•  Verify most users spend the majority time in a small number of places. Incorporate location in experiments lead to reduced FPRs

37

•  Hypothesis: the micro-behavior a user interacts with the soft keyboard reflects his/her cognitive and physical characteristics.

Cognitive fingerprints: typing rhythms, correction rate, delay between keys, duration at each key…. Physical characteristics: area of pressure, amount of pressure, position of contact, shift …

38

•  Keystroke Dynamics are a popular subject Many papers—focusing primarily on desktops

Great success for passwords, good success for arbitrary text

Typing rate, key-to-key latencies are the primary features

Once people are skilled at typing, they develop natural rhythms (on desktops)

•  Detecting keystroke patterns on mobile phones is challenging Focus on Desktop-like attributes

Typing rate, timing, di-graphs, tri-graphs, etc.

•  Use background applications to “sniff” keystrokes Without direct access to keyboard

Successful demonstrations using accelerometers

38

Cisco Confidential © 2010 Cisco and/or its affiliates. All rights reserved. 39

•  Frequent use •  Typically single user

•  Context awareness •  Protected applications vs. Non-

protected •  Current and historical patterns

•  Touchscreens provide wealth of data •  Touch location, pressure, finger

size, finger drift •  Wide variety of other sensors

•  Accelerometers, gyroscopes

•  Limited computing power •  Need to use efficient algorithms

•  Finite battery life •  Users are sensitive to battery life

impact •  Highly mobile

•  Typical usage: lying down, sitting, walking, passenger in car/train/subway system

•  Need to behave gracefully

40

•  Location pressed on keys •  Length of press

key down to key up •  Force of press

Varies across device types •  Change of force over key press •  Size of finger

Varies across device types •  Drift of finger during press •  Recent accelerometer history

44

•  From finger down to finger up

46

•  13 initial users after short recruiting drive

•  2 week long collection period

•  86,000 keystrokes

•  430,000 data points @ ~5/keystroke

•  Data split into training and testing:

•  99% accuracy with just 1 key stroke

Training Data for Model 50%

CV 15%

Training for Keys 15%

Final Testing 15%

CV for Keys 10%

48

Anomaly Detection Rate: 67.7%

FRR(FPR):4.6%

49

Anomaly Detection Rate:84.8%, increased from 67.7% for 5 strokes

FRR(FPR): 2.2%, reduced from

4.6% for 5 strokes

51

•  Alpha test started in 6/2012, 1st Google Play Store release in 10/2012

•  False Positive: 13% FPR still annoying users sometimes

•  75% alpha users mentioned feeling annoyed when SenSec prompted for passcode, at least once. Some of them are confused by the UI.

“I couldn’t get the passcode right multiple times when trying to answer an important phone call because I was single-handed”

“I just entered my passcode a min ago and it asked me again” “I was so lost about this UI. What’s this training and testing knob?”

52

•  Use adaptive model •  Adding the trace data shortly before a false positive to the training data and

update the model

•  Add sliding pattern in addition to passcode validation

•  A confirmed false positive will grant a “free ride” for a configurable duration

•  Assumption: just authenticated user should control the device for a given period of time

•  “free ride” period will end immediately if abrupt context change is detected.

•  Interaction Metrics added as part of the sensory fusion •  Motion, location and interaction metrics work together •  Different user flow: Reactive triggering vs. Proactive triggering

53

6.1 USER INTERFACE AND USER EXPERIENCE CHALLENGES 118

Figure 6.3: SenSec Protection Options: Secured, Protected via either Passcode or Sliding Pattern

Figure 6.4: SenSec Home Screen

54


Figure 6.2: SenSec Tutorial


Figure 6.3: SenSec Protection Options: Secured, Protected via either Passcode or Sliding Pattern

Figure 6.4: SenSec Home Screen

55

•  Data collected from physical and soft sensors •  Identify various behavioral factors from sensor streams •  Build behavioral models to generate quantifiable metrics •  Apply these models to a mobile security application: SenSec

Study the fundamental scientific problem

modeling a mobile user’s behavior

heterogeneous sensory time-series

of from

56

•  Propose a concept Behaviometric to study the fundamental scientific problem of modeling a mobile user’s behaviors from heterogeneous sensor time series

•  Adopt a Language approach to solve Behaviometric problems via various NLP techniques

•  Unsupervised algorithm Helix to discover the hierarchical structure, i.e. grammar, in activity recognition

•  Investigate existing statistical learning algorithms being adopted and applied to Behaviometric context when NLP approach is less sufficient

•  Derive an effective yet simple “activity level” metric from time series to be used in identification and detection. Use location metrics to augment motion metrics to reduce FPR.

•  Develop and deploy versions of SenSec app and adapt through user feedbacks.

57

•  Extended data set for System Metrics TCP, UDP traffic; sound; ambient lighting; battery status, etc.

•  Data and Modeling Gain more insights into the data, features and factorized relationships among various sensors

•  Enhanced security of SenSec components prepared for commercial release

Integration with Android security framework and other applications

•  Privacy as expectation (Liu et al., 2012) Users need to know where the data resides, how the data is going to be used and shared. Whom to trust the data with?

•  Energy efficiency

58

•  CyLab at Carnegie Mellon

•  Northrop Grumman Cybersecurity Research Consortium

•  Cisco Research aware for “A Language Approach in Behavioral Modeling” Research award for “Privacy Preserved Personal Big Data Analytics through Fog Computing’’

•  Special thanks: Dr. Hao Hu, Jiatong Zhou, Dr. Flavio Bonomi, Cisco Systems; Sky Hu, Twitter Inc.; Yuan Tian, Google Inc.; Pang Wu, Samsung Research North America.

58

Cybersecurity Research Consortium

59

“Mobile behaviometrics: Models and applications” In Proceedings of the Second IEEE/CIC Inter- national Conference on Communications in China (ICCC), Xi’An, China, August 12-14 2013., [with H.Hu, S.Hu, P.Wu, J.Zhang]

“MobiSens: A Versatile Mobile Sensing Platform for Real-world Applications”, MONE, 2013, [with P.Wu, J.Zhang]

"SenSec: Mobile Application Security through Passive Sensing," to appear in the Proceedings of International Conference on Computing, Networking and Communications. (ICNC 2013). San Diego, USA. January 28-31, 2013 [with P.Wu, X.Wang, J.Zhang]

“Towards Accountable Mobility Model: A Language Approach on User Behavior Modeling in Office WiFi Networks”, accepted to ICCCN 2011, Maui, HI, Aug 1-5, 2011 [with Y.Zhang]

"Retweet Modeling Using Conditional Random Fields," in the Proceedings of DMCCI 2011: ICDM 2011 Workshop on Data Mining Technologies for Computational Collective Intelligence, December 11, 2011.[ with H.Peng, D.Piao, R.Yan and Y.Zhang]

“Mobile Lifelogger - recording, indexing, and understanding a mobile user's life", in the Proceedings of The Second International Conference on Mobile Computing, Applications, and Services, Santa Clara, CA, Oct 25-28, 2010 [With S.Chennuru, P.Cheng, Y.Zhang]

"SensCare: Semi-Automatic Activity Summarization System for Elderly Care", MobiCase 2011, Los Angeles, CA, October 24-27, 2011. [with Pang Wu, Huan-kai Peng,Joy Ying Zhang] "Helix: Unsupervised Grammar Induction for Structured Human Activity Recognition," to appear in the Proceedings of The IEEE International Conference on Data Mining series (ICDM), Vancouver, Canada, Dec 11-14, 2011.[with Huan-Kai Peng, Pang Wu, and Ying Zhang] "Statistically Modeling the Effectiveness of Disaster Information in Social Media," to appear in the Proceedings of IEEE Global Humanitarian Technology Conference (GHTC), Seattle, Washington, Oct. 30 - Nov. 1st, 2011.[with Fei Xiong, Dongzhen Piao, Yun Liu, and Ying Zhang] "A dissipative network model with neighboring activation," to appear in THE EUROPEAN PHYSICAL JOURNAL B.[with F. Xiong, Y. Liu, J. Zhu, Z. J. Zhang, Y. C. Zhang, and J. Zhang]

"Opinion Formation with the Evolution of Network," to appear in the Proceedings of 2011 Cross-Strait Conference on Information Science and Technology and iCube, TaiBei, China, Dec 8-9, 2011.[with F.Xiong, Y.Liu, Y.Zhang]

Thank you.

61

Location

Time of the Day

Day of the week

R =f(handholding,

indoor loc, app)Alert!

Activity

R =f(WiFi trace,

app, time)

Traveling Speed

Gait R =f(geo trace,

app, time)

At home At work Between home and work

8am-6pm Other

Mon.-Fri. Sat. & Sun idle

Walking Driving

1

Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series

Technology

Transcript of Behaviometrics: Behavior Modeling from Heterogeneous Sensory Time-Series