Bringing iot data to life, IoT Israel 2014
-
Upload
iotisrael -
Category
Technology
-
view
728 -
download
0
Transcript of Bringing iot data to life, IoT Israel 2014
6
IoTA: Analytics in Action!
Mobility Pattern Analytics
Behavior Learning & Prediction
Crowd Analytics
Anomaly Detection Anomaly Detection
7
Anomaly Detection Answers Difficult Questions
What just happened that shouldn‘t have?
• What does something that shouldn‘t have happened look like?
How can I find it in time?
• Before there is serious damage
• Before supply chains, customers, competitors and VIPs are impacted
Why are you disturbing my sleep?!
• False alarms are costly
8
Anomaly Detection Addresses Multiple Problems
Anomaly Detection
Framework
Traffic Incidents
Electrical Grid
Smart Home
Social Media
Crowd
Dike Stability
9
Traffic Anomaly Detection
86%
14%
Detection performance
True
positives
False
positives
Customer problems addressed
• Ensure road network efficiency, safety
• Minimize impact of traffic incidents
• Real-time, automatic detection of abnormal traffic congestion based on sensor data
2.00
0.11 0.00
5.00
Alert rates per day and road segment
Rule-based detection
Anomaly DetectionAnomaly detection
10
Traffic Anomalies Captured Live
Different anomalies are identified depending on index threshold and event filtering.
Abnormal traffic congestion identified
Construction worker caused traffic changes
Taxi parked for >15 min caused traffic changes
28-Nov 13:15 22-Nov 04:15 20-Nov 21:26
11
Engine Configuration for Traffic Incident Detection
Use-Case Specific Plugin
Generic Plugin
Detection and Classification
API
Context Dependency Modeling
Preprocessing
Normality Model Learning
Robust Density Estimation
Support Vector Machines
Apache Thrift
Timestamp Discretization
Auto Partitioning
Discrete Context
Switching
Hypothesis & Persistence Test
Noise Reduction
Data Imputation
Nearest Neighbor
Python Storm Spark MapReduce
Principal Component Analysis
Clustering
Manifold Learning
Feature Extraction
Traffic Parameter Extraction
Enhanced HMM System
Identification
Random Forest
Event Filter
Normalization
Robust Density Estimation
Timestamp Discretization
Auto Partitioning
Discrete Context
Switching
Hypothesis & Persistence Test
Noise Reduction
Data Imputation
Storm MapReduce
12
Traffic Anomaly Detection – Data and Extracted Patterns
Raw Data: LPR, VA Characteristic Features Normal Pattern Model
• Speed and volume per lane
• High frequency noise (10% - 30% std. dev.)
• Full/partial sensor outages
• 4 feature vectors:
• Average speed
• Total volume
• Lane average speed
• Lane speed difference
• Aggregation to 1 min interval
• Data cleaning
• Noise filtering
Multi-dimensional model
• Traffic features (4 dim.)
• Context dependency
• Time of day
• Day of week; public holidays
• Covariance optimization for robustness against anomalies in training set
13
Traffic Anomaly Detection – Anomaly Index
Anomaly Example – Features and Model
Mahalanobis Distance
• Mahalanobis Distance indicates magnitude of deviation between model and measurements
• Index threshold (red line) determines detection sensitivity
• Anomalies affect multiple traffic characteristics
• Deviation vectors used to further classify the type of anomaly
14
Results from Large-scale Deployment
1,000 LPR cameras 16 million vehicle detections/day
230 road segments analyzed
Same configuration applied across highways, on-ramps, urban arterials and side streets Events validated
on CCTV Low false alert rate Recurring congestion
suppressed
15
Moonscape Ventures
• Corporate development and investment company
• Launched August 2014; operates in TLV, NYC, Silicon Valley
• Grows startups: IoT, smart cities, big data, news and media, other
• Invests in late-seed stage, Series A round
• Led by Tammy Mahn