The Pothole Patrol: Using a Mobile Sensor Network forRoad Surface MonitoringJakob Eriksson, Lewis Girod, Bret Hull, Ryan

Newton, Samuel Madden, Hari Balakrishnan

MIT Computer Science and Artificial Intelligence Laboratory

Outline

Introduction Architecture Data Acquisition Algorithm Performance Related Work Discussion

P2 : A mobile road surface monitoring system

Hazardous to drivers and increasing repair costs due to vehicle damage

Determine “which” roads need to be fixed Static sensors will not do well – requires mobility! P2 is first of its kind Challenge : differentiate potholes from other road

anomalies (railroad crossings, expansion joints)

Challenge : coping with variations in detecting the same pothole. (speed, sensor orientation)

P2 successfully detects most potholes (>90% accuracy on test data)

P2 Architecture Vehicles have GPS and 3-axis accelerometer

<time,location,speed,heading,3-axis acceleration>

Opportunistic WiFi/Cellular connections with dPipe to cope network outages

Taxi Testbed 7 Toyota Priuses1

Soekris 48012 Embedded Linux Wifi Card Sprint EVDO Rev A3 Network card GPS

Some numerical facts 9730 total kms 2492 distinct kms 7 cabs 174 km with >10 repeated passes

1. http://www.carbuyersnotebook.com/archives/Toyota_Pruis_2006.jpg2. http://www.pkgbox.org/Soekris-4801.jpg3. http://gizmodo.com/gadgets/peripherals/two-new-sprint-evdo-rev-a-cards-pantech-px500-and-sierra-wireless-aircard-595-200423.php

1

2

3

P2 ArchitecturePotholeRecord

Clustering

Cab 1GPS

3 Axis Accelerometer

LocationInterpolator

PotholeDetector

Cab 2GPS

3 Axis Accelerometer

LocationInterpolator

PotholeDetector

Central Server

P2 Architecture

Distance Traveled vs. Total Hours

Across All Taxis

Lower line represents unique roads

Segments of roads that were repeatedly covered

258,021 unique road segments

DATA ACQUISITION

Accelerometer placement Dashboard Windshield Embedded Computer

GPS Accuracy Standard deviation 3.3m

DATA ACQUISITION

Hand Labeled Data Smooth Road Crosswalks/Expansion

Joints Railroad crossing Potholes Manholes Hard Stop Turn

DATA ACQUISITION

Loosely Labeled Training Data We know only types of

anomalies and their rough frequencies

Exact numbers and locations are unknown

Extends available training set

ALGORITHM Features of accelerometer data High energy events are potholes?

Not really! Rail road crossings, expansion joints, door

slamming are high energy events Accelerometer data is processed by

embedded computer 256-sample windows Pass through 5 different filters

ALGORITHM - Filtering

Input Raw accelerometer data 256-sample windows

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM - Filtering

Speed Car is not moving or moving slowly Rejects door slam and curb ramp events

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM - Filtering

High-Pass Removes low-freq components in x and z axes Filters out events like turning, veering, braking.

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM - Filtering

z-peak Prime characteristic for significant anomalies Rejects all windows with absolute z-acceleration < tz

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM - Filtering

xz- ratio Assumes potholes impact only side of the vehicle Identifies anomalies that span width of the road (rail crossings,

speed bumps) Rejects all windows with

xpeak within Δw (=32) samples from zpeak < tx X zpeak

Or, ( Xpeak/ zpeak )< tx

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM - Filtering

speed vs. z ratio At high speeds, small anomalies cause high peak accelerations Rejects windows where Zpeak < ts X speed

or, (Zpeak /speed ) < ts

INWindowsof all event classes

Speed High-pass z-peak

xz-ratiospeed vs. z ratio

OUTPothole Detections

ALGORITHM – Sample Traces

ALGORITHM - Training

Tuning parameters t={tz,tx,ts} are computed using exhaustive search over a set of values

For each set t, we compute detector scores(t) = corr – incorr2

Corr is no. of pothole detections when sample was labeled as “pothole”

Maximize s(t) Include loosely labeled data

s(t) = corr – incorr2labeled – max(0,incorrloose – countr)

ALGORITHM - Clustering

Improve accuracy Cluster of at least k events must happen in the

same location with small margin of error(Δd) Clustering algorithm

Place each detection in Δd X Δd grid. Compute pairwise distances in same or neighboring grid

cells Iteratively merge pairs of distances in order of distance Max intra cluster distance < Δt Reported location is the centroid of the locations within it

ALGORITHM – Blacklisting &

False Negatives Well-known anomalies like bridges, railroad crossings, speed bumps etc can be located from GIS sources and blacklisted

GPS errors Pothole avoidance Biased detection will focus on critical

anomalies

PERFORMANCE EVALUATION Goals

Minimize false negative rate for smooth roads Never a flag a smooth road as anomaly

Missing a few potholes is acceptable Evaluation

1. Classification accuracy on hand-labeled data2. Performance improvement using loosely labeled

data3. Performance on loosely labeled roads4. Spot-checks

Performance on Labeled Data Randomly divided into training set and test set

False positive rate is 7.6% Not accurate

PERFORMANCE EVALUATION

Class Hand Labeled w/ Loosely Labeled

Pothole 88.9% 92.4%

Manhole 0.3% 0.0%

Expansion joints 2.7% 0.3%

Railroad Crossing 8.1% 7.3%

PERFORMANCE EVALUATION Estimating the false-positive rate

Ran the detector on loosely labeled roads

Helps set upper bound on false positive rate (at most 0.15%) on good roads.

Road # potholes # windows # detections rate

Storrow Dr. few 1865 3 0.16%

Memorial Dr. few 1781 2 0.12%

Hwy I-93 few 2877 5 0.17%

Binney St. some 6887 25 0.63%

Beacham St. many 1643 231 14%

PERFORMANCE EVALUATION Impact of features and thresholds

1. Only Z peak 2. w. xz-ratio filter 3. w. speed vs. z ratio

tx=1.5 tx=2.5ts=5

PERFORMANCE EVALUATION Performance under uncontrolled conditions

Slamming doors Fiddling with the sensor equipment Driving behaviors Deliberately avoiding potholes

Use clustering k=4

PERFORMANCE EVALUATION Spot Checks

Typical pothole Manhole Expansion joint

RELATED WORK

Surveys Falling weight deflectometer Machine vision – cameras, robots Accelerometer Microsoft Trafficsense – smartphones

DISCUSSION This is what I think

Innovative Ground truth establishment is tedious, expensive in dense

road networks Will it work in hilly areas ,slopes? Future work?

Driver feedback – Interactive embedded computers Smartphones – Cheaper solution, greater coverage

Comments/Questions ???

REFERENCES

The Pothole Patrol: Using a Mobile Sensor Network forRoad Surface Monitoring Jakob Eriksson, Lewis Girod, Bret Hull, Ryan Newton, Samuel Madden, Hari Balakrishnan MIT Computer Science and Artificial Intelligence Laboratory

U. Lee, E. Magistretti, B. Zhou, M. Gerla, P. Bellavista, and A. Corradi. MobEyes: Smart Mobs for Urban Monitoring with a Vehicular Sensor Network. IEEE Wireless Communications, 2006.

TrafficSense: Rich Monitoring of Road and Traffic Conditions using Mobile Smartphones Prashanth Mohan, Venkata N. Padmanabhan, and Ramachandran Ramjee {prmohan,padmanab,ramjee}@microsoft.com Microsoft Research India, Bangalore

http://research.microsoft.com/apps/pubs/default.aspx?id=70573