Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft...

78
Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th , 2009

Transcript of Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft...

Page 1: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Hidden Markov Map Matching Through Noise and Sparseness

Paul Newson and John KrummMicrosoft ResearchACM SIGSPATIAL ’09November 6th, 2009

Page 2: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Agenda

• Rules of the game• Using a Hidden Markov Model (HMM)• Robustness to Noise and Sparseness• Shared Data for Comparison

Page 3: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Rules of the GameSome Applications:• Route compression• Navigation systems• Traffic Probes

Page 4: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Map Matching is Trivial!

“I am not convinced to which extent the problem of path matching to a map is still relevant with current GPS accuracy”- Anonymous Reviewer 3

Page 5: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Except When It’s Not…

Page 6: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Our Test Route

Page 7: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Three Insights

1. Correct matches tend to be nearby

2. Successive correct matches tend to be linked by simple routes

3. Some points are junk, and the best thing to do is ignore them

Page 8: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Mapping to a Hidden Markov Model (HMM)

Page 9: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Three Insights, Three Choices

1. Match Candidate Probabilities

2. Route Transition Probabilities

3. “Junk” Points

Page 10: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Match Error is Gaussian (sort of)

0 2 4 6 8 10 12 14 16 18 200

0.02

0.04

0.06

0.08

0.1

0.12

GPS Difference Probability

Data Histogram Gaussian Distribution

Distance Between GPS and Matched Point (meters)

Page 11: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Route Error is Exponential

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

1

2

3

4

5

6

7

Distance Difference Probability

Data Histogram Exponential Distribution

abs(great circle distance - route distance) (meters)

Page 12: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Three Insights, Three Choices

1. Match Candidate Probabilities

2. Route Transition Probabilities

3. “Junk Points”

Page 13: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 14: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 15: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 16: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 17: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 18: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 19: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 20: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 21: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 22: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 23: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 24: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 25: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 26: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 27: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 28: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 29: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 30: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 31: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 32: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 33: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 34: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 35: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Match Candidate Limitation

• Don’t consider roads “unreasonably” far from GPS point

Page 36: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Route Candidate Limitation

• Route Distance Limit• Absolute Speed Limit• Relative Speed Limit

Page 37: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 38: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 39: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 40: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 41: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 42: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 43: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 44: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 45: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 46: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 47: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 48: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 49: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 50: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 51: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 52: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 53: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 54: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 55: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 56: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 57: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 58: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 59: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 60: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Robustness to Sparse Data

1 2 5 10 20 30 45 60 90 120

180

240

300

360

420

480

540

600

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Error vs. Sampling Period

Sampling Period (seconds)

Rout

e M

ismat

ch F

racti

on

Page 61: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 62: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 63: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 64: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Robustness to Sparse Data

1 2 5 10 20 30 45 60 90 120

180

240

300

360

420

480

540

600

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Error vs. Sampling Period

Sampling Period (seconds)

Rout

e M

ismat

ch F

racti

on

Page 65: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 66: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 second sample period 90 second sample period

Page 67: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 second sample period 90 second sample period

Page 68: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 second sample period 90 second sample period

Page 69: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Robustness to NoiseAt 30 second sample period

4.07 10 15 20 30 40 50 75 1000

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Accuracy vs. Measurement Noise

Noise Standard Deviation (meters)

Frac

tion

of R

oute

Inco

rrec

t

Page 70: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.
Page 71: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 seconds, no added noise

30 seconds, 30 meters noise

Page 72: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 seconds, no added noise 30 seconds, 30 meters noise

Page 73: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 seconds, no added noise 30 seconds, 30 meters noise

Page 74: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 seconds, no added noise 30 seconds, 30 meters noise

Page 75: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

30 seconds, no added noise

30 seconds, 30 meters noise

Page 76: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Data!http://research.microsoft.com/en-us/um/people/jckrumm/MapMatchingData/data.htm

Page 77: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Conclusions

• Map Matching is Not (Always) Trivial• HMM Map Matcher works “perfectly” up to

30 second sample period• HMM Map Matcher is reasonably good up to

90 second sample period• Try our data!

Page 78: Hidden Markov Map Matching Through Noise and Sparseness Paul Newson and John Krumm Microsoft Research ACM SIGSPATIAL ’09 November 6 th, 2009.

Questions?Hidden Markov Map Matching Through Noise and Sparseness

Paul Newson and John KrummMicrosoft ResearchACM SIGSPATIAL ’09November 6th, 2009