The Empirical Bayes Method for Safety

Estimation

Doug HarwoodMRIGlobal

Kansas City, MO

Key Reference

Hauer, E., D.W. Harwood, F.M. Council, M.S. Griffith, “The Empirical Bayes method for estimating safety: A tutorial.” Transportation Research Record 1784, pp. 126-131. National Academies Press, Washington, D.C.. 2002

http://www.ctre.iastate.edu/educweb/CE552/docs/Bayes_tutor_hauer.pdf

The Problem

You are a safety engineer for a highway agency. The agency plans next year to implement a countermeasure that will reduce crashes by 35% over the next three years. To estimate the benefits of this countermeasure, what safety measure will you multiply by 0.35?

What Do We Need To Know?

You need to know – or, rather, estimate – what would be expected to happen in the future if no action is taken

Then, you can apply crash modification factors (CMFs) for the known effects of planned actions to estimate their effects quantitatively

Common Approach: Use Last 3 Years of Crash Data

2008 2009 2010

Observed Crashes

30 19 21

More Data Gives a Different Result

2001 2002 2003 2004 2005 2006 2007 2008 2009 2010

ObservedCrashes

22 23 16 16 9 14 17 30 19 21

RTM Example with Average Observed Crashes

0

1

2

3

4

5

6

7

1993 1995 1997 1999 2001 2003 2005Year

Cra

sh

es

Long-term average (m)

3- Year average (Xa)

Random error

0

1

2

3

4

5

6

7

1993 1995 1997 1999 2001 2003 2005Year

Cra

sh

es

Long-term average (m)

3- Year average (Xa)

0

1

2

3

4

5

6

7

1993 1995 1997 1999 2001 2003 2005Year

Cra

sh

es

Long-term average (m)

3- Year average (Xa)Long-term average (m)

3- Year average (Xa)

Random error

“True Safety Impact of a Measure”

0

1

2

3

4

5

6

7

1993 1995 1997 1999 2001 2003 2005 2007

Year

Cra

shes

Long-term average (m)

3-year average ‘before’ (Xa)

True safety effect

Observed safety effect

3-year average ‘after’

Regression to the mean problem …

High crash locations are chosen for one reason (high number of crashes!) – might be truly high or might be just random variation

Even with no treatment, we would expect, on average, for this high crash frequency to decrease

This needs to be accounted for, but is often not, e.g., reporting crash reductions after treatment by comparing before and after frequencies over short periods

The “imprecision” problem …

Assume 100 crashes per year, and 3 years of data, we can reliably estimate the number of crashes per year with (Poisson) standard deviation of about…

However, if there are relatively few crashes per time period (say, 1 crash per 10 years) the estimate varies greatly …

or 5.7% of the mean

or 180% of the mean!

Things change…

BEWARE about assuming that everything will remain the same ….

Future conditions will not be identical to past conditions

Most especially, traffic volumes will likely change

Past trends can help forecast future volume changes

Focus on Crash Frequency vs. AADT Relationships: Use of Crash Rates May Be Misleading

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh F

requ

ency

Before After

C2

C1

E1 E2

R1

F1

F2

F3

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh F

requ

ency

Before After

C2

C1

E1 E2

R1

F1

F2

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh F

requ

ency

Before After

C2

C1

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh F

requ

ency

Before After

C2

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh F

requ

ency

Before After

C2

C1

E1E1 E2E2

R1R1

F1F1

F2F2

F3F3

The Empirical Bayes Approach

Empirical Bayes: an approach to estimating what will crashes will occur in the future if no countermeasure is implemented (or what would have happened if no countermeasure had been implemented)Simply assuming that what occurred in a recent short-term “before period” will happen again in the future is naïve and potentially very inaccurate

Yet, this assumption has been the norm for many years

The Empirical Bayes Approach

The observed crash history for the site being analyzed is one useful and important piece of information

What other information do we have available?

The Empirical Bayes Approach

We know the short-term crash history for the site

The long-term average crash history for that site would be even better, BUT… Long-term crash records may not available If the average crash frequency is low, even the long-term average crash frequency may be imprecise

Geometrics, traffic control, lane use, and other site conditions change over time

We can get the crash history for other similar sites, referred to as a REFERENCE GROUP

Empirical Bayes

Increases precisionReduced RTM biasUses information from the site, plus …Information from other, similar sites

3-17

Safety Performance Functions

SPF = Mathematical relationship between crash frequency per unit of time (and road length) and traffic volumes (AADT)

0

5

10

15

20

25

30

0 5000 10000 15000 20000 25000 30000

AADT

Cra

sh

Fre

qu

en

cy

3-18

How Are SPFs Derived?

SPFs are developed using negative binomial regression analysis

SPFs are based on several years of crash data

SPFs are specific to a given reference group of sites and severity levelDifferent road types = different SPFsDifferent severity levels = different SPFs

The overdispersion parameter

The negative binomial is a generalized Poisson where the variance is larger than the mean (overdispersed)

The “standard deviation-type” parameter of the negative binomial is the overdispersion parameter φ

variance = η[1+η/(φL)] Where …

μ=average crashes/km-yr (or /yr for intersections) η=μYL (or μY for intersections) = number of crashes/time φ=estimated by the regression (units must be complementary with L, for intersections, L is taken as one)

3-20

Regression model for total crashes at rural 4-leg intersections with minor-road STOP control

SPF Example

where:

Np = Predicted number of intersection-related crashes per year within 250 ft of intersection

ADT1 = Major-road traffic flow (veh/day)

ADT2 = Minor-road traffic flow (veh/day)

Np= exp(-8.69 + 0.65 lnADT1 + 0.47 lnADT2)

3-21

Calculating the Long-Term Average Expected Crash Frequency

The estimate of expected crash frequency:

Ne = w (Np) + (1 – w) (No)

Weight (w; 0<w<1) is calculated from the overdispersion parameter

ExpectedAccident

Frequency

PredictedAccident

Frequency

ObservedAccident

Frequency

Weight (w) Used in EB Computations

w = 1 / ( 1 + k Np)

w = weight k = overdispersion parameter for the SPF Np = predicted accident frequency for

site3-22

3-23

Graphical Representation of the EB Method

Predicting Future Safety Levels from Past Safety Performance

Ne(future) = Ne(past) x (Np(future) / Np(past))

Ne = expected accident frequency

Np = predicted accident frequency

3-24

Predicting Future Safety Levels from Past Safety Performance

3-25

The Np(future)/Np(past) ratio can reflect changes in:Traffic volume Countermeasures (based on CMFs)

CMFs—How to Use Them

CMFs are expressed as a decimal factor:CMF of 0.80 indicates a 20% crash reductionCMF of 1.20 indicates a 20% crash increase

CMFs—How to Use Them

Expected crash frequencies and CMFs can be multiplied together:

Ne(with) = Ne(without) CMF

Crashes Reduced = Ne(without) - Ne(with)

3-28

CMFs—Single Factor

CMF for shoulder rumble stripsRural freeways (CMFTOT = 0.79)

Ne(with) = Ne(without) x 0.79

3-29

CMF Functions

CMFs for Lane Width (two-lane rural roads) (Harwood et al., 2000)

CMFs for Combined Countermeasures

CMFs can be multiplied together if their effects are independent:

Ne(with) = Ne(without) CMF1 CMF2

Are countermeasure effects independent?

EB applications

HSMIHSDMSafety Analyst

EB applications

HSM Part C Estimate long-term expected crash frequency

for a location under current conditions Estimate long-term expected crash frequency

for a location under future conditions Estimate long-term expected crash frequency

for a location under future conditions with one or more countermeasures in place

HSM Part B Evaluate countermeasure effectiveness using

before and after data

EB applications

Site-Specific EB MethodBased on equations in this presentationProject-Level EB MethodIf project is made up of components with different SPFs, then there is no single value of k, the overdispersion parameter

EB Before-After Effectiveness EvaluationSee Chapter 9 in HSM Part B

Questions?