State Data Quality State Data Quality Review ProgramReview Program

Improving the Quality of State Crash and Inspection Data Uploaded to MCMIS

State Data Quality Review Program: Components

Select state Preanalysis On-site visit Post Visit

– Document data reporting processes, observations and recommendations

– State selects which recommendations to pursue– FMCSA technical assistance is available

State Data Quality Program: Components

Select a state Preanalysis On-site visit Post Visit

– Document data reporting processes, observations and recommendations

– State selects which recommendations to pursue– FMCSA technical assistance is available

How do we decide where to go?

State Selection Methodology

State Safety Data Quality (SSDQ) Results

Contribution to the Nation’s data quality performance

State’s willingness to participate

SSDQ Results: June 2006

5%

Crash AccuracyCrash Timeliness

13%18%

13%

5%

9%

State Data Quality Program: Components

Select state Preanalysis On-site visit Post Visit

– Document data reporting processes, observations and recommendations

– State selects which recommendations to pursue– FMCSA technical assistance is available

Preanalysis Completed Before the On-Site Visit

SSDQ results (A&I On-Line) Crash statistics (A&I On-Line) Ad hoc MCMIS reports Crash form analysis

State Data Quality Program: Components

Select state Preanalysis On-site visit Post Visit

– Document data reporting processes, observations and recommendations

– State selects which recommendations to pursue– FMCSA technical assistance is available

Who Participates in an On-Site Visit? Federal team

– HQ representation (when possible)– MCMIS crash and inspection data– Crash form– Collection & reporting process– SafetyNet use

State managers and support staff– Crash and inspection data– Collection, reporting, training

What HappensDuring an On-Site Visit?

Talk to managers Talk to support staff Use as series of questions to guide the group

through the discussions– Define collection and reporting process– Identify bottlenecks and practices resulting in

compromised data quality

Last day Federal team provides draft list of observations

On-Site Visit: Covers Entire Crash Process

CrashEvent

Input toState

System

Upload/Input

To SNET

Upload toMCMIS

DataCollected

Observation: Crash Collection Process Compromises Data Accuracy

CompleteCrashForm

Input toState

System

Upload/Input

To SNET

Upload toMCMIS

DataCollected

Observation: Crash Collection Process Compromises Data Accuracy

No distinction between interstate and intrastate trucks on crash form

Crash reports sent to the State with common errors (USDOT#, GVWR, cargo body types)

Prior versions of the crash form used

Observation: Crash Reporting Process Compromises Data Accuracy

CompleteCrashForm

Input toState

System

Upload/Input

To SNET

Upload toMCMIS

DataCollected

Observation: Crash Reporting Process Compromises Data Accuracy

Missing data filled-in with incorrect information

Conversion of State data elements to SafetyNet elements

Observation: Some Practices and Procedures at the MCSAP Office Compromise Data Quality

CompleteCrashForm

Input toState

System

Upload/Input

To SNET

Upload toMCMIS

DataCollected

Observation: Some Practices and Procedures at the MCSAP Office Compromise Data Quality

SafetyNet carrier match process not being used before sending data to MCMIS

Infrequent census updates to SafetyNet

MCSAP office uploads from SafetyNet to MCMIS occurring infrequently

Observation: Not all FMCSA Reportable Crashes are Uploaded to MCMIS

CompleteCrashForm

Input toState

System

Upload/Input

To SNET

Upload toMCMIS

DataCollected

Observation: Not all FMCSA Reportable Crashes are Uploaded to MCMIS

State and FMCSA CMV definitions do not match

Officers trained to identify FMCSA reportable crashes according to State’s definition.

State’s definition used to extract FMCSA reportable crashes from State’s database.

State Data Quality Program: Components

Select state Preanalysis On-site visit After the on-site visit

– Document data process flows, observations and recommendations

– State selects which recommendations to pursue

– FMCSA technical assistance is available

After the On-Site Visit: Document Crash Process Flow

Officer fills out DMV-349

6

Scan DMV-349s11

Is Crash a Fatal?

9

No

Import into SafetyNet

21

Review SafetyNet log files

22

Contains fatal errors?

23

Send log file to DMV

24

Send export file to MCMIS

27

Review MCMIS confirmation report

28

Contains fatal errors?

29

END

Photocopy Fatal DMV-349 and send to

FARS Team

10

Crash Event

No

DMV-349s sent to DMV

7

DMV-349s are sorted (CMV, CMV-Fatal,

Non-CMV)

8

Is CMV info complete?

12

Yes

Send DMV-349 to Officer

13

Input into CRS

15

No

Yes

Is Crash Record Saved

Successfully?

16

Officer fills out electronic NCCRF

1

Upload electronic NCCRF

2

Officer corrects information

14

Notification sent to Officer

4

Officer corrects errors

5

Extract CMV reportable records

17

Apply "Finalist" Changes

18

Generate CMV export file and email to MCSAP agency

19

Save and Convert export file

20

Correct on-site?

25

Make Corrections

26

No

North Carolina Law Enforcement Agencies

(SHP and LP)

Division of Motor Vehicles (DMV)

Motor Carrier Safety Assistance Program

(MCSAP) Office

Contains errors?

3Yes

Yes

Yes

Yes

No

No

Yes

No

After the On-Site Visit: The State is in Control

State decides which recommendations to pursue

FMCSA Technical Assistance is Available– Crash form design to capture FMCSA data elements– Officer training for crash form completion– SafetyNet training

FMCSA Learns Too

Opportunity for the state to give feedback to FMCSA

– Improve SafetyNet and MCMIS activity log formats

– SafetyNet training course needed for new user

For more information, please contact:

Betsy BenkowskiFMCSA, Analysis Division202-366-5387

Beth DeysherUSDOT, Volpe Center617-494-2570