Evaluating Driver Education Programs - ADTSEA ... 2 Evaluating Driver Education Programs:...

Click here to load reader

  • date post

    04-Oct-2020
  • Category

    Documents

  • view

    1
  • download

    0

Embed Size (px)

Transcript of Evaluating Driver Education Programs - ADTSEA ... 2 Evaluating Driver Education Programs:...

  • 607 14th Street, NW Suite 201

    Washington, DC 20005 800-993-7222

    www.aaafoundation.org

    October 2006

    Prepared for

    E v a l u a t i n g D r i v e r E d u c a t i o n P r o g r a m s

    �������� ����

    Prepared by

    Kathryn M. Clinton Larry Lonero

    Northport Associates 182 Bagot Street Cobourg, Ontario Canada K9A 3G2 905 - 377- 8883 www.northportassociates.com

    C o m p r e h e n s i v e G u i d e l i n e s

  • 2 E v a l u a t i n g D r i v e r E d u c a t i o n P r o g r a m s : C o m p r e h e n s i v e G u i d e l i n e s

    This report was funded by the AAA Foundation for Traffic Safety in Washington, D.C. Founded in 1947, the AAA Foundation is a not-for-profit, publicly supported charitable research and education organization dedicated to saving lives by preventing traffic crashes and reducing injuries when crashes occur. Funding for this report was provided by voluntary contributions from AAA/CAA and their affiliated motor clubs, from individual members, from AAA-affiliated insurance companies, as well as from other organizations or sources.

    This publication is distributed by the AAA Foundation for Traffic Safety at no charge, as a public service. It may not be resold or used for commercial purposes without the explicit permission of the Foundation. It may, however, be copied in whole or in part and distributed for free via any medium, provided the AAA Foundation is given appropriate credit as the source of the material.

    The opinions, findings, conclusions, and recommendations expressed in this publication are those of the authors and are not necessarily those of the AAA Foundation for Traffic Safety, any individuals who peer-reviewed this report, or advisory group members. The AAA Foundation for Traffic Safety assumes no liability for the use or misuse of any information, opinions, findings, conclusions, or recommendations contained in this report.

    If trade or manufacturer’s names are mentioned, it is only because they are considered essential to the object of this report and their mention should not be construed as an endorsement. The AAA Foundation for Traffic Safety does not endorse products or manufacturers.

    ©2006, AAA Foundation for Traffic Safety

  • (continued)

    3E v a l u a t i n g D r i v e r E d u c a t i o n P r o g r a m s : C o m p r e h e n s i v e G u i d e l i n e s

    Contents

    Glossary of Acronyms ➣ 5

    List of Tables and Figures ➣ 7

    Acknowledgements ➣ 9

    Preface ➣ 11

    Executive Summary ➣ 13

    1. Introduction ➣ 23 Overview of the Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 How to Read the Guidelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Target Audiences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Driver Education Programs in the U.S. and Canada . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Approaches to Driver Education Program Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

    2. Evaluation Model and Framework ➣ 31 What is Program Evaluation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Why is Evaluation Important? Lessons from Related Fields . . . . . . . . . . . . . . . . . . . . . . . 33 What is Good Evaluation? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Depicting Programs through Logic Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Developing an Evaluation Model and Framework for Driver Education . . . . . . . . . . . . . . . . . 43 Driver Education Evaluation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Integrated Driver Education Evaluation Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

    3. Driver Education Evaluation Guidelines ➣ 55 Applying the Framework—How to Use Evaluation Effectively . . . . . . . . . . . . . . . . . . . . . . 55 The Evaluation Team . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 Guiding Questions for Driver Education Program Evaluation . . . . . . . . . . . . . . . . . . . . . . . 56 Steps for Evaluating Driver Education Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 STEP 1: Focus the Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 STEP 2: Select Evaluation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 STEP 3: Develop Data Collection Plan and Select Data Collection Tools . . . . . . . . . . . . . . . .101 STEP 4: Gather, Analyze, and Summarize Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 STEP 5: Interpret and Act Upon the Evaluation Findings . . . . . . . . . . . . . . . . . . . . . . . . .129

    4. Basic Driver Education Program Evaluation Scenario ➣ 137 Program Logic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .138 Evaluation Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .140 Evaluation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .145 Evaluation Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149

  • (Table of Contents—continued)

    4 E v a l u a t i n g D r i v e r E d u c a t i o n P r o g r a m s : C o m p r e h e n s i v e G u i d e l i n e s

    5. Advanced Driver Education Program Evaluation Scenario ➣ 151 Program Logic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Evaluation Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Evaluation Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .157 Evaluation Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .164

    6. Conclusion ➣ 165

    References and Evaluation Resources ➣ 167 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167 Evaluation Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

    APPENDIX A: Review of the Literature: Driver Education Evaluation—Past and Present . . . . . 179

    APPENDIX B: Glossary of Terms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .213

    APPENDIX C: Program Evaluation Standards Definitions . . . . . . . . . . . . . . . . . . . . . . . . . .231

    APPENDIX D: Overview of Evaluation Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .235

    APPENDIX E: Worksheets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .239

    APPENDIX F: Data Collection Tools: Basic Evaluation Scenario . . . . . . . . . . . . . . . . . . .251

    APPENDIX G: Data Collection Tools: Advanced Evaluation Scenario . . . . . . . . . . . . . . . . 281

    APPENDIX H: Attitude and Risk-Taking Questionnaires . . . . . . . . . . . . . . . . . . . . . . . . 313

    APPENDIX I: Benchmark Program Standards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317

    APPENDIX J: Hiring an Evaluator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .329

  • 5E v a l u a t i n g D r i v e r E d u c a t i o n P r o g r a m s : C o m p r e h e n s i v e G u i d e l i n e s

    GLOSSARY OF ACRONYMS

    AAA Formerly American Automobile Association

    AAAFTS AAA Foundation for Traffic Safety

    ADTSEA American Driver and Traffic Safety Education Association

    CAA Canadian Automobile Association

    CIPP Context, Input, Process, and Product Evaluation

    CPM Critical Path Method

    DSAA Driving School Association of the Americas

    GDL Graduated Driver Licensing Progr