A Comprehensive Review of Dissemination and Implementation Science Instruments

Cara C. Lewis, Cameo Borntrager, Ruben Martinez, Phil Fizur, & Kate Comtois

Agenda

1. Background

2. Goals

3. Framework & Scope of Review

4. Overview of Review Process &

Findings

5. Evidence-Based Assessment Criteria

6. SIRC Instrument Review Task Force

7. Shared Product

Background

Impetus for the project:

✦ Measurement redundancy

✦ Constructs with no measures

✦ Prioritize both measure quality and usability

Goals

Step 1

Instrument Repository

Step 3

Consensus Battery

Step 2

Comprehensive Review

Evidence Based Assessment

Framework & Scope

Consolidated Framework for Implementation Research

✦ 39 Constructs

Damschroder, Aron, Keith, Kirsh, Alexander, & Lowery, 2009

Framework & Scope

Outcomes for Implementation Research

✦ 17 Constructs

Proctor, Silmere, Raghavan, Hobmand, Aarons, Bunger, et al. 2011

Comprehensive Review

1 Literature Review

3 Measure Specific

4Individual and Listserv Requests

5 Comprehensive Measure List

2Construct Specific

215 measures

Comprehensive Measures List

SIRC Website: Main Page

www.seattleimplementation.org

SIRC Website: Main Page

www.seattleimplementation.org

SIRC Website: Construct Page

www.seattleimplementation.org

SIRC Website: Measure Page

www.seattleimplementation.org

Goals

Step 1

Instrument Repository

Step 3

Consensus Battery

Step 2

Comprehensive Review

Evidence Based Assessment

Criteria

Evidence Based Assessment Criteria

First Draft

SPG Feedback (N = 58)

Test Developer Input

IRT Pilot

Hunsley & Mash, 2008

Evidence-Based Assessment Criteria

EBA2 =

Adequate 3 = Good 4 = Excellent1 = Minimal0 = None

Psychometric Properties

✦ Norms

✦ Reliability Information

✦ Validity Information: Criterion, Content, Construct

✦ Validity Generalization versus Test Bias

✦ Validity Information: Predictive, Discriminant, Concurrent

Seattle Implementation Research Conference: Instrument Review Task Force

VA Center for Implementation Practice and Research Support

International Evidence Informed Practice Consortium

40 Implementation Science Experts from SIRC SPG

Universities of Washington, Montana, & Indiana U

IRT

SIRC Instrument Review Task Force

Gregory Aarons

Cassidy Arnold

Melanie Barwick

Rinad Beidas

Helen Best

Elisa Borah

Cameo

Borntrager

Craig Bryan

Adam Carmel

Mark Chaffin

Kate Comtois

Laura Damschroder

Dennis Donovan

Michelle Duda

Dean Fixsen

Howard Goldman

Carmen Hall

Amanda Jensen-

Doss

Sarah Kaye

Sara Landes

Cara Lewis

Marsha Linehan

Aaron Lyon

Michael

McDonell

Kate McHugh

Maria Mancebo

Shari Manning

Christopher

Martell

Brian Mittman

Sandra Naoom

Raphael Rose

Lisa Ruble

Joe Ruzek

Anju Sahay

Sonja Schoenwald

Jeremy Smith

Bradley Steinfeld

Phil Ullrich

Shannon Wiltsey

Stirman

Jennifer Villatte

Comprehensive Review

1 Literature Review

3 Measure Specific

4Individual and Listserv Requests

5 Comprehensive Measure List

2Construct Specific

215 measures

Specific Measure Review: Product for IRT

Specific Measure Review: Product for IRT

Specific Measure Review: Product for IRT

Summary of Review to Date

Constructs with several measures (i.e., more than 5):✦ Acceptability✦ Adoption✦ Relative Advantage✦ Organizational Culture and Climate✦ Readiness for Implementation✦ Knowledge & Beliefs about Intervention✦ Planning✦ Reflecting & Evaluating

Constructs with zero measures:✦ Institute of Medicine’s Service Outcomes: Efficiency, Equity,

Timeliness✦ Intervention Adaptability, Complexity, Design Quality and

Packaging, Source✦ Outer Setting: Cosmopolitanism, Policy & Incentives, Peer

Pressure✦ Implementation Climate: Relative Priority, Goals & Feedback✦ Engaging & Executing

Q&A