Challenges to Measuring Provider Fidelity in a Statewide Dissemination/Implementation Project
INTEGRATED RECOVERY The Evidence Base & Measuring Fidelity Dan Chandler, Ph.D. January 19, 2007.
-
Upload
ginger-rich -
Category
Documents
-
view
218 -
download
3
Transcript of INTEGRATED RECOVERY The Evidence Base & Measuring Fidelity Dan Chandler, Ph.D. January 19, 2007.
INTEGRATED RECOVERY
The Evidence Base&
Measuring Fidelity
Dan Chandler, Ph.D.
January 19, 2007
Integrated Recovery and the SAMHSA Toolkits
Summary of Evidence
• Standards used by CIMH• Effective
• Efficacious
• Promising
• Emerging
• Not effective or harmful
Other Toolkits
• ACT: Effective
• Supported Employment: Effective
• Family Psychoeducation: Effective
• Illness Management and Recovery:
Promising (but multiple components have
different levels of support)
The IDDT Toolkit
• Rating for Integrated Approaches to Co-occurring Disorders—Promising
• Rating for the IDDT Model Embodied in the Fidelity Scale—Emerging
2006 IDDT Summary by Developers
• “… after 20 years of research, there remains a lack of strong and clear evidence regarding effective engagement, treatment, and rehabilitation interventions for people with co-occurring disorders.”*
*McHugo, G. J., Drake, R. E., Brunette, M. F., Xie, H., Essock, S. M., & Green, A. I. (2006). Enhancing Validity in Co-occurring Disorders Treatment Research. Schizophrenia Bulletin, 32(4), 655-665.
Context
• Though evidence is limited, the IDDT Toolkit model is basically the best that exists
• Each component is a “black box” that can be filled in different ways based on developing research
• Recent research also supports:– Integrated residential treatment– Contingency (reward) based treatment
A consistent finding…
• If a program can engage and retain clients they will show positive change
• Clients in Engagement and Persuasion stages need intensive services, particularly outreach
Two Approaches to “fidelity”
The traditional approach
“Fidelity is adherence to the key
elements of an evidence-based practice,
as described in the controlled
experimental design, and that are shown
to be critical to achieving the positive
results found in a controlled trial.”American College of Mental Health Administration
The IDDT Approach
• How the fidelity scale was developed– Established “principles” of treatment
– Developed “anchors” for the principles
• It is not based on any particular effective programs as the ACT scale is
• High fidelity means the program is implemented as designed but may not correlate with good outcomes
How fidelity is measured
Each scale consists of 12-28 itemsItems rated on 5-point continuum • 1 = Not Implemented• 5 = Fully Implemented• ≥ 4.0 considered good
implementation
Achieving fidelity
• Some of the SAMHSA EBPs are easier to implement than others
• The time required to achieve full implementation varies greatly
National EBP Study
Fidelity by EBP 2 Years After Startup
4.444.17 4.00
3.58 3.43
1
2
3
4
5
SE ACT FPE IMR IDDT
(n = 9) (n = 13) (n = 4) (n = 12) (n = 11)
Fid
elit
y Sc
ale
Mea
n
National EBP Study (2)
Fidelity of ACT
2.8
4.1 4.2 4.2 4.2
1
2
3
4
5
Baseline 6 mo. 12 mo. 18 mo. 24 mo.
(n = 13) (n = 13) (n = 13) (n = 13) (n = 13)
DACT
S M
ean
Fidelity of IDDT Programs
2.42.9 3.2 3.3 3.4
1
2
3
4
5
Baseline 6 mo. 12 mo. 18 mo. 24 mo.
(n = 11) (n = 11) (n = 11) (n = 11) (n = 11)
IDDT
Fid
elity
Sca
le M
ean
Baseline 6 Months 12 Months0.0
1.0
2.0
3.0
4.0
5.0
3.2
1.9
3.4
2.8
3.73.3
California (N=8 in Rnd1-2, 6 in Rnd3)
National EBP Project (N=10)
National EBP Project Early Results vs. CaliforniaResults on Comparable Fidelity Items
Implementation as a start-up
Same fidelity items as prior graphFidelity Scores
4.3
4.9
1
2
3
4
5
Baseline Six Months