Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012...
-
Upload
damien-robart -
Category
Documents
-
view
214 -
download
0
Transcript of Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012...
![Page 1: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/1.jpg)
Joint Meeting on Adolescent Treatment EffectivenessWashington, DC – April 10, 2012
Pay-for-performance as a method to improve delivery of high-quality care: Results from the Reinforcing Therapist Performance (RTP)
experiment
Bryan R. Garner, Susan H. Godley, Michael L. Dennis, Brooke D. Hunter, Christin M. L. Bair, & Mark D. Godley
Chestnut Health SystemsBloomington-Normal, IL
R01AA017625 (PI: Garner)
Contract #270-07-0191
Opinions are those of the authors and not official positions of the government
![Page 2: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/2.jpg)
Learning Objectives
To learn more about…
1. Methods to improve the implementation of evidence-based practices
2. The effectiveness of pay-for-performance methods
3. The relationship between performance measures and client outcomes
![Page 3: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/3.jpg)
Group of treatment organizations that are: Receiving the same level of funding. Implementing the same evidence-based
treatment. Receiving the same comprehensive training
model. Documenting treatment implementation using
the same web-based program. Documenting client outcomes using the same
evidence-based assessment instrument.
AAFT
AAFT as the foundation of the Reinforcing Therapist Performance (RTP) Experiment
![Page 4: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/4.jpg)
AAFT
A-CRA Clinician Training and Certification Process
Feedback Monitor Clinician Fidelity
RecordTherapySessions
Achieve Basic A-CRA Certification
In 9 procedures
Treatment Manual
and Training
Workshop
Participant in Coaching Calls with Supervisors/Coaches
Adapted from Godley, Garner, Smith, Meyers, & Godley (2011)
![Page 5: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/5.jpg)
AAFT
A-CRA Training Workshop
A-CRA Treatment Manual
+
A-CRA Clinician Training and Certification Process
![Page 6: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/6.jpg)
AAFT
Feedback Monitor Clinician Fidelity
RecordTherapySessions
Achieve Basic A-CRA Certification
In 9 procedures
Treatment Manual
and Training
Workshop
Participant in Coaching Calls with Supervisors/Coaches
A-CRA Clinician Training and Certification Process
![Page 7: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/7.jpg)
AAFT
Digital Session Recordings
(DSRs)
A-CRA Clinician Training and Certification Process
![Page 8: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/8.jpg)
Upload DSR to EBTx.org website
![Page 9: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/9.jpg)
AAFT
Feedback Monitor Clinician Fidelity
RecordTherapySessions
Achieve Basic A-CRA Certification
In 9 procedures
Treatment Manual
and Training
Workshop
Participant in Coaching Calls with Supervisors/Coaches
A-CRA Clinician Training and Certification Process
![Page 10: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/10.jpg)
AAT 1:First Year
AAFT 1:Second Year
AAFT 1:Third Year
15 sites funded in October
2006
AAFT 2:First Year
AAFT 2:Second Year
AAFT 2:Third Year
17 sites funded in October
2007
Improving the ImplementationProcess within AAFT
How might we improve the process?
![Page 11: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/11.jpg)
Recommended Pay-for-Performance (P4P) as a promising method to improve the delivery of high quality of care.
“The literature evaluating the effectiveness of pay for performance consists of fewer than 20 studies, yielding mixed conclusions on overall impact.”
Rewarding Provider Performance: Aligning Incentives in Medicare (IOM, 2007)
![Page 12: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/12.jpg)
AAFT 1:First Year
AAFT 1:Second Year
AAFT 1:Third Year
15 sites funded in October
2006
AAFT 2:First Year
AAFT 2:Second Year
AAFT 2:Third Year
17 sites funded in October
2007
RTP study funded in October 2008
How might we improve the process?
Improving the ImplementationProcess within AAFT
![Page 13: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/13.jpg)
AAFT
RTP
Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes.
Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)
$50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification.
$200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010).
Targets: Monthly A-CRA Competence Target A-CRA
Aims and Design of RTP Experiment
![Page 14: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/14.jpg)
TargetA-CRA
6-month Remission
Status
+
+
Tre
atm
ent I
mpl
emen
tatio
n P
erfo
rman
ce M
easu
res
TreatmentEffectiveness
A-CRACompetence
RTP hypotheses to be tested
+
Assignment to ExperimentalPay-For-Performance
Intervention
![Page 15: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/15.jpg)
Sample size & Data structure
Therapist 104
Site 29Level 3:
29 Sites
Level 2: 105
Therapists
Level 1: 986 Clients
…
Therapist 105
1 2 3 4 5 6 7 8 9
Therapist 1
Site 1
Therapist 3Therapist 2 …
… 986985983982 984
![Page 16: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/16.jpg)
Therapist Characteristics (N=105)
a p < .05; b See Donnellan, Oswald, Baird, & Lucas (2006) for more information about Mini-International Personality Item Pool (Mini-IPIP)
% or Mean (SD)
IAU(n = 52)
P4P(n = 53)
Therapists (N=105)
Female 67% 81%
White 58% 51%
Age 37.2 (11.7) 36.0 (10.7)
Master’s degree or higher 60% 49%
Months experience 60.3 (80.2) 35.9 (37.5)a
In Recovery 6% 6%
Mini-IPIP personality factorsb
Extraversion 3.6 (0.73) 3.5 (0.77)
Agreeableness 4.4 (0.47) 4.4 (0.50)
Conscientiousness 3.9 (0.73) 4.2 (0.59)a
Neuroticism 2.5 (0.63) 2.5 (0.69)
Intellect/Imagination 4.0 (0.67) 3.9 (0.72)
![Page 17: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/17.jpg)
Client Characteristics (N=986)
% or Mean (SD)
IAU(n = 534)
P4P(n = 452)
Clients (N=986)
Female 22% 27%a
White 36% 28%a
Age 15.8 (1.34) 15.9 (1.47)Criminal Justice Involved 64% 68%Prior substance use treatment 37% 28%a
Percent-of-days-abstinent .58 (.37) .66 (.34)a
a p < .05;
![Page 18: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/18.jpg)
AAFT
RTP
Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes.
Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)
$50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification.
$200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010).
Targets: Monthly A-CRA Competence Target A-CRA
Results
![Page 19: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/19.jpg)
Results: A-CRA Competence
0%
5%
10%
15%
20%
25%
IAU P4P
9.2%
23.7%
Odds Ratio = 2.2p = .02
![Page 20: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/20.jpg)
AAFT
RTP
Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes.
Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)
$50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification.
$200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010).
Targets: Monthly A-CRA Competence Target A-CRA
Results
![Page 21: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/21.jpg)
Results: Target A-CRA
0%
5%
10%
15%
20%
IAU P4P
3.3%
15.9%
Odds Ratio = 5.2p = .01
![Page 22: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/22.jpg)
AAFT
RTP
Aims: Examine the effectiveness and cost-effectiveness of providing monetary incentives to therapists as a method to improve treatment service delivery and subsequent treatment outcomes.
Design: Randomly assigned AAFT grantees and their therapists to either: 1) Implementation-As-Usual (IAU) or 2) Pay-For-Performance (P4P)
$50 for each month that a randomly selected digital session recording (DSR) has at least one A-CRA procedure rated at or above the minimum level of competence required for A-CRA certification.
$200 for each adolescent who they deliver an empirically derived level of A-CRA treatment that has been shown to be significantly related improved treatment outcomes (Garner et al., 2009, 2010).
Targets: Monthly A-CRA Competence Target A-CRA
Results
![Page 23: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/23.jpg)
Results: Remission Status
![Page 24: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/24.jpg)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
IAU P4P
50.8%
41.8%
No Significant Difference
Average Remission Status was 24% in
Cannabis Youth Treatment (CYT) study
(Dennis et al., 2004)
Results: Remission Status
?
![Page 25: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/25.jpg)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
IAU P4P
65%56%
Poor Follow-up Rates
Remission Status
Unknownfor 44%
Remission Status
Unknownfor 35%
Differential Client Attrition!
![Page 26: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/26.jpg)
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
IAU P4P
65%56%
Poor Follow-up Rates
Remission Status
Unknownfor 44%
Remission Status
Unknownfor 35%
No SignificantDifferences
Significant Differences
Follow-up group was
significantly more severe at baseline
![Page 27: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/27.jpg)
TargetA-CRA
6-month Remission
Status Tre
atm
ent I
mpl
emen
tatio
n P
erfo
rman
ce M
easu
res
TreatmentEffectiveness+
A-CRACompetence
+
Assignment to ExperimentalPay-For-Performance
Intervention
Not Supported
Supported
Post hoc analyses
![Page 28: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/28.jpg)
Take home messages
1. Modest-sized incentives can have large impacts on improving treatment implementation
2. Despite the large impact of P4P on the two treatment implementation measures, there remained considerable room for improvement even in the P4P condition.
![Page 29: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/29.jpg)
Next Step
Examine the extent to which the improvements in A-CRA Competence and Target A-CRA were Cost-Effective.
![Page 30: Joint Meeting on Adolescent Treatment Effectiveness Washington, DC – April 10, 2012 Pay-for-performance as a method to improve delivery of high-quality.](https://reader037.fdocuments.in/reader037/viewer/2022103015/551636d5550346b2068b4efe/html5/thumbnails/30.jpg)
Thank You.