Achieving Our MissionAchieving Our Mission

The Role of CQI in Public Safety

Kimberly Gentry Sperber, Ph.D.

Efforts To DateEfforts To Date

• “What Works” Literature– Principles of Effective Interventions– Growing evidence based on individual

program evaluations and meta-analyses

• Continuing Gap Between Science and Practice– Few programs score as satisfactory on

CPAI/CPC

Importance of CPAI/CPC CriteriaImportance of CPAI/CPC Criteria

• Differences in recidivism rates based on CPAI scores:– Scores of 0-49% demonstrated 1.7% reduction

compared to comparison group.– Scores of 50-59% demonstrated 8.1% reduction.– Scores of 60-69% demonstrated 22% reduction.

What are implications for ODRC standards for correctional programs?

Why IsnWhy Isn’’t t ““ItIt”” Working? Working?Latessa, Cullen, and Gendreau (2002)Latessa, Cullen, and Gendreau (2002)

• 4 common failures of correctional programs:– Failure to use research in designing programs– Failure to follow appropriate assessment and

classification practices– Failure to use effective treatment models– Failure to evaluate what we do

What Happens When We DonWhat Happens When We Don’’t Measure Ourselves?t Measure Ourselves?

The Role of Fidelity

Washington State ExampleWashington State Example(Barnoski, 2004)(Barnoski, 2004)

• For each program (FFT and ART), an equivalent comparison/control group was created

• Felony recidivism rates were calculated for each of three groups, for each of the programs• Youth who received services from therapists

deemed ‘competent’• Youth who received services from therapists

deemed ‘not competent’• Youth who did not receive any services (control

group)

Functional Family Therapy Results: Functional Family Therapy Results: % New Felony% New Felony

139 6

2519

11

3227

17

0

10

20

30

40

50

60

70

6 Months 12 Months 18 Months

FFT Not Competent

Control group

FFT Competent

Results calculated using multivariate models in order to control for potential differences between groups

Project GreenlightProject Greenlight

• Short-term prison-based reentry program in New York– CBT Skills Training– Employment Services– Housing Services– Drug Education and Awareness– Family Counseling– Practical Skills Training– Community-Based Networks– Familiarity With Parole– Individualized Release Plans

Project Greenlight BenefitsProject Greenlight Benefits

• Participants received more service referrals

• Participants reported more contacts with community services after release

• Participants demonstrated significantly more familiarity with parole conditions

• Participants were more positive about parole

But Did It Work?But Did It Work?

17.214.4 13

34.1

26.824.2

0

5

10

15

20

25

30

35

Arrests at 6 Months Arrests at 12Months

Greenlight

No Services

Existing TransServices

What Went Wrong?What Went Wrong?

• Violation of the risk principle– Ceased use of risk assessment instrument when staff

deemed process too cumbersome

• Violation of the need principle– All offenders received same services whether

needed or not

• Violation of the fidelity principle– Staff modified delivery of the CBT curriculum

(shortened the duration, increased frequency, increased class size)

• Differential staff competence– Certain case managers produced worse outcomes

UC Halfway House/CBCF Study in Ohio: A UC Halfway House/CBCF Study in Ohio: A Look at Fidelity Statewide (2002)Look at Fidelity Statewide (2002)

• Average Treatment Effect was 4% reduction in recidivism

• Lowest was a 41% Increase in recidivism• Highest was a 43% reduction in recidivism

• Programs that had acceptable termination rates, had been in operation for 3 years or more, had a cognitive behavioral program, targeted criminogenic needs, used role playing in almost every session, and varied treatment and length of supervision by risk had a 39% reduction in recidivism

2010 UC Halfway House/CBCF Study in Ohio:2010 UC Halfway House/CBCF Study in Ohio:Adherence to CBT in Groups and Changes in RecidivismAdherence to CBT in Groups and Changes in Recidivism

-2

-1

0

1

2

3

4

5

6

7

-1 0 1

What Do We Know About What Do We Know About Fidelity?Fidelity?

• Fidelity is related to successful outcomes (i.e., reductions in recidivism, relapse, and MH instability).

• Poor fidelity can lead to null effects or even iatrogenic effects.

• Fidelity cannot be assumed• Fidelity can be measured and

monitored.

Bringing Mission to the Bringing Mission to the ForefrontForefront

Becoming a Mission-Driven Becoming a Mission-Driven OrganizationOrganization

• Knowing the mission statement• What does it mean to you?• What does it mean for your role?• Who is responsible for fulfilling the

agency’s mission statement?• How can QA/QI help us accomplish

our mission statement?

Providing Managed ChangeProviding Managed Change

• Achieving and maintaining quality is not accidental.

• It requires a deliberate effort on the part of everyone.

Achieving QualityAchieving Quality

• Responsibility for quality falls on both the organization and the individual.

• The individual and the organization should be linked in a formal framework designed to continually improve quality.

Quality Assurance (QA)Quality Assurance (QA)

• Retrospective review process• Emphasis on regulatory and

contract compliance• Catching people being bad leads to

hide and seek behavior

Continuous Quality Continuous Quality Improvement (CQI)Improvement (CQI)

• CQI is a prospective process• Holds quality as a central priority within the

organization• Focus on customer needs; relies on feedback

from internal and external customers• Emphasizes systematic use of data• Not blame-seeking• Trust, respect, and communication• Move toward staff responsibility for quality,

problem solving and ownership of services

Focus on Causes Focus on Causes notnot SymptomsSymptoms

• Focus on processes/systems rather than individuals or specific errors

• Identification of risk points and their contribution to the problem

• Identify changes in these processes that reduce risk of re-occurrence

Objectives of CQIObjectives of CQI

• To facilitate the Agency’s mission• To ensure appropriateness of services• To improve efficiency of services/processes• To improve effectiveness of directing

services to client needs• To foster a culture of learning• To ensure compliance with funding and

regulatory standards

Common ElementsCommon Elements

• Documentation Review• Performance Indicators

– Process vs. Outcomes

• Satisfaction – Client, Employee, Referral Source/Stakeholders

• Protocols for Data Use• Infrastructure

Focused Discussion:Focused Discussion:Why Examine Documentation?Why Examine Documentation?

• Clinical/Service Implications– Documentation is not separate from service delivery.– Did the client receive the services he/she needed?

• Operational Implications– Good documentation should drive decision-making.– Means of communication

• Risk Management Implications– If it isn’t documented, it didn’t happen.– Permanent record of what occurred in the program/unit.

• Source of Staff Training • Reflection of the provider and organization’s

competency:– EBP– Outcome of care

Action PlansAction Plans

• Plan of correction• Proactive approach to problem-

solving• Empowers staff• Using objective data to inform

decision making

Who Creates Action Plans?Who Creates Action Plans?

• Anyone and everyone can create action plans

• Focus should be on who has knowledge or expertise to contribute

• Focus should not be on the person’s title

Minimum RequirementsMinimum Requirements

• Buy-in from staff at all levels of the organization

• Sufficient resources allocated for staff training

• Sufficient resources allocated for staff to participate in the process– Meetings– Data collection, reporting, use

• Sufficient information systems

Why Invest the Resources?Why Invest the Resources?

What makes this effort too compelling to ignore?

A CEOA CEO’’s Perspectives Perspective

• Why invest in CQI?– Because it’s the right thing to do!– Better for clients (i.e., better outcomes)

• Mission-driven– Increased staff satisfaction– Increased staff retention– Improved referral source satisfaction– More business for related projects– Outcomes to sell to business community

and other payers– Demonstrates fiscal responsibility (i.e.,

effective use of dollars)

Strategic Use of CQI DataStrategic Use of CQI Data

• CQI data used to provide testimony before legislature

• CQI data and infrastructure used to secure new contracts and grants

• CQI data used in newsletters, media relations, levy campaigns, etc.

• CQI data used to negotiate programmatic changes with stakeholders

Performance ExamplesPerformance Examples• Men’s halfway houses employment with

benefits– 33% to 67% over 5 years

• Changes in HIT over time per strategic plan– 57.4% to 72% over 5 years

• CCC’s employment indicator performance– <10% to 75% over 5 years

How do these changes in performance link to mission?

Top 10 Practices for Reducing Top 10 Practices for Reducing Recidivism in Drug CourtsRecidivism in Drug Courts

(Carey & Finigan, 2013)(Carey & Finigan, 2013)

1. Program caseload less than 252. Participants have greater than 90 days

clean before graduation3. Judge spends average of 3 minutes or

more per person during status review hearings

4. Treatment communicates with court via email

5. Representative from treatment attends drug court team meetings

Top 10 Practices for Reducing Top 10 Practices for Reducing Recidivism in Drug CourtsRecidivism in Drug Courts

(Carey & Finigan, 2013)(Carey & Finigan, 2013)

6. Review of the data/program stats has led to modification in operations

7. Representative from treatment attends court sessions

8. Drug court allows non-drug charges9. Law enforcement is member of drug court

team10.Results of program evaluations have led

to modification in operations

Top 10 Practices for Increasing Cost Top 10 Practices for Increasing Cost Savings in Drug CourtsSavings in Drug Courts

(Carey & Finigan, 2013)(Carey & Finigan, 2013)

1. Review of the data/stats has led to modifications in operations

2. Results of program evaluations have led to modification in operations

3. Sanctions are immediate4. Defense attorney attends drug court

team meetings5. Must have a job or be in school to

graduate

Top 10 Practices for Increasing Cost Top 10 Practices for Increasing Cost Savings in Drug CourtsSavings in Drug Courts

(Carey & Finigan, 2013)(Carey & Finigan, 2013)

6. Representative from treatment attends court sessions

7. Team members are given copy of sanctioning guidelines

8. Drug test results back in 48 hours or less9. Drug tests collected at least 2x/week

during first phase10.Law enforcement attends court sessions

NPC Research on Drug CourtsNPC Research on Drug Courts

37%

16%

0%

10%

20%

30%

40%

YesN=20

NoN=15

Perc

ent I

mpr

ovem

ent i

n O

utco

me

Cost

s*

* "Percent improvement in outcome costs" refers to the percent savings for drug court compared to business-as-usual

Drug Court Uses Evaluation Feedback to Make Modifications

The Role of QA/QI in Community CorrectionsThe Role of QA/QI in Community Corrections(based on UC Halfway House and CBCF study)(based on UC Halfway House and CBCF study)

6

1

0

2

4

6

8

Internal QA No Internal QA

% C

hang

e in

Rec

idiv

ism

Bottom LineBottom Line

• Many programs are implementing evidence-based practices with little evidence of strong fidelity.

• Result is an ongoing gap between science and practice.

• This gap often results in null or even iatrogenic effects.

• Correctional organizations have a responsibility to ensure effective services.

• Responsibility for EBP needs to be aligned at all levels – administration, management, line staff.

• Need to focus on creating formal infrastructure to support EBP and mission achievement

Continuous Quality Continuous Quality ImprovementImprovement

Questions & Answers