Continuous Quality Improvement: Ideas from The Field

Becca Sanders, Carolyn SullinsEvaluators, Kalamazoo Wraps System of Care beccaanddan@gorge.net; csullins@wmich.edu

Objectives

From Theory to Practice in the Field• Challenges to implementing the model• Reorienting Evaluation

– Onlooker vs. In the Mix– CQI: “The Big 3” to be Tackled– Technical and Adaptive Components

How CQI? • Our “Non-model Model”

– New roles for evaluators– The Foundation: Key Components– What Matters– What Does it Look Like?

Model Challenges: CQI in an SOC Context

The Scope of SOC • Not a program The Scope of the SOC Audience• Relevance, right people at the right table- not so easyThe SOC Aggregate Data Only Problem• Moving target, changing system The Nature of the SOC Problem • Technical and adaptive problems require technical and

adaptive solutions

CQI looks good on paper:Moving from paper to action…It’s another story…What are the barriers?

“Check out our tight indicators and efficient quarterly review process… yeah… we’ll work on that…”

More Model Challenges:CQI in an SOC Context

Most SOCs Already Have:• Data demands from other

agencies• Plenty of (inconsistently

collected, underutilized) data• Concerns about how data

may be misinterpreted• Skepticism about the

usefulness of more data

Reorienting Evaluation: A Key Shift for Doing CQI Well

Data as “beside” the (systems) problem; a contributor to the solution.

• Help figure out if good “stuff” is happening in an initiative.

• If not, why not?• If not, what should we

do about it?• Clean, linear, solid

aggregate indicators

Data as “within” the (systems) problem; a contributor to the problem (and solution).

• Data culture chronically nonfunctioning or nonexistent.

• Data culture in need of CQI in and of itself.

• “Readiness for CQI:” Precedes all else

Reorienting Evaluation Onlooker vs. In the Mix for a Fix

Evaluation of Change EffortsMultiple Change Efforts

Practices/ Services

Procedures

Policies

Etc…

Multiple Change Efforts

Practices/ Services

Procedures

Policies

Etc…

Data Culture

Onlooker

In the Mix for a Fix

Does the data stuff…vs.

Helps us do the data stuff

What does Reorientation Look Like?CQI: “The Big 3” To Be Tackled

1) Data Existence: Collect relevant, useful data, consistently and accurately. Simplify this process!

2) Data Access: MIS system extraction, MIS data input, resources, expertise, power structures, trust, bureaucratic structures, agency culture, regulations, interest/ competence levels, data silos, data sharing agreements, understanding.

3) Data Dissemination: report on time, clearly and simply, link to individuals when appropriate, level “local” enough to be helpful.

When ducks in a row, data use follows

CQI: The Big Picture OutcomeBig Picture Outcome: Build a data-guided culture

across organizations/ stakeholders by helping communities improve “The Big 3:”

1.Data Existence

2.Data Access

3.Data Dissemination

A Mix of Technical and Adaptive Issues

See ya’ll in 10 years… this is going to take me a while.

Technical versus Adaptive

What do we mean?Technical: “WHAT there is to know.”

• Requires a knowledge transfer• The “teachables”

Adaptive: “WHY this is important.” “HOW you can use it.”

• Requires a value shift• The “less-teachables”

“The Big 3:” Require a Mix of Technical and Adaptive Solutions

Welcome to the Kalamazoo Wraps luxury!Hi! I’m an evaluator… I focus mostly on the adaptive stuff related to building a data guided culture: coaching, working one on one with stakeholders, helping people understand the value of data, upping interest and engagement in data related activities, making sure people understand reports and that they’re pretty. My budget is small.

Hi! I’m an evaluator… I focus mostly on the technical stuff related to building a data guided culture- collecting data, creating and managing databases, analyzing data, helping create and review instrumentation. I also run the national evaluation. My budget is large.

Hey… lets support one another’s roles!

Strategies for CQI Capacity BuildingTechnical and Adaptive Solutions

The Big 3 Highly Technical Highly Adaptive

Data

Existence

Data

Access

Data Dissemination

CoachingInstrument Review and Recommendations

Help with MIS systems set up

Data Collection

Technical Assistance

MIS system access

Analysis

Data management

MOU establishment with partnering organizations

Catered Reporting•Content•Timing•Stakeholder driven•Data splits

Product Review

CQI: What Matters?The Foundation: Key Components

• Initiative/ Leadership: Who Cares?

• Relationship building/ personality: What’s the engagement / hook

• Data splits for targeted responsesBy clinician, by individual…

• Fluidity: Product, Timing, Whose at the TableTweak, Tweak and Tweak some more

Beware of polite toleration of CQI efforts! (Adaptive)

Offer a service!(Adaptive)

Aggregate lacks utility! (Technical)

pi= 3.141592653…and on it goes (Adaptive and Technical)

CQI: What Can it Look LikeA “Close Proximity” Family Vacation:

A few key people have a vague notion of the kind of place they want to go…

So they hop in the car…

But there’s no map…

So you get lost…

But you know enough to get found…

Then you arrive…

At something close to what you were going for…

But you swear there are measures you can take so next year will be better…

So you start planning…

Then you hop back in the car…

If You Give a Mouse a Cookie…

In Conclusion

• Foundation/ key components to quality CQI- clear• Various facets of the CQI problem- clear• Various facets of the CQI solution- clear• Roles/ who does what regarding CQI- clear

• How to apply CQI: (intentionally) not so clear. Why?– fluid– catered– responsive– highly localized

x