An Illustration of the PracticeWise Model and Tools

Bruce F. Chorpita, Ph.D.University of California, Los AngelesPracticeWise, LLC

Eric L. DaleidenMichael A. Southam-Gerow John R. WeiszChild STEPs Research NetworkDavid H. BarlowRon DrabmanAnne Marie AlbanoRick Barth

Hawaii Departments of Health and EducationMinnesota Department of Human Services John D. & Catherine T. MacArthur

FoundationAnnie E. Casey FoundationCalifornia Alliance of Child and Family

ServicesUS Army CAF Behavioral Health ProponencyLA County Department of Mental HealthLos Angeles Unified School DistrictPasadena Unified School DistrictCalifornia Institute of Mental Health

How can we get more from the evidence we already have?

Isn’t it all really about knowledge management?

Evidence Consolidation Coordination Implementation Practice

Children and Families

Raw RCTs…

EBT “Lists”

Coaching, Feedback, Org/admin supports

Quality services, supervision

1st generation

2nd

generation

3rd

generation

Knowledge Inquiry

“the unmanageable multitude of primary studies or information of variable quality that is out there and that may or may not be easily accessed” (Graham et al., 2006, p. 18)

Knowledge Synthesis

“the application of explicit and reproducible methods to the identification, appraisal, and synthesis of studies or information relevant to specific questions. It is done to make sense of all the relevant knowledge. This knowledge often takes the form of systematic reviews, including meta-analysis and meta-synthesis” (p. 19)

Knowledge Tools

“…provide explicit recommendations with the intent of influencing what stakeholders do.” (p. 19)

• e.g., a Care Pathway is a knowledge tool• What PracticeWise calls “knowledge

appliances”

ACMEEvidence-Based Treatment Manual

ACME

Evidence-

Based

Treatment

Manual

ACME

Evidence-

Based

Treatm

ent

Manual

“How can I learn enough EBTs? “How do they fit with what I was doing

before?” “How will what I learn stay current?” “Are there EBTs for all the different kinds

of kids I see?” “What do I do if there are not?” “What do I do if a child does not respond

to an EBT?” “Aren’t there other forms of evidence?”

In addition to installing new programs in a system…

Develop a knowledge delivery architecture to improve the practices that are already there

See the evidence base as knowledge and not simply products…

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

School

Cognitive, Problem Solving…

Competency Checklist for Therapists

RADS, CDI

CBT (adapted CWD)

PhD, MSW

PhD Supervisor

From Kahn et al., 1990

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

GeneralServicesResearch

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Evidence-Based Services Model

Case-SpecificHistorical

Information

CausalMechanismResearch

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Individualized Case Conceptualization Model

LocalAggregateEvidence

Case-SpecificHistorical

Information

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Practice-Based Evidence Model

GeneralServicesResearch

LocalAggregateEvidence

Case-SpecificHistorical

Information

CausalMechanismResearch

Tx ProgramSelection

TherapeuticPractices

ServiceSetting

ClientProgress

TreatmentIntegrity

Ongoing Review of Children’s Treatment Research 464 research papers 44 years 1,205 study groups 864 treatment protocols 46,171 youth participants

GeneralServicesResearch

LocalAggregateEvidence

Case-SpecificHistorical

Information

CausalMechanismResearch

“Good to see you, Sally. As soon as I finish reading these research studies, we can start our session today.”

ProblemAgeGenderEthnicitySettingStrength of Evidence

67 protocols,

19 treatment families

DEPRESSION

Results: Depression

Cognitive Behavior Therapy (19)

Cognitive Behavior Therapy with Parents (4)

Cognitive Behavior Therapy and Medication (3)

Family Therapy (2)

Interpersonal Therapy (3) Expressive

Writing/Journaling (2) Client Centered Therapy

(2) Relaxation (2)

Best Support Good Support

CWD-A PASCETBeck’s

CBT

CBT

Cognitive CognitiveCognitive

Relaxation

ProblemSolving

Relaxation

These are the practice elements.

Relaxation

Among all the evidence-based treatments for a given problem, which elements are the most common?

0% 25% 50% 75% 100%

Cognitive

Psychoeducational‐Child

Activity Scheduling

Maintenance/Relapse Prevention

Problem Solving

Self‐Monitoring

Communication Skills

Social Skills Training

Goal Setting

Psychoeducational‐Parent

Self‐Reward/Self‐Praise

Behavioral Contracting

Relaxation

Guided Imagery

Talent or Skill Building

Modeling

Stimulus Control or Antecedent …

Therapist Praise/Rewards

Relationship/Rapport Building

Parent Coping

Assertiveness Training

Crisis Management

Insight Building

Tangible Rewards

Family Therapy

Motivational  Interviewing

Peer Pairing

Supportive Listening

Frequency of Practice Element

Not just ingredients…

Focus

Connect ConsolidateCultivate

Interference

Focus

Connect

EngagementPsychoeducation

Consolidate

MaintenanceBooster

Cultivate

Pleasant ActivitiesCognitiveProblem Solving…

Interference

Low Motivation: RewardsComplaining and Irritability: Active Ignoring

Tantrums: Time Out…

What if we could enroll every registered youth in a given service system into every randomized clinical trial that was ever published, and see who qualifies for which effective treatments?

Well, we can…

Can map entire service population• How many youth are “coverable” by anything

in the evidence base?• What is the minimum number of practices or

programs my staff need to learn to serve all “coverable” youth?

Example from Hawaii CAMHD Service System 2008

We still have to know the basic steps…right?

Objectives: to increase the amount of positive attention provided to the child, even if the child has misbehaved

at other times during the day

to teach the caregiver to attend to positive behaviors

to promote the child’s sense of self-worth Steps:

Provide rationale Emphasize the importance of providing positive attention to the child. Elicit the caregiver’s opinion about how attention affects behavior and

people’s motivation to do a good job. Have the caregiver describe his or her best and worst “managers”

and the caregiver’s motivation to work for each. Lead the caregiver to recognize that how he or she was treated

affected the caregiver’s desire to work. Discuss how the child’s behavior may be affected by the caregiver’s

behavior towards the child and how the child’s desire to behave can be increased by improving the caregiver-child relationship.

Set aside one-on-one time for caregiver and child

Encourage the caregiver to set aside a block of time (e.g., 10 minutes) each day devoted to joining the child in an activity the child has chosen.

Teach caregiver to provide positive and descriptive commentary

Show the caregiver how to demonstrate sincere interest in the child’s activities while they are playing.

Instruct the caregiver to provide enthusiastic descriptive (e.g., “You are drawing a tree”) and/or positive (e.g., “I like the way you stacked the blocks”) commentary and praise regarding the child’s behavior.

Encourage caregiver to engage in child’s activity

Suggest that the caregiver become actively involved in the play activity by imitating the child’s behavior in order to demonstrate approval.

Restrict criticism, questions, and commands

It is important that the child lead the activity; that is, the caregiver should refrain from making suggestions, asking questions, and criticizing the child.

Allow the child to use his or her imagination (e.g., coloring the green or making up new rules to a game) without caregiver input about the “correct” way to do things.

Anticipate difficulties When the procedure is initially implemented, the child may engage in negative behavior that characterizes the usual caregiver-child interaction. When this occurs, the caregiver should:

consistently ignore negative behavior by looking away; refrain from scolding the child so as to avoid providing negative

attention for misbehavior; end one-to-one time if disruptive behavior continues or is

dangerous. Over time, however, it is expected that consistent positive attending will result in decreased negative behavior and increased positive caregiver-child interactions.

Attending

Use This When:

To improve the quality of the caregiver-child relationship.

Practitioner Guide

For CaretakerFor Caretaker

One 2-sided page per practice

Coordinated library of evidence based procedures Available online and trained online to therapists

How do I know if it is working?

What do I do if it is not?

Clinical Dashboard

Progress

Practices

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child Mother Father Other

Progress Measure:Fear rating

RewardsCommands

Time OutPraise

Problem SolvingParent Monitoring

Response Cost Ignoring/DRO

Stimulus Control/AntecedentsAttending/Directed Play

ModelingCognitive

Parent PsychoeducationSelf-Monitoring

RelaxationExposure

MaintenancePsychoeducation

Activity SchedulingSkill BuildingSocial Skills

Self MonitoringOtherOtherOtherOtherOtherOtherOtherOther

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

GeneralServicesResearch

LocalAggregateEvidence

Case-SpecificHistorical

Information

CausalMechanismResearch

Wisdom is knowing when to apply our knowledge and when not to (Speigler, 2000).• So….how do we know which of these tools to use

when?

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Decisions

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Actions

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Guiding Information

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Selecting a Treatment Approach

This tells you the treatment types that work for this problem.

This tells you the practice elements associated with those treatment types.

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Getting on Track

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Treatment Going Along Well

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child Mother Father Other

Progress Measure:Fear rating

RewardsCommands

Time OutPraise

Problem SolvingParent Monitoring

Response Cost Ignoring/DRO

Stimulus Control/AntecedentsAttending/Directed Play

ModelingCognitive

Parent PsychoeducationSelf-Monitoring

RelaxationExposure

MaintenancePsychoeducation

Activity SchedulingSkill BuildingSocial Skills

Self MonitoringOtherOtherOtherOtherOtherOtherOtherOther

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child Mother Father Other

Progress Measure:Fear rating

RewardsCommands

Time OutPraise

Problem SolvingParent Monitoring

Response Cost Ignoring/DRO

Stimulus Control/AntecedentsAttending/Directed Play

ModelingCognitive

Parent PsychoeducationSelf-Monitoring

RelaxationExposure

MaintenancePsychoeducation

Activity SchedulingSkill BuildingSocial Skills

Self MonitoringOtherOtherOtherOtherOtherOtherOtherOther

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Treatment Not Going So Well

no

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child Mother Father Other

Progress Measure:Fear rating

RewardsCommands

Time OutPraise

Problem SolvingParent Monitoring

Response Cost Ignoring/DRO

Stimulus Control/AntecedentsAttending/Directed Play

ModelingCognitive

Parent PsychoeducationSelf-Monitoring

RelaxationExposure

MaintenancePsychoeducation

Activity SchedulingSkill BuildingSocial Skills

Self MonitoringOtherOtherOtherOtherOtherOtherOtherOther

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

Do the practices fit the problem?Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child Mother Father Other

Progress Measure:Fear rating

RewardsCommands

Time OutPraise

Problem SolvingParent Monitoring

Response Cost Ignoring/DRO

Stimulus Control/AntecedentsAttending/Directed Play

ModelingCognitive

Parent PsychoeducationSelf-Monitoring

RelaxationExposure

MaintenancePsychoeducation

Activity SchedulingSkill BuildingSocial Skills

Self MonitoringOtherOtherOtherOtherOtherOtherOtherOther

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Service Quality and Treatment Integrity

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

NotEngaged?

Crisis?

Poor Attendance,Complaints, etc.

Pursue engagement;Take appropriate action

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB

Prob. w/ TxSelection?

TreatmentIntegrity?

Practitioner Guide,Clinical and

Quality ReviewConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports

2. Change Intervention3. Further Consultation

4. Add intervention

PW DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

PWDatabaseStart

The MAP: Getting Unstuck…

Youth Network on Child Mental Health (Weisz, PI)

5-Year, multisite randomized trial• Boston, Honolulu

Anxiety, Depression, Conduct ProblemsCommunity therapistsStandard Manuals, MATCH, Usual CareN = 174 children ages 7-13Funded by John D. and Catherine T.

MacArthur Foundation

If therapists like it better, they will be more engaged

Attitudes Towards Evidence-Based Practices

p < .05

Borntrager, C. F., Chorpita, B. F., Higa‐McMillan, C. K., & Weisz, J. R. (2009). Provider attitudes towards evidence‐based practices: Are the concerns with the evidence or with the manuals? Mental Health Services Research, 60, 677‐681.

If therapists like it better, they will use it when the study is over

Therapist Satisfaction with Treatment (rated after each case)

p < .05 p < .05

CAFAS 8-Scale Total

-1.1

-1.5-1.3 -1.3 -1.4

-1.7 -1.8 -1.8

-2.3 -2.3-2.6 -2.6

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

2002.12002.2 2002.3 2002.4 2003.12003.2 2003.3 2003.4 2004.12004.2 2004.3 2004.4

Fiscal Quarter

Cha

nge

per M

onth

(M +

/- SE

)

Final Effect Size for Change = .07/mo, .84/yrDaleiden, Chorpita, Donkervoet, Arensdorf, & Brogan (2006).

$4,640$3,535

$2,087$1,080 $654 $564

$12,477

$9,325

$6,828

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,00020

02

2003

2004

2002

2003

2004

2002

2003

2004

Expe

nditu

res

(US$

)

CALOCUSLevel of Care

(0 - 5)

ASEBAParent CBCL

T-Score

CAFAS8-Scale Total

(0 - 240)

Service Expenditures per Unit of Improvement(Annual Cost per Youth / Average Annual Rate of Improvement)

There are many answers already in the evidence base

Knowing is not enoughWe need to develop and implement the

tools that will help guide system performance