Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results...

54
Improving Improving Practice and Results Practice and Results 2006 CSR Baseline 2006 CSR Baseline Results Results February 2007

Transcript of Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results...

Page 1: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Measuring and Improving Measuring and Improving Practice and ResultsPractice and Results

2006 CSR Baseline Results2006 CSR Baseline Results

Measuring and Improving Measuring and Improving Practice and ResultsPractice and Results

2006 CSR Baseline Results2006 CSR Baseline Results

February 2007

Page 2: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Transformation Goals Transformation Goals Goal 1: Americans understand that mental health is

essential to overall health.

Goal 2: Mental health care is consumer and family driven.

Goal 3: Disparities in mental health services are eliminated.

Goal 4: Early mental health screening, assessment, and referral to services are common practice.

Goal 5: Excellent mental health care is delivered, and research is accelerated.

Goal 6: Technology is used to access mental health care and information.

Goal 1: Americans understand that mental health is essential to overall health.

Goal 2: Mental health care is consumer and family driven.

Goal 3: Disparities in mental health services are eliminated.

Goal 4: Early mental health screening, assessment, and referral to services are common practice.

Goal 5: Excellent mental health care is delivered, and research is accelerated.

Goal 6: Technology is used to access mental health care and information.

Page 3: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Measuring Performance Measuring Performance

What are the results achieved? Who are successes? Who do we fail? Are we achieving highly consistent, high

fidelity performance in Practice? Are we implementing evidence-based

practices with fidelity and wisdom. Are we intervening earlier?

What are the results achieved? Who are successes? Who do we fail? Are we achieving highly consistent, high

fidelity performance in Practice? Are we implementing evidence-based

practices with fidelity and wisdom. Are we intervening earlier?

Page 4: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Measuring Performance Measuring Performance

Are we working together more collaboratively?

Are we keeping more kids at home, in school? Adults independent & working?

Are we identifying the critical path to improved performance and outcomes?

What is our evidence of progress? How can we be more efficient?

Are we working together more collaboratively?

Are we keeping more kids at home, in school? Adults independent & working?

Are we identifying the critical path to improved performance and outcomes?

What is our evidence of progress? How can we be more efficient?

Page 5: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

The Community Service The Community Service ReviewReview

The Community Service The Community Service ReviewReview

Is a CASE STUDYCASE STUDY technique that relies on a guided professional appraisal.

Uses various sampling strategies to “SPOT CHECKSPOT CHECK” daily front-line practice performance working conditions.

Shows WHAT’S WORKINGWHAT’S WORKING for adult service consumers (status indicators).

Is used for PRACTICE DEVELOPMENTPRACTICE DEVELOPMENT and refinements in PROGRAM DESIGN.

Is a CASE STUDYCASE STUDY technique that relies on a guided professional appraisal.

Uses various sampling strategies to “SPOT CHECKSPOT CHECK” daily front-line practice performance working conditions.

Shows WHAT’S WORKINGWHAT’S WORKING for adult service consumers (status indicators).

Is used for PRACTICE DEVELOPMENTPRACTICE DEVELOPMENT and refinements in PROGRAM DESIGN.

CSR:CSR:Person’s Status,Person’s Status,Daily Practice,Daily Practice,

Present ResultsPresent Results

A new LENS for A new LENS for Viewing OurViewing OurPractice & ResultsPractice & Results

Page 6: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

CSR: Focus on Practice & ResultsCSR: Focus on Practice & ResultsCSR: Focus on Practice & ResultsCSR: Focus on Practice & Results

Status of Persons ServedStatus of Persons ServedAre adults receiving servicesAre adults receiving servicesdoing well now or showingdoing well now or showing

good progress in well-being, good progress in well-being, functioning, risk reduction,functioning, risk reduction,

safe & sustainable supports?safe & sustainable supports?

Status of Persons ServedStatus of Persons ServedAre adults receiving servicesAre adults receiving servicesdoing well now or showingdoing well now or showing

good progress in well-being, good progress in well-being, functioning, risk reduction,functioning, risk reduction,

safe & sustainable supports?safe & sustainable supports?

Practice & PerformancePractice & PerformanceAre our practices workingAre our practices working

well for those being served?well for those being served?

Practice & PerformancePractice & PerformanceAre our practices workingAre our practices working

well for those being served?well for those being served?

Front-Line ConditionsFront-Line ConditionsHow are front-line workingHow are front-line working

conditions affecting practice,conditions affecting practice,performance, and results?performance, and results?

Front-Line ConditionsFront-Line ConditionsHow are front-line workingHow are front-line working

conditions affecting practice,conditions affecting practice,performance, and results?performance, and results?

USING CSR FINDINGS FOR LEARNING & CHANGEUSING CSR FINDINGS FOR LEARNING & CHANGEFindings are used to decide Findings are used to decide WHAT TO DO NEXT!WHAT TO DO NEXT! NotNot just to say: just to say:

““good or bad” or “pass/fail.” A key purpose is Learning and Change.good or bad” or “pass/fail.” A key purpose is Learning and Change.

USING CSR FINDINGS FOR LEARNING & CHANGEUSING CSR FINDINGS FOR LEARNING & CHANGEFindings are used to decide Findings are used to decide WHAT TO DO NEXT!WHAT TO DO NEXT! NotNot just to say: just to say:

““good or bad” or “pass/fail.” A key purpose is Learning and Change.good or bad” or “pass/fail.” A key purpose is Learning and Change.

Page 7: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

8

Coordinate and Coordinate and Deliver services while Deliver services while Advocating for those Advocating for those

not availablenot available

Core Functions in Child & Family PracticeCore Functions in Child & Family Practice

1

3

4 5

6

7 2

Engage Family Members/ Assemble Service Team

Engage Family Members/ Assemble Service Team

Assess & Understand Assess & Understand Current Situation, Current Situation,

Strengths, Needs, Wishes, Strengths, Needs, Wishes, Underlying FactorsUnderlying Factors

Assess & Understand Assess & Understand Current Situation, Current Situation,

Strengths, Needs, Wishes, Strengths, Needs, Wishes, Underlying FactorsUnderlying Factors

Plan Interventions, Plan Interventions, Supports, and Services Supports, and Services Following a Long-term Following a Long-term

Guiding ViewGuiding View

Plan Interventions, Plan Interventions, Supports, and Services Supports, and Services Following a Long-term Following a Long-term

Guiding ViewGuiding View

Access or Assemble Access or Assemble Necessary Resources in Necessary Resources in

Local CommunityLocal Community

Access or Assemble Access or Assemble Necessary Resources in Necessary Resources in

Local CommunityLocal Community

Implement Plan of Implement Plan of Interventions, Strategies, Interventions, Strategies,

SupportsSupports

Implement Plan of Implement Plan of Interventions, Strategies, Interventions, Strategies,

SupportsSupports

Monitor Progress, Monitor Progress, Evaluate Results, What’s Evaluate Results, What’s Working & Not WorkingWorking & Not Working

Monitor Progress, Monitor Progress, Evaluate Results, What’s Evaluate Results, What’s Working & Not WorkingWorking & Not Working

Adapt Delivered Services Adapt Delivered Services Through On-going Through On-going

AssessmentAssessment and Planningand Planning

Adapt Delivered Services Adapt Delivered Services Through On-going Through On-going

AssessmentAssessment and Planningand Planning

Key Functions in a Key Functions in a Basic Practice ModelBasic Practice Model: each function requires strategies & techniques: each function requires strategies & techniques

Find Child & Family in Find Child & Family in Need => Need => ENTRYENTRY

Find Child & Family in Find Child & Family in Need => Need => ENTRYENTRY

Make Transition & Make Transition & Safe Safe Case ClosureCase Closure => => EXITEXIT Make Transition & Make Transition & Safe Safe Case ClosureCase Closure => => EXITEXIT

9

CSR Tests theCSR Tests thePractice ModelPractice Model

Page 8: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

© Human Systems & Outcomes, Inc. • 2003

6 = OPTIMAL PERFORMANCE . Excellent, consistent, effectivepractice for this person in this function area. This level of perfor-mance is indicative of exemplary practice and results for the person.["Optimum” does not imply “perfection.”]

5 = GOOD PERFORMANCE. At this level, the system function isworking dependably for this person, under changing conditions andover time. Effectiveness level is consistent with meeting long-termgoals for the person. [Keep this going for good results]

4 = FAIR PERFORMANCE. This level of performance is minimally ortemporarily sufficient for the person to meet short-term objectives.Performance may be time-limited or require adjustment soon due tochanging circumstances.[Some refinement is indicated]

3 = MARGINAL PERFORMANCE. Practice at this level may be un-der-powered, inconsistent, or not well-matched to need. Performanceis insufficient for the person to meet short-term objectives. [With re-finement, this could become acceptable in the near future.]

2 = POOR PERFORMANCE. Practice at this level is fragmented, in-consistent, lacking in intensity, or off-target. Elements of practicemay be noted, but it is incomplete/not operative on a consistent basis.

1 = ADVERSE PERFORMANCE. Practice may be absent or not oper-ative. Performance may be missing (not done). - OR - Practice strat-egies, if occurring in this area, may be contra-indicated or may beperformed inappropriately or harmfully.

AcceptableRange: 4-6

UnacceptableRange: 1-3

CSR Interpretative Guide for Practice Performance

MaintenanceZone: 5-6

Performance is effec-tive. Efforts should bemade to maintain andbuild upon a positivepractice situation.

RefinementZone: 3-4

Performance is minimalor marginal and maybechanging. Further effortsare necessary to refinethepractice situation.

ImprovementZone: 1-2

Performance is inade-quate. Quick actionshould be taken to im-prove practice now.

Page 9: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Indiana Adult DataIndiana Adult Data

Page 10: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

18-29 years 30-49 years 50-69 years 70+ years0

10

20

30

40

14

11

30

36

24

36

2 3

Male

Female

Sample by Age and Gender

IN Adult CSR Combined Data 2006

1%

23% 23%

2%

15%

19%

9%

7%

Page 11: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Caucasian-American Indian

Latino-American

African-American

Euro-American

0 25 50 75 100 125 150

1

3

25

125

Number of Cases Reviewed

Race/Ethnicity Adult Review

6%

16%

81%

(2%)

(1%)

IN Adult CSR Combined Data 2006

n=154

Page 12: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

10+ placements

6-9 placements

3-5 Placements

1-2 Placements

No placement changes

0 10 20 30 40 50 60 70 80 90 100

1

2

7

47

95

Number of Cases Reviewed

Placement Changes - Adult Review

61%

4%

30%

IN Adult CSR Combined Data 2006

(1%)

(1%)

Page 13: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

37+ months

19-36 months

13-18 months

10-12 months

7-9 months

4-6 months

0-3 months

0 10 20 30 40 50 60 70

59

28

9

15

10

21

13

Number of Cases Reviewed

Length of Time in Current Living Arrangement

8%

13%

68%

IN Adult CSR Combined Data 2006

6%

10%

6%

18%

38%

Page 14: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Satisfaction: caregiver

Satisfaction: person

Social network: recovery

Social network: composition

Living arrangement

Income control

Income adequacy

Safety of others

Safety of the person

0% 20% 40% 60% 80% 100%

96%

92%

55%

62%

88%

79%

77%

94%

89%

Percent acceptable cases

Participant's StatusCommunity Living

IN Adult CSR Combined Data 2006

n=151

n=102

n=99

n=96

Page 15: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Person's Overall Status

Recovery activities

Work

Edu./career preparations

Voice & role in decisions

Mental health status

Substance use

Health/physical well-being

0% 20% 40% 60% 80% 100%

77%

51%

44%

39%

76%

65%

70%

81%

Percent acceptable cases

Participant's StatusWell-being/Life Activities

IN Adult CSR Combined Data 2006

n=155

n=56

n=69

n=79

n=144

Page 16: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Progress

Meaningful personal relationships

Improved social integration

Successful life adjustments

Risk reduction

Progress to recovery goals

Education/work progress

Personal responsibilities

Substance use

Reduction psychiatric symptoms

0% 20% 40% 60% 80% 100%

65%

55%

50%

68%

63%

49%

42%

64%

70%

70%

Percent acceptable casesIN Adult CSR Combined Data 2006

Participant's Recent Progress

n=54

n=125

n=150

n=147

n=78

n=140

n=103

n=145

n=150

n=155

Page 17: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Transition/adjustments

Social integration

Sustainable supports

Income/benefits

Recovery/relapse

Symptom/SA reduction

Personal recovery goals

Assessment & understanding

Teamwork: functioning

Teamwork: formation

Engagement of the person

0% 20% 40% 60% 80% 100%

50%

44%

65%

76%

52%

79%

45%

73%

67%

72%

79%

Percent acceptable cases

Practice PerformancePlanning Treatment & Support

IN Adult CSR Combined Data

2006

n=145

n=154

n=143

n=139

n=140

n=144

n=155

n=124

Page 18: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice Performance

Culturally appropriate

Recovery plan adjustments

Service coordination & continuity

Support for integration

Seclusion/restraint

Medication management

Urgent response

Intervention adequacy

Resources: use

Resources: availability

0% 20% 40% 60% 80% 100%

72%

96%

65%

78%

65%

89%

70%

65%

68%

76%

Percent acceptable cases

Practice PerformanceProviding and Managing Treatment

IN Adult CSR Combined Data

2006

n=100

n=70

n=0

n=129

n=149

n=155

n=45

Page 19: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Acceptability ofService SystemPerformance in

Individual Cases

Status of the Participant inIndividual Cases

Adult Case Review Outcome Categories

IN Adult CSR Combined Data 2006

n=156

Outcome 1:

Good status for the participant,ongoing services

acceptable.

65% (101 cases)

Outcome 3:

Good status for the participant, ongoing services mixed or

unacceptable.

12% (19 cases)

Outcome 2:

Poor status for the participant,ongoing services

minimally acceptable but limited inreach or efficacy.

8% (12 cases)

Outcome 4:

Poor status for the participant,ongoing services

unacceptable.

15% (24 cases)

AcceptableSystem

Performance73%

UnacceptableSystem

Performance27%

Favorable Person Status 77%Unfavorable Person Status 23%

Page 20: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice Performance

Overall Progress

Person's Overall Status

0% 20% 40% 60% 80% 100%

70%

85%

61%

81%

74%

89%

Non-ACT Cases, n=129

ACT Cases, n=27

Participant's Overall Ratings

IN Adult CSR Combined Data 2006

Page 21: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Level 1 Level 2 Level 3 Level 4 Level 5 Level 60%

20%

40%

60%

80%

100%

0% 2% 4%10% 11%

18%

37%42%

48%

28%

0% 0%

Percent of ACT cases, n-27

Percent of Non-ACT cases, n=129

Adult Overall Practice Performance

ADVERSE POOR MARGINAL FAIR GOOD OPTIMAL

IMPROVEMENT REFINEMENT MAINTENANCE

UNACCEPTABLE ACCEPTABLE

IN Adult CSR Reviews 2006

Page 22: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Adult CSR RatingsStatus and Perfromance

IN Adult CSR Regional Data 2006

South & Southwestern Regions

Eastern Region

Northeastern Region

Northwestern Region

Central Region

0% 20% 40% 60% 80% 100%

72%

76%

90%

93%

80%

80%

49%

63%

74%

74%

Percent Acceptable Perfromance

Percent Acceptable Status

Page 23: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Indiana Child GraphsIndiana Child Graphs

Page 24: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

0-4 years 5-9 years 10-13 years 14+ years0

5

10

15

20

25

30

35

5

2

33

19

28

17

28

12

Boys

Girls

Sample by Age and Gender

IN Child CSR Combined Data 2006

n=144

12%

1%

3%

23%

19%

8%

13%

19%

Page 25: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Biracial

Latino-American

African-American

Euro-American

0 10 20 30 40 50 60 70 80 90 100 110

5

6

23

104

Number of Cases Reviewed

Race/Ethnicity Adult Review

IN Adult CSR Combined Data 2006

n=138

75%

17%

(4%)

(4%)

Page 26: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Diagnosis and Co-Occurring Conditions

Number Percent

Mood Disorder 45 31%

Anxiety Disorder 23 16%

PTSD/Adjustment to Trauma 19 13%

Thought Disorder/Psychosis 5 3%

ADD/ADHD 89 62%

Anger Control 30 21%

Substance Abuse/Dependence 7 5%

Learning Disorder 25 17%

Communication Disorder 8 6%

Autsim 8 6%

Disruptive Behavior Disorder (CD, ODD) 55 38%

Mental Retardation 24 17%

Medical Problem 33 23%

Page 27: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Level 8 - 10 Mild Impairment

Level 6 - 7 Moderate Impairment

Level 1 - 5 Serious Impairment

0 10 20 30 40 50 60 70

17

65

60

Number of Cases Reviewed

Child Level of Functioning (GAF)

42%

45%

12%

IN Child CSR Combined Data 2006

n=142

Page 28: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

6-9 placements

3-5 Placements

1-2 Placements

No placement changes

0 10 20 30 40 50 60 70 80 90 100

2

14

36

90

Number of Cases Reviewed

Placement Changes - Child ReviewOver Past 12 Months

62%

10%

25%

IN Child CSR Combined Data 2006

n=142

(1%)

Page 29: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Educational Placement

Number Percent

Regular K-12 education 60 42%

Full inclusion 11 8%

Part-time special education 35 24%

Self-contained special education 22 15%

Alternative education 6 4%

Vocational education 1 1%

Adult basic/GED 1 1%

Day treatment program 6 4%

Completed/graduated 0 0%

Expelled/suspended 1 1%

Dropped out 1 1%

Page 30: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Special Education

Juvenile Justice

Child Welfare

0 10 20 30 40 50 60 70 80 90

80

16

35

Number of Cases Reviewed

Child Involvement with Other Agencies

24%

11%

IN Child CSR Combined Data 2006

56%

Page 31: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

5+ psychotropic medications

4 psychotropic medications

3 psychotropic medications

2 psychotropic medications

1 psychotropic medication

No psychotropic medications

0 10 20 30 40

2

8

26

34

36

37

Number of Cases Reviewed

Number of Psychotropic Medications

26%

6%

24%

25%

IN Child CSR Combined Data 2006

18%

(1%)

Page 32: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

37+ months

19-36 months

13-18 months

10-12 months

7-9 months

4-6 months

0-3 months

0 2 4 6 8 10 12

4

0

2

1

4

8

10

Number of Cases Reviewed

Length of Time in Current OOH Placement

34%

28%

IN Child CSR Combined Data 2006

14%

3%

7%

14%

n=29

Page 33: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Substance use: child/youth

Emotional well-being: school

Emotional well-being: home

Health/Physical well-being

Living arrangements

Permanency

Stability: school

Stability: home

Safety of others

Safety of the child

0% 20% 40% 60% 80% 100%

84%

71%

65%

94%

88%

85%

82%

81%

88%

92%

Percent acceptable cases

Child StatusSafety and Permanency

IN Child CSR Combined Data 2006

n=140

n=139

n=25

Page 34: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Child Status

Lawful behavior: parent

Lawful behavior: child/youth

Social connection & support

Present performance

Instructional engagement

School attendance

Educational placement

0% 20% 40% 60% 80% 100%

78%

93%

83%

66%

61%

63%

87%

81%

Percent acceptable cases

Child StatusDeveloping Life Skills

IN Child CSR Combined

Data 2006

n=141

n=138

n=139

n=140

n=138

n=93

n=121

Page 35: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Caregiver Status

Satisfaction:caregiver

Satisfaction: child/youth

Substance use: caregiver

Caregiver participation

Parenting capacities: reunify

Parenting capacities: present

Caregiver support of the child

0% 20% 40% 60% 80% 100%

88%

91%

91%

80%

86%

40%

80%

85%

Percent acceptable cases

Parent/Caregiver Status

IN Child CSR Combined Data 2006

n=114

n=44

n=15

n=139

n=136

Page 36: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Progress

Progress toward transition

Meaningful relationships: adults

Meaningful relationships: peers

Meaningful relationships: family

Risk reduction

School/work progress

Improved coping/self-mgt.

Substance use reduction

Symptom reduction

0% 20% 40% 60% 80% 100%

75%

55%

81%

64%

85%

73%

64%

70%

71%

76%

Percent acceptable cases

Child Progress

IN Child CSR Combined Data

2006

n=138

n=14

n=60

n=138

n=113

n=134

n=139

n=135

n=131

Page 37: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Outcomes & ending requirements

Assessment: family

Assessment: child

Teamwork: functioning

Teamwork: formation

Engagement

0% 20% 40% 60% 80% 100%

56%

67%

77%

51%

54%

85%

Percent acceptable cases

Practice PerformanceEngagement

IN Child CSR Combined

Data 2006

n=140

Page 38: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Transition/independence

Recovery/relapse

Crisis response

Sustainable supports

Behavior changes

Symptom/SA reduction

0% 20% 40% 60% 80% 100%

44%

50%

55%

50%

74%

78%

Percent acceptable cases

Practice PerformanceIntervention Planning

IN Child CSR Combined Data

2006

n=82

n=40

n=75

n=132

n=141

n=135

Page 39: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice

Tracking & adjustment

Adequacy of intervention

Resources: unit/placement-based

Resources: unique/flexible

Crisis response

Family support

0% 20% 40% 60% 80% 100%

60%

63%

67%

73%

58%

54%

65%

Percent acceptable cases

Practice PerformanceSupport & Resources

IN Child CSR Combined Data 2006

n=134

n=63

n=119

n=78

Page 40: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Acceptability ofService SystemPerformance in

Individual Cases

Status of Child/Family inIndividual Cases

Child Case Review Outcome Categories

Outcome 1:

Good status for child/family,ongoing services

acceptable.

56% (81 cases)

Outcome 3:

Good status for child/family,ongoing services mixed or

unacceptable.

22% (32 cases)

Outcome 2:

Poor status for child/family,ongoing services

minimally acceptable but limited inreach or efficacy.

4% (6 cases)

Outcome 4:

Poor status for child/family,ongoing services

unacceptable.

17% (25 cases)

AcceptableSystem

Performance60%

UnacceptableSystem

Performance39%

Favorable Child Status 78%Unfavorable Child Status 21%

IN Child CSR Combined Data 2006

n=144

Page 41: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice Performance

Overall Child Progress

Overall Parent Caregiver Status

Overall Child Status

0% 20% 40% 60% 80% 100%

59%

73%

75%

73%

88%

82%

78%

82%

Non-SOC Cases, n=133

SOC Cases, n=11

Overall Child Ratings

IN Child CSR Combined Data

2006

Page 42: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Level 1 Level 2 Level 3 Level 4 Level 5 Level 60%

20%

40%

60%

80%

100%

0% 0%

9% 8%

18%

33%27%

33%

45%

25%

0% 1%

Percent of SOC cases, n-11

Percent of Non-SOC cases, n=133

Child Overall Practice Performance

ADVERSE POOR MARGINAL FAIR GOOD OPTIMAL

IMPROVEMENT REFINEMENT MAINTENANCE

UNACCEPTABLE ACCEPTABLE

IN Child CSR Reviews 2006

Page 43: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice Performance

Overall Child Progress

Overall Parent Caregiver Status

Overall Child Status

0% 20% 40% 60% 80% 100%

60%

40%

75%

65%

62%

90%

85%

80%94%

75%

71%

85%

14+ years old, n=40

10-13 years old, n=45

5-9 years old, n=52

Overall Child Ratings by Age Groups

IN Child CSR Combined Data

2006

Page 44: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Practice Performance

Overall Child Progress

Overall Parent Caregiver Status

Overall Child Status

0% 20% 40% 60% 80% 100%

76%

65%

52%

100%

83%

58%

100%

86%85%

94%

88%

63%

Level of Functioning 8-10, n=17

Level of Functioning 6-7, n=65

Level of Functioning 1-5, n=60

Overall Child Ratings by Functioning

IN Child CSR Combined Data

2006

Page 45: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Child Service Data

Page 46: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Child Service Data

Page 47: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Child Service Data

Page 48: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Overall Child CSR RatingsStatus and Perfromance

IN Child CSR Regional Data 2006

South & Southwestern Regions

Eastern Region

Northeastern Region

Northwestern Region

Central Region

0% 20% 40% 60% 80% 100%

62%

81%

48%

66%

69%

94%

64%

76%

56%

76%

Percent Acceptable Perfromance

Percent Acceptable Status

Page 49: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Key Elements for TransformationKey Elements for TransformationKey Elements for TransformationKey Elements for TransformationCLEAR EXPECTATIONSCLEAR EXPECTATIONS: common understandings (a shared vision by all levels of organization) of an integrated, collaborative, system of care based on accepted operating principles, a basic practice model, and measured results.

LEADERSHIPLEADERSHIP:: consistent focus, communications, problem solving, team work, reinforcement of directions and efforts within and across agencies.

TRAINING, MENTORING, & COACHINGTRAINING, MENTORING, & COACHING:: building and sustaining adequate and consistent, case-level practice support and supervision across all frontline units.

FRONTLINE CAPACITYFRONTLINE CAPACITY:: building an effective array of community-based services & finding better ways of conducting daily practice with the current workforce.

FLEXIBLE FUNDINGFLEXIBLE FUNDING:: creating better and more timely ways of accessing what’s needed, when need, and where needed by children and families.

PERFORMANCE MEASUREMENT WITH FEEDBACK LOOPS and ACTION STEPSPERFORMANCE MEASUREMENT WITH FEEDBACK LOOPS and ACTION STEPS:: providing feedback about frontline PERFORMANCE and RESULTS so that people can change from current performance levels to desired performance levels in improving practice and getting better results for children and families receiving services.

CLEAR EXPECTATIONSCLEAR EXPECTATIONS: common understandings (a shared vision by all levels of organization) of an integrated, collaborative, system of care based on accepted operating principles, a basic practice model, and measured results.

LEADERSHIPLEADERSHIP:: consistent focus, communications, problem solving, team work, reinforcement of directions and efforts within and across agencies.

TRAINING, MENTORING, & COACHINGTRAINING, MENTORING, & COACHING:: building and sustaining adequate and consistent, case-level practice support and supervision across all frontline units.

FRONTLINE CAPACITYFRONTLINE CAPACITY:: building an effective array of community-based services & finding better ways of conducting daily practice with the current workforce.

FLEXIBLE FUNDINGFLEXIBLE FUNDING:: creating better and more timely ways of accessing what’s needed, when need, and where needed by children and families.

PERFORMANCE MEASUREMENT WITH FEEDBACK LOOPS and ACTION STEPSPERFORMANCE MEASUREMENT WITH FEEDBACK LOOPS and ACTION STEPS:: providing feedback about frontline PERFORMANCE and RESULTS so that people can change from current performance levels to desired performance levels in improving practice and getting better results for children and families receiving services.

Page 50: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Next Steps Next Steps

improve the alignment of funding sources with program philosophy and goals.

Develop data on access, service array, outcome indicators and quality measures of results and consistency.

Conduct reviews using data profiles and CSR by provider.

Make the commitment!

improve the alignment of funding sources with program philosophy and goals.

Develop data on access, service array, outcome indicators and quality measures of results and consistency.

Conduct reviews using data profiles and CSR by provider.

Make the commitment!

Page 51: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Let’s Talk!Let’s Talk!

Page 52: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Quality Management Quality Management ConcernsConcerns

Service population: client characteristics, demog. profiles

System flow: flow through the system- access entry/exit points, Timeliness, paths, gaps, waitlists

Service utilization: which services people use and why

Resource utilization: fund/resource use patterns, adequacy

Practice: assessment,crafting interventions, timely and skillful delivery, judging results.

System dependability:Fidelity to program model, reliability of implementation, continuity of care

Service results: immediate effects/benefits of services

Stakeholder Satisfaction: perceptions of the service system

General System Concerns Front-line Performance Concerns

Page 53: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

Innovation Curves: Spreading a New PracticeSee See Diffusion of InnovationDiffusion of Innovation; Everett Rogers, fifth edition, 2003; Everett Rogers, fifth edition, 2003

KnowledgeAttitudePractice

The Unfolding of Practice and Related Systems Change Over Time

Perc

ent o

f Pra

ctiti

oner

s

Understanding ofThe New Practice

Commitment toThe New Practice

Full Scale Use Of New Practice

Initial Awareness ofNew Practice

Beginning Acceptance of New Practice

Learning to PerformThe New Practice

Baseline 1 2 3 4 5 6

Confirmation & Integration of the Practice

Limited but Growing Implementation

Page 54: Measuring and Improving Practice and Results Practice and Results 2006 CSR Baseline Results Measuring and Improving Practice and Results Practice and Results.

© Human Systems & Outcomes, Inc., 2001

PracticeImprovementRequires:

n Knowledge of the Outcomes of Treatment

n Quick, continuous and Correct Feedback

n Continually improving Skill/ Craft knowledge of techniques andmethods of intervention

n Relationship of Process and Outcomes

n Ability to Contextualize knowledge to fit individuals

n Ability to Synthesize and Generalize Learning

Bickman, 1999