Heartland CIT: Toward Comprehensive Evaluation
-
Upload
citinfo -
Category
Health & Medicine
-
view
779 -
download
0
description
Transcript of Heartland CIT: Toward Comprehensive Evaluation
1
Heartland CIT: Toward Comprehensive Evaluation
International Crisis Intervention Team Conference 2010
San Antonio, TexasJune 1, 2010
Shinobu Watanabe-GallowaySteve Spelic
2
Acknowledgement
• Consumers, families, and advocates who made the implementation of Heartland CIT program possible
• Officers and agency leaders who have participated and embraced the program
• Providers and other professionals who have helped the planning and implementation
• CIT Council members and all other individuals and organizations involved
• University of Nebraska Medical Center College of Public Health Evaluation Team
3
Workshop Objectives
1. Learn about a comprehensive evaluation approach used by Heartland CIT Program
2. Learn how Heartland CIT used evaluation data to improve curriculum and identify training needs
4
Heartland CIT Program
U.S. Census 2009 Estimates-Nebraska: 1.8 million -Douglas County: 510,199 (Omaha: 419,949)-Sarpy County: 153,504
6
Background
• Nebraska Chapter of NAMI coordinated an effort for diversion programs including CIT– Consumers/families have played an important role
in planning and implementation• CIT Council established in December 2005• Began with 2 workshop per year with three
agencies to 3 workshops per year
7
8
CIT Participating Agencies
Police De-partments (208) 70%
Douglas Co. Sheriff's Dpt
(38) 13%
Douglas Co. Corrections
(30) 10%
Probation (9) 3% Other (12) 4%
Police Dpt’sOmaha 160Papillion 14Bellevue 12Boys Town 6Lincoln 4Norfolk 4LaVista 2Council Bluff 2Plattsmouth 2Blair 1Fremont 1Total 208
“Other”Metro Comm College 3NE Law Enf Training Ctr 1911 Center 6Emergency Coordinators 2Total 12
9
Heartland CIT Program EvaluationStep 1:
Identify purpose and objectives
Step 2:Choose evaluation
methods
Step 3:Collect data
Step 4:Review results
Step 5:Use results to improve
program
10
Wksp 1
Wksp 2
2006 2007 2008 2010
Wksp 3
Wksp 4
Wksp 5
Wksp 6
Wksp 7
Wksp 8
Wksp 9
Wksp 10
Wksp 11
Wksp 12
2009
Onsite Paper & Pencil Survey
Follow-up Mailed Survey
Follow-up Internet Survey
2007 January 2008 October
2009 Jun - Dec
Focus Groups
Encounter Forms
2010 Apr - May
11
Wksp 1
Wksp 2
2006 2007 2008 2010
Wksp 3
Wksp 4
Wksp 5
Wksp 6
Wksp 7
Wksp 8
Wksp 9
Wksp 10
Wksp 11
Wksp 12
2009
Onsite Paper & Pencil Survey
12
On-Site Survey
• To evaluate participant’s assessment of – Usefulness of each session in terms of content covered and
instruction methods used– Instructor / instruction – Logistics (e.g., facility, communication prior to workshop)– Benefits of overall CIT workshop
• Data has been used to:– Improve the overall curriculum (changing schedule, time
allocation, new topics added, etc)– Refine materials for specific sessions– Identify alternative instructors in a few cases
13
Onsite Questionnaires
- Paper-and-pencil survey- Curriculum / instruction survey
Session Survey1-page survey completed after each session
Summary Survey2-page survey completed at the end of 1-week workshop.
Session Survey Example
14Summary Survey Example
15
Onsite Surveys: Major Findings
• About 70% of officers reported that they did not have adequate knowledge prior to the workshop– Overview of mental illness, substance abuse– Older adults / children, developmental disabilities– Active listening, crisis de-escalation, suicide
prevention– Community resources– Cultural aspects
16
Major Findings (cont’d)
• More than 90% of officers reported enhancement of knowledge from all 21 topics except 3 (children – 89%; substance abuse 86%; co-occurring 78%)
• Sessions that received very positive ratings were:– Consumer / Family Perspectives– Scenario-based training – Hearing Voices– Site Visits
• Workshops provided opportunities for interaction and gaining insight about mental illness
Addressing CIT ClassThe Chief of Police and the Douglas County Sheriff addressing the class at the beginning of training.
Family PerspectivesFamily members sharing their stories with the class.
18
Hearing Voices & Virtual Hallucination SessionsTwo of the most popular sessions of CIT Workshop.
19
Scenario Based Training:Two officers just beginning a scenario and coming to a person’s apartment. The lady in the picture is one of our actors who is portraying a client in crisis. Officers are just showing up “on the scene” to deal with a mental health crisis.
Interaction – Outside of Scenario Based Training Shows interaction between actors of the scenario training, an officer, and a consumer (the officer and consumer are the evaluators of the scenarios) - this is occurring just outside the rooms where we do the scenario training.
20
Opportunities for Interactions:During the workshop, especially around scenario-based sessions, lots of interactions and conversations happened among officers, between officers and consumers and workshop coordinator.
21
Wksp 1
Wksp 2
2006 2007 2008 2010
Wksp 3
Wksp 4
Wksp 5
Wksp 6
Wksp 7
Wksp 8
Wksp 9
Wksp 10
Wksp 11
Wksp 12
2009
Onsite Paper & Pencil Survey
Follow-up Mailed Survey
Follow-up Internet Survey
2007 Fall/Winter
2008 Fall
22
5-Month Mailed Follow-up Survey of Sep 2006 Graduates (N=20 / 23) (January 2007)
• I used different questions and handled the person differently than in the past.
• I have a more calm and open approach with the person in crisis.
• I gave the party involved more time to process the questions that I asked.
• I have asked about their book if they have one or if they have a plan as to what we can do to assist them better especially when they have a history of documented mental illness.
Figure 1: Calls Received from Individuals with Mental Illness
0
1
2
3
4
5
6
7
8
9
1 to 4 5 to 9 10 to 14 15 to 19 20 or more
Number of Calls Received
Num
ber o
f Offi
cers
23
Figure 3: Percentage of participants that would like additional training in these topics
0 10 20 30 40 50 60 70 80 90
Overview of Mental illness
Medications and Side Effects
Age and Medical Conditions
Intervention with Children and Adolescents
Wellness Recovery Action Plan
Active Listening & Crisis De-escalation
Crisis Intervention, Negotiation and Tactical Aspects
Consumer Perspectives
Preventing Suicide
Site visit
Co-occurring Disorders
Developmental Disabilities
Assessing Treatment Resources
Managing Family, Friends, and Bystanders
Family Perspective
Effects of Culture on Psychiatric Crisis
Mental Health Law & Jail Diversion
CIT Officer Panel & Integration CIT Techniques
Hearing Voices
Scenario Based Training
Session Title
Percent
24
25
Internet Follow-up Survey of OPD Graduates (N=24/104) (October 2008)
• Survey monkey makes it easy to collect and analyze data
• The downside is low response rate
• Reminder and notice from the department may increase response rate
0 1 2 to 5 6 to 10 11+0
2
4
6
8
10
12
Figure 2: Calls Received from Individuals with Mental Illness
Number of Calls Received
Nu
mb
er
of
Off
ice
rs
26
Usefulness of CIT Classes
Strongly Disagree Disagree Neither Agree
Strongly Agree
I am better able to assess whether the person has a mental illness or not.
0.0% 4.3% 4.3% 56.5% 34.8%
I am more patient with the person who has a mental illness.
0.0% 0.0% 4.3% 43.5% 52.2%
When dealing with a person who has a mental illness, I have a better understanding of their needs.
0.0% 0.0% 4.3% 52.2% 43.5%
I am better able to communicate with a person who has a mental illness.
0.0% 0.0% 13.0% 43.5% 43.5%
I am better in my use of the de-escalation skills.
0.0% 0.0% 13.0% 52.2% 34.8%
I am better able to use the active listening skills than before.
0.0% 4.3% 13.0% 39.1% 43.5%
27
Wksp 1
Wksp 2
2006 2007 2008 2010
Wksp 3
Wksp 4
Wksp 5
Wksp 6
Wksp 7
Wksp 8
Wksp 9
Wksp 10
Wksp 11
Wksp 12
2009
Onsite Paper & Pencil Survey
Follow-up Mailed Survey
Follow-up Internet Survey
2007 January 2008 October
2009 Jun - Dec
Encounter Forms
28
Incidence Tracking Form
Pilot tested for Douglas Co. Sheriff’s Dept and Omaha PDJune – Dec. 2009
135 forms completed
CIT Non-CIT
Total
Douglas Sheriff
37 71 108
OPD 25 2 27
Total 62 73 135
Consumer Demographics
≤10 11 to 20 21 to 30 31 to 60 61 to 70 71 to 800
10
20
30
40
50
60
70
80
Age (yr)
Num
ber o
f For
ms
Female51%
Male49%
Chart Title
White87%
African A
meri-can
10%
Hispanic4%
30
N = 135 %Known person?Yes 44 32.6No 55 40.7Don't know 33 24.4Repeat call ?Yes 6 4.4No 93 68.9Don't know 29 21.5
N = 135 %Parent 31 23.0Self 21 15.6Partner/spouse 16 11.8Other family member 14 10.4Hospital/medical staff 14 10.4Friend/Acquaintance 10 7.4Boyfriend/girlfriend 8 5.9Otherwise unknown 6 4.4Sibling 6 4.4Police Observation 5 3.7Business Owner 2 1.5Don't know 2 1.5
Reporting Party
Known Person?
Repeated Call?
N = 135 %Suicide threat or attempt 73 54.1Disorderly/disruptive 28 20.7Neglect of self-care 31 23.0Drug-related offense 10 7.4Public intoxication 6 4.4Nuisance 6 4.4Theft 2 1.5
N = 135 %Threats/violence/weapon? Yes 20 14.81No 112 82.96Don't know 2 1.48Incident injuries ?Yes 20 14.81No 115 85.19Drug/alcohol involvement? Yes 54 40 No 72 53.3Don't know 9 6.7Med compliance reported ? Yes 29 21.5 No 106 78.5
Nature of Incident
(“Check all that apply”)
Diagnosis
(“If known” / “Check all that apply”)
N = 135 %Depressed 51 37.8Disoriented/confusion 32 23.7Disorganized speech 28 20.7Belligerent or uncooperative 27 20.0Manic 17 12.6Delusions 14 10.4Unusually scared or frightened 10 7.4Hallucinations 4 3.0
Behaviors at time of Incident
(“Check all that apply”)
N = 135 %Disoriented/confusion 32 23.7Delusions 14 10.4Hallucinations 4 3.0Disorganized speech 28 20.7Manic 17 12.6Depressed 51 37.8Unusually scared or frightened 10 7.4Belligerent or uncooperative 27 20.0
Encounter Outcomes
Douglas Sheriff OPDN = 108 % N = 27 %
Transported to treatment facility 92 85.2 15 55.6
No action/resolved on scene 8 7.4 3 11.1
On-scene crisis intervention 5 4.6 6 22.2
Arrested 4 3.7 3 11.1
Police notified CM or MHC 3 2.8 1 3.7Outpatient/Case management referral 1 0.9 0 0.0
34
Consumer Characteristics: Sheriff’s Dpt vs. OPD (CIT officers)
• Similarities: Age, gender, race, diagnosis• Differences in disposition of consumers
– Disorderly /disruptive (Sheriff: 41% vs. OPD:13%)– Neglect of self care (Sheriff: 30% vs. OPD: 14%)
35
CIT vs. Non-CIT (Sheriff’s Dpt Data)
• Disorderly/Disruptive disposition – Sheriff: 40.5% vs. OPD: 12.7%
• Female consumers – Sheriff: 64.9% vs. OPD: 45.1%
36
Wksp 1
Wksp 2
2006 2007 2008 2010
Wksp 3
Wksp 4
Wksp 5
Wksp 6
Wksp 7
Wksp 8
Wksp 9
Wksp 10
Wksp 11
Wksp 12
2009
Onsite Paper & Pencil Survey
Follow-up Mailed Survey
Follow-up Internet Survey
2007 January 2008 October
2009 Jun - Dec
Focus Groups
Encounter Forms
2010 Apr - May
37
Focus Groups – Preliminary Results
• Three focus groups– OPD– Douglas County Corrections– Douglas County Sheriff’s Department
• Conducted in March – April 2010• 4 – 10 people participated for each group• Five Questions
38
1. Skills or knowledge areas gained
1. Awareness and knowledge of mental illness
2. Knowledge of medication3. Communication and de-
escalation skills 4. Attitudes (tolerance and
openness) 5. More options for handling
crisis situations 6. WRAP plan helped to
establish trust7. Learned about community
resources and organizations
2. Training needs
1. Continuing education on new medications/ updates on mental illness and treatment
2. Expansion on communication techniques
3. Training tailored to meet specific needs of agencies
4. Intervention with youths 5. Section that addresses
mental health of officers (to remove stigma)
39
3. Refresher Course?
1. Follow up within 6 months2. Continuing education
annually 3. Give participants a chance to
go back to CIT workshop and help train others
4. Networking opportunities5. Update contact information
for current resources6. Video-tape session to make
the CIT available to officers who cannot attend
4. Youth Specific CIT Program Needed?
Yes because it will 1. Remove a current gap in the
CIT program2. Prepare officers for
interacting with youth3. Help remove stigma and
labeling of youth, and improve acceptance of youth with mental illnesses
4. Could be a 1-day program that would attract more participants
40
5. Other Suggestions
1. Have a ‘Corrections’ specific class: different needs and many facilities could participate
2. Develop a better policy for dealing with hospitals: need more standardization between hospitals and officers need to know which one is best equipped to handle different situations.
3. More information on interacting with the elderly.
4. More applied, hands-on learning.
41
Future Directions
• Information from multiple sources – Key stakeholders including consumers, families, providers
• System-level analysis– Economic analysis – Impact on community
42
Community
Criminal Justice
Law Enforcement
Providers
Consumers and Families
Community