The Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS): A...
1
The Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS): A One Year Case Study. Elizabeth Cook, B. A., Robert W. Johnson, M. A., Ashley Wynne, M. A., Melissa Tarasenko, M. A., William D. Spaulding, Ph.D. University of Nebraska-Lincoln Introduction There has been a distinct movement toward a comprehensive recovery-oriented philosophy for the treatment of people with serious mental illness (SMI) (Surgeon General’s Report on Mental Health, 1999; President’s New Freedom Commission on Mental Health 2003; Federal Action Agenda, 2005, Uniformed Mental Health Services Package, 2008). Until recently, there were not any comprehensive instruments that summarized the integration of recovery-oriented services across multiple service sites. The need for this type of instrument led to the development of the Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS). The CIMHRRS is a 52 item, theory driven instrument designed to capture a program’s level of integration of psychiatric rehabilitation and recovery-oriented services. Using a site visit format, evaluators analyze the structural and organizational characteristics of the program and functional processes of assessment and treatment provision. Because the CIMHRRS is a newly developed instrument, investigation of its utility is warranted. Thus, the purpose of this project is to evaluate the utility of the CIMHRRS in providing ongoing process monitoring in a single psychiatric rehabilitation and recovery-oriented service program. Between site visits, the program underwent a change in administrative auspices. In addition, it implemented a number of administrative and process changes in response to the initial site visit recommendations. It is expected that these changes will be reflected by changes in CIMHRRS item ratings. More specifically, it is hypothesized that: 1) CIMHRRS item scores for which the maximum number of points were earned at time 1 will remain stable over time. Also, it is hypothesized that: 2) for CIMHRRS items associated with areas in which recommendations were made at Time 1 and for which the program implemented administrative interventions, item scores will increase over time. Method The CIMHRRS was administered at the same program at two time points, with the second time point occurring approximately one year after the first. CIMHRRS ratings were made by independent raters within 8 domains of service provision (Program Mission, Program Demographics & Composition, Organizational Boundaries, Program Functioning, Treatment Team Structure & Process, Assessment Process, Treatment Planning, and Treatment Provision), and consensus ratings were derived. It should be noted that the Program Demographics & Composition and Program Functioning domains were not included in this analysis due to the fact that no subjective ratings were made within these domains. In addition, the Assessment Process domain was not included in this analysis due to changes in the CIMHRRS scoring system within this domain between time points. Individual items that had undergone significant revision between Time 1and Time 2 were omitted as were items that were not available during Time 1. It should also be noted that only one rater was the same between the two time points. Domain and item-by-item score comparisons across time were performed. Percentages of integration by domains were calculated by subtracting each raw item score by 1 in order to prevent artificial inflation, aggregating item scores within domains, and dividing by the total number of items within the domains. Results CIMHRRS domain scores were comparable across time. The Program Mission domain, which was the only domain in which percentages of integration fluctuated between time points, showed an increase in percentage of integration between Time 1 and Time 2 (69% versus 81%, respectively). See Figure 1 for a graphical depiction of percentages of integration by domain over time, and Figure 2 for a graphical depiction of scores within the Program Mission domain over time. Although the percentages of integration within the Treatment Provision domain were the same across time, item scores within this domain varied. See Figure 3 for a graphical depiction of scores within the Treatment Provision domain across time. Item scores did not differ within any of the other domains. Thus, scores within these domains are not graphically depicted. The CIMHRRS captured an increase in specific programming that was recommended at Time 1 (see Figure 4). At Time 1, the program did not offer any training in the identified areas. In other words, these areas comprised 0% of the staff trainings provided by the program or its parent organization at Time 1. At Time 2, these areas comprised 31% of the total staff training. Lastly, while not graphically represented, the CIMHRRS was able to identify changes in the coordination of consumer-led services. Discussion In support of hypothesis 1, CIMHRRS item scores for which the maximum number of points were earned at time 1 remained stable over time. As with any program evaluation, administrators are charged with considering recommendations and program response within the context of client population, program functioning, and funding resources. At Time 1, evaluators recommended five areas in which the program should focus its resources (Program Process Monitoring, Staff Training, Role of Consumer, Performance Improvement, Responsibility for Crisis Services). The program implemented plans of action for two of those areas (Program Process Monitoring and Staff Training), for which administrative interventions produced demonstrable effect s in program functioning. Thus, our second hypothesis was also supported. The domains of Performance Improvement , Responsibility for Crisis Services, and Role of Consumer remained stable over time. While the rating for Role of Consumer did not change within the program, the CIMHRRS demonstrated enough sensitivity to capture the program’s increasing attempts to coordinate consumer–led groups outside of the program. Results show that the CIMHHRS has the capacity to monitor changes over time as a result of specific administrative interventions (as demonstrated in Figures 1, 2, and 4) as well as nonspecific changes in program activity (Figure 3). Given the consistency of scores between two different sets of raters and stability of ratings over time, the CIMHRRS appears to be a promising instrument in evaluating SMI service programs. This study has implications for the utility of the CIMHRRS in providing clinicians, administrators, policy makers, and stakeholders with an objective comprehensive assessment of agency functioning over timeTables/Figures PercentagesofIntegration by Dom ain overa O ne YearPeriod 0 20 40 60 80 100 Program M ission Organizational Boundaries Treatment Team Structure & Process Treatment Planning Treatment Provision Dom ain Nam e Percentageof Integration Tim e 1 Tim e 2 1 2 3 4 5 Program Mission Program M odel Performance Improvement Program Monitoring CIMHRRSRating Item Nam e Program M ission Item Scoresovera One YearPeriod Tim e 1 Tim e 2 Treatm entProvision Item Scores overa O ne YearPeriod 1 2 3 Collab._Psychophar Rehab Counseling Social SkillsTrng. Prob. Sol. Skills Ind. LivingSkills Fam ilyCon, Edu, Tx ContingencyM gt. Supported_Housing Spec. Int. Tx. forCO Cognitive Spec. M odsInt / Traum a-based svcs PeerSupport Supported_Em ploy Ill/W ell_M gt._Skill CBT DBT Personal Therapy IPT Item Nam e CIM HRRSRating Tim e 1 Tim e 2 For more information on the CIMHRRS – Contact: Robert Johnson at [email protected] For more information on the Serious Mental Illness Research Group at the University of Nebraska-Lincoln http://www.unl.edu/dsc A special thanks is extended to the UNL Serious Mental Illness Research Group for their contributions to this project and the development of the CIMHRRS instrument. Figure 1. Figure 2. Figure 3. The CIMHRRS project is supported by Award Number F31MH079771 from the National Institute of Mental Health. The content of this research is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health. Adm instrative C hanges in S taffT rainings 0% 0% 0% 0% 0% 0% 0% 7% 7% 3% 7% 7% 0% 0% 0% 1% 2% 3% 4% 5% 6% 7% 8% RelapsePrevention (Psychiatric) RelapsePrevention (SubstanceAbuse) Combined Psychopharmocologica l Treatment Evidence-based practice EmpiricallySupported Treatment RiskManagement TraumaInform ed Services A reas ofTraining Percent of overall training Tim e 1 Tim e 2 Figure 4.
-
Upload
eleanore-lamb -
Category
Documents
-
view
212 -
download
0
Transcript of The Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS): A...
The Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS): A One Year Case Study.
Elizabeth Cook, B. A., Robert W. Johnson, M. A., Ashley Wynne, M. A., Melissa Tarasenko, M. A., William D. Spaulding, Ph.D.
University of Nebraska-Lincoln
Introduction
There has been a distinct movement toward a comprehensive recovery-oriented philosophy for the treatment of people with serious mental illness (SMI) (Surgeon General’s Report on Mental Health, 1999; President’s New Freedom Commission on Mental Health 2003; Federal Action Agenda, 2005, Uniformed Mental Health Services Package, 2008). Until recently, there were not any comprehensive instruments that summarized the integration of recovery-oriented services across multiple service sites. The need for this type of instrument led to the development of the Comprehensive Inventory of Mental Health and Recovery and Rehabilitation Services (CIMHRRS). The CIMHRRS is a 52 item, theory driven instrument designed to capture a program’s level of integration of psychiatric rehabilitation and recovery-oriented services. Using a site visit format, evaluators analyze the structural and organizational characteristics of the program and functional processes of assessment and treatment provision. Because the CIMHRRS is a newly developed instrument, investigation of its utility is warranted. Thus, the purpose of this project is to evaluate the utility of the CIMHRRS in providing ongoing process monitoring in a single psychiatric rehabilitation and recovery-oriented service program. Between site visits, the program underwent a change in administrative auspices. In addition, it implemented a number of administrative and process changes in response to the initial site visit recommendations. It is expected that these changes will be reflected by changes in CIMHRRS item ratings. More specifically, it is hypothesized that: 1) CIMHRRS item scores for which the maximum number of points were earned at time 1 will remain stable over time. Also, it is hypothesized that: 2) for CIMHRRS items associated with areas in which recommendations were made at Time 1 and for which the program implemented administrative interventions, item scores will increase over time.
Method The CIMHRRS was administered at the same program at two time points, with the second time point occurring approximately one year after the first. CIMHRRS ratings were made by independent raters within 8 domains of service provision (Program Mission, Program Demographics & Composition, Organizational Boundaries, Program Functioning, Treatment Team Structure & Process, Assessment Process, Treatment Planning, and Treatment Provision), and consensus ratings were derived. It should be noted that the Program Demographics & Composition and Program Functioning domains were not included in this analysis due to the fact that no subjective ratings were made within these domains. In addition, the Assessment Process domain was not included in this analysis due to changes in the CIMHRRS scoring system within this domain between time points. Individual items that had undergone significant revision between Time 1and Time 2 were omitted as were items that were not available during Time 1. It should also be noted that only one rater was the same between the two time points. Domain and item-by-item score comparisons across time were performed. Percentages of integration by domains were calculated by subtracting each raw item score by 1 in order to prevent artificial inflation, aggregating item scores within domains, and dividing by the total number of items within the domains.
Results
CIMHRRS domain scores were comparable across time. The Program Mission domain, which was the only domain in which percentages of integration fluctuated between time points, showed an increase in percentage of integration between Time 1 and Time 2 (69% versus 81%, respectively). See Figure 1 for a graphical depiction of percentages of integration by domain over time, and Figure 2 for a graphical depiction of scores within the Program Mission domain over time. Although the percentages of integration within the Treatment Provision domain were the same across time, item scores within this domain varied. See Figure 3 for a graphical depiction of scores within the Treatment Provision domain across time. Item scores did not differ within any of the other domains. Thus, scores within these domains are not graphically depicted. The CIMHRRS captured an increase in specific programming that was recommended at Time 1 (see Figure 4). At Time 1, the program did not offer any training in the identified areas. In other words, these areas comprised 0% of the staff trainings provided by the program or its parent organization at Time 1. At Time 2, these areas comprised 31% of the total staff training. Lastly, while not graphically represented, the CIMHRRS was able to identify changes in the coordination of consumer-led services.
Discussion In support of hypothesis 1, CIMHRRS item scores for which the maximum number of points were earned at time 1 remained stable over time. As with any program evaluation, administrators are charged with considering recommendations and program response within the context of client population, program functioning, and funding resources. At Time 1, evaluators recommended five areas in which the program should focus its resources (Program Process Monitoring, Staff Training, Role of Consumer, Performance Improvement, Responsibility for Crisis Services). The program implemented plans of action for two of those areas (Program Process Monitoring and Staff Training), for which administrative interventions produced demonstrable effect s in program functioning. Thus, our second hypothesis was also supported. The domains of Performance Improvement , Responsibility for Crisis Services, and Role of Consumer remained stable over time. While the rating for Role of Consumer did not change within the program, the CIMHRRS demonstrated enough sensitivity to capture the program’s increasing attempts to coordinate consumer–led groups outside of the program. Results show that the CIMHHRS has the capacity to monitor changes over time as a result of specific administrative interventions (as demonstrated in Figures 1, 2, and 4) as well as nonspecific changes in program activity (Figure 3). Given the consistency of scores between two different sets of raters and stability of ratings over time, the CIMHRRS appears to be a promising instrument in evaluating SMI service programs. This study has implications for the utility of the CIMHRRS in providing clinicians, administrators, policy makers, and stakeholders with an objective comprehensive assessment of agency functioning over time.
Tables/Figures
Percentages of Integration by Domain over a One Year Period
0
20
40
60
80
100
Prog
ram
Miss
ion
Orga
niza
tiona
lBo
unda
ries
Trea
tmen
tTe
am St
ruct
ure
& Pr
oces
s
Trea
tmen
tPl
anni
ng
Trea
tmen
tPr
ovisi
on
Domain Name
Perc
enta
ge o
f Int
egra
tion
Time 1
Time 2
12345
Program Mission
Program Model
Performance Improvement
Program Monitoring
CIMHR
RS Ra
ting
Item Name
Program Mission Item Scores over a One Year Period
Time 1
Time 2
Treatment Provision Item Scores over a One Year Period
1
2
3
Colla
b._P
sych
opha
rRe
hab
Coun
selin
gSo
cial
Skill
s Trn
g.Pr
ob. S
ol. S
kills
Ind.
Livi
ng S
kills
Fam
ily C
on, E
du, T
xCo
nting
ency
Mgt
.Su
ppor
ted_
Hous
ing
Spec
. Int
. Tx.
for C
OCo
gniti
veSp
ec. M
ods I
nt /
Trau
ma-
base
d sv
csPe
er S
uppo
rt
Supp
orte
d_Em
ploy
Ill/W
ell_
Mgt
._Sk
ill CBT
DBT
Pers
onal
The
rapy
IPT
Item Name
CIM
HRRS
Rati
ng
Time 1
Time 2
For more information on the CIMHRRS – Contact: Robert Johnson at [email protected]
For more information on the Serious Mental Illness Research Group at the University of Nebraska-Lincoln http://www.unl.edu/dsc
A special thanks is extended to the UNL Serious Mental Illness Research Group for their contributions to this project and the development of the CIMHRRS instrument.
Figure 1. Figure 2. Figure 3.
The CIMHRRS project is supported by Award Number F31MH079771 from the National Institute of Mental Health. The content of this research is solely the responsibility of the authors and does not necessarily represent
the official views of the National Institute of Mental Health or the National Institutes of Health.
Adminstrative Changes in Staff Trainings
0%0%0%0%0%0%0%
7% 7%
3%
7% 7%
0%0%0%1%2%3%4%5%6%7%8%
Rela
pse
Prev
entio
n(P
sych
iatri
c)
Rela
pse
Prev
entio
n(S
ubst
ance
Abu
se)
Com
bine
dPs
ycho
phar
moc
olog
ica
l Tre
atm
ent
Evid
ence
-bas
edpr
actic
e
Empi
rical
ly S
uppo
rted
Trea
tmen
t
Risk
Man
agem
ent
Trau
ma
Info
rmed
Serv
ices
Areas of Training
Perc
ent o
f ove
rall
train
ing
Tim e 1 Tim e 2
Figure 4.
