8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
1/29
Wellness Self-ManagementInitiative
Performance IndicatorsGuidebook
The New York State Office of Mental HealthVersion 5
SEPTEMBER 2009
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
2/29
Table of Contents
Performance Indicators Guide 3
Performance Indicators... 3
Completing Performance Indicators.. 6
Submitting Performance Indicators... 7
Internal CQI Reports 18
Quick Guide.. 19Performance Indicator Forms. 20
Indicator #1: Attendance. 21Indicator #2: Discontinuation.. 23Indicator #4: Fidelity. 24
Indicator #5: Personal Progress Checklist... 26Indicator #6: Group Leader Ratings.. 27
Additional Resources 28Key Concepts of Educational SupervisionLesson by Lesson Helpfulness ScaleParticipant Feedback SurveyGroup Leader Feedback Survey
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
3/29
INTRODUCTION
The New York State Office of Mental Health (OMH) recently partnered with the newlycreated Center for Practice Innovations (CPI) at Columbia University. This center is aresource available to mental health agencies, consumers and family members as well as
members of the Practice Improvement Network (PIN). The following guide and developmentof the web based data entry system has been developed by the CPI to support our collectiveefforts to better understand and evaluate important aspects of the Wellness SelfManagement Initiative.
The information on performance indicators and a description of the web based systemcurrently in development (http://nyebptac.networkofcare.org) is meant to support thegathering of practical and reliable information that enables agencies to determine the value ofWSM service and identify challenges affecting the starting, sustaining, and spreading ofWSM services. For OMH, your experiences implementing WSM helps us better understandthe system and programmatic issues that influence the adoption of innovations. It also helps
us better understand the usefulness of the training approach and resource materialsemployed in this initiative. In light of OMHs goal to promote and support the availability ofevidence based practices, it is critical that agencies have the tools to adopt, implement, andsustain quality practices.
The data your agency provides will be critical to furthering this goal. Reporting the datadescribed in this guidebook will enable your agency to 1) keep track of keyperformance indicators and 2) examine your data in relation to data from othermembers of your collaborative as well as regional and statewide data.
It is very important to emphasize that collecting data on specific performance indicators is
meant to support YOUR efforts to improve the quality of your services and OUR efforts todetermine effective methods to promote quality services. It will not be used to judge qualityof a specific agency or program.
As you begin using the web based system to enter data we very much want yourfeedback and suggestions.
PERFORMANCE INDICATORS
As part of the Quality Improvement plan for this initiative, each participating agency will reportdata related to six performance indicators at the intervals listed below (see Table 1, pg 5).We have developed several tools to assist you in gathering information on theseperformance indicators. Table 2 (pg 5) lists the persons responsible for completing the forms.The tools are included at the end of this guidebook.
Indicator #1: Attendance. This indicator provides a measure of how many consumersparticipated in WSM across the State. The QI representative of each agency/site shouldsubmit monthly attendance reports on the CPI secure website for each WSM group.
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
4/29
Indicator #2: Discontinuation. As part of our ongoing QI efforts, it is important for programsto know why consumers are discontinuing participation in the WSM group. The QIrepresentative of each agency/site should submit monthly discontinuation reports on the CPIsecure website for each WSM group, which includes the number of consumers whodiscontinued participation during that month and the primary reason for discontinuation. If no
one discontinued during the month enter 0.
Indicator # 3: Total number of lessons completed by each participant. We havestreamlined this indicator by including it as an item in indicator # 6 (Group leader Ratings:Item 1). For this reason, you may notice that there is no separate indicator #3.
Indicator #4: Fidelity. This indicator provides a measure of the degree to which WSMgroups were implemented based on the specific WSM techniques and values imbedded inthe program and the training provided. Optimally, clinical supervisors complete a fidelityrating at least three times over the course of the group (Time 1: during the first 8 lessonsTime 2: at the midpoint of the group around lesson 34 and Time 3: near the end of the
group around lesson 50) .The clinical supervisor meets with the group leader(s) before thesession, attend the session, and mark the check boxes based on their observations of howoften they observe the techniques being used by the group leader(s) and to what degree thegroup leader(s) incorporate WSM values into the session. The QI representative of eachagency/site should submit a fidelity report on the CPI secure website in after each fidelityreview conducted by the supervisor.
Indicator #5: Personal Progress Checklist. This indicator measures consumersperception of their progress in achieving their personal wellness goals. This information willnot be entered on the web, but rather, the QI representative of each agency/site should mailthe white copy of the checklist to the CPI; this is voluntary and should be done with the
verbal consent of the consumer. Agencies/sites should black out consumers names orother identifying information before mailing to maintain the consumers confidentiality.
Indicator #6: Group Leader Ratings: This indicator measures the group leader(s)assessment of the following for each consumer: number of lessons completed; level of groupparticipation; percentage of lessons for which action steps were completed; level ofinvolvement of others; level of health checkups; and level of progress on their goals. Thisassessment should be completed after completion of lesson 34 and lesson 57. The QIrepresentative of each agency/site should submit a fidelity report on the CPI secure websiteupon completion of lesson 34 and lesson 57.
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
5/29
Table 1. Schedule for Reporting the Six Performance Indicators
Monthly (due the 10th
After AfterIndicator of each month for the By end of Lesson LessonNumber Indicator Name previous month) Lesson 8 34 57
1 Attendance X
2 Discontinuation X
4 Fidelity (supervisor ratings) X X X
Personal ProgressChecklist (the checklistshould be filled out at
Lesson 7, Lesson 34 and
Lesson 57, and submitted5 at the end of the group) X
6Group Leader Ratings ofeach participant across 6items X X
Table 2. Schedule for Who Fills Out the Six Performance IndicatorsIndicatorNumber Indicator Name Group Leader Supervisor Consumer
1 Attendance X
2 Discontinuation X
4 Fidelity X
5 Personal Progress Checklist X
6 Group Leader Ratings X
5
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
6/29
COMPLETING PERFORMANCE INDICATORS
The CPI has setup a website to assist sites with the collection and entry of data for the sixperformance indicators. The process for collecting and entering data is as follows:
1. Each QI team should determine how best to collect the data requested, either usingagency forms or the forms provided in this guide.
2. Forms for collecting data for each indicator should be provided to each group leaderwith the instructions for completing each form and timelines for submitting these formsto the QI representative on the team. (See Tables 1 and 2, pg 5)
3. The QI representative and one other backup person (Users) will request a unique IDand password for accessing the CPI website. If user does not have an ID and password,please contact Melissa Hinds-Martinez and she will get one assigned.
4. The User(s) will access the data-entry website at http://nyCPI.networkofcare.org TheUser(s) will enter all required data on the CPI website monthly and/or specific timesoutlined above (see Table 1, pg 5) for each performance indicator.
5. All of the information that is reported through the website will be in reference to specificgroups that are operating. Some sites may have multiple groups running simultaneously.Each agency/site may choose to enter data on the first group started or on two or moregroups. Data from other groups operating will provide additional data to assist OMH andthe CPI in spreading WSM and assisting agencies in adopting, implementing, andsustaining other evidence based practices. Each time a new group is started, the Usermust assign a unique name to that group (the User should store these group names,with information that distinguishes the groups, in a safe place for future reference).
6 . Once a group has been assigned a name, the User selects which performance indicator
to access. Note that those indicators that are due monthly should be submitted by
the 10th
of each month for the previous month. CPI will send monthly reminders to allagency QIT members.
7. In addition to entering data, the website will eventually allow agencies to print summaryreports showing data for their site relative to the group, region, and State as a whole.These reports include rates of attendance over time, a report of the number discontinuingand completing the program over time, a summary of the fidelity scores, and a summary
of progress consumers made towards their goals.
8. If you have any questions about the data-entry, please contact:MelissaHinds-Martinez at 212-543-5941 or [email protected].
6
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
7/29
SUBMITTING PERFORMANCE INDICATORS
When the User is ready to enter the data, the User will access the website (located athttp://nyCPI.networkofcare.org)and enter their User ID and password as shown here:
7
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
8/29
If this is the first time the User has accessed the website, the User will be brought to thesection of the Home page where they can create a new Group Name. Because all ratingsare attached to a specific group, a unique name will need to be assigned to each group eachtime a new group starts. The figure below illustrates what information is needed, includingthe type of program in which the group is taking place, the number of group leaders, whetherone is a peer specialist, and the date that the group began. In the example below, the Usercreated a group name that was meaningful to them to help them keep track of their groupsgoing forward.
8
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
9/29
Once the User has assigned a name to each group, the User can begin entering theinformation for each respective indicator. The Existing Groups section of the Home pagewill show the User all of the available indicators for which they may enter data. This pagecan always be accessed by clicking the Home button. Typically, the User will begin byentering the Attendance data for the month. This is accessed by clicking on the Attendance
link on the Home page.
9
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
10/29
Performance Indicator #1: Attendance
When entering information regarding Enrollment and Attendance, report the data for eachweek during the reporting month. Most months will only have 4 weeks, but for those with 5, afield is available for reporting data. The website automatically fills in the weeks for each
respective month, by showing the date which falls on the Monday for each week. Note that ifthe last week of the month spills over into the next month, depending on what day of theweek the group runs, the User might need to enter the first weeks data in the last week ofthe previous month. There are spaces available for those agencies which run more than onesession per week; most agencies will only run one session per week, and so all of theirattendance data should be entered under Session #1 Attendance. Also, note that a personis no longer considered enrolled when they clearly indicate that they will no longer beattending the group, or if their circumstances change such that it is apparent that the personwill not be returning to the group. In many cases, determining enrollment will be a judgmentcall on the part of the group leader or the programs policies and procedures. If there are anyweeks during the month when the group is not conducting a session (e.g., starting partway
through the month, vacations, holidays), please just leave the enrollment column blank (thistells us that no session was expected to occur during that week). At any time, you can reviewand edit previous entries by clicking on the Existing Entries tab.
10
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
11/29
A note about enrollment and discontinuation: Every group has its own uniquecircumstances, and agencies are open to implement the program in the way thatmakes the most sense for the consumers being served. Additionally, we recognizethat attendance can be irregular as a new group is starting to form. New people may
join the group late in the program, or existing members may decide the program isnt
right for them. Occasionally, two groups may merge into one group, or one groupmay split if the group membership gets too big, and some groups may becomeinactive for a period and then reconvene with a few new members. We are mostinterested in how many consumers are receiving this intervention, why someconsumers decide to discontinue their involvement, and what impact the programhas on the lives of the consumers who complete the program.
Some things to keep in mind:
The number enrolled in a group should always equal or exceed the numberattending each week.
Consumers are considered enrolled when they state their intention to jointhe group and attend at least one session; a consumer who never attends thegroup should not be considered enrolled.
Only report a consumer as having discontinued when they leave theWSM altogether, and have no intention of returning; consumers who moveto another group should not be considered discontinued.
If a consumer switches to another group, that should be reflected in theenrollment numbers (enrollment should go up for one group and down foranother group), but the person should not be considered discontinued.
Because enrollment and attendance data can not be tied to identifiableconsumers, we have no way of knowing that a person who leaves one group
is the same as the person who shows up in another group. Consequently,please avoid reporting data that incorrectly suggests large changes inenrollment which are, in reality, reorganization of group memberships for verygood clinical and practical reasons. Cases where group enrollment convertsto zero and then stops completely, should be reserved for those cases wherethe group truly ended and was not moved elsewhere. Although individualcircumstances may vary, generally, if one group splits into two, the agencyshould report the data as if there were two groups running simultaneouslyfrom the beginning. Generally, if two groups merge into one, the agencyshould treat both groups as one single group from the beginning (pleasecontact Melissa Hinds-Martinez at 212-543-5941 for questions aboutenrollment and discontinuation).
11
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
12/29
Once the data have been entered, click on the Save button, and if the data were enteredcorrectly, the statement, Record Inserted Successfully will appear at the top.
12
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
13/29
Performance Indicator #2: Discontinuation
The number of participants (if any) who discontinue enrollment during the month should berecorded on Indicator #2 along with the reasons for discontinuation (this will be particularlyhelpful for OMH to understand how to make WSM even better for participants). We anticipate
that for many months, there will be no consumers discontinuing. In those cases, simply enterzero (0) for the number discontinuing during that month. As much as possible, try to useone of the pull-down options unless the situation is so unusual as to warrant a response ofother. You may still use the specify box to elaborate any reason for discontinuing, even ifyou select one of the pull-down options other than other, and being as descriptive aspossible can be useful for helping OMH understand the reasons for discontinuing. Note thatmoving to another group is not considered discontinued and a consumer who neverattended the group should neither be considered enrolled nor discontinued. At any time,you can review and edit previous entries by clicking on the Existing Entries tab.
13
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
14/29
Performance Indicator #4: Fidelity
The supervisor should complete the fidelity ratings for each group and hand the form to the
User (Note: the supervisor should only provide one rating at each of the 3 intervals, and theratings that are entered on the website should be for the group as a whole; althoughindividualized supervision is encouraged, the fidelity ratings must be about the group as awhole and not the individual group leaders). The User will then access the website and enterthe information as shown below. At any time, you can review and edit previous entries byclicking on the Existing Entries tab.
14
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
15/29
15
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
16/29
Performance Indicator #5: Personal Progress Checklist
The Personal Progress Checklist should not be entered on the web. Rather, after the lastlesson is completed, the consumer should tear off the white sheet and hand it to thegroup leader. The group leader will give them to the agency QI person who will mail them
to:
Melissa Hinds-MartinezAdministrative AssistantCenter for Practice InnovationsDepartment of Mental Health Services and Policy ResearchNew York State Psychiatric InstituteRoom 2739, Box 1001051 Riverside DriveNew York, New York 10032
Office: (212) 543-5941Fax: (212) 543-6535Email: [email protected]
Because this is voluntary, it is important to obtain verbal consent from the consumer. Asuggested script follows:
Our agency and OMH are interested in knowing how the Wellness program is helpingconsumers who participate in the groups. Over the past year, you have been recording yourprogress on the Personal Progress Checklist. This checklist provides a lot of informationabout how the program has specifically helped you. Would it be okay with you if I mailed acopy of your completed form to OMH? I will not put your name anywhere on the form. If yourname is on the form, please cross it out to insure confidentiality. This is completely voluntary.Would this be alright with you?
16
mailto:[email protected]:[email protected]:[email protected]8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
17/29
Performance Indicator #6: Group Leader Ratings
The group leader(s) of each group should evaluate the progress of each member of thegroup twice a year (at midpoint and endpoint). At midpoint, the group leader should fill out the
1
st
set of ratings and hand the form to the User, and the User will enter the information onthe web as shown below (note that only the 1st section should be entered at midpoint; at the
end of the program, the group leader should fill out the 2nd
set of ratings and hand it back tothe User to enter the endpoint data; for this indicator, it is very important to make sure thatthe consumer who is listed as #1 for the 1
stset of ratings is the same person who is #1 for
the 2nd
set of ratings, and so forth for all of the group members):
17
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
18/29
INTERNAL CQI REPORTS
Below are some sample reports which may be generated from the data that is entered online.Each agency will be able to see how their site is performing relative to the rest of the State,Region and similar Program Types. This functionality will be made available at a later date. In
the meantime, the Center will continue to provide summary reports by email on a regularbasis.
Attendance Rates Over Time For Your Agency
Compared to the Rest of the State100
90
80
70
60Attendance
50
40Percent
30 Statewide
Your Agency
20
10
0May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
2008 2009*Hypothetical Data
Time to Discontinuation From WSM Group
1
0.9Group
0.8
WSM
0.7
in
0.6
0.5Enrolled
0.4Still
0.3
Statewide0.2Your Agency0.1
Proportion
0
012345678910 11 12
Months Since Group Began *Hypothetical data!
18
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
19/29
Wellness Self-Management ProgramQuick Guide for Submitting Wellness Indicators
M
onthly (by the 10th
of each Month):
Group leaders fill out the forms for Indicators #1 and #2 and hand them to theQIperson responsible for entering the data (see below).
B
y the end of December 2009:
The supervisor makes the first fidelity rating for Indicator #4. The QI member of theQuality Improvement Team enters the data from the Core Competencies Checklist(see below).
M idway through the program (Lesson 34):
The group leader makes the first set of ratings for Indicator #6.
The supervisor makes the second fidelity rating for Indicator #4. All data entry forms are handed to the QI person who will enter this data alongwith the Monthly indicators (see below).
W
hen the group is nearing completion (lessons 54-57):
The group leader completes the second set of ratings for Indicator #6 (GroupLeader Ratings).
The supervisor completes the third fidelity rating for the group.
The group leader will also invite participants to tear out the white copy of the
Personal Progress Checklist and hand it in to the group leader who will mail it to theCPI.
The group leader will also invite participants to complete the feedback surveys as wellas the lesson by lesson helpfulness checklist that is also included in the workbook.Participants are asked to tear these forms from the workbook and hand in to the groupleader. NO IDENTIFIABLE INFORMATION SHOULD BE ON ANY OF THE FORMSHANDED IN BY GROUP PARTICIPANTS
All data entry forms are handed to the QI person who will enter this data alongwith the Monthly indicators (see below). For entering data:
1) The QI person accesses the website. (http://nyCPI.networkofcare.org ).
2) The QI person enters his/her user ID and password to view the secure data-entry screens for his/her agency.
3) The QI person selects the screen that is needed:
a) If a new group has started, the QI person needs to assign a group name
b) If data is being reported for an existing group that already has a group name, theQI person will select the appropriate indicator.
4) Follow the instructions on the screen and submit the data. NOTE: Make sureyouhave the correct Group Name listed for the group being reported.
Questions? Contact Melissa Hinds-Martinez at 212-543-5941
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
20/29
Performance IndicatorForms
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
21/29
Performance Indicator #1: Attendance
Agency _____________________ Program _________________Group Leader(s) _________________
Date Group Began ____________ Group Name_______
Submit data on a monthly basis that will provide information regarding the number ofconsumers enrolled and the actual number who attended their Wellness Self-Management group. Report separately for each group that is running. For those groupswhich run more than one session per week, there are spaces for you to report attendancefor up to three sessions during each week. For those groups that only operate one sessionper week, only fill in the attendance for Session #1. If no sessions were conducted during
a particular week (e.g., group hadnt started yet, vacations, holidays), please leave theenrollment column blank (a blank in the enrollment column tells us that no group wasexpected to attend that week). Weeks 1-5 refer to the weeks of the month for which youare reporting.
Month Reporting (e.g. 6/08) ______________
# enrolled each week # attending each weekSession 1 Session2 Session3
Week 1
Week 2
Week 3
Week 4
Week 5 (if applicable)
21
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
22/29
ATTENDANCE TRACKING FORM: DATA TO ASSIST IN COMPLETING PERFORMANCE
INDICATOR # 1 AND # 6 (Item 1: total number of lessons completed)
Program ______________Group Leader(s) ______________ Date Group Began _________
Group Name___________________________________
# Initials Lessons Completed: Circle the number for each lesson that the person attended in a scheduledgroup or completed in a make-up session.
1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
2 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
3 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
4 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
5 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
6 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
7 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
8 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
9 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
10 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
11 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
12 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2425 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 4647 48 49 50 51 52 53 54 55 56 57
22
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
23/29
Performance Indicator #2: Number and Reasons for Discontinuation
Agency _____________________ Program _________________Group Leader(s) _________________
Date Group Began ____________ Group Name_______
Month Reporting __________ # of consumers who discontinued this month ______For each person who discontinued this month, please indicate the primary reason for discontinuing participation in the
group (where possible, based on self report)
Person#
Primary Reason for Discontinuation (please check one box)
1 Work schedule School scheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________________________________________
2 Work Schedule School ScheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________________________
________________
3 Work Schedule School ScheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________
________________________________
4 Work Schedule School ScheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________________________________________
5 Work Schedule School ScheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________________________________________
6 Work Schedule School ScheduleFamily/Home DemandsRelapse/De-compensationHospitalizedIncarceratedLoss of contactDeceased
Didnt get along with group membersDid not follow ground rulesDischarged from the agency/program Schedule conflict with other servicesWorkbook is too difficultGroup not helpfulPhysically unable to attend
Other, specify:________________________________________________
23
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
24/29
24
PERFORMANCE INDICATOR # 4: Core Competencies Checklist for use by
group facilitators and supervisors
Agency _____________________ Program _________________Group Leader(s) _________________
Date Group Began ____________ Group Name_______
Person who completed ratings __________________________ Date ratings completed ____________
One aspect of the implementation of WSM relates to the clinical competencies demonstrated by
group facilitators. Four core competencies have been identified:
Engagement and motivational techniques
Educational/teaching techniques
Cognitive-behavioral techniques
Group skills utilizing the ROPES format
This checklist will be utilized by supervisors and group facilitators to support the continuous
improvement of skills in implementing the WSM group program. It focuses on the four corecompetencies plus the four WSM values, and allows supervisors to provide feedback to group
facilitators after observing the implementation of group session(s). Furthermore, group facilitators may
use this checklist to assess their knowledge and skills and identify areas they may want to improve.For each skill component listed below, supervisors and/or group facilitators indicate, by checking the
box next to each item, whether the component was Not observed, Sometimes observed or
Frequently observed.
Not Sometimes Frequently
Observed Observed Observed
I. Engagement and Motivational Techniques
Connects topic to members goals and values Uses reflective listening and empathic responding
(avoids judgmental and critical comments)
Emphasizes the benefits of learning the topic area
Makes inspiring comments that promote hope
Expresses appreciation for participants efforts
Not Sometimes Frequently
Observed Observed Observed
II. Educational/Teaching Techniques
Engages people in reading out loud
Asks questions about main points to increase comprehension
Respectfully assists member to stay on topic Clarifies ideas via examples relevant to participants lives
Asks questions to check on participants comprehension
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
25/29
25
Not Sometimes Frequently
Observed Observed Observed
III. Cognitive-Behavioral Techniques
Reframes ideas or beliefs that are self-defeating
Breaks information down into small segments to shape understanding
Models behavior via demonstrations and/or self disclosure
Provides specific feedback to participants Provides positive reinforcing comments to participants
Not Sometimes Frequently
Observed Observed Observed
IV. Group Skills Utilizing the ROPES Format
Prior to group session, describes the plan for the session with specific goals that can be observed
Prior to the group session, describes the progress made by each participant
Checks on the outcome of Action Steps from previous
session Reviews participants experiences with and knowledge
about todays topic
Presents an Overview of todays session
Leads a discussion about the importance of todays topic for participants
Leads a discussion focusing on theImportant Information presented in todays lesson
Assists participants with the completion ofPersonalized Worksheets
In session, assists members planning forAction Steps
Encourages participants to Summarize the important points
of the lesson
Not Sometimes Frequently
Observed Observed Observed
V. WSM Values (Please rate how often the group leaderincorporated the 4 values of the WSM Program)
Hope (emphasizes positive possibilities, acknowledges setbacks without a sense of defeat)
Choice (emphasizes options, encourages informed decision making, helps people identify pros and cons)
Involvement (emphasizes learning from others, encourages action about getting needs met)
Recovery (emphasizes personal strengths, values and goals)
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
26/29
QUALITY OF LIFE PROGRESS CHECKLIST
TIME 1 (Lesson 7) TIME 2 (Lesson 34) TIME 3 (End)Quality of Life Areas I want to improve Mid Point in the WSM At the end of the WSM
Working at a paid job (part or full time)that I like
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Contributing to my community in a usefulway(e.g., volunteering, joining a self help
or peer advocacy group)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Working towards an academic degree,continuing my education or learning a
tradein school
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Living in a place that I like and canmanage successfully
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Having a good relationship with one ormore family members
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Socializing with friends (spendingenjoyable time with others)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Enjoying hobbies, leisure and recreationalactivities
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Engaging in creative activities (music, art,writing, dance etc.)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Being confident that I can handle mymental health problems and not relapse
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Being hopeful about my future (confidentthat I will find success and satisfaction in
important areas of my life)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Stopping or reducing my use of alcohol,drugs or cigarettes
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Using medication in a way that works forme.
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Improving my physical health( healthyeating, exercise, getting regular checkups
for medical, dental and vision problems)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Improving my spiritual/religious side( e.g.,being part of a supportive spiritual
community)
No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
Add your own: No Improvement Improvement Ive achieved this
No Improvement Improvement Ive achieved this
PERFORMANCE INDICATOR 5: Participant perception of progress in
self identified goal areas
26
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
27/29
PERFORMANCE INDICATOR # 6: WSM GROUP LEADER RATINGS:RATING GUIDELINES : To answer the questions accurately, you may want to meet individually with the
participant. Place numbers in the corresponding boxes for each member of the WSM group across two (2) points of
time: lesson 34 and the end.
Agency _____________________ Program _________________Group Leader(s) _________________
Date Group Began ____________ Group Name_______1st Ratings at midpoint (lesson 34)
Date 1st Rating Completed ___/___/___
2nd Ratings at completion of WSM program
Date 2nd Rating Completed ___/___/___
Group
Member
Number
GROUP
MEMBER
INITIALS
(for local use,DO NOT
ENTERINTO WEB-
BASEDDATABASE
1)LESSONS
COMPLETED
2)
PARTICIPATION
3)ACTIONSTEPS
4)INVOLVEMNT
O
FOTHERS
5)HEALTH
CHECKUP
6)GOAL
PROGRESS
1)LESSONS
CO
MPLETED
2)PARTICIPATION
3)ACTIONSTEPS
4)INVOLVEMNT
O
FOTHERS
5)HEALTH
C
HECKUP
PROGRESS
1
2
3
4
56
7
8
9
10
11
12
1) LESSONS COMPLETED: Please indicate the
number of lessons completed by each member of thegroup for two time-points (Lesson 34 and lesson 57).
This is the total number completed since the group
began and include all lessons completed during the
group session, as well as lessons completed in individual
meetings.
4) INVOLVEMENT OF OTHERS: Indicate the extent to which
each consumer involved other people (e.g., family /friends/peers) ingetting the most out if the WSM program.
1 = No involvement
2 = Low level of involvement (discuss few of the lessons with others
3 = Moderate level of involvement (e.g. discuss many lessons with
others, received support such as help with reading some lessons)
4 = High level of involvement (discuss most lessons and received he
with reading and completing action steps; others met with group
leader to discuss their involvement
2) PARTICIPATION
1 = Infrequent participation in group discussion even
when prompted
2 = Usually participates only when prompted3 = Often participates without prompting
4 = Very active participation throughout in group
discussion throughout the program without prompting
5) HEALTH CHECK UP: How many times did the person visit
physical healthcare professional (e.g., doctor, dentist, nurse,
optometrist) in the past 6 months?
0 = never1 = once
2 = twice
3 = three or more times
3) ACTION STEPS: the person completed Action Steps
for:
1 = less than 25% of the lessons he/ she completed
2 = 26-50% of the lessons he/ she completed
3 = 51-75% of the lessons he/ she completed
4 = 76-100% of the lessons he/she completed
6) GOAL PROGRESS1 = Person is doing worse in many of their goal areas
2 = Little or no progress made in any goal area
3 = Significant progress made in one or more goal areas
4 = Achieved one or more goals
27
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
28/29
28
Additional Resources
Educational Supervision informationWSM Participant Feedback SurveyWSM Group Leader Feedback SurveyLesson by Lesson Helpfulness Scale
8/6/2019 WSM Performance Indicators Guidebook FINAL for PINv2_20100127
29/29
Key Concepts of Educational Supervision
The outcome of a successful educational supervisory process is that the supervisee finds ithelpful
Supervisor and supervisee engage in a professional development process designed to reinforceand enhance the skills and knowledge of both parties.
Educational supervision is NOT part of a formal performance evaluation process unless bothparties choose to make it so.
The supervisor is NOT assumed to have superior expertise in conducting WSM groups. In fact,the supervisee may have greater expertise. Nonetheless, the supervisor typically has theresponsibility to support and promote the professional development of those he/she supervises.
Educational supervision creates a time and place for colleagues to review the standards ofpractice associated with conducting a WSM group. In this way, fidelity to the WSM service isreinforced.
The information from educational supervision assists the NYS OMH to evaluate theeffectiveness of the training provided as well as evaluating the learning collaborative method ofdisseminating WSM.
The educational supervisory method emphasizes the importance of supervisees engaging in aSelf-Assessment process. Consequently, educational supervision begins with the superviseereviewing, completing and sharing his/her self assessment using the Core CompetenciesChecklist.
The supervisor shares his/her observations in a manner that emphasizes strengths. The supervisorengages the supervisee in a discussion, The following questions may be considered:
o What are your thoughts about todays group?o What were you pleased with? What would you have wanted to do differently?o In what way were you able to accomplish what you set out to accomplish?o What challenges did you face in todays group?o How helpful do you find the workbook and the use of the ROPES framework?o In what way could I be helpful in addressing areas you are concerned about?
Top Related