Evaluation Plan: Nonviolent Crisis Intervention Tammy …...CPI offers a variety of products,...

of 30 /30
Running head: EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 1 Evaluation Plan: Nonviolent Crisis Intervention Tammy Randall Purdue University EDCI 577 October 11, 2014

Embed Size (px)

Transcript of Evaluation Plan: Nonviolent Crisis Intervention Tammy …...CPI offers a variety of products,...

  • Running head: EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 1

    Evaluation Plan:

    Nonviolent Crisis Intervention

    Tammy Randall

    Purdue University

    EDCI 577

    October 11, 2014

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 2

    Identification of Evaluation Goals and Scope

    Instructional Product Overview: Nonviolent Crisis Intervention

    The Crisis Prevention Institute (CPI) is a resource for staff training that provides training in safe,

    respectful, noninvasive methods for managing disruptive and assaultive behavior in a way that is

    compatible with staff's duty to provide the best possible care.

    CPI offers a variety of products, including the Nonviolent Crisis Intervention Training program

    (CPI, 2013), which is considered the standard for crisis prevention and intervention. The hybrid

    course examined here consists of two main modules that blend online learning with in-class

    instruction and simulations. The first module is a self-paced online presentation that provides the

    background information for all learners, including instructional videos. This module contains a

    quiz at the end of each unit to ensure that each learner has mastered the basic concepts of the

    program, and a participant may not advance to the next unit without a score of at least 80%. The

    second module is an in-class presentation that provides learners with more hands-on instruction,

    including a review of Module 1, simulations of crisis situations, and a final post-test.

    The table below summarizes the content of the Hybrid course.

    Unit Module Location Title

    1 1 Online The CPI Crisis Development Model

    2 1 Online Nonverbal Behavior

    3 1 Online Paraverbal Behavior

    4 1 Online Verbal Intervention

    5 1 Online Precipitating Factors, Rational Detachment, Integrated

    Experience

    6 1 Online Staff Fear and Anxiety

    7 1 Online CPI’s Personal Safety Techniques

    8 1 Online Nonviolent Physical Crisis Intervention and Team Intervention

    9 1 Online Custom unit on Indiana State Law updates

    10 2 In class In-class overview of Units 1 - 9

    11 2 In class Situational Role-Plays

    12 2 In class In-class post-test

    The goal of this training is to change the culture of the target organization, to modify the

    professional behavior of the participants, and to change the relationships between the learners

    and those they serve. Specifically, all participants should learn how to defuse disruptive behavior

    before it becomes a crisis.

    The course presented will include a special section for Indiana schools to help ensure compliance

    with recent changes to Indiana state law concerning the use of restraint and seclusion in schools.

    Under Senate Bill 345, all Indiana schools must have restraint and seclusion plans in place for

    the 2014-15 school year. The law requires the use of preventive techniques, Positive Behavior

    Intervention and Supports (PBIS), and conflict de-escalation to minimize or eliminate the use of

    restraint and seclusion. The law also requires each school to adopt a plan to implement these

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 3

    goals, as well as recurring training of the school’s staff in that school’s adopted plan and

    documentation of the plan and of all incident reports (CPI, 2013).

    The objectives of this course are summarized below (CPI, 2013):

    Reduce the risk of injury due to physical interventions to staff and to those they serve.

    Comply with any legal requirements and minimize the risk of legal liability.

    Meet standards for accreditation.

    Improve staff retention by providing critical skills.

    Improve communication by establishing a common vocabulary.

    Reduce stress and anxiety that may result from uncertainty in a crisis.

    Create a safe, caring, and respectful environment for all staff members and those they serve.

    Audience & Context

    The target for this training is any group of professionals who must interact with clients in such a

    way that crisis situations may arise. This includes members of a variety of professions, including

    education, law enforcement, correctional staff, and medical and mental health staff. It is expected

    that these learners have a clear understanding of their own professions and the specific situations

    that might arise in their workplace. For some participants, certification in crisis intervention and

    prevention is required for their employment. In most cases, the training is sponsored by the

    employer, who provides the facilities for the in-class portion of the training and provides

    payment.

    There are two levels of learners that are targeted by this training:

    Novice learners who have not had crisis prevention / intervention training in the past, and whose goal is to gain the knowledge and skills to deal with crisis situations in their

    workplace.

    Experienced learners who have taken this or similar courses and need to complete the course to refresh their skills or to reestablish their certification. In some environments,

    the re-certification is required periodically, in some cases as often as yearly.

    For this presentation of the course, participants will be teachers, administrators, and staff from

    the XYZ School District in Indiana. During the course of their work day, these individuals may

    encounter students or visitors in disruptive or crisis situations, and they must be prepared to deal

    with any situation in a way that minimizes injury to all participants and that complies with school

    policy and Indiana law.

    Participants must complete an online module before the presentation of the in-class instruction.

    They will need to have access to an internet-capable computer running a recent version of one of

    the popular web browsers. The online course materials will be available for one month before the

    scheduled class date, and all units must be completed with a passing grade of 80% before the

    start of the in-class portion of the course. This portion of the course consists of nine units, each

    of which may be completed in approximately one half hour.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 4

    The in-class module (Module 2) will be presented at the client’s location. This module will

    require one full day, with the morning dedicated to review of the Module 1 concepts and the

    afternoon dedicated to simulations, with approximately one hour reserved for the post-test.

    Access to a computer lab will be provided in order to automate the testing. One half hour will be

    reserved for completing the Level 1 evaluation.

    All participants who complete the course with a passing grade and complete the level 1

    evaluation will receive a “Blue Card” certification and will be granted access to an online

    account that provides access to ongoing trends and review materials.

    Description of Process

    A detailed schedule, including a Gantt chart, is included in Appendix F. The key dates are

    summarized in the table below:

    Task Name Start Finish

    Discovery and Planning 09/08/14 09/10/14

    Identify training goals of the XYZ School District 09/08/14 09/08/14

    Present training overview to stakeholders 09/09/14 09/09/14

    Get buy-in / sign agreement 09/10/14 09/10/14

    Perform Learner Analysis 09/11/14 09/12/14

    Preliminary analysis of trainees 09/11/14 09/11/14

    Group trainees 09/11/14 09/12/14

    Evaluation Preparation 09/15/14 09/18/14

    CPI Learner Survey 09/15/14 09/15/14

    Assess and Customize 09/15/14 09/15/14

    CPI Post-test 09/16/14 09/16/14

    Assess and Customize 09/16/14 09/16/14

    CPI Behavior Survey 09/17/14 09/17/14

    Assess and Customize 09/17/14 09/17/14

    CPI Facility Survey 09/18/14 09/18/14

    Assess and Customize 09/18/14 09/18/14

    Implement hybrid training 09/19/14 10/24/14

    Online course will be available for five weeks prior to

    the start of the in-house training.

    09/19/14 10/23/14

    In-house training 10/24/14 10/24/14

    Evaluation Implementation (Data Collection) 10/24/14 10/26/15

    CPI Learner Survey (level 1) 10/24/14 10/24/14

    CPI Post-test (level 2) 10/24/14 10/24/14

    CPI Behavior Survey (level 3) 04/27/15 04/27/15

    CPI Facility Survey (level 4) 04/27/15 04/27/15

    CPI Behavior Survey (level 3) 10/26/15 10/26/15

    CPI Facility Survey (level 4) 10/26/15 10/26/15

    Data Analysis 10/27/14 11/16/15

    CPI Learner Survey (level 1) 10/27/14 10/31/14

    CPI Post-test (level 2) 11/03/14 11/07/14

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 5

    CPI Behavior Survey (level 3) 05/12/15 05/18/15

    CPI Facility Survey (level 4) 05/12/15 05/18/15

    CPI Behavior Survey (level 3) 11/10/15 11/16/15

    CPI Facility Survey (level 4) 11/10/15 11/16/15

    Feedback / Presentation 12/08/14 12/22/15

    Prepare initial report 12/08/14 12/12/14

    Present initial report results 12/15/14 12/15/14

    Prepare six month assessment 06/16/15 06/22/15

    Present six month results 06/23/15 06/23/15

    Prepare 12 month assessment 12/15/15 12/21/15

    Present 12 month results 12/22/15 12/22/15

    As shown in the schedule, the analysis of the special needs of the XYZ School District, as well

    as the target audience for this module, have been analyzed and all required updates have been

    incorporated into the standard CPI training modules. During Discovery and Planning, CPI

    representatives met with the XYZ District School Board to identify the goals of the training, to

    plan the training schedule, and to discuss the simulation requirements and the required facilities.

    These events are reflected in the schedule summary above, and in Appendix F. At this point, the

    corporation also provided summaries of all crisis reports from the previous school-year to use as

    a baseline against which to compare future results.

    The evaluation strategy has been agreed upon by CPI and the school board. Evaluation

    instruments have been customized to reflect the changes required by the XYZ School district.

    Level 1 and 2 evaluations will be performed immediately following completion of the course,

    and an initial report will be generated and presented based on these instruments. At six and

    twelve month intervals, Level 3 evaluations will be performed to allow participants to assess

    their own behavior and how it has changed as a result of the course. Along with those

    evaluations, administrators will complete the Facility survey to assess the overall value of the

    course in meeting the goals of the district. The results of the six and twelve month evaluations

    will be presented on June 23, 2015, and December 22, 2015, respectively.

    Instruments & Instrument Descriptions

    The evaluation strategy will follow levels 1 and 2 of the Kirkpatrick and Kirkpatrick model.

    Level 1 Evaluation

    Instrument Description

    According to Kirkpatrick & Kirkpatrick (2006, p. 27), Level 1 evaluations focus on learner

    reactions to the training, and they liken it to “measuring customer satisfaction” with the training.

    There are two main uses for the level 1 data that is collected. First, the information will be used

    to evaluate the training program, and as feedback to improve the quality of the course materials

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 6

    and presentation in future programs. It also serves as a way to inform the learners that their input

    is valued and will be taken into account (Kirkpatrick & Kirkpatrick, 2006, p. 27).

    The level 1 evaluation will be performed using a reaction questionnaire containing questions

    assessing the learners’ reactions to the quality of the material and presentation, the relevance of

    the training to their own workplace, and their overall satisfaction with the hybrid course. The

    reaction questionnaire is included in Appendix A. The initial reaction questionnaire must be

    completed at the end of the in-class module.

    Data Collection

    The client has provided computer facilities with web access, and the online version of the

    reaction questionnaire will be used, allowing the data collection and analysis to be done more

    quickly. The results of the reaction surveys will be aggregated and the statistical results will be

    shared with the client and fed back within CPI to improve future courses.

    The reaction questionnaire contains three types of information:

    Identifying information about the participant, which should be recorded for comparison to other individuals or institutions .

    A number of questions that measure the participant’s reactions using a five-point Likert scale. For these questions, only the numerical value must be recorded.

    Two questions provide the ability to provide free-form answers. For these questions, the complete text provided by the participant should be recorded.

    Data Analysis

    The same evaluation tools will be used at all clients, allowing data to be analyzed and compared

    across multiple client sites. For each client, information will be requested regarding the type of

    business (e.g. an elementary school, a mental hospital, etc.) and the profession of each

    participant. All the data will be compiled and the statistics will be anonymously presented to the

    management of both the client and CPI. This will provide an opportunity to CPI to continuously

    improve the course materials and presentation, and will provide the client with an assessment of

    how well the course met the objectives of the training. Since the presentation to the client will

    include data for three categories (participants from the client company, participants in similar

    professions or work environments from other CPI presentations of this course, and all

    participants from all CPI presentations of this course), each client will be able to assess how well

    their participants did compared to those in similar training situations.

    Data Reporting

    For the level 1 Reaction Assessment, the following table will be provided to the client and to CPI

    management, along with an anonymous compilation of all free-form responses. Each value in the

    table will contain an average value from 1 to 5 based on the user responses to the reaction

    questionnaire.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 7

    Client Average Industry Average Global CPI Average

    Overall Satisfaction

    Quality of presentation

    / materials

    Relevance to

    employment

    Level 2 Evaluation

    Instrument Description

    A level 2 (learning) evaluation of the training attempts to determine the extent to which learners

    gain new knowledge, acquire or improve skills, or change their attitude as a result of the training.

    Evaluating learning is critical for this training, since it will result in a certification permitting the

    participant to work in potentially dangerous crisis situations. A level 2 evaluation should focus

    on these three questions (Kirkpatrick & Kirkpatrick, 2006, p. 42):

    What knowledge was acquired?

    What skills were developed or enhanced?

    What attitudes were changed?

    All three of these questions are important to the objectives of this training. The participant must

    gain an understanding of the CPI methodology and the common vocabulary associated with it,

    develop the skills to provide maximum safety in crisis situations, and develop the attitudes to

    employ the acquired knowledge and skills safely and effectively.

    Both qualitative and quantitative data will be gathered during the learning process using three

    evaluation tools:

    During Module 1 training, quizzes will be presented after each unit, and the learner will need to pass all unit quizzes in order to advance to the next unit, and ultimately to the

    Module 2 training.

    During Module 2 training, simulations will be presented to assess the knowledge and skills of the participants and to allow the facilitator to correct and further instruct the

    learners.

    Following completion of the course, a post-test will be given that will assess both knowledge and attitude. Successful completion of the post-test with a score of at least

    80% will be required in order to achieve “Blue Card” certification. See Appendices B

    and C for the Crisis Prevention and Intervention Post-test and the answer key.

    Note that no pre-test will be given, since the degree of change in knowledge, skills, and attitude

    is not as important as each learner achieving at least the minimum level in each of these areas.

    While Kirkpatrick and Kirkpatrick (2006, p. 43) recommend using a control group whenever

    practical, doing so in this case would not be practical since all possible participants will be

    completing the training at approximately the same time.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 8

    Data Collection

    The client has provided computer facilities with web access, and the online version of the post-

    test (see Appendix B) will be used. All participants must pass the post-test with a score of 80%

    or higher in order to successfully complete the course.

    The post-test contains two types of information:

    Identifying information about the participant, which should be recorded for comparison to other individuals or institutions.

    A number of questions that measure the participant’s knowledge using true/false and free-form questions. For these questions, only the percentage correct, as checked against

    the answer key in Appendix C, needs to be recorded.

    Data Analysis

    The same evaluation tools will be used at all clients, allowing data to be analyzed and compared

    across multiple client sites. For each client, information will be requested regarding the type of

    business (e.g. an elementary school, a mental hospital, etc.) and the profession of each

    participant. All the data will be compiled and the statistics will be anonymously presented to the

    management of both the client and CPI. This will provide an opportunity to CPI to continuously

    improve the course materials and presentation based on the areas in which participants scored

    poorly. It will also provide the client with an assessment of how well the course met the

    objectives of the training. Since the presentation to the client will include data for three

    categories (participants from the client company, participants in similar professions or work

    environments from other CPI presentations of this course, and all participants from all CPI

    presentations of this course), each client will be able to assess how well their participants did

    compared to those in similar training situations.

    Data Reporting

    For the level 2 Learning Assessment, the following table will be provided to the client and to CPI

    management. The first value reported will be the percentage of participants in the training who

    successfully completed the course and passed the post-test. The second value will report the raw

    average of the post-test scores as a percentage.

    Client Average Industry Average Global CPI Average

    Percentage of

    participants receiving

    “Blue Card”

    certification

    Average post-test score

    Level 3 Evaluation

    Instrument Description

    A level 3 (behavior) evaluation of the training attempts to determine the extent to which learners

    modify their behavior as a result of the training. This evaluation attempts to determine whether

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 9

    the knowledge, skills, and attitudes learned in the course have been transferred to the job

    (Kirkpatrick & Kirkpatrick, 2006, p. 61). Evaluating behavior is critical for this training, since it

    will determine whether the participants are working in compliance with current law, and if they

    are behaving in a manner that will minimize injury to everyone.

    Data Collection

    The Crisis Prevention and Intervention Behavior Survey (see Appendix D) will be made

    available online to all participants six months and twelve months after course completion, and all

    participants will be required by the school district to complete the surveys. The surveys will give

    the participants an opportunity to describe if and how they have used the training, and whether

    they were able to apply the learned behaviors in a crisis situation. As Kirkpatrick and Kirkpatrick

    (2006, p. 62) note, the tabulation of results from questionnaires can provide a good indication of

    changes in behavior.

    The behavior survey contains three types of information:

    Identifying information about the participant, which should be recorded for comparison to other individuals or institutions.

    A number of questions that request feedback on the use of the training in job-related situations.

    o For questions 1 and 2, record the answer for each question. o For question 3-11, record the total number of yes and no responses.

    Question 12 provides the ability to provide free-form answers. For this question, the complete text provided by the participant should be recorded.

    Data Analysis

    The level 3 evaluation instrument is customized for Indiana schools, but comparison data may be

    used across all schools that use the CPI training. All the data will be compiled and the statistics

    will be presented to the administration and school board with no personal identifying

    information.

    The statistics that will be calculated will be the following:

    Percentage of participants with no prior crisis prevention training / some training / extensive training.

    Average response (1-5) for question 1 regarding confidence to act appropriately.

    Percentage of participants choosing each of the answers to question 2.

    Aggregate percentage of yes answers to questions 3-11.

    Data Reporting

    There are two purposes for the level three data: to measure all participants against the minimum

    standards of the state of Indiana and of the school’s policy for crisis situations, and to compare

    participants from the participating school against other schools where CPI training has been

    presented.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 10

    For the level 3 Behavior Survey, the following table will be provided to the client and to CPI

    management. This will provide all relevant information, and will permit comparison to other

    schools in Indiana who have completed the CPI training. Ideally, the data should reflect

    improvement from the six-month to the twelve-month survey, as participants have more

    opportunities to utilize the new skills, and both values should compare favorably to the overall

    average. If the results of the surveys do not indicate proficiency and confidence in the new skills,

    then students may review all course materials online. In some cases, additional training may be

    recommended for some participants.

    6-month Survey 12-month

    Survey

    Average (All

    Schools)

    No Prior training

    Some Prior training

    Extensive Prior training

    Average confidence level (question 1)

    No opportunity to use training

    (percentage)

    Not confident to use training

    (percentage)

    Have seldom used training

    Have frequently used training

    Average number of yes responses (3-

    11)

    Level 4 Evaluation

    Instrument Description

    A level 4 (results) evaluation of the training attempts to establish the tangible value of the

    training to the stakeholders (Kirkpatrick & Kirkpatrick, 2006, p. 69). The results of this training

    should be the accomplishment of the goals of the school district: stricter compliance with school

    policy in accordance with state law, reduced injury to everyone in crisis situations, and fewer

    escalations of situations to the point where restraint or seclusion is used.

    The Crisis Prevention and Intervention Facility Survey (see Appendix E) will be used to perform

    the level 4 evaluation. This survey will gather information on the number of incidents, the use of

    restraint or seclusion, and appropriateness of the participants’ actions in response to the situation.

    Since Indiana State Law requires the documentation of all incidents involving restraint or

    seclusion, these surveys will simply collect the data already available to the administrators. In

    order to determine what part of any improvements may be attributed to the CPI training course,

    the survey will also collect aggregate statistics about the school’s staff to determine what portion

    of the staff have completed the CPI training, and what portion had other prior crisis management

    training. This will permit a more quantifiable assessment that the goals have been met, rather

    than relying solely on self-assessments of behavior in level 3. However, as Kirkpatrick and

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 11

    Kirkpatrick (2006, p. 69-70) point out, proof is sometimes impossible to get, so we must be

    satisfied with evidence of improvement.

    Data Collection

    The Crisis Prevention and Intervention Facility Survey will be made available online to school

    administrators six months and twelve months after course completion, as shown in the schedule

    in Appendix F. The survey information provided during course planning will provide a baseline

    against which to compare the later evaluations. Kirkpatrick and Kirkpatrick (2006, p. 66) note

    that it is important to allow time for results to be achieved, and most teachers and staff members

    will rarely face crisis situations. For this reason, the level 4 surveys will be conducted six and

    twelve months after course completion, allowing time for new behaviors to be used.

    The facility survey contains four types of information:

    Identifying information about the facility.

    Questions determining the percentage of staff members with prior training, and who have taken the CPI training. For these two questions, only the percentages need to be recorded.

    A number of questions that request information on crisis incidents and how they were handled. The provided values should be recorded for each question.

    Question 7 provides the ability to provide free-form answers. For this question, the complete text provided by the participant should be recorded.

    Data Analysis

    The level 4 evaluation instrument is customized for Indiana schools, but comparison data may be

    used across all schools that use the CPI training. All the data will be compiled and the statistics

    will be presented to the administration and school board with no personal identifying

    information.

    The statistics that will be calculated will be the following:

    Percentage of staff members with prior crisis prevention training.

    Percentage of staff members who have completed the CPI training.

    The raw data for questions 1-6, normalized to a six month period.

    Data Reporting

    For the level 4 Facility Survey, the following table will be provided to the client and to CPI

    management. For comparison, the incident information is presented that was obtained during the

    planning period, along with that obtained in the six- and twelve-month facility surveys. Data

    from all schools that have taken the CPI training is also presented. Note that all values are

    normalized to a six-month period for meaningful comparison.

    Note that the information provided regarding the percentage of the staff who had prior crisis

    management training, and those who have taken the CPI training, is meant to make the data more

    meaningful by exposing influences that might bias the presented values.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 12

    Prior to

    training

    6-month

    survey

    12-month

    survey

    All CPI

    Schools

    Prior training (percentage)

    Completed CPI training

    (percentage)

    Number of incidents

    Number of incidents resulting in

    injury

    Number of incidents resulting in

    restraint

    Number of incidents resulting in

    seclusion

    Number of incidents where state

    law was not followed

    Number of incidents in which

    school policy was not followed

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 13

    Evaluation Plan Executive Summary

    Discussion of Need

    School administrators and staff are faced with the daunting task of keeping all students and staff

    members safe and to control all students to prevent crisis situations from arising and escalating.

    Recent changes to Indiana state law concerning the use of restraint and seclusion in schools will

    require policy changes for many schools, and training to ensure that the law and adopted school

    policy is followed.

    This CPI course is a hybrid of online learning and on-site training and simulation, and is

    designed to meet the requirements of Indiana law. This course will provide all staff members

    with the tools to effectively handle crisis situations, to help avoid escalation and to affect de-

    escalation, and to help prevent injury to all participants. This training will help schools comply

    with Indiana law, and will help provide a safe, secure environment for both students and staff.

    Most schools must carefully schedule training to minimize time away from students and to fit

    into tight staffing budgets. By permitting most of the course to be completed online in the

    privacy of staff members’ homes, the time away from the classroom will be minimized, and the

    course may be completed in a single day of on-site training.

    Objective of Evaluation

    The goals of this training are to increase safety and to reduce the number of incidents that

    escalate to the use of restraint or seclusion. By the end of the training, all participants should feel

    confident that they have the knowledge and skills to handle crisis situations effectively and

    safely. The purpose of the evaluation is to ensure that these goals are met to the satisfaction of all

    stakeholders.

    Summary of Evaluation Plan

    A four-level evaluation will be performed. Level 1 evaluations are designed to ensure that

    participants are satisfied with the training, and level 2 evaluations ensure that participants have

    learned all the knowledge, skills, and attitudes required in the training. Level 1 and 2 evaluations

    will be performed immediately following completion of the course, and an initial report will be

    generated and presented based on these instruments. Level 3 evaluations determine if the

    participants have modified their behavior in response to the training. At six and twelve month

    intervals, Level 3 evaluations will be performed using the Behavior Survey to allow participants

    to assess their own behavior and how it has changed as a result of the course. Findings from

    these surveys will be collected and presented to all stakeholders. Level 4 evaluations assess the

    results of the training to determine its value. At six and twelve month intervals, administrators

    will complete the Facility Survey to assess the overall value of the course in meeting the goals of

    the district. Findings from these surveys will be presented along with the Level 3 findings.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 14

    Key Findings

    The online portion of the course is currently available to all participants, and the on-site training

    is scheduled to take place on October 24. Reaction and learning data will be collected during the

    training, and the initial report based on this data will be presented on December 15. The

    behavior and results data will be collected six months and twelve months after the course has

    been completed, and additional reports will be presented on June 23, 2015, and December 22,

    2015.

    Some of the results from level 1 evaluations will include the following:

    Percentage of participants whose overall reaction was positive.

    Percentage of participants who found the training relevant to their jobs.

    Percentage of participants who found the training engaging.

    Percentage of participants who found the training content sufficient.

    Some of the results from level 2 evaluations will include the following:

    Percentage of participants who have mastered the key information and techniques required by the training.

    The average score for participants in the training.

    Some of the results from level 3 evaluations will include the following:

    Percentage of participants who had an opportunity to use their training in their jobs.

    Percentage of participants who have transferred the knowledge and techniques to their jobs.

    Some examples of how the training was used in real crisis situations.

    Some of the results from level 4 evaluations will include the following:

    To what extent have the objectives of increased safety and increased adherence to school policy been achieved.

    To what extent can the CPI training be shown to be responsible for any increase, taking into account prior training and level of training of current staff.

    Recommendations

    Some possible recommendations to the school district may include the following:

    Based on the results of the level 2, 3, and 4 evaluations, further training may be

    recommended for some or all participants.

    If the level 3 evaluation indicates excessive non-compliance with the policy, then policy

    changes may be recommended to add enforcement or to prevent individuals from acting

    alone in crisis situations.

    If the level 4 evaluation indicates that safety and compliance have not increased to the

    standards of other Indiana schools, then policy changes may be recommended.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 15

    References

    Crisis Prevention Institute, Inc. (CPI). (2013). CPI: Training and consulting in

    behavior management & dementia care. Retrieved from

    http://www.crisisprevention.com/.

    Kirkpatrick, D. and Kirkpatrick, J. (2006). Evaluating training programs: The four levels (3th

    ed). San Francisco, CA: Berrett-Koehler Publishers, Inc.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 16

    Appendices

    Appendix A: Crisis Prevention and Intervention Learner Survey .........................................17

    Appendix B: Crisis Prevention and Intervention Post-test ....................................................19

    Appendix C: Crisis Prevention and Intervention Post-test Answer Key ...............................20

    Appendix D: Crisis Prevention and Intervention Behavior Survey .......................................21

    Appendix E: Crisis Prevention and Intervention Facility Survey ..........................................24

    Appendix F: Crisis Prevention and Intervention Project Plan and Timeline .........................26

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 17

    Appendix A: Crisis Prevention and Intervention Learner Survey

    Instructions: Your impressions of the Crisis Intervention course are important to help us create

    the best possible learner experience. Please answer each question below so that we can derive

    they clearest possible picture of your impressions of this course.

    Using the five-point scale shown, please choose the number that best represents your opinion.

    Student Registration Number: _________________________________________________

    Type of business (e.g. educational facility): ______________________________________

    Your Profession (e.g. elementary teacher): ______________________________________

    Do you have prior experience with crisis prevention training before this course?

    None Some Extensive

    Strongly

    Disagree

    Disagree Neutral Agree Strongly

    Agree

    1. The videos presented in Module 1 helped me understand the concepts

    presented in the course.

    1 2 3 4 5

    2. The online quizzes for Module 1 were a reasonable test of the prerequisite

    knowledge for the Module 2 training.

    1 2 3 4 5

    3. After completing this training I feel I have the tools and knowledge to be

    well equipped to manage a difficult

    situation to help keep all stakeholders

    safer.

    1 2 3 4 5

    4. I would recommend this training to other colleagues in my profession.

    1 2 3 4 5

    5. Please share something new you learned or of which you gained better

    understanding:

    6. I found the nonviolent crisis intervention training to be applicable

    for my professional needs.

    1 2 3 4 5

    7. I believe this training course in nonviolent crisis intervention has

    reduced my risk of injury.

    1 2 3 4 5

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 18

    8. The material presented was well organized, understandable and there

    were good examples of the content in

    useful situations.

    1 2 3 4 5

    9. This course has improved my confidence in handling a crisis

    situation.

    1 2 3 4 5

    10. The group simulations in Module 2 added understanding to the Module 1

    self-paced online training and group

    class lecture presentation.

    1 2 3 4 5

    11. The group leader handled the simulations in a professional manner

    (e.g. learners were not made to feel

    unnecessarily uncomfortable when

    performing paired simulations).

    1 2 3 4 5

    12. I feel this course was a good fit for blended learning, with the right amount

    of online self-paced modules, group

    lecture presentation, and group

    simulations for optimal understanding.

    1 2 3 4 5

    13. Please add additional comments providing reasons why, in your opinion,

    this course was a good fit or not for

    blended learning.

    14. I felt that taking a qualifying post-test immediately after completing the

    Module 2 course was a reasonable

    requirement (e.g. I felt prepared to take

    the test at that point, and the timing was

    appropriate).

    1 2 3 4 5

    15. After completing this course, I would handle a crisis substantially differently

    than I would have before this training.

    1 2 3 4 5

    16. The speed of the class was about right for me to master the material.

    1 2 3 4 5

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 19

    Appendix B: Crisis Prevention and Intervention Post-test

    Please answer each question below based on what you have learned from this course.

    Student Registration Number: _________________________________________________

    Type of business (e.g. educational facility): ______________________________________

    Your Profession (e.g. elementary teacher): ______________________________________

    Do you have prior experience with crisis prevention training before this course?

    None Some Extensive

    1. List two ways in which a hostile person may vent aggression or hostility.

    2. List the four distinct and identifiable behavior levels present in every crisis development situation.

    3. (True / False) Individuals in crisis always exhibit the behaviors in the exact order described in the CPI Crisis Development Model.

    4. What approach is characterized by reassuring the client and using positive verbal and non-verbal approaches to determine what is upsetting the client?

    5. What approach is characterized by offering the client choices and setting reasonable limits with reasonable consequences?

    6. What approach is appropriate as a last resort when others have been placed in imminent danger?

    7. What approach involves attempting to re-establish communication once the client has begun to calm down and show remorse?

    8. Under what condition is setting limits the preferred approach?

    9. (True / False) Physical intervention may sometimes be used as a punitive measure.

    10. Describe the CPI supportive stance.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 20

    Appendix C: Crisis Prevention and Intervention Post-test Answer Key

    1. Verbal and physical 2. Anxiety; defensive; Acting-out Person; Tension Reduction 3. False 4. Supportive 5. Directive 6. Nonviolent Physical Crisis Intervention 7. Therapeutic Rapport 8. Only when the individual begins to lose rationality. 9. False 10. Place your body at an angle to the person and keep a distance of at least a leg length.

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 21

    Appendix D: Crisis Prevention and Intervention Behavior Survey

    Instructions: Please answer each question below regarding your behavior since taking the CPI

    training. Please note that only a summary of statistical information will be presented, and that

    your comments will be anonymous.

    Student Registration Number: _________________________________________________

    Type of business (e.g. educational facility): ______________________________________

    Your Profession (e.g. elementary teacher): ______________________________________

    Your Role: Support Staff Professional / Certified Management / Executive

    Do you have prior experience with crisis prevention training before this course?

    None Some Extensive

    1. How confident are you in your ability to correctly use the conflict resolution techniques learned in the CPI course?

    a. Not at all b. Somewhat unconfident c. Neutral d. Somewhat confident e. Extremely confident

    2. Have you had an opportunity to use your CPI training since attending the course? a. No, I have not had the opportunity (go to question 12) b. No, I did not feel confident enough when the opportunity arose (go to question 12) c. Yes, seldom d. Yes, frequently

    3. If you have used the CPI training, did you feel that the training prepared you to handle the situation confidently and safely?

    a. No b. Yes

    4. If you have used the CPI training, did you consciously consider your position relative to the acting-out person and take a supportive stance?

    a. No b. Yes

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 22

    5. If you have used the CPI training, did you use paraverbal communication to consciously control the tone, volume, and cadence of your voice to create a calm, non-threatening

    atmosphere?

    a. No b. Yes

    6. If you have used the CPI training with a person in the defensive stage, did you use the directive approach to attempt to take control of the situation by setting limits?

    a. No b. Yes

    7. If you have used the CPI training, did you attempt to establish a therapeutic rapport with the acting-out person?

    a. No b. Yes

    8. If you have used the CPI training, was the situation effectively resolved without injury? a. No b. Yes

    9. If you have used the CPI training, was the situation effectively resolved without using seclusion or restraint?

    a. No b. Yes

    10. If you have used seclusion or restraint, were you able to remember and employ the personal safety techniques?

    a. No b. Yes

    11. Overall, among your co-workers who have taken the CPI training, do you feel more confident in the ability of the group to work together in partnership using the CPI techniques

    for conflict resolution?

    a. No b. Yes

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 23

    12. Please share a description of an incident in which crisis intervention was required, and your specific actions in that situation. Please omit names and identifying information.

    __________________________________________________________________________

    __________________________________________________________________________

    __________________________________________________________________________

    Thank you for sharing!

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 24

    Appendix E: Crisis Prevention and Intervention Facility Survey

    Instructions: Please answer each question below regarding the number and types of incidents in

    your facility over the past 6 months, and the ways in which those incidents were handled. Please

    note that only a summary of statistical information will be presented, and that your comments

    will be anonymous.

    Facility Registration Number: _________________________________________________

    Type of business (e.g. educational facility): ______________________________________

    Does your staff have prior experience with crisis prevention training before this course?

    a. None b. Fewer than 25% have prior training c. Fewer than 50% have prior training d. More than 50% have prior training e. Most or all have prior training

    What percentage of your current staff have taken the CPI training?

    a. None b. Fewer than 25% have prior training c. Fewer than 50% have prior training d. More than 50% have prior training e. Most or all have prior training

    1. During the evaluation period, how many incidents have occurred in your facility requiring crisis intervention? ____________

    2. During the evaluation period, how many incidents have occurred in your facility in which

    injuries were sustained by staff or students? ____________

    3. During the evaluation period, how many incidents have occurred in your facility in which a student needed to be restrained? ____________

    4. During the evaluation period, how many incidents have occurred in your facility in which a student needed to be secluded? ____________

    5. During the evaluation period, how many incidents have occurred in your facility in which the state recommendations for restraint and seclusion were not followed? ____________

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 25

    6. During the evaluation period, how many incidents have occurred in your facility in which the school policy for restraint and seclusion were not followed? ____________

    7. Please note any intangible benefits that you have seen as a result of this training (e.g. greater cooperation among staff members).

    ________________________________________________________________________

    ________________________________________________________________________

    ________________________________________________________________________

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 26

    Appendix F: Crisis Prevention and Intervention Project Plan and Timeline (Gantt Chart)

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 27

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 28

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 29

  • EVALUATION PLAN: NON-VIOLENT CRISIS INTERVENTION 30

    Task Name Duration Start Finish Discovery and Planning 3 09/08/14 09/10/14

    Identify training goals 1 09/08/14 09/08/14

    Present training overview 1 09/09/14 09/09/14

    Get buy-in / sign agreement 1 09/10/14 09/10/14

    Perform Learner Analysis 2 09/11/14 09/12/14

    Preliminary analysis of trainees 1 09/11/14 09/11/14

    Group trainees 2 09/11/14 09/12/14

    Evaluation Preparation 4 09/15/14 09/18/14

    CPI Learner Survey 1 09/15/14 09/15/14

    Assess and Customize 1 09/15/14 09/15/14

    CPI Post-test 1 09/16/14 09/16/14

    Assess and Customize 1 09/16/14 09/16/14

    CPI Behavior Survey 1 09/17/14 09/17/14

    Assess and Customize 1 09/17/14 09/17/14

    CPI Facility Survey 1 09/18/14 09/18/14

    Assess and Customize 1 09/18/14 09/18/14

    Implement hybrid training 26 09/19/14 10/24/14

    Online course available 25 09/19/14 10/23/14

    In-house training 1 10/24/14 10/24/14

    Evaluation Implementation 262 10/24/14 10/26/15

    CPI Learner Survey (level 1) 0 10/24/14 10/24/14

    CPI Post-test (level 2) 0 10/24/14 10/24/14

    CPI Behavior Survey (level 3) 0 04/27/15 04/27/15

    CPI Facility Survey (level 4) 0 04/27/15 04/27/15

    CPI Behavior Survey (level 3) 0 10/26/15 10/26/15

    CPI Facility Survey (level 4) 0 10/26/15 10/26/15

    Data Analysis 276 10/27/14 11/16/15

    CPI Learner Survey (level 1) 5 10/27/14 10/31/14

    CPI Post-test (level 2) 5 11/03/14 11/07/14

    CPI Behavior Survey (level 3) 5 05/12/15 05/18/15

    CPI Facility Survey (level 4) 5 05/12/15 05/18/15

    CPI Behavior Survey (level 3) 5 11/10/15 11/16/15

    CPI Facility Survey (level 4) 5 11/10/15 11/16/15

    Feedback / Presentation 272 12/08/14 12/22/15

    Prepare initial report 5 12/08/14 12/12/14

    Present results 1 12/15/14 12/15/14

    Prepare six month assessment 5 06/16/15 06/22/15

    Present six month results 1 06/23/15 06/23/15

    Prepare 12 month assessment 5 12/15/15 12/21/15

    Present 12 month results 1 12/22/15 12/22/15