10Program Evaluation
-
Upload
ccusa-volunteer-managers-network-resource-library -
Category
Documents
-
view
5 -
download
0
description
Transcript of 10Program Evaluation
Training Busy StaffTraining Busy Staff to Succeed with Volunteers: to Succeed with Volunteers:
The 55-Minute Training SeriesThe 55-Minute Training Series
Copyright © 2007Betty Stallings
Energize, Inc.
Volunteer Program Evaluation
© 2007, Betty Stallings - Energize, Inc.
S-2
Purposes of this Session►Identify the steps needed to establish a
volunteer program evaluation.►Explore 4 types of volunteer program
evaluation. ►Learn to identify significant
stakeholders who need results of the evaluation communicated to them.
© 2007, Betty Stallings - Energize, Inc.
S-3
Learning Objectives►Discuss the purpose and values of carrying
out a volunteer program evaluation.► Identify key steps and questions to be
answered prior to designing a volunteer program evaluation.
►Understand the difference in purpose and value of 4 types of volunteer program evaluation techniques.
► Identify key stakeholders who must be engaged in the creation/approval of the evaluation process and given the results derived from it.
© 2007, Betty Stallings - Energize, Inc.
S-4
Definition►Volunteer program evaluation is a
process to determine the strengths, weaknesses, and impact of your volunteer program through use of appropriate instruments or techniques to gather this information.
© 2007, Betty Stallings - Energize, Inc.
S-5
Warm-up Exercise►What are some advantages to
evaluating the success of the volunteer program in your organization, unit, department, or for yourselves as individuals?
© 2007, Betty Stallings - Energize, Inc.
S-6
Advantages to Evaluation► Can be used to set goals for future volunteer
involvement.► Creates buy-in from all who are involved in any
aspect of the volunteer program,► Can educate staff and volunteers about the
elements of a successful volunteer program.► If given to appropriate stakeholders, may
stimulate/support needed changes.►May anticipate future concerns that can be
worked on to avert significant future problems.
© 2007, Betty Stallings - Energize, Inc.
S-7
Advantages to Evaluation ► Builds professionalism into the volunteer program.► Can help diffuse negative feelings. ► Can analyze cost/benefits of the volunteer
program.► Great tool for a new staff person.► Gathers various viewpoints from those involved in
the delivery of the volunteer program.► Gains a profile of the current program: the big
picture.► Can be the basis for a staff retreat to enhance the
program.
© 2007, Betty Stallings - Energize, Inc.
S-8
4 Key Concepts of Volunteer Program Evaluation
1. Assessment begins with clearly determining what information you want, the best process to obtain that information, and which stakeholders might be interested in the results.
2. The most common methods of gathering data for evaluating volunteer programs are: Evaluating Output Statistics Exploring Customer Satisfaction Comparison to Outside Standards
© 2007, Betty Stallings - Energize, Inc.
S-9
4 Key Concepts of Volunteer Program Evaluation
3. The ultimate information to derive from Volunteer Program Evaluation is the impact the volunteer program is having on the mission of the organization.
4. After data and impact of volunteer activity is measured, effectively communicate these to significant stakeholders.
© 2007, Betty Stallings - Energize, Inc.
S-10
Key Concept 1
Assessment begins with: ►clearly determining what information
you want, ►the best process to obtain that
information, and ►which stakeholders might be
interested in the results.
© 2007, Betty Stallings - Energize, Inc.
S-11
Preparation for Designing Volunteer Program Evaluation
For what purpose is the evaluation being done? What do you want to be able to decide as a result of the evaluation?
Who are the audiences that will want this information?
Do we have the support of those who will be able to carry out the suggestions arising from the evaluation?
© 2007, Betty Stallings - Energize, Inc.
S-12
Preparation for Designing Volunteer Program Evaluation
What kinds of information are needed to make decision and from whom do we need to gather that information?
What methods can be used to access that information?
What roles will be taken and by whom to: Develop/pilot test/approve the questions. Carry out the tabulation/analysis/interpretation of the
results. Present the material to appropriate audiences.
© 2007, Betty Stallings - Energize, Inc.
S-13
Discussion Question
►Have you ever carried out an evaluation and left out one of these 6 steps? What impact was there on the effectiveness/value of the evaluation?
© 2007, Betty Stallings - Energize, Inc.
S-14
Additional Issues to Consider► How confidentiality will be handled.► Do you have the necessary resources to carry out an
evaluation?►Who will lead the process?► How you obtain information will be impacted by the size
and culture of your organization.► Always involve a task force to carry out the evaluation.►Make certain that those who are in a position to carry out
the recommendations are in support of the evaluation.
© 2007, Betty Stallings - Energize, Inc.
S-15
Key Concept 2
►The most commonly used methods of gathering data for evaluating volunteer programs are: Evaluating Output Statistics Exploring Customer Satisfaction Comparison to Outside Standards
© 2007, Betty Stallings - Energize, Inc.
S-16
Evaluating Output Statistics► This method measures quantitative
information about a volunteer program: Numbers of volunteers, hours, clients served,
etc. Dollar value of donated time Numbers of volunteers serving in different
capacities Specifics about volunteers, such as: gender,
age, ethnicity, education, etc. Percentage of volunteers who also gave
financial donations
© 2007, Betty Stallings - Energize, Inc.
S-17
Methods of Computing the Value of Volunteer Hours► Minimum Wage
Lowest equivalency► Imputed Wage System
Department of Labor figure calculated annually in the US by Independent Sector
► Equivalent Wage System Calculating cost of equivalent paid job
classifications, plus fringe benefits.
© 2007, Betty Stallings - Energize, Inc.
S-18
Output Statistics – Potential Benefits►Change in numbers or kinds of
volunteers in your organization.►Information for volunteer insurance
purposes.►Volunteers who are giving financial
support to the organization.
© 2007, Betty Stallings - Energize, Inc.
S-19
Output Statistics – Potential Shortcomings► Numbers do not deal with impact or
quality of the work donated. ► There are many other quantity markers
beyond “hours served” that would be useful such as: Sources of new volunteers Average length of service Numbers of clients assisted Range of jobs performed by volunteers
© 2007, Betty Stallings - Energize, Inc.
S-20
Exploring Customer Satisfaction► Asking for feedback from your “customers”
Paid staff Volunteers Executive management Clients (if appropriate) Funders Community
► Qualitative information elicited by: Surveys Interviews Focus groups
© 2007, Betty Stallings - Energize, Inc.
S-21
Customer Satisfaction – Potential Benefits►Satisfaction from customers greatly
impacts the success of the volunteer program.
►Determines major weaknesses and strengths in the program from perspective of customers.
►Gains buy-in from staff for volunteer program
►Great form of volunteer appreciation.
© 2007, Betty Stallings - Energize, Inc.
S-22
Customer Satisfaction – Potential Shortcomings
►Answers not accurate if there are organizational trust issues.
►Must prepare those taking the survey regarding reasons for survey and what you will do with the results/ recommendations.
© 2007, Betty Stallings - Energize, Inc.
S-23
Activity►Look over the examples of customer
surveys in Handout #3. ►What are the benefits and
shortcomings of each example?►Which questions would you keep or
discard for your organization and what others would you add? Why?
© 2007, Betty Stallings - Energize, Inc.
S-24
Comparison to Outside Standards►Comparison to objective management
standards created by an outside body or authority.
►Examples of such standards: The Volunteer Management Audit (Energize, Inc.) The Canadian Code for Volunteer Programs
(Volunteer Canada) Points of Light Paradigm Project and Volunteer
Program Evaluation
© 2007, Betty Stallings - Energize, Inc.
S-25
Comparison to Standards – Potential Benefits
►Education of excellence in standards of volunteer management.
►Identification of areas of weakness that may be greatly impacting your program.
►Gives you information that will help you develop goals for improving the program.
© 2007, Betty Stallings - Energize, Inc.
S-26
Comparison to Standards – Potential Shortcomings
►Takes dedication/time.►Not helpful if no action occurs after you
identify areas needing improvement.►May be challenging to compare your
program to a set of generic standards.
© 2007, Betty Stallings - Energize, Inc.
S-27
Key Concept 3
►The ultimate information to derive from Volunteer Program Evaluation is the impact that the volunteer program is having on the mission of the organization.
© 2007, Betty Stallings - Energize, Inc.
S-28
Impact or Outcome-Based Evaluation►Focuses on results – not just activities.►Question asked is: What has changed
in the lives of individuals, families, organizations, or the community as a result of this program?
►Keep asking: “So what?”
© 2007, Betty Stallings - Energize, Inc.
S-29
Impact or Outcome-Based Evaluation►An outcome-based evaluation discovers:
What target groups will change? What about the target will change and in what direction
will it change? What will happen to create that change? How will the change will be identified and tracked? How will the change will be measured? How will we know that our service is what impacted the
change? How long must the change remain in effect before being
considered successful?
© 2007, Betty Stallings - Energize, Inc.
S-30
Challenges of Impact Evaluation► Some attempts and research tools to
demonstrate the impact of volunteers.► None, to date, are without result flaws and all
are very demanding in terms of time.►We can’t strictly evaluate the exact impact of a
volunteer, but in we can measure discrete planned change in the recipients of volunteer service in
Knowledge gain Values altered Behavioral adjustment
Alteration in social or economic status
Attitude change Improved skills Client reaching set
goals Mood, stress
reduction
© 2007, Betty Stallings - Energize, Inc.
S-31
Example►The mission of the mentoring program
is to keep high-risk students out of gangs and involved in further education.
►To describe the impact of the mentoring volunteers, begin to describe the program. For each statement, ask: So what? To whom might this matter? Is this really evidence of something important?
© 2007, Betty Stallings - Energize, Inc.
S-32
Example. continued► We had 90 volunteers working on the mentor project for
3 months. (So what?)► The volunteers were mentoring high risk students at the
continuation high school. (So what?)► Most students reported liking their mentor. (So what?)► Most students began to turn in their homework more
regularly. (So what?)► The students began to develop more pride in their work
and began to get passing grades. (So what?)► As a result of receiving better grades, many were able
to transition back into regular high school. (So what?)
© 2007, Betty Stallings - Energize, Inc.
S-33
Example, continued► 59% of high risk students being mentored for a minimum
of 3 months by a caring adult tested higher on standardized tests, got after-school jobs that they had kept for 2 months, and were expressing interest in additional education to prepare them for the work world. (So what?)
► It has been statistically shown that students who are able to keep an after school job and are motivated to get additional training/education for future employment have a 70% less chance of breaking the law and becoming dependent on drugs and involved with street gangs. OH, WOW, GREAT!!! So that is the impact! I
would love to be involved and/or support that program!
© 2007, Betty Stallings - Energize, Inc.
S-34
Activity►Find a partner.►One person explains the impact of some
volunteer work on the mission of the organization.
►The other person keeps asking, “So what?” until both of you are satisfied that you understand the impact of the volunteer work being done.
© 2007, Betty Stallings - Energize, Inc.
S-35
Key Concept 4
►After data and impact of volunteer activity is measured, effectively communicate these to significant stakeholders.
© 2007, Betty Stallings - Energize, Inc.
S-36
Communicating to Stakeholders
►Who needs the information? Volunteers Paid Staff Executive Team or Executive Director Board Clients Funders The Community
© 2007, Betty Stallings - Energize, Inc.
S-37
Activity►What is your experience with volunteer
program evaluations?
►How might you establish an evaluation system for whatever involvement you have with the volunteer program?
►To whom would you give the results of that evaluation to help improve the program in your area and/or in the larger organization?
© 2007, Betty Stallings - Energize, Inc.
S-38
Summary► Answer key questions before designing your
evaluation.► Select the appropriate method(s) to access the
information needed: Statistical Outputs Customer Satisfaction Standards-based evaluation Impact/Outcome Evaluation
►Share Information with key stakeholders. Summary of key findings Complete report on results and recommendations