Measuring the Impact of Your Volunteer Program Barbra J. Portzline, Ph.D. Liz Benton, MBA.

Measuring the Impact of Your Volunteer Program

Barbra J. Portzline, Ph.D.Liz Benton, MBA

Introductions

• Name• Organization• Role/Position• Experience (if any) with evaluation

© 2013 Barbra Portzline, used with permission

Evaluation…

• What comes to mind when you hear the word evaluation…


Definition of Evaluation

• Evaluation is an intentional, purposeful, systematic and useful inquiry designed to collect information about a program, process, system, structure, organization, individual, and/or group in order to inform decisions or actions


Benefits of Evaluation

• Evaluation ensures quality programs• Helps prioritize resources• Contributes to organization members learning• Evaluation encourages accountability• Helps build a case to convince others of the

effectiveness of certain programs and initiatives• Shows volunteers that their time is worthwhile


Some Things to Evaluate

• Extent to which program met its goals• Impact of program on clients• Awareness of program• Impact of volunteers on the program• Impact of volunteers on the clients• Costs and benefits of a program• Implementation of program• Marketing efforts• Extent that staffing is sufficient• Recruitment of volunteers• Satisfaction• Transfer of skills


Evaluation Planning Process

• Rationale• Purpose• Stakeholders• Key Questions• Methods• Budget• Timeline• Communication Strategy


Focusing the Evaluation

• Helps to ensure that the evaluation concentrates on those issues that will provide the most useful information, yield the greatest insights, and/or provide the most solid basis for action

• It is never possible to look at everything; focusing involves a choice

• “I would really like to know ___ about the work we do”


Reasons to Link Evaluation Efforts to Programs

• Data Driven Decisions• Efficient use of resources• Systematic and Purposeful• Credible results


Key Questions

• The Key Questions should be broad, overarching, questions that guide the evaluation• In what ways are we improving

families’ lives?• To what extent do our volunteers

feel valued?• How can our relationships with key

stakeholders be improved?


Paired Interviews

• In pairs, interview your partner using the questions provided

• Take notes as needed• Share the key elements of a “exceptional volunteer

program” with the larger group


Developing Key Questions

• Based on the strengths discussed, develop 4-6 key questions, that if answered will help you reach your vision

• Examples:• To what extent are our volunteers satisfied with

the level of communication provided? How can communication be improved?

• To what extent is our program meeting volunteers needs? How can it be improved?

• What impact are volunteers having on our clients well-being?


Data Collection Methods

• Interviews (individual and focus group)• Observation• Surveys and questionnaires• Document analysis• Testimonials


Methods Depend On…

• Key Questions• Money• Time• Accessibility of respondents• Requirements


Data Collection Matrix

• Matrix is used to present the key questions, data collection methods, and data sources• Across the top the key questions are written• Methods are listed in the column to the left• Inside each cell are the data sources


Evaluation Data Collection Matrix M

etho

ds

Key Questions

Interviews (focus group, face-to-

face, phone)

Survey

Document Review and Analysis

Observation

Other

16© 2013 Barbra Portzline, used with permission

Evaluation Plan

• The Matrix is the “down and dirty” plan• Often you need a rationale or justification for

evaluation work.


Reasons to Communicate and Report

• To convey information about the program • To describe how a program is working and to

what effect• To demonstrate results and accountability • To aid decision-making about continued

funding, prospective funding, and replication at other sites/organizations

• To learn, grow, and improve the program• To inform decision making by program staff

and management about changes that will improve the program

• To build support for evaluation


Reporting to Stakeholders

• Primary Stakeholders– Usually request the evaluation– Typically are major decision makers– Can consist of program staff, supervisors,

senior managers, and funders



• Secondary Stakeholders– Usually are involved, but with little or no

daily contact– Can consist of program participants, their

supervisors or managers, and other affected by the evaluation



• Tertiary Stakeholders– Usually are more distant but are possibly

interested in findings– Can consist of future program participants,

the general public, and members of the same profession


Reporting During the Evaluation

• To include stakeholders in decision making about evaluation design/activities

• To inform different stakeholders about upcoming evaluation activities

• To keep stakeholders informed about the progress of the evaluation

• To convey information about the program and its evaluation

• To demonstrate results and accountability• To learn, grow, and improve the program


Some Formats

• Comprehensive written reports• Working sessions• Executive summaries• Personal discussions• Newsletters, bulletins, briefs, brochures• News media communications• Video presentations• Memos and postcards• Verbal presentation• Posters• Email


Keep in Mind

• Think about what purposes your communicating and reporting can (and should) serve

• Review and identify the evaluation stakeholders and think about the purposes for which they should be considered a communicating and reporting audience

• Choose the best format – based on purposes (that is, content), audiences, and timing


Measuring the Impact of Your Volunteer Program Barbra J. Portzline, Ph.D. Liz Benton, MBA.

Documents

Transcript of Measuring the Impact of Your Volunteer Program Barbra J. Portzline, Ph.D. Liz Benton, MBA.