Making a Difference:Measuring Your

Outcomes

Montgomery County Volunteer Center

February 4, 2014

Pam Saussy and Barry Seltser, Consultants

Today’s Goals

Help you think about outcomes

Give you an opportunity to develop your own outcomes

Reduce anxiety

2

Measuring Success

“At the end of the year how do we know we accomplished our goals?”

- Outputs, such as:• Number of program offerings• Materials provided to clients

        - Outcomes, such as:

• Increased client knowledge• Living status improvement

3

Why do outcomes matter?

Your funders care

Your program needs them to get better

You care about them

4

What do your stakeholders need?

Executive Director and staff

Board members

Current and potential funders

Partners

Clients

Handout 1—Stakeholder Needs

5

The Logic of Outcomes

Logic Model approach to understanding your program

For each program, connect potential outcomes to:

- Goal of the program (why does it exist?)

- Inputs/Resources (who and what is being invested?)

- Activities (what are you doing?)

- Outputs (what is being produced, what services are provided, what products are offered or delivered?)

6

Logic Model Ladder

7

Questions to Help Generate Outcomes

Handout 2– Questions to Help Generate Outcomes

8

Outcome Indicators

Indicators need to be:- Specific and measurable- Relevant to mission and goals- Feasible to collect- Useful- Accurate, valid, and reliable

9

If possible, make your indicators attributable to your program outcomes

Classifying Outcomes

Handout 3: “A Classification of Outcome Types”

Handout 4: A Logic Model Framework for Outcomes

The value of “sequenced” outcomes

10

Exercise: Identifying Outcomes

Use Handout 5 to identify 2-3 specific sequenced outcomes you use or would like to use

Discuss some examples with the full group

11

Setting Standards/Targets

Try to find a balance between “realistic” and “stretch” standards

Use your targets to track improvements or gaps

12

Are you meeting them?

If so, what changes could you make to meet

even higher ones?

If not, why, and what could you change to get

closer?

Evidence

What information will you need to collect and analyze?

How will you do this?

How strong is the evidence likely to be?

Is it feasible to collect and analyze it?

13

Measurement and Collection Issues

Measurement Challenges (Handout 6)

Tips for Collecting Information (Handout 7)

14

Evidence and Belief

“…often enough our faith beforehand in an uncertified result is the only thing that makes the result come true.” – William James

The power and temptation of belief

Outcomes Measurement 15

How Much Evidence of Success Do You Need?

Certainty or Strong Evidence- Usually requires control or comparison group, pre-

and post-measures, and many resources

Likelihood of Success- Usually requires pre- and post-measures, a strong

logic model, and preferably a comparison group

Examples or Suggestions of Success- Can rely on anecdotal stories, personal reports,

evidence of post-program conditions

16

Assessing Your Measures

Indicator Assessment Instrument (Handout 8)

17

Summary of Key Points

Start where you are, and develop stronger measures over time

Think sequentially about your outcomes

Outcome information should help you improve—outcomes are your tools, not your masters

18

Other Help and Resources

Consult experts in measurement and evaluation design as needed

Use internet, libraries, other organizations to find examples and measurement tools

Handout 9

19

Thank you for your participation!

For information or help, please contact:

Kathleen.stobie@montgomerycountymd.gov