1

Theory of Change

Chesapeake Bay Funders Network

Program Evaluation

Training Workshop

OMG Center for Collaborative Learning

January 9-10, 2008

2

Building an Evaluation Plan

I. Theory of Change/Pathway Map

II. Outcomes

III. Indicators

IV. Methods

V. Putting Evaluation to Work for You

3

To evaluate how well you’re doing, you must havesome place you’re trying to get to.

Or, in Yogi Berra’s words,

“If you don’t know where you’re going, you’ll end up somewhere else.”

4

 

• People hold beliefs or theories that guide our analysis of problems and the way that we respond to them.

• Clarity about the problem and the theory about how to address it is critical - these beliefs guide our work.

• Helps us to plan our work as well as evaluate it.

Why Theory of Change?

EXAMPLE: Injury Free Coalition for Kids Dog Bite Prevention Program

Strategic FocusTo reduce dog bites among children in Providence through a pilot dog bite prevention and literacy program

Long-term Outcomes

The number of emergency room visits by children due to dog bites is significantly reduced.

Short-term Outcomes

90% of participants are significantly more accurate at determining safe and unsafe dog situations

Program is expanded to include additional classrooms

ActivitiesDevelop a three session program that educates students about dog bites

Deliver program to six elementary schools

Assumptions Assumptions

Contextual Analysis

In Rhode Island, dog bites are the leading cause of emergency room visits by children

In other communities, evidence has shown that educating children about the proper way to interact with dogs reduces the number of bites that occur

Assumptions

6

What is a Theory of Change?

A theory of change is an individual, group of individuals, or organization’s belief about how to positively change conditions or behaviors

It is based on assumptions about what is needed to make these changes

Sound theory should be based on experiences and logical judgments about what works

The process of articulating the assumptions, actions and strategies for change and illustrating them visually is often called pathway mapping

7

The Elements of Pathway Mapping: The Way it Works

 

1. Define key problems/issues challenging our communities and constituencies

1. Describe what we hope to change (desired outcomes) in our communities through our interventions

2. Describe how (strategies, actions, interventions) we hope to accomplish those outcomes

3. Articulate our underlying assumptions about the way things work

8

The Elements of Pathway Mapping: The Way it Works

Examining assumptions about what works….While assumptions are often based on experiences from the field, sometimes they are built on intuition. Challenging these assumptions allows organizations to look more closely at their theory and reflect on the choices that they are making.

As we discuss our pathways to change, surface and discuss the assumptions that we are making concerning why a particular

intervention might work.

Challenge those assumptions – what experiences or information are they based on?

9

Linking strategies, actions and desired outcomes….

In order for a pathway map to be useful, a logical path must be created that connects strategies and actions to reasonable and measurable outcomes.

Closely examine the links between our strategies, actions and outcomes and ensure that they are logical, achievable and direct.

Identify and address gaps in our strategies – paths that might not get us where we want to go.

The Elements of Pathway Mapping: The Way it Works

Here’s the model that will help us create our own pathway maps

Strategic Focus

The overall approach to conducting your work. If you cannot implement all of the desired strategies at once, prioritize them.

Long-term Outcomes

Positive changes that you hope to achieve in the long-run.

Short-term Outcomes

Identify the short-term changes in your target entity (water body, population, etc.) that you hope to achieve by your program/activities.

Actions/Activities

Activities are the individual projects that your organization works on: this could include water testing, organizing clean up events, or contacting legislators.

Assumptions

Contextual Analysis

Identify the major conditions that

serve as the backdrop to your

work: what are the opportunities and

challenges surrounding your

work?

Assumptions

Assumptions

IFCK EXAMPLE WITH ASSUMPTIONS

Strategic Focus

To reduce dog bites among children in Providence through a pilot dog bite prevention and literacy program

Grantee Activities

Develop a three session program that educates students about dog bites

Deliver program to six elementary schools

Contextual Analysis

In Rhode Island, dog bites are the leading cause of emergency room visits by children

In other communities, evidence has shown that educating children about the proper way to interact with dogs reduces the number of bites that occur

AssumptionsChildren can be taught behavior-modifying material in three sessions

Schools will be receptive to this type of programming

Programming learned in the classroom will be applied to situations in real life

Short-term Outcomes

90% of participants are significantly more accurate at determining safe and unsafe dog situations

Program is expanded to include additional classrooms

Long-term Outcomes

The number of emergency room visits by children due to dog bites is significantly reduced.

EXAMPLE: Fullojunk Watershed Alliance

Strategic Focus

Improve the environmental quality of the Fullojunk Watershed through clean up efforts, education, and advocacy.

Long-term Outcomes

Short-term Outcomes

Decreased tons of trash captured in netting system

Alliance ActivitiesOrganize watershed clean up events

Provide education programming to schools and other orgs

Contact lawmakers to advocate for policies that benefit the watershed

Assumptions

Contextual Analysis

The watershed is in a highly populated area, so there is a large amount of pollution in the water

The local public and policymakers have limited knowledge of the state of the watershed

Current legislation does very little to protect the Fullojunk Watershed

Assumptions

Educating kids and community members about effects of trash on water quality will lead to changes in their behavior

13

How do we develop a Theory of Change

1. Convene all stakeholders for a planning meeting (staff, Board, volunteers, key partners, community members)

2. Designate a facilitator

3. Address each element of the TOC and reach consensus

4. Share with key stakeholders and revise based on feedback

14

How do we Use Theory of Change

As a planning and communication tool

Involving stakeholders in planning and helping them get clarity and consensus around program or project activities and outcomes

Providing opportunities to correct faulty underlying assumptions and modify the program’s design before it is launched

15

How do we Use Theory of Change

As a planning and communication tool (cont.)

As a check to help inform decisions about changes as the program evolves – Is this within our focus? – Will it help us get to out outcomes? – What are our assumptions?

16

How do we Use Theory of Change

As an evaluation tool

• Laying the foundation for the evaluation plan through the identification of outcomes that will be measured

• Helping stakeholders in a partnership decide what outcomes they are responsible for

17

How do we Use Theory of Change

As an evaluation tool (cont.)

• Providing a map to revisit implementation steps to inform your understanding of program outcomes (Why did this result happen?)

18

Exercise

Use the blank pathway map worksheets in your folder to develop the Contextual Analysis, Strategic Focus, Assumptions, and Activities for your own organization.