Post on 04-Jan-2016
outcomemapping.ca
Introduction to Outcome Mapping
Ziad Moussa, LebanonJan Van Ongevalle, Belgium
Heidi Schaefer, Canada
22 September 2014 – Dar Es Salaam – OM Lab
Workshop Objectives
Clarify what Outcome Mapping is and doesIntroduce and apply Outcome Mapping
concepts and toolsConsider if and how Outcome Mapping could
be useful in our work
Agenda Overview
Key concepts in Outcome Mapping
Acknowledgements
This presentation makes use of various materials that were shared by members of the global OM community. Without being exhaustive, special thanks goes to Terry Smutylo, Steff Deprez, Jan Van Ongevalle, Robert Chipimbi, Daniel Roduner, Kaia Ambrose and many others.
Source: A guide for project M&E: IFAD
TimeSource: Ricardo Wilson-Grau (inspired by Jeff Conklin)
Conventional thinking…
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Time
… clashes with relationships of cause and effect that are unknown
Source: Ricardo Wilson-Grau (inspired by Jeff Conklin)
ACTIVITIES
OUTPUTS
OUTCOMES
IMPACT
INPUTS
Social change can be…
• Complex: involve a confluence of actors and factors
• Unstable: independent of project duration
• Non-linear: unexpected, emergent, discontinuous
• Two-way: intervention may change
• Beyond control: but subject to influence
• Incremental, cumulative: watersheds & tipping points
Source: Terry Smutylo
OUTPUT
OUTCOME
ACTIVITY
INPUTSACTIVITY
INPUTS
ACTIVITY
INPUTS
OUTPUT
OUTPUT
ACTIVITY
OUTPUT
OUTPUT
OUTCOME
OUTCOME
OUTCOME
OUTCOME
OUTCOME
OUTCOME
Source: Ricardo Wilson-GrauTime
INPUTS
Evaluation challenges1. Establishing cause & effect in open systems
2. Measuring what did not happen
3. Reporting on emerging objectives
4. Justify continuing “successful” interventions
5. Timing – when to evaluate
6. Encouraging iterative learning among partners
7. Clarifying values
8. Working in ‘insecure’ situations
Source: Terry Smutylo
Brief definition of OM
• A participatory method for planning, monitoring and evaluation
• Focused on changes in behaviour of those with whom the project or program works
• Oriented towards social & organizational learning
OM’s answer
Source: Terry Smutylo / OM Lab 2012
Support people to build their own well-being
Enable interventions to adapt as they engage
Apply a systems understanding
Start from observable behaviour change
Recognise that all interventions have limited influence
Embrace different perspectives
“Being attentive along he journey is as important as, and critical to, ariving at the destination ”
Michael Quinn Patton
Three key concepts in OM:
1. Sphere of influence
2. Boundary Partners
3. Outcomes understood as changes in behaviour
There is a limit to our influence
ProjectPeople the project works with/through
People who benefit from the project
Sphere of control Sphere of concern
There is a limit to our influence
Inputs, activities, outputs
Outcomes: Changes in behavior
Impact: Changes in state
Sphere of control Sphere of concern
Participatory research on demonstration farms to develop approaches to drip irrigation
Farmers participate in field trials
Participating farmers learn how to use drip irrigation equipment
Extension workers visit demonstration farms
Training of extension workers
Publication of performance of different set-ups
Increased knowledge of techniques
Extension workers promoting drip irrigation
Farmers adopting drip irrigation methods
Reduced numbers of new wells
Greater quantities of groundwater available
Source: Terry Smutylo
Who are your boundary partners?
Programme
Beneficiaries Stakeholders Boundary Partners
Focus of Outcome Mapping
Outcome Mapping
Community ownership
increases
Program influence decreases
Inputs Activities Outputs Outcomes Impacts
✓OM is a guide to the journey we take with our partners. We co-create the map.
✓It focuses on the intention and what happens along the way
✓The map is not the territory, it shows the route taken
✓“The only real voyage of discovery exists, not in seeing new landscapes, but in having new eyes” (Marcel Proust)
Where is the map?
Step 1: Vision
improved human, social, & environmental wellbeing
I have a dream!
Martin Luther King, Jr.August 28, 1963
A vision statement..• Guides• Motivates and inspires• Is an ‘accountability-free zone’
37
Vision in graphic form, Nagaland (India)
Vision statement in narrative form
Local authorities, communities, and international organizations in developing
countries in Africa recognize the value of HIV/AIDS intervention as an integral part of
social & economic development. Municipal, regional, and national governments
actively support HIV/AIDS prevention activities by formulating and implementing
effective public health policies. Using research findings, they have developed a
comprehensive public health strategy to slow down the infection rate. Formerly
marginalized groups (e.g. women and youth) are organized into advocacy groups that
can effectively formulate their needs to policy makers. All groups have access to
reliable and relevant technical information about HIV/AIDS prevention and are able to
make informed choices. In essence, there are healthier, happier, and wealthier
communities.
Vision facilitation questionImagine that, 5-10 years from now, the program has been extremely successful. Things have improved beyond your most ambitious dreams.
• What changes have occurred? • What (& how) are your intended beneficiaries doing?• What are your partners doing? • Describe the better world you are seeking.
Step 2: Mission
The mission is that “bite” of the vision statement on which the program is going to focus.
• How the program intends to apply its resources in support of the vision
• The areas in which the program intends to work
• How the program will support the achievement of outcomes by its direct partners
A mission statement describes:
Example Mission Statement
In support of this vision and on behalf of its donors, the program will work in the areas of
research, dissemination, capacity building, & coordination. It will contribute to the production,
synthesis, & dissemination of research data, position papers, & other information that will
sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the
range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in
order to produce credible information for local, national, & international policy-making &
program development. It will promote an interest in HIV/AIDS research among new researchers
by providing research fellowships, mentorship, & training opportunities. The program will
contribute to the development of linkages between Northern & Southern researchers &
encourage partnerships between research organizations, advocates, & decision makers. It will
increase its visibility & credibility among the donor community & will convince them of the utility
of supporting HIV/AIDS prevention.
Summary
Vision✓ About the future
✓ Observable
✓ Idealistic
✓ Not about the program
Mission✓ Feasible
✓ Identifies activities and relationships
✓ About the program
Exercise : Correcting Vision and Mission Statements
1. Across rural India, women and girls are contributing to and sharing equitably in the benefits of development.
2. They utilize and benefit from appropriate health care, education, food and water security.
3. They are free from violence in the home and in the community.
4. The project seeks to improve their well being through a wide range of interventions.
5. Women are able to earn their own livelihoods, accessing freely the markets, credit, banking and municipal services they need to pursue their economic goals.
6. Villages are fully served by public transport, are well lit at night and have police forces that enforce all laws fully and equitably.
7. The consortium of donors responds favorably to the project’s progress reports and decides to continue funding for a second phase
1. The Swayamsiddha Project works with government, non-government and community-based organizations to strengthen their individual and combined efforts towards improving women’s health and empowerment.
2. The development of women’s self help groups is facilitated by the project, providing women with moral, material and educational support in identifying and taking action to make community and government services more responsive to their health and livelihood needs.
3. Primary and reproductive health care at the local level is directly accountable for, and responsive to, the needs of women and girls.
4. Enrolment of girls in primary schools increases by 35% as a result of project interventions
5. Research is undertaken in the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services.
Step 3: Boundary Partners
Step 3: Boundary partners
Outcome mapping focuses on outcomes as changes in behaviour or relationships of those with whom the program or project is working directly (= boundary partners)
Boundary Partners are...
Those individuals, groups, & organizations with whom a program:•interacts directly to encourage change as a contribution to the vision.•can anticipate some opportunities for influence•engages in mutual learning
Who are your boundary partners?
Programme
Beneficiaries Stakeholders Boundary PartnersStrategic partners
CIDA
IDRC
BAIF
StateNGO
StateNGO
StateNGO
StateNGO
StateNGO
StateNGO
SHG Police CommunityLeaders
Families Banks PHCs
Swayamsiddha
Boundary partners have their own boundary partners - Often many actors and results at different levels
Facilitation questions• In which individuals, groups, or organizations is
your program trying to encourage change as a contribution to the vision?
• With whom will you work directly? • Are you choosing boundary partners because
you want to influence the ways they help or influence others?
• On whose actions does your success depend?
Strategic partners• Selected for their potential to contribute to the
mission
• A person or group with whom the program works directly to achieve the mission, without necessarily wanting to change the partner’s behaviour as part of the mission
• E.g. Donor agency, contracted service providers, NGOs doing similar work, media agencies
Facilitation questions• In which individuals, groups, or organizations is
your program trying to encourage change as a contribution to the vision?
• With whom will you work directly? • Are you choosing boundary partners because
you want to influence the ways they help or influence others?
• On whose actions does your success depend?
Screening of boundary partnersA program has normally not more than 4 or 5 types of boundary partners
•What if too many BP’s? What if you are not sure whether it is really a BP?
– Some boundary partners might be boundary partners of boundary partners!
– Discuss how the boundary partners are possibly contributing to the vision
– Can we realistically influence the boundary partners?– Where will we put most efforts and resources?– Sometimes similar actors can be clustered as one type of boundary
partners– Who are your ultimate beneficiaries / target group? There are not your
boundary partners– Maybe some partners are actually strategic partners?
Exercise : Boundary partners
Boundary partners: exercise
• Read the handout with the case of the VVOB teacher education and vulnerability programme
• On the basis of the case of the VVOB programme (see handout), try to position the various actors in (or out) of the various influence spheres on the flip chart provided.
• Solution: see handout!
20-04-23 59
Developing an actor focused TOC (case of VVOB Zimbabwe)
Inputs, activities, outputs
Sphere of control Sphere of interest
Final beneficiaries
PupilsPupils
College adminsCollege admins
Student peer educator clubsStudent peer
educator clubs
Student support structures
Student support structures
Boundary partners = Direct target groupsBoundary partners = Direct target groups
Staff devpt committeesStaff devpt committees
Earch Childhood depts
Earch Childhood depts
VVOB support
team
VVOB support
team
College studentsCollege
students
College LecturersCollege
Lecturers
Indirect target groupsIndirect target groups
Outcomes Impact
UnicefUnicef
DGDDGD
PLAN International
PLAN International
Local resource persons
Local resource persons
Step 4: Outcome Challenge
Outcome Challenge• Describes behaviour of a single boundary
partner
• Sets out the ideal actions, relationships activities
• Describes the boundary partner’s contribution to the vision
Facilitation questions• Ideally, in order to contribute to the vision,
what would the boundary partner be doing?
• With whom would they be interacting?
• How could this boundary partner contribute maximally to the vision?
Women’s self help groups are taking action to make community and
government services more responsive to the health and livelihood needs of
women and girls.
They influence banks, police, health and social service providers, local
officials and state and national government agencies in relationships of
mutual respect and joint action to improve women’s well being.
Women’s self help groups arrange bank loans for members and for life skills
training for girls to be included in the school curriculum.
They influence local, state and national government policies and
expenditures on community improvement and transportation and support
women candidates to run for election to local government office.
Step 5: Progress Markers
(Deep transformation)
(Active engagement)
(Early positive responses)
Love to see
Like to see
Expect to see
Progress Markers
Love to see
Like to see
Like to see
Expect to see
Like to see
Like to see
Expect to see
Like to see
Like to see
Expect to see
Why Graduated Progress Markers?
• Articulate the complexity of the change process
• Allow negotiation of expectations between the program and its partners
• Permit early assessment of progress
• Encourage the program to seek the most profound transformation possible
• Help identify mid-course improvements
?
Greater awareness…Empowered women…Community ownership…Reduced conflict…Increased collaboration…Governmental commitment…Gender sensitivity…Equal access…Budgetary transparency…Active participation… Poverty alleviation…Strengthened capacity…
How can we measure:
Examples• Project expects to see Tanzania Forest Service
– Consult with communities on the new forest policy and the national REDD strategy;
– Develop management plans for all Nature Reserves;– Continue a policy that is supportive of participatory
forest management.
Examples• Project would like to see Tanzania Forest Service
– respond promptly to information from communities about illegal activities and other forest management issues and take action to resolve the issue.
– incorporate recommendations from communities and civil society organistions in the revised forest policy and the national REDD strategy;
– approve voluntary social and environmental timber standards;– implement and monitor management plans for nature reserves
and other high biodiversity forest reserves with the participation of communities;
– approve independent forest monitoring and cooperate fully in its implementation;
– incorporated Biodiversity indicators in NAFORMA
Examples• Project would love to see Tanzania Forest Service
– successfully advocate for wider application of good timber standards throughout Tz gov’t systems.
– consistently and effectively apply the Forest Act and regulations within the timber and charcoal trade thereby increasing the revenue that they capture and reducing illegal timber harvesting and charcoal production;
– incorporated social and environmental standards in revised harvesting regulations and other documents related to the national forest programme;
– Applying good social and environmental practices in respect of reserve management
Facilitation questions• How can the programme know the boundary
partner is moving toward the outcome?
• What would they be doing?
• What milestones would be reached as the boundary partner moves towards their intended role in contributing to the vision?
Progress Marker ChecklistEach Progress Marker:Describes a changed behaviour by the boundary
partnerCan be monitored & observed
As a set, Progress Markers:Are graduated from preliminary to more profound
changes in behaviourDescribe the change process of a single boundary
partner
Exercise : Arranging Progress Markers
Step 6: Strategy Maps
Strategy Maps
For each Boundary partner:• What will the Project do to SUPPORT desired changes in the
Partner towards the Outcome Challenge?
• Diverse range (an array) of ACTIVITIES:
- Aimed at Partner
- Aimed at Partner’s environment
- Cause, Persuade, Support
Strategy Map
causalpersuasiv
esupportiv
e
Partner
Environment
6 types of support strategies
causal persuasive supportive
Iaimed at individual boundary
partner
directInfluence
(e.g. funding, prepare a report, …)
arouse new thinking;
build skills, capacity
(e.g. skill enhancement,
methodological workshops, training)
on-going support
supporter/mentor who guides change over
time,involvement is more frequent and sustained,
nurturing for self-sufficiency
(e.g. program member who provides regular guidance and input, coaching, expert, …)
Eaimed at boundary partner’s
environment
alter the physical or regulatory
environment(e.g. technical transfer, policy change, internet access, terms of reference, rules,
guidelines)
broad information
dissemination/access to new
info(e.g. Radio, TV,
website, publications, conferences, workshops)
create / strengthen
peer networkspartners working
together and collectively
supporting each other)
(e.g. research network, …)
Example strategy map: for extension agents
80
Causal Persuasive Supportive
Directed to the boundary partner
Provide research and training funds
Run technical training workshops for extension agencies and activities (e.g. ISFM)
Coordinate training on farmer participatory action research approaches to extension agencies, service providers and research teams
Conduct training on protocols development and project monitoring
Contract a farmer-training specialist to work with extension on farmer mobilization, action research, data collection and documentation
Conduct quarterly monitoring and review workshops
Directed to the environment of the boundary partner
• Building model storage facility for grain and backstopping
Develop extension materials and policy brief on adaptation to climate change
Facilitate establishment of exchange forums for sharing of information, knowledge and experience among partners
Establish a network of extension agents and researchers that facilitate knowledge and information sharing
Engage researchers and policy makers as conveners for discussions on climate change impact and alternative measures
Exercise : Strategy Map
Exercise on strategy mapsInstructions
•Your table will be given a number of cards with activities written on them. Discuss which type of strategy each of the activities represents. The Outcome Challenge has been provided so that you are able to think about what the program is contributing to.
•Select someone from your table to plot the activities in the appropriate section of the strategy map on the wall. Be prepared to explain your reasoning for plotting the different strategies as I-1, I-2, I-3, E-1, E-2, or E-3.
•!Remember, a strategy map describes what the program will do to try to encourage change(s) in a boundary partner. It breaks down depending on whether they are aimed at the individual boundary partner or the environment and whether they are causal, persuasive, or supportive.
Time: 30 minutes
mission vision
Girls & Women
Community Leaders
Women’s Self Help Groups
Families
PoliceState NGOs
Banks
Public Health Clinics
Strategic Partners
Strategies Project’s OutcomesBoundary
PartnersBP’s outcomes
BAIF
Step 7: Organisational Practices
Organizational Practices
How does your team or organization stay relevant, viable and effective?
8 practices1. Prospecting for new ideas, opportunities, and
resources
2. Seeking feedback from key informants
3. Obtaining the support of your next highest power
4. Assessing and (re)designing products, services, systems, and procedures
8 practices5. Checking up on those already served to add
value
6. Sharing your best wisdom with the world
7. Experimenting to remain innovative
8. Engaging in organizational reflection
M&E planning and journals
5 kinds of M&E Information
Program Partner
outcomes(behaviour changes in the partners)
implementation(interventions by the program)
relevance & viability(actions of the program)
C o n t e x t u a l I n f o r m a t I o n
State, status or situational data
Strategies
M&E Planning Worksheet
What information will be collected?
How will it be collected, from what sources and when?
Who will collect it?
Who will make sense of the info & how?
Who will manage the collection-to- usage processes?
What resources are needed & who contributes?
Outcome journal• What, when, who, how much• Contributing factors• Sources of evidence• Unanticipated change• Lessons, required changes, action
Strategy journal• What did you do, with whom, when• (How) did it influence change• Outputs• Suggested changes/follow up• Lessons
Performance journal• Progress against internal commitments – e.g.
organisational practices• Reviewing the logic of the programme
Journal for Monitoring OutcomesOutcome Monitoring Journal
Period of monitoring and data collection: Contributors to Monitoring Update: Name of the person(s) who compiled the journal: Outcome Challenge:
Based on your analysis of each progress marker, please indicate the number on the scale that best represents the boundary partner's progress in achieving the progress marker (1 indicates no activity towards the progress marker while 7 represents full achievement of the progress marker) Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted Progress markers What happened (who, how,
what, where) Date (when)
Strategy Used and its Efectiveness
Follow up/corrective measures Evidence (documents etc.)
1)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
2)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
3)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
4)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
5)
Not started yet/no activity towards progress marker 1 2 3 4 5 6 7 Progress marker reached/strong activity towards progress marker noted
Name of the boundary partner
Work dating from/to:
Name(s) of the person(s) who compiled the journal:
Outcome Challenge:.
Progress Markers Remarkable facts, what happened
Follow up / corrective measures
Narrative of a success story:
Unanticipated changes :
Which support strategies from the ICCO alliance where helpful or require further follow-up or action during the next term?
Contributing or limiting factors and actors towards achievement of progress markers: :
Summary of lessons learned/recommendations :
Outcome Journal
Thank you!For more info on OM:
•Visit the OM learning community:www.outcomemapping.ca
•Contact:Simon Hearns.hearn@odi.org.uk
Sharing information to improve evaluation
outcomemapping.ca