Increasing development impact niarobi nov 2016 final

Increasing development impactIDIAR Course

CSIRO AGRICULTURE AND FOOD BUSINESS UNIT & LAND AND WATER BUSINESS UNIT

Michaela Cosijn (International Development Research Scientist) & Jennifer Kelly (Senior Innovation Broker)2-3 November 2016

Day 1Innovation and impact at scale

Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly2 |

Objectives


Understand …

i. the nature of innovation as a wider process than research

ii. that innovation takes different forms

iii. that there are different ways of organizing innovation each with its own set of tools

iv. the rationale for using multi-stakeholder platforms for innovation, including

innovation platforms

v. how & when to use innovation platforms, including what works well and typical

challenges

Session 1Partnerships and platforms: Different ways of organizing agricultural innovation


But first … What is innovation?

• It is the creative use of new or existing ideas, technologies or ways of

doing things in response to social and economic needs and opportunities.

Innovation can only thrive in a sound institutional environment.

• It is not research or technology, but might involve both.


Innovation vs innovations

• Invention or innovations is about the creation of new knowledge

new to the world, usually by research organisations, but also by

artisans and others

• Innovation on the other hand, is about the use of the

knowledge.

• For example, the knowledge is novel for a farmer, firm, neighbour

and/or competitor, but not necessarily new globally


So what does innovation look like - Types of innovation?

• Technological innovation: farmers adopting a new crop variety, a

new agronomic practice, or animal feeding regime.

• Organizational innovation: farmers work collectively to market

produce.

• Institutional innovation: researchers form new partnerships with

farmers and companies to deliver solutions that give income and

profits

7 | Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly

Types of innovation continued

• Business innovation: companies develop new products and service

or new ways of delivering these that create profit and other value.

• Value chain innovation: Value chain actors use new ways to

procure, add value or market products.

• Policy innovation: regulations, rules and incentives that add value

to social and economic activity. Food standards, pesticide

approvals, new ways of financing farming and business investment.


Why is this important …

Why do I need to know about innovation

I am a researcher and interested in

research / creating knowledge?


Changing approaches to investing in innovation capacity

Early 1980s and beyond -

•Bricks and mortar. The period before the mid-1980s emphasized expanding public sector research by investing in physical infrastructure, equipment, and human resource development. In many cases the investments created centralized national agricultural research systems (NARS).

Late 1980s

•Management systems. From the late 1980s the emphasis shifted to improving the management of existing public sector research organizations through better planning, improved financial management, greater accountability, and increasing the relevance of programs to clients

Mid- to Late1990s

•Down to the grassroots. In the mid- to late 1990s, the instability and inefficiency evident in many public research organizations led to an emphasis on development of pluralistic agricultural knowledge and information systems (AKISs) with greater client participation and financing.

Current

•Innovation systems. More recently, the Bank’s approach has moved towards the concept of “agricultural innovation systems” (AIS) and focuses on strengthening the broad spectrum of science and technology activity of organizations, enterprises, and individuals that demand and supply knowledge and technologies and the rules and mechanisms by which these different agents interact.

Source: The International Bank for Reconstruction and Development / The World Bank (2006). Enhancing Agricultural Innovation: How to Go Beyond the Strengthening of Research Systems


Changing conditions …

Research organisations have / are experiencing …

• Research funders expecting results / outcome orientated projects

• Poor uptake of technology and weak demand-orientation research

The agriculture sector more broadly is facing an…

• Increasingly complex food & nutritional security needs, environment requirements, poverty reduction, and changing consumer demands

• New players and more prominence for the private sector

• Improved understanding of how ideas and technology come into use


Research into Use (RIU) - Findings by Hall (2011)

• As few as 1-2% of 1600 research projects failed to result in innovation and

impact if they relied solely on technology transfer or new technology as the

starting point

• Implementation leads to new research questions which are often about

political and institutional contexts

• Research plays a different role at different times on the innovation trajectory.

• Research use is part of a wider process of innovation


Research into Use (RIU) - Findings by Hall (2011)

• Innovation emerges from networks of interacting players and associated

policy and institutional developments that support chains of actors

• RIU needs development of innovation capacity.

• Involves linking different organisations, tackling policy bottlenecks and

creating policy and institutional conditions that enable innovation process

and make them more responsive to economic, social and environmental

ambitions of society


So what does this mean …?

There is a “knowledge market failure” in creating

conditions where different stakeholders can:

• Exchange information.

• Identify new solutions and opportunities.

• Align activities

• Address systems challenges beyond the scale of their

current activities (e.g. farmers, small businesses)


How to address this failure?

15 |

• Depends on the nature of the problem or opportunity being addressed.

• Development and testing of new maize varieties: Simple product development, markets and policy systems exist few stakeholders. Finance issues with scale out. More like a partnership or simple research and farmer seed company innovation platform.

• Creating aflatoxin free food systems in East Africa: Highly complex, requires technical change, new market and policy systems, behavioral change

among many and diverse stakeholders. More like a combination of partnerships, alliances, farm level and policy level platforms. Requires a

range of multi-stakeholder process over many years.

Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly

But what can I do?

There are broadly 2 views for organizing for innovation:

A technology transfer pipeline view

A “systems” views involves different types of innovation

coupled together through a network of actors (individuals

or groups) that interact to produce, share and use

knowledge


Neither is universally correctThere is no blueprint for innovation.Each approach needs to be tailored to task at hand.

17 •

Agricultural Research Partnerships

Agricultural innovation delivery partnerships

National Agri-food systems innovation partnerships

Global development innovation partnerships

Agricultural research organizations collaborate to develop new knowledge on discreet technical dimensions of prioritized problems and opportunities.

Agricultural research organizations collaborate in agricultural production and agribusiness innovation that delivers new products and services that create value for farmers and companies.

Agricultural research organizations participate in the efforts of public policy and private sector to catalyse innovation in agri-food systems that creates social, economic, and environmental value in line with national development plans.

Agricultural research organizations participate in efforts of national and global public and private sector stakeholders to catalyse innovation in economic and social systems to achieve social, economic, and environmental development targets set by the SDGs.

Mode 1: Mode 3:Mode 2: Mode 4:

SolutionsLocal

ImpactsFood

Systems Impact

Pervasive change

Solutions

Local Impacts

Food Systems Impact

Long term enduring impacts at global scale

Long term, but enduring impacts at value chain or national scales

Quick wins, but restricted to scale of project, mission or commercial opportunity

Dependent on linkages to other delivery, innovation and societal change processes


Source: ISPC, 2015. Strategic study of good practice in AR4D partnership. Rome, Italy. CGIAR Independent Science and Partnership Council (ISPC), viii + 39pp + annex 49pp

Trends in organising for innovation

Presenter

Presentation Notes

Examples Mode 1: Pest resistance, analytical f/works, models & platform technologies Mode 2: Agricultural productivity/ business competitiveness Mode 3: Food security/poverty reduction/economic growth Mode 4 Development challenges framed by SDGs

Which mode of Innovation Platforms fit ?


Agricultural Research Partnerships

Agricultural innovation delivery partnerships

National Agri-food systems innovation partnerships

Global development innovation partnerships

Mode 1: Mode 3:Mode 2: Mode 4:

SolutionsLocal

ImpactsFood

Systems Impact

Pervasive change

Solutions

Local Impacts

Food Systems Impact

Presenter

Presentation Notes

Question Who think innovation platforms fit into mode 1 .. 2… etc

What is your experience with innovation multi-stakeholder partnership / platforms?

- who is involved?- benefits?

- challenges?- how could it be done differently?


Presenter

Presentation Notes

Pick a group based on which model you think you work in … In groups discuss … benefits, challenges, who is involved, how you could work differently 10 min discussion 10 min feedback

What is an Innovation Platforms (IPs)

It is a partnership innovation mechanism

It is a

• space where actors can come together and interact including diagnosing

problems, identify opportunities and find ways to achieve their goals

• space for learning and creating change

An IP may act as a mechanism for designing and implementing activities, or a

mechanism for coordinate activities designed by individual members


An Innovation Platform (IP) is not ...

• A fixed method, approach or specific process• It has to include changes of skills, mindsets and attitudes, organisationalpractices and cultures of partners, as well as the ways in which the partners

interact as part of the wider “innovation system”

• An everlasting interaction that needs to be facilitated by researchers = an IP can be dissolved if issues are addressed …

• A solution looking for a problem


Presenter

Presentation Notes

I think that the last bit is also important – we talk about sustainability because our projects have most of the time been short = 2-3 year. If you have a longer collaboration there might be a point in which the IP is dissolved or its focus changes because new issues emerge.

What an IP involves?

1. Bring together different ideas and resources from different sources

2. Aligning related activities, e.g. research and farming, policy making and

business objectives

3. Creating conditions (systems) for using ideas: changing markets and

business systems; changing the way organisations and individuals work;

changing policy setting; creating new connections in the system.


What an IP involves continued


• Vertically Iinkages – across policy levels

• Horizontally linkages - with other platforms at the same level)

Source: Tucker, J., Schut, M. and Klerkx, L.. 2013. Linking action at different levels through innovation platforms. Innovation platforms practice brief 9, Nov 2013

7 Stages of an IP

Source: Homann-Kee Tui, S., Adekunle, A., Lundy, M., Tucker, J., Birachi , E., Schut, M., Klerkx, L., Ballantyne, P.Alan Duncan, A. , adilhon, J. and Paul Mundy, P.(2013) What are innovation platforms? Innovation platforms practice brief 1, November 2013


Brainstorm

• Why establish an innovation platform?


Presenter

Presentation Notes

5 minutes of brainstorm

Why have an IP?• Capacity development

• Identify need-based capacity building of actors• Dialogue• Reflection• Cross-learning

• Allows innovation research• Identification of research issues (demand-driven)• Disseminate research outputs• Action research and learning by platform members

• Better communication & decision-making• Facilitate upward communication – creating spaces for weaker partners to have a voice & ability to

negotiate• Better informed decision-making• Facilitate dialogue and understanding amongst stakeholders


Why have an IP continued

Creates ownership, buy-in and motivation

• Identify and create shared goals and interests in the value chain actors,

opportunities, common problems and bottlenecks, and solutions

• Use understanding of value chain to identify upgrading / scaling options

– including technical, organizational, institutional, service delivery and

policy innovations

• Define activities and actions and, roles and responsibilities of various

actors in implementation of agreed options for value chain improvement


Why have an IP continued

• Allows for enhanced impact improved market functioning, agricultural productivity

etc Implementation and scale up if interventions are

successful.

• Allows for processes for monitoring actions for upgrading / scaling

• Create spaces for long-term learning processes from experiences through iterative action-reflection-learning cycles that support innovation

They are particularly useful in complex

systems with multi-stakeholders e.g.

agriculture, environmental


How research can contribute to IP’s


Source: Lema, Z. and Schut, M. (2013) Research and innovation platforms. Innovation platforms practice brief 3, November 2013

Network buildingDeveloping adaptive capacity for innovationAddressing institutional constraints and power asymmetries between stakeholdersReflective monitoring, evaluation and strategic adjustment

Facilitating joint knowledge production and learningKnowledge sharingProcess documentationInternal and external communicationAction research and joint monitoring and evaluationTranslating and packaging of research outcomes

Production of authoritative, objective and value-free knowledgeBaseline studies, impact evaluation

Traditional Research

Knowledge management and joint action research

Create an enabling environment for innovation

Researchers as an Innovation Brokers

• Researchers are often seen as neutral

• They be external or internal

• Researchers often understand the whole picture and interact with all stakeholders

• Play a number of roles - capacity building, facilitation, carrying risk, seeding ideas,

inspiration, active collaboration, building relationships, facilitates learning & exchange &

action


Questions



Morning coffee break

Session 2An Innovation Platform in Practice:The Case of IMGoats Innovation Platforms in Mozambique & India


Acknowledgements


The IMGoats project


• Funded by EC through IFAD

• Started January 2011 and ended June 2013 (30 months)

• International Livestock Research Institute (ILRI) was the research institution coordinating the

project.

• CARE (NGO) implemented in Mozambique building on an existing project funded by CIDA

• Bharatiya Agro Industry Foundation (BAIF) (NGO) implemented in India

Its aim was to transform small holder goat production and marketing to a sound and profitable enterprise and model that taps into a growing market.

Innovation Platforms were the key innovation mechanism used

35 |

Project locations

Need map of Indian sites



Topic Udaipur district, Rajasthan State– India*

Inhassoro district, Inhambane Province– Mozambique

Population density 196/km2 11/km2Participating households 1000 524Literacy levels 58.62% 51%Average annual rainfall 600mm 600-800mmLivelihoods Small land and livestock holdings (subsistence

agriculture); wage labour important source ofincome

Small land and livestock holdings (subsistenceagriculture); crop production main occupation;cattle numbers very low

Main crops Maize, wheat, barley, chickpea, rape andmustard

Maize, groundnuts, beans, cassava, millet

Average goat herd size 6.2 (range 1-16) 8.4 (range 1-30)

Marketing practices During main festive period (October to December) and ad hoc throughout the year to meet household demands

During festive period (December) and ad hoc throughout the year to meet household demands

Nearest goat market 50Km (Udaipur) 200Km (Massinga)

Main goat value chainconstraints

Lack of improved bucks; limited access toanimal health services; low number of goatsavailable for sale; limited knowledge aboutimproved husbandry practices

Low number of goats; limited access toanimal health services; lack of organizationof producers; lack of infrastructure; limitedknowledge about improved husbandryPractices

Main value chain actors Producers; CAHWs; local traders/butchers;long distance traders; local pharmacist; AnimalHusbandry Department; BAIF; research (ILRI,veterinary college)**

Producers; CAHWs; local traders/butchers;local retailers; District (SDAE) and Provincial(SPP) Veterinary Services; CARE; research(ILRI)**

Purpose of the IPs Overarching purpose

Establishment of Innovation Platforms for reducing poverty and increasing food security in dryland areas of India and Mozambique to increase incomes and food security in a sustainable manner by enhancing pro-poor small ruminant value chains in India and Mozambique

Key aspects:• Increasing production of goats – both quality (health & weight) and

quantity• Increasing access to markets


IMGoats IPs supported innovation for …


Producers – poor quality & quantity of goats, few

buyers

Buyers – no secure suppliers

Innovation platform

Solution: improve production & market construction

Producers receive better prices & invest in production. Buyers access good animals.

Types of innovation the IP supported include …• In Mozambique

• Technological– Animal production

– Improved kraals– Improved animal husbandry– Improved animal production techniques including

Animal husbandry service delivered by CAHAW• Organisational

– Retailers supply veterinary inputs to CAHAW• Institutional

– Introduction of use of weighing scales and live weight pricing to guarantee fair price

• A combo Tech / Org and Institutional– 12 Communal pastures focusing on

• Guidelines / training on sustainable management based on local situation

• Legal demarcation of the area in process by government• Community management (ie water, herding, security and

fire management


• In India• Technological

– Animal health service delivery by CAHAW– Alternative feeds and new feeding techniques– Better breading practices (eg improved make goats

availability, castrating inferior male goats, etc)• Organisational

– Aggregation of animals by CAHWs for selling– Organisation of goat fairs and exploration of new

markets– Organisation of health camps for vaccination

• Institutional– Linkages with government, agricultural training

institute through a series of trainings on good practice

– New ways of collaborating between CAHWs and veterinary services for faecal sample testing, including a linkage with the regional disease diagnostic laboratory

Other key areas of innovation process in both countriesInstitutional• Coordination in implementation with government to ensure

sustainability

Organisational• Group formation and strengthening (elections,

accountability, advantages of working in groups) -

Mainstreaming• Focus on gender and vulnerable households (FHH, PLWHA) • Environmental assessment


Exercise:Role Play


Exercise: Instructions1. Form groups of 7-8

2. At least one person in your group has been assigned a role, each with a specific agenda which is

unknown to the other characters.

3. Spend 5-10 minutes individually reminding yourselves about the case we discussed before lunch

and reading the handouts (characters for the role play)

4. When all the members of your group have finished reading their roles the person assigned as the

project leader will start the role play by inviting your characters to a meeting to set up the IP.

5. You will have 45 minutes for the role play.

6. At the end of the 45 minutes we will break and debrief as a larger group.


Exercise: Role Play characters and tips

A representatives from

- The Government

- The NGO

- The private sector

- Trader

- Retailer

- The Farmers & Paravets

- The research organsiation


What your group needs to set up /

Agree on ...

− Purpose of IP

− Members of IP

− How often IP will meet?

− Who will lead the IP?

− What are the key challenges that

will be addressed

What worked well?

What challenges did you encounter?

How did you overcome them?

What lessons would you take forward?

Exercise: Debrief


IP Formation• The IP formation process was inclusive; including producer groups

• Project partners conceptualised vision & objectives, challenges & opportunities;

• Potential tasks identified

• Assessment of knowledge/skills among producers and CAHWs was thoroughly explored

• Roles of some VC actors in the innovation process were insufficiently explored.

• Problem identification was participatory with a focus on production and marketing -> linked to key

constraints.

• Project partners lead in facilitation and management; mechanisms were established to hand-over

• Project funded initial resources.


IP Management • Participation varied across the VC actors.

• Information flow from platform to producer groups was good, but weak inversely

• CAHWs formed an important link with producers.

• The IP tapped into the knowledge/skills of some VC actors, especially India;

• Problem solving followed a systematic innovation process (technological,

organizational, and institutional elements);

• Some interventions were highly predictable, others not (flexible planning);

• In Mozambique, there was a stronger reflection on the IP as an institutional innovation itself. The key

management structure was secretariat.


IP Management • Capacity building through training and exposure/exchange visits (focus was on producers and

CAHWs)

• IP meetings = capacity building through systematic reflection.

• Innovation brokering included multiple diverse tasks;

• Facilitation was gradually handed over to local actors, but project

partners continued to play an important role.

• IPs were (human) resource intensive; including:

• extra efforts to get endorsement and support from community leaders and producer groups

• creating strategic linkages with government agencies.


Actors in the IPs and their changing roles


Agricultural innovation platforms challenges

• Usually established as community level IPs

• Disconnected from platforms and other groups at higher scales

• Impacts are at local scales and often restricted to project cycle funding

• Emphasis where impact needs to happen and this is a key operational interface

• Without any link to higher-level groupings: Little scope for tackling overarching policy and

institutional constraints or aligning with longer-term (and wider-scale) development goals

and plans.

50 |

“Pilots never fail, but never scale” WHY?


General challenges and lessons learnt• Buy-in & trust for success

• Inclusion and representation (women, private sector, contextual factors –language, illiteracy)

• Incentives for participation (demand for goats, existing networks)• Dynamic nature of participation• Power dynamics• Generating tangible benefits

• Capacity building is key• Creating experiential learning (CAHW, cross-visits)• Building skills in management structure & facilitation• Reflection & discussions• Building skills with in NGO’s• Private sector skill development – neglected! – assumed skills were there…


Challenges are evolutionary,

continuous, always changing, and

integrated.

• Technical innovation • not ‘new’ but supported by organisational changes (producer

groups, communal grazing areas, health camps) & existing institutions (legislation, fairs)

• Facilitation and management• IP’s are complex and sometimes political• As a result, they can be costly and time-consuming to

implement• Who should facilitate• When to start and end an IP• Who should finance what?

Key message –

Short cuts are risky – IP’s are

mechanisms for promoting

systems thinking not only

forums for technology

transfer and dissemination

General challenges and lessons learnt continued


M&E can be difficult

3 aspects to monitor –

• activities

• process changes (knowledge, attitude and practice)

• impacts on the poor

• time lags between activities and impact hard to measure

(e.g. Capacity building, communication benefit,

complexity of stakeholders, complexity of measuring

behaviour changes)


General challenges and lessons learnt continued

53 |

Questions



Lunch break

Session 3Partnering:Tips and tricks


Key aspects• No single partnership is like any other.

• All partnerships are highly context---specific and reflect the surrounding circumstances.

• Sometimes it is necessary to settle for a ‘good enough’ partnership

• Until the local conditions change.• [This is not an excuse for sloppy partnering!]

• Often the partnership itself can help to influence and bring about change.

Challenges to partnerships

Why do we need the 3 core principles

What makes successful partnerships

… each sector plays its appropriate part and contributes from core competencies and strengths

Good partnership practice• WORKING FROM FACTS

• BREAK-THROUGH NOT BREAK-DOWN• REQUESTING VS. COMPLAINING

• CREATING QUALITY PARTNERING CONVERSATIONS• MANAGING MEETINGS WELL

• KEEPING RECORDS• CREATING A LEARNING CULTURE

• SETTING GROUND RULES

• REMEMBER

• Golden rule l - BUILD ON SHARED VALUES

(because successful partnerships are values-driven)

• Golden rule 2 - BE CREATIVE

(because every partnership is unique)

• Golden rule 3 - BE COURAGEOUS

(because all partnerships involve risk)

Exercise:What would you do ?


Exercise: ScenarioThree organisations (a development donor, a research organisation and a research funder) form an innovation

partnership to link national and international research expertise with a view to improve impacts and outcomes from

agriculture and food security programs.

The partners did this by working together to create a National Agri-food systems innovation partnership /platform

that could:

1) strengthen the analytical and evidence base for the designing food security interventions,

2) promote the use of effective research and development methods (emphasizing multidisciplinary and systems

approaches), and

3) improve links between research and development practice

The partners had previously worked together in individual partnership, but this was the first time they had attempted

to work together in a tripartite partnership


Exercise: ScenarioSix months into the partnership, the world changed …. An election changed the government resulting in reduced total

funding for international development.

This in term impacted the partnership and the individual partners, including

- development of a new international development strategy that created a void in the policy direction

- number and scale of food security projects were reduced,

- the focus changed from capacity building and partnering at a national level to a project level and

- all three partner organisations restructuring, which resulted in changes of key personal in the partnership

Despite these changes the project continued to operate according to the plan it developed before the changes despite

the fact that the objectives of the partnership were becoming less aligned / priority with the objectives / priorities of the

individual partner organisations.


Exercise: Scenario

… Until the twelve month mark, when an independent review was commissioned to review the progress of the

initiative.

The findings were not positive … The independent reviewer highlighted a number of challenges, including:

- the need to change the direction of the partnership to be in line with the new policy which was developed

- a dysfunctional management structure, including conflicting priorities and misaligned objectives between the

partnership and the partner organisations, and

- a non-tradition mix of skills needed for a non traditional research project.

On receipt of the review findings the partnership steering committee met to discuss option for going forward.


Exercise: Discussion

If you were the steering committee what would you do ????

- How would you get the partnership back on track and focus on impact?


Debrief: What the Steering Committee did ...

The partnership went to marriage counselling … not really but the SC did engaged an independent partnership broker

to facilitate a workshop with all partnership and develop a partnership agreement.

The agreement included

- understanding joint and individual objectives

- developing values and behaviours to work together

- identified and developed a plan to manage risks

- identified what each organisation contributed (resources … beyond cash) to the partnership

Outcome … trust the process … the partnership developed stronger relationships, including trust, mutual respect

and each partner gained a stronger understanding of the other partners organisational requirements (key

ingredients for success)

Partnering tools


The Partnership Cycle

Partnership agreements and reviews

Resource Mapping - Identify and Value Contributions from Partners

Risk ManagementTypes of Risks• Reputation impact - all organisations and

institutions value this• Loss of autonomy - working in collaboration

inevitably means• Conflicts of interest - whether at strategic or

operational levels,• Drain on resources - partnerships typically

require a heavy ‘front• Implementation challenges - once a partnership

is established

Risk Matrix

So in conclusion …When they work well partnerships have the potential to do great things … but there is no

magic pill to make them work well … they need guidance and attention.

Somethings that can help include …Ensuring that :• there are clear decision-making protocols / procedures agreed and in place• do not over burden the partnership with governance

• most day-to-day decisions are carried by individuals or small groups on behalf of the partnership• only major decisions (for example, of policy or expenditure) are brought to the partners as a

whole group• there is regular, easily accessible and succinct information-sharing between the partnersAsking for help

• Resolve conflicts as quickly as possible and preferably internally• Use independent facilitators to help with objectivity

• There are resources dedicated to brokering / managing the partnership

Questions


Useful Resources


Useful Resources ILRI IP resources - Series of 12 briefs on IP’s

https://cgspace.cgiar.org/handle/10568/33667/browse?value=INNOVATION+SYSTEMS&type=ilrisubject

KARI/ACIAR IP http://aciar.gov.au/aifsc/sites/default/files/images/innovation_guide.pdf

Wageningen UR critical issues for reflectionhttps://www.wageningenur.nl/en/Publication-details.htm?publicationId=publication-way-343535383133

Partnership Brokering Association (PBA)https://www.partnershipbrokers.org


https://cgspace.cgiar.org/handle/10568/33667/browse?value=INNOVATION+SYSTEMS&type=ilrisubject

http://aciar.gov.au/aifsc/sites/default/files/images/innovation_guide.pdf

http://aciar.gov.au/aifsc/sites/default/files/images/innovation_guide.pdf

https://www.partnershipbrokers.org/

Papers & ReportsAndy Hall (2011). Putting agricultural research into use: Lessons from contested visions of innovation. UNU merit Working paper Series no #2011-076http://portal.unu.edu/calendar/?go=event.page&id=4244Kees Swaans et al (2014). Operationalizing inclusive innovation: lessons from innovation platformss in livestock value chains in India and Mozambiquehttp://dx.doi.org/10.1080/2157930X.2014.925246The International Bank for Reconstruction and Development / The World Bank (2006). Enhancing Agricultural Innovation: How to Go Beyond the Strengthening of Research Systemshttp://siteresources.worldbank.org/INTARD/Resources/Enhancing_Ag_Innovation.pdfWorld Bank, 2012. Agricultural Innovation Systems: A Investment Sourcebookhttp://siteresources.worldbank.org/INTARD/Resources/335807-1330620492317/9780821386842.pdfISPC, 2015. Strategic study of good practice in AR4D partnership. Rome, Italy. CGIAR Independent Science and Partnership Council (ISPC) Coming soon


http://portal.unu.edu/calendar/?go=event.page&id=4244

http://dx.doi.org/10.1080/2157930X.2014.925246

http://siteresources.worldbank.org/INTARD/Resources/Enhancing_Ag_Innovation.pdf

http://siteresources.worldbank.org/INTARD/Resources/335807-1330620492317/9780821386842.pdf

Day 2Designing & managing projects for impact at scale


Objectives of today


Understanding

1. the important of considering MEL during

Project Design

2. the range of tools used for different aspects

of monitoring evaluation and learning.

3. how to develop the activities, output,

outcome, impact logic of projects

4. how to select relevant tools for different

MEL tasks

Session 1

Monitoring, Evaluation & Learning: Introduction


What is the MEL System about?

84 •

What: Describing the change (planned & actual)

Why: Explain the change (the logic)

Is it good enough: Evaluative part (targets)

DATA

DIALOGUE


• Inputs: resources used to implement the activities. Eg $, people,

equipment, etc

• Activity: Actions implemented by the project, including research.

• Output: New information, ideas, technologies created by the

project.

• Outcome: describes the changes in behaviour of farmers, traders,

policy makers and other actors.

• Purpose: describes the development challenge that the project is

contributing to resolving. Also known as impact.


Before we delve into the detail lets recap on some key terms …

85 |

• Results chain / impact pathway / project logic / Theory of Change: articulates and illustrates how & why the planned components of project or programs will deliver its outputs and desired changes (ieoutcome (behavioral) and impact (economic, social and environmental).

• Indicator / KPI / measure: a metric used to monitor and assess progress and success against the planned outcomes and impacts

• Milestone: a significant stage or event in the project. Helps to track progress against planned activities & outputs.

• Baseline: a quantitative and/or qualitative performance assessment of a project or program starting point.

• Target: A aspirational goal set against an indicator to indicate success. Ideally a target should be Specific, Measurable, Achievable, Relevant, and Time-bound (SMART)

• Assumption: describes what has been accepted as true or certain to happen to justify the causal linkages described in a results chain / impact pathway / etc


More key terms …

86 |


More key terms …

87 |

A typical journey ….


Presenter

Presentation Notes

The pathway may look linear but the innovation journey is anything but …

• Articulating planned changes and the logic underpinning how these changes will resulting from

planned activities (short, medium and long term changes).

• Organising & communicating information

• periodically to enable decision makers to make evidence based judgments and adjust plans as

necessary.

• to funders/donors and other stakeholders about the projects contribution to the planned outcomes

and purpose (impact) for accountability.

• about generic lessons from the implementation of projects/ clusters of projects that can inform

project and program practice in the future.


M&E&L helps to navigate this journey by …

89 |

You have 10 minutes to discuss in groups of three some of the challenges you

have experienced.

Write on cards your top two challenges (one per card).

We will then ask you to report your top two challenges back to the group for

discussion.

Exercise: What are your challenges?


Typical MEL challenges?• Technology adoption and impacts:

– selecting indicators and data collection methods

• Tracking effectiveness of collaboration:

– Quality or quantity? Scope, Relevance?

• Tracking capacity development to innovate at multiple levels

• How stakeholders and systems respond:

– Unpredictable and unexpected. But project needs this information to keep on track.

• Resources

• Dedicated resources for MEL can often be overlooked at the planning time


Different MEL Tools & approaches for different purposes


You have 5 minutes to think about the following questions. Use

your note pads to capture your thoughts. After 5 minutes of

individual reflection we are going to discuss as a group.

Question 1: What tools do you use?

Question 2: Do you use them for accountability or learning

purposes?

Question 3: Would you recommend any particular tool, if so

why?

Question 5: Would you not recommend any particular tool, if

so why?

Exercise: What tools do you currently use & why?


Design

Implementation

Review


Source: CSIRO (2014) Impact Framework

Impact pathway

Problem trees


19. Increased planting area for cassava

21. Increased net incomes of existing cassava farmers

Cassava Results ChainDRAFT (June 2016)

14. Clusters buy more cassava (due to improved productivity) from farmers &

process into chips

2. UNEJ & PT BCM establish new clusters, coops, organic fertilizer production

facilities, sheep pens involving target cassava & sheep farmers

1. UNEJ and PT BCM provide TA to coops &

clusters on all aspects of business model

13. Decreased cassava input costs via organic

fertilizer substitution for existing cassava farmers)

22. Increased net income for new cassava farmers (improved profitability

compared to former crops)

10. Farmers adopt improved practices for cassava production

6. UNEJ & PT BCM disseminate improved production practices to key cassava

farmers in coops

5. Coops provide loans, inputs, organic fertilizer, sheep and

TA to cassava & sheep farmers

8. Key cassava farmers disseminate improved production practices to

other farmers

9. Sheep farmers adopt fattening practices for

sheep

20. Increased net incomes of sheep farmers

18.Share of increased local coop profits from sheep &

organic fertilizer flows back to farmers

4. Capacity building by ARISA (Partnership, Technical, Gender, Ethics, Market Development and others TBD) to UNEJ & PT BCM

7. More effective partnership between PT BCM & UNEJ, improved technologies, improved outreach and sustainability of income increases, and international-quality research publications

15. Stronger linkages among clusters, farmers, coops, PT BCM

12. Sheep farmers sell sheep to investors/NGO via coops

16. Other farmers outside the system copy best practices for cassava production or other elements of integrated farming system

3. UNEJ engages with government actors

11. Increased government policy and/or financial support

23. Improved scale-out and sustainability of

business model

17.Share of increased coop profits from sheep re-

invested into expanding sheep farming scheme

KEYred lines – institutional changesgreen lines – DCED copying = = impacts

= intermediary outcomes (behavioral changes by market actors)

= activities/direct outputs expected by project partners/ARISA

Results chains / logic models


Indicators

97 |

INDICATOR ASSUMPTIONSMEANS OF

VERIFICATION


Presenter

Presentation Notes

The terms “measure”, “metric” and indicator” are often used interchangeably and their definitions vary across different documents and organisations. Hence, it is always useful to check what these terms mean in specific contexts. Defined during Design stage; refined, monitored and reported against during implementation and review stages Typically used for outcomes and impacts

Activity plans and milestones

98 • Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly

Presenter

Presentation Notes

Defined during Design stage; refined, monitored and reported against during implementation and review stages Used to track progress of activities and outputs

Internal activity reporting

Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly99 •

Presenter

Presentation Notes

Uses milestones and activity plans to communicate activity to project team and other key internal project stakeholders such as steering committees if requested

Rubrics


Presenter

Presentation Notes

A rubric sets out clearly criteria and standards for assessing different levels of performance. A rubric is a participatory process that involves rating the performance of a project / programme, which can be generic (eg from 'Very poor' to 'Excellent') or customised (eg ''Detrimental' to 'Highly Effective'). A rubric can also be known as a global assessment scale. Advice for rubrics users Involve a mix of people in the development of the rubrics to ensure that all important criteria have been included and the rubric is seen as legitimate by those who will be using its results. If possible, first piloted or field test the matrix to create a common understanding of expectations and check that the criteria at each level are defined clearly enough to ensure that scoring is accurate, unbiased and consistent. Ensure that the criteria and expectations in the rubric directly aligned with the overall objectives of the project.

Most Significant Change (MSC)


Presenter

Presentation Notes

The Most Significant Change (MSC) approach involves generating and analysing stories or personal accounts of change to determine what was the most significant – and why. MSC is not just about collecting and reporting stories but about having processes to learn from these stories. It provides some information about impact and unintended impact but is primarily about clarifying the values held by different stakeholders. It can be very helpful in explaining HOW change comes about (processes and causal mechanisms) and WHEN (in what situations and contexts). In particular it can be useful to support the development of program theory of change or design logic.

102 |Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly

It is a process standard for

donor projects that work

with the private sector. It

includes a criteria for

compliance criteria for

each component and a

compliance audit function.

It is updated every few

years to maintain

relevance.

DCED Standard

Based on Jim Tanburn (DCED)

Key components include:

• Articulating the Results Chain or

programme logic

• Defining indicators of change based on

the logic

• Measuring changes in indicators, applying

good practice

• Estimating attributable changes

• Capturing wider changes in the system or

market

• Tracking associated programme costs

• Reporting results in a responsible way

• Managing the system for results

measurement

Presenter

Presentation Notes

It was developed in 2008. by donors & consultants working on private sector development projects, It built on lessons from a number of donor projects including: DFID’s ENABLE in Nigeria, USAID’s MEDA in Pakistan, UNCDF in Nepal, & DFAT’s Enterprise Challenge Fund in Pacific & Asia. Currently 100+ programs in 50+ countries are using the standard

Outcome mapping (OM)


• It focuses on understanding outcomes or the results that emerge from the initiatives activities

towards the longer term economic, environmental, political and demographic changes

• It provides tools that can be used standalone or in combination with other ME&L systems to

identifies, plan, monitor & evaluate behavior changes in individual, groups or organisation's that

the project works with.

• It is a participatory process – involves all actors in M&E

• Good for projects that:

Monitor behavior change of stakeholders / partners require capacity building work involve knowledge and decision-making processes, where technical concerns can obscure the development challenges complex issues

Presenter

Presentation Notes

Outcome mapping (OM) is a methodology for planning, monitoring and evaluating development initiatives in order to bring about sustainable social change … it is orientated towards social & organizational learning In particular it monitors - non-linearity and contribution, not attribution and control At the planning stage, the process of outcome mapping helps a project team or program be specific about the actors it intends to target, the changes it hopes to see and the strategies appropriate to achieve these. For ongoing monitoring, OM provides a set of tools to design and gather information on the results of the change process, measured in terms of the changes in behaviour, actions or relationships that can be influenced by the team or program. As an evaluation approach, OM unpacks an initiative’s theory of change, provides a framework to collect data on immediate, basic changes that lead to longer, more transformative change, and allows for the plausible assessment of the initiative’s contribution to results.

Technology adoption tracking tools

Partnerships, platforms & managing for impact • Michaela Cosijn & Jen Kelly104 •

Provides quantitative and/or qualitative information to the project and donors about

the difference a project has made at the scientific and community level … Focuses on

the uptake of the science.

Can include tools like:

• Baseline surveys and end of project / post project surveys

• Rapid appraisals and ad hoc studies – valuable for keeping an eye on progress and

getting feeds back on weather the project activities are leading to expected results.

Presenter

Presentation Notes

If you have worked on an ACIAR project before you may have been involved in one of these?

Institutional History (IH)

• A narrative that records key points

about how institutional arrangements –

new ways of working – have evolved

over time and have created and

contributed to more effective ways to

achieve project or programme goals.

• An IH is generated and recorded in a

collaborative way by scientists, farmers

and other stakeholders.

• A key intention behind institutional

histories is to introduce institutional

factors into the legitimate narrative of

success and failure in research

organizations


Reviews and external evaluations

106 |

• Relevance: The extent to which the aid activity is suited to the priorities and policies of the target group,

recipient and donor.

• Effectiveness: A measure of the extent to which an aid activity attains its objectives.

• Efficiency: Efficiency measures the outputs - qualitative and quantitative - in relation to the inputs. This generally

requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient

process has been adopted.

• Impact: The positive and negative changes produced by a development intervention, directly or indirectly,

intended or unintended. This involves the main impacts and effects resulting from the activity on the local social,

economic, environmental and other development indicators.

• Sustainability: Sustainability is concerned with measuring whether the benefits of an activity are likely to

continue after donor funding has been withdrawn.


Presenter

Presentation Notes

So in summary what do we know …


There is no silver bulletand

design and MEL are not separate functions … they

inform each other

Presenter

Presentation Notes

Key Fit for Purpose … this is the test for a good MEL system

Morning coffee break


Session 2:Developing a project logic / impact pathway / results chain



Source: CSIRO (2014) Impact Framework

Impact pathway

Example scenario• Program purpose : Growth, development and innovation of food systems in Indonesia

• Actors / Partners : Researchers, farmers, private sector

• Assumptions :

• If farmers apply new knowledge or existing knowledge in a new / novel way (innovation), they will improve productivity and household incomes, which will in turn lead to economic growth and development in the food system in Indonesia

• The market / private sector is functioning relatively well and has capacity to connect / partner with public researchers and farmers

• Policy makers both national and local will are connected and responsive to farmer and market needs and will ensure the policy and regulatory environment enables growth and economic development in the agri-food sector in Indonesia


• Projects to increase incomes of farmers in:

• Cassava

• Sugar

• Beef

• Geography :

Some ground rules used to develop the impact pathways

• The program and the supporting project needed to develop an impact pathway

• The impact pathway logic was used to articulated and agree how the project’s

purpose (impact) contributed to the program purpose. e.g. Improving the welfare

of smallholder cassava farmers in the southern part of East Java through

integrated cassava system (from production to processing)

• All projects outcomes needed to contribute to the programs purpose, including:

• Adoption of technology/policy/standards + capacity development

• Researchers, farmers, private sector, government, NGO, etc. working in new ways

that deliver solutions.


Tip:

Don’t start with the

planned research.

Start with the

problem you want to

solve.

Presenter

Presentation Notes

Built on earlier research and research teams Extensive collaborator and stakeholder workshops and consultations to develop problem trees to identifying a interconnected set of problems) develop project outcomes, outputs and activities needed to address those problems. develop of a results chain (aka project logic, theory of change, etc), and develop of an MEL systems to track progress on outcomes using DCED standards


Example program logic

Source: Adapted from CSIRO 2015 ARISA Program Design Document

Impa

ct Smallholder farmers in East Indonesia have increased productivity and

household incomes

Inclusive growth and productivity in the agri-food sector in East

Indonesia


Example project logic



Example Approach - how the partnerships were set up to implement these projects and program


Exercise: Developing an Impact Pathway


1. Break into your IP groups (the one you formed yesterday for the IMGoats project)

2. Your task is to develop a impact pathway for your IP

3. Use the information in the handout as background information for your impact

logic. You will have 30 minutes

4. At the end of the 30 minute you will have 5 minutes to present your results chain

5. We will then your discuss lessons, insights, etc as a group

Exercise: Questions to considerations ..?1. What problem am I helping solve? (impact)

2. Who is the client?

3. Who is the beneficiary?

4. Who do I need to work with & What is their role?

5. What product or service do they want? (outputs)

6. When do they want it by?

7. What people, $, equipment do I need (inputs)

8. What will the beneficiary do different if I successfully deliver the product or service? (outcome

/ behavioural change)

9. What do I need to do to deliver the product / service? (activities)

10. What approach would you use for developing a logic?


Exercise: Tips


1. Start with the end in mind

Think about what problems you agreed to work on and if successful what you want to see changed end of the project

Think about what you logically need to do as a group or individually to achieve these changes

2. Be a clear and specific as possible when writing your results

This helps to build agreement and avoid confusion later down the track

3. There is no right or wrong outcome or logic – your aim is fit for purpose

The important things to focus on are : 1) does the proposed logic make sense and 2) do the stakeholders agree

4. Remember your logic

Impact, outcomes, outputs, activities, inputs

5. Remember your desired impact is not in your control … focus on how you think you can contribute and

influence changes that will support the desired impact being achieved

6.

Now that everyone has presented their impact pathways:

What did you observed about everyone’s impact pathways?

What did you find the hardest part in developing it?

Was the process useful, and if so how was it useful?

Exercise: Debrief



Lunch break

Case Study:Managing an Innovation Platform –What MEL info do we need?


Key aspects to managing and IPSo now that we have formed our IP and developed an impact pathway ... Now what???

• What information do we need to monitor to show we are on track, learning and able to make decisions?

• Who does this?

• When does it need to happen?

• How do we communicate this information to the different IP actors?


Role of the IP facilitator / brokerEach actor may have the same, similar and/or different information needs for learning and

accountability purposes.

The facilitator needs to help the group

- find the common ground in terms of information needs … something that works for the IP

and the individual organisations

- Establish a monitoring systems that is feasible, efficient and useful

- Ensure accountability and responsibilities are clear for collecting, analysing and reporting

monitoring results

- Agree on timing for periodical monitoring data collection, reporting and decision making


TIP: You don’t need to know every

thing at the start … info needs can

evolve over time.


Monitoring project logics


Example: What information do we need to manage the IP?


Session 3Exercise:Role Play


Exercise: Monitoring the IP we set up on yesterday?

ROLE PLAY: Develop a draft MEL plan for the IP


1. Reform the groups that set up the IP yesterday

2. In your groups take on the same role you played yesterday

3. You will have 30 minutes to develop an M&E plan – think

about the questions on the next slide

Exercise: Characters and discussion questions

A representatives from

- Government

- NGO

- Farmer representative(s)

- Paravets

- Private Sector

- Trader and retailer

- Research institution (Broker)

Questions for consideration

- What information do you need to know if the platform is

working and if co-created activities are working as

expected?

- Why do you need it?

- When do you need it and how often?

- Where will you source the information from?

- What tools will you use?


Questions for group debrief …

What did you find most challenging from each actors perspective?

What helped you come to a consensus if you go to one?

What limited the consensus making?

What lessons would you take forward?

Exercise: Group debrief


Questions


Useful Resources


Resources• Online resource portals for MEL practitioners

• Better Evaluation http://betterevaluation.org

• Online community of practices• Eval Partners http://www.mymande.org/evalpartners

• S.Stone-Jovichi (2015) The journey to fit for purpose: The development of a MEL system for a complex project. Practice Note Series by Food Systems Innovation • www.foodsysteminnovation.org.au

• The Donor Committee for Enterprise Development (DCED) Standard • http://www.enterprise-development.org/measuring-results-the-dced-standard/


http://betterevaluation.org/

http://www.mymande.org/evalpartners

http://www.foodsysteminnovation.org.au/

Michaela CosijnInternational Development Research ScientistLand and Water

t +61 7 3833 5579e [email protected] www.csiro.au

Jennifer KellySenior Innovation BrokerAgriculture and Foodt +61 2 6218 3474e [email protected] www.csiro.au

Thank you

Increasing development impact niarobi nov 2016 final

Science

Transcript of Increasing development impact niarobi nov 2016 final