CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

28
Philippe ELLUL, Senior Science Officer ECOP meeting Rome, 12-13 Nov. 2015 Consortium/System Office Science Team Monitoring, Evaluation & Learning

Transcript of CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Page 1: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Philippe ELLUL, Senior Science Officer

ECOP meeting

Rome, 12-13 Nov. 2015

Consortium/System Office Science Team

Monitoring, Evaluation & Learning

Page 2: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

System Office and new SRF

Finance

IP & Legal

Strat. & Partn.

ICT & KM

Science

System Council

Portfolio / CRPs / Centers

ind

epen

den

t ..

. IEA

ISPC

IAU New

SR

F

Page 3: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Science Team at the CRP level

To implement the reform process in alignment with SRF by

– maintaining an ongoing direct dialogue with CRPs to provide a sounding board and share best practices – concerns across programs

– ensuring that top quality and effective R4D proposals are designed and proposed to CB/ FC for approval (Extension, draft & pre-proposal phase2)

– assessing annually the planning (POWB) and reviewing performance of CRPs (Annual Reports) by monitoring progress along impact pathways and advise on resource re-allocation

– advising on the potential use of technological and behavioural innovations (eg pre-breeding approaches, policies issues on germplasm exchange, big data opportunities, system dynamics or agro-biodiversity studies

– supporting CRPs to set up an efficient Result-Based Management (RBM) framework, promoting an outcome-based approach; MEL CoP

Page 4: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

1) The ST supports CGIAR scientists & stakeholders in shaping a coherent, innovative and well-balanced Portfolio of CRPs that

• enhances synergies across the system

• address research gaps

• minimize duplication

Science Team’s role at the Portfolio level

Page 5: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

2) The ST produces and facilitates adherence to CGIAR research policies, guidelines, and standards that facilitate increased coherence and impact

• Harmonized reporting templates (POWBs and Annual Reports)

• Guidelines for Extension Proposals 2015-16

• Guidelines for CRP 2nd call 2017-22

• CB/FC Gender Strategy

• Open Access & Open Data strategy (harmonized implementation)

Science Team’s role at the Portfolio level

Page 6: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

3) The ST deals with increased Portfolio efficiencies by steering the entire package towards increased efficiency, relevance, and scientific innovation and quality by:

• identifying concerted actions (pre-breeding, GB++, RBM, etc...)

• harmonizing responses on cross-cutting strategic issues

• coordinating strategic sharing of research infrastructure across CRPs

Site/country integration (Partnership)

Data repositories, metadata, interoperability (OA-DM, ICT)

User-friendly flexible IT system for RBM results information

Innovation platforms, Phenotyping platforms

Etc...

Science Team’s role at the Portfolio level

Page 7: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Science Team’s role at the Portfolio level

Page 8: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

1st MEL CoP meeting (Rome, 10-11 November 2015)

Page 9: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

System Office & CRP performance monitoring

CCEE

MonitoringEvaluation

CRP Life Cycle

Pro

port

ion

of

rese

arc

h p

ortf

olio

Impact Assessment

CONSORTIUM –SYSTEM-OFFICE

IEA

IAU

Page 10: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

But generic ToC far too simplistic about how science caninfluence rest of world

Essential to evolve realistic & dynamic ToC overtime (e.gprivate seed companies w/ 5-year plan revisited annually)

Need + flexibility, + willingness to learn from mistakes,asking key questions, re-inventing program

3rd round of CRP Annual Reports (2012-2013-2014)

• quality reporting

• + structured CRPs

• visibility on progress towards IP

• Performance reporting strategy

Page 11: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Annual Reports 2013, 2014 and 2015

A. KEY MESSAGES (1 ½ page)

B. IMPACT PATHWAY AND INTERMEDIATE DEVELOPMENT OUTCOMES (IDOS) (1/4 page)

C. PROGRESS ALONG THE IMPACT PATHWAY

The CRP should complete Table 1, in Annex 1 and provide a narrative (C.1 to C.3), referring to those indicators from the table in Annex 1 that are relevant to the CRP

C.1 Progress towards outputs ( 2 pages)

C.2 Progress towards the achievement of research outcomes and IDOs ( 2 pages)

C.3 Progress towards Impact (1/4 page)

D. GENDER RESEARCH ACHIEVEMENTS (1 page)

E. PARTNERSHIPS BUILDING ACHIEVEMENTS (1 page)

F. CAPACITY BUILDING (1/2 page)

G. RISK MANAGEMENT (less than 1/2 page)

H. LESSONS LEARNED (1 page)

I. CRP FINANCIAL REPORT (L series, 9 Excel files)

Page 12: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Annual Reports 2013, 2014 and 2015

• KNOWLEDGE, TOOLS, DATA (12)

• CAPACITY ENHANCEMENT AND INNOVATION PLATFORMS (5)

• TECHNOLOGIES/PRACTICES IN VARIOUS STAGES OF

DEVELOPMENT (10)

• POLICIES IN VARIOUS STAGES OF DEVELOPMENT (5)

• OUTCOMES ON THE GROUND (2)

34 indicators in Table 1 / Annex 1

Page 13: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Indicator #9 Number of publications in ISI journals produced by CRP

Web-link to the complete list of papers published in 2014

Funding acknowledgement (W1, W2, W3 or bilateral) is suitable in all publications (mapping the publications to CRPs’ contribution)

A classification based on ISI impact index would provide information on science quality and content

Present a summary on the CRP research staff (numbers, grades, gender; see FTA AR 2014)

2014 CRP ISI papers

1 FTA 328

2 GRiSP 218

3 WLE 150

4 A4NH 137

5 CCAFS 114

6 GL 108

7 WHEAT 107

8 PIM 98

9 RTB 92

10 DS 72

11 MAIZE 64

12 AAS 49

13 L&F 48

14 DC 47

15 HT 44

TOTAL 1676

Page 14: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Performance / Science quality

Informed by

Elsevier bibliometric analysis

2014

Page 15: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

34 indicators of the Annual Report (Table 1 / Annex 1)

TECHNOLOGIES/PRACTICES IN VARIOUS STAGES OF DEVELOPMENT

18. Number of technologies/NRM practices under research in the CRP (Phase I)

=> Novel genes, promoters or QTLs with known major effect(s) on specific traits, molecular markers, breeding or lines with improved traits (NILs, RILs, transgenic, …), DHLs, animal lines with specific traits

23. Number of technologies /NRM practices field tested (phase II)

=> Superior genotypes, breeds, lines, crosses or hybrids with improved traits, transgenic lines, improved variety for which certification is being sought

27. Number of technologies /NRM practices released by public and private sector partners globally (phase III)

=> Improved varieties, cultivar, lines or breeds with various desirable traits, management practices, ready to be dessiminated

Page 16: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

TECHNOLOGIES / PRACTICES IN VARIOUS STAGES OF DEVELOPMENT

18. Number of technologies/NRM practices under research in the CRP (Phase I)

23. Number of technologies /NRM practices field tested (phase II)

27. Number of technologies /NRM practices released by public and private sector partners globally (phase III)

0

20000

40000

60000

80000

100000

120000

140000

160000

180000

200000

WHEATMAIZE

DCGL

RTB

Phase I nº18 Phase II nº23 Phase III nº27

WHEAT MAIZE DC GL RTB

Phase I nº18 200000 34123 6329 1684 78

Phase II nº23 2151 2063 2173 23 48

Phase III nº27 24 70 8 32 22

Page 17: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

From guidelines to creativity... MAIZE Annual Report 2014

Page 18: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Only 2 outcome-based indicators (Table 1 / Annex 1)

OUTCOMES ON THE GROUND

33. Number of hectares under improved technologies or management practices as a result of CRP research

34. Number of farmers and others who have applied new technologies or management practices as a result of CRP research

Page 19: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Outcome indicators 33 (a) and 34 (b)

From CGIAR Research Programs (CRP) Annual Reports 2013 and 2014

CRPs Number of hectares (a) Number of farmers reached (b)

2013 2014 2013 2014

A4NH 231 135,000 645,075 1,089,139

AAS (c) 242,133 7,408 584,953 -

CCAFS 450,000 (d) 185,000 2,000,000 3,900,000

Dryland Cereals 80,000 267,800 70,000 377,000

Dryland Systems 4,000,000 7,525,065 4,000,000 469,348

FTA 196,500,000 65,000,000 870,000 2,000,000

Grain Legumes 253,224 4,763,000 1,717,249 2,451,917

GRiSP 4,763,000 91,996 7,741,000 7,741,000

HumidTropics 12,000 430,000 94,466 91,000

Livestock & Fish (c) 331,070 632,169 24,000 488,000

MAIZE 417,000 2,305,462 1,100,000 4,089,000

PIM - - - -

RTB 398,000 430,000 1,180,000 1,180,000

WHEAT 1,650,000 2,096,240 2,802,000 10,310,000

WLE 15,471,050 4,845,000 35,000 205,946

Total 224,567,708 88,714,140 22,863,743 34,392,350

Page 20: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Intermediate users’ adoption

End of ADB

project

HRDC Formed

38 members

HRDC w/ 68 public & private members

Kindly provided by Dr. Achim Dobermann (IRRI)

Page 21: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

End users’ adoption

Drought Tolerant Maize for Africa (DTMA) project

Page 22: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Drought Tolerant Maize for Africa (DTMA)

http://www.cgiar.org/consortium-news/partnerships-lead-to-measurable-impacts-for-drought-tolerant-maize-for-africa/

DTMA Project indicators 2013

Number of new varieties released (2007-2013) 140

Seed production in 2013 (new varieties) 30,000 t

Number of NARs - or companies - involved in seed production 13

Number of stakeholders adopting the new varieties: cooperatives;

regional, national, small & medium enterprises (SMEs);

international enterprises; large companies

118

Hectares planted with the new varieties 1,230,000 has

Number of households reached 2,900,000

Number of consumers benefiting these adoptions (Millions) 20 M

Page 23: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Portfolio Report 2014

Page 24: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Portfolio Report 2014

Page 25: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Portfolio Report 2014

Page 26: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

CRP Portfolio Report 2014

8. RISKS TO PORTFOLIO PROGRESS AND THEIR MITIGATION

Risks of a CRP portfolio biased towards short-term priorities and results

New financial risk

Risk of continued disconnection between financial and programmatic planning and management

Risk of mobilizing insufficient human resources to undertake research on complex systemic approaches

Risk of a CRP leadership not fully empowered

Risk of a patchy approach to integrity and ethics in research for development at portfolio level

Page 27: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Science Team’s role at the CGIAR-Institutional level

How to optimize the implementation of the CGIAR reform process?

To participate to the design of the new Strategic Research Framework (SRF) for the CGIAR

To explain the way in which the new System Council functions, what it decides and the rationale in the decisions, to communicate clearly to the CRPs/stakeholders what donors’ expectations are

To communicate and work + collaboratively with ISPC, IAU, IEA, regional organizations (GCARD3), donors, development agencies

To maintain and build external partnerships (to provide leadership in agricultural development, share / learn from others)

To ensure a system-wide focus demonstrating global visibility on development impact for smallholders’ benefit

Page 28: CGIAR Consortium/System Office - Monitoring, Evaluation and Learning

Thanks a lot for your attention...