Evaluation of the CGIAR Research Program on Climate Change, … · 2019-11-06 · This evaluation...

I

Evaluation of the CGIAR Research Program on

Climate Change, Agriculture and Food Security (CCAFS)

Volume II – Annexes

April 2016

Simon Anderson Fawad Khan Carmenza Robledo Christian Roth

This evaluation has been commissioned by the Independent Evaluation Arrangement (IEA) of CGIAR.

The Independent Evaluation Arrangement (IEA) of CGIAR encourages fair use of this material provided proper citation is made.

Correct citation: CGIAR-IEA (2016), Evaluation of the CGIAR Research Program on Climate Change, Agriculture and Food Security (CCAFS). Rome, Italy: Independent Evaluation Arrangement (IEA) of CGIAR.

http://iea.cgiar.org/

http://iea.cgiar.org/

ii

Evaluation of CGIAR Research Program on CCAFSEvaluation of CGIAR Research Program on CCAFS

iea.cgiar.org

Table of Contents

ANNEX A: CCAFS EVALUATION TIMELINE ...............................................................1

ANNEX B: Evaluation team profiles .........................................................................3

ANNEX C: Existing review and evaluations consulted .............................................5

ANNEX D: List of interviews conducted ...................................................................6

ANNEX E: CCAFS researcher survey – results .......................................................... 11

1. OVERVIEW ........................................................................................................ 11

2. RESULTS ............................................................................................................ 12

PART I: INTRODUCTION ................................................................................... 12

PART II: YOUR INVOLVEMENT IN CCAFS ......................................................... 15

PART III: YOUR RESEARCH ............................................................................... 19

PART IV. PARTNERSHIPS, GENDER AND CAPACITY ENHANCEMENT .............. 24

PART V. CCAFS WORKING CONDITIONS .......................................................... 28

PART VI. CCAFS VALUE ADDED ....................................................................... 29

ANNEX F: Bibliometrics analysis ............................................................................. 30

ANNEX G: Review of journal articles ...................................................................... 35

1

Evaluation of CGIAR Research Program on CCAFS Evaluation of CGIAR Research Program on CCAFS

iea.cgiar.org

ANNEX A: CCAFS EVALUATION TIMELINE

MAIN EVENTS PERIOD/DATE ACTIVITIES PEOPLE INVOLVED PREPERATORY and INCEPTON PHASE

Oct 2014 – Feb 2015

Preparatory Phase • Finalizing ToR • Recruitment of Evaluation Team • Establishment of Reference Group

IEA

1st RG consultation (by e-mail)

27 Oct 2014 • Feedback on draft ToR RG + IEA

Attendance of CCAFS meetings, Washington

28 – 31 Oct 2014 • Observing PMC meeting, ISP meeting • Identifying together with CCAFS

management the key evaluation issues

Simon Anderson

Inception meeting, Rome

10 – 13 Mar 2015 • Work on evaluation methodology • Start preparing the Inception Report • Briefing on CCAFS program and

interaction with CCAFS management

IEA + ET + CCAFS

2nd RG consultation 28 May 2015 • Feedback on draft Inception Report RG + TL + IEA Final inception report Jun 2015 • Final inception report published on

IEA homepage IEA

INQUIRY PHASE Mar – Jun 2015 Field visits Key points covered: • Kenya 10 Apr – 6 May • Interaction with regional CCAFS

management and researchers and partners

• Participation in CBA 9 conference • Visit 2 CSVs

Simon Anderson Fawad Khan

• Senegal 26 Apr – 2 May • Interaction with regional CCAFS management and researchers and partners

• Visit 3 CSVs in Kaffrine and inetrview with local partners and beneficiaries

Carmenza Robledo

• India 15 – 28 May • Interaction with regional CCAFS management and researchers and partners

• Visit 3 CSVs in Haryana state and 2 CSVs in Ludhiana

Fawad Khan Christian Roth

• Vietnam • Lao

15-18 Jun 18-22 Jun

• Interaction with regional CCAFS management and researchers and partners

• Visit 1 CSV near Hanoi and 1 CSV near Vientiane

Christian Roth

2


iea.cgiar.org

• Colombia 7- 16 Jun • Interaction with regional CCAFS management and researchers and partners

• 2 farm visits

Simon Anderson Carmenza Robledo

Research staff survey June 2015 • Design and piloting of survey • Conduct of survey

ET + IEA

Other inquiry phase activities

June-Aug 2015 • Interviews with partners and stakeholders and external experts

• Documentation review

ET

ANALYSIS AND REPORTING PHASE

Jun -Oct 2015

Synthesis of existing reviews/evaluations

June 2015 • Review of 10 reviews and evaluations • Synthesis of main findings

IEA

Bibliometric analysis June 2015 • Citation, journal frequency and H index analysis

IEA

Analysis meeting in Edinburgh

24-26 June 2015 • Consolidation and verification of findings

• Identification of information gaps

ET + IEA + CCAFS representation

Publication review July 2015 • Qualitative assessment of sample publications

External panel (4 members)

Drafting of report July-Aug 2015 • Drafting of evaluation report ET 3rd RG consultation

• RG + TL + IEA

Feedback and comments • CCAFS management and RG provide feedback and comments

•

CCAFS +RG

Incorporation of comments • Review and revisions of draft report TL +IEA Final Evaluation Report • Final Evaluation Report incl. CCAFS

management response submitted to CGIAR Fund Council

IEA

Dissemination phase • Communications products CCAFS + TL + IEA

ET= Evaluation Team, TL = Evaluation Team Leader.

3


iea.cgiar.org

ANNEX B: Evaluation team profiles

Simon Anderson is the Head of the Climate Change Group at the International Institute for Environment and Development. He has worked in international development for some 30 years, mainly on natural resources management and environmental change. His current research focus on climate change adaptation effectiveness and his expertise covers agro-economic systems and agricultural science as well as policy analysis, programme management and monitoring and evaluation. In 2009 he led Joint External Evaluation on the Operation of the Least Developing Countries Fund (LDCF) for adaptation to climate change. Simon previously worked for DFID as Research Manager and Evaluation Advisor and was also a Principal Research Fellow at Imperial College. Simon has a PhD Agricultural Science from the University of London.

Fawad Khan is CEO and founder of Institute for Social and Environmental transition in Pakistan, a non-profit research institute. He focuses on the evaluation of adaptation effectiveness, economics of adaptation strategies, community based local adaptation planning, and exploring factors contributing to resilience. His areas of expertise are the economics of climate change in South and South East Asia, community based adaptation, and monitoring and evaluation. Fawad previously worked as an Institutional Development Specialist for the World Bank in South Asia, a Senior Coordinator on Monitoring and Evaluation for the International Union for Conservation of Nature and has over 20 years’ experience as a development consultant. He holds an MSc from the Faculty of Economics at the London School of Economics and Political Science in UK, a BSc from the School of Engineering and Applied Sciences at Columbia University and a BA in Liberal Arts from Middlebury College in USA.

Carmenza Robledo has almost 20 years experience on climate change and sustainable management of natural resources in developing countries. In her work she combines scientific research, policy advice and project implementation. She has project experience in Latin America, Africa and Asia, as well as experience advising international organizations including ITTO, FAO, World Bank, UNDP, UNEP, CIFOR, GEF, UNFCCC Secretariat or IUCN. She participated in the Fifth Assessment Report of the IPCC as a Lead Author in the Working Group III - mitigation - and as a reviewer in the Working Group II – Vulnerability and adaptation. During the period 2013-14 she was member of the FTA evaluation team, where she was responsible for climate change as well as for gender issues. Carmenza has a PhD in geography from the University of Stuttgart.

Christian Roth over 30 years of research experience in tropical land and water management. He is currently working with CSIRO’s Land and Water Flagship based in Brisbane, Australia. Over the last twelve years, his main focus has been designing, commissioning and conducting inter- and transdisciplinary research for development programs and projects in Australia, South and South-East Asia in water resource management, climate change adaptation, smallholder farming systems and conservation agriculture. He has also led or participated in a range of research project and program evaluations in Australia and Asia. His main research interests reside in integration of social sciences and biophysical research to influence the research for development agenda in South and South-East Asia, specifically in the domains of

4


iea.cgiar.org

climate adaptation and agricultural development. He has published his research extensively in about 180 publications and research reports. Christian has a PhD in soil hydrology from the University of Göttingen.

5


iea.cgiar.org

ANNEX C: Existing review and evaluations consulted

Reviews conducted by donors, the consortium, auditors and the Internal Evaluation Arrangement • IAU Advisory PHASE I Review of CRP 7 – Climate Change, Agriculture & Food Security

(CCAFS) (2015) • EU review of CCAFS (2012) CRP-Commissioned External Reviews (CCERs). • Management and governance review (2013) • Review of CCAFS Theme by region matrix management (2013) • Theme 3 review (2014) Core Team Commissioned Reviews • Review of CCAFS Scenarios activities (2014) • Theme 2 review on climate information services activities (2014) • Review of capacity enhancement activities (2014) • Evaluation of CCAFS Data and Tools (2014) • CGIAR citations in IPCC reports: a summary report (2015)

6


iea.cgiar.org

ANNEX D: List of interviews conducted

Surname Name Gender Position Organization

Aggarwal Pramod Male Regional Program Leader CCAFS

Ampaire Edidah Female Project Coordinator IITA

Arango Jacobo Male Scientist CIAT

Aryal Jeetendra Male Agricultural Economist CIMMYT

Asekenye Cresensia Female Research Associate IITA

Bathily Samba Male Deputy Parliament of Senegal

Berre David Male Farming System Agronomist, CIMMYT

Bewket Woldeamlak Male Associate Professor Addis Ababa University

Bonilla Osana Female FP1 assistant CIAT

Bustamante Mercedes Female Profesor University of Brasilia

Campbell Bruce Male Professor University of Copenhagen/CCAFS

Cardozo Carlos Ivan Male Scientist Universidad Nacional de Palmira

Chacon Adriana Female Climate change expert CATIE

Chirinda Ngoni Male Scientist CIAT

Cong Lan Vu Male ?

National Institute of Agricultural Planning and Projection ...

Corner-Dolloff Caitlin Female Climate Change Adaptation Specialist CIAT

Crane Todd Male Climate Adaptation Scientist ILRI

de Pinto Alex Male Scientist IFPRI

Del Cid Jose Miguel Male Climate change expert Secretaria de Agricultura, Honduras - SAG

Diatta Rama Female Deputy Parliament of Senegal

Dieng Mbaye Male Research Program Officer LEAD Francophone Africa

Dieng Talla Male Director Radios Communataires

Dieye Bounama Male Platform facilitator Min Agriculture

Diouf Birame Male n/a Reseau Environment Developpement - CONGAD

Ericksen Polly Female Programme Leader, Livestock Systems and Environment ILRI

Escobar Daniel Male Scientist CIAT

Forch Wiebke Female Science Officer ILRI

Friis Anette Female Head of Program Coordination-CCAFS CCAFS

7


iea.cgiar.org


Gathenya John Male Associate professor Maseno University

Giraldo Diana Female Scientist CIAT

Girvetz Evan Male Climate Change and Soils CIAT

Goel SK Male Principal Secretary, (Agriculture and Marketing) Government of Maharashtra

Gonsalves Julian Male Senior Program Advisor International Institute of Rural Reconstruction

Hanson James Male Flagship Leader CCAFS

Hellin Jon Male Value Chain and Poverty Specialist CIMMYT

Hillier Jon Male Scientist The Unviersity of Aberdeen

Hymann Glenn Male Scientist CIAT

Inthavong Thavone Male Director

Agriculture and Forestry Policy Research Centre, Agriculture and Forestry Research Institute (NAFRI)

Jarvis Andrew Male Director of the Decision and Policy Analysis Area CIAT

Jarvis Andy Male Flagship Leader CCAFS

Jat ML Male Soil Science, IWMI

Jat RK Male n/a Borlaug Institue for South Asia

Jimenez Manuel Male Climate change expert Consejo Agropecuario Centroamericano - CAC

Joarmahir Babacar Sene Male Director Agropasteur (farmers newspaper)

Joshi PK Male n/a IWMI

Joshi Pramod K Male Director for South Asia IFPRI

Keophosay Anousith Male Research Officer IWMI

Khatri Chhetri Arun Male Science Officer IWMI

Kinyangi James Male Regional Program Leader CCAFS

Kishur Avinash Male Associate Research Fellow IFPRI

Kumar Suresh Male Addl. Director Agriculture Extension, Haryana

Lacombe Guillaume Male Senior Researcher – Hydrologist IWMI

Ladha JK Male Principal scientist for soils and agronomy research IRRI

Lien Huong Do Female Head, Strategy and Policy Research Division IPSARD

Loboguerrero Ana Maria Female Regional Program Leader CCAFS

Londono Sebastian Male Owner and manager Family Farm San Jorge

Lopez-Ridaura Santiago Male Agronomist CIMMYT

8


iea.cgiar.org


Majumdar Kaushik Male Director IPNI South Asia

Martinez Daissy Female CCAFS Martinez Baron Deissy Male Scientific Coordinator CCAFS

Mathieu Henri Male Professor University Cheikh AntaDiop

Mathur Prem Male Regional Director (ad interim) Bioversity

Meincke Holger Male Professor University of Tasmania

Mittal Surabhi Female Agricultural Economist CIMMYT

Mixay Somsanouk Male Vice President Lao Journalists Association

Molina Carlos Hernando Male Owner and manager Family Farm El Hatico

Mude Paul Male Researcher ILRI

Ndiaye Cheikh Tidiane Male Deputy Parliament of Senegal

Ndione Jacques-Andre Male Head of Research Centre de Suivi Ecologique - CSE

Ndiyae Ousama Male Head of Climate Change ANACIM

Neufeld Henry Male Focal contact point ICRAF

Ngoc Quyen Luu Male Deputy Head NOMAFSI

Nguyen Hung Male Senior Scientist ILRI

Nguyen Duong Female Vietnam Coordinator Redraw The Line Media

Nhuong Tran Male Scientist WorldFish

Nyangaga Julius Male Consultant Right Track Africa

Nyasimi Mary Female Gender and Policy Specialist CCAFS

Oyou, Anthony Male Researcher ICRISAT

Parker Louis Male Crop and Climate Modelling CIAT

Pavelic Paul Male Principal Researcher – Hydrogeology IWMI

Phengvichith Vanthong Male Deputy Director General NAFRI

Phongoudome Chanhsamone Male Deputy Director General NAFRI

Plapallil Shri Joseph Male Managing Director Agriculture Insurance Company of India Limited

Radney Maren Female Science Officer ILRI

Rao Kolli Male Crop Insurance Industry Expert n/a

Recaman Liliana Female link between NGO/empresa de acueducto Fundacion Rio Piedras

Recha John Male Participatory Action Research (PAR) Specialist ILRI

Rice Chuck Male Profesor University of kansas

Robinson Lance Male Environmental Governance and Resilience Specialist ILRI

9


iea.cgiar.org


Rosenstock Todd Male Agroecologist ICRAF

Roswal Tomas Male n/a In pension

Sall Moussa Male Scientist/Agroeconomist ISRA: Institute senegalese de researches agricoles

Sandoval Jose Manuel Male National Coordinator of the Colombian Low Carbon Development Strategy

Ministry of Environment and Sustainable Development - MADS

Sane Oumar Male Director de l'Agriculture Min Agriculture

Sanogo Diaïte Diaminatou Female Scientist/focal point ICRAF ISRA: Institute senegalese de researches agricoles

Sapkota Tek Male Agricultural Systems/Climate Change Mitigation CIMMYT

Sarr Ousmane Fall Male Chairmann

Senegalese National Climate Change Committee - COMNAC

Sarre Diouf Madeleine Female Climate Change Unit Min Environment

Sebastian Leo Male Regional Program Leader CCAFS

Seck Madieng Male Director Agri Infor (farmers newspaper)

Sharma DK Male Director Central Soil Salinity Research Institute (ICAR)

Sidhu BS Male Punjab Commissioner Agriculture Sidhu HS Male Senior research engineer BISA

Sikka Alok Male DDG ICAR

Simelton Elizabeth Female Climate Change Scientist ICRAF

Singh Rajbir Male Zonal Project Director ICAR

Smith Pete Male Profesor University of Aberdeen

Tapasco Jeimar Male Scientist CIAT

Thi Sen Pham Female Head of the Science and International Relations Department NOMAFSI

Thiam Djibril Male Coordinator AGRECOL Afrique (NGO)

Thiao Ibrahima Paul Male Regional coordinator

Federation de Organisationen Non Gouvernamentales du Senegal - FONGS

Thioye Yoro Idrissa Male Counsellier en Politique Agricole

Conseil national de concertation et de coopération des ruraux - CNCR

Thornton Phil Male Flagship Leader CCAFS

10


iea.cgiar.org


Van Son Duong Male Lecturer Thai Nguyen University

Van Trinh Mai Male Agricultural expert Institute for Agricultural Environment

Veeger Marieke Female Scientist

Universidad para la Cooperacion Internacional - UCI

Vermeulen Sonja Female Head of Research CCAFS

Vidal, PhD Alain Male Strategy Director a.i. & Senior Partnerships Advisor CGIAR Consortium

Vinh Bui Male n/a CIAT

Wassmann Reiner Male Coordinator of Climate Change Research IRRI

Whitbread Anthony Male Scientist ICRISAT

Wollenberg Lini Female Flagship Leader Univ. of Vermont/CCAFS

Zaidi PH Male n/a CIMMYT

Zougmoré Robert Male Regional Program Leader ICRISAT/CCAFS

11


iea.cgiar.org

ANNEX E: CCAFS RESEARCHER SURVEY – RESULTS

1. OVERVIEW

No of researchers Sent out (2 June) 407 Responses: 158 In % 38.8% Complete responses by mid July

128

In % 31.4%

Composition of researchers by home institution (total 407):

5

32

14

63

7

15

6 7

42

35

23

7

26

12

25

66

22

0

10

20

30

40

50

60

70

12


iea.cgiar.org

2. RESULTS

PART I: INTRODUCTION

Figure 1: QUESTION 1: Please indicate your home institution

Composition of researchers:

TOTAL SENT % TOTAL RESPONDED AfricaRice 5 1% 4 3% Bioversity 32 8% 7 4% CCAFS 14 3% n/a CIAT 63 15% 24 15% CIFOR 7 2% 5 3% CIMMYT 15 4% 7 4% CIP 6 1% 5 3% ICARDA 7 2% 3 2% ICRAF 42 10% 5 3% ICRISAT 35 9% 12 8% IFPRI 23 6% 7 4% IITA 7 2% 4 3% ILRI 26 6% 17 11% IRRI 12 3% 6 4% IWMI 25 6% 14 9% Partner 66 16% 31 20% WorldFish 22 5% 7 4% TOTAL 407 158

47

24

57

53

5

12

74

17

6

14

31

7

0

5

10

15

20

25

30

35

13


iea.cgiar.org

Figure 2: QUESTION 2: Please indicate your gender

Figure 3: QUESTION 3: What is your role within CCAFS?

Female, 34.8%

Male, 65.2%

Principal Investigator, 13,

8%

Activity Leader, 25, 16%

Project Leader, 26, 16%

Regional Program

Leader, 4, 3%Theme

leader/Flagship Leader, 4, 3%

Researcher, 48, 30%

Management, 6, 4%

Other (please specify), 32,

20%

14


iea.cgiar.org

Other:

Science Officer Focal Point Partner Contact Point

Communication Support Consultant

Figure 4: QUESTION 4: In what discipline/field is your highest level of academic education?

Figure 5: QUESTION 5: Since when have you been working with your current organization? Please indicate the year you joined

CONSOLIDATED 2011-2015 72 2009-2010 21 pre 2010 71

Agricultural science, 60,

38%

Climate science, 6, 4%Environmental

science, 25, 16%

Social sciences, 44, 28%

Other (please specify), 23,

14%

0

15

27

141615

6

12

57

4

96

31 1

31 0

20 0 0 1 0 1

4

0 1 0 0 0 1 0 0 0 1 1 0 10

5

10

15

20

25

30

https://www.surveymonkey.net/analyze/hWc7xtuhawCeXuxj_2F8Bn8UJnYgHFcAB8NW3JfESj2Po_3D







15


iea.cgiar.org

Table 1: QUESTION 6: In what country are you currently based?

COUNTRY Answers Kenya 23 Colombia 19 United States 17 India 15 Philippines 10 Indonesia 5 Peru 5 Benin 4 Costa Rica 4 Denmark 4 Ethiopia 4 Sri Lanka 4 United Kingdom 4 Australia 3 Malaysia 3 Mali 3 Uganda 3 Bangladesh 2 Burkina Faso 2 Germany 2 Italy 2 Laos 2 Morocco 2 Senegal 2 Vietnam 2 Zimbabwe 2 Canada 1 China 1 France 1 Ghana 1 Jordan 1 Mexico 1 Nepal 1 Netherlands 1 Niger 1 South Africa 1

PART II: YOUR INVOLVEMENT IN CCAFS

Figure 6: QUESTION 7: For which CRP(s) do you currently work? Please estimate the

16


iea.cgiar.org

proportion of your total working time spent on each CRP.

Figure 7: QUESTION 8: CCAFS is organized around four Flagships and a cross-cutting theme on gender. To which do most of your own research activities contribute? (Indicate only one.)

Figure 8: QUESTION 9: To which additional Flagships do you also contribute? (Indicate all

2

2119

13

6

14

73

9

3

43

0

5

10

15

20

25

30

35

40

45

50

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

74

16 18

39

40

1020304050607080

17


iea.cgiar.org

that apply.)

Figure 9: QUESTION 10: Which entity/organization do you primarily identify with?

52

37 3741 40 37

0

10

20

30

40

50

60

My home organization,

115, 72%

CCAFS, 36, 23%

Other, 8, 5%

18


iea.cgiar.org

Figure 10: QUESTION 11: How well do you know CCAFS?

16 17

35

52

20

11

0

10

20

30

40

50

60

I hav

e pa

rtic

ipat

ed in

desig

ning

CCA

FS P

hase

1

I hav

e pa

rtic

ipat

ed in

desig

ning

CCA

FS P

hase

2

I hav

e no

t par

ticip

ated

inde

signi

ng C

CAFS

but

but

am

quite

fam

iliar

with

the

prog

ram

stru

ctur

e,ob

ject

ives

, im

pact

…

I hav

e so

me

know

ledg

e of

the

prog

ram

stru

ctur

e,ob

ject

ives

, im

pact

path

way

s, ID

Os a

nd k

eyor

gani

zatio

ns in

volv

ed.

I kno

w o

nly

the

Them

es/F

lags

hips

that

my

wor

k co

ntrib

utes

to.

I kno

w v

ery

little

or n

othi

ngab

out C

CAFS

. I o

nly

know

abou

t the

rese

arch

I am

wor

king

on.

19


iea.cgiar.org

PART III: YOUR RESEARCH

Figure 11: QUESTION 12: What is your perception of the factors influencing the choice of research topics in the Flagship you mostly work for?

Figure 12: QUESTION 13: Regarding the balance among different kinds of activities in the CCAFS Themes you mostly contribute to, please indicate in percentages your perception of the current balance; and whether emphasis should be changed.

Note: For local research and adaptive research the current average was rated higher than for the other options.

3

4

5

3

5

11

15

2

1

7

5

13

21

25

8

10

16

21

24

28

26

26

28

35

36

22

26

24

51

53

36

43

44

21

19

43

32

32

19

20

12

7

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

CCAFS strategy

Flagship priorities

Expressed needs of clients or beneficiaries

Donor priorities/potential to attract funding

Researchers' own assessments of scientificinterest,priorities, needs, and knowledge gaps

Center strategy

Continuation of work that started before CCAFS begun

1 - no influence 2 3 4 5 6 - primary influence

0 5 10 15 20 25 30 35 40

“Blue sky”, high risk research

Strategic research to produce InternationalPublic Goods

Local research to produce feed-back andsynthesis for global research

Adaptive research

Scaling-out of research results

Extension

AVERAGE CURRENT

AVERAGE SHOULD BE

20


iea.cgiar.org

Figure 13: QUESTION 14: Are you familiar with the different funding sources of CCAFS (W1/2 and W3 and bilateral funding)?

Figure 14: QUESTION 15: CCAFS receives funding from different sources where the Windows 1 and 2 are of least restricted type. What is your view of how W1/2 funds are used in CCAFS?

Note: only answered by those people who had “YES” as a response for Question 14

Yes, 82, 57%

No, 61, 43%

2

0

3

2

3

1

8

3

7

10

14

17

18

14

17

19

15

16

16

9

9

16

14

13

12

22

10

20

16

19

11

16

16

14

13

14

18

11

15

9

17

7

14

8

6

2

4

2

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Increase integration between different areas of research

Increase gender relevant research

Open calls for competitive grants

Leverage bilateral funding

Fill gaps in research funding

Improve relevance through ex ante studies

Provide opportunities for long-term, high risk research

Provide accountability through ex post impact studies

1 - not at all 2 3 4 5 6 - main purpose

21


iea.cgiar.org

Figure 15: QUESTION 16: In your view, how well are the following aspects managed in CCAFS for enhancing effectiveness of research?

Figure 16: QUESTION 17: How well have you been trained/prepared in Theory of Change (ToC) and Impact Pathway (IP) thinking prior to the proposal development for Phase 2?

03132234

25

29

836678910

129

1415

1017

2222

2013

1725

261925

32

1732

232231

2825

2640

2735

24

403331

34333130

2322

2426

22

46262628282221

1711910

6

Communications and outreachInfuencing policy and decision makers

Impact pathway development at Flagship levelAnnual progress monitoring

Engagement with most appropriate partnersImpact pathway development at program levelImpact pathway development at Regional level

Feedback between regional programs and…Analysis of target groups, such as gender and the…

Strategic allocation of fundingCapacity enhancement activities

Feed-back to research from adoption and impact…

1 - not well at all 2 3 4 5 6 - very well

27

51

21

35

0

10

20

30

40

50

60

Very well Adequately Not so well I have not receivedany training in ToC

and IP thinking

22


iea.cgiar.org

Figure 17: QUESTION 18: To what extent have your research teams developed explicit impact pathways for your individual research projects?

Figure 18: QUESTION 19: In your view, what are the primary impact pathways for research through which CCAFS aims to have impact? Please score using a scale where 1=not important at all and 6=primary impact pathway.

29

56

43

6

0

10

20

30

40

50

60

To a high degree To a substantialdegree

To a modestdegree

To a negligibledegree

5

7

3

11

13

12

11

19

19

15

23

26

20

39

41

41

42

49

32

34

41

33

28

20

17

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Contribute to national planning processes related toagriculture, development and food system policies

Producing strategies for tackling food insecurity in the faceof climate change (IPG)

Work directly with food security and Climate Smart Agriculture (CSA) implementing agencies national …

Provide open data and information in the shortest possibletime, so that research partnerships and additional outputs…

Provide capacity enhancement to utilise agriculture-climateknowledge effectively

1- not important at all 2 3 4 5 6 - primary impact pathway

23


iea.cgiar.org

Figure 19: QUESTION 20: In your view, how effectively are the measures listed below managed in your Center/CCAFS for assuring and enhancing high quality of research?

1

1

2

2

2

1

3

4

9

7

7

10

14

12

16

16

23

24

29

22

15

23

23

31

33

40

35

35

44

34

28

29

35

32

29

29

27

32

29

23

20

14

18

14

15

13

19

12

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Availability and quality of research support staff

Knowledge management

Research data management

Availability and quality of technical facilities/…

Acceptance and encouragement of innovative …

Allocation of competences and appropriate skill…

Quality assurance processes such as internal peer…

Personal incentives from management, including…

1 - no attention at all 2 3 4 5 6 - very effectively

24


iea.cgiar.org

PART IV. PARTNERSHIPS, GENDER AND CAPACITY ENHANCEMENT

Figure 20: QUESTION 21: Please indicate how important different partners, as listed below, are for the work you do

Figure 21: QUESTION 22: In your view, to what extent are your partners in CCAFS involved in Program activities as listed below?

21242375453

8

454

716

1110

9101014

15

81213

1515

1821

2226

2230

27

262930

2828

382733

3636

3242

4437

4536

4341

3745334324

25

4344

3537

2217

2514

1812

1811

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

National governmentsOther CRPs or CGIAR centers

National research institutions in developing…Farmers and their organizationsDonor implementation agencies

Universities in developing countriesNational agricultural extension systems

Local NGOsUniversities and research organization in…

International NGOsPrivate sector

Civil society organizations

1- not important at all 2 3 4 5 6 - very important

0

3

2

2

4

6

2

6

6

8

10

7

9

10

9

9

6

22

16

26

13

21

26

21

24

28

32

23

32

26

30

34

37

23

38

36

40

19

33

30

49

35

22

24

19

34

20

13

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Conducting project activities

Planning of projects

Participate in building research capacity

Analyzing data and interpreting results

Feed-back from clients/users back to research

Research prioritization

Publishing research result

Reporting to managers and sponsors

1 - not involved at all 2 3 4 5 6 - very much involved

25


iea.cgiar.org

Figure 22: QUESTION 23: In your view, to what extent do the current partnerships increase the likely effectiveness of your research in areas listed below?

QUESTION 24: Please elaborate to what extent do these partnerships add value that outweighs the time and effort of managing the partnerships?

Open question

Figure 23: QUESTION 25: Please indicate your agreement with the following statements that relate to partnerships

1

3

1

3

5

3

4

2

2

6

6

11

12

11

9

9

13

14

22

36

41

29

28

22

28

40

36

31

40

41

42

49

30

24

29

47

43

44

27

20

17

10

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Increase the relevance of your research

Help in scale out and uptake of the research results

Increase the effectiveness of your research

Help in collecting data for research activities

Help in attracting funding

Increase the state-of-art in research methodology

Help in attracting new talents

1 - not at all 2 3 4 5 6 - very much

0

1

1

4

2

11

4

3

7

1

9

17

14

17

19

23

20

24

32

42

35

35

35

29

77

65

65

64

58

42

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Strategic partnerships are important to ensuring that…

It is important to develop innovative approaches to…

Partnerships are essential to capture the attention of policy…

Strategic partnerships are critical so that space is created for…

Participatory action research is a cornerstone to achieve…

The research agenda is predominantly co-designed and co-…

1- strongly disagree 2 3 4 5 6 - strongly agree

26


iea.cgiar.org

Figure 24: QUESTION 26: Please indicate your agreement with the following statements that relate to mainstreaming of gender issues in your work and CCAFS.

QUESTION 27: Please add any comment on gender mainstreaming in CCAFS and how it has effected your work:

Open question

2

4

6

3

5

9

5

12

8

31

10

7

10

8

15

15

17

19

12

19

16

14

16

12

25

23

23

26

25

14

17

25

19

31

31

16

31

24

24

18

29

31

25

28

27

25

22

24

17

12

53

41

29

7

11

16

11

10

4

20

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Attention to gender issues is important for our research…

Gender dis-aggregated data on results are collected…

Training and mentoring are equitably targeting men and…

The CCAFS gender and inclusion toolbox is a useful…

The CCAFS gender and equity strategy has been well…

There is sufficient funding to implement the gender-…

The CCAFS gender and equity strategy provides useful…

The CCAFS gender and equity strategy influences the…

The CCAFS gender and equity strategy influences the…

There is too much emphasis on gender in CCAFS


27


iea.cgiar.org

Figure 25: QUESTION 28: Please indicate your agreement with the following statements that relate to capacity enhancment (CE) in your work and in CCAFS.

QUESTION 29: Please add any comment on capacity enhancement in CCAFS and how it has effected your work:

Open question

1

1

1

3

9

4

10

1

5

5

16

7

10

9

6

16

18

19

17

25

20

18

26

25

26

17

35

28

41

48

45

41

33

20

25

55

22

23

15

15

9

4

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

CE activities are an essential element for delivering…

CE activities are designed to increase the capacity of…

CE activities are designed to increase the capacity of…

CE activities are well integrated into research

My research teams are now putting more emphasis on…

CCAFS is addressing institutional and organizational CE…

CCAFS is addressing CE needs of individuals very well.


28


iea.cgiar.org

PART V. CCAFS WORKING CONDITIONS

Figure 26: QUESTION 30: Please indicate how satisfied you are with the following working conditions for your work

6

8

10

6

16

8

7

12

19

22

47

13

15

17

22

23

25

24

26

19

26

27

30

29

27

27

26

27

32

28

35

34

27

38

34

27

34

18

28

33

28

14

22

11

29

28

28

25

20

23

21

21

16

9

3

12

14

14

9

21

10

6

12

20

10

8

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Amount of time spent on coordination among partners

Amount of time spent on travelling and meetings

The usefulness of the P&R platform to support me in…

Incentives for cooperating with non-CGIAR partners

Transparency in the allocation of W1/2 funding within CCAFS

Incentives for working across themes and disciplines

Incentives for cooperating with other CGIAR partners

Share of time for research compared to time for…

Fairness in the allocation of W1/2 funding within CCAFS

Reliability and predictabiliy of bilateral project funding

Reliability and predictabiliy of W1/2 funding

1 - very dissatisfied 2 3 4 5 6 - very satisfied

29


iea.cgiar.org

PART VI. CCAFS VALUE ADDED

Figure 27: QUESTION 31: Please indicate your agreement with the following statements related to the value the CCAFS has had or is likely to have influencing the success of your research compared to past Center-based implementation of the research.

QUESTION 32: Please add any comment on the value-added or negative value from research implementation through CCAFS that you have observed or expect in the future.

Open question

QUESTION 33: Please add any suggestions on what could be done differently.

Open question

3

3

3

3

3

2

2

3

2

6

8

8

7

14

8

8

11

13

12

20

11

20

12

16

24

20

26

22

13

28

21

19

34

23

24

21

34

39

29

28

37

25

24

22

21

33

23

23

26

24

18

16

12

11

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Research is becoming strategically better focused on…

CCAFS is creating or enhancing synergies between…

CCAFS has good potential to help streamline…

CCAFS is enabling research to be better aligned to…

CCAFS provides a better framework for guiding and…

Research is becoming better integrated across…

CCAFS has improved the way in which gender issues…

CCAFS has good potential to help streamline…

Due to CCAFS, capacity development is now…


30

Evaluation of CGIAR Research Program on CCAFSEvaluation of CGIAR Research Program on CCAFS

iea.cgiar.org

ANNEX F: BIBLIOMETRICS ANALYSIS CCAFS publication database:

Publication type 2010 2011 2012 2013 2014 Grand Total % of total CCAFS briefs, info notes 1 23 16 5 21 66 5% CCAFS report 1 5 3 4 6 19 2% CCAFS working paper 12 23 23 13 71 6% CCAFS manual, strategy, progr. doc 3 3 0% Conference 38 24 38 100 8% Journal article 35 65 101 165 108 474 39% Book and book chapter 9 26 24 32 16 107 9% Policy Brief 6 18 5 18 47 4% Report 1 41 13 55 5% Working paper 3 6 11 7 27 2% Other 4 114 13 57 47 235 20% Grand Total 60 310 261 362 211 1204

Source: CCAFS Annual reports.

Journal articles - citations

Citations 2010 2011 2012 2013 2014 TOTAL

0 1 4 6 13 21 45 1 -5 2 9 26 62 47 146 6-10 4 10 20 25 20 79 11-20 3 14 22 37 14 90 21-50 11 17 19 21 6 74 >50 13 7 6 6 0 32 TOTAL 34 61 99 164 108 466

Source: Google scholar.

Books and book chapters - citations

Citations 2010 2011 2012 2013 2014 TOTAL 0 3 6 9 4 22 1 -5 3 8 9 8 4 32 6-10 4 3 5 3 1 16 11-20 0 2 1 3 0 6 21-50 1 4 1 0 0 6 >50 1 1 0 1 0 3 TOTAL 9 21 22 24 9 85

Source: Google scholar. Ten most cited articles

YEAR TITLE Citations Lead institute 2010 Herrero M et al. (18 authors). 2010. Smart investments

in sustainable food production: revisiting mixed crop-livestock systems.Science 327 (5967): 822–825.

280 ILRI

31


iea.cgiar.org

2010 Thornton PK. 2010. Livestock production: recent trends, future prospects. Philosophical Transactions of the Royal Society Series B, 365 (1554): 2853–2867.

276 ILRI

2013 Asseng S. et al. (52 authors) 2013. Uncertainty in simulating wheat yields under climate change. Nature Climate Change 3: 827-832.

170 Agricultural and Biological Engineering Department, University of Florida

2010 Vermeulen S, Cotula L. 2010. Over the heads of local people: consultation, consent, and recompense in large-scale land deals for biofuels projects in Africa. J. Peasant Studies 37(4): 899–916.

168 CCAFS

2012 Vermeulen SJ, Campbell BM, Ingram JSI. 2012. Climate change and food systems. Annual Review of Environment and Resources 37:195-222.

141 CCAFS

2010 Sikor T. et al. (7 authors) E. 2010. REDD-plus, forest people’s rights and nested climate governance. Global Environmental Change 20 (3): 423–425.

113 School of International Development, University of East Anglia

2010 Brussaard L. et al. (8 authors). 2010. Reconciling biodiversity conservation and food security: scientific challenges for a new agriculture. Current Opinion in Environmental Sustainability 2: 34–42.

111 Diversitas AgroBiodiversity Science Committee, Dept. of Soil Quality, Wageningen Univ

2013 Vanbergen AJ et al. (40 authors). 2013. Threats to an ecosystem service: pressures on pollinators. Frontiers in Ecology and Environment 11, 251-259.

102 NERC Centre for Ecology & Hydrology, Edinburgh

2011 Varshney, R.K., Bansal, K.C., Aggarwal, P.K., Datta, S.K., Craufurd, P.Q. 2011. Agricultural biotechnology for crop improvement in a variable climate: hope or hype? Trends in Plant Science 16, 363-371

101 ICRISAT

2010 Thornton PK, Jones PG, Alagarswamy G, Andresen J, Herrero M. 2010. Adapting to climate change: agricultural system and household impacts in East Africa. Agricultural Systems 103(2):73–82.

93 ILRI

Source: Google scholar.

32


iea.cgiar.org

Most frequent journals JOURNAL TITLE No of articles Impact factor Category Environmental Research Letters 27 3.906 Environmental sciences PLoS ONE 15 3.234 Multidisciplinary sciences Global Environmental Change 14 5.089 Environmental sciences Water International 13 0.686 Engineering, civil Field Crops Research 12 2.976 Agronomy Agricultural and Forest Meteorology 11 3.762 Agronomy Climatic Change 11 3.43 Environmental sciences Agriculture, Ecosystems and Environment 11 3.402 Agriculture, multidisciplinary Nature Climate Change 10 14.55 Environmental sciences Current Opinion in Environmental Sustainability 10 3.491 Environmental sciences Regional Environmental Change 10 2.628 Environmental sciences Agriculture and Food Security 9 n/a Global Change Biology 9 8.044 Biodiversity conservation Proceedings of the National Academy of Sciences 8 9.674 Multidisciplinary sciences Agricultural Systems 8 2.906 Agriculture, multidisciplinary Mitigation and Adaptation Strategies for Global Change 8 2.669 Environmental sciences Food Security 8 1.495 Food science and technology Experimental Agriculture 6 1.079 Agronomy Plant and Soil 5 2.952 Agronomy Geoderma 5 2.772 Soil science Source: Scopus.

Journals with highest Impact Factor (higher than 6) Journal No IF Nature 1 41.46 Science 4 33.61 Nature Climate Change 10 14.55 Trends in Plant Science 1 12.93 Nature Communications 1 11.47 Proceedings of the National Academy of Sciences 8 9.674 Global Change Biology 9 8.044 Frontiers in Ecology and Environmen 1 7.441 Conservation Letters 2 7.241 Philosophical Transactions of the Royal Society - B 2 7.055 Plant Cell and Environment 1 6.96 Global Ecology and Biogeography 2 6.531 Remote Sensing of Environment 1 6.393 Journal of Hydrometeorology 1 6.182

Source: Scopus.

33


iea.cgiar.org

Co-authors with CCAFS 2010-2014

Institute No of articles published together University of Copenhagen 8 CIAT 6 CSIRO 6 ILRI 6 INRA 6 Bioversity International 4 Charles Darwin University 4 FAO 4 University of Leeds 4 University of Vermont 4 Wageningen University and Research Centre 4 IUCN 4 CIP 3 CIRAD 3 Open University 3 University of Oxford 3 University of Reading 3 World Agroforestry Centre 3

Source: Scopus.

CCAFS Team leaders (12) – H index

Name Role H INDEX SCOPUS Ana Maria Loboguerrero Regional program leader 1 Andrew J. Challinor Flagship program leader 24 Andy Jarvis Flagship program leader 19 Bruce Campbell Flagship program leader 28 James Kinyangi Regional program leader 15 James Hansen Flagship program leader 24 Lini Wollenberg Flagship program leader 4 Philip Thornton Flagship program leader 27 Pramod Aggarwal Regional program leader 20 Robert Zougmore Regional program leader 9 Sebastian Leocadio Regional program leader 0 Sonja Vermeulen Flagship program leader 11 Source: Scopus.

34


iea.cgiar.org

Spread of H index of CCAFS contact points, Flagship Program leaders and Regional Program Leaders

H index Number of researchers

0 3 1 to 5 3 6 to 10 5 11 to 20 10 24 to 28 4 39 1 TOTAL: 26

Source: Scopus.

Source: Scopus.

0

1

2

3

4

5

6

7

0 1 to 5 6 to 10 11 to 20 24 to 28 39

FP leader

RP leader

Center contact point

35


iea.cgiar.org

ANNEX G: REVIEW OF JOURNAL ARTICLES PROCESS:

• This component of the Quality of Science analysis was coordinated and managed by the IEA • CCAFS provided the evaluation team with a database of 1204 CCAFS outputs, of which 474

were classified as journal articles. • Since better alignment of publications with CCAFS research and objectives was expectd or

articles published more recently, it was decided to select the period of 2012-2014 for the qualitative assessment

• Ultimately the assessment should cover 30% of articles published between 2012 and 2014 • Out of the 374 articles (2012-2014) a random sample was taken and articles were classified

by four subject areas (see below) • Since economics and social science papers were comparatively less frequent, the sample for

economics papers was lower than for the other subject areas • Four external experts (all professors at Universities) were recruited to conduct the

assessment • Articles which were clearly not aligned or which were only published in a language not

spoken by the respective reviewer were excluded from the sample and the next one in the list was reviewed

• A template (see below) was used and additionally reviewers were asked to provide an overall assessment (1-2 pages) to the IEA

PEER REVIEWERS:

SUBJECT AREA Name of peer reviewer

Position INST

Agriculture, crop production Chris Atkinson Professor of Sustainable Agriculture and Climate Change, Department of Agriculture Health and Environment

Natural Resources Institute, University of Greenwich

Climate change issues John Roy Porter Professor Climate and Food Security, PLEN, Faculty of Science

University of Copenhagen AND Natural Resources Institute, University of Greenwich

NRM and environment Johan Six Professor, Sustainable Agroecosystems

ETH Zurich

Economics Baris Karapiner Assoc. Prof. Boğaziçi University Short profiles of peer reviewers:

Professor Christopher Atkinson joined the University of Greenwich from East Malling Research in late 2012. Previously he worked for Unilever Research and for the Agriculture Food Research Council at Rothamsted

36


iea.cgiar.org

Research Station, in Harpenden (1971–76). Professor Atkinson's research interests focus on understanding the impacts of environmental stress on the growth and development of crops, with particular expertise in perennial woody crops.

Professor John Roy Porter is an internationally known scientist in crop ecology and physiology, biological modelling and agricultural ecology. Main contribution has been multi-disciplinary and collaborative work in the response of arable crops, energy crops and complex agro-ecosystems to their environment with an emphasis on climate change and ecosystem services. He has published more than 100 papers in reviewed journals out of a total of about 350 publications.

Professor Johan Six received his PhD in Soil Science in 1998 from Colorado State University. He recently took up a chair position in Sustainable Agroecosystems at ETH-zurich, where he will continue his research program developed at UCDavis, but with more of an emphasis on landscape analyses and global Food Security. He led and was involved in many projects investigating the effect of land use change and management on greenhouse gas fluxes in agricultural, grassland and forest ecosystems.

Dr. Baris Karapinar is the Managing Director and member of the Executive Board of the TEMA Foundation, Turkey's biggest environmental NGO

37


iea.cgiar.org

SAMPLE: No of journal articles

assessed STATUS

Agriculture, crop production (AGR) 34 completed Climate change issues (CC) 33 completed Economics (ECO) 16 completed Natural resource management (NRM) 32 completed TOTAL 115

FRAMEWORK USED

Criterion Assessment approach 1. Methodological rigor and coherence of data analysis

Rating Scale 1=poor 2=mediocre 3=good 4=excellent

2. Originality, innovativeness Rating Scale 0 =not applicable 1 = no originality 2= standard methods, established knowledge 3= rather original 4= very original, new research, analytical or theoretical concepts

3. Referencing (whether referencing is up to date, balanced across relevant disciplines, indicating that the publication takes account of earlier work)

Rating Scale 1=referencing is poor 2= referencing is limited 3= referencing is good 4=referencing is excellent

4. Do the results (knowledge) presented in the paper represent broadly applicable knowledge (International Public Goods) relevant to agriculture and climate change?

Rating Scale 0= results not relevant to agriculture and climate change 1=no broader applicability (local relevance only) 2= potentially broader applicability, but not spelled out 3= broader applicability is presented 4= significant international applicability

5. Quality (and appropriateness) of publication venue

Observation of low-quality or inappropriate venue relative to subject and quality of paper

6. Co-authorship Observation of extent of co-authorship, with whom and is it appropriate?

7. Overall quality of publication (including additional criteria at evaluator discretion)

Brief overall assessment (around 100-150 words)

38


iea.cgiar.org

SCORING RESULTS 1. Methodological rigor and coherence of data analysis

RATINGS AGR % CC % ECO % NRM % TOTAL 1=poor 0 2 6% 5 31% 1 3% 8 2=mediocre 8 24% 9 27% 3 19% 5 16% 25 3=good 14 41% 15 45% 6 38% 11 34% 46 4=excellent 12 35% 7 21% 2 13% 15 47% 36 TOTAL 34 33 16 32 115

2. Originality, innovativeness

RATINGS AGR % CC % ECO % NRM % TOTAL 0 =not applicable 0% 2 6% 1 6% 0% 3 1 = no originality 0% 1 3% 5 31% 2 6% 8 2= standard methods, established knowledge 17 50% 18 55% 6 38% 8 25% 49 3= rather original 17 50% 6 18% 4 25% 18 56% 45 4= very original, new research, analytical or theoretical concepts 0% 6 18% 0% 4 13% 10 TOTAL 34 33 16 32 115

3. Referencing (whether referencing is up to date, balanced across relevant disciplines, indicating that the publication takes account of earlier work)

RATINGS AGR % CC % ECO % NRM % TOTAL 1=referencing is poor 2 6% 0% 0% 2 6% 4 2= referencing is limited 6 18% 4 12% 8 50% 5 16% 23 3= referencing is good 16 47% 22 67% 6 38% 14 44% 58 4=referencing is excellent 10 29% 7 21% 2 13% 11 34% 30 TOTAL 34 33 16 32 115

39


iea.cgiar.org

4. Do the results (knowledge) presented in the paper represent broadly applicable knowledge (International Public Goods) relevant to agriculture and climate change?

RATINGS AGR % CC % ECO % NRM % TOTAL 1=no broader applicability (local relevance only)

0% 9 27% 0% 2 6% 11 2= potentially broader applicability, but not spelled out 9 26% 16 48% 3 19% 4 13% 32 3= broader applicability is presented 11 32% 4 12% 9 56% 17 53% 41 4= significant international applicability 14 41% 4 12% 4 25% 9 28% 31 TOTAL 34 33 16 32 115

Evaluation of the CGIAR Research Program on Climate Change, … · 2019-11-06 · This evaluation...

Documents

Transcript of Evaluation of the CGIAR Research Program on Climate Change, … · 2019-11-06 · This evaluation...