IMPROVING INSTITUTIONS FOR PRO-POOR GROWTH

A DFID Research Programme Consortium

MID-TERM REVIEW

August 2009

Neil McCulloch Institute of Development Studies

Candida Blaker

North South Consultants Exchange

This report has been prepared for the Department for International Development by Neil McCulloch, Consultant supplied by Institute of Development Studies and Candida Blaker, Consultant supplied by North South Consultants Exchange through the Governance and Social Development Resource Centre Framework. The views expressed herein are those of the authors and do not necessarily represent the view of Coffey International Development, the consortium members of GSDRC or DFID.

Contents

A Executive Summary 1

B List of Recommendations 2

1 Introduction 5

2 Review of Purpose 6

3 Review of Outputs 11

4 Communications and Policy Influencing 14

5 Capacity Development 24

6 Gender 29

7 Partnership Management and Effectiveness 31

8 Links with DFID and Related RPCS 36

9 Conclusions and Overall Recommendations 39

Annexes

A Terms of Reference for the MTR

43

B People Consulted during the MTR

50

C List of Research Outputs

51

D Other Documents Consulted during the MTR

54

E Capacity Building with iiG Research Partner Organisations

55

F iiG–Funded Projects (spreadsheet of all projects, status and funding)

58

G Proposed Changes to the Logframe

63

H Project Scoring Annual Review

69

GLOSSARY

ACEHS African Centre for Economic and Historical Studies, Ethiopia BRAC Bangladesh Rural Advancement Committee CSAE Centre for the Study of African Economies, University of Oxford EPRC Economic and Policy Research Centre, Uganda iiG Improving Institutions for Pro Poor Growth in Africa and South Asia,

Research Programme Consortia IPAR Institute of Policy Analysis and Research, Kenya ISEC Institute for Social and Economic Change, India IPPG Institutions for Pro Poor Growth, Research Programme Consortia LSE London School of Economics and Political Science QEH Queen Elizabeth House, University of Oxford STICERD Suntary-Toyota International Centre for Economics and Related

Disciplines, London School of Economics and Political Science UI University of Ibadan, Nigeria

1

A EXECUTIVE SUMMARY Positives

The iiG is successfully applying the latest economic methods, notably those of impact evaluation, to a range of institutional innovations in order to determine their impact on the inclusion of the poor in growth processes.

The programme has been good at generating extremely high quality academic outputs and at ensuring a high level of methodological rigour in its work.

Considerable methodological capacity on impact evaluation has been built in certain key partners.

iiG have also created strong connections and ownership by key stakeholders, including NGOs and government ministries, through their approach of working with strongly motivated partners well connected with the policy process.

Negatives

It is not clear how some of the projects undertaken are linked the central theme of the RPC – understanding how institutions can be improved to promote pro-poor growth.

The programme has struggled with difficult relationships with a couple of its core partners. In addition, a significant number of research projects have been unsuccessful and have therefore been stopped.

The programme has significantly underinvested in communications work to date although this is now changing rapidly.

The programme is also significantly under spent, although this situation may change as projects reach the higher spending period of their cycle.

Institutional capacity building has not been a focus of the programme. Areas Requiring Action

The programme needs to clearly articulate and effectively communicate how the work that it is doing is related to the central theme of the RPC, and develop a stricter set of criteria for future project selection.

iiG‟s management needs to adjust the composition of core partners and involve them more in the management of the programme.

Significantly more resources need to be spent on communicating the key ideas of the programme to maximise the policy impact of the research outputs.

Conclusion

The iiG programme is generating high quality innovative research on the connections between a range of institutions and pro-poor growth. Going forward it needs to clarify how its work answers critical questions relating to pro-poor growth and engage more systematically in communicating the key messages from the programme to a wide range of stakeholders.

2

B LIST OF RECOMMENDATIONS For iiG R1. Make the theories which are being tested by the research more explicit and

show how testing them provides information which will be useful for the design of institutions that promote pro-poor growth.

R2. Translate the conceptual framework into a set of clear criteria for project

selection (and exclusion) and use these to narrow down and make more coherent the range of activities undertaken by the programme.

R3. Start to outline, now, the big messages coming out from the research program

and attempt to ensure greater complementarily between the projects under each theme to facilitate drawing out these messages in the second half of the programme.

R4. Devote more resources to the Institutions for Manufacturing Exports theme

and in particular the exploration of which types of institutional functioning‟s matter most for growth.

R5. Put more effort into documenting, disseminating and encouraging the use of

the datasets which have been collected. R6. Place a stronger emphasis on comparative work, drawing on common

elements across the programme. R7. Foster stronger linkages between the research team in Oxford and the team

in LSE to encourage more sharing of ideas, approaches and the generation of comparative work.

R8. Encourage the active involvement of political scientists that understand the

value of economic approaches to institutional analysis in the programme, in order to draw on their knowledge of the broader political science literature on the issues of relevance to the programme.

R9. Alter logframe „output‟ to make it more possible to track, measure and report

on communications outcomes. R10. Update iiG communications strategy, making a distinction between iiG and

CSAE strategies R11. Discuss Communications strategy and share successes among RPC

members at the 6-monthly meetings each year. R12. Develop selected project communications strategies, in collaboration with

southern counterparts. R13. Roll out training on Communications, including writing Policy Briefs, to the iiG

research team, north and south.

3

R14. Devote significantly larger resources to communications activities in the

second half of the program, including through local partners, ensuring that Communications accounts for (at least) 10% of the total spend, as stipulated by DFID for RPCs..

R15. Broaden the outreach base for policy-relevant results. For example, make

systematic use of DFID‟s Communications Corner to share experience of advocacy, and send Policy Briefs to the most appropriate of the 17 programmes DFID funds to help get research into use.

R16. Clarify the distinction between iiG – IFG and IPPG on the R4D website.

(Recommendation for DFID/R4D) R17. Undertake capacity building of researchers and staff in RPC consortium in

advocacy and media communications R18. Link between iiG communications officer and people responsible for

communications in southern partner organisations to discuss capacity building needs and opportunities.

R19. Explore ways to extend and build on institutional capacity development work

in order to maximise impact and leave a legacy for the southern partner institutions.

R20. Devise and implement a plan to maximise opportunities for south-south exchange and inter-partner learning.

R21. Identify more clearly what resources are allocated to capacity development,

and increase the overall spend. R22. Encourage the systematic examination of gender issues throughout the three

research themes. R23. Consider producing a synthesis of the lessons about the relationship between

institutional design and the welfare of gendered groupings from the various projects which examine different aspects of this issue.

R24. Seriously consider whether continued investment in some Southern partners

partnerships will actually deliver commensurate outputs. If not, the programme should formally end relationships with Southern partners that are not delivering results.

R25. Do more to facilitate learning, information sharing and comparative work

across its Southern partners and more broadly amongst the programme members.

R26. Construct a small formal management committee chaired by the Director and

involving at least two permanent Southern partners and meet (virtually) regularly to discuss the management of the programme.

4

R27. Formally change the Directorship of the programme to Prof. Stefan Dercon. R28. Keep oversight of the programme with the Oversight Committee, but set up a

small informal Advisory Group specific to the iiG programme (akin to the non-functioning US Advisory team but involving non-US people too) to provide in-depth strategic advice once per year. A representative of DFID should be on this Advisory Group.

R29. Set up systems for the systematic collection of the indicators given in the

logframe to ensure that the programme is capable of demonstrating its success.

R30. The iiG should contact the Directors of the IPPG and arrange for a joint

workshop to encourage mutual learning from the two programmes. R31. iiG and IPPG should both put links on their websites linking to each other with

agreed text delineating the different focae of the research programmes. For DFID R32. DFID need to take seriously the impact of constantly changing staff on their

RPCs. There is always a high level of turnover within DFID positions for understandable institutional reasons. However, it would be within the power of DFID to require staff taking up Advisor positions in the research Department to stay in the same position for at least 2 years (e.g. by making them ineligible for other posts with that period). To compensate for this disincentive to take research Advisor positions, there could be an element of preference for subsequent advancement associated with the positions, or some other form of incentive. This is a complex institutional issue – but DFID should address it since their ability to shape and to obtain the maximum benefit from their large research investments is being affected by the constant staff changes.

R33. DFID should encourage RPCs to contact relevant DFID country offices

directly and should facilitate initial introductions. All RPCs should be informed of this.

R34. DFID should tidy up the confusion between iiG and IPPG on the R4D website. R35. DFID should request the IGC Co-Directors to produce and disseminate clear

written rules about the process for applying to the IGC for research funding and the procedures that will be followed in making decisions about funding, to preclude any suggestion that iiG (or any other researchers) will have preferential treatment in the allocation of research funding.

5

1. INTRODUCTION

1.1. The Improving Institutions for Pro Poor Growth in African and South Asia (iiG)

is a DFID-funded Research Programme Consortium (RPC). It was launched in January 2007 and was allocated £2,749,891 by DFID until December 2011.

1.2. The iiG is the second RPC to address the Pro Poor Growth theme identified in

DFID‟s Research Funding Framework, 2005-2007. The other is Institutions for Pro Poor Growth (IPPG), co-managed by University of Manchester and University of York.

1.3. The iiG aims to contribute to a new body of policy-relevant research on

institutional reforms to promote pro-poor growth in South Asia and sub-Saharan Africa. It is premised on the growing evidence that the impediments to generating pro-poor growth in South Asia and Sub-Saharan Africa are institutional in nature, and that social, legal, economic and political institutions affect the pattern of investment and growth and the extent to which the poor participate in that growth.

1.4. The RPC focuses on Southern Asia (India and Bangladesh) and Western and

Eastern Africa (with partners in Ethiopia, Kenya, Nigeria and Uganda), though some related research projects are also underway in Tanzania, Liberia, Sudan, and Ghana.

1.5. The RPC is formally directed by Prof. Paul Collier of the Centre for the Study

of African Economies (CSAE) at the University of Oxford. The collaborating Institutes from the UK are the Suntory-Toyota International Centre for Economics and Related Disciplines (STICERD) at the London School of Economics (LSE); Queen Elizabeth House (QEH), University of Oxford. The Southern formal partners are: the Institute of Policy Analysis and Research (IPAR), Kenya; Bangladesh Rural Advancement Committee (BRAC); Economic and Policy Research Centre (EPRC), Makerere University, Uganda; Department of Political Science, University of Ibadan (UI), Nigeria; Institute for Social and Economic Change (ISEC), India; and the Africa Centre for Economic and Historical Studies (ACEHS) in Ethiopia.

1.6. Some of the projects in the iiG programme receive considerable co-funding

from a range of other foundations and donors, including the William and Flora Hewlett Foundation, and the Open Society Institute, and several of the projects that the iiG is evaluating are funded by donors, including DFID.

1.7. The objective of this Mid-Term Review (MTR) is to measure and report on

performance to date and to indicate adjustments that may need to be made to ensure the success of the research programme. The ToR for the MTR is in Annex 1.

6

2. REVIEW OF PURPOSE

2.1. The iiG‟s purpose is “to provide governments, civil society in Africa and Asia,

and development agencies with usable and useful knowledge on how institutions can be redesigned to be more conducive to growth and empowerment.” This is an extremely relevant and important purpose.

2.2. To assess the extent to which the programme is likely to achieve this purpose,

it is useful to review the conceptual foundations of the RPC and how these have evolved over time.

2.3. The original conceptual narrative for the RPC was based on the idea that

different geographical and endowment characteristics of countries (coastal/land locked, natural resource rich/poor, good/bad neighbours) influence their long-run growth. This conceptual approach drew on the idea of “growth traps” in Collier‟s Bottom Billion book.

2.4. The iiG website still features Collier‟s Bottom Billion work prominently (even

though it was not funded by iiG). However, most of the work undertaken by the programme does not follow the conceptual narrative suggested by both the book and the original proposal. The original narrative may have had some influence upon the initial selection of countries in which to work, but, other than that, appears to have played no significant role.

2.5. The original proposal to DFID did however, outline four research themes:

institutions and growth (focussing on the analysis of micro-data from firms); institutions for accountability and empowerment (focussing on different types of institutions for accountability and public service delivery); institutions for property rights and contracting (focussing on the impact of legal, financial, and social institutions on investment, growth and poverty reduction); and institutions for regulation and trade (focussing on the regulation of competition and trade). The programme has been reasonably faithful to this original narrative which has focussed on the microeconomics of institutions which mediate the ability of the poor to participate in growth.

2.6. As the programme has developed, these have evolved into three themes of

work, as reflected in the Inception Report: Institutions for Manufactured Exports Institutions for Accountability Institutions for Rural Transformation and Development A full list of all the projects currently being undertaken by the programme, broken down by research theme is included as Annex 4.

Institutions for Manufactured Exports 2.7. The Institutions for Manufactured Exports theme has already produced some

very high quality outputs. For example, Collier and Venables work on how European trade preferences might be redesigned to encourage manufacturing exports from Africa is both innovative, and appears to have had an influence on EU policy. In addition, iiG has funded some of Collier‟s

7

time to produce papers which continue some of his previous work looking at the explanations of the natural resource curse, as well as the role of aid in mitigating external shocks.

2.8. The iiG website states that the work on Manufactured Exports is being taken

forward to explore the extent to which manufacturing exports are becoming clustered by task. In addition, work will examine the determinants of export success. Specifically it is intended to examine the extent to which export success is related to five functions provided by institutions: price regulation; predation; contract enforcement; public service delivery; and network services. This work is clearly and directly related to the aim of the RPC (to look at the role of institutions in generating pro-poor growth), but, to date, no outputs have been produced in this area. The programme also plans further research to examine the institutional foundations for the provision of these functions and how the institutional forms created are shaped by historical and political circumstances.

2.9. The Institutions for Manufactured Exports theme also intended to look at

Export Processing Zones in Africa. Unfortunately this work has been abandoned due to the unavailability of the relevant data. Separate work is ongoing, however, looking at the impact of the ethnic violence in Kenya on flower exports.

2.10. In addition, Robin Burgess and co-authors have produced a very high quality

and influential paper on how state-level labour regulations mediated the influence of de-licensing reforms in India.

2.11. Notwithstanding the excellent quality of the papers which have been produced

under this theme, it would appear that both of these pieces of work were continuations of existing work with minimal resource input from the iiG programme. Since this theme is the one most directly linked to economic growth, it is disappointing that so few of the iiG‟s resources have been devoted to this area. Only 4 of the 30 projects underway as part of the iiG programme fall under this theme (including the closed EPZ project).

2.12. It should be noted, however, that one of the reasons for iiG placing less

emphasis on this programme was because, as iiG was developing this work programme in late 2007, UNIDO became interested and subsequently funded several CSAE staff to work on this theme under the leadership of Paul Collier and John Page for their Industrial Development Report 2009. Since this work was being done for UNIDO, iiG felt it more appropriate to focus its resources elsewhere. Nonetheless, some of the background papers from the Industrial Development Report may be developed further for iiG.

Institutions for Accountability/Institutions for Rural Transformation 2.13. The work in the Accountability and Rural Transformation themes focuses less

on country-level growth and more on understanding the success or failure of different institutional interventions.

8

2.14. Almost half of the projects being undertaken by the iiG come under the theme of Institutions for Accountability. Accountability has been defined broadly by the iiG to include both political accountability (e.g. work on political violence in Nigeria) as well as evaluating alternative contractual arrangements for service delivery (e.g. alternative approaches to school governance in Kenya and Uganda).

2.15. The Institutions for Rural Transformation theme, by contrast, has a stronger

focus on understanding the role of social norms and values in perpetuating poverty (e.g. how psychological defeatism creates poverty traps in Ethiopia, the effectiveness of the ultra-poor program in Bangladesh, or the ability to raise self-esteem among adolescent girls in Uganda and Tanzania). In addition, it incorporates a range of projects which have a strong rural focus (e.g. water in Ghana), even if they are not primarily about social norms and values.

2.16. The Annual Report for July 2007-September 2008 probably is most accurate,

when it classifies the nature of the microeconomic work done by the iiG as explorations into three areas: social norms and values, political economy and accountability; and contract theory for service delivery. Almost all of the projects being undertaken in these two themes could be categorised into these three areas.

2.17. However, the shifting classifications of the work undertaken between

proposal, inception report, and Annual Report has led to some confusion about what the programme is actually about, and a concern, in DFID at least, that it is losing its focus on growth. Certainly, the iiG‟s conceptualisation of “growth” is very different from a traditional macro or sectoral approach, focussing much more strongly on learning about how different institutional interventions can influence the willingness to invest, the capacity to invest (through better health and education), and the ability to influence public policy (through stronger political accountability). The assumption is that such improvements aggregate up into stronger and more pro-poor overall economic growth.

2.18. In practice, the single unifying factor across the majority of projects is not, in

fact, thematic or conceptual, but methodological. Most of the large projects have been chosen because they allow the application of the latest scientific approaches to programme evaluation, while the specific institutions and issues examined have been determined primarily by existing relationships with key actors in the selected countries and their enthusiasm to participate in the implementation of a rigorous approach to evaluation.

2.19. It should be noted that this approach has some advantages: it ensures high

and consistent quality in the methodological approach to institutional problems; it encourages research collaboration between Northern and Southern partners that are substantive; and, because in many cases it is evaluating major NGO and government programs, it automatically creates direct policy influence. However, although iiG say that projects have been rejected because of their lack of connection with pro-poor growth, the

9

impression remains that the projects are picked without sufficient consideration of their relevance to growth (pro-poor or otherwise). The project list reads as, and to some extent probably is, a set of activities where an opportunity arose to learn something in a methodologically rigorous fashion about a theory that the researcher happened to be interested in. Although all good research programmes should encourage such exploratory work, the iiG should elaborate more clearly which core theories, when tested by the research, are likely to tell us most about how to build better institutions for pro-poor growth, and to make clear how each piece of work relates to these core theories.

2.20. The evolution of the programme described above has a direct bearing upon

the extent to which it is likely to achieve its purpose. It is clear that the outcome of the microeconomic work of this RPC will be a much deeper and more rigorous understanding of the efficacy of a variety of interventions in shaping institutions which we know to be important for growth e.g. social norms, electoral behaviour and violence, the quality of service delivery. In this sense it will achieve its purpose by providing “usable and useful knowledge as to how institutions can be redesigned so as to be more conducive to growth and empowerment.” We regard the likelihood of the programme achieving this purpose as a 2 using the DFID Scoring System.

2.21. But the microeconomic work of the RPC is not designed to identify which

types of institutional functioning‟s are most important for growth in any given context. It would be good if the macroeconomic work of the programme could focus more on providing some insights on this. The Accountability and Rural Transformation work will have a bigger impact on understanding the institutions important for empowerment, as well as the types of interventions which are more successful for ensuring the inclusion of the poor.

2.22. More generally, it will be important for the iiG to do more to ensure that the

overall research programme of the RPC is more than the sum of its parts. This would involve further efforts to show how the individual research projects under each theme complement one another, in order to provide a coherent set of messages about the relationship between different kinds of institutions (economic, political, service delivery, or social) and pro-poor growth.

2.23. Significant elements of the programme are also likely to continue to achieve

the purpose of the programme after the end of the grant period. This is particularly true of the major evaluations (of educational governance in Kenya/Uganda, health insurance in India, and empowerment programmes through BRAC) where there is strong ownership and significant funding for these activities from the government or NGO involved. The likelihood of continued immediate policy influence for many of the smaller projects once funding is finished is significantly lower, although the results of the research may continue to feed into policy and practice through the work of the NGOs involved, and some may make important intellectual contributions which may change the way in which future policy is framed.

10

Recommendations R1. Make the theories which are being tested by the research more explicit and

show how testing them provides information which will be useful for the design of institutions that promote pro-poor growth.

R2. Translate the conceptual framework into a set of clear criteria for project

selection (and exclusion) and use these to narrow down and make more coherent the range of activities undertaken by the programme.

R3. Start to outline, now, the big messages coming out from the research program

and attempt to ensure greater complementarity between the projects under each theme to facilitate drawing out these messages in the second half of the programme.

R4. Devote more resources to the Institutions for Manufacturing Exports theme

and in particular the exploration of which types of institutional functionings matter most for growth.

11

3. REVIEW OF OUTPUTS

3.1. The logframe for the programme describes 4 outputs:

a) New comparative and context specific quantitative data sets on institutions b) Country-specific research insights on links between institutions, growth

and poverty, quantitatively validated and with specific recommendations for institutional and policy design

c) Lessons from comparative research across countries, quantitatively validated, on the links between institutional design, development policies, growth and poverty, with specific recommendations on institutional design issues

d) An up-to-date research agenda reflecting changing policy priorities on institutions and development

3.2. To date, the iiG RPC has produced 10 published articles, 13 Working papers,

5 Briefing Papers, 4 Reports, and 5 presentations. In addition the programme reports four further presentations, four podcasts and one book, relating to Prof Paul Collier‟s book „The Bottom Billion: Why the Poorest Countries are Failing and What Can be Done About It.‟ although this was not funded by the programme. The programme has therefore been quite successful in producing high quality academic economic outputs.

3.3. Another area of success has been the production of context specific

quantitative datasets. The programme‟s strong focus on impact evaluation methodology, has led to the design and fielding of several new surveys, in many cases with more than one round. Understandably, work so far has focussed on the design and collection of data and the subsequent analysis and write up. As a result, only one of these datasets has been made publically available so far, nor have there been any efforts yet to disseminate and encourage use of the datasets beyond the project consortium (although several PhD students are now using the data). This may be an area in which the RPC may wish to focus more effort in the second half of the grant period.

3.4. The programme has produced fewer comparative datasets, since most of the

microeconomic research is strongly dependent on the particular issues and interventions of each country. However, there are exceptions – for example, the educational interventions in Kenya and Uganda have been designed together allowing for comparison; similarly the programme on empowering adolescent girls in Tanzania and Uganda are designed to provide comparative results.

3.5. The programme is well on the way to producing a wealth of country-specific

research insights on the efficacy of particular types of institutional innovation. As noted above, the links between these insights and growth are somewhat weaker than the linkages with poverty reduction. However, the results so far give a reasonable confidence that most of the research projects will give useful and specific recommendations for the institutional design of a wide range of interventions.

12

3.6. The RPC also aims to produce specific recommendations on institutional design issues from comparative research across countries. As noted, this has not been so much of an emphasis for the programme. Most of the cross-country work has come from the macro programme and is focussed on trade preferences, or on the relationship between commodity dependency, institutions and growth. There is also useful sub-national comparative work, notably the work on how differences in state-labor regulations have mediated the impact of de-licensing on state level performance. However, aside from these few projects, the programme has not yet generated specific recommendations from comparative work. Since some of the projects are clearly capable of generating useful comparative results, it would be nice to see more effort in this area going forward.

3.7. One way in which such comparative work might be encouraged would be

through encouraging stronger connections between the work undertaken by Oxford and that being done by LSE. Although the teams are aware of each others‟ work and meet formally every 6 months at the programme meetings, there do not currently appear to be any systems set up to encourage sharing and learning across the streams of work led by the two institutions. It may be useful for the programme to consider ways of increasing communication across these streams of work. We note that one iiG researcher, closely involved in the work in India is becoming a British Academy post-doc, and will be dividing his time between Oxford and LSE. Hopefully this will facilitate this process.

3.8. The logframe points to the creation of a up-to-date research agenda reflecting

changing policy priorities on institutions and development. Here the programme has been quite successful. There is obvious excitement and enthusiasm for the research agenda among those involved and its constant evolution, although posing problems of focus and consistency, is also evidence of the way in which the research agenda is being shaped by and, in turn, affecting policy priorities on institutional issues.

3.9. The reviewers have read a selection of outputs from the 30 projects currently

underway in the iiG. Most of these studies have been undertaken with scientific rigour, and the quality of the outputs is generally high. They add value to existing knowledge and, in some cases, provide valuable evidence about the underlying behavioural theories which have broader relevance for the design of policy.

3.10. One characteristic of the outputs is, however, striking. They are all from the

same school of quantitative economic analysis. This is by design – indeed having an unashamedly strong disciplinary and methodological focus for the RPC may have assisted in producing high quality economic work. It does however, leave the programme open to the criticism that the relatively narrow methodological focus on quantitative identification of impact (whether through randomisation or instrumental variable approaches) precludes the possibility of investigating phenomena that are not amenable to such approaches.

13

3.11. In particular, a significant component of the programme tackles issues associated with the institutions of political accountability. There is a huge political science literature on this topic, but most of the outputs of the programme barely mention it. This is slightly surprising since one political scientist has been involved from the outset in the work on Kenya, and a couple of other researchers have a background in politics as well as economics. The programme has also drawn on a political scientist as an international advisor. It would appear that the programme is already addressing this apparent weakness, having recently hired two researchers with political science PhDs. would add depth to the political analysis undertaken without affecting the overall methodological approach of the programme.

3.12. Overall we assess the likelihood of achieving the outputs in the grant period

as a 2 using the DFID Scoring System. Recommendations R5. Put more effort into documenting, disseminating and encouraging the use of

the datasets which have been collected. R6. Place a stronger emphasis on comparative work, drawing on common

elements across the programme. R7. Foster stronger linkages between the research team in Oxford and the team

in LSE to encourage more sharing of ideas, approaches and the generation of comparative work.

R8. Encourage the active involvement of political scientists that understand the

value of economic approaches to institutional analysis in the programme, in order to draw on their knowledge of the broader political science literature on the issues of relevance to the programme.

14

4. COMMUNICATIONS AND POLICY INFLUENCING

Achievement of Communications Outputs 4.1. The logframe contains a set of Outputs directly related to communications and

policy influencing: a. Accessible body of research output for skilled policy makers on the

themes of the RPC b. Accessible, non-technical output based on the research on institutions,

growth and poverty c. Web-based accessible information on the research of the RPC as a

platform for user-friendly engagement with media, policy-makers and civil society

Progress has been made on the achievement of all of these Outputs. 4.2. A range of research outputs has been developed, aimed both directly and

indirectly at policy makers and other stakeholders. According to R4D‟s target audience classification, 17 of the 21 listed outputs are considered to be relevant to or aimed at policy. An equal number fall within „academic‟, five „popular‟ and four „miscellaneous‟. Aside from academic papers, the outputs include Policy Briefs, a website, podcasts and a blog. As far as we are aware, other popular media – such as radio and stories - have not yet been exploited for in-country communications outputs.

4.3. The iiG website contains clear information on the programme‟s aims and

research themes. The website was substantially updated in May 2009, with the introduction of a number of additional publications and working papers. The site is accessible to low bandwidth users. It is all in English, as this is the main language of the research countries.

4.4. The Annual Report cites 4 Policy Briefs, but by May 2009, iiG had drafted a

series of 8 such Briefs which highlight the policy relevance of a number of research projects. Some of the forthcoming Policy Briefs will draw comparisons across projects, while others will focus on particular local contexts. They are aimed at unspecified range of policy makers and media relevant to each, and intended to be topical: they will be released to coincide with relevant influencing opportunities. The early drafts given to the MTR reviewers tended to contain more information than necessary on the information-gathering process. Following a training workshop in May 2009, these were revised so as to be more fit-for-purpose.

Policy Influence and Stakeholder Engagement 4.5. One strength, from a policy influencing perspective, of the program‟s focus on

microeconomic evaluation, is that, because it is explicitly associated with evaluating interventions, it is very closely linked to immediately relevant operational work. Moreover, because the work entails a detailed understanding of the context in which such interventions are implemented, the research programme has become more embedded within and achieved a much greater sense of ownership from operational stakeholders (governments and NGOs) than is typical of most research programmes. As a

15

result it is likely that, at least some elements of research will be have a direct and substantial influence on institutional/program policy design and implementation in specific countries and sectors.

4.6. The RPC does enjoy close engagement with national policy makers in several

countries. The degree to which this happens is variable, and depends on the partner, often flowing from their pre-existing relationship or reputation. According to the iiG programme bid, partners were chosen partly on account of their policy engagement. This is certainly the case with the RPC partners in Uganda, Kenya, Bangladesh and India.

4.7. Beyond this, national and local policy makers are sometimes closely involved

with the research implementation process itself. The workshop in September 2008 for Kenyan Members of Parliament, representatives of civil society and academics to discuss findings of the 2007 Elections highlighted the collaboration with the Kenyan Education Ministry over the Local Accountability Under Free Primary Education project. This government partnership is seen by iiG as a „unique opportunity‟ in which they can involve and report to a range of officials at every stage of the process, and thus have more chance of effective policy influence. The risk, however, is of co-option. Similarly, the parallel work in Uganda relates closely to the national Ministry of Finance and Ministry of Education, as well as to district and local authorities.

4.8. Information on actual policy outcomes arising from the iiG research is

somewhat lacking, so far. This is partly due to lack of agreed monitoring systems, but also because of the nature as well as stage of the research. For example it is not clear whether there were direct policy outcomes of the meeting with MPs in Kenya. Local dissemination events have not taken place following the success of the work on violence in the elections, nor has tracking of impact in Nigeria been discussed with the implementation partner, ActionAid: the findings are, rather, feeding in to the iiG‟s ongoing body of work on Accountability.

4.9. However, we were informed of some important policy outcomes: apparently

an EU official stated that the Collier & Venables work on Trade Preferences directly contributed to changing the rules of origin for EPAs. This thinking also strongly influenced the recent UNIDO Industrial Development Report, launched in March 2009 in London and at the UN in New York. The new, Sierra Leonean, Director General of UNIDO is keen to get research done on how Africa could break in manufacturing and these ideas were reinforced again at the recent African Development Bank meetings. Similarly, work on labour regulation by Burgess and co-authors was apparently significant in leading to policy changes in West Bengal, and has drawn the attention of the Indian business sector. The Indian State Government of Karnataka has approached ISEC for advice in other areas because of their joint work in evaluating the national health insurance scheme. The authority and reputation of some of the iiG work is also spreading in international policy circles: the BRAC Ultrapoor model, evaluated in partnership with LSE-STICERD, is now being rolled out in 12 countries, with the involvement of the World Bank, among others. And the Institute of Child Health asked the iiG for assistance in

16

designing and implementing a survey on social networks among pregnant women in rural Nepal, having learned of the iiG‟s work on networks in Bangladesh.

Media Coverage and Communication to Broader Audiences 4.10. The program has also produced outputs, notably Paul Collier‟s recent books

and presentations, which explain new thinking on poverty reduction, institutional accountability mechanisms and conflict, to the public as well as to policy makers. They effectively draw upon findings both from iiG and other research in which the CSAE are engaged. Presentations have been made by the main iiG RPC leaders to high-level regional or multi-lateral donor and policy for a held by the World Bank, African Development Bank, EU etc.

4.11. From a communications perspective, the iiG is fortunate to have as its

figurehead Professor Paul Collier. Professor Collier is an internationally renowned development economist and there is therefore huge interest in his thinking on a wide range of issues. It is therefore entirely sensible that one of the communications strategies employed the programme is to “piggy-back” on his standing, both by placing him and his work (iiG related or otherwise) on the website, as well as by “feeding him” key insights and results from the ongoing research programme. There is no doubt that this approach has allowed the research of the programme to reach a much wider audience than it might otherwise have done. The programme should clearly continue to draw on Professor Collier‟s tremendous assets, not only as a researcher, but as a well known communicator of development thinking.

4.12. However, it should be noted that the close association of Professor Collier

with the programme also creates a few challenges. Firstly, there is inevitably a perception that what DFID is funding under the iiG are the things that he is promoting in his books. In fact, the work described in the Bottom Billion book was not funded by the iiG program (despite it and associated outputs appearing prominently on the iiG website) and the iiG-funded work that he is involved in is, perhaps regrettably, a relative minor component of the overall programme. The more recent Wars, Guns and Votes book does draw on iiG funded research, and has recently been featured on the iiG website.

4.13. As noted, it is very sensible for iiG to use the enormous interest in the ideas contained in Collier‟s book as a way of engaging and informing people of the work that iiG is doing. Visitors to iiG‟s website care about the issues and ideas, not who funded what. But it is important that iiG‟s reporting to DFID is completely clear about what is, and what is not, being funded by the program. iiG kindly provided the review team with a detailed breakdown of budget allocation by project which makes this clear. It would be sensible for this detailed breakdown to be updated and made available to DFID on a regular basis.

17

4.14. Data is not systematically collected on media coverage related to the iiG. However, Collier, Burgess and Dercon have given numerous interviews to national and international media. We were also told that international media are beginning to show an interest in the LSE work (Burgess) on climate change. Collier also indicated that he routinely passed journalists onto his colleagues to encourage them to draw on a broader range of expertise from the programme. The recently-appointed communications officer has already instituted a system of replying to media enquiries (which are all channelled through her) and of monitoring and collecting data on media coverage, including through google alert). Notwithstanding, the program could consider doing more to build the capacity of key researchers within the program – north and south - to be able to respond to media enquiries, as well as further systematising the mechanisms used to provide more effective outreach.

4.15. Local and southern national media uptake has been significant for a few

projects. The Kenya meeting with MPs in September 2008 apparently generated local media coverage for weeks after the event. This was largely because iiG invited the participation of a particular well-known Kenyan communicator (and controversial figure), as opposed to the efforts of the Institute of Economic Affairs which the RPC had contracted to assist with communications . The Does Crime Pay for Politicians in India research was widely covered in the Indian media, as a result of the efforts of a local campaigning NGO. The senior research fellow and head of the service sector at iiG partner organisation, EPRC, recently appeared on Ugandan national television to present the Primary Education Accountability research. He is a reasonably well-known figure and has substantial media experience.

Working with NGOs 4.16. Several of the projects rely on collaboration with International NGOs as

opposed to, or as well as, local research institute partners, for implementation. This is an interesting and fairly unusual model for an RPC, and may not only have a beneficial effect on the quality and nature of information gleaned, but also increase the chances of longer term policy impacts and the sustainability of the outcomes beyond the research period, at a local and national level.

4.17. In Kenya and Uganda, the primary education projects work with World Vision

and a Dutch NGO (SNV) in Uganda. In Bangladesh, Tanzania and Uganda, the Ultrapoor programme is being run by the large international NGO BRAC. The Nigerian election project was carried out with ActionAid, which used innovative popular culture and media (theatre) to communicate effectively with villagers. Both the Kenya and Liberia successes count on effective stakeholder involvement within the research process. In Liberia, they are working with the US Institute of Peace and the Carter Centre, along with 14 local stakeholders. These are invested in the policy process, have an international reputation and provide models of conflict mediation: the Carter Centre works with government and the USIP‟s mandate is to create policy reform.

18

Capacity Strengthening 4.18. According to early iiG documents, the African and Asian collaborating

institutions were chosen, in part, because of their history of interaction with their own governments. Furthermore, CSAE has strong links with Ministries of Finance and Central Banks in Nigeria and in Uganda, and STICERD has strong links with the Ministry of Finance and Panchayat Raj Ministry in India. CSAE‟s links with Nigerian NGOs and with BRAC in Bangladesh were considered appropriate conduits to strengthen interaction with civil society. Arguably, the iiG would therefore have a limited direct role in building communications and advocacy capacity among these partner organisations.

4.19. However, the capacity for communications and advocacy work among iiG‟s

partners appears to be rather variable. Some of the partners, notably BRAC and ISEC seem to have well established capacity. Other formal partners, notably IPAR, Ibadan, ACEHS and EPRC appear to have limited ability in this area. Indeed, during a teleconference with the MTR reviewers, the EPRC in Uganda expressed a desire for such training. However, the RPC has not hitherto offered training to partners in research communication or advocacy per se, with most training so far focussed on building technical capacity on impact evaluation. Nevertheless, this is changing and in a recent development, the iiG is intending to hold a communications workshop focusing on writing skills in Uganda in September 2009, alongside a regular RPC meeting.

4.20. Nevertheless, developing capacity of researchers to write policy relevant

material is now underway. So far, this has been run as a pilot workshop with senior Oxford researchers, but it could potentially be extended to other countries. The CSAE/iiG administrator is also piloting capacity building of a group of local economists and journalists in Kenya in May 2009.

Communications Strategy, Logframe, Monitoring and Evaluation 4.21. The RPC Communications Strategy, contained within the revised Inception

Report, affirms the „unique opportunity‟ for influencing policy environments afforded by the Millennium Development Goals. The selection of research themes and projects is “driven by a dual objective: creating high quality academic research with a direct relevance for policy.”

4.22. A comprehensive and ambitious list of target audiences is identified,

encompassing governments and political leaders not only in Africa and South Asia, but also in Europe and the United States of America; multilateral agencies; the research community; and the broader development community across the world, including NGOs; and the broader public. The only sector omitted from the list is the business sector. The strategy and subsequent workplans are not so clear on how the policy targets will be reached and by whom, nor the extent to which some of these targets are also stakeholders and users, and could be collaborators in the research process. Nevertheless, the MTR reviewers found that policy-makers had indeed been significantly

19

engaged in the research processes in several cases (notably in the Accountability work).

4.23. The Strategy mentions the intention to identify clear policy-related messages;

to base their communication on solid evidence; and to stimulate evidence-based policy making. It identifies as a strength of the program its „modus operandi of consulting and working directly with policy makers‟. Indeed this is a distinctive strength of the programme. However, the phrasing of the communications-related outputs in the logframe have encouraged the RPC to report on activities as opposed to on outcomes or policy impact.

4.24. The Communications Officer recognises the need to develop the indicators,

and for systematic monitoring and data collection in order to be able to evaluate communications and policy success. According to the Inception phase report, the success of RPC‟s communications strategy will be measured not only via periodic programme reviews, but also through monitoring of specific quantitative indicators outlined in the logframe and the Communication Plan (Inception Report, table 2). However, in practice, these do not appear to have been monitored. In addition, the RPC intends to measure qualitative effectiveness in the form of response to the outputs and the evolution of policy, but this has not been done systematically. The questionnaire-based evaluation sheets suggested in the logframe for monitoring events have not been used:It transpires that the team does not in fact consider such tools to be valuable, and prefers to gather feedback „via discussions and post event follow up.‟Country communications strategies have not yet been forthcoming, despite the choice of southern partners partly because of their policy experience or expertise.

4.25. The Annual Report (July 2007 – September 2008) affirms the validity of the

original strategy, so it has not been amended. However, with the Communications Officer now in post, it will be appropriate to develop a more specific as well as achievable Communications strategy and action plan.

Process and Structure 4.26. Responsibility for Communications and Policy influence within iiG has been

shared amongst the staff in both Oxford and LSE. In Oxford, several staff have been involved in communicating the outputs of the program including Paul Collier, Stefan Dercon, Tessa Bold, Andrew Zeitlin and Bilal Siddiqi, although in practice Professor Collier is usually the spokesperson or the one cited in the media. The Administrator, Rose Page, along with Erland Berg (LSE) and others in the senior research team have been responsible for disseminating information through the website. The DFID annual Communications workshop was attended in 2008 by the Administrator.

4.27. The iiG communications strategy recognises that team members have

expertise in reaching different audiences. However, there is no mention of the role of southern researchers or research partners in this respect, despite, the statement in the Annual Report that one of the lessons learnt was to exploit

20

the leverage and importance of high profile figures from the South within the RPC (Lessons Learnt, p12).

4.28. The Communications Officer was recruited in part to help spread the load of

media and policy influencing profile among the iiG team. The principle mechanism so far identified to do this is to put forward a variety of iiG researchers to the media, and to maximise opportunities for conference presentations and other dissemination events by a range of researchers. Identifying those with the appropriate skills, both in the UK and the South, will be an ongoing and cumulative process.

4.29. Local policy networks and advisory groups are not mentioned in the

Communications Strategy. There is no advisory group specifically for public policy outreach and communications, though the RPC‟s advisory body is the CSAE‟s Policy Committee. (See Section 7, for further discussion of the role of the Policy Committee.)

4.30. Although the Inception Report states the intention to recruit a designated

Communications Officer by October 2007, with the promise of co-funding from the Gates Foundation, the subsequent Annual Report (June 2007 – September 2008) reasons that despite significant progress in engaging in conferences, there was “room for improvement, and implementation of the communications strategy requires more management time and coordinated effort”; and again identifies the need to recruit expertise in order to translate research into marketable policy-oriented material. The implication is that the RPC had not initially thought a dedicated Communications role was necessary. Indeed, several aspects of the communications role had been co-ordinated or implemented by the Administrator. A permanent part-time Communications Officer, Karin Loudon, took up the post in January 2009. She was appointed as Communications officer for CSAE, with „approximately‟ 50% of her time dedicated to iiG.

4.31. In her first few months in office, the Communications Officer has already

engaged in a number of significant activities: arranging training of senior Oxford researchers in writing for policy communications; assisting the iiG Administrator in planning of a forthcoming workshop for local journalists and economists in Kenya (Africa Means Business); and drafting a CSAE Communications Strategy and Action Plan. She has also begun to lay the foundations for greater systematisation by establishing mechanisms to track timescales for expected research outputs and identifying key external influencing opportunities, as well as compiling a database of international media contacts.

4.32. The Communications Officer sees her principle roles as matching research

products to policy influencing targets and helping to draw out stories across the projects, guided by the researchers, and she hopes to establish CSAE as a centre for excellence in research communications. However, it would be beneficial to ringfence the iiG communications strategy and outline how it dovetails and overlaps with CSAE, so that there is a distinct public face and tracking.

21

4.33. Currently, the Communications Officer‟s focus is predominantly „northern‟. For

example, the influencing „pegs‟ and media contacts are largely UN markers. She is keen to develop more southern and local input, but this will require time and the building of relationships with appropriate southern collaborators. The gateway contact is with the northern researchers who have a direct relationship with the southern counterparts, who appear to be keen not only for policy dialogue to be embedded within the research process but also for the in-country communications side of the work to be developed.

4.34. The iiG communications officer had not been fully briefed on the iiG

communications strategy nor relevant correspondence with DFID, and had limited contact with LSE and the Asia side of the RPC at the time of the MTR reviewers‟ initial meeting with her. Nor was she aware of an agreed communications link in each of the partner organisations, with whom she could relate on the development and implementation of an overall communications strategy. She had admittedly only been in post for a few weeks. Since then, and rapidly, she has made links with partners, in which the lead researcher is the communications link. Aside perhaps from BRAC, the iiG partner organisations do not have the institutional capacity to have dedicated communications specialists

4.35. Following meetings with the MTR reviewers, the Communications officer and

Administrator are intending to discuss communications at the forthcoming RPC meeting. This would ideally become a regular agenda item. Once the systems are more developed and the researchers are more aware of the communications needs and possibilities, it will be easier to track success. If the pilot training workshops with local journalists and researchers were rolled out, this would significantly contribute to building local capacity and to the creation of collaborative communications strategies.

Budget 4.36. Given that the programme is only half way through and that the

communications officer has only just been appointed, one would expect significant underspending on communications. The reported budget expenditures bear this out. Total expenditure on communications and workshops/conference from July 2007 – September 2008 was only GBP 27,566 (GBP 35,000 by March 2009). This is 6.5% of total expenditure during this period. This expenditure came overwhelmingly from CSAE (with no expenditure recorded for LSE). BRAC and ISEC also spent around 7% of their expenditure communications and workshops, but none of the other partners record any significant expenditure on communications (although this is in part because partner workshops are funded from the central budget).

4.37. It should be noted that these figures do not reflect efforts since September

2008. Nor do they incorporate any estimate of the staff time spent on communications, notably the salary of the newly appointed communications officer. Nonetheless, it is clear that the programme will need to devote

22

considerably more resources to communications in the second half of the program, as it is beginning to do.

Use of R4D, Other Networks and DFID 4.38. The iiG makes good use of the R4D website, which lists all its outputs, and it

also lists some of the LSE-generated outputs on the Poverty Action Lab (MIT) website. However, the RPC has not hitherto made use of the Communications Corner or taken advantage of some of DFID‟s 17 other programmes that are funded to help RPCs get „research into use‟.

4.39. The RPC suffers from having –used the acronym IFG (Institutions for Growth)

in its Inception Report but becoming iiG (Improving Institutions for Pro-Poor Growth) in public documents, and from the similarity in name to Institutions for Pro Poor Growth (IPPG), also a DFID-funded RPC. The R4D site does not make a clear distinction in its listings or search facility between the iiG and the IPPG. Indeed the confusion may go further, as indicated in the Review of DFID Research 10% Communications Policy (ODI, February 2009) which does not refer to „iiG‟ but lists both the „IFG‟ and „IPPG‟ as „Improving Institutions for pro-poor Growth‟ (Annex 3); and cites „International Forum on Globalisation‟ for the IFG acronym (Glossary).

4.40. DFID‟s communications adviser provided useful feedback on the Inception

Report. This was not obviously taken up as is reflected in subsequent reports. 4.41. While some policy outputs and outcomes at the multilateral level have been

significant, the southern voice and input could arguably have been stronger, the integration of policy dialogue more systematic, and the monitoring of communications outputs and outcomes clearer, if the RPC had appointed a dedicated Communications Officer earlier in its life. There is potential to develop the above dimensions in the remaining period of the RPC.

Recommendations R9. Alter logframe „output‟ to make it more possible to track, measure and report

on communications outcomes. R10. Update iiG communications strategy, making a distinction between iiG and

CSAE strategies R11. Discuss Communications strategy and share successes among RPC

members at the 6-monthly meetings each year. R12. Develop selected project communications strategies, in collaboration with

southern counterparts. R13. Roll out training on Communications, including writing Policy Briefs, to the iiG

research team, north and south. R14. Devote significantly larger resources to communications activities in the

second half of the program, including through local partners, ensuring that

23

Communications accounts for (at least) 10% of the total spend, as stipulated by DFID for RPCs.

R15. Broaden the outreach base for policy-relevant results. For example, make

systematic use of DFID‟s Communications Corner to share experience of advocacy, and send Policy Briefs to the most appropriate of the 17 programmes DFID funds to help get research into use.

R16. Clarify the distinction between iiG – IFG and IPPG on the R4D website.

(Recommendation for DFID/R4D)

24

5. CAPACITY DEVELOPMENT

5.1. iiG states that it is “committed to assist in capacity building for the Southern

partners institutions and other stakeholders in the partners countries. As we intend to achieve evidence-based policy change on key themes of our research agenda, our capacity building activities have a number of target groups: a) Researchers in our partner institutions in African and South Asia b) The broader research community in our partner countries c) The stakeholders in the development community in the countries of our

partner institutions, including in government, aid agencies and NGOs.” 5.2. The logframe‟s stated expected output is “Strengthened Capacity of partner

institutions in RPC in analyzing the role of institutions and their reform for growth and poverty.“ This relates to analytical and research capacity, and not to capacity to communicate and to influence. The stated Verifiable Indicator has also been rather narrow, citing only joint authorship of papers.

5.3. While the logframe defines capacity development as institutional, in practice it

also encompasses individual training, fellowships, pairing northern and southern researchers, and some peer review work. Some of this has been done quite informally, as a result of establishing direct researcher-to-researcher relationships.

5.4. In practice, the capacity building work has followed four main areas:

a) Training in statistical techniques for policy research – econometric analysis for impact evaluation; panel data techniques; survey implementation and cross-cutting designs

b) Joint policy research on impact evaluation c) Joint North-South research and co-authorship of papers d) Mentoring southern researchers

5.5. To date, statistical research methodology training has taken the form of 7

workshops, in Uganda, Kenya, and Ethiopia, but not in India or Bangladesh. One of these was a multi-country workshop on Econometric Methods for Programme Evaluation, in Uganda; facilitated by the University of Oxford and intended to strengthen the in-country capacity to take forward this methodology. It was aimed at research partners, and included teams both from IPAR (Kenya) and EPRC (Uganda), as well as the Uganda Bureau of Statistics and the Bank of Uganda, and the Faculty of Economics and Management, Makere University. A workshop on Collaborative Project Design with Key Stakeholders in the Education Sector was held with the EPRC, and included key research collaborators from various Ugandan statutory and civil society organisations. Of four methodological workshops so far carried out in Kenya, most involved The Ministry of Education and other users such as Central Bank of Kenya, National Bureau of Statistics, and the University of Nairobi. Prof. Collier also informed the MTR reviewers of capacity development for Governors of all African Central banks at a forthcoming CSAE workshop on managing national resources in the financial crisis (funded by JP Morgan). Considerable capacity building efforts have been

25

made in survey implementation, for example training BRAC staff as enumerators; and for double data entry upon survey collection.

5.6. Two Impact and Evaluation workshops have been carried out in Uganda, for government education sector and civil society, and for health care managers of regional and district referral hospitals. The latter was done by EPRC without CSAE involvement, using material gleaned from earlier CSAE training. Similarly, in Kenya three workshops have been held for a range of Ministry of education staff, Kenya National Bureau of Statistics and for community development motivators (in partnership with World Vision). And in Tanzania, a three day seminar of impact evaluation of land policies included participants from the relevant Government Ministry.

5.7. One strength of the workshop design is that the events are aimed not only at

the partner organisations but also at other stakeholders. So these will have benefited not only individual researchers and some organisations, but also policy makers and civil society. However, the degree of benefit and the outcomes are not explicit.

5.8. There have been some direct North-South researcher relationships, joint

research and visiting fellowships. Exchange visits have taken place with Uganda (Zeitlin from Oxford, and Bategeka to Oxford), and in Kenya, (Sandefur and Bold from Oxford, and Prof. Mwabu and Kimenyi to Oxford), and in India and Bangladesh (Burgess from STICERD and Prof Rajasekhar to STICERD. A student from ISEC will soon spend 3 months with the LSE/STICERD to work on one of the projects. BRAC has a PhD student at LSE working on the Ultrapoor Programme, part-funded by the RPC. The LSE/STICERD staff report that capacity building work with counterparts in India and Bangladesh has tended to be „informal‟, consisting mostly of dialogue between researchers, and collaborative design of surveys and other methodological inputs.

5.9. The final area is mentoring of junior researchers in partner organisations. This

has taken place in Uganda, in which young research fellows at the Economic Policy Research Centre were mentored by a DPhil researcher from QEH, to develop and write scientific papers. One of these papers was presented at the CSAE conference in March 2009. The proposed peer-review scheme has not yet begun except in Uganda. According to the iiG team in Oxford, this is because partners have not put forward many projects.

5.10. Researchers listed on 3 main themes on the iiG website are predominantly

northern. None of the 10 published articles involves a researcher from a Southern partner (and only one involves a Southern based researcher); only 1 of the 10 working papers on the website involves a Southern-based researcher. Although more jointly authored work may be in the pipeline, it is clear that the approach of the programme has been to produce high quality research by working with a small number of already highly skilled Southern researchers, rather than to invest heavily in junior Southern researchers. However, the MTR reviewers have been assured that as southern institutions are now beginning to generate more data, a number of papers are in the

26

process of being produced with junior researchers in BRAC, EPRC, Ministry of Education in Kenya, etc.

5.11. In lessons learned cited in the iiG Annual Report June 2007 to September

2008, the RPC notes the lack of partner demand for capacity building. This may reflect a difference between the type of capacity building on offer (predominantly training on impact evaluation followed by informal mentoring on the implementation of such evaluations) – and the capacity building desired by the partners. For example, the iiG‟s management said that the Nigerian partner did not appear to be interested in capacity development (it is true that they are not interested in quantitative impact evaluation), whereas in a telephone conversation with the MTR reviewers, the recently-appointed Director of Department in Ibadan suggested that he would indeed be interested in capacity building, although he did not relate this to any clear indication of needs.

Achievement of Capacity Building Outputs 5.12. The RPC has gone some way to meet its expected output of “Strengthened

Capacity of partner institutions in RPC in analyzing the role of institutions and their reform for growth and poverty.” This does not apply across the board, but is noticeable with respect to EPRC and BRAC, and largely concerns increased institutional capacity in methodologies of economic impact assessment.

5.13. The EPRC (Uganda) says its institutional capacity to undertake, and to train

others in, Impact Assessment has been increased as a result of the iiG programme. Following the initial training workshops undertaken by iiG staff from Oxford, the EPRC has run both Impact Assessment and Project Design workshops themselves. According to Dr Mugisha, the principle advantage to them of being part of the iiG programme is the methodological support from Oxford: “Impact Evaluation is new to EPRC; we need technical support until we can get to the end of the process. We can now carry it out and teach it, but we want our own expertise [in this area].”

5.14. Although the EPRC is a highly respected Ugandan organisation with a degree

of expertise in Capacity Building, it recognises the wider need for reduced southern dependence on the North. The iiG is playing an important role in filling a perceived gap. Ideally the EPRC would also like capacity building for junior researchers (university graduates) in the form of fellowships in the UK, and media communications training for staff and researchers.

5.15. While the RPC role in formal capacity building (in the form of workshops) in

Asia may have been relatively limited, the marked development in methodology used for BRAC‟s Phase II Ultrapoor programme is evidence of the institutional capacity that has been built as a result of participation in the iiG RPC. There are, furthermore, plans for LSE to help with capacity building for BRAC offices as they open in Uganda and Sierra Leone. This is testament to the extent to which BRAC values the LSE/STICERD‟s input.

27

5.16. Some partners (ISEC and EPRC) also commented on the institutional benefit

to them of participating in the RPC meetings – whereby they could glean information about the wider context and learn from others‟ experience. Greater emphasis could be placed on facilitating such institutional capacity building opportunities by reorganising the advisory and management arrangements to ensure more southern partner input [see Section 7].

5.17. Furthermore, in order to increase the impact and to ensure a lasting legacy,

the achievements in institutional capacity development with EPRC and BRAC could be extended, communicated and multiplied in other parts of the programme.

5.18. In particular, it would be good to consider increasing the role of South-South

learning. To date, training has come from northern iiG partners with the recent exception of EPRC. However, as the programme matures and expertise is gained within Southern partners on different topics, there may be a useful role for the programme in facilitating and encouraging learning across Southern partners.

Budget for Capacity Building 5.19. As with communications, it is difficult to assess the true extent of resources

which have been spent on capacity building since these fall across a number of budget categories including personnel time and workshops. The total amount spent under the explicit capacity building category in the Annual Report from July 2007 – September 2008 was only GBP 9,172 – only 2.2% of total spend – with no expenditure indicated by any of the Southern partners. Clearly this significantly underestimates the true resources devoted to capacity building. At the same time, the evidence from joint-authorship as well as discussion with Southern partners confirm the impression that capacity building is not a major emphasis of the programme.

Recommendations: R17. Undertake capacity building of researchers and staff in RPC consortium in

advocacy and media communications R18. Link between iiG communications officer and people responsible for

communications in southern partner organisations to discuss capacity building needs and opportunities.

R19. Explore ways to extend and build on institutional capacity development work

in order to maximise impact and leave a legacy for the southern partner institutions.

R20. Devise and implement a plan to maximise opportunities for south-south exchange and inter-partner learning.

28

R21. Identify more clearly what resources are allocated to capacity development,

and increase the overall spend.

29

6. GENDER

6.1. The initial project bid does not contain a detailed gender analysis, but rather

acknowledges that while shifts of power arise from institutional change “it is not envisaged that any particular group of poor or marginalised people will be adversely affected by the programme.” Similarly, the RPC reports do not contain gender-disaggregated information.

6.2. Despite this, some of the projects specifically address the situation faced by

women and look closely at gender relations. The Female Empowerment through Inheritance Rights project in India is one such project. Also, notable is the impact evaluation of the BRAC program for Human Capital, Financial Capital, and the Economic Empowerment of Female Adolescents. This work began in Bangladesh and has been extended to Tanzania and Uganda. Aiming for empowerment, it encompasses the not only the productive but also the „community‟ role of these young women, providing life skill training as well as addressing wider livelihood issues. The project also seeks to learn about parents‟ hopes and expectations for their adolescent sons, and how this affects the opportunities available to their daughters.

6.3. In addition, the Ultrapoor Programme in Bangladesh looks closely at gender

relations. Most but not all of the beneficiaries are women, and the study collects data on the changes in their relationships with, for example, traders, money lenders, food exchangers, local decision-makers, social networks etc. as a result of the programme intervention. Data is also differentiated according to whether they are female-headed households. The BRAC programme is training these young women in political and social awareness as well as providing practical skills. The Law Without Lawyers project in Liberia is mindful of the fact that men and women have differentiated access to justice, that household dynamics are changing with the process of redefining the rape law; and that paralegals may be particularly beneficial to women. The Kenya education programme broadly considers the implications of differentiated attendance by boys and girls.

6.4. Thus, although gender is not explored explicitly within the programme, the

three guiding iiG themes – accountability, manufactured exports, and rural transformation – do have interesting gender dimensions. Given the significant amount of work in the program which examine womens‟ rights and empowerment, there would seem to be an opportunity to more systematically draw out lessons from the programme about how different institutional designs can help or hinder the welfare of women. For example, the accountability work could consider a range of issues such as differential access to power, voting patterns, vulnerability to violence, candidates‟ promises etc., between men and women. Similary, within the manufactured industry theme, the Kenyan flower export industry project, for example, could consider some socio-economic implications from the fact that the employees are almost exclusively women.

6.5. The issue has neither been raised as a „requirement‟ by DFID, nor has

anyone made a point of looking at or drawing out the gender dimension within

30

the RPC as a whole. In the opinion of the MTR reviewers, the iiG could be enriched if researchers were encouraged to consider gender relations as a matter of course, and if relevant findings were drawn together. The UK-based researchers clearly have the capacity to analyse gender issues relating to their research topics, so there is potential to pursue this dimension further.

Recommendations R22. Encourage the systematic examination of gender issues throughout the three

research themes. R23. Consider producing a synthesis of the lessons about the relationship between

institutional design and the welfare of gendered groupings from the various projects which examine different aspects of this issue.

31

7. PARTNERSHIP MANAGEMENT AND EFFECTIVENESS

7.1. The iiG RPC is a consortium of CSAE, QEH, and LSE in the UK, IPAR in

Kenya, EPRC in Uganda, University of Ibadan in Nigeria, ISEC in India, ACEHS in Ethiopia, and BRAC‟s Africa operations.

7.2. The iiG Programme has struggled with certain partner relationships since the

outset of the programme. The relationship with IPAR in Kenya appears to have effectively ended. The former Director and Finance Director of the IPAR left suddenly, and, although the relationship with the current Director is satisfactory, the emphasis of IPAR appears to be towards consultancy rather than research. No proposals for research projects have been received from IPAR. The partnership arrangement therefore has de factor turned into a relationship with particular senior non-IPAR Kenyan researchers already well known to the CSAE team (Kimanyi, Mwabu) as well as with the Ministry of Education for the service delivery project.

7.3. Similarly, relations with the Nigerian partner, the University of Ibadan, have

been poor. The Ibadan team in the Department of Political Science was keen to undertake a survey of candidate selection and campaign finance. The current head of department says that the pilot survey of campaign finance is complete and that a draft report will be available imminently, although none was received by the time of writing of this MTR. We were told that the survey being conducted on candidates has interviewed 50 of the 75 candidates intended, but is apparently being slowed down by the difficult in obtaining interviews with incumbent state governors.

7.4. However, there appears to be a major methodological divide between the

approach adopted by the Department in Ibadan and that of Oxford and LSE. The Department would appear to have little experience of conducting professional quantitative surveys and no knowledge or interest in the type of impact evaluation approaches adopted by the iiG programme. Rather their approach appears to involve qualitative semi-structured interviews. Their view is that this is more appropriate to the Nigerian context since much official quantitative data is unreliable.

7.5. The Oxford/LSE team, on the other hand, feels that, despite “constant

optimism” from University of Ibadan, there is little indication of their seriousness to conduct research and is therefore waiting to see whether there are any concrete results from the initial pilot before committing any further resources. In January 2009 iiG attempted to rekindle the relationship between the Department and researchers in Oxford through the involvement of political scientists from the Department of African Studies. The outcome of these efforts was not clear at the time of writing.

7.6. It is also interesting to note that, despite these problems in the relationship

with the official iiG partner, interesting and innovative iiG research on violence and voting has taken place in Nigeria. This was conducted by an Oxford post-doc working independently with Action Aid, rather than through the University of Ibadan.

32

7.7. Other institutional partner relationships have suffered from their dependence

on key individuals as well as political and institutional uncertainty. For example, the Director of the partner in Ethiopia (ACEHS) – Dr. Alemayehu Seyoum Tafesse - left to join the IFPRI office in Ethiopia. Since the research project in Ethiopia was joint between Professor Dercon and the Director of ACEHS, CSAE now intends to shift the partnership arrangement to the ESSP programme of EDRI (the Ethiopian Development Research Institute) – a joint programme with IFPRI - to ensure that the research collaboration can continue.

7.8. In a few cases, the institutional partnership appears to be working well. The

relationship with EPRC in Uganda is productive and supportive. A call for proposals within EPRC resulted in a major research project between a senior EPRC researcher and a CSAE researcher. The project is undertaking an impact evaluation of alternative approaches to improving school quality. This topic is of critical importance to Uganda which has recently experienced a huge increase in enrolment after the introduction of free schooling, and is now concerned about how to maintain quality. The work is being done in close conjuction with the government and the results are likely to be influential in determining the approach that the government may adopt to monitoring school performance. EPRC researchers value the association with the Oxford team for two reasons: because the Oxford researchers were happy to support a project originated from the EPRC, rather than imposing a project from outside, and because they have introduced and provided extensive training in the latest techniques of impact evaluation as well as support for the implementation of the project.

7.9. The relationship with ISEC also appears to be working well. ISEC were involved right from the onset of the project, in writing the original proposal for the RPC. They are currently working closely with researchers from LSE on an evaluation of the impact of health insurance on poverty, as well as the impact of alternative approaches to providing information on uptake of such programmes. As a long established and well respected research institute, they have taken the lead on identifying the key research questions. Again they value the relationship with the LSE team because it provides access to the latest methodologies of impact evaluation. And again, ISEC is extremely well connected with both the state and national government and so the results of the work are likely to have significant influence on policy.

7.10. The relationship with BRAC reflects the programme‟s demand driven

approach to partnerships. BRAC, as a major international NGO, have clear objectives, structures and systems. Part of these entail undertaking rigorous evaluations of program interventions. They have therefore worked closely with LSE researchers on mutually agreed evaluation of key BRAC projects. This has enabled a significant increase in the ability of BRAC to ensure that its programmes are evidence based.

7.11. In addition to the formal members of the consortium, CSAE and LSE

researchers have entered into relationships with a variety of key institutions

33

related to the particular issue under study. For example, work in Liberia exploits the links between one Oxford researcher and the Carter Centre; research in Burundi (now stopped) exploited a connection between another Oxford researcher and a large local NGO (ACCORD); the service delivery research projects in Kenya and Uganda have close links with the Ministries of Education and, in Uganda, with World Vision.

7.12. Despite this profusion of collaborators (as opposed to formal partners), the

programme has, so far, done relatively little to encourage dialogue and learning across southern organisations with which it works. Formal partners are invited to the iiG meetings every six months, and clearly value this engagement, but more could be done to facilitate learning, both between partners on methodologies, comparative work on similar themes in different contexts, and even mutual learning across NGOs and implementing agencies on different approaches to similar problems.

7.13. The tendency of the program towards a “hub and spokes” arrangement for the

consortium (in fact 2 hubs – Oxford and LSE – with different spokes) is also reflected in the program‟s management arrangements. Although the programme management committee does involve Southern researchers, it does so for a year or so at a time and then rotates in another partner. The only permanent members of the management committee are Paul Collier, Stefan Dercon, Robin Burgess and Rose Page. In practice all significant decisions about which projects to undertake and resource allocations are taken by Stefan Dercon, in the case of the Oxford linked projects, and Robin Burgess, in the case of the LSE linked projects, with both consulting Paul Collier when appropriate.

7.14. This management arrangement provides rapid decision-making and great

flexibility. Nor did any of the Southern partners have any major objection to this mode of operation, given the collaborative nature of the research projects themselves. However, the programme may wish to consider whether it is missing the opportunity to draw more (and more frequently) on the managerial expertise and ideas of their Southern partners.

7.15. Related to this is the issue of the Directorship of the programme. Currently

this resides with Paul Collier. However, it is quite clear that the day-to-day management of the programme (which is formally led by CSAE) is done by Stefan Dercon. This then begs the question as to whether it might be better for Prof. Dercon to formally assume the Directorship of the programme. Prof. Collier has stated that we could be perfectly happy to pass the Directorship to Prof. Dercon and Prof. Dercon appears to be willing to undertake this role.

7.16. The iiG programme also has an informal US advisory team consisting of four

leading political scientists and lawyers working in the field. Such advisory teams can be invaluable as they provide a knowledgeable but disinterested view on the project. Unfortunately, this team does not seem to have functioned as a team and only one member of the team appears to have been significantly engaged with the programmes work.

34

7.17. The programme also has a formal Oversight Committee. Because the programme is housed in CSAE it was decided that this committee would be the same as the Oversight Committee for CSAE which meets twice a year to consider all the work done by the Centre. This group contains many distinguished and experienced individuals. Professors Collier, Dercon, and Burgess all sit on the Committee, as do a number of other members of CSAE and at least two senior African representatives. DFID withdrew from the Oversight Committee because it they felt that their presence would represent a conflict of interest. The iiG programme is a standing item on the Committee‟s agenda, ensuring that the programme‟s progress is discussed at every meeting. For brevity this usually consists of a presentation from one of the projects currently underway, followed by a general discussion.

7.18. Notwithstanding the enormous experience amongst the members of the

Oversight Committee, the use of the Oversight Committee of the CSAE as the oversight body of the iiG programme does have significant disadvantages. First, it is a large committee, making it more difficult to go into different elements of the programme in detail. Second, it is (understandably) heavily biased towards those with experience of Africa; but a large component of the iiG programme works in South Asia. Third, its role and structure has led DFID to decide to exclude itself from participation. Yet it is very helpful for DFID staff to play an advisory (rather than an oversight or management) role for RPCs, since, among other things, they can contribute experience of the approach of other RPCs and DFID funded programmes which may be helpful to the iiG programme. It may be helpful for the RPC to consider ways of accessing more detailed advice across all its projects, including input from DFID.

7.19. Finally, the programme currently monitors and evaluates its progress rather

informally, through frequent contacts between the relevant researchers in Oxford and LSE and their counterparts in Southern partners or collaborators. This approach ensures “real-time” monitoring of success (and failure) and the ability in principle to respond flexibly to changing developments as they emerge. However the generally good quality of practical management is not yet matched by an effective formal monitoring and evaluation system. Basic facts about the progress of each project against milestones are not routinely or systematically collected, nor is there consistent collation of information about communication and dissemination activities. There does not appear to be any systematic effort to collect some of the Objectively Verifiable Indicators provided in the logframe – particularly those relating to the project purpose, rather than outputs.

Recommendations R24. Seriously consider whether continued investment in some Southern partners

partnerships will actually deliver commensurate outputs. If not, the programme should formally end relationships with Southern partners that are not delivering results.

35

R25. Do more to facilitate learning, information sharing and comparative work across its Southern partners and more broadly amongst the programme members.

R26. Construct a small formal management committee chaired by the Director and

involving at least two permanent Southern partners and meet (virtually) regularly to discuss the management of the programme.

R27. Formally change the Directorship of the programme to Prof. Stefan Dercon. R28. Keep oversight of the programme with the Oversight Committee, but set up a

small informal Advisory Group specific to the iiG programme (akin to the non-functioning US Advisory team but involving non-US people too) to provide in-depth strategic advice once per year. A representative of DFID should be on this Advisory Group.

R29. Set up systems for the systematic collection of the indicators given in the

logframe to ensure that the programme is capable of demonstrating its success.

36

8. LINKS WITH DFID AND RELATED RPCS

8.1. The relationship between iiG and DFID has not been entirely harmonious. In part

this may reflect the unusual circumstances surrounding the initial award of the RPC. This was compounded by the impression in DFID that the programme has deviated significantly from the original proposed work as described above.

8.2. The difficulties in communication between the programme and DFID have been

compounded by constant changes in staff in DFID. There have been three Advisors responsible for the programme in the 2.5 years of the programme. In some cases, iiG claim that they were not even informed that the relevant contact person in DFID was changing.

8.3. The programme also claims that it has received little or no feedback from DFID

on the documents which have been sent (apart from the initial Inception report). This situation may have improved recently – the review team saw a dialogue between the latest DFID Advisor and the programme which indicated that documents are being read and commented upon.

8.4. There also appears to be confusion about the extent to which the RPC can

contact DFID offices in the field. The RPC say that they were told not to contact field offices except through the central research department. We understand that this is not DFID‟s view. However, the reviewers have heard a similar account from another RPC. It is important for DFID to clarify the protocol for all RPCs. It is our view that DFID should encourage RPCs to contact relevant DFID offices directly and should facilitate initial introductions. It seems to us unfortunate that DFID, because of its understandable concern not to overload local offices, appears to be inadvertently discouraging the communication about DFID funded research to its own staff in the countries in which this work is being done.

8.5. Interestingly, the RPC has a distinct advantage over some other RPCs in

engaging local DFID offices, since the programme works directly with implementing agencies, some of which are receiving DFID funding for their work. For example, the DFID office in Bangladesh is well informed of the RPC‟s work because of their funding for BRAC. Given the relevance of much of iiG‟s research to practical policy interventions it should be incumbent on both the programme and DFID to ensure that local DFID offices well informed of the results of the research.

8.6. Links with other RPCs working on similar issues have been minimal. In

particular, there is another RPC on Institutions for Pro-Poor Growth run by Manchester and York Universities. Although the methodological bias of the IPPG programme is rather different from iiG, there is useful work being done on institutions in this programme from which iiG researchers might learn. However, although there have been conversations between senior staff in the two RPCs and there are strong links between the two administrators, there has been little contact between the iiG and IPPG researchers, and very little sharing of the outputs of the two programmes.

37

8.7. Despite being distinct research programmes, the similarity in their names is still causing some confusion for those outside of the two programmes. Although iiG have been consistent in the use of iiG in their public documents, its initial proposal was entitled “Improving Institutions for Pro-Poor Growth in Africa and South-Asia”, whilst the inception report drops the “in Africa and South-Asia” and uses the acronym IFG. The current name is the same as in the Inception Report but with the acronym iiG. As a result the ODI Review of the DFID Research Communications strategy in Feb 2009 lists IFG in RPC acronyms as „International Forum on Globalisation‟, yet in the Annex detailing the RPCs says IFG is the „Improving Institutions for pro-poor growth‟. It also says that the IPPG is „Improving Institutions for pro-poor growth‟ which it is not. There was also confusion at the DFID Communications meeting, and on the R4D web search.

8.8. There are potentially close links between the iiG and the new International

Growth Centre (IGC) not least because the co-Directors of the IGC are Paul Collier and Robin Burgess. However, these links are likely to be informal, since the modus operandi of the IGC is quite different from that of the iiG. Thus there is no sense in which iiG research programmes will merely be continued under the IGC. However, it is quite likely that related pieces of research will be proposed to the IGC research theme directors as part of the IGC‟s open call for proposals. Since four of the iiG research theme directors are members of the LSE iiG team and the final decisions regarding research funding will be made by Paul Collier and Robin Burgess, some care will need to be taken to ensure proper procedures for those responsible for making decisions to recuse themselves from bids that involve parties linked to themselves. Prof. Collier assured the reviewers that this would be done.

Recommendations For iiG R30. The iiG should contact the Directors of the IPPG and arrange for a joint

workshop to encourage mutual learning from the two programmes. R31. iiG and IPPG should both put links on their websites linking to each other with

agreed text delineating the different focae of the research programmes. For DFID R32. DFID need to take seriously the impact of constantly changing staff on their

RPCs. There is always a high level of turnover within DFID positions for understandable institutional reasons. However, it would be within the power of DFID to require staff taking up Advisor positions in the research Department to stay in the same position for at least 2 years (e.g. by making them ineligible for other posts with that period). To compensate for this disincentive to take research Advisor positions, there could be an element of preference for subsequent advancement associated with the positions, or some other form of incentive. This is a complex institutional issue – but DFID should address it

38

since their ability to shape and to obtain the maximum benefit from their large research investments is being affected by the constant staff changes.

R33. DFID should encourage RPCs to contact relevant DFID country offices

directly and should facilitate initial introductions. All RPCs should be informed of this.

R34. DFID should tidy up the confusion between iiG and IPPG on the R4D website. R35. DFID should request the IGC Co-Directors to produce and disseminate clear

written rules about the process for applying to the IGC for research funding and the procedures that will be followed in making decisions about funding, to preclude any suggestion that iiG (or any other researchers) will have preferential treatment in the allocation of research funding.

39

9. CONCLUSIONS AND OVERALL RECOMMENDATION

9.1. The iiG is a good research programme producing innovative research which

will improve our understanding of the way in which a variety of institutions impact upon the opportunities and wellbeing of poor communities in Africa and South Asia.

9.2. However, iiG‟s opportunistic, indeed almost promiscuous, way of picking

collaborators and projects poses challenges to both the iiG and to DFID. 9.3. A key challenge for the iiG is to try and ensure intellectual coherence across

the program. It is essential that iiG clarify and communicate their vision for how the predominantly microeconomic projects on accountability and rural transformation will answer questions critical to promoting pro-poor growth.

9.4. Moreover, the opportunistic way in which projects are identified and funded

creates a higher than usual risk that the individual pieces of work will not add up to more than the sum of their parts. iiG needs to take seriously the challenge of ensuring that its multitude of activities provide real added-value. In particular, the programme needs to develop clearer criteria for the inclusion and exclusion of projects in the program going forward.

9.5. The iiG‟s approach has also generated a higher than usual rate of project

failure. For example, substantial resources were devoted to the Irrigation Tanks work which has now stopped due to a lack of support from the government; the land reform work in Uganda also stalled; the EPZ work in Kenya stopped for lack of data; the Burundi project stopped because the pattern of refugee movements was not as originally thought. These are in addition to the problems with partners mentioned above. To some extent such failures are the price one pays for interesting opportunistic research. Moreover, it is good research management to pull the plug on work where it becomes clear that the research questions cannot be properly addressed in the new circumstances that have arisen. However, one cannot avoid the impression that the programme might have been better to take a more cautious approach and to do more background investigation before launching into new projects and collaborations.

9.6. However, the iiG programme also posed some challenges for DFID. DFID‟s

RPC model is based upon the idea of a set of long-term Southern partners who would have a strong sense of ownership of the entire programme of work and who would work closely with Northern research institutions to build capacity and generate jointly high quality research.

9.7. For some of the larger projects this is the way in which the iiG operates.

However, for many it is not. Although there are senior Southern researchers from partner institutions involved in the Management Advisory Group, they are rotated every year. There is consultation, but most of the key decisions about project selection and funding are taken by Prof. Stefan Dercon and Prof. Robin Burgess. The research partnerships between Northern and Southern researchers are, in many cases, substantial, but they are so precisely

40

because they are based upon proven interest from the relevant Southern institution (whether ministry, NGO, or research institute) to conduct research.

9.8. This way of working is not typically what DFID expect from an RPC. The

challenge to DFID, is that it also appears to work. Insisting on genuine demand from Southern partners and researchers means that most of the research that actually gets done has committed Southern partners involved in it. Centralised control of funding, means that substantial funds are not spent until initial pilot pieces of work have shown real potential. Centralised control of methodology ensures the highest possible level of methodological consistency and quality control. The result is a smorgasbord of generally high quality, innovative but opportunistic research at relatively low cost. In short, the iiG approach is a good way of generating interesting new research, but at the cost of doing less by way of institutional capacity building for Southern partners. At the same time, it seems likely that the methodological capacity building which has happened in several partners may have a significant long-term impact on the way in which these institutions work in the future.

Value for Money 9.9. One area where the programme is performing well is on Value for Money. The

iiG is excellent value for money, in the sense that it has successfully leveraged significant external funds for many of its projects. One example puts this into perspective. The evaluation of BRAC‟s ultra-poor programme by iiG is costing GBP 130,000 – but this is part of almost GBP 1.3 million which BRAC are spending on the programme (drawn from other programme funding, including DFID). Throughout the iiG programme one sees examples in which iiG have strategically invested relative small amounts of money to enable the scientific exploration of issues embedded in much larger government or NGO programmes.

9.10. The down side of the iiG‟s good value-for-money is that it is seriously

underspent. Total expenditure to the end of September 2008 was GBP 422,487. Although, the rate of spending should have increased substantially over the last year, this nonetheless represents a spend rate of around GBP 360,000 per year. This would imply and overall programme expenditure of GBP 1.8 million, compared to a programme budget of GBP 2.7 million. Moreover, almost all of the expenditure to date has been by Oxford and LSE, with minimal expenditure recorded by the RPC‟s formal partners. In part this reflects problems that iiG has had with a couple of partners (see the section on Partnership Management below); later figures may also reflect more expenditure by ISEC and BRAC in particular.

9.11. The significant underspend does pose a problem, because it is not clear how

it can be effectively addressed. One of the strengths of the programme is that money only flows to a partner when there is indication of real commitment and substantive collaboration. But even when this is the case, the total expenditure required is not always large, precisely because the programme

41

injects expertise strategically to learn from institutional innovations that are, in many cases, already funded from other sources.

9.12. There are two obvious ways in which programme expenditure could be

expanded. First, the programme could undertake more projects. However, as noted above, the programme already has difficult in ensuring intellectual coherence and value-added from the projects that it already has – adding more small projects will not help this. Second, iiG could expand its contribution to existing projects. But this might entail providing finance for programmatic activities which would have taken place anyway. Neither of these options for expanding expenditure are attractive.

9.13. A third option might be for the programme to invest significantly more

resources into capacity building and communications. As noted above, much of the capacity building work to date has been informal and relatively small scale. Yet the value of the techniques being used is enormous, and BRAC already has plans for creating a centre in Kampala to manage such work. Helping to set up and seed fund a major initiative based in the South extending the evaluation methodologies used by the iiG could be a very valuable use of the programmes funds.

9.14. A further option would be to devote more time to communicating the value of

the work that the iiG is doing to a broader audience, not only among academics and donors, but also policymakers and NGOs in the South. Non-economists could easily dismiss the outputs of the programme as “a set of evaluations proving the obvious”. But the best work in this programme is in fact testing theories. More resources could usefully be devoted to explaining to a wider audience what these theories are and why they matter for development.

9.15. The iiG management have indicated that some of these broader programme

wide activities may be constrained by the fact that a proportion of the budget is earmarked for partners. However, they believe that the underspend issue is one of research start-up and will be resolved in year 4 and 5 of the programme, particularly as they formalise new partnerships.

Risk Assessment 9.16. The project has a well developed Risk Assessment framework (contained in

Annex 3 of the Annual Report) which remains appropriate. 9.17. Nonetheless, given the relatively high rate of project failure, the project may

wish to re-assess the risks facing new projects. In particular, it would appear that the programme may have underestimated the impact and probability of government non-compliance on project success.

9.18. The programme should also consider introducing a more systematic risk

evaluation process for each project in order to avoid devoting significant

42

resources to projects which might reasonably be predicted to be highly risky at the outset.

Overall Recommendation

9.19. It is recommended that the programme continues with the modifications set

out in the recommendations above.

43

iiG MTR Annex A

Terms of Reference for Mid-Term Review of DFID-contracted Research Programme: Institutions for Pro-Poor Growth: iiG University of Oxford

Background

DFID-contracted Research Programmes are centres of specialisation around a particular research and policy theme. They are groups of researchers from a number of institutions which may include NGOs, civil society organisations, academic and commercial organisations. Funding is provided for five years. DFID Research (formerly Central Research Department (CRD)) provides support for the research programmes through the Human Development, Growth, Sustainable Agriculture and Social, Political and Environmental Change teams.

It is a DFID requirement that all programmes are subject to regular reporting and monitoring processes. DFID Research‟s agreement with research programmes provides for an external Mid-Term Review (MTR) of each programme. MTRs will be held after two years of operation of each research programme, to enable the research programmes to gain the maximum benefit from the review. Each programme has a statement of Purpose and a number of Outputs (deliverables) agreed by DFID and the relevant institution(s), which are set out in the programme‟s Logical Framework, together with indicators of achievement.

Benefits of the Mid-Term Review to Research Programmes

Encourage ownership and participation;

Ensuring continuous learning and quality control;

Ensure funds are used effectively and efficiently to deliver outputs/outcomes;

An opportunity to engage with DFID Research and DFID advisers.

Benefits of the Mid-Term Review to DFID

Part of the ongoing monitoring and review of research programmes;

An opportunity to work closely with research programme members and external reviewers, including to identify added value and comparative advantage of the research programme.

An opportunity to discuss lessons learned by research programmes in some detail and shape future direction to increase operational and policy relevance;

MTRs of programmes are a key process in the programme management cycle, comparing actual progress against targets and value for money (VFM);

44

To review the risk analysis of the programme.

Objectives

The purpose of the MTR is to measure and report on performance to date and to indicate adjustments that may need to be made to ensure the success of the research programmes. The MTR should be used by all of the research programme partners and management to inform their future work.

The MTR should include an analysis of the following areas:

1. Review of Purpose

a) The degree to which the purpose is relevant to decision-makers in policy and practice in developing countries.

b) The likelihood of achieving the purpose during the grant period. Use the scoring system outlined below*** in paragraph 6.

c) The likelihood of further progress in achieving the purpose after the grant period (this captures the issue of influence continuing after the end date of 31/12/2011, especially if the programme has built sustainable partnerships. It allows for the slowness of some influencing processes).

d) Value for money (VFM) (The score provided for the review of purpose is used by DFID internal systems for a calculation on value for money for DFID Research and DFID as a whole. In the absence of benchmarks for the VFM of impact of research on policy and practice, it is worth looking at the degree to which changes in policy and practice can be attributed to the programme, as assessed by reviewers who are familiar with that particular policy environment, e.g. by interviewing target decision makers. This could lay the foundation for a retrospective review of VFM when the benchmarks are clearer).

2. Review of Outputs

a) Degree to which the outputs have been achieved and likelihood of achieving them by the end of the grant period. Use the scoring system outlined below*** in paragraph 6.

b) Relevance, added value to existing knowledge and theory, sufficiency and scientific rigour of knowledge outputs (whether or not these are specified in the output indicators).

c) Relevance, sufficiency and quality of communication process and outputs (whether or not specified in the indicators).

45

d) Degree to which the outputs capture the gender dimensions of the research topic (whether or not this is specified in the indicators).

3. Capacity Development

a) Degree to which the research programme has developed the capacity of research programme members in research and persuasive, targeted communication (whether or not these are specified as separate outputs).

b) Is capacity building adequately dealt with in the logframe/workplans?

c) How has it been defined - as “training” or organisational capacity building?

d) Has what is promised been delivered; does it meet the needs of the partners a) to be better researchers and b) to build sustainable organisations?

e) Do members of the research programme have adequate capacity to analyse gender issues relating to the research topic? And/or what additional support do they need?

f) To what extent does the consortium have the internal skills to deliver both training/personal capacity building and organisational capacity building?

g) What additional financial assistance is needed to ensure that the agreed levels of capacity building are delivered in future DFID research programmes?

4. Partnership Management and Effectiveness

a) Degree to which all research programme partners have been involved in research and communications, as envisaged in their proposals and inception reports.

b) Effectiveness of the research programme‟s inter-partner management systems (e.g. steering committees, working groups, communications).

c) Effectiveness and influence of the Advisory Group in decision making.

d) Degree to which a consortium approach adds value to research and persuasive communications, compared with a non-consortium approach.

e) Effectiveness of the research programme‟s M&E systems, including the quality of internal systems and annual reports.

f) Lessons learned on effective research programme working.

g) Outreach to other relevant research, especially DFID Research-funded research and to the work of the International Growth Centre.

h) Effectiveness of the division of funding between the different partners, and any recommendations about any changes in the relative amounts of funding for each partner.

i) Recommendations on how future funding should be disbursed between developing country partners for maximum benefit for the research.

46

j) How the partnerships and the networks which have been built would be best sustained after 2010?

k) Recommendations on the organizational arrangements in order to strengthen the role of southern partners in overall leadership and coordination of the network? What capacities for this already exist, and what additional activities should be taken in the final years of the programme to ensure sustainability and vitality of the network?‟

5. Links with DFID

a) Degree to which DFID Research meets the research programme‟s needs in terms of management and administration.

b) Degree to which the research programme has access to DFID advisers for reasons related to achievement of its outputs and purpose.

c) Degree to which DFID advisers have access to the research programme, for similar reasons.

6. Miscellaneous

a) Utility of links with related research or influencing programmes or institutions that would increase the probability of the Research Programme meeting its objectives, or missed opportunities for links.

b) Risk assessment of the research programme, using the DFID risk assessment processes (annex 1).

c) Identification of opportunities to introduce/strengthen the use of a gender perspective in the analysis of research findings.

*** Please use the following scoring system

1. = Likely to be completely achieved. The outputs /purpose are well on the way to completion (or completed)

2. = Likely to be largely achieved. There is good progress towards purpose completion and most outputs have

been achieved, particularly the most important ones.

3. = Likely to be partly achieved. Only partial achievement of the purpose is likely and/or achievement of some

outputs.

4. = Only likely to be achieved to a very limited extent. Purpose unlikely to be achieved but a few outputs likely to

be achieved.

5. = Unlikely to be achieved. No progress on outputs or purpose

47

Outputs from the Review

The MTR team will produce a report (no more than 15 pages plus appendices) that includes:

A one page summary (which will be put on DFID‟s R4D – www.research4development.info);

A full list of recommendations that makes reference to evidence contained within the main body of the report;

A one page outline of the extent to which the programme is currently addressing gender equality issues, highlighting future opportunities for the programme to strengthen the use of a gender perspective in analysis.

A DFID „Project Scoring Annual Review‟ in accordance with DFID ARIES guidelines (Annex 2);

Recommendations about any revisions required to the logical framework with track changes;

A recommendation that the programme:

Continues as is,

Continues with modification or,

Does not continue.

Methodology

The MTR team may comprise DFID Research staff, independent consultants and other DFID staff (e.g. link advisers) as appropriate. One of the independent consultants will be the designated Team Leader. Consultants may be subject area specialists, research/policy specialists or organisational management specialists. The exact composition of the MTR team will be decided in negotiation with DFID Research.

Process

DFID will recruit the MTR team and organise key dates and deadlines for the review process for each research programme.

Each MTR team meets and holds a briefing session with DFID to agree the overall review methodology and flag any other issues e.g. any concerns known in advance.

48

The consultation and review process should include:

Interviews with the research programmes‟ partner institutions to collect information on outputs to date, achievements to date and management aspects of work;

Interviews with key stakeholders and users of the research programmes work, to include questions on the degree to which programmes have met their outputs and intended impact; what are the gaps; and what could have been done differently or better?;

Interviews with DFID Departments to ascertain the degree to which the research work has permeated their thinking and decisions;

Research programmes/DFID provides the MTR team with key documents, as outlined below.

Hold a formal MTR meeting with the review team, DFID and relevant research programme staff to discuss findings so far;

Write up findings of the MTR as per expected outputs;

Submit and finalise reports based on DFID comments;

Documents to include in the MTR

Project proposal document;

Inception phase report;

Research programme annual reports;

Documents containing research programme‟s response to any written questions from the MTR team;

Key communication successes which should keep the focus on upstream activities, not academic, peer-reviewed papers publications. These items should be identified by the research programme and focus on examples of the most persuasive communications, with an explanation of why the research programme thinks it is important. This could include anecdotes of decisions having been taken, policies or programmes that have changed or communication material that may have an impact on decision making.

Documentation from the research programmes, which identifies lesson learning from successes and challenges;

Other documents as negotiated with the MTR team, DFID and research programmes.

Timing

The Mid-Term Review will start on 6 April 2009 with a completion date of 19 June 2009. The Mid-Term Review report will be submitted by Tuesday 26 May. It is

49

envisaged that up to 15 days input for the main consultant and up to 10 days for the second team member will be needed.

Growth Team

DFID Research

24 March 2009

iiG Annex B

50

People Involved in IIG Consulted by MTR Reviewers

Lawrence Bategeka Uganda

Erlend Berg UK LSE

Tim Besley UK LSE

Tessa Bold UK CSAE

Robin Burgess UK LSE

Paul Collier UK CSAE

Stefan Dercon UK QEH

Maitreesh Ghatak UK LSE

Selim Gulesci UK LSE

Karin Loudon UK CSAE

Frederick Mugisha EPRC, Uganda

Germano Mwabu Kenya (US)

Bayo Okunade University of Ibadan

Rose Page UK CSAE

Rajasekhar Durgam India, Institute for Social and Economic Research, Bangalore,

Justin Sandefur UK CSAE

Bilal Siddiqi UK QEH

Munshi Sulaiman Bangladesh BRAC,

Alemayehu Seyoum Taffesse

Ethiopia, African Centre for Economic and Historical Studies,

Pedro Vicente Ex UK CSAE

Andrew Zeitlin UK CSAE

iiG MTR Annex C

51

List of Research Outputs

A Published Articles

1. Special issue of the Oxford Review of Economic Policy, on the Political Economy of Economic Development. Forthcoming 2009 (edited by Christopher Adam and Stefan Dercon). Toke S. Aidt, "Corruption, Institutions and Economic Development" Gani Aldashev, "Legal Institutions, Political Economy and Development" Paul Collier, "The Political Economy of State Failure" Indermit Gill and Homi Kharas, "Gravity and Friction in Growing East Asia" Ragnar Torvik, "Why do some resource abundant countries succeed while

others do not?" Pedro C. Vicente and Leonard Wantchekon, "Clientelism and Vote Buying:

Lessons from Field Experiments in African Elections" 2. Stefan Dercon and Pramila Krishnan, “Land Rights Revisited”, in The

Microeconomics of Institutions, Tim Besley and Raji Jayaraman (eds.), MIT Press (forthcoming)

3. Paul Collier and Tony Venables in "Rethinking trade preferences: How Africa can diversify its exports" (forthcoming, World Economy)

4. Philippe Aghion, Robin Burgess, Stephen Redding, and Fabrizio Zilibotti, 2008. "The Unequal Effects of Liberalization: Evidence from Dismantling the License Raj in India." American Economic Review, 98(4): 1397–1412.

5. Paul Collier and Benedikt Goderis, "Does Aid Mitigate External Shocks", forthcoming in Review of Development Economics

6. Paul Collier and Anke Hoeffler, "Testing the Neocon Agenda: Democracy in Resource-Rich Societies", forthcoming in European Economic Review. Non-gated version

7. Paul Collier and Benedikt Goderis, "Prospects for Commodity Exporters: Hunky Dory or Humpty Dumpty?" in World Economics, 2008

8. Gutiérrez Romero, R., Kimenyi, M,. Collier, P. 2009. „Elections and Institutions in Kenya‟, in Adam, C. Collier, P. (Eds.) Kenya Policies for Prosperity, Oxford University Press, forthcoming.

9. Tessa Bold, Mwangi Kimenyi, Germano Mwabu, and Justin Sandefur. “Determinants of Educational Achievement in Kenya since the Introduction of FPE.” Forthcoming in in Adam, Collier and Ndung‟u, Kenya: Policies for Prosperity.

B Working Papers 1. Christopher Ksoll, Rocco Macchiavello & Ameet Morjaria: "Guns and Roses: the

effects of Ethnic Violence on Kenyan Flower Exporters". iiG working paper 2. Rocco Macchiavello & Ameet Morjaria: "The Value of Relational Contracts:

Evidence from a Supply Shock to Kenyan Flower Exporters" iiG working paper 3. Mwangi S. Kimenyi & Roxana Gutierrez Romero, Identity, Grievances, and

Economic Determinants of Voting in the 2007 Kenyan Elections, University of Connecticut, Department of Economics Working paper 2008-38, 2008

4. Mwangi S. Kimenyi & Roxana Gutierrez Romero, Tribalism as a Minimax-Regret Strategy: Evidence from Voting in the 2007 Kenyan Elections, University of Connecticut, Department of Economics Working paper 2008-35, 2008

iiG MTR Annex C

52

5. Michael Bratton and Mwangi Kimenyi, Voting in Kenya: Putting Ethnicity in Perspective, Afrobarometer Working Paper No.95, 2008

6. Paul Collier and Benedikt Goderis, Does Aid Mitigate External Shocks, CSAE WPS/2008-18, 2008

7. Paul Collier and Pedro C. Vicente, Votes and Violence: Evidence from a Field Experiment in Nigeria, CSAE WPS/2008-16, 2008

8. Paul Collier and Benedikt Goderis, Commodity Prices, Growth, and the Natural Resource Curse: Reconciling a Conundrum, CSAE WPS/2007-15, 2007

9. Stefan Dercon and Daniel Ayalew, Land Rights, Power and Trees in Rural Ethiopia, CSAE WPS/2007-07, 2007

10. Fafchamps, Marcel, and Pedro C. Vicente (2009), Political Violence and Social Networks: Experimental Evidence from a Nigerian Election, Oxford University, Working Paper

11. Tanguay Bernard, Alemayehu Seyoum Tafesse, and Stefan Dercon. “Aspirations Failure and Well-Being Outcomes in Ethiopia: Towards and Empirical Exploration. December 2008.

12. Gutiérrez Romero, R. Dercon, S. "Decentralization, Accountability and the MPs Elections in Kenya" iiG working paper

13. Tessa Bold and Stefan Dercon. 2009. “Contract Design in Insurance Groups”. CSAE/iiG Working Paper

C Briefing Papers 1. Roxana Gutiérrez Romero, Mwangi S. Kimenyi and Stefan Dercon, "The 2007

Elections, Post-Conflict Recovery and Coalition Government in Kenya", iiG Briefing Paper, 26 September 2008

2. Roxana Gutiérrez Romero, 'Decentralization, Accountability and the MPs Elections: The Case of the Constituency Development Fund in Kenya' iiG Briefing paper, May 2009

3. Stefan Dercon, Michael Bratton, Mwangi Kimenyi, Roxana Gutierrez-Romero, and Tessa Bold, "Ethnicity, Violence and the 2007 Elections in Kenya", Afrobarometer Briefing Paper No.48, 2008

4. Paul Collier, "Survival of the Fattest", in The American Interest, 2007 5. Paul Collier and Anthony J. Venables, “Rethinking Trade Preferences to Help

Diversify African Exports”, CEPR Policy Research Insight No. 2, June 2007. D Reports 1. Pathways Out of Extreme Poverty: Findings from round I survey of CFPR phase

II 2. Baseline report on Food distribution, skill development, and financial services:

An evaluation of BRAC South Sudan‟s FFTIG program 3. From Current Practices of Justice to Rule of Law: Policy Options for Liberia‟s

First Post-Conflict Decade. Field Research Primary Findings Report. Draft April 2009. In collaboration with the United States Institute of Peace and George Washington University.

4. Management and motivation in Ugandan primary schools: Report on baseline survey. Draft May 11, 2009. Economic Policy Research Centre, Uganda and Centre for the Study of African Economies, University of Oxford, UK

iiG MTR Annex C

53

E Presentations 1. Tessa Bold, Mwangi Kimenyi, Germano Mwabu, and Justin Sandefur. “The

Determinants of Academic Performance under FPE” Presentation to the Permanent Secretary, Ministry of Education, Government of Kenya. November 2008

2. Tessa Bold, Mwangi Kimenyi, Germano Mwabu, Justin Sandefur: Free Primary Education in Kenya: Enrolment, Achievement and Local Accountability, CSAE Conference 2009, 23 March 2009.

3. Christopher Ksoll, Rocco Macchiavello, Ameet Morjaria: Kenyan Flower Exports during the Violence: a Quantitative Assessment, April 2009

4. Erlend Berg, Maitreesh Ghatak, R Manjula, D Rajasekhar, Sanchari Roy: Information and Health Care: A Randomized Experiment in India, iiG Workshop, Oxford University 21 March 2009.

5. Paul Collier: 4 ways to improve the lives of the 'bottom billion', TED (Technology, Entertainment, Design), 2008.

6. Paul Collier: Collier on the Bottom Billion, Podcast, Library of Economics and Liberty, 2008.

7. Paul Collier: The Bottom Billion, Carnegie Council, 2008. 8. Paul Collier: 'The Divergence of the Bottom Billion', DESTIN and STICERD

public lecture, Oct 2007 - abstract and podcast 9. Jonathan Conning, Klaus Deininger, Justin Sandefur, Andrew Zeitlin: Evaluating

the award of Certificates of Right of Occupancy in urban Tanzania, Methodology workshop Impact evaluation of land-related projects, World Bank. 5 March 2009

iiG MTR Annex D

54

Other Documents Consulted

In addition to the documents listed in Annex C – iiG Research Outputs, the following documents were consulted in the course of the MTR. iiG Documents

iiG RPC bid, October 2006

Submission IPPG2 Project Proposal logical framework

IFG Inception Report January 2007 to June 2007, revised, plus covering letter

iiG Annual Report July 2007 – September 2008

Correspondence with DFID accompanying revised RPC proposal and 5 yr. work plan

Minutes CSAE Policy Committee: 11.06.07; 17.01.08; 09.06.08; 10.11.08

CSAE Communications Strategy, DRAFT, May 2009

Draft Policy Briefs x 8 (2 since finalized)

Financial summaries …. iiG Documents Produced for the MTR

Capacity Building, May 2009

iiG Projects spreadsheet 220509

iiG Partners, spreadsheet, May 2009

Logframe, revised May 2009 DFID and other RPC Documents

Communications Assessment of iiG Inception report 280709

DFID feedback on iiG Annual Report 2008

Brief Synoposis Collier Institutions for Growth (undated)

RPC Collier the Big Idea and Research Themes (undated)

DFID minute 200607, and various other staff communications and file notes

iiG CRD meeting on Inception Report, note 270107

Various DFID emails to iiG

Monitoring and Evaluation: A Guide for DFID-contracted Research Programmes 0506

Communication of Research: Guidance Notes for Research Programme Consortia, 1005

DFID Communications Strategy, Draft for Consultation 2008 (undated)

Research for Development (R4D) website and Communications Corner

IIG MTR Annex E

55

Capacity Building with iiG Research Partner Organisations

Multi-Country

1/2008. Workshop on Econometric Methods for Program Evaluation, “Improving Institutions for Pro-Poor Growth” (iiG) Research Programme Consortium (with Germano Mwabu and Justin Sandefur). Theory and applied statistical work, providing training with Stata software. All iiG African partner research organisations were invited to attend. Approx. 25 participants attended from our Kenyan and Ugandan partners, as well as a small number of additional participants from the Ugandan research community (Makerere University, Uganda Bureau of Statistics).

Uganda

5/2008. Dual-purpose workshop: (1) Impact Evaluation workshop for Government & Civil Society (Uganda education sector); (2) collaborative project design with key stakeholders in the education sector. 25 participants including research collaborators from Economic Policy Research Centre, Uganda; Ministry of Education and Sports; Uganda National Examinations Bureau; Uganda Bureau of Statistics; District Education Office staff from four project districts; civil society (ADRA; SNV; World Vision).

12/2008. Work on Impact Evaluation workshop for Health Care Managers of Regional and District Referral Hospitals. This is in response to government efforts to mainstream impact evaluation in government programs and projects. 30 participants from Regional and District Referral Hospitals, including participants from Ministry of Health and Civil Society Organisations.

2/2009. Mentoring of young research fellows at Economic Policy Research Centre to develop and write scientific papers. One of the papers was presented at the recent CSAE conference.

Joint research and co-authored papers. Extensive in-country collaboration through Andrew Zeitlin and visit of Lawrence Bategeka to Oxford.

Kenya

3/2008. 3 day training workshop on panel data techniques and impact evaluation in Nairobi for the Ministry of Education and Kenya National Bureau of Statistics officials and some researchers from the Central Bank of Kenya.

4/2009. Research retreat for Ministry of Education and University of Nairobi research team on research cross-cutting designs and implementation.

5/2009. Research retreat for Ministry of Education and University of Nairobi research team on survey design and conduct.

5/2009. Training day for junior and senior Ministry staff organised by Mr Obiero is planned on utilitsing the EMIS data base and randomised impact evaluation in evidence-based policy making.

IIG MTR Annex E

56

5/2009. Workshop planned with World Vision on evaluation techniques and training of community development motivators to collect data on SMCs and parents in 192 treatment and control schools.

Joint research and co-authored papers. Extensive in-country collaboration through Justin Sandefur and Tessa Bold. Prof. Mwabu and Prof. Kimenyi visited Oxford.

Ethiopia

9/2008. A Trust fund from the World Bank provided funding training for poverty research using panel data in Ethiopia in September/October, with inputs from iiG staff (Sandefur, Dercon and Seyoum). Staff linked to the iiG both in Ethiopia and Oxford (Seyoum and Dercon) provided additional time to act as resource persons providing supervision and comments on research papers produced by researchers from regional universities on poverty and growth.

Joint research and co-authored papers. Extensive in-country collaboration through Stefan Dercon. Alemayehu Seyoum visited Oxford.

Nigeria

They have declined.

India

The preferred capacity building route was chosen to be joint research, given their access to alternative capacity building initiatives in India.

Professor Rajasekhar visited LSE.

Manjula from ISEC is coming to the LSE this autumn for two months, she will attend lectures/seminars and work with on our project.

Bangladesh

A training workshop on survey, data analysis and impact evaluation techniques at BRAC in Bangladesh this summer will be offered by iiG/LSE staff.

Munshi Suliaman (BRAC) is now doing a PhD at the LSE and his research will focus on the BRAC iiG work in Africa.

An externality of the iiG work is that there is now a joint LSE/BRAC effort to build up the capacity of the BRAC offices in Uganda. Munshi Suliaman is involved with this and will be able to provide more information.

There is also a joint LSE/JPAL/BRAC effort to build up the capacity of the BRAC office in Sierra Leone, planned for later this year.

Robin Burgess visited Bangladesh.

IIG MTR Annex E

57

Capacity Building in other iiG Projects Tanzania

8/2008. Three-day seminar on tools of impact evaluation, with applications to evaluating land policies. Approx 15 participants from Ministry of Lands, Housing and Human Settlements Development (Department of Policy and Planning and Strategic Plan for Implementation of Land Laws project staff)

iiG MTR Annex F

Projects Partly or Wholly Funded

by iiG

Researchers Start

year

Category

A: Mainly funded by the

project

B: Only fieldwork and/or

RA

C: Minor contribution

Costs Comments Outputs

Institutions for Accountability

Accountability and Economic

Performance

Paul Collier (CSAE, Oxford)

Anke Hoeffler (CSAE, Oxford)

Benedikt Goderis (CSAE,

Oxford)

Lisa Chauvet (DIAL)

2007 A only staff time DFID (Hoeffler) Ongoing, several

policy and

research outputs

A6, A7, B6, B8,

C4, E5, E6, E7,

E8

Community-Based Justice and the

Rule of Law in Liberia: A Field

Experiment Using Mobile Paralegals

Bilal Siddiqi (CSAE, Oxford)

Justin Sandefur (CSAE, Oxford)

Andrew Zeitlin (CSAE, Oxford)

Stephen Lubkemann (GWU)

Deborah Isser (USIP)

2008 A £12,000 OSI

staff time DFID (Siddiqi)

Baseline

completed;

intervention

underway.

Baseline data

being cleaned and

analyzed. Policy

presentations end

of May

D3

Determinants of Leadership Turnover Paul Collier (CSAE, Oxford)

Anke Hoeffler (CSAE, Oxford)

2008 A staff time DFID (Hoeffler) Ongoing

Does Crime Pay for Politicians in

India

Timothy Besley (LSE)

Robin Burgess (LSE)

2007 B RA time DFID Ongoing

Elections, Ethnic Polarization and

Managing Post-Electoral Conflict in

Kenya

Mwangi S. Kimenyi

(Connecticut)

Roxana Gutiérrez Romero

(DID, Oxford)

Stefan Dercon (DID, Oxford)

Michael Bratton

(Afrobarometer)

Pedro Vicente (CSAE, Oxford)

Tessa Bold (CSAE, Oxford)

2007 A £11,500 VCEP

£9,500 Hewlett

£31,000 DFID (CSAE)

£37,000 DFID (QEH)

staff time DFID (Romero)

Two rounds of

surveys

completed.

Several research

outputs and policy

outreach events

A8, B3, B4, B5,

C1, C3

iiG MTR Annex F

Projects Partly or Wholly Funded

by iiG

Researchers Start

year

Category

A: Mainly funded by the

project

B: Only fieldwork and/or

RA

C: Minor contribution

Costs Comments Outputs

Managing Irrigation Tanks: How do

Institutions Matter in Karnataka, India

D Rajasekhar (ISEC)

R Manjula (ISEC)

Suchitra J Y (ISEC)

Tim Besley (LSE)

Maitreesh Ghatak (LSE)

Erlend Berg (LSE)

2007 A £13,448 DFID

plus staff time DFID (Berg)

Closed -

Political Violence in Nigeria: A Field

Experiment around the 2007

Elections

Paul Collier (CSAE, Oxford)

Marcel Fafchamps (CSAE,

Oxford)

Pedro Vicente (CSAE, Oxford)

Michael Bratton

(Afrobarometer)

2007 A £5,000 OSI

£73,994 Hewlett

Two rounds of

surveys

completed.

Several outputs

available on web

A1, B7, B10,

fieldwork

reports

(multiple)

Randomized Evaluation of Policies to

Create Local Accountability Under

Free Primary Education in Kenya and

Uganda

Germano Mwabu (Nairobi)

Mwangi Kimenyi (Connecticut)

Lawrence Bategeka (EPRC)

Frederick Mugisha (EPRC)

Abigail Barr (CSAE, Oxford)

Tessa Bold (CSAE, Oxford)

Justin Sandefur (CSAE, Oxford)

David Johnson (DOE, Oxford)

Geeta Kingdon (CSAE, Oxford)

Stefan Dercon (DID, Oxford)

Andrew Zeitlin (CSAE, Oxford)

2007 A Uganda £23,400 VCEP [EPRC

subcontract]

£7,904 DFID

staff time DFID (Zeitlin)

Kenya £12,500 OSI

£14,788 Hewlett

£21,441 DFID

staff time DFID (Bold and

Sandefur)

Ongoing A9, D4, E1

Repatriation, conflict strategies and

mediation in Burundi

Christopher Ksoll (CSAE,

Oxford)

2008 C £1,000 OSI

£3,151 VCEP

£504 DFID

no staff time

Stalled

Sao Tome Vote Buying Pedro Vicente 2007 C £4,636 Hewlett Ongoing A1

iiG MTR Annex F

Projects Partly or Wholly Funded

by iiG

Researchers Start

year

Category

A: Mainly funded by the

project

B: Only fieldwork and/or

RA

C: Minor contribution

Costs Comments Outputs

Sierra Leone Criminal Justice

Paralegals

Bilal Siddiqi (CSAE, Oxford)

Justin Sandefur (CSAE, Oxford)

2008 C £3045 VCEP Early stage

Social Funds and Political Patronage

in Kenya

Stefan Dercon (DID, Oxford)

Roxana Gutiérrez Romero

(DID, Oxford)

2007 A [see below for IPAR contract

cost in 2007]

staff time DFID (Romero)

Initially with IPAR,

in process of

being revived.

B12, C2

The Election Process in Nigeria:

Candidate Selection, Finance and

Outcomes

John Ayoade (Ibadan)

Bayo Okunade (Ibadan)

Pedro Vicente (CSAE, Oxford)

Michael Bratton

(Afrobarometer)

Paul Collier (CSAE, Oxford)

Jennifer Widner (Princeton)

Adrienne LeBas (Oxford)

Nic Cheeseman (Oxford)

2007 A £21,413 DFID Ongoing A1

Institutions for Manufactured Exports

Contractual Relationships and Ethnic

Violence: Theory and Evidence from

Kenyan Flowers Exports

Christopher Ksoll (CSAE,

Oxford)

Rocco Macchiavello (CSAE,

Oxford)

Ameet Morjaria (LSE)

2008 B £3,000 DFID

£1,000 VCEP

no staff time

Analysis ongoing.

Papers presented

in several

conferences and

one policy

workshop

B1, B2, E3

EPZs and Social Funds IPAR 2007 B £7,104 DFID Closed

Growth in Manufactured Exports

Across African States

Paul Collier (CSAE, Oxford)

Francis Teal (CSAE, Oxford)

Justin Sandefur (CSAE, Oxford)

2007 A only staff time DFID (Sandefur) Ongoing A3, C5, E5, E6,

E7, E8

Liberalization Meets Investment

Climate: The Case of India

Robin Burgess (LSE) 2007 B only RA time DFID Paper published A4

Institutions for Rural Transformation and Development

iiG MTR Annex F

Projects Partly or Wholly Funded

by iiG

Researchers Start

year

Category

A: Mainly funded by the

project

B: Only fieldwork and/or

RA

C: Minor contribution

Costs Comments Outputs

A sustainable transition: food aid and

income generation in South Sudan

Oriana Bandiera (LSE)

Robin Burgess (LSE)

Selim Gulesci (LSE)

Imran Rasul (UCL)

2008 B staff time DFID (Gulesci) Baseline

completed;

intervention

underway

D2

Aspiration Failure and Rural

Development in Ethiopia

Alemayehu Seyoum (ACEHS)

Stefan Dercon (DID, Oxford)

Tessa Bold (CSAE, Oxford)

2007 A £4,500 OSI

staff time DFID (Bold and

Dercon)

Ongoing B11

Cyclones and development in India Stefanie Sieber (LSE) 2008 C RA time DFID Early stage

Empowering the Very Poor: The

Ultrapoor Programme in Bangladesh

Oriana Bandiera (LSE)

Robin Burgess (LSE)

Selim Gulesci (LSE)

Imran Rasul (UCL)

Imran Matin (BRAC)

Munshi Sulaiman (BRAC)

Farhana Hussain (BRAC)

2007 A £59,590 DFID

staff time DFID (Gulesci)

Baseline

completed;

intervention

underway

D1

Female Empowerment through

Inheritance Rights: Evidence from

India

Sanchari Roy (LSE) 2008 B RA time DFID First round of

results available

Human Capital, Financial capital, and

the Economic Empowerment of

Female Adolescents: Evidence from a

Randomized Intervention in Uganda

and Tanzania

Oriana Bandiera (LSE)

Robin Burgess (LSE)

Markus Goldstein (World Bank)

Selim Gulesci (LSE)

Imran Rasul (UCL)

2008 B staff time DFID (Gulesci) Ongoing

Information and Healthcare: A

Randomized Experiment in India

Erlend Berg (LSE)

Maitreesh Ghatak (LSE)

R Manjula (ISEC)

D Rajasekhar (ISEC)

Sanchari Roy (LSE)

2008 A £14,360 DFID

staff time DFID (Berg)

Baseline

completed; data

being cleaned

E4

Infrastructure and Development – The

Political Economy of Road Placement

in East Africa

Ameet Morjaria et al (LSE) 2008 C RA time DFID Early stage

iiG MTR Annex F

Projects Partly or Wholly Funded

by iiG

Researchers Start

year

Category

A: Mainly funded by the

project

B: Only fieldwork and/or

RA

C: Minor contribution

Costs Comments Outputs

Property Rights in East Africa Lawrence Bategeka (EPRC)

Andrew Zeitlin (CSAE, Oxford)

Stefan Dercon (DID, Oxford)

2007 C £3,000 VCEP

no staff time

Ongoing A2, A9, B9, E9

Psychosocial skills and Aspirations:

Evaluating the impact of a long-term

intervention in India

Sofya Krutikova (EDI)

Pramila Krishnan (Cambridge)

K. Sriram (India)

2008 C £3,000 DFID Early stage

Pure home water in Ghana Greg Fischer (LSE) 2009 C RA time DFID Early stage

Risk, Shocks, Growth, Aid and

Poverty

Paul Collier (CSAE, Oxford)

Stefan Dercon (DID, Oxford)

Benedikt Goderis (CSAE,

Oxford)

2007 C limited RA time Ongoing A5, B6, E5, E6,

E7, E8

The welfare and behavioural impact

of group based insurance provision in

Ethiopia

Stefan Dercon (DID, Oxford)

Tessa Bold (CSAE, Oxford)

2008 B £7,909 DFID

staff time DFID (Bold)

Ongoing B13

Annex G

Narrative Summary (NS)

Verifiable Indicators (OVI)

Means of Verification (MOV)

Comments

Goal: (DFID) The production and uptake of technologies and policies that will contribute to poverty reduction and the achievement of the MDGS

(No need to complete)

(No need to complete)

This version 22/05/2009 08:41:57

Purpose: To provide governments and civil society in Africa and Asia, and development agencies, with usable and useful knowledge as to how institutions can be redesigned so as to be more conducive to growth and empowerment.

1. Specific policy change in the form of changes in institutional design in countries and localities, consistent with knowledge generated by the programme. 2. Engagement of local and international RPC researchers in high-level policy dialogue on institutional design in the context of growth and poverty reduction.

1. Reference in key policy documents, including the PRSP, to the RPC team’s research findings, as a basis of informing institutional change and policies. 2. Listing of at least six invited addresses and contributions on institutions, growth and poverty reduction based on RPC findings and related evidence, to top level officials and core civil society representatives, internationally and in the RPC partner countries.

1.

Collier & Venables EU EBA trade preferences policy

Collier Extractive Industries Transparency Initiative

2. See Communications Strategy document for details

Outputs: 1. New comparative and context specific quantitative data sets on institutions.

1. High quality data sets available initially to collaborators, and subsequently on the web.

1. An assessment of data sets on the web, and supporting documents on the data (from year 3, at least one per year

1.

Datasets compiled for several projects; many are now being cleaned and

Annex G

2. Country-specific research insights on links between institutions, growth and poverty, quantitatively validated and with specific recommendations for institutional and policy design. 3. Lessons from comparative research across countries, quantitatively validated, on the links between institutional design, development policies, growth and poverty, with specific recommendations on institutional design issues.

Data sets involved include data for specific research in Nigeria, India, Kenya and Ethiopia. At least 2 data sets will combine data on political power and process, socio-economic data and outcomes related to efficiency and equity. 2. At least six research papers per year with country-specific analysis on themes of the RPC, circulating and presented widely and published in peer-reviewed journals. 3. At least three research papers per year with comparative evidence across countries on the themes of the RPC circulating and presented widely and published in peer-reviewed journals.

posted). 2. A yearly systematic listing of all papers written, by nature of publication, including peer-reviewed journals, and of all presentations based on the research. 3. A brief yearly research summary document, by theme, on the research conducted and its dissemination.

analyzed.

All datasets are available to local partners and will begin to be posted from year 3, beginning with the Kenya elections dataset compiled with Afrobarometer

Kenya education dataset will be released soon.

2. See Project List document 3. The web has become the clearest repository for this information, and no research ‘summary’ was produced

iiG research-based literature reviews can be generated once there is enough critical research

Annex G

4. Strengthened capacity of partner institutions in RPC in analyzing the role of institutions and their reform for growth and poverty. 5. An accessible body of research output for skilled policy makers and other decision-makers on the themes of the RPC.

4. Two papers per year at least co-written and presented by developing country based collaborators, especially by the African teams, some using data collected by the project and showing capacity building in evidence-based policy research; 5. Widely distributed reports, papers and policy briefs, based on RPC research, targeted to specific audiences of policy makers and decision-makers

4. Yearly specific documentation on all the capacity building activities and the outputs of partners, especially Africa-based, to show capacity building 5. A report on the communication performance, including records of presentation, targeted policy papers and briefing notes, and media coverage. Monitoring of policy and practice outcomes that could be attributed to iiG work, in UK, internationally and in partner countries.

on an issue (e.g. elections, education accountability, aspirations)

Policy briefs produced similarly

The latest Collier book has been strongly influenced by research ongoing in iiG

4. See Capacity Building document.

Selected joint papers (see list of Research Outputs) A8, A9, B3, B4, B5, B11 5. Several policy papers and briefing notes available (see list Research Outputs) See Communications Strategy document

Annex G

6. Accessible, non-technical output based on the research on institutions, growth and poverty. 7. Web-based accessible information on the research of the RPC as a platform for user-friendly engagement with media, policy-makers and civil society. 8. An up-to-date research agenda reflecting changing policy priorities on institutions and development.

6. Per year, two briefing notes, and well-designed presentations for meetings and other events, and for targeted media coverage. 7. An up-to-date website with accessible information and research results 8. A log-framework and other research coordination documents that reflect the current research and includes quantified specific targets in the log-framework

7. Access statistics on the website. 8. Updated log-framework approved by DFID with clear quantitative and other targets and agreed means of evaluation.

6. Same as above 7. To be obtained 8. Under review

Activities: 1. Consultative planning of research, communications and capacity building

Inputs: 1. Planning meetings with senior researchers at all institutions (two joint project RPC planning workshops per year, plus one meeting per year of senior RPC staff with

1. Evidence of sufficient time and funding allocated to each of these elements of the RPC, including research time, managerial inputs, research coordination, website design

1. On track; see point #3 below

Annex G

2. Conducting data collection and collating secondary data sources. Archiving data 3. Analyzing data and writing research papers 4. Presentations and other interaction with academic and policy audience to improve

country team, each involving other stakeholders). 2. Commissioning specific research around themes and projects (ensuring that there is always at least one ongoing specific project involving each of the partners at any point of time involving original research on newly compiled or collected data). 3.1 Data collection,

cleaning and archiving activities, in line research projects under each theme.

3.2 Data analysis and research on themes of the RPC. 4. Research workshops with IFG teams and other invited researchers/resource

and management, data collection, capacity building, producing research outputs, presentation, dissemination strategy and other dissemination activities.

2. Details on website.

IPAR dropped.

New project started with EDRI Ethiopia and ESSP/IFPRI in Ethiopia.

3. Ongoing 4. Specific sessions at CSAE conferences (last afternoon)

2007: Session on keynotes by John

Annex G

research analysis and stimulate policy change 5. Specific capacity building activities targeted at African and other developing country partners (training and collaborative research) 6. Implementing dissemination and communication activities, including targeted dissemination towards high level policy makers and core civil society representatives

persons. (Two internal workshops, of which one in one of the partner institutions from 2008, and one public research conference per year.) 5. Dissemination meetings and presentations, focusing on target audiences (Including one specific dissemination event of results in each of the partner countries per year from year 3)

3. 6. Specific researcher exchanges, visit and training workshops focused on capacity building and for hands-on collaboration in analysis (including a Visiting

Bellows & Paul Collier

2008: Round table on elections in Africa (David Anderson, Nic Cheeseman, Paulo Vicente, Mwangi);

2009: presentation of papers on OxREP issue on political economy of development, corruption, politics

5.

Kenya dissemination meeting candidate selection, election violence, schools, corruption

Similar meeting in India next year.

See Policy Outreach document for further details

6.

Alemayehu Seyoum, Lawrence Bategeka, Germano Mwabu, Mwangi Kimenyi on extended research visits to Oxford

D Rajashekar visited LSE

Annex G

7. Regular monitoring and review of scope and impact of RPC to update research and dissemination plans.

Fellowship scheme at CSAE, one per year for up to 3 months – 4 in total; one training workshop or similar event in year 1 or 2 on quantitative evaluation methods for each partner country, and at least yearly visit of senior researcher to southern partner institution).

7. Producing policy briefing notes for media interaction, including posted in blog style. (Selection of three specific dissemination themes per year on which for two of them briefing notes will be produced, from year 2, and brief yearly research summary for wider distribution.) 8. Regular meetings of Management Advisory Group (twice per year) and Oversight Committee (twice a year)

Justin Sandefur, Tessa Bold, Andrew Zeitlin visits to Kenya/Uganda/Tanzania

Stefan Dercon visit to Ethiopia

Robin Burgess visit to Bangladesh

Tim Besley visit to India 7. See Policy Outreach document and list of Research Outputs 8. Taken place as planned (minuted)

Annex H

Part F. Knowledge Sharing

Knowledge Sharing

1. Working with partners

2. Best Practice/ Innovation

3. Project Management

See comment on working with partners above

This section allows you to share any knowledge you have gained from the project over the past year.

Lessons learned and suggested dissemination. (You must complete at least one of these sections).

Key Issues/ Points of Information

If appropriate, please comment on the effectiveness of the institutional relationships involved with the project eg comment on

processes and how relatonships have evolved

The project has had excellent relationships with some key partners, but very poor relationships with

others. It should seriously consider whether continued investment in some Southern partners

partnerships will actually deliver commensurate outputs and, if not, formally end relationships with

Southern partners that are not delivering results. It should also do more to facilitate learning,

information sharing and comparative work across its Southern partners and more broadly amongst

the programme members

By working in and through government bodies and large NGOs in testing the effectiveness of

interventions the project is ensuring that the lessons from the research become directly embeded in

policy and practice.

Project management has been effective on content and value for money - but should be more

focussed and exploit more formal management tools to be able to demonstrate progress when

made.

Annex H

See Annex to MTR

Please list any key documentary evidence which is available to support the conclusions of this report. List any supporting

documents annexed to this report. Give Quest numbers where applicable.

Annex F

Part G. Additional Comments

If you have any additional comments or would like to go into any issues in more detail please use the spaces below.

Annex F

PRISM/ARIES SummaryDO NOT TYPE INTO THIS SHEET

This information comes from the other sheets which you have filled in

The information highlighted in yellow below should be entered into PRISM/ARIES.

Note there is a character limit on some of the fields

Review Type Annual Review

Review Date 00/01/1900

Project Scoring Assessment

Purpose Statement

Purpose Objectively

Verifiable Indicators

(OVIs)

Project Purpose Score 2

Project Purpose Risk Low

Purpose Justification

Output

Impact

Weights

Output

Risk Score

1 10% Low 2

2 25% Low 1

3 10% Medium 3

4 10% Medium 3

5 15% Low 2

6 10% Low 2

7 10% Low 2

8 10% Low 2

9 0% 0 0

10 0% 0 0

Scoring - Sources of

Information

Scoring - Partners

Involved

To provide governments and civil society in Africa and Asia, and development

agencies, with usable and useful knowledge as to how institutions can be

redesigned so as to be more conducive to growth and empowerment.

1 Specific policy change in the form of changes in institutional design in countries

and localities, consistent with knowledge generated by the programme. 2

Engagement of local and international RPC researchers in high-level policy

dialogue on insti

Some excellent, innovative and influential research has been done. Greater

An up-to-date research agenda reflecting changing policy priorities on institutions and development.

0

0

An accessible body of research output for skilled policy makers and other decision-makers on the themes of the RPC.

Accessible, non-technical output based on the research on institutions, growth and poverty.

Web-based accessible information on the research of the RPC as a platform for user-friendly engagement with media, policy-makers and civil society.

Impact Weighted

Output Scoring

New comparative and context specific quantitative data sets on institutions.

Country-specific research insights on links between institutions, growth and poverty, quantitatively validated and with specific recommendations for institutional and policy design.

Lessons from comparative research across countries, quantitatively validated, on the links between institutional design, development policies, growth and poverty, with specific recommendations on institutional design issues.

Strengthened capacity of partner institutions in RPC in analyzing the role of institutions and their reform for growth and poverty.

Annex F

Lessons Learned

Working with partners

Best Practive/ Innovation

Project Management

Conditionality

There may be only 1 box in PRISM to fill

Disbursement

Suspended? 0

Cause?

Date Suspended? 0

Consequences? 0

The project has had excellent relationships with some key partners, but very poor

relationships with others. It should seriously consider whether continued

investment in some Southern partners partnerships will actually deliver

commensurate outputs

0

By working in and through government bodies and large NGOs in testing the

effectiveness of interventions the project is ensuring that the lessons from the

research become directly embeded in policy and practice.

Project management has been effective on content and value for money - but

should be more focussed and exploit more formal management tools to be able to

demonstrate progress when made.