Impact evaluation as a learning and accountability tool for agricultural extension programs:...

Impact Evaluation as Learning and Accountability Tool for

Agricultural Extension Programs: Challenges and

Prospects HAILEMICHAEL TAYE BEYENE

INTERNATONAL LIVESTOCK RESEARCH INSTITUE (ILRI)

Outline Introduction and Objectives Materials and Methods Results and Discussions Challenges of IE of Agricultural Extension Programs Future Prospects of IE of Agricultural Extension

Programs Conclusions and Recommendations

Introduction and Objectives… Agricultural extension has long been recognized as an

important factor in promoting agricultural development (Birkhaeuser et al., 1991; and Anderson and Feder, 2007).

Consequently, the Sub-Saharan African (SSA) countries have been implementing various agricultural extension programs to improve agricultural production and productivity.

Impact evaluation is an attempt to establish a casual relationship between an intervention and change in outcomes.

If impact evaluations are not conducted appropriately, could lead to wrong conclusions which may lead to wrong decisions.

•

Introduction and Objectives… The difficulty of tracing the casual relationship

between extension input and its impact is appreciated by various authors (Anandajayasekeram et al., 2008; Davis. K, 2008; Anderson, 2007).

Several evaluations have been conducted on agricultural extension programs in SSA by various parties.

This paper, based on analysis of some impact evaluation studies conducted in SSA attempts to identify challenges and future prospects of impact evaluation of agricultural extension programs in the region.

Materials and Methods Review of some impact evaluation reports on

agricultural extension programs in SSA. A total of 21 impact evaluation results from 10

countries were collected from various websites for the analysis.

Agricultural productivity trends reported by various authors are used.

Mainly based on qualitative analysis. The quantitative data are analyzed using simple

descriptive statistics mainly percentages.

Results and Discussions Agricultural Production and Productivity

Most literatures agree that the performance of the agricultural sector in SSA has been disappointing.

Per capita agricultural production was deteriorating in Africa during the past 45 years (Steven H and Peter B. R. H, 2010)

Due to this the continent has turned from a food exporter to a net food importer.

Yields per ha is stagnant, particularly for cereals, in contrast to substantial yield increases in other regions of the world in the years from 1960 to 2005 ( Staatz. JM & Dembele NN, 2007).

Yield per ha is stagnant shows that the yield production growth is due to mostly area expansion (limited technological change )

Results and Discussions...Agricultural Extension Impact Evaluation Results 15(71%) impact evaluation studies reported positive

significant impact. 6 (29%) reported non significant impact. Some of the impact evaluation studies also contradict

each other. For instance: Bindlish & Evenson (1997) vs Gautam &

Anderson (1999) Ayele et al (2005) and IFPRI (2008) vs

EEA/EEPRI (2006)

Results and Discussions...

Contradictions Agricultural productivity trends..low

technological change. The IE results themselves.

Such exaggerations and contradictions lead to skepticism on the results (Davis, K. 2008).

Challenges of IE of Agricultural Extension Programs

Impact evaluation is about attribution. it is difficult to attribute a change in outcomes to a specific intervention since there are so many different factors involved. Two challenges:

Controlling the extraneous effects and design effects for estimating the net impact.

The other inherent challenge in impact evaluation is estimating the counterfactual.

Challenges of IE of Agricultural Extension Programs…

Comparable Control Group: difficult to establish control groups both from operational and ethical considerations.

Selection bias: People involvement in agricultural extension programs is based on interest and resource availability. Additionally, trying new technology requires the capacity to take risks.

Endogeneity in extension-farmer interaction: Extension contact which plays central role for success of extension programs is highly affected by farmers’ characteristics and actions.


Other sources of information and technology: There are sources of information and technology other than the extension program.

Extension to be effective needs other support services.

Agriculture especially in the Sub-Saharan African countries is highly dependent on nature and it is complex, diverse and risk prone.


Methodological Factors Indicators used to measure outcomes: This is

related to the definition of impact evaluation. ‘Impact’ as the final level of the causal chain. The difference in the indicator of interest (Y) with the

intervention (Y1) and without the intervention (Y0). That is, impact = Y1 – Y0. It is about attribution by identifying the counterfactual value of Y (Y0) in a rigorous manner.

The two definitions could overlap if Y is an outcome

indicator.


Impact Evaluation Designs: impact evaluations in SSA have shortfall in employing appropriate evaluation designs.

Sampling: The sample should not only be large enough but also random so that it can represent the population and provide adequate statistical power.

Statistical analysis: Regression techniques are preferred for analysis in impact evaluations so that the effects of confounding factors could be controlled.

Research Design and Implementation: In SSA due to lack of skilled manpower, budget, time and logistics evaluations are done in tight schedule and in a hurry.


Lack of baseline and reliable data Shortage of Skilled Manpower to Conduct Rigorous

Impact Evaluation Poor M&E system Budget constraints for Impact Evaluation Studies

Future Prospects of IE in SSA Paradigm shift in agricultural research and extension

calls for review the linear M&E approaches. Creating casual relationships between interventions and

outcomes will not be easy due to the complex, dynamic and institutional pluralism.

More focus on learning. Shift from proving to improving impact.

Conclusions and Recommendations Conclusions Agricultural productivity is disappointing. Majority of IE reported positivity results. In some cases contradictory results. There are methodological, data and capacity related

factors that affect the results of IEs.

Recommendations Building evaluation capacity, establishing appropriate

M&E systems. Employing easy, less costly ,flexible, participatory,

and learning focused tools. From proving to improving impact.

Impact evaluation as a learning and accountability tool for agricultural extension programs:...

Education

Transcript of Impact evaluation as a learning and accountability tool for agricultural extension programs:...