Evaluation ·

Abdillahi Aboud Delius Asiimwe Mustafa Babiker

DHP PUBLICATIONS SERIES No.6, June 200 t

anual

Organization for Social Science Research in Eastern and Southern Afr ica (OSSREA)

DRYLAND HUSBANDRY PROJECT (DHP)

© 2001 Organization for Social Science Research in Eastern and Southern Africa (OSSREA)

All Rights Reserved Published 2001 Printed in Ethiopia

ISSN 1608-8891

Typesetting: Misrak Kirifemichael

This Manual is the exclusive property of the Organization for Social Science Research in Eastern and Southern Africa (OSSREA). Any use, copy, reproduction, or quotation of any nature must be accompanied by the prior consent of The Regional Project Coordinator, DHP/OSSREA.

Cover photographs from DHP by:

Tegegne Teka, Kazo County, Mbarara, Uganda E.N. Sabiiti, Workshop in Kazo County, Mbarara, Uganda Diress Tsegaye, Aba'ala, Afar Region, Ethiopia

Organization for Social Sc ience Research in Eastern and Southern Africa P.O.Box 3 1971 , Addis Ababa, Ethiopia Fax: 251-1-551399 E-maiL ossrea@telecom.net.et

pub. ossrea@telecom.net.et http: //www.ossrea.org

Tegegne Teka: Edi tor, DHP Publications Series & Regional Project Co-ordinator Dryland Husbandry Project

Regional Advisory Committee of the Dryland Husbandry Project

Prof. Abdel Ghaffar M. Ahmed (OSSREA, Ethiopia) Prof. Anders Hj ort afOrnas (EPOS, Linkbping University, Sweden) Dr. Kassim O. Farah (P INEP, University of Nairobi, Kenya) IGAO (OJ ibouti) Dr. Hashim EI Atta (University of Khartoum, Sudan) Prof. E. N. Sabiiti (Makerere University, Uganda) Dr. Nashon Musimba (University of Nairobi, Kenya) Dr. Bisrat Gebru (University of Asmara, Eritrea) Dr. Mitiku Haile (Mekelle Uni versity, Ethiop ia) Dr. Tegegne Teka (OSSREA, Ethiopia)

Editorial Address:

DHP Publications Series OSSREA P.O. Box 3197 1 Addis Ababa, Ethiopia Tel: 251-1-551163/553281 Fax: 251-1-551399 E-mail: ossrea@ telecom.net.et http: //www.ossrea.org

The Dryland Husbandry Project (OSSREA) acknowledges the support of the Swedish International Development Cooperation Authority (Sida/SAREC).

Table of Contents

List o/Tables

Acronyms and Abbreviations

Foreword

Pre/ace

Acknowledgements

•

PART I: INTRODUCTION

.. VII

• • • VIlI . IX

. XI . . .

XIII

I

I. The Dryland Husbandry Project I

2. The Project Objectives, Outputs and Activities 2

PART II: EVALUATION MEASUREMENTS AND STANDARDS 6

Evaluation Tools 6

Tool \:

Tool 2:

Tool 3:

Tool 4:

ToolS:

Tool 6:

Tool 7:

Schematic Review Approach

Plan of Operation

Monthly/Quarterly Summary of Project Activities

Monthly/Quarterly Summary of Project Results/Outputs

Findings and Recommendations of the Project Evaluation

Mission

Questionnaire for National Steering Committee Level

Questionnaire for Field Management Level

6

7

\I

13

15

16

16

deve lopment projects at the community level. Contri butions to its furth er improvement in fo rm and conten t co uld come from those in DHP and from others in the deve lopment prac ti ce.

The Regiona l Offi ce of DHP (OSS REA) is thankfu l for the team of consultants who prepared this Evaluation Manual. The fo ll owing team members are selected on the bas is o f the ir association and fam il iarity w ith D HP acti vities in their respecti ve countries .

Prof. Abdillahi Aboud. Prof. Aboud is a sen ior academic and Dean at Egerton Uni versity, Nj oro, Kenya. He was commissioned because of hi s long-term assoc iation with DHP Kenya . Moreove r, he led the interna l evaluati on for DH P Kenya in 1997 and hi s contribut ion in teachi ng in the Pastoral Info rmation Network Programme (PINEP) based in Nai robi Uni versity.

Mr. Delius Asiimwe. Mr. Asii mwe is a senior research fe ll ow at Makercre Ins titu te of Social Research (MI SR), Makerere Univers ity, Kam pa la, Uganda. He was comm issioned on the accoun t of his long-term associati on w ith DHP Uganda and led the internal eva luat ion for DHP Uganda in 1997.

Dr. Mustafa Babiker. Dr. Bab iker is a senior research fe ll ow at the Deve lopment Studies and Research Centre (DSRC), Uni versity of Kharto um , Khartou m, Sudan . He was commiss ioned on the strength of hi s long years of invo lvement in the Nat ional Steering Committee (NSC) of DHP Sudan. The NSC directs DI-IP at the country level. He was al so a Lia ison Officer of OSSREA in the Sud an from 1992 to August 2000.

Dr. Tegegne Teka DHP Regional Co-ordinator & Editor, DHP Publications Series OSSREA P.O.Box 31971

•

Addis Abeba, Ethiopia June 2001

Tool 8:

Tool 9:

Questionnaire for Local Community Projects

DHPs Local Community Project Activities and . . .

Contributions

Tool 10: Financial Management and Accouptability

Tool II: Itinerary and Evaluation Activities

Tool 12: People Interviewed

Tool 13: Baseline Data and Impact Assessment

PART III: DHP Standardized Evaluation Reporting Format

17

17

19

21

23

25

26

Annex I: Questionnaire at the National Steering Committee Level 29

A. Respondents Demographic Data 29

B. Perceptions of Long-term Objectives of DHP 30

C. Perceptions oflmmediatt'. Objectives

D. Perceptions ofDHP Outputs

E. Perceptions of DHP Activities and Linkages

in Implementation

Annex II: Questionnaire at the Field Management Level

Annex III: Questionnaire at the Local Community Level

REFERENCES

33

36

39

42

46

50

•

List of Tables

Table I: Plan of Operation 9

Table 2: Monthly/Quarterly Summary Of Project Activities 12

Table 3: Monthly/Quarterly Summary of Project Results/Outputs 14

Table 4: Findings and Recommendations of the Project Evaluation Mission

Table 5: DHPs Local Community Projects, Activities And Contributions

Table 6: Financial Management And Accountability

Table 7: Itinerary and Evaluation Activities

Table 8: People Interviewed

Table 9: Perceptions of Long-telItl Objectives ofDHP

Table 10: Perceptions oflmrnediate Objectives

Table II: Perceptions of DHP Outputs

Table 12: Perception ofDHP Activities and Linkages in Implantation

15

18

20

22

24

30

33

36

39

DHP:

DSRC:

EPOS:

IGAD:

IGADD:

MISR:

NGO:

NSC:

OSSREA:

PDANS:

PINEP:

PoO:

REDPA:

SAREC:

SlDA:

TOR:

ACRONYMS AND ABBREVIATIONS

Dryland Husbandry Project

Development Studies and Research Centre (University of

Khartoum)

Environmental Policy and Society (Institute of Tema Research,

Linkoping University)

Inter-Governmental Authority on Development

Inter-Governmental Authority on Drought and Development

Makerere Institute of Social Research

Non-Governmental Organisation

National Steering Committee

Organisation for Social Science Research in Eastern and

Southern Africa

Pastoral Development AgentIV eterinary Scout

Pastoral Information Network Programme

Plan of Operation

Regional Dryland Programme in Eastern Africa

Department for Research Cooperation .

Swedish International Development Authority

Terms of Reference

FOREWORD

The Dryland Husbandry Project (DHP) attempts to work with pastoralists and agro­pastoralists in an endeavour to change the quality of life in the drylands. It has been in operation at the field level since 1996 though the project was launched in 1995. DHP is in operation in the Inter-Governmental Authority on Development (IGAD) countries of Eritrea, Ethiopia, Kenya, Sudan and Uganda. It is co-ordinated by a Regional DHP Office at OSSREA in Addis Abeba, Ethiopia with a full financial support by SidaiSAREC

DHP believes in.a bottom-up approach. [t implements action-oriented participatory activities, conducts field research, employs extension methodology and encourages continuous dialogue between pastoralists and planners and policy makers to WOl k for a sustainable development. DHP is also into trials research, paravet training, raises gender issues in qevelopment, ethnoveterinary practices and conducts regional workshops and seminars. It also underlines a dialogue between local knowledge and formal knowledge and the use of local knowledge and local technologies for assuring creativity and sustainability in food security and in technology development.

DHP is into capacity building that will help pastoralists and agro-pastoralists attain some potential to better manage their environment. It puts the priorities of what people think are most pressing and most useful at the community level. Some of the specific activities include the introduction of improved grass species for cattle feed, enclosures, researching into better water harvesting and management and training members of the pastoralists and agro-pastoralists as paravets.

Although women have been involved in DHP development activities at various levels, existing cultural and institutional barriers in some countries did not encourage gender issues to be discussed at the community level.

Since DHP was launched, it did not have a manual to follow up its development activities. That is why this Evaluation Manual has become necessary. What is presented here is an attempt to provide some measurement standards that can help DHP country projects to trace and (0 monitor their achievements and to look at the constraints that surround project implementation. The Evaluation Manual can also help to generate comparable data between country projects so that one country can learn from the other. Moreover, it can show the impacts of DHP and how the project can grow and expand within and outside of the country. Furthermore, the E;valuation Manual can help internal evaluators/assessors to see what DHP has achieved at the grassroots level and can serve present and future DHP activities and similar

IX

PREFACE

This is an Evaluation Manual deve loped for the use of the Oryland Husbandry Project (OHP), intended to serve as a guideline for the eva luati on of the current and future OHP programmes. The Manual is to be used by country project managers as well as internal and ex ternal evaluators. It provides a pre­designed format of approaches, methods and too ls for evaluation purposes.

Evaluation is an assessment, as systematic and as objective as possible of an on-going or completed project, programme· or poli cy. The design, implementation and results of the evaluation is to help determine the relevance and fulfilment of the objectives, effectiveness, impact and sustainability of the project. Moreover, an evaluation should provide information that is credible and useful to enable the incorporation of lessons learned into the decision-making process of bo(h the recipient and donors. Project/programme evaluation includes quantitative and qualitative measurements of the costs, services provided, service quality and outputs and achievements. The compari sons based on these measurements and the resulting informatio n are used in policy and management judgement.

In the case of OHP, the main project document provides two types of evaluations to be conducted fo r each country project. One is the ' internal ' evaluation to be led by OSSREA and the implementing agency, focusing on documenting and analysing the perceptions, concerns, and on-go ing re­assessment of the objecti ves by different actors engaged in project implementation. The other is the ' external' evaluation to be led by SIDA/SAREC, the funding agency, to assess the progress of the project and to suggest guidelines for an eventual proposed phases.

The internal and ex terna l evaluations of Phase I of the OHP, were undertaken and identified some limitations. Most important of these was the lack of detai led action plan and indicators that cou ld be used to assess proj ect progress. Furthermore, there was no standardised format used to carry out the evaluations to enable compari sons across projects and countries. Speci fi cally, there were no measures and standards providing evaluation tool s upon which country Oryland Husbandry Projects could be systematically and adeq uately evaluated . In the absence of such guidel ines, one could expect that the subjectivity and biases of evaluators and project managers could be introduced into the evaluation exercise. Moreover, objective compari sons of OHP in performances and progress were not possible in the region.

DHP Evaluation Manual has been developed by a team of three members commissioned by the Organisation for Social Science Research in Eastern and Southern Africa (OSSREA). The purpose of this team of consultants was to develop measurement standards that are to be used as guidelines in the current and future evaluations of national DHP performances and progress. The team was to draw from their experience of the evaluation work of DHP conducted in 1997. The output of the team would be an Evaluation Manual for Dryland Husbandry Project that would , inter alia, comprise the DHP objectives and activities; evaluation approaches, tools and reporting format.

Abdillahi A. Aboud, Delius Asiimwe and

Mustafa Babiker

Addis Abeba

October 2000

ACKNOWELGEMENTS

. Sincere gratitude are extended to Ms. Misrak Kinfemichael, Ad mi nistrati ve Assistant. OSSREA for her excellent contri bution to the overall organi sation of this exercise and her remarkable dedication to the producti on of this manual even beyond offi ce hours. We also recognize the secretarial support provided by Ms. Selamawit Getachew towards the end of the publi shing work.

Gratitude also to all OSSREA staff who ki ndly availed ti me from the ir usual busy schedules to respond to the demands of the team of consultants.

Finally, our appreciation to Dr. Tegegne Teka, the Regional DHP Co­ordinator of entrusting us wi th thi s important assignment as we ll as the support and encouragement without which thi s task wou ld have not been accomplished as was planned.

Abdillahi A. Aboud, Delius Asiimwc and

Mustafa Babikcr

DHP Evaluation Manual • I

PART I: INTRODUCTION

1. The Dryland Husbandry Project

The Origills. The Inter-Governmental Authority on Drought and Development (IGADD) was establi shed in 1986 by OJ ibouti , Ethiopia, Kenya, Somalia, Sudan and Uganda as a platform for regional cooperation concerning environment and development. In January 1992, IGADD formerly. requested support for a regional range management programme to implement a consolidated five-year plan, focusing on food security and environmental protection. An identification mission launched by the Swedish International Development Agency (SlDA) in February 1992 explored the joint interests and basic concepts for cooperation related to dryland husbandry within the framework of IGAD five-year plan. This was followed by SlDA supported project preparation based on the results of the exploratory mission. Uppsala University research programme on Environmental Policy and Society (EPOS) assisted IGADD and selected participating institutions in the member countries in the preparation of a regional Dryland Husbandry Programme to address environmental and associated socio-economic problems in the region.

The Project. The Dryland Husbandry Project is a regional initiative designed with a view to bringing together a variety of stakeholders in pastoral development to identi fy and develop strategies for addressing the crisis of African drylands in the 1990s and beyond. While the regional DHP is based within the Organization for Social Science Research in Eastern and Southern Africa (OSSREA), Addis Ababa, Ethiopia and nfltional proj ects were set up and operated in Eritrea, Uganda, Kenya, Sudan, and Ethiopia. A network approach is applied to rai se issues of mutual concern to researchers and practitioners with special emphasis on sustainable service provision and pastoral resource management.

The first phase of the DHP which covered the period October 1995-1998 followed a decentralized approach in the five countries in the region. It is useful to note that there are many similarities and a number of differences between the country projects, as reflected in their objectives, management, implementation and level of funding. One such similarity in each participating country is the levels of administration of the National DHP. First, is the field

DHP Evaluation Manual L

station level which consists of researcl!ers and extensionists, supported by cooperating agencies at the community level. The second is the National Steering Committee (NSC) level consisting of the national lead institution and members of the collaborating institutions and agencies. The day-to-day administration of the project activities and financial management is the responsibility of the Project Manager through the supervision of the national coordinator in collaboration with field staff and committees Irom pastora l communities.

The second phase of the project, covering the period \999-2000 is a continuation of the first one, designed to strengthen 'the achievements of the first phase as well as to incorporate the lessons learned in order to overcome the project weaknesses.

2. The Project Objectives, Outputs and Activities

DHP has the following long-term, immediate objectives, outputs and activities:

Long - term objectives

• To improve resource management, especially water and livestock by developing new techniques together with related- institution-building at the community level;

• To increase livestock productivity through improved di sease control and range conditions managed by pastoralists and local institutions;

• To improve the standards of living of pastoralists by creating an enabling environment in which they and their herds may better survive recurrent droughts;

• To increase the competence and capacity of IGAD member states to implement and coordinate environmentally related development policies and viable service provision for dryland management;

DHP Evaluation Manual 3 •

• To promote inter and intra-regional cooperation among research institutions, policy makers, and local implementing institutions (governmental and non-governmental) in dryland management;

• To contribute to an action-research process whereby research and extension insti tutions reconsider their relations with (a) each other (b) pastoral communiti es, and (c) other potential cooperating private, public, formal and informal sector insti tutions;

• To develop capacity of participating institutions to consider the impact of their actions on gendered resource use;

• To establish a conceptual and structural framework fo r two-way exchange of technological innovation among pastorali sts and participating institutions.

Immediate objectives

• To further strengthen the already establi shed structure for training and subsequent fo llow-up support to Pastoral Development AgentsNeteri nary Scouts/ (PDANS) fro m among the local pastorali st communities;

• To develop a methodology for participatory research tri als emphasising water management, and animal disease contro l invo lving pastoralists, researchers, extensionists, GOs and local authorities;

• To develop dialogue, based on parti cipatory action research among local actors in range management (pastorali sts, researchers, extensionists, NGOs arid local authoriti es) concerning how to sustain a viab le structure of services and research activities which address the locally perceived needs;

• To develop dia logue among the local actors in range management concerning gender specific needs, such as different demands on water resources for animal and human consumption, different management methods fo r small , large and lactating li vestock; thereby leading to the establ ishment of strategies which meet diverse needs;

,

DHP Evaluation Manual 4

• To continue to work with participating institutions and other organisations with previous experi ence in developing similar technologies and in community-based service provision;

• To collect data concerning local methods of li vestock disease contro l, natural resource tenure and water management, and to initiate a process whereby participating institutions plan how to integrate this information into their on going work.

Outputs

• Conduct trials in co-operation with local pastorali sts using a variety of technologies fo r increased rangeland productiv ity through improved water management and simple veterinary treatments;

• Establish locally managed viab le, support structures including supply of inputs such as veterinary drugs and tree seedlings (through publ ic or private sector sources) and regular contacts with research and extension;

•

•

•

•

Establi sh a working structure of regular exchange of experience among a broad variety of research and implementation insti tutions engaged in dryland husbandry projects in the lOAD region;

Address a gendered resource use related to di verse management needs of water (for household/animal consumption and range improvement) and livestock (i ncluding small stock and lactating an imals);

Apply action-oriented research methodology (emphasising dialogue among di fferent actors, and analysis and utili sation of local approaches to natural resource tenure, botany and veterinary care) ;

Establish a structure for training and subsequent fo llow-up support to Pastoral Development Agents/Veterinary Scouts from among the loca l pastoral communities;

DH P Evaluation Manual

• Address policy dialogue issues through sensitisation, lobbying, workshops and seminars.

Activities

.:. Training and manpower development. The types of training provided are on a regional level for the staff of participating national institution, and at the research sites for pastoralists and staff of locally participating institutions;

• • .:. Participatory research trials. Each country programme will continue to

work in developing a methodology for participatory research trials, in cooperation with pastoral communities and other local institutions. Trials focus on water management techniques, and low-cost veterinary care utilizing locally available resources;

.:. Action research on local knowledge. The project emphasises working with local actors with a view to bridging the gap between research and implementation by creating 'contact points for a meaningful dialogue between different spheres of knowledge concerning range management. This dialogue will be developed through the process of implementation of research trial s and through a series of regular field meetings;

.:. Gender-based acti vities. In most pastoral development projects women's participation have consistently been relegated to a secondary position; if considered at all. The social and structural reasons behind thi s are believed to be complex and can perhaps be addressed in a

, long-telln perspective. OI-lP will therefore emphasise a process of highlighting gaps in the research and extension agenda related to gendered resource use. This will be carried out by training existing female village health workers as pastoral development agents/veterinary scouts research and extension on small stock, location and design of water resources and their effect on gendered resource demands;

• :. Networking Information

of institutions, NGOs researchers and policy makers . exchange through study tours and creation of