Evaluating a Humanitarian Action Project

Why Evaluation

• To assess the quality of the project• To see if it fits into the mandate of the

organization• To see if the needs and aspirations of the

community is captured properly• Value for money• Make decisions: select, fine tune, reject

Key Indicators Used for Evaluation

• Relevance• Effectiveness• Efficiency• Capacity and Support• Sustainability• Imp (whether all the documents required are in

place)

The Project

• A Post Emergency Disaster Risk Reduction Project in Northern Sri Lanka

• Submitted to European Commission Humanitarian Aid and Civil Protection

• Duration: 12 Months• Proposed Budget: Euros 128.806,00• Financing Mechanism: 100% from ECHO

Relevance

• Appropriateness of the specific objective, pertinence of the action in regards to the real needs of the population

• Appropriateness of the action to the physical and political environment

• Link of the principal objective of the action with DG ECHO strategy

Project Context

• The project seems to be in line with local needs and aspirations of the community therefore it is highly relevant

• Installation of the early warning system could reduce the impact of disaster

• Weather conditions can lead to heavy rains, thunderstorms and floods

• N E Sri Lanka is also vulnerable to cyclone and tsunamis.• The impact of civil war has further enhanced the

vulnerability of the community

Efficiency

• Measuring outputs in relation to the inputs• Alternative approaches to achieving the same

output• How economically resources were utilized:

Human Resources, finance and time

Cost Efficiency

• Cost broken down by sector is rationally allocated• Cost for transportation: further information

needed on what kind of a vehicle will be used for transportation

• Time allocated for transportation seems a bit unrealistic: duration of the project 12 months – time for planning

• HR Cost: Expatriate: 33% of the budget• Program cost V/s Program support cost = 60:40

Time Efficiency: realistic management schedule

• Three phases: establishment, capacity building & implementation

• Difference between M&E

Human Resources Efficiency

• Should reflect staff responsibility, support, development and well being

• The proposal does not state anything about the criteria and selection for team recruitment

Effectiveness

• Are the objectives effective compared to the problem statement?

• Do the results correspond with the aimed objectives?

Output, outcome and impact

Output

• Objectives are coherent with objectives of ICDM and their former projects

• No clear percentages in objectives: difficult to assess– F.e.: “participating communities are aware of their

risk situation” and “participation of women is ensured” what percentage of the participating communities and what percentage of the women?

• Objectives should be more specific

Outcome and Impact

• In general: the stated results (outcome) match the stated objectives

• However, one of the results implicates a raised awareness of Disaster Risk Reduction:– The area is already prone to natural disasters

are there not already existing coping mechanisms in use?

– Not investigated by ICDM could negatively effect the impact of the project

Capacity and Support

• Whether the organization has the internal and external means to successfully implement the proposal

• Whether the organization has the ability to critically assess its organizational capacity

• Whether it has the strategies and resources in place to improve the organizational capacity

Internal Organizational Capacity

• ICDM is well equipped and suitable for the project

• Coordination and capacity of HR is not clear• It should consider adding training schemes for

its team

Implementing Partner Capacity

• Seva Lanka fits the mandate of the project• No point of contact has been mentioned • No information about the HR of the partner

organization

Evaluation Methods

• Method of evaluation proposed seems appropriate

• As per ECHO guidelines the focal person for evaluation needs to be mentioned

• No information on sharing the report with HQ.• Sharing of the evaluative feedback with

ALNAP, UN and other international knowledge based organizations

Sustainability as Reflected

• Involvement of community and other stakeholders in the project cycle

• Co-operation and coordination between different stakeholders

• Designing of the institutional mechanism• Availability of the local skills and knowledge to

maintain the systems

Involvement of the Community

• Enhances ownership• Management of the project after phase out• The Transfer of ownership should be a gradual

process• Whether built into Project Cycle

Organization Perception

• Presence of the organization• Its image• Interface between the organization and the

community• Program Mode V/s Project Mode

Coordination between Stakeholders

• Linkages and leverages• Mobilizing resources• Sharing expertise• Comprehensive project• Community, Community Based Organizations,

Government, Civil Society, UN Clusters and Donors

Availability of Local Skills

• Crucial when some of the components are imported from outside

• Enhancing the skills of the community to manage innovation

Proposal: Organizations Presence

• Pg 5 and Pg 13?• Present since 2004 and long term relationship

with the community• Till date implemented six projects• Local partnership with Seva Lanka

Proposal: Involvement of the community

• Pg 10, 19 and 35?• Designing: through community consultation

involving the most vulnerable• Focuses on community contribution• Participatory M&E

Proposal: Coordination Between Stakeholders

• Pg 39• Formalization of the relationships with key

stakeholders• Robust mechanism for involving local level

stakeholders• Silent on coordination at national level:

Disaster Management Bureau, Un Clusters, agencies providing relief

Proposal: Local skills

• Pg 31, 34, 35, 36• Gives due emphasis on creating village level

volunteer for S&R and FA• EWS (imported): capacity building is reflected• Task Force: One time capacity building during

the program. After that? Mechanism for constant capacity enhancement

Recommendations

• More robust log frame based on SMART• Ratio between program and operational cost• Efforts should be made to develop a better linkage with national

level institutions• M&E framework• Mechanisms are not yet in place to ensure sustainability of the

capacity of the community.• Mechanism to ensure engagement of the community after the

closure of the project. • Fine-tune capacity building strategies• Knowledge sharing mechanism• Conflict and DRR

Thank You

The teamKiona BoltSilvi HurkmansYaoliang PengMaike BenemmaShakeb Nabi