WELCOME

Project Monitoring and Evaluation : Dr. Bijaya Bhusan Nanda, bijayabnanda@yahoo.com

What is a project?

a starting and an ending point

clear objectives

a plan of the work to be done

a budget specific performance requirements that

must be met

INPUTS- PROCESS RESULTS

OUTPUTS-

PRODUCTS/

SERVICES

Outcome

Impact

SYSTEM

Impact

Outcome

Reduced mortality rates for children under 5 years old

Improved use of ORT for managing childhood diarrhea

Results

Outputs

Inputs

15 media campaigns completed· 100 health professionals

trained· Increased maternal knowledge

of ORT services· Increased access to ORT

· Launch media campaign to educate mother

· Train health professionals in ORT

· Trainers· ORT supplies· Funds· Participants

Imple-mentation

Activities

Project to Reduce Child Mortality Rate in A Particular Area

What is Monitoring? A continuous management function. Provides programme/projects managers and

key stakeholders with regular feedback and early indications of progress or lack thereof in the achievement of intended results.

Tracks the actual performance against planned or expected.

Involves collecting and analysing data on programme/project processes and results.

And recommends corrective measures. Generally internal.

Monitoring

Regular feed back Early

indication of

Progress or

Lack thereof

Tracks P

erform

ance

Collecting & Analysing

data

LINK

ACTIVIT

IES &

RESOURCES

TO

OBJE

CTIVES

RE

CO

MM

EN

DI

NG

Cor

rect

ions

Purpose & Goals for the Monitoring Tool

~Collect information regarding project quality~Support the stake holders in project

implementation.~Lead the beneficiaries to possible solutions of

the problems detected in the implementation.~ Be familiar with the real resources.~ Assess the progress of the projects.~ Disseminate examples of good practices.~ Establish a close relation between the Stake

holders and the organisation

Physical activity – Quality, timeliness

Financial – Cost of component, target

Performance – Overall performance of system

What to Monitor

Inputs: Money, Material, human resources Activities: A set of tasks performed Process: A set of activities, towards a common

purpose Output Outcomes: Series of effects of some action and

activities (e.g. Crop diversity, reduced alkalinity, Increased Water table)

Impact: Changes in peoples lives and livelihoods (e.g. Increased income, reduced mortality, Increased capacity to bargain)

What to Monitor

Principles of developing Monitoring System Demand driven

Simple and easy Participatory Not for policing or punishment rather for learning Mix of qualitative and quantitative data Aggregation of data 2 way feedback Meaningful use of analysed information at source

12

MONITORING DICHOTOMY

Progress monitoring orInput-output monitoring or

Target-achievement monitoring

Process monitoring

Emphasis on physical

achievements vis-à-vis targets

Emphasis on the way the target has

been achieved

A complete monitoring system will have both progress and process monitoring

Mechanism

Reports: Regular , specific Visit/observations/ transects, both at field and

office Meetings/reviews – Regular, specific Studies – Impact studies, other special studies Special tools – for measuring specific

processes or outputs.

Levels of monitoring are to be decided based on the project management level

Monitoring ToolManagement Information System (MIS) A system for

Collection, Organisation, Maintenance, Analysis, Interpretation of data For assessment, Better decision, Corrective measures,

for better performance at various level

Criteria of a good (MIS)

Regularity, Continuity,

Timeliness,Relevant Indicators,

Forward & Backward feedback,Automated

Hence a computerized MIS is a must Records & registers MPRs, QPRs, APRs

TRIPLE A PROCESS

AnalysisAction

Assessment

TRIPLE A

Assessment: Identify the “what” - Analysis: Identify the “why” - Talking and

understanding the causes Action: Identify the “how”-

praise for good practice, agree on actions to improve the situation

Broken Triple A

A supervisor during her field visit finds that in an Anganwadi Centre the number of children in grade II, III and IV has increased. She advises the AWW to take more care of these children. However in her next visit she finds that the situation has not improved.

What is missing?

Broken Triple A A supervisor during her field visit finds that

in an Anganwadi Centre the number of children in grade II, III and IV has increased. She discusses the causes with the AWW and is informed that a number of children are suffering from diarrhoea. She checks whether the AWW has stock of ORS and advises her to give ORS to the affected families.

What is missing?

What is Evaluation?A time-bound exercise.Assess systematically and objectively the relevance, performance and success, or the lack thereof, of ongoing and completed programmes.Evaluation is undertaken selectively to answer specific questions to guide decision-makers and/or programme managers, and to provide information on whether underlying theories and assumptions used in programme development were valid, what worked and what did not work and why. Evaluation commonly aims to determine the relevance, validity of design, efficiency, effectiveness, impact and sustainability of a programme.

Evaluation- concerned

with

RelevanceProgramme

continues to meetneeds

Sustainability

After

withdraw

al

An

ticip

ate

d/

Un

an

ticip

ate

d re

su

ltsCausality

Factors Validity of

DesignEffe

ctiv

enes

s

Achie

vem

ent

ofre

sults

Eff

icie

ncy

Res

ults

vs.

co

sts

Altern

ativ

e

stra

tegie

s

to th

e

prob

lem

Source: ILO, 1997.

Why evaluate?• To inform decisions on operations, policy, or strategy

related to ongoing or future programme interventions; • To demonstrate accountability to decision-makers • To enable learning and contribute to the body of

knowledge on what works and what does not work and why;

• To verify/improve programme quality and management;• To identify successful strategies for extension/

expansion/ replication;• To modify unsuccessful strategies;• To measure effects/benefits of programme and project

interventions;• To give stakeholders the opportunity to have a say in

programme output and quality;• To justify/validate programmes to donors, partners and

other constituencies.

Three Common Evaluation Purposes

To improve the design and performance of an ongoing programme – A formative evaluation.

To make an overall judgment about the effectiveness of a completed programme, often to ensure accountability – A summative evaluation.

To generate knowledge about good practices.

Monitoring Continuous Keep track; oversight,

analyses and documents progress

Focuses on inputs, outputs, process, continued relevance, likely results at purpose l

Translate Objectives to performance indicators

Collect data on Indicators routinely

Report progress to stake holders & Alert them to problems and provides options for corrective actions

Internal Self assessment

Evaluation Periodic: Mid-term, End

term, Appraisal In-depth analysis compare

planned with actual achievement

Focuses on out-puts in relation to inputs, results in relation to cost, processes, overall relevance, impact and sustainability

Answers why and how results were achieved.

Contributes to building theories and model

Provide managers with strategy and policy options

Internal and or External

Monitoring vrs. Evaluation

Evaluation

Evaluative Activities: Activities such as situational analysis, baseline surveys, applied research and diagnostic studies. Evaluation Questions: A set of questions developed by the evaluator, sponsor, and/or other stakeholders, which define the issues the evaluation will investigate and are stated in such terms that they can be answered in a way useful to stakeholders.

Evaluation

Evaluation Standards: A set of criteria against which the completeness and quality of evaluation work can be assessed. The standards measure the utility, feasibility, propriety and accuracy of the evaluation. Evaluation standards must be established in consultation with stakeholders prior to the evaluation. Ex-ante Evaluation: An evaluation that is performed before implementation of a development intervention. Related term: appraisal.

EvaluationEx-post Evaluation: A type of summative evaluation of an intervention usually conducted after it has been completed. Its purpose is to understand the factors of success or failure, to assess the outcome, impact and sustainability of results, and to draw conclusions that may inform similar interventions in the future. External Evaluation: An evaluation conducted by individuals or entities free of control by those responsible for the design and implementation of the development intervention to be evaluated (synonym: independent evaluation).

Participatory Monitoring & Evaluation

Strengthening self development initiatives Public accountability of programmes to

communities Encouraging institutional reform towards

participatory structures Organization building and learning Capturing social dynamics

29

M&E Stakeholders

Fundingagency

Implementingagency

Beneficiaries NGOs

Need & InterestOf all should be taken care of

30

 

is a different approach which involves –

   Local People,

Development Agencies, and

Policy Makers

 deciding together how progress should be measured, and results acted upon.

It can reveal valuable lessons and improve accountability.

  However, it is a challenging process for all concerned since it encourages people to examine their assumptions about what constitutes progress, and to face up to the contradictions and conflicts that can emerge.

Participatory monitoring and evaluation

31

At the heart of P M & E, however, are four broad principles:

'Participation' –

‘Negotiation' to reach agreement about what will be monitored or evaluated, how and when data will be collected and analysed, what the data actually means, and how findings will be shared, and action taken.

This leads to 'learning' which becomes the basis for subsequent improvement and corrective action.

Since the number, role, and skills of stakeholders, the external environment, and other factors change over time, 'flexibility' is essential.

32

Steps involved in PM&E

(Clockwise)

Identify who should and want

to be involved

Clarify participants Expectations of the Process,and in what

Way each personOr group wants to

contribute

Define the prioritiesFor M&E

Identify indicatorsThat will provide The information

needed

Agree on the methods,Responsibilities and

Timings of informationcollections

Collect the information

Analyse the information

Agree on how The findings areTo be used and

By whom

Clarify if the PM&E Process needs to be

Sustained and If so how

33

Key Characteristics of Participatory M&E- Draws on local resources and capacities

Recognizes the innate wisdom and knowledge of the end-users

Demonstrates that end-users are creative and knowledge about their environment

Ensures that stakeholders are part of the decission-making process

Uses facilitators who act as catalysts and who assist stakeholders in asking key questions

34

Benefits of participatory M&E

All Stakeholders owns the M&E Process & Results.

Correction, redesigning of the policy, plan of action, budgeting become easy.

Better decision making by insiders

Insiders develop evaluation skills

Outsiders have better understanding of insidersInsider to insider communication is strengthenedInformation is useful for ongoing management of projectEntry point for the participatory approach

35

STEPS FOR EVALUATION

FIRST Step: Review objectives and activities Second step :Review reasons for evaluationThird step: Develop evaluation questionsFourth Step: Decide who will do the evaluation.

Fifth step: Identify direct and indirect indicators.

Sixth step :Identify the information sources for evaluation questions

36

Seventh Step: Determine the skills and labour that are required to obtain information

Eighth step: Determine when information gathering and analysis can be done.

Ninth step : Determine who will gather information.

Tenth step: Analyze, present and use results.