M & E: WHAT, WHY AND HOW?S. Venkatraman

UIS-AIMS, UNESCO Bangkok

WHAT IS M&E?

Monitoring and evaluation (M&E) of development activities provides government officials, development managers, and civil society with better means for learning from past experience, improving service delivery, planning and allocating resources, and demonstrating results as part of accountability to key stakeholders

WHAT IS M&E?

It provides the information needed to answer areas of concern (needs and gaps) about the system

used to determine, using reliable and valid evidence, the extent to which goals/targets have been achieved

Must be demand-driven and based on information need by country

Both Need-based and evidence-based Necessary process that helps decision

makers/managers to manage, plan and make sound policy decisions

THE MONITORING

The monitoring process looks at what is being done and how it is being done. It could involve continuous tracking of activities, review (measure) of the flow of services and activities provided by the program, compliance with laws, regulations, guidelines, etc.

Much of the information desirable to monitor a program is also valuable for the evaluation component.

THE EVALUATION

Evaluation looks at performance against goals. This can and should take place while the program or research project is underway, and is concerned with evaluating how the intervention is meeting its performance goals.

It overlaps with monitoring to the extent that it uses the data gathered during monitoring that is specifically performance related.

TOP DOWN & COUNTRY SPECIFIC Top-down approach to M&E results in narrow

perception of process and lack of commitment at country level, sub-national level resulting in questions about sustainability

Country driven approach in M&E indicates commitment towards improving delivery of services and ownership of process. It is more based on the real need (contextual) & demand (problems & gaps)

Having M&E or data or tools alone does not guarantee progress

M&E: THE REAL VALUE

M&E true benefit comes when information is used at all levels & all stages resulting in progress

M & E is used for measuring the quantity, quality. It looks at input, process and outputs, and the outcomes.

Impact of the programme can then be assessed

Information collected through M&E process -> to identify and understand ‘why’, ‘where’, ‘what’-> need and gaps of performance, assess progress and shortfalls

M&E could be carried out at project, program or sector level,

Also at national, sub-national or local level

ACTIVITIES- OUTCOMES- IMPACT

IMPLEMENTING M&E

Ownership of M&E is important in ensuring sustainability and progress rather than simply satisfying external need/requirement or physical deliverables

Need a efficient system in place to ensure collection of relevant and appropriate information and appropriate skills built at all levels so that such information is used effectively

-(eg) ‘down the drain’!

DOWN THE DRAIN?

WELL CONNECTED?

Without sound M & E system in place -> difficult to manage and measure progress & achieve goals

M & E system does not guarantee improvement but necessary basis to improve

INSTITUTIONALIZING M & E M & E not only to monitor progress for donor

funded projects, instead should be part of regular process of trying to improve overall performance-> need to internalize

Need to ensure that M & E system in place is sustainable, and information collected is relevant, timely* (SL), used in all aspects & all levels of operation; policy formulation/ revision, planning, for better transparency and accountability, and management including resource allocation

Effectiveness of Institutionalization of M & E depends on how countries internalize-> internal capacity to collect & use information

Use of information depends on -> analysis and evaluation of information to identify key issues relating to progress & performance (evaluation) and determining further info need (continuous process)

Current practice of M&E on techno-centric approach and on information collection -> little effort on identifying “what is the emerging, trend, the gaps & messages (stats: stories)? it conveys

INDENTIFYING RELEVANT INDICATORS

Selecting appropriate Indicators. Underpinning all M&E activities is the need to determine what it is that is being measured, and for what purpose.

The identification, selection and prioritization (in some cases) of both quantitative and qualitative indicators are therefore critical before M&E can be undertaken

The need for specific indicators may also evolve and change over time, as circumstances change.

INDICATORS:

Indicators are measures of inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies.

When supported with sound data collection—analysis and reporting, indicators enable managers to track progress, demonstrate results, and take corrective action to improve service delivery

WHAT CAN WE USE INDICATORS FOR?

Setting performance targets and assessing progress toward achieving them

Identifying problems via an early warning system (potential problems)to allow corrective action to be taken.

Indicating whether an in-depth evaluation or review is needed

ADVANTAGE

Effective means to measure progress toward objectives.

Facilitates benchmarking comparisons between different organizational units, divisions/province, and over time.

Poorly defined indicators are not good measures of success.

DISADVANTAGE

Tendency to define too many indicators, or those without accessible data sources or non-availability of data, making system costly, impractical, (often ‘shopping list’) and likely to be underutilized.

Often a trade-off between picking the optimal or desired indicators and having to accept the indicators which can be measured using existing data

COUNTRY DRIVEN M&E

Clear Goals & targets with well-defined indication of what is/are to be achieved

Indicators identified and targets set for each goal with proper tools & methodologies for collecting , processing and analysis of information

Establish system to monitor (regularly) and evaluate progress made in each goal and reported

Key stakeholders involved in addressing each goal (progress and planning)

Identification, role clarifies to each stakeholder for collecting relevant information in relation to each indicator (eg: Early childhood)

WHY EVALUATE PERFORMANCE? Helps implementation, rethink strategy and

re-designing programs, review existing tools (eg. expensive data collections) and resource allocations to improve program/sector’s chances of achieving target

Encourages comparison among programs, coordination (eg. Literacy-> Census, NFE, MOE) and supports efficient and effective use of resources

Promotes transparency and accountability in implementation and use of resources

Encourages good practice in results-based management

IMPACT EVALUATION: WHAT IS IT?

Impact evaluation is the systematic identification of the effects – positive or negative, intended or not – on individual households,

institutions, and the environment caused by a given development activity

Impact evaluation helps us better understand the extent to which activities reach the poor and magnitude of their effects on people’s welfare.

They can range from large-scale surveys, to small-scale rapid assessment and participatory appraisals

MAINSTREAMING M&E

Realizing and internalizing the need for information at all levels of decision making

Making the commitment to set up an effective sustainable system for M&E

Ensuring sustainability of M & E system through capacity building, allocating necessary resources

Move towards results-based management

NEVER HAVE THIS SITUATION!

THANK YOU! VINAKA!