ENTITIES FOR A UN SYSTEM EVALUATION

FRAMEWORK

17th MEETING OF SENIOR FELLOWSHIP OFFICERS

OF THE UNITED NATIONS SYSTEM AND HOST COUNTRY

AGENCIES

BY DAVIDE TONINI

IMO, LONDON, 17-19 NOVEMBER 2008

JOINT INSPECTION UNIT (JIU)

“The Inspectors shall provide an independent view through Inspection and evaluation aimed at improving management and methods at achieving greater coordination between organizations.”

“Without prejudice to the principle that external evaluation remains the responsibility of appropriate intergovernmental bodies, the Unit, with due regard to other responsibilities, may assist them in carrying out their responsibilities for external evaluation of programmes and activities. On its own initiative or at the request of the executive heads, the Unit may also advise organizations on their methods for internal evaluation, periodically assess these methods, and make ad hoc evaluations of programmes and activities.”

UNITED NATIONS EVALUATION GROUP (UNEG)

UNEG, as one of the main interagency bodies providing strategic guidance in evaluation, plays a very important role in strengthening the objectivity, effectiveness, and visibility of the evaluation function across the UN system by bringing together the professional units and individuals responsible for evaluation within the UN system..

UNEG is comprised of 43 member organizations and chaired by UNDP, aiming to provide a forum for the discussion of evaluation issues within the UN system and to promote simplification and harmonization of evaluation reporting practices among UNDP and the executing agencies.

JIU STANDARDS & GUIDELINES

THE GOAL OF EVALUATION

• Formative/Process Evaluation: Appraising processes of ongoing programme in order to propose modifications and/or improvements

• Impact/Outcome Evaluation: Finding out how well programme works in order to provide info for continuation/expansion/reduction decisions

CONCLUSION OF DESIGN PHASE

• Identify existing criteria related to expected outputs

• Identify relevant executive or legislative actions• Understand interest/concern of Member States• Clarify purpose of evaluation• Understand programme objectives & legislative

history• Identify applicable rules & regulations for

implementation of mandate• Identify relevant and related internal controls

STRUCTURED DESIGN PLAN FOR EVALUATION

• Description & justification of sampling methodology details (if sampling used)

• Specification & description of measurement methods and instruments and estimate of their reliability & validity

• Justification that appropriate procedures & instruments have been specified

UNEG STANDARDS FOR EVALUATION

IN THE UN SYSTEM

INSTITUTIONAL FRAMEWORK

UN organizations should:

• have an adequate institutional framework• develop an evaluation policy and regularly

update it• ensure submission of evaluation plans for

approval• ensure evaluation follow-up mechanisms

MANAGEMENT OF THE EVALUATION FUNCTION

Head of Evaluation:

• has a lead role in ensuring that the evaluation function is fully operational and that evaluation work is conducted according to the highest professional standards

• is responsible for ensuring the preparation of evaluation guidelines

• should ensure that the evaluation function is dynamic, adapting to new developments and changing needs both within and outside the organization

COMPETENCIES

Persons engaged in designing, conducting and managing evaluation activities:

• Should possess core evaluation competencies

Evaluators should have:

• Relevant educational background, qualification and training in evaluation

• Relevant professional work experience• Specific technical knowledge of

methodology/approach applied

ETHICS

Evaluators should:

• be sensitive to beliefs, manners and customs and act with integrity and honesty in their relationships with all stakeholders

• ensure that their contacts are characterized by respect

• protect anonymity and confidentiality of informants

• be responsible for their performance and products

CONDUCTING EVALUATIONSDESIGN (1)

The evaluation design should:

• ensure relevant, timely, valid and reliable information

• include TORs that provide the purpose and description of process & product

• include clearly stated purpose and context• ensure clear description of subject• ensure realistic and achievable objectives &

clearly defined scope

DESIGN (2)

The evaluation design should:

• clearly specify the evaluation criteria • ensure methodologies are sufficiently rigorous

and ascertain a complete, fair and unbiased assessment

• assess cost effectiveness• include considerations regarding the extent of

UN system’s commitment to the human rights-based approach incorporated in the design

PROCESS

• Relationship between evaluator and commissioner(s) must be characterized by mutual respect and trust

• Stakeholders should be consulted in the planning, design, conduct and follow-up

• A peer review/reference group composed of external experts may be useful

SELECTION OF TEAM

Evaluations should be:

• conducted by well-qualified evaluation teams• composed of gender balanced and

geographically diverse teams

IMPLEMENTATION

• conducted in a professional and ethical manner

REPORTING

The final evaluation report should:

• contain evidence-based findings• contain conclusions• contain lessons and recommendations• be free of irrelevant information• be presented in an accessible and comprehensible

manner

FOLLOW-UP

Evaluation requires an explicit response by the governing authorities and management addressed by its recommendations

EVALUATION REPORTS

• Key basic info in title and opening pages• Executive Summary• Clear description of subject evaluated• Role & contributions of UN & stakeholders• Purpose & context• Explanation of evaluation criteria• Explanation of evaluation objectives/scope• Indication of extent to which gender issues & human

rights considerations were incorporated• Transparent description of applied methodology• Complete description of stakeholders’

participation/contributions

EVALUATION REPORTS (2)

• Discussion of extent to which design included ethical safeguards

• Inputs, outputs, outcomes/impacts should be measured

• Reasons for accomplishments and difficulties of subject evaluated

• Conclusions substantiated by findings consistent with data & methodology

• Recommendations based on evidence & analysis, relevant & realistic, with priorities for action made clear

• Lessons should be generalized beyond immediate subject

• Complete and relevant annexes

Lunchtime!