Decentralized evaluation for evidence-based decision makingWFP Office of Evaluation

Decentralized Evaluation Quality Assurance System (DEQAS)

Quality Checklist for Decentralized Evaluation ReportVersion June 2018

[title of the decentralized evaluation]Overall

General Comments/StatusLength: Report is concise, not exceeding 40 pages excluding Annexes and Executive SummaryAccessibility Written in a clear and accessible manner The structure of the findings allow readers to understand

evaluation findings Key messages are summarised and highlighted The report presents a good balance between descriptive and

analytical information Terminology is used correctly The report is free from grammar, spelling, or punctuation errors The tone of the report is constructive The language used in the report is simple and clear and the

report is reader friendly

Credibility The report presents successes and failures, positive and

negative sides in a balanced way. The report raises critical issues when necessary

The report’s findings and conclusions are fully evidence-based.Editing

WFP format and templates are applied Acronyms are spelt out the first time they are used Paragraphs and pages are numbered electronically Table of contents is included and lists tables, graphs, figures and

annexes Tables and diagrams are used as relevant and are numbered List of acronyms is included When data or quotes are provided, the report indicates the

source (either in a footnote or directly below a table/graph).Cover Page

Uses WFP DEQAS Report Template Title of the decentralized evaluation is identical to that in the

TOR Date and status of the report (draft/final) indicated on the cover

page

Executive SummaryOverall: The Executive Summary should provide a complete and balanced synthesis of the evaluation findings, conclusions and recommendations in not more than 4 pages.

Expected Content Assessment criteria Comments/status Introduction: main features of Executive Summary is

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 2 | 10

the evaluation including evaluation type, period being evaluated and commissioning office

Purpose and objectives, main users/ intended audience, context, subject of the evaluation.

Main features of the methodology

Key findings on all the evaluation questions

Overall summarized conclusionsandrecommendations

coherent, self-explanatory and self-contained

The key findings form the largest part of the section

All recommendations are presented in brief

The Executive Summary could be used as a standalone aid to decision-making

Length does not exceed 4 pages

1. IntroductionOverall: Succinct overview of the evaluation subject, context and features, providing clarity on why and how the evaluation was carried out, in not more than 5 pages.

Expected Content Assessment criteria Comments/status Clear reasons for the evaluation Specific objectives Scope of the evaluation Stakeholders in the evaluation Intended users and how they will

use the evaluation findings

Introduction clearly sets the scene for the evaluation, including its rationale, scope, stakeholders and users.

Information is succinct and focused

1.1 Overview of the Evaluation SubjectFactual information regarding the evaluation subject including:o Type of evaluation (operation,

activity, thematic area, transfer modality, pilot project)

Information is relevant and important to understanding the subject of the evaluation:o What it iso When it was designed

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 3 | 10

o Geographic scope of the evaluation subject

o Relevant dates: Approval date; start date; end date

o Planned outputs at design o Beneficiary numbers (planned and

actual) disaggregated by sex/age/ activity

o Any other planned and actual outputs

o Amount of planned and actual tonnage/cash/vouchers, disaggregated by year and by activity (if multiple activities)

o Planned and actual outcomes Key activities

o Main partners (Government; NGOs; Bilateral; Multilateral)

o Resources (% funded of total requirements) and key donors. If subject funded from pooled funds or is a component/activity of an operation, show resource allocated to the subject of the evaluation

o Assessment of the Logical Framework or similar tool from evaluation perspective

o Other relevant preceding/ concurrent activities/ interventions/operations

o Any amendments to initial designo Include reference to:o Past evaluations/reviews related

to the subjecto Maps/graphs for illustration

o What are the key inputs ($ value)

o What are the planned and actual outputs? (beneficiaries, MT, Cash &Voucher $)

o What is the target/scope?o What are the planned and

actual outcomes? o Who is involved in its

implementation? Highlights relevant issues

from past evaluations and reviews that are relevant to the evaluation

Gender dimensions explained Differences between original

design and implementation are explained if appropriate

1.2 Context

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 4 | 10

Overview of the surrounding context directly relevant to the evaluation including: Poverty, food and nutrition security Government policies and priorities Key data and trends related to SDG

2 / SDG 17 in the context (region, country sub-national/local level)

Humanitarian issues Gender dimensions of the context Key external events Features of international assistance

in the area Other WFP work in the area Work of other key actors

Information is relevant and necessary to understand the implications for the evaluation subject

Relevant indicators and latest information have been identified and trend data used

Gender context of country is described

Information draws from and is consistent with inception report, updated where appropriateInformation is explicitly geared to the evaluation subject, rather than being generically presented.

1.3 Evaluation Methodology and Limitations Methodology used for the

evaluation is summarized Evaluation criteria applied to the

evaluation are listed and justified (and if possible interpreted for the subject)

The main evaluation questions are set out

Summary of the overall methodological approach is provided

Specific methods and sampling used are provided

A statement on how validity and reliability was addressed is included

Limitations of methods or data availability encountered, and mitigation strategies applied/implications for the

Information allows reader to understand how the evaluation was conducted

Methodological approach is comprehensive and presents a systematic approach that generates trust in the credibility of the evaluation

Methodological approach is coherent, logical and in line with the TOR and inception report

Specific methods and sampling applied are relevant to the methodological approach and are feasible/appropriate

Limitations and their effects D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 5 | 10

findings, are explained Gender considerations for the data

collection methods are explained Describe here how the evaluation

has considered and managed ethics, and what safeguards were in place during the evaluation

Full details in a specific Annex

on the evidence base are included

Statement on validity and reliability generates confidence in the evidence base

Methods demonstrate consideration of gender equality and women’s empowerment

Statements on how the evaluation has managed ethics

Full methodology provided in an Annex (updated from Inception Report in relation to its actual implementation).

2. Evaluation FindingsOverall: This section should be the most substantial of the evaluation report. It should present information that supports the conclusion(s) of the evaluation team against the key evaluation questions. It should not exceed 30 pages.

Expected Content Assessment criteria Comments/status Presentation of clear data

analysis against all evaluation questions, including triangulated information

Findings are substantiated by credible evidence that has been checked for accuracy, consistency and reliability

Limitations or gaps in evidence are presented

Indication of where evidence is inconclusive

Information sources are provided and accurately quoted

The findings have been formulated clearly and are based upon evidence collected

Findings are triangulated Findings use credible external

sources of information Findings address the

evaluation criteria chosen Information sources are

provided, and where evidence is inconclusive this is clearly

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 6 | 10

Findings are adequate to substantiate the conclusions and recommendations made by the evaluation team (next section)

Findings use gender (and equity) sensitive language throughout and data is disaggregate.

Findings are presented in a gender and culturally sensitive manner.

stated. Findings address any

limitations or gaps in evidence

Visual aids are used in a clear an accessible way to illustrate the argument

Information sources are quoted

The findings balance detail and synthesis

Findings reflect gender analysis and address gender sensitivity including use of gender disaggregated data Findings are presented by evaluation question

3. Conclusions and RecommendationsOverall: Conclusions and Recommendations are core components of the evaluation report; Lessons learned and Good Practices should be included if appropriate. The section should not exceed 5 pages.

Expected Content Assessment criteria Comments/status3.1 Overall Assessment/Conclusion

Conclusions are logically derived from findings and substantiated by evidence

There is clear referencing back to findings

Report contains a brief and balanced assessment of the evaluation findings related to the selected evaluation criteria and

Conclusions follow logically from the findings, with clear and explicit linkage to them

A conclusive picture is presented, formed from the previous sections of the report

A summary against evaluation criteria is included

Conclusions are succinct, D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 7 | 10

questions. This may be organised by criteria, theme and/or by evaluation question

synthesising common findings

Conclusions include recognition of different stakeholder groups and gender aspects

No new evidence is introduced that has not been presented in the findings

Conclusions are impartial and free of bias

3.2 Lessons Learned and Good Practices (optional) Lessons are presented

that WFP and partners can use in future contexts to improve WFP projects and programmes.

Lessons include any positive and negative lessons

Good practices are presented which represent successful strategies or interventions that have performed well;

Emerging good practices can be presented (i.e. an intervention that has potential to perform well)

Lessons contribute to wider organizational learning

Lessons specifically indicate what should be repeated or avoided in future contexts to guide future action

Good practice is potentially replicable

Good practices describe why things have worked

3.3 Recommendations Short paragraphs

describing up to 10 Recommendations follow

logically from the findings

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 8 | 10

recommendations, which are all relevant to the subject of the evaluation

Recommendations logically derive from the findings and conclusions and are clearly linked to the evidence base

Recommendations are realistic/feasible

Recommendations are prioritised, sequenced and addressed to specific target groups

Gender issues are addressed where appropriate in recommendations

and conclusions They are based on the

analysis and evidence collected

They are realistic (implementable), presenting plausible options for improvement

They are clear and specific and clustered /presented in a logical order

They are prioritised and have a clear timeframe for implementation

The target group for each recommendation is clear and relevant to the intended users of the evaluation

Gender is integrated into the recommendations and where feasible, recommendations for improving WFP’s performance on gender are provided.

Annexes Comment Comments/statusAnnexes should support and expand on text in the main report. Including: Terms of Reference Documents Reviewed Stakeholders

Interviewed Evaluation matrix Data Collection tools Bibliography

Annexes listed and numbered

Annexes referenced, where appropriate in the main report

Data collection tools are included

Not all working documents

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 9 | 10

List of Acronyms are included

D E R e p o r t Q C – V e r s i o n J u n e 2 0 1 8 P a g e 10 | 10