Evaluation at USAID Cynthia Clapp-Wincek Director of USAID Learning, Evaluation & Research 1.
USAID/UGANDA PERFORMANCE AND IMPACT EVALUATION FOR ...
Transcript of USAID/UGANDA PERFORMANCE AND IMPACT EVALUATION FOR ...
QUARTERLY PERFORMANCE REPORT
FY2019, FIRST QUARTER REPORT
OCT - DEC 2018
USAID/UGANDA PERFORMANCE AND
IMPACT EVALUATION FOR LITERACY
ACHIEVEMENT AND RETENTION
ACTIVITY (LARA)
Contract No. AID-617-C-16-00003
This publication was produced for review by the United States Agency for International Development. It was
prepared by NORC at the University of Chicago. The authors’ views expressed in this publication do not
necessarily reflect the views of the United States Agency for International Development or the United States
Government.
AID/UGANDA PERFORMANCE
AND IMPACT EVALUATION FOR
LITERACY ACHIEVEMENT AND
RETENTION ACTIVITY (LARA)
QUARTERLY PERFORMANCE
REPORT
FY2019, FIRST QUARTER REPORT
OCT - DEC 2018
Prepared under Contract No.: AID-617-C-16-00003
January 31, 2019
Submitted to:
USAID/Uganda
Submitted by:
NORC at the University of Chicago
Attention: Ritu Nayyar-Stone
Phone:+1-301-634-9538
DISCLAIMER The authors’ views expressed in this publication do not necessarily reflect the
views of the United States Agency for International Development or the United States
Government.
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QPR 1 i
TABLE OF CONTENTS
TABLE OF CONTENTS ..................................................................................................... i
ACRONYMS ........................................................................................................................ ii
INTRODUCTION ............................................................................................................... 1
IMPACT EVALUATION ACTIVITIES ............................................................................ 2
UPDATE ON IMPACT EVALUATION ACTIVITIES .......................................................................................... 2
MAJOR ACCOMPLISHMENTS AND CHALLENGES OF PAST QUARTER ............................................... 3
MAJOR ISSUES THAT NEED TO BE RESOLVED ............................................................................................... 3
PERFORMANCE EVALUATION ACTIVITIES .............................................................. 3
UPDATE ON PERFORMANCE EVALUATION ACTIVITIES ........................................................................... 3
MAJOR ACCOMPLISHMENTS AND CHALLENGES OF PAST QUARTER ............................................... 4
MAJOR ISSUES THAT NEED TO BE RESOLVED ............................................................................................... 4
P&IE PLANNED ACTIVITIES FOR NEXT QUARTER ................................................ 4
ANNEX A. OCT 17 2018 R&A AND CRO PRESENTATION ..................................... 5
ANNEX B. LIST OF ATTENDEES (OCT 17 2018 R&A AND CRO
PRESENTATION IN KAMPALA) .................................................................................. 28
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QPR 1 ii
ACRONYMS
CRO Classroom Observations
EGR Early Grade Reading
FY Fiscal Year
LARA Literacy Achievement and Retention Activity
MoES Ministry of Education and Sports
NORC NORC at the University of Chicago
P&IE Performance and Impact Evaluation
RWI Research World International
R&A Retention and Attendance
SRGBV School-Related Gender-Based Violence
USAID United State Agency for International Development
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 1
INTRODUCTION
NORC at the University of Chicago (NORC), in partnership with subcontractor Panagora Group, is
pleased to submit this Quarterly Performance Report covering the period October to December 2018
under the USAID/Uganda Performance and Impact Evaluation (P&IE) of the 5-year Literacy
Achievement and Retention Activity (LARA) in Uganda.
LARA is a 5-year (April 7, 2015 to April 6, 2020) USAID-funded initiative to improve reading skills for
1.3 million learners in 28 districts throughout Uganda. The project, which is being implemented by RTI
International, is designed to support the Ministry of Education and Sports (MoES) - Uganda in its efforts
to improve early grade reading (EGR) and retention. LARA has two main objectives:
• Result 1 (R1) focuses on strengthening the capacity of MoES and other educational stakeholders
to deliver EGR. To this end, the activity focuses on improved reading skills in three local
languages (Luganda, Runyoro-Rutooro, and Runyankore-Rukiga) and English for early primary
grade learners P1-4.
• Result 2 (R2) focuses on promoting a safer primary school learning environment to prevent and
reduce incidents of school-related gender-based violence (SRGBV).
The activity hypothesizes that reducing SRGBV will increase learners' retention because they will be
able to focus on their lessons and feel secure in their learning environment, thereby improving their
ability to learn to read. To realize these objectives, the activity focuses its efforts on systemic capacity
building of the education system, school-level support, and community- and household-level support
and participation.
The Uganda LARA P&IE activity (April 2016 - April 2021) has two objectives: (1) to assess the impact
of LARA on learners’ literacy skills and retention rates; and (2) to assess the performance of LARA in
terms of project management, learning, design, implementation, results, and sustainability. The P&IE
activity is focused on LARA cluster 2 schools.
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 2
IMPACT EVALUATION ACTIVITIES
UPDATE ON IMPACT EVALUATION ACTIVITIES
Presentation on Retention and Attendance and Classroom Observations
In a presentation to USAID/Uganda and project stakeholders in Kampala on October 17th, 2018, the
LARA P&IE Evaluation Team Leader (Dr. Ritu Nayyar-Stone) and the Evaluation Specialist (Dr. Alicia
Menendez) shared the results from data collection on School Retention and Attendance (R&A) over
the past school year. The presentation (see Annex A) showed findings on enrollment and absenteeism
for students, teachers’ absenteeism, and differences across grades, gender, and LARA treatment status.
Dr. Ursula Hoadley, the Education and Classroom Practices Expert, gave a presentation on the findings
from the Classroom Observations (CROs) conducted in LARA P&IE study. She discussed observations
of literacy lessons and the differences across control and treatment schools, and provided insight into if
and how LARA curriculum and practices were implemented in the classroom.
A total of 20 participants attended the presentations. Participants were from MoES, USAID,
implementing partners and civil society (see Annex B for details).
Evaluation Team Leader and Evaluation Specialist met with USAID and LARA in
Kampala
On October 16th, prior to the presentation on findings, the LARA Evaluation Team met with
USAID/Uganda (Kay Leherr, Samuel Hargadine, William Osafo) to discuss ideas and the feasibility of
additional data collection and an endline study to understand whether midline results remain at the
same levels after the LARA program ends. As a result of this discussion, the NORC LARA P&IE
Evaluation Team prepared a concept note with proposed activities and requirements for
USAID/Uganda consideration, which was shared in November 2018.
On October 18 the LARA P&IE Evaluation Team met with the LARA Implementer Partner (Chief of
Party, Geri Burkholder and literacy team) to continue the conversation initiated during the
presentation about classroom practices in LARA schools and their implications for the program.
Training to collect Year 2 Term 3 R&A and CRO data
NORC’s local data collection partner for EGR, Research World International (RWI), held an R&A and
CRO data collection training prior to the start of fieldwork. Training included 4 classroom training
days and 1 pilot day during the week of October 15, 2018. Evaluation Specialist Dr. Alicia Menendez
and Education and Classroom Practices Expert Dr. Ursula Hoadley attended the training as well.
Fieldwork to collect Year 2 Term 3 R&A data
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 3
RWI undertook R&A data collection for year 2 term 3 students in 71 cohort 2 schools that are
targeted by the P&IE. Data was collected for a total sample of 3,5051 learners currently in P1 through
P5 (most are in P2 and P5) over the course of approximately four weeks starting on October 22nd,
2018.
Fieldwork to conduct Year 2 Term 3 CROs
In conjunction with the R&A data collection, RWI also performed CROs in a sub-sample of 31 schools.
Data collection involved paper-and-pencil data entry, and audio recordings that were later transcribed.
The team captured audio recordings in a subset of 8 schools from the CRO sample.
MAJOR ACCOMPLISHMENTS AND CHALLENGES OF PAST
QUARTER
The major accomplishment of this past quarter was:
• Successful presentation of preliminary findings after one full year of R&A and CRO data
collection.
MAJOR ISSUES THAT NEED TO BE RESOLVED
None.
PERFORMANCE EVALUATION ACTIVITIES
UPDATE ON PERFORMANCE EVALUATION ACTIVITIES
Since the final performance evaluation will take place in March/April 2020, no activities have been
initiated for this component. NORC intends to use midline findings of the impact evaluation (Oct
2019) to inform the performance evaluation (Feb 2020) in such matters as the locations of schools to
be selected for field visits and the questions to be asked during the KII. The 2017 baseline was an
indication of the status in cluster 2 schools prior to LARA intervention, Yand NORC expects
considerable changes between the baseline and the midline data collection.
1 The number has decreased due to the unfortunate death of 2 learners.
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 4
MAJOR ACCOMPLISHMENTS AND CHALLENGES OF PAST
QUARTER
None.
MAJOR ISSUES THAT NEED TO BE RESOLVED
None.
P&IE PLANNED ACTIVITIES FOR NEXT
QUARTER
Preparation for R&A Data Collection (Year 3, Term 1)
In preparation for the data collection in the first quarter of 2019, NORC and RWI will review the data
collection instruments and protocols from the last data collection round and updated them based on
lessons learned from fieldwork and data cleaning. NORC’s Nfield programmer will update the student
instrument to show the correct grades in which the pupils should be enrolled, assuming that all were
promoted to the next grade at the end of the 2018 school year. Additionally, the teacher instrument
will be updated and no names will be preloaded as we expect new teachers to be assigned to P3 and
P6 in 2019.
Fieldwork to collect Year 3 Term 1 R&A data
RWI will conduct R&A data collection for year 3 term 1 in 71 schools for a total sample of 3,505
learners in P2 through P6 (most will be in P3 and P6) over the course of approximately three weeks
starting on March 11th, 2019. RWI will hold an R&A data collection training on March 9th prior to the
start of data collection.
Initiate Preparation for the Midline Impact Evaluation
Based on the evaluation design, NORC will conduct the midline impact evaluation for R1 and R2 in
2019. NORC plans to collect the SRGBV data in term 2 (June/July 2019) and the EGRA data in term 3
(October 2019) to avoid burdening the schools with data collection and to ensure quality control.
Thus preparation for the SRGBV data collection will start in the first quarter of 2019 with an update of
all LARA activities, followed by additional survey questions to capture information on implemented
activities.
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 5
ANNEX A. OCT 17 2018 R&A AND CRO
PRESENTATION
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 6
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 7
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 8
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 9
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 10
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 11
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 12
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 13
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 14
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 15
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 16
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 17
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 18
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 19
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 20
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 21
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 22
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 23
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 24
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 25
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 26
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 27
Uganda LARA – AID Contract No. AID-617-C-16-00003
QUARTERLY PERFORMANCE REPORT 2019 QTR 1 | 28
ANNEX B. LIST OF ATTENDEES (OCT 17 2018
R&A AND CRO PRESENTATION IN KAMPALA)
Names Organization / Agency Title
Eonnie Katzler Redearth Education Director
Jody Underrheiner Reads for Life Executive Director
Jamses Kassaga
Avinaitne Teach for Uganda Director
Klinnie Nakanwagi Teach for Uganda Training and Support
Stephen Backman RTI SHRP Education Specialist
William Asazo USAID COP
Komukama Gloria Teach for Uganda Development and Communication
Officer
Charlotte Iraguhan Teach for Uganda CPO
Fiona Waaya USAID M&E Specialist
Martin Muwaga USAID Senior M&E Specialist
Ammanyire Daniel MOES Ass. Gender Unit
Consilous Rwanyonga RTI LARA Deputy Chief of Party T
Prossy Naayombi RTI LARA Senior Program Specialist
Peto Mayingo RTI LARA M&E Coordinator
Geri Burkholder RTI LARA COP
Victoria Brown Mango tree Research Technical Adviser
Sherry Ahititne Koica Program officer
Vinvent Funi Dusabe NCDC Curriculum Specialist
Sanit Kizido Omale Makerere University School of
Statistics Lecturer & Researcher
Richard Wamimbi NORC Local Evaluation Manager
Vicky Kengozi STIR Education Senior Program Manager