Education for Accountability Workshop, June 22 nd, 2009 Overview of the Evidence: Interventions in...

Education for Accountability Workshop, June 22nd, 2009

Overview of the Evidence: Interventions in Information for Accountability

Deon FilmerLead Economist, DECRG


Evidence: Information for Accountability (1)

Country

Authors Intervention Methodology

Findings

CHILE Mizalal et al 2007

Test-based school rankings published in newspapers & based on index of student test scores + student socio-economic characteristics. (primary & sec. schools) (SIMCE)

Multivariate Regression Analysis

Rural school enrolling poorer students have higher probability of receiving a low score, thus SIMCE scores do not control for student socio-economic background.

CHILE Mizalal & Urquiola 2007

Test-based school rankings published in newspapers & MoE website; based on index of student test scores, socio-economic characteristics, basic school data, inter-cohort test score gain measures. Schools ranked w/in pre-determined homogenous gps. (primary & sec. schools) (SNED) (1996-present)

Regression Discontinuity

No effect on 1st or 9th grade enrolment; No effect on school tuition

CHILE Contreras et al 2002

SNED school rankings (as above)

Multivariate Regression Analysis

Increased student performance in winning schools



Country


Findings

United States

Carnoy , Loeb 2002; Hanushek & Ramond 2003; Loveless 2005; Chicago: Jacob 2005; Florida: Charkrabarti 2006; Figlio & Rouse 2006; Rouse et al 2007; West & Peterson 2006; Kentucky: Clark 2003; Texas: Haney 2000 & 2002

Federal government legislated test-based school rankings along with school and aggregate (district or state) level report cards. Rankings made publically available in media (varies by state). (No Child Left Behind Act)

Multivariate regression analysis; cross-state comparisons; panel data –based comparisons

Increased public awareness of relative school performance; Increased gaming & other unintended consequences (cheating, teaching to the test, student exclusions); Ambiguous results on student outcomes.

United Kingdom

Bradley et al 2000; Burgess et al 2005; Goldstein 2001; Tymms and Fitz-Gibbon 2001

Government publishes school performance on national exams (passage rates) and ranks schools accordingly (rankings appear in newspapers).

Multivariate regression analysis exploiting different identification strategies

Improvement in school performance (raw test scores) but unclear identification – not possible to attribute effects to accountability reform; Creation of perverse incentives (student exclusions, cheating etc.); Lower performance of low ability students



Country

Authors

Intervention Methodology

Findings

UGANDA

Reinnikka Svensson 2005 & 2006; Bjorkman 2006

Participatory Tracking Expenditure: Published amounts & dates of per capita grants to district education officers/ primary schools in local & national newspapers.

Difference in Difference/ Instrumental Variables

A school close to a newspaper outlet suffered 25 percentage points less from capture as compared to a school 1 SD (~ 30 km) further away from a newspaper outlet; 1 SD reduction in distance (30 km) led to a 0.09 SD increase in ave. test scores; Increase in enrolment in schools where expenditure shares increased.

LIBERIA

forthcoming

Administration of Early Grade Reading Assessment & results disseminated > school community.

Randomized Trial

forthcoming

BENIN forthcoming

Impact of school based management trainings w/ and w/o village radio campaigns (broadcasts concerning school performance & progress)

Randomized Trial

forthcoming



Country Authors


Findings

INDIA Khemani, Banerjee et al 2008.

Neighborhood school report cards (including student literacy/ numeracy rates and student enrolment rates) were generated & discussed at village meetings.

Randomized trial

No effect on student enrolment, teacher effort or student learning; No impact on parent activism/ VEC activities

INDIA Pandey, Goyal, Sundararaman 2008

Information campaigns (re: roles and responsibilities of school community in school management) conducted in village meetings.

Randomized trial

Increase (14-27%) in reading scores in 2 of 3 states (in 1 of 3 grade levels); Increase in writing (15%) & math scores (27%) in 2 different grade levels in 3rd state.

PAKISTAN

Das, Andrabi, Khwaja 2009

Provision of report cards with child information and information on all schools in village at a meeting in every school (for 2 years)

Randomized trial

Average test-scores (in govt’ schools) increased by 0.10 std dev; Little increase in switching; Decrease private school fees by 21%

TODAY***


Overview of the Evidence: Interventions in School-

Based Management

Harry PatrinosLead Education Economist,

HDNED


Evidence: School-Based

Management (1) Country Authors Intervention Methodolog

yFindings

MEXICO Murnane et al 2006

Schools given resources for implementing a school plan in consultation with parents (funding goes to both infrastructure & teaching resources). PEC (Quality School Program)

Difference in Difference (w/ systematic check of equal trends bt. treatment & control groups

Positive impact on dropout rates; No effect on repetition

MEXICO Skoufias & Shapiro 2006

Schools given resources for implementing a school plan in consultation with parents (funding goes to both infrastructure & teaching resources). PEC (Quality School Program)

Matching estimation w/ DD (one year pre-intervention trend

Positive impact on dropout, failure, and repetition rates

MEXICO Lopez-Calva & Espinosa 2006

Parents associations given small amounts of money for civil works and infrastructure. AGES (Support for School Management)

Matching, cross section

Positive impact on test scores


Country Authors


Findings

NICARAGUA

King & Ozler 1998

Gov’t establishment of consultative school councils (1991); Councils transformed into monitoring boards w/ wide scope of autonomous decisions (teachers vote to enter program) (1993). (ASP)

Matching, panel data

De jure autonomy yielded no impact. Real autonomy (to hire and fire teachers) yielded a positive impact on test scores

NICARAGUA

Parker 2005

Gov’t establishment of consultative school councils (1991); Councils transformed into monitoring boards w/ wide scope of autonomous decisions (teachers vote to enter program) (1993). (ASP)

Matching, panel data

Positive impact on test scores

BRAZIL Paes de Barros & Mendonca 1998

Direct transfer of funds to schools, election of principals, and creation of local school councils (gov’t decentralization)

Difference-in-difference , no pre-intervention trends

Positive impact on repetition and dropout rates; No impact on test scores

Evidence: School-Based

Management (2)


Country Authors Intervention Methodology Findings

EL SALVADOR

Jimenez & Sawada 1999 & 2003

Community associations are responsible for administering funds, hiring/firing teachers, and monitoring and maintaining infrastructure (EDUCO)

Heckman correction model; exclusion restriction: gov’t prioritizing targeting formula

Increases reading scores & decreases absenteeism (1999); Increases retention (2003).

HONDURAS

Di Gropello & Marshall 2005

School councils have autonomy over hiring and firing teachers, monitoring attendance, managing funds, & maintaining infrastructure (PROHECO)

Heckman correction model; exclusion restriction: presence of potable water & community services

No effect on test scores;Small changes in dropout rates

SEVERAL COUNTRIES

King, Orazem, & Gunnarsson 2003 & 2004

School-based management interventions across several countries, spanning several years

IVs: prinicipal’s attributes & legal structure

No effect on test scores (2003 & 2004); Positive impact on parental participation (2004)

Evidence: School-Based Management (3)


Country Authors


Findings

NEPAL Chaud-huryforthcoming

Communities express desire to take over management of schools (receive 1-time incentive grant); Both gov’t (control) and community-managed schools experiment with school report cards.

Randomized trial

forthcoming

PAKISTAN Das et al.More forthcoming

School councils constituted following certain rules; NGO hired to manage school together w/ school council; Each SC received $4000.

Randomized trial (but not upheld); used intention to treat.

Thus far, no change in enrollment or teacher absenteeismMore forthcoming

MADAGASCAR

Nguyen, Lassibille 2008More forthcoming

Training of all key actors in education sector on their respective roles & responsibilities; Equipping key actors w/ management manual; Increasing information flow through use of report cards (at school & other administrative levels)

Randomized trial

Increased student attendance by 4.3% percentage pts; Increase of 0.1 std. dev in test scores (but only in schools where school-level actors trained); Improved provider behavior (when intervention @ school-level)

Evidence: School-Based Management (4)


Country


Findings

KENYA Vermeersch forthcoming

Training of school committees to monitor teachers on teacher performance and committee-based hiring of teachers (versus headmaster hiring of new teachers)

Randomized trial

forthcoming

MEXICO

Gertler, Patrinos, Rubio-Codina2006 & forthcoming

Parental empowerment program trains school administrators, teachers and parents to engage in joint strategic planning about how to improve their schools; school committees entrusted with monetary resources also.

Quasi-experimental – Panel Data & DID

Reduction (by 4-5%) in grade repetition and failure rates; More forthcoming

TODAY***Evidence: School-Based

Management (5)

Education for Accountability Workshop, June 22 nd, 2009 Overview of the Evidence: Interventions in...

Documents

Transcript of Education for Accountability Workshop, June 22 nd, 2009 Overview of the Evidence: Interventions in...