Latin American Experience with

Enhancing Quality and Measuring

Quality

READ Global Conference

Moscow, October 1, 2009

Eduardo Velez

World Bank Sector Manager for Education

Human Development Sector

East Asia and Pacific

Current Situation in Latin America

Assessment Systems

Most countries have at least an incipient national assessment system based on standardized student achievement test, periodically applied to samples or all students of certain key grades in core academic subjects

Some countries have sub-national assessments systems

Most have participated in one or more international test

A few countries and sub-national entities have been producing school- and system-report cards

Assessment Systems: What

kinds of results are found?

- Quality lower than expected

- Huge differences in averages between rural and urban population, public and private schools, poor and non-poor, indigenous and non-indigenous populations

- Increasingly small, if any, differences between girls and boys

- Slow change in outcomes

Assessment Systems: What kinds of

results are found? Positive results

Important inputs: school climate, high expectations, principal’s leadership and permanence, homework, peer effects, educational materials, teacher’s satisfaction and knowledge of subject matter, active pedagogy, parents’ SES and participation, use of classroom assessment as a pedagogical tool, time on homework, interest in subject, student perception of relationship with teacher, understanding that science & math associated w/ better job opportunities & future financial security, parent’s (especially mother’s) education, home educational resources, assessment systems, school autonomy (process and human resources), less influence from unions, ……..

just like everywhere else!!!!!!!!!!!!!!

Assessment Systems: What kinds of

results are found? Negative results

The following characteristics have a negative impact:

Memorization, rote learning

Mother’s employment

Number of siblings

High student-teacher ratio

Frontal teaching method

Technology has mixed results

Assessment Systems: Big

Challenge

Quality of Education is the challenge

It should take 2 to 3 (at very most) grades to learn to read. If it is taking 4 to 6, are systems working at about 50% effectiveness?Poorest 53 countries spend $16 billion on primary education, are they “wasting” $8 billion of it?

International Comparisons

GDP per capita versus PISA scores (Math 2006)

PISA 2000: Math Scores & Dispersion

ISR

ARG

ROMBUL

BEL

GRC

PER

DEU

ALB

RUS

LIE

LVA

CHE

POL

NZL

USACZE

HUN

BRA

MKD

SWE

CHL

HKG

AUT

GBR

NO

LUX

AUS

PRTITA

ESP

FRA

NLDJPN

DNK

CAN

ISL

IRL

IDN

THA

MEX

KORFIN

Mean

S

co

re

Dispersion

200

400

600

280240 340 380 420

300

350

400

450

500

550

600

240 260 280 300 320 340 360 380

Dispersion

Sc

ore

MEX

Finland

IDN

Thailand

Tunisia

Serbia

Uruguay

BZL

TUR

Greece

Belgium

US

Italy

HK

GermanyIreland

Canada

Latvia

Macao

Korea

NLD

RUSPurtugal

JPN

CZK

AUS

HUN

SWEIceland

DEN

POL

ESP

Swit

NZD

NOR

Austria

FRA

FRA

PISA 2003: Math Scores & Dispersion

Assessment and Accountability

The Uruguay Experience (1)

Participation, consensus building and face to face discussion with teachers, principals, and supervisors (sample, all schools get results, all teachers can apply the test to their students, and can compare with national results)

Assessment and Accountability

The Uruguay Experience (2)

In-service teacher training as the first consequence of the assessment (starting with assessments’ results, voluntary and collective --involves the teacher team of a school--, exchange of experience with other teams, all year round—once a fortnight; participants receive 20% of salary; schools in poor areas are priority; focus on how to teach; and emphasis on practical activities in the classroom)

Assessment and Accountability

The Uruguay Experience (3)

Evidence on the impact-70% of teachers support national assessment-70.2% read MOE’s publications-55% changed teaching and evaluating practices-78% apply school based assessment

“When MOE appeared I had a brick in each hand… Little by little they convinced us… Now it is a valuable experience. We have changed our practice”

Evolución de resultados por contexto - Lenguaje en porcentajes de

alumnos suficientes

51,4

46,7

37,1

54,8

60,8

48,5

66,4

62,3

58,4

75,5

73,970,2

87,7

88,085,4

66,5

61,6

57,1

30

40

50

60

70

80

90

100

1996

1999

2002

1996

1999

2002

1996

1999

2002

1996

1999

2002

1996

1999

2002

1996

1999

2002

Muy Desfav. Desfavorable Medio

Favorable Muy Favorable TOTAL PAÍSEVOLUTION OF THE RESULTS BY SOCIAL CONTEXT

Language, percentage of students achieving an acceptable level in

the test /Primary 6th grade. The Uruguay experience (4)

Poor ………………………………………Rich

Assessment and Accountability

The Mexican Experience (1)

Increase AutonomyTo improve quality, efforts are needed to move decision-making to the school level, thus increasing school autonomy.

Increasing school autonomy can compensate disadvantaged schools.

Autonomy can help raise the schooling outcomes of indigenous peoples.

School autonomy reinforces the role of homework, learning styles and future value of education.

With more autonomy, schools could determine the appropriate mix of technology for their students.

Assessment and Accountability

The Mexican Experience (2)

Improve Accountability

Accountability mechanisms can improve school quality.

Accountability mechanisms that put people at the center of service provision can go a long way in making services work and improving outcomes.

Flexible and wide-ranging accountability mechanisms could encompass various types of services.

To improve quality, efforts are needed to move decision-making to the school level, thus increasing school autonomy.

Assessment and Accountability

The Mexican Experience (3)

Continue learning from the assessmentAssessment testing can be used to inform policy decisions.

Analysis of assessments can foster public and civil society involvement in education reform.

However, governments must be proactive in encouraging public debate using assessment results.

Expand coverage of the national assessments. National and international assessments could be

used to inform school reform process.

PISA 2003: Mathematics

GreciaDinamarca

Portugal

Colima DF

AguascalientesUruguay Serbia TurquiaTailandia

Mexico

350

400

450

500

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Sub1 1 2 3 4 5

Youth

%

Achievement level

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Albania

Macedonia

Sub1 1 2 3 4 5

Youth

%

Achievement Level

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Albania

Macedonia

Mexico

Chile

Sub1 1 2 3 4 5

Youth

%

Achievement Level

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Albania

Macedonia

Latvia

Hungría

Mexico

Chile

Sub1 1 2 3 4 5

Youth

%

Achievement level

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Albania

Macedonia

Latvia

Hungría

Suecia

Mexico

Chile

Sub1 1 2 3 4 5

Youth

%

Achievement level

0

10

20

30

40

50

60

1 2 3 4 5 6

Perú

Albania

Macedonia

Latvia

Hungría

Suecia Finlandia

Mexico

Chile

Sub1 1 2 3 4 5

Youth

%

Achievement level

Some Lessons for Assessment Policy

Implementing units must be committed withproducing materials useful for policy-makers, managers, instructor, teachers, beneficiaries with dissemination and use of results. (Reports with useful information about their performance and activities)

Technical legitimacy of test methodology is crucial

Timely and accurate data to inform policymaking

Unit doing it must be autonomous from political interests

Importance of detailed planning of actions and coherent implementation

Some Lessons for Assessment Policy

You need an assessment strategy, not just to administrate tests… policymakers use the results of evaluation of existing interventions to inform design and implementation of policies

A “user-friendly" approach to assessment facilitates its use by teachers and others

Articulate dissemination of results with an effective in-service training program

Trainers, teachers need space and time to meet, study, discuss, try new things

Some lessons for Assessment Policy

Autonomy… with support. With assessment’s information more autonomous schools can implement appropriate education policies

Accountability. A more accountable system will encourage more active participation by parents, teachers, and others, which is key to improving learning outcomes

Assessment. A system that is based in constant assessment and participation in international benchmarking exercises will improve cost-effectiveness