A comparisson of teacher and test based assessment for
Spanish primary and secondary education
Óscar D. Marcenaro-Gutiérrez*Anna Vignoles**
*Departamento de Estadística y Econometría (UMA) y Fundación Pública Centro de Estudios Andaluces (FCEA)
** University of Cambridge
XI Conference FESMadrid, 10-12th July 2013
G13
1
Outline:
Motivation
Aim
Data
Methodology
FES Madrid
Conclusions
3
FES Madrid
Motivation:
1. Needless to say: importance of education as an engine of socio-economic change.
4
The profitability of investment in education is higher
that investment in capital goods (Psacharopoulos, 1985)
FES Madrid
Importance of training as an engine of socio- economic change: Budgetary effort.
Motivation:
FES Madrid
Motivation:
Central to Economics of Education: Do we have precise measures of students’ outcomes? To what extent does the nature of the assessment criteria produce social class inequalities in academic outcomes?
FES Madrid
Motivation:
2. Teacher assessment matters, because:- Affects how students perceive their own academic ability (self-esteem, etc.).- It is used to determine whether a child can proceed to the next grade.
1. Virtual lack of datasets allowing the comparison “Teachers’ actual assesment-external test assesment”; novelty of the paper.
7
It is crucial to determine whether such assessment is a good guide to pupils’
actual level of achievement:PISA? Actual exams?....
FES Madrid
Aim:
Are assesment tests (test-based assessment –TestA-) and official marks (teachers-based assessment –TeachA-) telling different things? We explore this empirically.
Do these differences vary systematically with pupil characteristics?
8
gender
ethnicity
socio-economic background
FES Madrid
Aim:
Are assesment tests (test-based assessment –TestA-) and official marks (teachers-based assessment –TeachA-) telling different things? We explore this empirically.
Do these differences vary systematically with pupil characteristics?
Differences in achievement levels of pupils in public (100%) schools and semi-private schools.
Do this potential gap discourage further study, in general or in any particular subject area (such as science, arts, etc.), after completing compulsory education? (i.e. future success)
9
FES Madrid
10
Literature:
Gender or Ethnicity of the teacher might interact with that of the student to produce systematic differences in pupil achievement (Dee, 2005 & 2007): teachers over grade students of the same gender to themselves.Comparing teacher assessments of pupil achievement and test scores: Reeves et al (2001) , Gibbons and Chevalier (2008) –UK-, TestA-TeachA consistent.
Gutierrez and Adserá (2012) –as opposed to Calero and Waisgrais (2008)- public school students higher grades: might be explained by differences in grading practices between public and private school teachers.
FES Madrid
Data:
PISA: 15-years-old students -4th year ESO-(OECD countries).
Dependent variables: - Math Test scores (normalised).- Reading Test scores (normalised).
Independent variables:- Student characteristics- Family background 26 variables- School characteristics
PISA-2009: Spain: 25.887 students within 889 schools.Andalusia: 1700 students within 51 schools
11
Data (1st stage):
FES Madrid
Linked databases (novelty):
- Andalusian Social Survey: Education and Homes in Andalusia (Encuesta social: Educación y Hogares en Andalucía, ESOC10); includes diagnostic assesment tests.
- Administrative marking records for those in the survey sample.
Data (2nd stage):
FES Madrid
ASS10:- Students born in 1994 + students born in
1998.
15 years 11 years
- Pseudo-panel: past and future outcomes.
- Very important: Includes interviews aimed at parents and children.
13
Crossing information
(consistency)
Data (2nd stage):
FES Madrid
ASS10:- ADDITIONAL added value: link ASS10 with
information from an administrative information system.
14
Information entered by the educators from public and semi-
private schools
• Repeaters.• Disabled (or with special educational requirements).• Private schools (only 2.5%) –data non avaialble-.
Data (2nd stage):
Excluding
FES Madrid
Andalusia 2009, huge differences with
the rest of Spain?
16
• Differences between natives and immigrants men more substantial in Andalusia as compared to the rest of Spain
• Public school Vs Private school: same differences as for the rest of Spain.
• Mother educational level: variables even more relevant.
• Once again…. IMPORTANCE OF HUMAN CAPITAL ACCUMULATION OF THE PROGENITOR AS A WAY OF ACADEMIC SUCCESS FOR CHILDREN, PARTICULARLY FOR WOMEN.
A) School failure:
B) Performance determinants:
FES Madrid
17
¿Consistency? (PISA-ASS)0
.1.2
.3.4
Den
sity
-4 -2 0 2 4Normalized scores
PISA reading TestA
ASS10 reading TestA
Reading
0.1
.2.3
.4D
ensi
ty
-4 -2 0 2 4Normalized scores
PISA math TestA
ASS10 math TestA
MathSubstantial overlap
FES Madrid
High grade of consistency between PISA-2009 y ASS10-SEN conditional effects:
18
ReadingTestA
(ASS10)TestA
(PISA09)Women 25,93**
*
6,73***
Inmigrant -36,07**
-34,73**
Semi-private 40,10***
8,40*
Mother ed.: primary -9,53 14,01Mother ed.: lower secondary 1,42 8,73Mother ed.: upper secondary 25,54** 20,88*Mother ed.: Higher ed. 35,77*** 41,83***Constant 504,70*
**
477,37**
Observations 1287 843R2 0,12 0.07
¿Consistency?
FES Madrid
0.1
.2.3
.4D
ensi
ty
-4 -2 0 2 4Normalized scores
Difference in assesments (reading)
0.1
.2.3
.4D
ensi
ty
-4 -2 0 2 4Normalized scores
Difference TestA-TeachA (female students)
Difference TestA-TeachA (male studnets)
Difference in assesments (maths)
women men
Differences TestA-TeachA:
FES Madrid
Multivariate analysis
Multinomial Probit
Probability of: Dropping out or
repeating. Enrol in Vocational
training. Academic track
(Sciences, etc.)
OLS regressions/FE/RE
Effects of variables on the difference TestA-
TeachA(ASS-SEN)
20
FES Madrid
Table 2.a. Discrepancies TestA - TeachA (1994 cohort).
21
Specification VReading Maths
Female -0.03 -0.50***Immigrant 0.10 0.26Semi-private school (=1) 0.49*** 0.20**Education level of father:
Degree or higher -0.06 -0.05Education level of mother:
Degree or higher -0.11 -0.11Constant -0.14** 0.12*Number of observations 1,087 1,051R-squared 0.04 0.06
Differences TestA-TeachA: (conditional analyses)
FES Madrid
Table 2.b. Discrepancies TestA - TeachA (1994 and 1998 cohorts).
22
Differences TestA-TeachA: (conditional analyses)
Specification V
Reading Maths
Cohort 1994 (=1) -0.05 -0.06Female -0.06 -0.34***
Immigrant -0.12 -0.02Semi-private school (=1) 0.37*** 0.22***
Education level of father:
Degree or higher -0.07 -0.05Education level of mother:
Degree or higher -0.01 -0.03
Constant -0.07* 0.09**
Number of observations 2,778 2,725R-squared 0.03 0.04
Teachers over estimating achievement of girls?
(preconceived ideas about ability’?)
Teachers under estimating achievement of students at
Semi-private schools.(Do teachers measure the child’s
achievement relative to other????)RE: Virtually identical results (2-3 students from each school).
FES Madrid
23
¿Difference TestA-TeachA varies
across the ability distribution?
Differences TestA-TeachA: (conditional analyses)
FES Madrid
24
Note: Including controls for immigrant, father education level and mother education level.
Table 4: Controlling for quintile in the previous yearSpecification V
Reading MathsPrior achievement -Reading- (TeachAt-1):
2nd quintile -0.113rd quintile -0.64***4th quintile -0.33***5th quintile (top) -0.74***
Prior achievement -Math- (TeachAt-1):
2nd quintile -0.023rd quintile -0.65***4th quintile -0.49***5th quintile (top) -0.60***
Female 0.08 -0.40***Semi-private school (=1) 0.54*** 0.22***Constant 0.21 0.44***Number of observations 995 1,001Number of centres 447 451
Differences TestA-TeachA: (conditional analyses)
pupils TeachAt-1 narrows gap
FES Madrid
25
Note: Including controls for immigrant, father education level and mother education level.
Table 5: Controlling for TeachAt-1
Differences TestA-TeachA: (conditional analyses)
Specification VReading Maths
Prior achievement -Reading- (TeachAt-1): -0.26***
Prior achievement -Math- (TeachAt-1): -0.20***
Female 0.03 -0.50***Semi-private school (=1) 0.48*** 0.17**Household cultural index level:
Average cultural index -0.01 0.07Higher cultural index 0.15* 0.18*
Constant -0.18*** 0.11Number of observations 1,057 1,030Number of centres 466 460
≈
“teachers’ views about the ability of pupils are
persistent”
FES Madrid
26
Table 6: By school type
Differences TestA-TeachA: (conditional analyses)
≈
Specification VSemi-private
schoolPublic school
Reading Maths
Reading
Maths
Female -0.12 -0.50**
*
-0.00 -0.49***
Constant 0.49*** 0.42***
-0.16** 0.11
Number of observations
265 248 822 803
Number of centres 148 137 334 331
FES Madrid
27
¿ To what extent TeachA-TestA at 15 predict education outcomes at age 16?
Choice at 16:
1) Repeating or dropping out of school at age 16 (ref.).
2) Undertaking vocational study;
3) For those staying on the academic track, enrolled in: Sciences, Arts & Humanities or Social Sciences)
FES Madrid
28
Table 7: Exit to:
VocationalTraining
Arts SciencesSocial
SciencesTestA-TeachA (reading) -0. 49*** -0.33** -0.60*** -0.47***Female 0.05 0.25 0.07 0.41***Immigrant -0.05 0.50 -0.34 -0.27
Semi-private school (=1) 0.41 0.19 0.50*** 0.41**Constant -0.71*** -1.31*** 0.97*** 0.90***Number of observations 1164
Χ2 90.39***
“pupils who are under estimated by teachers in terms of their
achievement are more likely to drop out”
Choice at 16:
FES Madrid
Conclusions
High relevance of parents human capital (particularly mothers); this is deeper in Andalusia, relevant to economic growth and labour market opportunities of women!!! (higher performance and lower failure)
29
PISA09 and ASS10 similar results, TestA consistent.
FES Madrid
Conclusions
Systematic differences between teacher assessments and actual test scores.
Girls’ achievement is over estimated by teacher assessments relative to test scores (preconceived ideas?????).
Teachers under estimating achievement of students at Semi-private schools (measuing relative acievements –peer effect-?????).
Pupils who are under estimated by teachers in terms of their achievement are more likely to drop out→ TestA-TeachA has a longer term impact on pupil outcomes.
30
FES Madrid
Thank you!
31
FES Madrid
A comparisson of teacher and test based assessment for
Spanish primary and secondary education
Óscar D. Marcenaro-Gutiérrez*Anna Vignoles**
*Departamento de Estadística y Econometría (UMA) y Fundación Pública Centro de Estudios Andaluces (FCEA)
** University of Cambridge
XI Conference FESMadrid, 10-12th July 2013
G13
32
Top Related