UNIVERSITY OF JYVÄSKYLÄ

Characteristics of low and top performers in reading and mathematics. Exploratory

analysis of 4th grade PIRLS and TIMSS data in Nordic countries

Sari Sulkunen, Department of Languages

Kari Nissinen, Finnish Institute for Educational Research

Pekka Kupari, Finnish Institute for Educational Research

University of Jyväskylä

UNIVERSITY OF JYVÄSKYLÄ

Introduction

Consider three student groups: – Top performers– Low performers– ’In-betweens’, intermediate performers

Vast majority of students are neither low or top performers– ’In-between’, intermediate pupils– Define these pupils as the reference group of analyses

UNIVERSITY OF JYVÄSKYLÄ

Introduction

How to minimize the number of low performers and maximize the number of top performers?

Look for characteristics which distinguish the low performers and top performers from the reference group– Student– Home– Teaching – School

Are there key characteristics that could be managed by school and educational system?

UNIVERSITY OF JYVÄSKYLÄ

Previous findings

Analyses of earlier large-scale data sets (TIMSS, PIRLS, PISA, national surveys etc.)

=> Certain characteristics repeatedly appear important

1. Socio-economic background– Parents’ education and occupation– Resources at home– Appreciation of education and culture at home– Often these explain school differences also

UNIVERSITY OF JYVÄSKYLÄ

Previous findings

2. Student’s attitudes and engagement– Motivation to school work– Free-time reading vs other activities– Self-concept

3. Student’s basic skills– E.g. reading accuracy and fluency

• Lack of basic skills makes all learning difficult

4. Student’s language background– Command of instruction language highly important

• ’Everyday’ vocabulary not enough?

UNIVERSITY OF JYVÄSKYLÄ

Previous findings

Role of gender?– Girls outperform boys in reading– Gender differences usually smaller in mathematics

– Gender itself not always significant in multivariate analyses

=> question of attitudes and engagement more than question of sex?

UNIVERSITY OF JYVÄSKYLÄ

Previous findings

School and class-level factors play minor role (in Nordic countries)– Student characteristics explain much of (already small) school

and class differences

Why is this? Are teachers’ and schools’ efforts relevant at all?– Overall level of teachers’ and schools’ efforts and resources

relatively standard (high)?– Question of teacher-student interaction; student’s contribution

(engagement) crucial– Difficult to catch with questionnaire data!

UNIVERSITY OF JYVÄSKYLÄ

Data

We utilize Nordic PIRLS 2011(reading) and TIMSS 2011 (mathematics) data sets– 4th grade pupils (except 5th grade for Norway)

– Several sources of data: • student questionnaire• parent questionnaire (PIRLS)• teacher questionnaire• school questionnaire

UNIVERSITY OF JYVÄSKYLÄ

Data

Finland, Norway and Sweden implemented TIMSS and PIRLS on the same schools and students

In Denmark PIRLS and TIMSS had separate student samples

Norway assessed 5th grade students also (with smaller sample!)– Comparable with other countries’ 4th grade

FinIand, Norway and Sweden implemented an additional test of student’s basic reading skills

UNIVERSITY OF JYVÄSKYLÄ

Data

Number of schools participating in PIRLS

Students assessed in PIRLS

Number of schools participating in TIMSS

Students assessed in TIMSS

Denmark 4th 232 4594 216 3987Finland 4th 145 4640 145 4638Norway 5th 53 1258 54 1270Sweden 4th 152 4622 152 4482

UNIVERSITY OF JYVÄSKYLÄ

Overall results

In reading (PIRLS), all four Nordic countries performed internationally relatively high– Finland (568) and Denmark (554) in Top 10– Norway 5th grade (549) and Sweden (542) in Top 20

• Scale midpoint = 500

In mathematics (TIMSS), Norway 5th, Finland and Denmark performed high while Sweden was mediocre– Norway 5th grade (548) and Finland (545) in Top 10– Denmark (537) in Top 20– Sweden (504) close to scale midpoint (500)

UNIVERSITY OF JYVÄSKYLÄ

Performance groups

Define low performers as students below 475 points475 = intermediate international TIMSS and PIRLS benchmark

Define top performers as students over 625 points625 = advanced international TIMSS and PIRLS benchmark

UNIVERSITY OF JYVÄSKYLÄ

Group percentages / reading

% low performersin reading: girls

% low performersin reading: boys

% top performersin reading: girls

% top performersin reading: boys

Denmark 4th 10 14 14 11Finland 4th 6 10 23 14Norway 5th 9 13 12 8Sweden 4th 13 17 11 7

UNIVERSITY OF JYVÄSKYLÄ

Group percentages / mathematics

% low performersin maths: girls

% low performersin maths: boys

% top performersin maths: girls

% top performersin maths: boys

Denmark 4th 19 17 9 12Finland 4th 17 17 10 13Norway 5th 14 13 9 16Sweden 4th 34 32 2 4

UNIVERSITY OF JYVÄSKYLÄ

Statistical modelling

Three-level logistic regression models for each country

A number of explanatory variables were introduced in the multivariate regression

– Suggestions from previous research– Interesting variables from the TIMSS and PIRLS background

questionnaires

UNIVERSITY OF JYVÄSKYLÄ

Statistical modelling

After preliminary steps a reduced variable list was used in the final models

– Variables appearing significant in at least one country

– Two-way interactions were originally considered but left out from the final regression models

Recall: multivariate model => effects are adjusted for all other variables in the model

UNIVERSITY OF JYVÄSKYLÄ

Results

Student’s basic skills – Important predictor of performance in every case– Only minor variation between countries– Strong relation between skills at school start and 4th grade

performance (stability)• Early literacy activities with child predict top reading

performance

Student’s attitudes– Confidence <=> performance– Motivation and engagement in learning, no show???– Liking to read related to reading performance

• Not found with mathematics!

UNIVERSITY OF JYVÄSKYLÄ

Results

Student’s activities– Reading activities:

• No role in distinguishing low performers from intermediate ones

• Instead, top performers stand out from the rest– Free-time computer use negatively associated with

performance

Home resources (SES)– Important in every country, with minor variations– Educational resources, books, parent’s education,…– Wealth of neighborhood (DEN, SWE)

UNIVERSITY OF JYVÄSKYLÄ

Results

Language at home / language difficulties in class– Some role found in every country

Parental and teacher support– Highly significant in every country (in various forms)– But: more support <=> lower performance

• Top performers’ need for support not recognized?• Top performers do not express need for support?

UNIVERSITY OF JYVÄSKYLÄ

Results

Male gender– Low performance in reading: DEN– Top performance in maths: FIN, NOR, SWE

Female gender– Low performance in maths: SWE– Top performance in reading: FIN

UNIVERSITY OF JYVÄSKYLÄ

Results School-related variables largely non-significant

(given the student, home and teaching related variables)– Discipline and safety at school – School emphasis on academic success– School size and location– School resources

– Only occasional exceptions:• Sweden: low performance associated with urban schools• Denmark: top reading performance associated with urban

schools and good resources• Denmark: low maths performance associated with small

schools

UNIVERSITY OF JYVÄSKYLÄ

Results

Teaching-related variables completely non-significant except for supporting low performers(given the student, home and school related variables)– Teacher’s experience and education– Student ability grouping– Activities in engaging students– Teacher’s confidence in teaching maths– Memorization or elaboration in teaching maths– Calculator or computer use in class– Amount of given homework in maths– Characteristics of reading materials

UNIVERSITY OF JYVÄSKYLÄ

Conclusions

Gap between low and top performers seems stable– School cannot even out the impact of background and

differences in basic skills at school start

Þ support for struggling students still insufficient?

In reading:– Early help with problems in linguistic development– Support in reading engagement, also for (disadvantaged)

families • Bring school’s and student’s textual landscapes closer

(text materials)• Family literacy programs

UNIVERSITY OF JYVÄSKYLÄ

Conclusions

In reading:– Immigrants: need for intensified support both in instruction

language and in mother tongue– Development of reading and learning strategies

• Special benefit for struggling readers!

In mathematics:– Support in reading!– Self-concept / confidence / positive attitudes crucial

• Influence of teacher actions and attitudes• Appropriately tuned tasks and homework• Bring school’s and student’s ’mathematical landscapes’

closer (tasks, learning materials)

UNIVERSITY OF JYVÄSKYLÄ

Conclusions

What to do with top performers?– Teachers and parents may overlook them since everything

goes fine => implications e.g. for school enjoyment?– Mild interest from adults’ side and lack of challenges may

hinder reaching full potential

Need for individualized pedagogical solutions, how about resources?

Need for continuing professional development of teachers, how about resources?

UNIVERSITY OF JYVÄSKYLÄ

Tak! Takk! Tack!Kiitos!