PERSPECTIVES ON QUALITY ANDEQUITY FROM LARGE-SCALE
ASSESSMENT STUDIES
John Ainleyand
Eveline Gebhardt
Outline• Properties of large-scale assessments• Perspectives from NAPLAN: 2008 to 2013
– National changes in reading achievement– Improvements in reading for Indigenous students– Differences in changes across jurisdictions
• Messages from PISA– Changes in reading and mathematics
achievement– Student background – Jurisdictional changes in reading and
mathematics– Variation between and within schools
• Summary
Perspectives from NAPLAN
National mean reading scores: 2008-2013
2008 2009 2010 2011 2012 2013 Difference 2008–13
Year 3 401 411 414 416 420 419 19
Year 5 484 494 487 488 494 502 18
Year 7 537 541 546 540 542 541 4
Year 9 578 581 574 580 575 580 2
National mean numeracy scores: 2008-2013
2008 2009 2010 2011 2012 2013 Difference 2008–13
Year 3 397 394 395 398 396 397 0
Year 5 476 487 489 489 489 486 10
Year 7 545 544 548 545 538 542 -3
Year 9 582 589 585 583 584 584 2
The story so far:
• Improvement in reading in years 3 and 5• No change in reading in years 7 and 9• No significant national change in numeracy
at Years 3, 5 (note exception of Queensland), 7 or 9
Changes in mean reading scores:Indigenous and non-Indigenous students
2008 2009 2010 2011 2012 2013 Difference 2008–13
Year 3
Indigenous 314 327 331 332 333 344 30
Non-Indigenous 405 415 419 420 424 423 18
Year 5
Indigenous 403 414 410 410 409 439 36
Non-Indigenous 489 498 491 492 498 506 17
Jurisdictional mean Year 3 reading scores: 2008-2013
2008 2009 2010 2011 2012 2013 Difference 2008–13
QLD 371 386 393 400 409 408 37
NT 307 322 329 323 332 339 33
ACT 421 434 439 443 444 442 21
WA 387 396 399 400 408 406 19
Jurisdictional mean Year 5 reading scores: 2008-2013
2008 2009 2010 2011 2012 2013 Difference 2008–13
NT 405 421 412 403 405 437 32
QLD 466 478 469 469 480 497 31
WA 474 482 478 480 483 495 22
ACT 503 513 509 516 519 519 16
SA 478 484 477 478 484 492 14
VIC 497 506 502 504 504 510 13
The story extended:
• Improvement in reading in years 3 and 5• No change in reading in years 7 and 9• No significant national change in numeracy
at Years 3, 5, 7 or 9• Larger improvement in reading for
Indigenous than non-Indigenous students (Years 3 & 5)
• Differences among jurisdictions with large increases in:– Queensland– Northern Territory
Interpretation
• Emphasis on the early years (school and pre-school)– Programs and resources– Focus on reading in the early school and
pre-school years• Programs intended to improve outcomes for
Indigenous students• Structural change in Queensland
– Introduction of a foundation/preparatory year
– Accompanied by a strong school improvement strategy
Messages from PISA
Change in reading literacy:2000 - 2012
2000 2003 2006 2009 2012Change
2000-2012
Poland 479 497 508 500 518 39
Germany 484 491 495 497 508 24
Portugal 470 478 472 489 488 18
Australia 528 525 513 515 512 -16
Finland 546 543 547 536 524 -22
Sweden 516 514 507 497 483 -33
Change in mathematical literacy:2003 - 2012
2003 2006 2009 2012Change
2003-2012
Poland 490 495 495 518 27
Portugal 466 466 487 487 21
Germany 503 504 513 514 11
Australia 524 520 514 504 -20
Finland 544 548 541 519 -26
Sweden 509 502 494 478 -31
Changes in achievement and student background
Reading 2000-2009 Mathematics 2003-2012
2000 2009 Gender
Females 546 533 Males 513 496 Difference 34 37 Indigenous status
Non-Indigenous 531 518 Indigenous 448 436 Difference 83 82 Language background English at home 535 518 LBOTE 504 509 Difference 31 10 Location
Metropolitan 535 521 Non-metro 518 496 Difference 17 25
2003 2012 Gender
Females 494 491 Males 505 502 Difference -11 -11 Indigenous status
Non-Indigenous 526 507 Indigenous 440 417 Difference 86 90 Language background
English at home 529 506 LBOTE 505 509 Difference 24 -3 Location
Metropolitan 528 511 Provincial 515 486 Remote 493 444 Diff (met-prov) 13 25
Changes in achievement & socioeconomic background (ESCS): overall gradient
Reading 2000-2009Mathematics 2003-2012
2003 2012
Australia 42 42
Canada 34 31
USA 45 35
UK na 41
Finland 33 33
Germany
47 43
2000 2009
Australia
47 46
Canada 38 32
USA 52 42
UK na 44
Finland 25 31
Germany
52 44
The story so far
• Modest declines in reading and mathematics among 15 year-olds
• Extent of decline is similar for reading and mathematics
• Does not appear to be associated with differences in student background.
Jurisdictional change in reading literacy:2000 -2012
PISA 2000
PISA 2009
PISA 2012
Difference 2000–2012
Victoria 516 513 517 1
Queensland 521 519 508 -13
Western Australia 538 522 519 -19
Northern Territory 489 481 466 -23
New South Wales 539 516 513 -26
ACT 552 531 525 -27
Tasmania 514 483 485 -30
South Australia 537 506 500 -37
Jurisdictional change in mathematical literacy: 2003 -2012
PISA 2003 PISA 2012 Difference 2003-2012
Victoria 511 501 -10
Queensland 520 503 -16
New South Wales 526 509 -17
Tasmania 507 478 -30
ACT 548 518 -30
Western Australia 535 516 -32
Northern Territory 496 452 -45
South Australia 548 489 -46
Summary of jurisdictional changes in PISA reading and mathematicsReading change(00-12)
Jurisdiction Change
VIC 1
QLD -13
WA -19
NT -23
NSW -26
ACT -27
TAS -30
SA -37
Mathematics change (03-12)
Jurisdiction Change
VIC -10
QLD -16
NSW -17
TAS -30
ACT -30
WA -32
NT -45
SA -46
The story extended
• Modest declines in reading and mathematics among 15 year-olds
• Extent of decline is similar for reading and mathematics
• Does not appear to be associated with differences in student background
• Differences among jurisdictions in the extent of decline
• Changes in reading and mathematics achievement appear to run parallel
Variation between and within schools• Variance
– dispersion calculated as the mean of the squared deviations of observed values from a mean
– total variance made up of: • variance within schools and • variance between schools’ mean scores
• Percentages of total variance• Within / total• Between / total
• % variance between schools is an indicator of the extent to which schools differ
Between-school variance in reading and mathematics in selected countries
% variance between schools Reading Mathematics PISA 2000 PISA 2009 PISA 2003 PISA 2012 Finland 8% 9% 5% 8% Sweden 9% 18% 9% 13% Canada 20% 22% 17% 20% Australia 20% 26% 22% 28% United States 30% 36% 24% 24% Germany 59% 60% 57% 53% OECD average 36% 37% 33% 35%
Change in mean mathematics score (2003-12) and change in % between-school variance
The story extended a little further• Modest declines in reading and mathematics among
15 year-olds• Extent of decline is similar for reading and
mathematics• Does not appear to be associated with differences in
student background• Differences among jurisdictions in the extent of
decline• Changes in reading and mathematics achievement
appear to run parallel• Declines in Australia associated with larger differences
among schools• In general for mathematics larger changes in between
school variance are associated with larger declines
Speculation
• Differentiation appears to be associated with lower overall achievement (Willms, 2010)
• Differentiation among schools may be associated with differentiation in teaching expertise among schools
• Possibly a reciprocal relationship arising from attractiveness of schools to teachers
• Caution about policies that exacerbate differences.
Thank you
QuestionsDiscussion
Top Related