TIMSS Survey

Highlights from PIRLS and TIMSS 2011

Jack Buckley

National Center for Education Statistics

Washington, DC

December 11, 2012

OverviewWhat are PIRLS and TIMSS and which education

systems participate in them?

How are U.S. students performing compared to their international peers in: 4th-grade reading? 4th- and 8th-grade mathematics? 4th- and 8th-grade science?

How are students performing in participating U.S. states?

2

What are…?

3

• Progress in International Reading Literacy Study (PIRLS)

• 4th-grade reading assessment

* In 1999, no grade 4 assessment.

TIMSSPIRLS

2001 2006 2011

1995 1999* 2003 2007 2011

• Trends in International Mathematics and Science Study (TIMSS)

• 4th- and 8th-grade mathematics and science assessment

• Developed by the TIMSS & PIRLS International Study Center at Boston College, under contract to the International Association for the Evaluation of Educational Achievement (IEA)

PIRLS participating education systems (in 4th grade)

4

26 education systems

201132 education systems53 education systems

5

TIMSS participating education systems (in either/both grades)

44 education systems37 education systems47 education systems57 education systems74 education systems

2011

Participating states

6

• 9 states participated as separate entities to obtain scores in PIRLS or TIMSS or both

• Each participated as part of the nation and on its own• NCES funded participation at grade 8 of all states except Florida as

part of a study to statistically link NAEP (National Assessment of Educational Progress) and TIMSS (Florida received other Education Department funding)

PIRLS TIMSS grade 4 TIMSS grade 8

Florida FloridaNorth Carolina

FloridaNorth CarolinaAlabamaCaliforniaColoradoConnecticutIndianaMassachusettsMinnesota

U.S. national sample size

4th gradeSchools: 370

Students: 12,726

4th grade Schools: 369

Students: 12,569

8th gradeSchools: 501

Students: 10,477

7

TIMSSPIRLS

U.S. state sample sizes(public schools only)

4th grade(FL)

Schools: 77

Students: 2,598

4th grade (FL, NC)

Schools: 46 – 77

Students:

1,792 – 2,661

8th grade(AL, CA, CO, CT, FL, IN, MA, MN, NC)

Schools: 53 – 82

Students:

1,712 – 2,614

8

TIMSSPIRLS

READING

PIRLS

Definition The ability to understand and use those written language forms required by society and/or valued by the individual. Young readers can construct meaning from a variety of texts. They read to learn, to participate in communities of readers in school and everyday life, and for enjoyment.

Content Literary and informational texts

Cognitive dimensions Focus on and retrieve Make straightforward inferencesInterpret and integrateExamine and evaluate

10

PIRLS 2011 framework

What is on the PIRLS assessment?

11

Percentage of 4th-graders answering correctly:U.S.: 90%Int’l Avg.: 89%

What is on the PIRLS assessment?

12


13

U.S. average score (556) higher than the PIRLS scale average (500)

Higher than U.S.

Not measurably different than U.S.

Lower than U.S.

14

U.S. average score (556) lower than in 5 education systems

Higher than U.S.


Lower than U.S.

15

U.S. average (556) not measurably different than in 7 education systems

Higher than U.S.


Lower than U.S.

16

U.S. average score (556) higher than in 40 education systems

Higher than U.S.


Lower than U.S.

17

Average reading scores of 4th-grade students increased from 2006 to 2011 in 13

education systems, including the U.S.

*p < .05. Change in average scores is significant.NOTE: Education systems ordered according to average reading score in 2011.

Change in average score

Hong Kong-CHN

Singapore

UNITED STATES

Denmark

Chinese Taipei-CHN

England-GBR

Slovenia

Poland

Norway

Georgia

Trinidad and Tobago

Iran, Islamic Rep. of

Indonesia

-20 -10 0 10 20 30 40

7*

9*

16*

8*

18*

12*

9*

6*

9*

17*

35*

36*

24*

Alberta-CAN

Sweden

Italy

Germany

Hungary

Bulgaria

Austria

Lithuania

-30 -20 -10 0 10 20 30

-12*

-8*

-10*

-7*

-12*

-15*

-9*

-9*


18

Average reading scores of 4th-grade students decreased from 2006 to 2011 in 8

education systems

*p < .05. Change in average scores is significant.NOTE: Education systems ordered according to average reading score in 2011.

PIRLS international reading benchmarks

19

Grade 4Advanced (625) Students can interpret figurative language, distinguish and

interpret complex information from different parts of text, and integrate ideas across text to provide interpretations about characters’ feelings and behaviors.

High (550) Students can recognize some textual features, make inferences on the basis of abstract or embedded information, and integrate information to recognize main ideas and provide explanations.

Intermediate (475) Students can identify central events, make straightforward inferences from the text, and begin to make connections across parts of the text.

Low (400) Students can retrieve explicitly stated details from literary and informational texts.

Percentages of U.S. 4th-grade students reaching PIRLS reading benchmarks were higher than international medians in 2011

Low Intermediate High Advanced0

102030405060708090

100 98

86

56

17

95

80

44

8

United StatesInternational median

Benchmark

Percent

20

NOTE: All U.S. percentages are significantly higher than the corresponding PIRLS international median at the .05 level of statistical significance.

NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure.

21

Two systems had higher percentages of 4th-grade students reaching Advanced than the U.S.

Higher than U.S. (p < .05)

Not measurably different than U.S. (p < .05)

Singapore

Florida-USA

Russian Federation

Northern Ireland-GBR

Finland

England-GBR

Hong Kong-CHN

UNITED STATES

Ireland

Ontario-CAN

0 10 20 30 40 50 60

24

22

19

19

18

18

18

17

16

15

Percentage reaching Advanced benchmark

MATHEMATICS

TIMSS 2011 mathematics framework

23

TIMSS mathematicsContent dimensions

Grade 4 Number Geometric Shapes and Measures Data Display

Grade 8NumberAlgebraGeometryData and Chance

Cognitive dimensions

KnowingApplyingReasoning

24

GRADE 4


What is on the TIMSS mathematics assessment?


25

GRADE 4


26

GRADE 8


GRADE 8


27

GRADE 8



28

U.S. average score (541) higher than the TIMSS scale average (500)

Higher than U.S.


Lower than U.S.

29


Higher than U.S.


Lower than U.S.

30


Higher than U.S.


Lower than U.S.

31


Higher than U.S.


Lower than U.S.

32


Higher than U.S.


Lower than U.S.

33


Higher than U.S.


Lower than U.S.

34


Higher than U.S.


Lower than U.S.

35


Higher than U.S.


Lower than U.S.

36

Average mathematics scores of 4th-grade students increased from 2007 to 2011 in 12

education systems, including the U.S.

*p < .05. Change in average scores is significant.NOTE: Education systems ordered according to average mathematics score in 2011.

Chinese Taipei-CHN

Japan

UNITED STATES

Denmark

Quebec-CAN

Slovenia

Czech Republic

Norway

Dubai-UAE

Georgia


Tunisia

-20 -10 0 10 20 30 40

15*

17*

12*

14*

14*

11*

24*

22*

24*

12*

28*

32*


Average mathematics scores of 8th-grade students increased from 2007 to 2011 in 10

education systems

37


Korea, Rep. of

Singapore

Chinese Taipei-CHN

Russian Federation

UNITED STATES

Italy

Ukraine

Dubai-UAE

Georgia

Bahrain

Palestinian Nat'l Auth

-20 -10 0 10 20 30 40

16*

18*

11*

27*

1

19*

17*

17*

22*

11*

37*


38


Average mathematics scores of 8th-grade students decreased from 2007 to 2011 in 6

education systems

Hungary

Sweden

Malaysia

Thailand

Jordan

Syrian Arab Republic

-40 -30 -20 -10 0 10 20

-12*

-7*

-34*

-14*

-21*

-15*


TIMSS international mathematics benchmarks

Grade 4 Grade 8

Advanced (625) Students can apply their understanding and knowledge in a variety of relatively complex situations and explain their reasoning.

Students can reason with information, draw conclusions, make generalizations, and solve linear equations.

High (550) Students can apply their knowledge and understanding to solve problems.

Students can apply their understanding and knowledge in a variety of relatively complex situations.

Intermediate (475)

Students can apply basic mathematical knowledge in straightforward situations.

Students can apply basic mathematical knowledge in straightforward situations.

Low (400) Students have some basic mathematical knowledge.

Students have some knowledge of whole numbers and decimals, operations, and basic graphs.

39

Percentages of U.S. 4th-graders reaching TIMSS mathematics benchmarks were

higher than international medians in 2011


10

20

30

40

50

60

70

80

90

100 96

81

47

13

90

69

28

4


Benchmark

Percent at or above

40

NOTE: All U.S. percentages are significantly higher than the corresponding TIMSS international median at the .05 level of statistical significance.




41

Seven systems had higher percentages of 4th-grade students reaching Advanced than the U.S.

Finland

UNITED STATES

Russian Federation

Florida-USA

North Carolina-USA

England-GBR

Northern Ireland-GBR

Japan

Chinese Taipei-CHN

Hong Kong-CHN

Korea, Rep. of

Singapore

0 10 20 30 40 50 60

12

13

13

14

16

18

24

30

34

37

39

43



10

20

30

40

50

60

70

80

90

10092

68

30

7

75

46

17

3


Benchmark

Percent at or above

42


Percentages of U.S. 8th-graders reaching TIMSS mathematics benchmarks were

higher than international medians in 2011


43

Eleven systems had higher percentages of 8th-graders reaching Advanced than the U.S.



California-USADubai-UAE

New ZealandLithuaniaRomania

Quebec-CAN UNITED STATES

Indiana-USATurkey

HungaryColorado-USA

Florida-USAEngland-GBR

AustraliaConnecticut-USA

IsraelMinnesota-USA

North Carolina-USARussian Federation

Massachusetts-USAJapan

Hong Kong-CHNKorea, Rep. of

SingaporeChinese Taipei-CHN

0 10 20 30 40 50 60

555556

777

8888

910

1213

1414

1927

344748

49


SCIENCE

TIMSS 2011 science framework

45

TIMSS science

Content dimensions Grade 4Earth scienceLife sciencePhysical science

Grade 8BiologyChemistryEarth sciencePhysics

Cognitive dimensions

KnowingApplyingReasoning

What is on the TIMSS science assessment?

46

GRADE 4 Percentage of 4th-

graders answering correctly:U.S.: 96%Int’l Avg.: 83%


47

GRADE 4



48

GRADE 8



49

GRADE 8



Higher than U.S.


Lower than U.S.

50

51


Higher than U.S.


Lower than U.S.

52


Higher than U.S.


Lower than U.S.

53


Higher than U.S.


Lower than U.S.

54


Higher than U.S.


Lower than U.S.

55


Higher than U.S.


Lower than U.S.

56


Higher than U.S.


Lower than U.S.

57


Higher than U.S.


Lower than U.S.

58

*p < .05. Change in average scores is significant.NOTE: Education systems ordered according to average science score in 2011.

Average science scores of 4th-grade students increased from 2007 to 2011 in 9

education systemsJapan

UNITED STATES

Czech Republic

Sweden

Netherlands

Denmark

Norway

Georgia


Tunisia

-20 -10 0 10 20 30 40

11*

5

21*

9*

8*

11*

17*

37*

17*

27*


59


Average science scores of 4th-grade students decreased from 2007 to 2011 in 5

education systems

Hong Kong-CHN

England-GBR

Italy

Australia

New Zealand

-30 -20 -10 0 10 20 30

-19*

-13*

-11*

-12*

-7*


60


Average science scores of 8th-grade students increased from 2007 to 2011 in 9

education systems

Singapore

Korea, Rep. of

Minnesota-USA

Russian Federation

UNITED STATES

Quebec-CAN

Ukraine

Norway


Palestinian Nat'l Auth

-20 -10 0 10 20 30 40

23*

7*

15*

13*

5

13*

16*

8*

15*

16*


61


Average science scores of 8th-grade students decreased from 2007 to 2011 in 7

education systems

Hungary

Bahrain

Thailand

Jordan

Malaysia

Syrian Arab Republic

Indonesia

-50 -40 -30 -20 -10 0 10

-17*

-15*

-20*

-33*

-44*

-26*

-21*


TIMSS international science benchmarks

62

Grade 4 Grade 8Advanced (625) Students can apply knowledge

and understanding of scientific processes and relationships and show some knowledge of the process of scientific inquiry.

Students communicate an understanding of complex and abstract concepts in biology, chemistry, physics, and earth science.

High (550) Students apply their knowledge and understanding of the sciences to explain phenomena in everyday and abstract contexts.

Students demonstrate understanding of concepts related to science cycles, systems, and principles.

Intermediate (475)

Students have basic knowledge and understanding of practical situations in the sciences.

Students recognize and apply their understanding of basic scientific knowledge in various contexts.

Low (400) Students have some elementary knowledge of life science and physical science.

Students can recognize some basic facts from the life and physical sciences.


10

20

30

40

50

60

70

80

90

100 96

81

49

15

92

72

32

5


Benchmark

Percent at or above

63


Percentages of U.S. 4th-graders reaching TIMSS science benchmarks were higher

than international medians in 2011


64



Three systems had higher percentages of 4th-grade students reaching Advanced than the U.S.

Singapore

Korea, Rep. of

Finland

Russian Federation

Chinese Taipei-CHN

UNITED STATES

Japan

Florida-USA

Hungary

North Carolina-USA

0 10 20 30 40 50 60

33

29

20

16

15

15

14

14

13

12



10

20

30

40

50

60

70

80

90

10093

73

40

10

79

52

21

4


Benchmark

Percent at or above

65

Percentages of U.S. 8th-graders reaching TIMSS science benchmarks were higher

than international medians in 2011


NOTE: Education systems with lower percentages of students reaching the Advanced benchmark than the percentage of U.S. students reaching the Advanced benchmark are not included in figure. 66



Twelve systems had higher percentages of 8th-graders reaching Advanced than the U.S.

Singapore

Massachusetts-USA

Chinese Taipei-CHN

Korea, Rep. of

Japan

Minnesota-USA

Colorado-USA

Connecticut-USA

Russian Federation

England-GBR

Slovenia

Florida-USA

Finland

North Carolina-USA

Alberta-CAN

Israel

Australia

Indiana-USA

UNITED STATES

Hong Kong-GBR

New Zealand

Hungary

Turkey

0 10 20 30 40 50 60

40

24

24

20

18

16

14

14

14

14

13

13

13

12

12

11

11

10

10

9

9

9

8


Summary of change in average U.S. scores over time

67

PIRLS Reading2006-2011

TIMSSMathematics2007-2011

TIMSS Science2007-2011

TIMSSMathematics2007-2011

TIMSSScience2007-2011

Grade 4

Grade 8

Change over time in U.S. average scores was statistically significant.

Change over time in U.S. average scores was not measurably different.

For more information

PIRLS at NCES:

http://nces.ed.gov/surveys/pirls/

International Data Explorer:

http://nces.ed.gov/surveys/pirls/idepirls/

Contact:

Sheila Thompson

NCES

[email protected]

202-502-7425

68

TIMSSPIRLSTIMSS at NCES:

http://nces.ed.gov/timss/

International Data Explorer:

http://nces.ed.gov/timss/idetimss/

Contact:

Stephen Provasnik

NCES

[email protected]

202-502-7480

http://nces.ed.gov/surveys/pirls/

http://nces.ed.gov/surveys/pirls/idepirls/

mailto:[email protected]

http://nces.ed.gov/timss/

http://nces.ed.gov/timss/idetimss/

mailto:[email protected]

TIMSS Survey

Education

Transcript of TIMSS Survey