International comparisons of science and mathematics education : PISA and TIMSS

International comparisons of science and mathematics education: PISA and TIMSS

Harrie Eijkelhof

August 2013 Summerschool

Outline of this session

• Why comparing countries?• PISA

• Mathematics• Science

• TIMSS• Grade 4 & 8• Video study

• Implications of comparing countries• Learning from other countries


Why comparing countries in science and mathematics education?

• Economic importance of science and mathematics education

• Internationalisation: mobility of students• Tool for educational policy: monitoring, reform• Learning from other systems: relative strengths

and weaknesses


Programme for International Student Assessment (PISA)

• Created in 1997 by OECD• Aim: to monitor outcomes of educational

systems in terms of student achievement• Age: 15 years• Surveys every 3 years since 2000 in reading,

mathematical and scientific literacy• 2000 reading, math, science [43]• 2003 reading, math, science [41]• 2006 reading, math, science [57]• 2009 reading, math, science [65]• 2012 reading, math, science [65]• 2015 reading, math, science [?? ]http://www.youtube.com/watch?v=q1I9tuScLUA

http://www.youtube.com/watch?v=q1I9tuScLUA


Focus of PISA

• Collaborative effort: OECD + non-member partner economies

• To measure how well students at age 15 are prepared to meet challenges in future life

• Beyond school-based approach: use of knowledge in everyday tasks and challenges

• Based on model of lifelong learning• Not only assessing knowledge: also how to

apply knowledge in real-life issues


PISA-definition of Mathematical literacy

An individual’s capacity:

• to identify and understand the role that mathematics plays in the world,

• to make well-founded judgements,

• to use and engage with mathematics in ways that meet the needs of that individual’s life as a constructive, concerned and reflective citizen.


PISA-definition of Scientific literacy


PISA 2009 Mathematical Literacy Framework

• Contexts, variety of situations: Personal Occupational Public Scientific

• Mathematical content, 4 overarching ideas: Space and shape Change and relationships Quantity Uncertainty

• Mathematical competencies, 3 clusters: Reproduction Connections Reflection


PISA 2015 Scientific Literacy Framework

Summerschool

Contexts for the PISA 2015 science assessment

Personal Local/National Global

Health Maintenance of health, accidents, nutrition

Control of disease, social transmission, food choices, community health

Epidemics, spread of infectious diseases

Natural Resources Personal consumption of materials and energy

Maintenance of human populations, quality of life, security, production and distribution of food, energy supply

Renewable and non-renewable natural systems, population growth, sustainable use of species

Environmental Quality Environmentally friendly actions, use and disposal of materials and devices

Population distribution, disposal of waste, environmental impact

Biodiversity, ecological sustainability, control of pollution, production and loss of soil/biomass

Hazards Natural and human-induced decisions about housing

Rapid changes [earthquakes, severe weather], slow and progressive changes [coastal erosion, sedimentation], risk assessment

Climate change, impact of modern warfare

Frontiers of Science and Technology Scientific aspects of hobbies, personal technology, music and sporting activities

New materials, devices and processes, genetic modifications, health technology, transport

Extinction of species, exploration of space, origin and structure of the Universe

August 2013


Scientific knowledge

Of science:• Physical systems• Living systems• Earth and space systemsAbout science:• Procedural knowledge: variables, measurement,

replicability, representing data, control of variables

• Epistemic knowledge: purposes, nature, values, justification, errors, collaboration and critique

Attitudes in science (PISA2015)

• Interest in science:e.g. curiosity, willingness to acquire knowledge, career

aspirations

• Valuing scientific approaches to enquiry:e.g. commitment to evidence, criticism

• Environmental awareness:e.g. concern, disposition to sustainable behaviour



Sample PISA-items (released)

• Study 5 sample questions• Look at

• the question type (open/closed)• difficulty

• Give your opinion on the nature and level of the questions: feasible in your country?

PISA-questions MATHEMATICS EXAMPLES

Question 1: Savings Account

1 000 zed is put into a savings account at a bank. There are two choices: one can get an annual rate of 4% OR one can get an immediate 10 zed bonus from the bank, and a 3% annual rate. Which option is better after one year? After two years?



Question 2: SCIENCE TESTS

In Mei Lin’s school, her science teacher gives tests that are marked out of 100. Mei Lin has an average of 60 marks on her first four Science tests. On the fifth test she got 80 marks.What is the average of Mei Lin’s marks in Science after all five tests?

Average: ...............................................


Question 3: EXCHANGE RATEMei-Ling from Singapore was preparing to go to South Africa for 3 months as an exchange student. She needed to change some Singapore dollars (SGD) into South African rand (ZAR).

Question A: EXCHANGE RATE Mei-Ling found out that the exchange rate between Singapore dollars and South African rand was: 1 SGD = 4.2 ZARMei-Ling changed 3000 Singapore dollars into South African rand at this exchange rate.How much money in South African rand did Mei-Ling get?Answer: .................................................

Question B: EXCHANGE RATE On returning to Singapore after 3 months, Mei-Ling had 3 900 ZAR left. Shechanged this back to Singapore dollars, noting that the exchange rate had changedto: 1 SGD = 4.0 ZARHow much money in Singapore dollars did Mei-Ling get?Answer: .................................................

Question C: EXCHANGE RATEDuring these 3 months the exchange rate had changed from 4.2 to 4.0 ZAR per SGD.Was it in Mei-Ling’s favour that the exchange rate now was 4.0 ZAR instead of 4.2ZAR, when she changed her South African rand back to Singapore dollars? Give anexplanation to support your answer.


Question 4: SUPPORT FOR THE PRESIDENT

In Zedland, opinion polls were conducted to find out the level of support for thePresident in the forthcoming election. Four newspaper publishers did separatenationwide polls. The results for the four newspaper polls are shown below:

Newspaper 1: 36.5% (poll conducted on January 6, with a sample of 500 randomlyselected citizens with voting rights)



Newspaper 4: 44.5% (poll conducted on January 20, with 1000 readers phoning in tovote).

Which newspaper’s result is likely to be the best for predicting the level of support forthe President if the election is held on January 25? Give two reasons to support your answer.


Question 5: BOOKSHELVES To complete one set of bookshelves a carpenter needs the following components:4 long wooden panels,6 short wooden panels,12 small clips,2 large clips and14 screws.The carpenter has in stock 26 long wooden panels, 33 short wooden panels, 200 small clips, 20 large clips and 510 screws.How many sets of bookshelves can the carpenter make?

Answer: .................................................

PISA-questions: Science example (1)

TOBACCO SMOKINGTobacco is smoked in cigarettes, cigars and pipes. Research shows that

tobacco related diseases kill nearly 13 500 people worldwide every day. It is predicted that, by 2020, tobacco-related diseases will cause 12% of all

deaths globally.Tobacco smoke contains many harmful substances. The most damaging

substances are tar, nicotine and carbon monoxide.

Question 8: TOBACCO SMOKING Tobacco smoke is inhaled into the lungs. Tar from the smoke is deposited in

the lungs and this prevents the lungs from working properly.Which one of the following is a function of the lungs?A To pump oxygenated blood to all parts of your bodyB To transfer some of the oxygen that you breathe to your bloodC To purify your blood by reducing the carbon dioxide content to zeroD To convert carbon dioxide molecules into oxygen molecules

SEC Advanced Topic 12/13


Question 9: TOBACCO SMOKINGTobacco smoking increases the risk of getting lung cancer

and some other diseases.Is the risk of getting the following diseases increased by

tobacco smoking? Circle “Yes” or “No” in each case.

Is the risk of contracting this disease increased by smoking?

Bronchitis Yes / NoHIV/AIDS Yes / NoChicken pox Yes / No



Question 10: TOBACCO SMOKING Some people use nicotine patches to help them to give up smoking.

The patches are put on skin and release nicotine into the blood. This helps to relieve cravings and withdrawal symptoms when people have stopped smoking.

To study the effectiveness of nicotine patches, a group of 100 smokers who want to give up smoking is chosen randomly. The group is to be studied for six months. The effectiveness of the nicotine patches is to be measured by finding out how many people in the group have not resumed smoking by the end of the study.

Which one of the following is the best experimental design?A All the people in the group wear the patches.B All wear patches except one person who tries to give up

smoking without them.C People choose whether or not they will use patches to help give

up smoking.D Half are randomly chosen to use patches and the other half do

not use them.



Pisa tests in schools

• Selection of schools• Schoolcoordinator appointed• Random selection of 35 15year olds per school• Permission from parents• Tests session organized by test administrator• 13 different booklets with sets of questions• 2 hours for knowledge testing of students• 30 minutes for student questionnaire: personal

background, learning habits, attitudes, motivation• School questionnaire (headmaster): demographic

characteristics, quality of the learning environment

Proficiency levels mathematics PISA

OECD %Level 6 > 669 3.1Level 5 607 - 669 9.6Level 4 545 – 607 18.9Level 3 482 – 545 24.4Level 2 420 – 482 22.0Level 1 358 – 420 14.0Below 1 < 358 8.0



PISA 2009 – cognitive performance

Braz Fin Indo Korea Kyrg. NL Shangh Turkey USA

Sci 405 554 383 538 330 522 575 454 502

Math 386 541 371 546 331 526 600 445 487

Read 412 536 402 539 314 508 556 464 500

U. S. Math Performance inGlobal Perspective (Nov. 2010)E. Hanushek, P. Peterson, L. Woessmann (Harvard University)

“We give special attention to math performance because math appears to be the subject in which accomplishment in secondary school is particularly significant for both an individual’s and a country’s economic well-being.Existing research, though not conclusive, indicates that math skillsbetter predict future earnings and other economic outcomes than other skills learned in high school.”

“This subject is particularly well suited to rigorous comparisons across countries and cultures. There is a fairly clear international consensus on the math concepts and techniques that need to be mastered and on the order in which those conceptsshould be introduced into the curriculum.”


Percentage of students at advanced level in math in countries participating in PISA 2006

allNetherlands 17.6

Germany 13.1 USA 6.0

Turkey 3.6Brazil 0.8

Indonesia 0.2


Percentage of students at advanced level in math in countries participating in PISA 2006 and

some U.S. states

all allNetherlands 17.6 Mass. 11.4

Minn. 10.8Germany 13.1 NY 6.3USA 6.0 Texas 6.2

N. Dakota 4.8Turkey 3.6 Wyoming 3.5Brazil 0.8 Arkansas 3.0

Louisiana 1.7Indonesia 0.2 Mississippi 1.3


Percentage of students at advanced level in math in countries participating in PISA 2006and in U.S. states (all, whites)

all all whiNetherlands 17.6 Mass. 11.4 12.6

Minn. 10.8 12.3Germany 13.1 NY 6.3 7.7USA 6.0 Texas 6.2 10.6

N. Dakota 4.8 5.3Turkey 3.6 Wyoming 3.5 3.8Brazil 0.8 Arkansas 3.0 3.8

Louisiana 1.7 2.6Indonesia 0.2 Mississippi 1.3 2.4


Percentage of students at advanced level in math in countries participating in PISA 2006and in U.S. states (all, whites, college educ. parent)

all all whi c. ed.par.Netherlands 17.6 Mass. 11.4 12.617.1

Minn. 10.8 12.3 15.7Germany 13.1 NY 6.3 7.7 10.0USA 6.0 Texas 6.2 10.6 11.7

N. Dakota 4.8 5.3 6.4Turkey 3.6 Wyoming 3.5 3.8 5.3Brazil 0.8 Arkansas 3.0 3.8 5.4

Louisiana 1.7 2.6 3.0Indonesia 0.2 Mississippi 1.3 2.4 2.2August 2013 Summerschool

U. S. Math Performance inGlobal PerspectiveE. Hanushek, P. Peterson, L. Woessmann (Harvard University)

Conclusion“In sum, the U.S. trails other industrialized countries in bringing its students up to the highest levels ofaccomplishment in mathematics. It is not a story of some states’ high performance being offset by the low performance of other states. Nor is it a story of immigrant or disadvantaged or minority students hiding the good performance of better prepared students. Comparatively small percentages of white students in the states achieve at a high level. And only a small proportion of the children of our college-educated population is equipped to compete with students in a majority of OECD countries.”



How to explain PISA-results?

Give at least five reasons which might explain differences between countries in results.


How to explain PISA-results?

• Cultural factors• Value of knowledge in society• Value of arguing• Science in the media (TV, journals, newspapers)

• Educational factors • Organization of education• Nature of the school curricula• Quality of teaching• Quality of assessment• Investments in education

• PISA-factors• Attitude towards PISA-test 15-year olds• Nature of the PISA-questions


PISA-sources

• http://www.pisa.oecd.org/

• http://www.oecd.org/pisa/pisaproducts/

http://www.pisa.oecd.org/

http://www.oecd.org/pisa/pisaproducts/




Trends in International Mathematics and Science Study (TIMSS)

• Started in 1995 by IEA (International Association for the Evaluation of Educational Achievement)

• Aim: to measure math and science knowledge and skills aligned with curricula in participating countries

• Age: grades 4, 8, 12• Surveys every 4 years in many countries:

Grade 4 Grade 8Grade 12

• 1995 26 41 5• 1999 -- 38 --• 2003 25 46 --• 2007 36 48 10 (2008)

• 2011 50 42 --http://timss.bc.edu/timss2011/index.html

http://timss.bc.edu/timss2011/index.html


TIMSS Frameworks and tests

• Mathematics • Grade 4

Content domains: number, geometric shapes and measures, data display

• Grade 8 Content domains: number, algebra, geometry, data and chance

• Science• Grade 4

Content domains: life science, physical science, earth science

• Grade 8 Content domains: biology, chemistry, physics, earth science

• Mathematics + Science Cognitive domains: knowing, applying, reasoning

Content and Cognitive Domains Grade 4

Science

• Content Domains 45% Life Science35% Physical Science20% Earth Science

• Cognitive Domains 40% Knowing40% Applying20% Reasoning

Mathematics

• Content Domains 50% Number35% Geometric Shapes and

Measures15% Data Display

• Cognitive Domains 40% Knowing40% Applying20% Reasoning


Sample of released math items TIMSS grade 4

1. In a car park, 762 cars were parked in 6 equal rows. How many cars were in each row?

2. Subtract 5.3 – 3.8

3. The temperature at 7 a.m. one morning was 12º. It increased by 2ºC every hour until it reached 20ºC at 11 a.m. What was the temperature at 9 a.m.?a. 14ºC b. 15ºC c. 16ºC d. 17ºC

4. Last year there were 92 boys and 83 girls in Fairmont School. This year there are 210 students, and 97 boys. How many more girls are there this year than last year? Show your work.


Sample of released math items TIMSS grade 8


Sample of released science items TIMSS grade 4

• What do birds, bats and butterflies have in common?a. feathersb. hairc. internal skeletond. wings

• Write down one form of energy Earth receives from the sun

• Which of these soil changes is due only to natural causes:a. loss of minerals due to farmingb. deserts forming due to tree cuttingc. flooding due to dam constructiond. minerals washing out due to heavy rain


Released science item TIMSS grade 4


Sample of released science items TIMSS grade 8


Released science item TIMSS Grade 8


TIMSS Results 2011Grades 4 & 8 Math & Science

Grade 4 (N=50)Math Science

Singapore 606 (1) 583 (2)Korea 605 (2) 587 (1)Hong Kong 602 ( 3) 535 (9)Chinese Taipei 591 (4) 552 (6)Japan 585 (5) 559 (4)Belgium (Flemish) 549 (7) 509 (27)Finland 545 (8) 570 (3)England 542 (9) 529 (15)Russian Feder. 542 (10) 552 (5)USA 541 (11) 544 (7)Netherlands 540 (12) 531 (14)Germany 528 (16) 528 (17)Norway 495 (29) 494 (33)Tunisia 359 (47) 346 (48)Morocco 335 (49) 264 (49)Yemen 248 (50) 209 (50)No significant difference with NL

Grade 8 (N=42)Math Science

Korea 613 (1) 587(3)Singapore 611 (2) 590 (1)Chinese Taipei 609 (3) 564 (2)Japan 570 (5) 558 (4)Russian Feder. 539 (6) 542 (7)Finland 514 (8) 552 (5)USA 509 (9) 525 (10)England 507 (10) 533 (9)Norway 475 (20) 494 (19)Tunisia 425 (32) 439 (29)Qatar 410 (33) 419 (37)Indonesia 386 (38) 406 (40)Morocco 371 (40) 376 (41)Ghana 331 (42) 306 (42)

Botswana (G9) 397 404South Africa (G9) 352 332


Trends Between 1995 and 2011 TIMSS Grade 4 Mathematics

Countries Improving• Australia• England• Hong Kong SAR• Iran• Japan• Korea• New Zealand• Norway• Portugal• Singapore• Slovenia• United States

Countries Declining• Austria• Czech Republic• Netherlands


Trends Between 1995 and 2011 TIMSS Grade 4 Science

Countries Improving

• Hong Kong SAR• Hungary• Iran• Japan• Korea• Portugal• Singapore• Slovenia

Countries Declining

• Norway


Progress in USA at local levels

Minnesota:http://www.youtube.com/watch?v=r4Xbej4Ncow

Michigan:http://www.youtube.com/watch?v=AtGlPOp-96M&playnext=1&list=PLF81F896453A0BC40&feature=results_video


http://www.youtube.com/watch?v=r4Xbej4Ncow

http://www.youtube.com/watch?v=r4Xbej4Ncow

http://www.youtube.com/watch?v=AtGlPOp-96M&playnext=1&list=PLF81F896453A0BC40&feature=results_video




TIMSS 1999 Video Study Mathematics

• http://timssvideo.com/

http://timssvideo.com/


TIMSS 1999 Video Study Mathematics

Country N math lessonseighth grade

N schools

Australia 87 85

Czech 100 100

Hong Kong 100 100

Japan 50 100

Netherlands 78 85

Switzerland 140 93

USA 83 76


TIMSS 1999 Video Study Mathematics:features

• Purpose• Reviewing previous lesson• Introducing new content• Practicing/applying this new content

• Classroom interaction• Public: dialogue directed by teacher or students• Private: students work at seats

• Content activity• Independent problem: class working on single problem• Concurrent problems: class working on set of problems• Non-problem: mathematical information


TIMSS video 1999: Dutch lesson signature

• Little convergence in purpose dimension: Dutch teachers differ in time allocation between purposes

• Covergence:• Public interaction until middle of the lessons• Rest of the lessons private interaction• Content activity

First third: independent problems Rest of the lessons: concurrent problems


TIMSS video 1999: Japanese lesson signature

• Strong convergence on all of the points• Review: first 10% of the lessons• Throughout lessons: introducing new content• Throughout lessons: mainly public and some

private interaction• Throughout lessons: working on independent

problems

8 -1-2013 SEC Advanced topic

TIMSS video: educators’ judgements of Dutch math lessons

• Teacher’s role: not dominating, guiding role, friendly, remains close to the textbook

• Students’ role: active, some groups, low expectations, help from classmates or teacher

• Content: not demanding, textbook dominates, structured scheme, focus on rules and results, superficial, no development of higher order thinking

• Climate: casual between teacher and students, informal, laissez faire, very permissive, relaxed, lack of discipline, high noise level, chaotic

Policy Implications of TIMSS & PISA

• Assessment of problems/successes• Looking abroad for solutions• Focus on assessment• Developing new curricula• Policy shifts, e.g.:

• focus on excellence• investing in backward regions or populations

• Acting on comparisons at system level


The Learning CurveLessons in country performance in

educationPearson,2012

http://thelearningcurve.pearson.com/videos





The Learning CurveLessons in country performance in

educationPearson,2012• 40 countries compared

• Index cognitive skills & educational attainment• Top-countries Finland and South Korea• NL in subtop (7)

• www.thelearningcurve.pearson.com


http://www.thelearningcurve.pearson.com/

School responsibility & autonomy compared


Country profiles


SouthKorea

Finland Nether-lands

Indonesia

TIMSS Grade 4 561 585 529 392PISA Grade 8 541 543 519 385HE-graduation 63% 49% 42% 13%% education in GNP 16% 12% 12% 17%Teachers’s salaries compared to mean

1,6 0,9 1,3 0,8

Global index of cognitive skills and educational attainment


Comparing educational systems

South Korea

• exam driven• rigid curriculum• reproduction oriented• frontal teaching• private crammer schools

(hagwans)• large classes• rel. high salaries teachers• importance of teacher

training• learning seen as moral

duty to family and society

Finland

• late start in schooling• less lessons• no homework• only some frontal teaching• focus on applying

knowledge• small classes• rel. low salaries teachers• importance of teacher

training• education seen as act of

social justice



Strong Performers and Successful Reformers in Education

A video series profiling policies and practices of education systems that demonstrate high or improving performance in the PISA tests.

Belgium (Flanders), Brazil, Canada (Ontario), Finland, Germany, Japan, Korea, Netherlands, Poland, Portugal, Shanghai, Singapore

http://www.pearsonfoundation.org/oecd/

http://www.pearsonfoundation.org/oecd/

International comparisons of science and mathematics education : PISA and TIMSS

Documents

Transcript of International comparisons of science and mathematics education : PISA and TIMSS