©2016 Northwest Evaluation Association. PISA-based … · ©2016 Northwest Evaluation Association....
Transcript of ©2016 Northwest Evaluation Association. PISA-based … · ©2016 Northwest Evaluation Association....
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
PISA-based OECD Test for Schools: A Blueprint
for School Improvement
Presented by
Dr. Terri L. Breeden, Shaker Heights City SD
and
Rob Johnson, NWEA Regional Manager
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Thoughtful Questions to Consider about our Schools
OECD: Organisation for Economic CO-operation and Development
PISA
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Know how students feel about their own learning, their confidence for learning and how their belief systems have a
direct correlation to their performance?
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Know where your school fits nationally and internationally
regarding students’ processing, analyzing and thinking?
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
PISA in brief5
• Over half a million students…
– representing 28 million 15-year-olds in 72 countries/economies
… took an internationally agreed upon 2-hour test…
– Goes beyond testing whether students can reproduce what they were taught…
– Mathematics, reading, science, problem-solving, financial literacy
… and responded to a 30 minute questionnaire on…
– their personal background, their schools and their engagement with learning and school
• Parents, principals and system leaders provided data on…
– school policies, practices, resources and institutional factors that help
explain performance differences .
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
The structure of the PISA assessment
6
2000 2003 2006 2009 2012 2015
Reading Reading Reading Reading Reading Reading
Mathematics Mathematics Mathematics Mathematics Mathematics Mathematics
Science Science Science Science Science Science
Problem Solving Digital ReadingProblemSolving
Collaborative Problem Solving
PISA helps Countries Learn
The tests are designed to assess to what extent students at the end of compulsory education, can apply their knowledge to real-life situations and be equipped for full participation in society.
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
PISA-based OECD Test for Schools
• Requested by schools
• Spearheaded by America Achieves
• Developed with grant funds
• 2012 Paper/Pencil pilot started
• Taken by over 400 US schools
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Includes International PISA PISA-Based OECD Test for Schools
Utilizes the common PISA scale
Administration Country level; online via thumb drive School-level &Online
Sample size at each school 35 students No more than 85 students
Test intervals Every three years On-demand
Includes student survey questions on
engagement, school culture and
climate
Country level comparable data to
“Participating PISA countries
comparison data”
School level reporting None 159-Page reports for schools
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
5048
53 > 20 schools
10-19 schools
1-9 schools
31
34
8
1
2
12
21
1
16
5
10 206
6
3
48
14
40
5
10
12
5
15
11
1
2
2
OECD Test for Schools (based on PISA)Participation Through 2015 In The United States
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
LEADING
COMPANY
Each School Receives Actionable Information
• 159 page report for each school
• Detailed analyses of how your school(s) compare on
multiple measures- math, reading, science
• Not just scores, but context including climate and
culture
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Overview of Results Report
Student Engagement and the Learning Environment at Your School
Disciplinary climate
Teacher/student relations
Students’ reading habits
Instrumental motivationSelf-efficacy
Your School Compared With Similar Schools in Your Country
Your School’s Results in an International Context
14
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Your school
US average
Below International Baseline
Medium-High Performers
Top Performers Internationally
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Are my students critical thinkers? Are they able to conceptualize, generalize, utilize and link information sources and flexibly translate between them?
OECD VS Consortia Proficiency Levels
Comparison.xlsx
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
OECD Sample Test Item
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Sample Question: Science
Reverse Osmosis
One way of producing drinking water from seawater is by reverse osmosis. Reverse osmosis is a type of filtration. Seawater is pushed through a semi-permeable membrane. Pressure is applied to the seawater. Semi-permeable means salt is trapped on one side of the membrane, but water can pass through. The trapped salts form a ‘seawater concentrate’ on one side of the membrane.
QuestionWhere do the trapped salts go?
A. Ocean
B. Storage tank
C. Delivery pipeline
D.Water distribution system
See more sample questions at: www.oecd.org/pisa/pisa-basedtestforschools.
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Why do schools care?Evolution of employment in occupational groups defined by PIAAC problem-solving skills
-20
-15
-10
-5
0
5
10
15
20
25
%
Employment of workers with advanced
problem-solving skills
Employment of workers with poor problem-solving skills
Employment of workers with medium-low problem-solving skills (PIAAC)
Source:PIAAC 2011
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
How We Compare to our Peers Nationally
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Exploring Reader ProfilesHow is your reading Instruction addressing the deep and wide?
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
What types of readers are my
students?
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Comparison of
Reader Profiles
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
What Students Say…
The test was simple, and
the navigating through
them was easy. The test
also had some interesting
articles and facts I
enjoyed reading
It challenged my mind to solve real-world problems that I felt like were relevant.
Some questions allowed for a broad typed out open-ended answer. This allowed me to better portray my thoughts through each answer.
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
How can we learn from other schools?
• Access to the Global Learning Network
• Facilitated by America Achieves
• Participate in Virtual, Regional and National Convenings
• Engage in peer-to-peer learning
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®globallearningnetwork.americaachieves.org
GOALS FOR THE CONVENING OF
WORLD-LEADING SCHOOLS
Participants identify concrete
actions that will further this
work in their schools and
districts
Create a sense of community among high-
achieving and committed
individuals to establish a
peer network
Improve participants’
understandingof their OECD
Test for Schools results
Build understanding
of how this work fits in with other efforts to improve student
outcomes
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Let’s hear from a school
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
Singapore
Hong Kong-ChinaChinese Taipei
Korea
Macao-China
Japan Liechtenstein
Switzerland
Netherlands
Estonia Finland
CanadaPoland
Belgium
Germany Viet Nam
AustriaAustralia
IrelandSlovenia
DenmarkNew Zealand
Czech RepublicFrance
United Kingdom
IcelandLatvia
Luxembourg Norway
Portugal ItalySpain
Russian Fed.Slovak Republic United StatesLithuania
SwedenHungary
Croatia
Israel
Greece
Serbia Turkey
Romania
Bulgaria
U.A.E.
Kazakhstan
Thailand
ChileMalaysia
Mexico
410
420
430
440
450
460
470
480
490
500
510
520
530
540
550
560
570
580
Mean score
High mathematics performance
Low mathematics performance
… Shanghai-China performs above this line (613)
… 12 countries perform below this line
Average performance
of 15-year-olds in
MathematicsFig I.2.13
US
Massachusetts
Connecticut
Florida
26% of American 15-year-olds do not reach PISA
Level 2 (OECD average 23%, Shanghai 4%, Japan 11%, Canada 14%, Some estimate long-term
economic cost to be US$72 trillion )
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
• North Star Academy, Newark NJ
• Math & Science below average
• Student interest in math & science below average
• Revamped curriculum
• One year later: >50% of seniors planned to major in STEM
Are my students engaged and motivated?
http://hepg.org/hel-home/issues/30_3/helarticle/quietly,-a-new-test-gains-
advocates
©2016 Northwest Evaluation Association. NWEA.org | Partnering to help all kids learn®
What Students Say…
The test was simple, and
the navigating through
them was easy. The test
also had some interesting
articles and facts I
enjoyed reading
It challenged my mind to solve real-world problems that I felt like were relevant.
Some questions allowed for a broad typed out open-ended answer. This allowed me to better portray my thoughts through each answer.