Post on 19-Jan-2016
description
Launching PIRLS 2011
Ina V.S. Mullis and Michael O. Martin
1st Meeting of PIRLS 2011 National Research Coordinators
Madrid, Spain
February 20-22, 2008
2
Launching PIRLS 2011Launching PIRLS 2011
2008
Feb 20-22 1st NRC Meeting, Madrid
• Plan upcoming activities
• Begin implementation
– Sampling
– Framework update
– Passage selection
– Other initiatives
3
Launching PIRLS 2011Launching PIRLS 2011
Contexts for Discussion
• Maintaining trend
• Schedule
• Resources
4
** Important Lesson **** Important Lesson **
When measuring change, do not change the measure.
Albert E. Beaton
John W. Tukey
5
** Extension to Important Lesson **** Extension to Important Lesson **
When measuring change, you sometimes have to change the measure because the world is changing.
Ina V.S. Mullis
Michael O. Martin
6
Changing WorldChanging World• Education policies
• Curriculum frameworks
• Societal changes
• Technical changes
• Shifting demographics
• Policies for greater inclusion
• Methodological advances
• Catastrophic events
7
If don’t change the measure to some extent
– May be making changes anyway since the contexts have changed
– Cannot stay at the forefront of providing high-quality measures
– Cannot provide information on topics policymakers and educators find important
8
Resolving Tension Between– Maintaining continuity with the past procedures
– Maintaining current relevance in a changing context
9
Keep Present as Point of Reference
– Link backwards while moving forwards
– Keep substantial portions of assessment constant (e.g., 3 literary and 3 informational passages)
– Introduce new aspects carefully and gradually (e.g., 2 literary and 2 informational passages)
– Plan as trend assessment
10
In Summary, Measuring TrendsIn Summary, Measuring Trends
– Is fundamental to educational improvement
– Is extremely complicated
– Needs to be done with common sense
– Needs to use highest methodological standards
11
Overview of PIRLS 2011 ScheduleOverview of PIRLS 2011 Schedule
2008 - Year of the Framework and Passages
2009 - Year of Instrument Development
2010 - Year of Field Test and Finalizing Instruments – Southern Hemisphere
2011 - Year of Data Collection and Processing
2012 - Year of Data Analysis and Reporting
2013 - Month of the International Database
12
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2008
Apr 30 NRCs finish submitting passages
Jul 30-Aug 1 1st RDG meeting
- Finalize framework
- Review passages (10 + 10)
- Refine passages (6 + 6)
- Review PIRLS 2011 Item and Scoring Guide Development Manual
Aug 30 Framework to printer
13
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2008
Nov 2-7 2nd NRC Meeting, Amsterdam
• Distribute published framework
• Select final field test passages (4 + 4)
• Participate in item writing workshop
• Review background questionnaires
14
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2009
Mar 2-5 2nd RDG meeting to review items and scoring guides for field test
Apr 1-3 1st QDG to review and update background questionnaires
May 31-Jun 5 3rd NRC meeting, Vilnius to review all field test materials
15
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2009
September Translation and layout verification begin
October Within-school sampling for field test
IEA DPRC conducts data entry seminar
16
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2010
Jan 31-Feb 5 4th NRC meeting to conduct constructed-response
scoring training for field test
Feb 15-March 15
*** Conduct Field Test ***Apr 30 All field test data to IEA DPRC
17
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2010
June 3rd RDG to review field test results and recommend passages and items
June 2nd QDG to review field test results and recommend questionnaires
July 18-23 5th NRC meeting to finalize PIRLS 2011 assessment materials
18
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2010
Aug 15 Data collection materials posted on web
October-December
*** Data Collection ***Southern Hemisphere
19
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2011
Feb 13-18 6th NRC meeting to conduct constructed-response
scoring training and finalize Encyclopedia outline, curriculum questionnaire, etc.
March-June
*** Data Collection ***Northern Hemisphere
20
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2011
Jul 30 All data to IEA DPRC
Sep 25-30 7th NRC meeting, Hamburg to review country data and
international report outline and exhibits
2012
Feb 22-24 4th RDG meeting to conduct scale anchoring
21
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2012
Apr 22-27 8th NRC meeting to review draft international report
July Print PIRLS 2011 International Report
Sep 25 Release PIRLS 2011 International Report
22
Upcoming Schedule HighlightsUpcoming Schedule Highlights
2013
Jan 4 Distribute PIRLS 2011 International Database
Jan 20-25 9th NRC meeting to conduct international database
training
Launching PIRLS 2011
Ina V.S. Mullis and Michael O. Martin
1st Meeting of PIRLS 2011 National Research Coordinators
Madrid, Spain
February 20-22, 2008