Assessing Student Achievement with PARCC The New Jersey ......Assessing Student Achievement with...
Transcript of Assessing Student Achievement with PARCC The New Jersey ......Assessing Student Achievement with...
Assessing Student Achievement with PARCC The New Jersey Leadership Summit
Jeffrey Nellhaus
Director of Policy, Research and Design
PARCC, Inc.
April 3, 2014
• The Consortium
• Goals and Priorities
• Assessment System Design
• Reporting Schedule and Reporting Metrics
• Technology
• Field Test
• Practice Tests
• Rollout
Presentation Topics
2
The PARCC Consortium
• Led by member state Chief State School Officers with significant support from SEA and LEA staff
• PARCC, Inc., Project
Management Partner • Consultants • Contractors
3
Develop tests
worth taking
Texts worth reading
Essays worth writing
Problems worth solving
Accessible to all students
Goals …
4
Report results worth using
Valid Reliable Timely
Actionable
Determine whether students are college- and career- ready or “on track”
Provide tools to assess student learning and support instruction during the
school year
Generate data, including growth metrics, for accountability uses
Report comparable data across schools,
districts and member states
Priorities
Use technology to create efficiencies
and engage students in the testing process
5
Design of the Assessment System
6
Formative Tools For use during the school year
• Grades 2-8
• Reading, Writing, Math
• Computer adaptive
• Designed to pinpoint students’ learning needs
• Links to interventions/enrichments
Diagnostic Assessments Mid-Year/Interim Assessments
• Grades 3-11
• ELA/Literacy and Math
• Computer- and paper-based
• Built from released PBA tasks
• Can be used for assessment at individual, classroom, school levels
K-1 Tools Speaking & Listening Tools • Grades K-1 • Reading and math • Checklists, running records,
performance tasks • Links to interventions/enrichments
• Grades 3-12 • Performance-based activities • Spontaneous oral response to oral
prompt; share findings of research in an oral presentation
7
Summative Assessments Measure and report achievement and growth
ELA/Literacy
Writing essays drawing evidence from sources, including multi-media, some comprehension
Math
Solving multi-step problems that require reasoning and address real world situations
ELA/Literacy
Demonstrating comprehension of literary and informational texts
Math
Demonstrating understanding of concepts and procedures and carrying out short applications
Performance-Based Component (PBA)
End-of-Year Component (EOY)
Overall Score = Combination of PBA + EOY 8
• Other Resources to Support Instruction
– Annual release of test items and benchmarked student work
– Guidance for using formative tools and interpreting results of summative assessments
• Test Design
– End-of-course math tests: Alg. I, Geo., and Alg. II
– ELA/L tests in each of the grades 9, 10, and 11
– Tests for high schools with block schedules
• Accessibility
– Computer-delivered accommodations for SWDs & ELs
– Translations of math assessments for ELs
Additional Features
9
PARCC’s Accessibility Features
10
Accessibility Features for All
Students
Accessibility Features
Identified in advance
Accommodations
Selected features
highlighter answer masking
magnification line reader …
color contrast
answer masking, math text-to-speech …
ELA text-to-speech
scribe extended time …
• The goal is to return results of the summative assessments prior to the end of the school year
• Standard setting will occur in
Summer 2015, with year 1 results
reported in September
• Reporting schedule for year 2 is
under discussion
Reporting Schedule
Claim Sub-Claim Performance Level
Scale Score Sub-Score
ELA/L X X
Reading X
Reading Literary Text
X
Reading Informational Text
X
Vocabulary X
Writing X
Written Expression X
Knowledge of Conventions
X
English Language Arts & Literacy Claims and Reporting Metrics
12
Claim Sub-Claim Performance Level
Scale Score Sub-Score
Math X X
Major Content X
Additional & Supporting Content
X
Expressing Mathematical Reasoning
X
Modeling and Applications
X
Mathematics Claims and Reporting Metrics
13
• PARCC will report results of summative assessments using 5 performance levels
• Allows for finer classifications of student performance and supports reporting of improvement and growth
• Performance level descriptors at www.parcconline.org/plds
• Standard setting event will occur in summer 2015
• K-12 and HE educators will serve on standard-setting panels
Performance Levels
14
Technology Number and Types of Devices
• Rule of thumb: In a school with 3 tested grades, at least one computer for every 2 students in the largest tested grade
• Desktop, laptop, netbook, and thin client/VDI computers (Windows, Mac, Chromebook), and tablets
• All devices must meet the established hardware, operating system, and networking specifications http://www.parcconline.org/technology
15
• Technology purchases should consider instructional needs, not only assessment
• Proctor caching allows schools with limited bandwidth to administer PARCC’s computer-based assessments by reducing the amount of data being transmitted over the internet during testing
• Caching involves downloading the test content to a local server prior to testing, then distributing the test to students through the school’s internal network.
• Caching will be available to all PARCC schools as part of the Technology Platform (TestNav 8) and only requires 5 kbps per simultaneous test-taker.
Proctor Caching: A Solution for Schools with Limited Bandwidth
16
• Not designed to yield individual student, school, or district results
• Also an opportunity for a number of important research studies
Field Tests – Testing the Test
The field tests are designed to evaluate the quality of the test questions, delivery platform and administration protocols.
17
Study Brief Description Participating States
1. Mode Comparability Can paper- and computer-based assessments be reported on the same scale?
ALL
2. Device Comparability Are assessment results of tablet and desktop/laptop administrations comparable?
MA, AR, CO
3. Quality of Items and Tasks Do the items measure what was intended to be measured? Do any items show bias? Was human scoring reliable?
ALL
4. Text-to-Speech Validity Does the text-to-speech accommodation provide desired differential boost to those who need it?
MD
5. High School Math Comparability Can traditional and integrated EOC assessments be reported on the same scale?
ALL
6. Quality of Test Administration
Do test administrators understand administration protocols? Do students understand test directions?
Surveys in ALL Site observations in MD, TN, OH, NM, IL
7. Feasibility of International Benchmarking
What points on PARCC scale correspond to benchmarks from PISA, PIRLS and TIMSS?
ALL
8. Psychometric Studies Can assessment results be put on a vertical scale? What is the best way to combine results from the PBA and EOY?
ALL
Field Test Research Studies
18
Schools in 14 States & DC are Participating in the Field Tests
19
• Over 1 million students are participating
• Nearly 16,000 schools
• In most cases, 1-3 grades per school, 2 classrooms per grade
• Most student take one component in one content area
• https://pearsonstsprod.service-now.com/parcc/
• With the exception of a few glitches, the delivery platform is functioning well
• Most students are able to use the platform, especially those who practiced first with sample questions and tutorials
• Schools that conducted a “dress rehearsal” have had fewer issues
• Social media is a factor: on the one hand, immediate feedback; on the other, threat to security
• Some directions need to be clarified: responding to essay questions, closing out of test sessions
20
Field Tests Already Providing Great Feedback
PARCC will get additional feedback from test
administrator and student questionnaires
– Practice tests, sample items and navigation tutorial now available
http://practice.parcc.testnav.com/#
Practice Tests and Tutorial Helping Students Get Ready
21
Next Steps For PARCC
22
SEPTEMBER
States launch PARCC
SUMMER
Model Content Frameworks
Released
AUGUST Item
Prototypes Released
APRIL Test
Blueprints released
We are here!
WINTER/SPRING
Field Test/Practice Test Online
SPRING
First Administration
of New Tests
SUMMER
Establishment of Cut Scores
FALL
Release of Diagnostic and
Formative Assessments
2010 2011
2012 2013
2014 2015
FALL
Use of Cut Scores for IHE
Placement
2016
Next year
APRIL PARCC
Becomes Independent Non-Profit
SUMMER
PARCC Reports Research From
Field Test Results
• NJ has made great strides transitioning to the CCSS over past several years
• Stay the course. Getting kids ready for college and careers is the right thing.
• PARCC states are committed to developing a high quality assessment system
• We invite your feedback and will strive to improve the system each year
• It’s about tests worth taking, reports results worth using!
Final Comments
23
Thank You!
Please visit PARCC online at: www.parcconline.org
Or follow us on Twitter at:
www.twitter.com/PARCCPlace
24