Post on 22-Nov-2014
description
On the Horizon
Board of Education PresentationJanuary 9, 2012
Lynn K. McMullin
OBJECTIVES
Introduction
Changes in Testing on the Horizon
Impact
CHANGE is imminent…
FACT: This year’s kindergarteners will never take the CMT!
FACT: SBCAT (?) will soon be as well known to you as CMT
OVERVIEW of Changes
In June 2010, 31 states, including Connecticut, joined to
form the SMARTER Balanced Assessment Consortium (SBAC)
Submitted application in the ‘Race to the Top’ Assessment
Competition
OVERVIEW
The USDE awarded two
‘Comprehensive Assessment
Systems’ grants in September
2010
• One to SBAC• Other to the
Partnership for Assessment of Readiness for College and Careers (PARCC)
PARCC
» Through-course Assessments #1 and #2 ELA and Math; 1 – 3 tasks in a class period
» Through-course Assessment #3 ELA and Math; taken over several sessions or classes
» Through-course Assessment #4 ELA Speaking and Listening; each student presents;
teacher scores» End-of-Year Assessment
On Computer 45 – 60 questions
» Field testing 2012 – 2014 – Operational 2014
Report
scores in 2
weeks
OVERVIEW» CT is a GOVERNING state in SBAC
Computer-Adaptive Testing: CAT
An Adaptive Test is a test that dynamically
adjusts to the trait level of each examinee as the
test is being administered.
• Get it right, it moves you up
• Get it wrong, it moves you down
• Makes continuous adjustments as you work
Computer-Adaptive Testing: CAT
Developed in France around 1905, Alfred Binet’s IQ Test
Is still used in schools today
Is the standard against which IQ tests are compared
Incorporates all the elements of an adaptive test.
A different starting point can be used for each child
It uses an adaptive item-selection procedure.
Scoring method allows a common score to be
obtained from different subsets of items.
Length can vary by child with the use of a variable
termination rule.
Based on a 43,000 calibrated
item bank
Computer-Adaptive Testing: CAT
A starting rule for selecting the first item
A procedure for scoring item responses and
estimating trait level
A method of selecting the next item
A rule for ending the test
Pre-calibrated 43,000 item bank
Computer-Adaptive Testing: CAT
Range of item complexity…
Mike is using cubes that measure ¼ inch on each side to fill a box that has
a height of 5 ¼ inches, width of 3 inches, and length of 2 ½ inches. How many ¼ inch cubes will Mike need to
fill the box?
Mike is using cubes that measure ½ inch on each side to fill a box that has a height of 5
¼ inches, width of 3 inches, and length of 2 ½ inches. How many ½ inch cubes will Mike
need to fill the box?
Mike is using cubes that measure ½ inch on each side to fill a box that has a height
of 5 ½ inches, width of 3 inches, and length of 2 ½ inches. How many ½ inch
cubes will Mike need to fill the box?
ADVANTAGES OF A CAT
CAT equalizes the psychological environment of the test across all ability levels.• High-ability students will get about 50% of the
questions correct.• Low-ability students will get about 50% of the
questions correct.
ADVANTAGES OF A CAT
Efficiency: CATs are more efficient than conventional tests—they generally reduce test length by 50% or more.
Precision: A properly designed CAT can measure all examinees with the same
degree of precision.
Reporting: More accurate placement of students who previously scored
‘advanced’ and ‘below basic.’
ADVANTAGES OF A CAT
Reporting: Results can be made available more quickly (computer-based)
Test Security/Item Exposure: Students are presented with different test items
More Flexibility for Computer Capacity: Students do not need to be assessed on
the same schedule
ADVANTAGES OF A CAT
Strategies will change…
Students cannot change an answer to an item once they have submitted it.
Test prep will need to include this.
Because CAT is dynamic, it can recover from an occasional student
error in answering an item.
Literature shows little or no gain from answer changing.
ITEMS will change…
Animations, simulations, on-line access to information, video or audio
stimulus, moveable modelsTest prep will need to include this.
Elicit a response from the student (e.g., selecting one or more points on a graphic, dragging and dropping a
graphic from one location to another, manipulating a graph)
Innovative items are coming…
Artificial Intelligence (AI)
All constructed response items in the CAT will be AI scored
Items not scored with AI delivered outside of the CAT ‘engine’ (e.g., some elements of
performance tasks)
SBAC will require 10 - 20% read behind to ensure accuracy
AI scoring is nearly 100% reliable
SBAC Testing
Summative Assessment
(CAT)
Mandatory comprehensive assessment in
grades 3–8 and 11
Testing window within
the last 12 weeks of the instructional
year
Selected response,
short constructed response, extended
constructed response,
technology enhanced,
and performance
tasks
Snapshot
Accountability
High-level analyses of tr
ends
District & State
SBAC Testing
Interim Assessment
(CA)Optional content-cluster
assessment
Learning progressions
Available throughout
the year
Selected response,
short constructed response, extended
constructed response,
technology enhanced,
and performance
tasks
Actionable Feedback
Progress
Monitoring
Teacher & School Comparisons
New MODEL
The teacher shares the learning goals with
students and provides opportunities for
students to monitor their ongoing progress.
New Curriculum: Common Core State Standards
Importance of focusing time, energy and
resources on implementing the CCSS beginning this coming school year: Teachers
must read the standards
Mathematics
FOCUS, FOCUS, FOCUS – Deeper understanding of fewer concepts
COHERENCE – One year builds to next
FLUENCY – Standards expect speed and accuracy
DEEP UNDERSTANDING – fewer standards allow for this
APPLICATION – ability to apply what they know
SB…CAT (?) Testing
Spring 2011 – 2012 – Smarter Balanced will pilot SBCAT with some schools’ -- email to Superintendents asking about participation **
Spring 2012 – 2013 – Larger SBCAT pilot
Spring 2013 – 2014 – Every district will be required to pilot a portion of the SBCAT test
CT has applied for a CMT/CAPT moratorium for 2013-2014
Significant changes
12 week window for testing – to meet 1-to-1 computer requirements
End-of-year window – to allow for interim assessments
Less time required; more precise
43,000 test item bank
Significant changes
Innovative test items to match ‘real world’ applications• You wouldn’t really use a protractor on a
computer, rotate it, etc.• You really would click on a word to get its
definition or hear it pronounced
Significant changes
Text Selection
Text Complexity
Range and Quality of Texts
High-Quality Text-Dependent Questions and Tasks
Students’ Ability To Read Complex Texts Independently
Academic Vocabulary
Students encounter appropriately complex texts at
each grade level to develop skills and the conceptual knowledge they need for success in school and life.
Includes extensive work in class with texts that are at or
above grade level.
Includes short tests that require close reading;
50% literary, 50% informational (K – 6)
50% literary non-fiction (7-12)
80 – 90% of all questions should be text dependent questions which require
close reading, vs. skimming.
Words that readers will find in all types of
complex texts from different disciplines.
Now
• Join the limited pilot in 2012, if possible• Examine existing CAT’s
Now
• Study and align to the CCSS• Develop new instructional strategies
Now
• Plan for a 1-to-1 solution by 2015• Define a new set of test-taking strategies
Next Steps ??