TOP 10 Things You Should Know About Iowa Assessments Comprehension Test!
A New Generation of Assessments: 3 Things You Need to Know
-
Upload
catapultlearn -
Category
Education
-
view
188 -
download
1
description
Transcript of A New Generation of Assessments: 3 Things You Need to Know
A New Generation of Assessments Three Things You Need to Know
October 30, 2013
Sean McGrew, Ph.D.
Poll 1: What is this?
A: A tool for chopping wood
B: A murder weapon
Three Things to Know
1. Everything depends on use of assessments
2. Assessment distinctions have fuzzy borders
3. Computer-based testing is (finally) changing the practice
Validity in the Context of Use
“Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests. Validity is, therefore, the most fundamental consideration in developing and evaluating tests. The process of validation involves accumulating evidence to provide a sound scientic basis for the proposed score interpretations. It is the interpretations of test scores required by proposed uses that are evaluated, not the test itself.” AERA/APA/NCME Standards for Educational and Psychological Testing, 1999.
Reliability
Reliability “refers to the consistency of . . . measurements when the testing procedure is repeated on a population of individuals or groups” (AERA, 1999)
Poll 2: What’s the most important attribute?
What’s the most important attribute of an educational assessment?
Utility
Validity
Reliability
It depends
Fuzzy Borders
Formative vs. Summative
Norm-Referenced vs. Criterion-Referenced
Interim/Benchmark/Formative
Test/Assessment
Poll 3: OK: Of a formative or diagnostic?
What’s the most important attribute of an diagnostic of formative assessment used to diagnose skill deficits and/or inform instruction?
Utility
Validity
Reliability
Poll 4: Of a high-stakes test?
What’s the most important attribute of a high-stakes test used to rank or rate students?
Utility
Validity
Reliability
Poll 5: Reporting Categories Important to You
Poll 6: Scores
Poll 7: Logistics
Computer-Based Coming of Age
PROS
Easy to edit/update
Rapid scoring and reporting
Possibility of multimedia content
Possibility of adaptive testing
CONS
Hardware requirements
Often need internet access
Assumes comfort level with computers
Harder to ensure same experience for all students
Less proven metrics for reliability (if get away from multi-choice)
Not all Computer-Based tests are Adaptive
Adaptive Tests
Basic idea goes back to Binet’s 1905 intelligence tests
First done by computers in 1970s and 1980s
Widespread by late 1990s
PROS
Shorter
More accurate for wider range of students
CONS
Harder to know ‘what’s on the test’
Harder to review item and standard results for a group of students
Common Core Test Consortia
Smarter Balanced: Computer-Based ADAPTIVE
PARCC: Computer-Based, NOT ADAPTIVE
Next-Generation Reporting
Focus on Longitudinal Analysis
Teachers expected to incorporate results provided online
GROWTH
INTERIM TEST REPORTING TOOL Catapult Learning’s Common Core
21
22
23
Grade 3 Assessment Blueprint
24
Standard Section
Number
of Items
2.OA.A.1: Use addition and subtraction within 100 to solve
one- and two-step word problems. Pre-req 4
2.OA.C.4: Use addition to find the total number of objects
arranged in rectangular arrays with up to 5 rows and 5
columns. Pre-req 4
2.OA.B.2: Fluently add and subtract within 20 using mental
strategies Pre-req 4
3.OA.A.2: Interpret whole-number quotients of whole
numbers Unit 1: A,B 4
3.OA.A.3: Use multiplication and division within 100 to solve
word problems in situations involving equal groups, arrays,
and measurement quantities. Unit 1: A,B 4
3.OA.C.7: Fluently multiply and divide within 100 Unit 1: A,B 8
Total 28 items
To teach, or re-teach?
To teach, or else re-teach, that is the question:
Whether ’tis Nobler in the mind to suffer
The Slings and Arrows of outraged Bosses,
Or to take Arms against student weaknesses,
And by opposing end them: Re-teach, press on
No more; and by Re-teach, to say we end
The Heart-ache, and the thousand Natural shocks
Common Core enforces. ’Tis a consummation
Devoutly to be wished. Re-teach, press on,
Press on, perchance to fail; Aye, there’s the rub.
Adapted from William Shakespeare, Hamlet III.i
Connect with Us!
Stay connected with our latest updates, blog posts, upcoming webinars, and more.
Catapult Learning on Twitter @catapultlearn
Dr. Sean McGrew on Twitter @SeanPMGrew
Share your successes! If you put our methods into action, let us know about it. Use the hashtag #catapulting
26