Post on 27-Mar-2015
Understanding Assessment
Mark LealDirector of AssessmentAlaska Department of Education and Early Developmentmark_leal@eed.state.ak.usOffice phone number- 465-8691
STANDARDS-BASED REFORMSTANDARDS-BASED REFORM
Content and Performance Content and Performance
StandardsStandardsStateState clear expectations clear expectations
Standards-based Standards-based teststestsTest Test StandardsStandards
Adjust instructionAdjust instruction to achieve standards
RemediationRemediationstandards-based instruction
Standards-based Standards-based
instructioninstructionTeach Teach StandardsStandards
EXPECTATIONS FOR STANDARDS-BASEDASSESSMENT SYSTEMS
Rigorous Rigorous standardsstandards
TestsTests based on and aligned with based on and aligned with standardsstandards
Fair, valid, reliable Fair, valid, reliable teststests and and assessment assessment systemsystem
Systems that include Systems that include ALLALL students students
Results reported to public/stakeholdersResults reported to public/stakeholders
STANDARDS-BASEDSTANDARDS-BASEDASSESSMENT ASSESSMENT SYSTEMSSYSTEMS
Individual studentsIndividual students
reportreportresultsresults
forfor
comparecompareresultsresults
by subgroup forby subgroup for
StatewideStatewide
DistrictsDistricts
SchoolsSchoolsDisability status
English proficiency
Low-income status
Migrant status
Gender/Race/Ethnicity
USING ASSESSMENT ASSESSMENT SYSTEMSSYSTEMS TO MAKE DECISIONS
ASSESSMENT SYSTEMS(Possible Components)
Norm-referenced testsNorm-referenced tests
compares student performance to norm group
Standards-based testsStandards-based tests
compares student performance to standard
Blended assessment systemsBlended assessment systems
combines norm-referenced and standards-based assessments
STANDARDS-BASED TESTSSTANDARDS-BASED TESTS
Also known as criterion referenced testsAlso known as criterion referenced tests compares performance to standards or criteriacompares performance to standards or criteria
Sixmile Creek is a comfortable distance from Anchorage. However it is not a comfortable river. Sixmile has similar character to many Washington and Oregon whitewater runs: lush forest, deep canyons, and tremendous rapids. The river is easily accessible from Seward Highway. Sixmile Creek has three distinct sections. The Upper Canyon rates a Class IV. The toughest stretch is the lower canyon, where the river rampages along at a constant Class V for two killer miles.
The Wildest One!Difficulty: Class V Season: May thru September Trip Length: Half-day to one day
Whitewater Classifications
Most experts use a I to VI rating system for whitewater difficulty. Here’s a quick rundown of what to expect from each classification:
I- Small rapids with low waves and no obstructions
II- More frequent rapids with few or no obstructions
III- Waves up to four feet, with some maneuvering around obstructions
IV- Very Difficult, extended rapids. Scouting is necessary.
V- Long and violent rapids. Large waves that are unavoidable. Complex course. Scouting is a must.
VI- Maximum difficulty. Involves serious risk of life.
TRIAL USFSA JUDGING FORMAdult Gold Free Skating TRIAL
Candidates’ Name _________________________________________USFSA # ____________________________
OFFICIAL JUDGES
TRIAL COMMENTS
Technical Merit
Presentation
Total Pass Retry Judge-on charge overwrite total in ink
Judge-in-Charge Signature_____________________________________________________ USFSA # ______________________Duration: Not to exceed 2 minutes, 40 seconds to the music of the skater’s choicePassing Average 3.0Candidate must give a strong performance and demonstrate a command of the following required elements showing very good form and carriage, strong true edges, effortless flow and preciseness to steps.JUMPS: ____Axel
____Lutz____Two different additional single jumps
________________________________
SPINS: ____Two different solo spins (minimum of 4 revolutions)
____One of the following: Split, Stag,
Host Club___________________________________________Date _______________________________
Falling Leaf, or Half Loop____One jump combination consisting of two
(2) different single jumps (no turnor change of foot between jumps)
________________ ________________
____One combination spin with at least one change offoot (minimum 4 revolutions in position)
STEPS: ____One step sequence – either straight line or circular) ____Connecting moves consisting of spirals, strong edges
good use of music and full utilization of ice surface
Extra elements may be added without penalty. Only two of the required elements may be retried, if necessaryTrial Judge’s Signature _______________________________________USFSA # _________________________Trial Judge’s Club ___________________________________Current Judging Status_______________________
SkaterTechnical
Merit Mark
Presentation Mark
Total of Marks
Ordinal Value
Amber 2.7 2.7 5.4 2
Jill 2.8 2.7 5.5 1
Tori 2.5 2.6 5.1 3
Haley 2.3 2.3 4.6 5
Nikki 2.4 2.4 4.8 4
STANDARDS-BASED TESTSSTANDARDS-BASED TESTScompares student performance to standardscompares student performance to standards
Results report levels of proficiency Advanced or exemplaryexemplary Proficient or masterymastery Below proficient or basicbasic Not-proficient or novicenovice
Items look more like classroom work
Multiple-choice Open-ended Extended Constructed-response and essay Short Constructed-response
StandardsStandards-Based TestsUSEUSE
Monitor mastery of Monitor mastery of standardsstandardsMeasure performanceMeasure performanceSample depth of Sample depth of student knowledge & student knowledge & skillsskillsAccurate with validity Accurate with validity & reliability& reliability
MISUSEMISUSEAs sole indicator of As sole indicator of learninglearning
As stand-alone As stand-alone evidence of a evidence of a school’s successschool’s success
As accurate without As accurate without validity & reliabilityvalidity & reliability
1115
75
66
1419
0 1
0102030405060708090
100
Novice NeedsImprovement
Proficient Advanced
District
State
NORM-REFERENCED NORM-REFERENCED TESTSTESTS
compares performance (data) to norm group.
Reference Population: Standard Normal Curve
95th5th
50th Percentile
Body Mass Index- Norm Referenced Measure
Boys: 2 to 20 years
BMI BMI
BMI BMI
Indicators of Nutritional StatusIndicators of Nutritional Status
<5th percentileStunting/shortness length or height-for-age
Head circumference-for-age
<5th percentile
<5th percentile>95th percentile
Underweight weight-for-length BMI-for-age
NORM-REFERENCED TESTSNORM-REFERENCED TESTS compares student performance to norm group
Results are reported asResults are reported as Standard scoreStandard score
Percentile scorePercentile score
Grade equivalentGrade equivalent
Half performed betterHalf performed better
Half performed worseHalf performed worse
Referenced to 50Referenced to 50thth %ile %ile
Norm-Referenced TestsUSE
To measure group To measure group
performanceperformance
To compare to norm groupTo compare to norm group
To measure subgroups To measure subgroups
represented in norm grouprepresented in norm group
As reliable & validAs reliable & valid
MISUSE
To measure individual To measure individual
student performancestudent performance
To compare to specific To compare to specific
standards through standards through
general domainsgeneral domains
To measure To measure
excluded/unaccounted excluded/unaccounted
for subgroupsfor subgroups
As modified without As modified without
new new
validation/reliabilityvalidation/reliability
05
101520253035404550
1st Qtr 2nd Qtr 3rd Qtr 4th Qtr
Majority
Hispanic
Black
Norm-Referenced Tests Does the NRT measure OUR studentsOUR students?
Does the score tell if the standardstandard is MET?
Do questions match OUR standardsOUR standards?
Did OUR experts match items/tasks to standardsstandards?
Are “alignment” studies specific to OUR OUR standardsstandards?
Does the NRT measure ALL students against OUR OUR standardsstandards?
BLENDED ASSESSMENTSBLENDED ASSESSMENTS
combines norm-referenced and standards-based testscombines norm-referenced and standards-based tests
Norm-referenced tests provide Comparisons to national normsComparisons to national norms
High levels of technical qualityHigh levels of technical quality
Familiarity and confidenceFamiliarity and confidence
Standards-based tests provide Comparisons to standardsComparisons to standards
Feedback on learning what is intendedFeedback on learning what is intended
Data useful to classroom teaching and Data useful to classroom teaching and learninglearning
Developmental Profile
Benchmark Tests
High School Graduation Qualifying Exam
Terra Nova CAT/6
National Assessment
of Educational
Progress
September 1 to
November 1
March
4, 5 & 6
October
28, 29 & 30
March
4, 5 & 6
March
4 - 21
January 27 to
March 7
K X K
1 X# 1
2 2
3 X 3
4 X X 4
5 X 5
6 X 6
7 X 7
8 X X 8
9 X 9
10 X 10
11 R R 11
12 R R 12
AlaskaLarge-Scale Assessment Matrix
LEGAL ISSUESTechnical quality of results
Validity and reliability of results Accuracy of results Appropriate use of results
Stakes for schools Stakes for students and teachers Legal issues
Rewards and sanctions Opportunity to learn Accounting for/Accommodating AllAll
Changes to the HSGQE
Framework for Continuous Renewal
SB 133
Committee Actions and Recommendations
Affirmed the Appropriateness of the Performance Standards
Reviewed bank of test questions Suggested that a number of questions be
changed or eliminated.
Suggested that the test be refocused
Suggested that certain portions of the test be revised.
Refocused HSGQE
Examination focusing on assessing minimum competencies in essential skills
Data used to insure that all students who receive a diploma in the state of Alaska can demonstrate minimum competencies in essential skills
Revised HSGQEMore specific direction in items and questions According to the article… Give a reason and list two examples
Format of Test Reading or charts on facing pages Writing check list
Varied Reading PassagesLength of TestReal World Focus and Application
Process for Development of Refocused and Revised HSGQE
New Test Item Maps Constructed
New Test Forms Generated based on new item maps Spring 2002 (Refocused)
New passing scores established August 2002
HSGQE Test Blueprint
ReadingMultiple choice
Short Response
Extended Response
Initial Test -Totals 40 13 1
R4.1 Use context clues 9 0 0R4.4 Summarize Information; Make connections 5 2 1R4.5 Support Main Idea; critique arguments 13 1 0R4.6 Read and apply multi-step directions 4 1 0R4.7 Analyze literary conventions & techniques 4 1 0R4.8 Analyze narrative elements 1 3 0R4.9 Make and support assertions 2 2 0R4.10 Analyze and evaluate themes 2 3 0
Refocused Test -Totals 35 15 0
R4.1 Use context clues 4 0 0R4.4 Summarize Information; Make connections 12 2 0R4.5 Support Main Idea; critique arguments 7 3 0R4.6 Read and Apply multi-step directions 4 3 0R4.9 Make and support assertions 4 4 0R4.10 Analyze and evaluate themes 4 3 0
Percent of Emphasis
100%
12%19%19%8%11%10%8%12%
100%
6%24%19%16%19%16%
HSGQE Test Blueprint
WritingMultiple choice
Short Response
Extended Response
Percent of Emphasis
Initial Test Totals 40 5 1 100%
W4.1/2 Write compositions /Demonstrate elements of discourse 0 4 1 32%W4.3 Use conventional English 18 1 0 35%W4.4 Revise Writing 22 0 0 32%
Refocused Test Totals 26 8 1 100%
W4.1/2 Write compositions / Demonstrate elements of discourse 0 4 1 54%W4.3 Use conventional English 14 2 0 27%W4.4 Revise Writing 12 2 0 19%
Percent of Emphasis
100%
10%14%14%21%21%20%
HSGQE Test Blueprint
MathematicsMultiple choice
Short Response
Extended Response
Intial Test Totals 52 10 2
A1: Numeration 6 1 0A2: Measurement 7 2 0A3: Estimation and Computation 7 2 0A4: Functions and Relationships 11 3 0A5: Geometry 11 1 1A6: Statistics and Probability 10 1 1
Refocused Test Totals 50 6 2
A1: Numeration 12 1 0A2: Measurement 10 2 0A3: Estimation and Computation 10 2 0A4: Functions and Relationships 6 0 1A5: Geometry 5 1 0A6: Statistics and Probability 7 0 1
100%
20%20%20%14%10%16%
What does this mean for the classroom?
HSGQE Refocused to assess minimum competencies of essential skills
Teachers need to continue teaching all of the standards to all of the students
Student performance needs to be evaluated using multiple assessments
Alaska State Assessment and TerraNova-CAT/6
Second EditionCopyright 2000K-12 Basic Skills Achievement BatteryThree assessment models Multiple Assessments Complete Battery Survey
TerraNova-CAT/6 Complete Battery Plus
Selected Response Items Subtests Reading/Language Arts* Mathematics* Science Social Studies Vocabulary* Language Mechanics* Computation* Spelling
TerraNova-CAT/6 Content
Objective Structure Developed across levels Thematic approach in
reading/language Real life context Model for CSSA item
Participation Guidelines
New booklets will be sent to schools by early December. Indicates 2002/2003 on cover
Will include changes specified by SB 133 Optional assessment program
Test SecurityBoth NRT, HSGQE and Benchmark Tests must be administered by school personnel at a school test center. Correspondence programs must
arrange to have tests administered by school personnel at school test centers.
Test materials must be stored in a secure manner
Test Security-continued
All test questions are confidential-no unauthorized access- TerraNova/CAT,HSGQE,Benchmark
HSGQE Associate test coordinator
Alaska Certification required Proctor ratio 1:30
Alaska Certification required Cannot be teachers who, at time of testing have
the student in class (in subject tested)
Duties of School Personnel
Ensure confidentiality of test questionsResponsible for test administration Inventory and track materials Control distribution, storage,
administration and collection of tests Ensure that no test is copied in any
manner
Duties of School Personnel- continued
District Test Coordinator-HSGQE Assure that each HSGQE test booklet
and test administration manual is accounted for from the time the materials arrive at the district office until the time they are returned to the test publisher.
Ensure adequate supervision of students taking the HSGQE
Duties of School Personnel- continued
School Test Center Code tests according to directions Inventory and track materials Securely store tests before and after use Control distribution of tests Ensure that:
no test is copied examinees only use reference materials specified
in directions no student receive a copy of test or learns
questions in advance Examinees do not exchange information Examinee’s answer is not altered after testing is
completed
Results returned to parents/students
HSGQE within 15 days of receiving or before
end of the school year.
Benchmark Exams within 20 days of receiving or before
end of the school year.
Norm Referenced Test In a timely manner
Results distributed to teachers
Class and individual results to each teacher for the teachers classes
Intent is that teachers have assessment data for students in their classes
ConfidentialityIndividual student results are confidentialFamily Educational Rights and Privacy Act (FERPA) Scores can be shared with school and
other education officials to complete their duties
May not be publicly reported without parent consent Fewer than 5 students tested/3 in any cell
Issues in Reporting Data
Confidentiality Family Educational Rights and Privacy Act
Technical Issues Small Schools
Release of Information Results not reported if fewer that 5 students
tested or fewer than 3 students are reported in any cell.
Protocol for Reporting ResultsTwo Categories of Achievement
Complete results will be reported as long as at least 3 students are reported in any cell. If either proficiency level contains 0,1, or 2 students the number of students will be eliminated and the percentage of students at each achievement level will be reported as a range. In reporting a range of performance the following rules will apply. NOTE: No results will be reported if less than 5 students are tested.
Number of Achievement Level ReportedStudents Tested
5 60% or more proficient (or not proficient)40% or fewer not proficient (or proficient)
8 75% or more proficient (or not proficient)
25% or fewer not proficient (or proficient)
10 80% or more proficient (or not proficient)20% or fewer not proficient (or proficient)
20 90% or more proficient (or not proficient)
10% or fewer not proficient (or proficient)
40 95% or more proficient (or not proficient) (or more) 5% or fewer not proficient (or proficient)
Reporting Protocol
Making effective use ofthe DATA
Provide results to parents, students and teachers School wide Individual
Devise ways to provide easy access to assessment data to teachers Include as much information as possible
Designation API
Mining the DATALook beyond the mean Disaggregate data to determine the effect of
program on various groups Quartiles, item analysis API Scores
Look at trends over time
Work with staff to determine what you can learn from the data how you will react to the data
Using the Results to Make Using the Results to Make DecisionsDecisions
About school improvement planningschool improvement planning
About policy changes and program evaluation
About rewardsrewards and sanctionssanctions
Use the results to:focus instructiontarget student interventionidentify needsdevelop programsevaluate programs and initiativesguide Staff Development activitiesset goals for improvement
Examining the Data to Set Goals
Weakest results Domain scores = reading?
Specific standards = use comprehension strategies?
Specific skills within the standard = paraphrase key points?
Demographics Population impact = low socio-economic majority?
Opportunity to learn = poor attendance rates?
SCHOOL IMPROVEMENT PLAN Selecting successful instructional interventions
For the area of concern For the age, grade, and skill level of the students For the demographics of the schoolFor long term effectiveness
Study and training Support through coaching, observation, and feedback Follow-up for ongoing training and reflection Monitoring through supervision and results
Specific Reasonable
Time lines and assignment
Planning staff development
Data CollectionImportance of Proper Coding Demographic Data Program data District or school data
Importance of Accurate and timely Data SubmissionsImportance of Verification of Data
Data CollectionChanges for 2002/2003
Increase in OASIS information Special Education Fields Summer Collection
Changes to Common Core Data-October 2002 Reporting Teacher Assignments Certification
Implementation of Unique Student Identification Number
Questions, Comments, Concerns