1 New England Common Assessment Program Test Coordinator Workshop April 8, 14, and 19, 2011.
New England Common Assessment Program
description
Transcript of New England Common Assessment Program
![Page 1: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/1.jpg)
1
New England Common Assessment Program
Guide to Using the October, 2006 NECAP
Reports:
Companion PowerPoint
Presentation February 2007
![Page 2: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/2.jpg)
2
Welcome and Introductions
Mary Ann Snider
Director of Assessment and Accountability
RI Department of Education
![Page 3: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/3.jpg)
3
Measured Progress Service Center: 1-877-632-7774
Welcome and Introductions
Tim Crockett
Assistant Vice-President
1-800-431-8901 x2106
Harold Stephens
NECAP Program Director
1-800-431-8901 x2235
Amanda Smith
NECAP Program Manager – Grade 11
1-800-431-8901 x2259
Elliot Scharff
NECAP Program Manager – Science
1-800-431-8901 x2126
Josh Evans
NECAP Program Manager – Grades 3-8
1-800-431-8901 x2244
Tina Haley
NECAP Program Assistant
1-800-431-8901 x2427
Jennifer Varney
NECAP Program Assistant
1-800-431-8901 x2115
Mellicent Friddell
NECAP Program Assistant
1-800-431-8901 x2355
![Page 4: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/4.jpg)
4
• To assist schools and districts in using the variety of reports and support material issued by the NECAP states in conjunction with the release of results
• To provide school and district personnel an opportunity to ask questions regarding the reports and support materials
Purpose of the Workshop
![Page 5: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/5.jpg)
5
• Development of Grade Level Expectations
• Test Item Review Committees• Bias and Sensitivity Review Committees• Classroom Teacher Judgment data• Standard Setting Panelists• Technical Advisory Committee
Involvement of Local Educators
![Page 6: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/6.jpg)
6
• Fall test – previous year’s GLEs• Reading and Mathematics – grades 3-8• Writing – grades 5 and 8• Variety of Item Types
• Reading: multiple choice and constructed response• Mathematics: multiple choice, short answer (one point), short answer
(two points), and constructed response• Writing: multiple choice, constructed response, and extended
response
• Common and Matrix Items
Basics of Test Design
![Page 7: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/7.jpg)
7
Two Basic Types of Educational Tests
• Norm-Referenced: What is the student’s status or rank in a designated group?
• Criterion or Standards-Referenced: Has the student learned the skills or achieved the standard?
NECAP is a Standards-Referenced Test
![Page 8: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/8.jpg)
8
The Family Educational Rights and Privacy Act (FERPA)
• Access to individual student results is restricted to:• the student• the student’s parents/guardians• authorized school personnel
• Superintendents and principals are responsible for maintaining the privacy and security of all student records.
• Authorized school personnel shall have access to the records of students to whom they are providing services when such access is required in the performance of their official duties.
• FERPA website: http://www.ed.gov/policy/gen/guid/fpco/ferpa/index.html
Family Educational Rights and Privacy Act
![Page 9: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/9.jpg)
9
National Council on Measurement in Education
Educating Others
Evaluation and Research
Interpretation and Use
Scoring
Development
Marketing and Selling
Selection
Administration
www.ncme.org
![Page 10: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/10.jpg)
10
Questions about the purpose of theworkshop and/or NECAP background?
Questions and Answers
![Page 11: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/11.jpg)
11
Rhode Island Results
Suburban
Urban Fringe
Urban
![Page 12: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/12.jpg)
12
Rhode Island Results
READING MATHEMATICS
2005 2006 2005 2006
All Students
58% 62% 50% 53%
Native Americans
46 49 33 38*
Asian 56 64* 52 59*
Black 35 41* 24 30*
Hispanic 31 38* 24 31*
White 68 71 59 62
Female 63 67 50 53
Male 54 58 50 54*
IEP 24 27 20 23
ELL 17 23* 14 20*
Poverty 38 44* 30 36*
* Indicates that gain exceeds the gain made by all students
![Page 13: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/13.jpg)
13
• Student Report• Item Analysis Report• School/District Results Report• School/District Summary Report• District Student Level Data Files
Types of Reports
![Page 14: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/14.jpg)
14
Student Report
• Two copies mailed to the school:• color copy for parents/guardians• black and white copy for local use
• Dual sided: text side and data side
• Spanish version of the student report available on the RIDE website for the Office of Assessment and Accountability
![Page 15: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/15.jpg)
15
![Page 16: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/16.jpg)
16
![Page 17: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/17.jpg)
17
![Page 18: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/18.jpg)
18
![Page 19: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/19.jpg)
19
• Review to be sure you have a student report for every student tested in October, 2006.
• Prepare a letter to provide your interpretation of the results for your school and/or district.
• Be sure teachers are prepared and comfortable discussing individual student results with families; including any instructional interventions.
• Use your district data file to analyze growth between students you had last year and this year.
Fun with Student Level Reports!
![Page 20: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/20.jpg)
20
Questions about the Student Report?
Questions and Answers
![Page 21: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/21.jpg)
21
• Rationale• Inclusion of students in “Teaching
Year” reports
Teaching School vs. Testing School
![Page 22: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/22.jpg)
22
Schools Can View Reports for Testing Year (2006-07)
Teaching School vs. Testing School
![Page 23: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/23.jpg)
23
Teaching School vs. Testing School
Or Teaching Year(2005-06)
![Page 24: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/24.jpg)
24
Testing Year(released this month)
Teaching Year (accountability)
![Page 25: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/25.jpg)
25
Item Analysis Report
• Available to schools and districts on-line• These reports are confidential and the
website is password-protected. • One per content area• Use in conjunction with Released Items
documents
![Page 26: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/26.jpg)
26
![Page 27: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/27.jpg)
27
![Page 28: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/28.jpg)
28
Released Item Documents
![Page 29: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/29.jpg)
29
Grade 5 2006 Released Mathematics Item
DOK: 3
GLE: DSP 4.1
![Page 30: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/30.jpg)
30
Grade 5 Practice Test Item (page 4)
DOK: 2
GLE: DSP 4.1
![Page 31: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/31.jpg)
31
Fun with Released Support Materials!
• Build practice tests
• Share writing samples as models
• Mock scoring sessions to calibrate teacher judgments
• Compare released items (GLE and DOK) with classroom instruction and assessments and student work.
• Map instruction and scaffolding practices to items based on student performance.
![Page 32: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/32.jpg)
32
Released Item Documents
• RIDE website at: http://www.ride.ri.gov/assessment/NECAP.aspx
• New information provided in the Released Item Support Materials documents includes:• For all content areas, providing the multiple-choice items
and the GLE code and text that each item is linked to• For math, indicating which items were included in session 1
of the math test by adding a “no calculator” symbol above the items
• For reading, listing the passage name and type of text• For writing, listing passage name and mode of writing
![Page 33: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/33.jpg)
33
Questions about the Item Analysis Report or the Released Item documents?
Questions and Answers
![Page 34: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/34.jpg)
34
• Available on-line at the Measured Progress website
• Separate report for each grade level tested
• District report contains no individual school data
School/District Results Report
![Page 35: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/35.jpg)
35
http://iservices.measuredprogress.org
![Page 36: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/36.jpg)
36
New thisyear
Please note:
You must always log in through this page to view confidential
reports and data
![Page 37: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/37.jpg)
37
![Page 38: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/38.jpg)
38
![Page 39: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/39.jpg)
39
![Page 40: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/40.jpg)
40
![Page 41: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/41.jpg)
41
![Page 42: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/42.jpg)
42
• Page 2 of the NECAP School/District Results Report
• Summary of participation and results
Grade Level Summary Report
![Page 43: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/43.jpg)
43
![Page 44: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/44.jpg)
44
• Review last year’s grade level summary report with this year’s
• If gains were made, identify what strategies, interventions, and programs may have worked
• If no progress was made, hypothesize why and what can be done at the school or district level to help students make progress
• Identify where your results differ from the district and state results. Hypothesize why they may differ
Fun with Grade Level Summaries!
![Page 45: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/45.jpg)
45
Questions about the Grade LevelSummary Page?
Questions and Answers
![Page 46: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/46.jpg)
46
Historical Data
• Data is included for two years and is combined in the Cumulative Total row
• In the Cumulative Total row for the achievement level columns:• The N columns are summed• The % columns are found by dividing the total
number of students in the level by the total number of students tested
• The Mean Scaled Score column is a weighted average
![Page 47: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/47.jpg)
47
![Page 48: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/48.jpg)
48
Comparisons of NECAP Scores Across Years
• School and District-Level Scaled Scores and Achievement Levels
• Student-Level Scaled Scores and Achievement Levels
• Content Area Subscores
![Page 49: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/49.jpg)
49
• Pages 3, 5, and 7 of the NECAP School/District Results Report
• Provides information about achievement in subtopics of tested content areas
Content Area Results
![Page 50: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/50.jpg)
50
![Page 51: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/51.jpg)
51
Number of Scaled Score Points Denoting Minimally Statistically Significant Difference for Average Group Results*
Grade SubjectNumber of Students Tested in Group (Class, School etc.)
10 25 50 100 200
3Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
4Reading 6 4 2 2 1
Mathematics 5 3 2 2 1
5
Reading 5 3 2 2 1
Mathematics 5 3 2 2 1
Writing 7 4 3 2 2
6Reading 6 4 3 2 1
Mathematics 6 4 3 2 1
7Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
8
Reading 6 4 3 2 1
Mathematics 5 3 2 2 1
Writing 6 4 3 2 1
*Standard error of the mean difference with assumption n1=n2 and s1=s2
![Page 52: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/52.jpg)
52
Percentage Difference in Student Achievement Level Classification Denoting Minimally Statistically Significant
Differences for Group Results*
Percentages of Students in Achievement Level(s)
Number of Students Tested in Group (Class, School etc.)
10 25 50 100 200
10 9 6 4 3 2
20 13 8 6 4 3
30 14 9 6 5 3
40 15 10 7 5 3
50 16 10 7 5 4
60 15 10 7 5 3
70 14 9 6 5 3
80 13 8 6 4 3
90 9 6 4 3 2
*0.05 level of statistical significance
![Page 53: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/53.jpg)
53
Questions About the Content AreaResults Page?
Questions and Answers
![Page 54: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/54.jpg)
54
• Pages 4, 6, and 8 of the NECAP School/District Results Report
• Provides information about achievement for different groups in a school/district
Disaggregated Content Area Results
![Page 55: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/55.jpg)
55
![Page 56: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/56.jpg)
56
• Compare last year’s Disaggregated Report with this year• Highlight in yellow, groups that had significant positive
increases• Highlight in blue, groups that remained stable or had decreases
• Establish cross-grade and content groups to hypothesize what the school can do to close gaps and maintain increases• Access to rigorous curriculum?• Professional development in a specific area?• Adequate support programs?• Adequate family involvement?
Fun with Disaggregated Results!
![Page 57: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/57.jpg)
57
Questions About the Disaggregated Content Area Results Page?
Questions and Answers
![Page 58: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/58.jpg)
58
• Separate report for each grade level tested
• Provides details about student achievement by content area
School/District Summary Report
![Page 59: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/59.jpg)
59
School/District Summary Report
• The first line designates the year the test was administered
• The second line is the name of the report• The third line differentiates between
“teaching year” and “testing year”
![Page 60: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/60.jpg)
60
![Page 61: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/61.jpg)
61
• Are there consistent patterns of achievement across grades? Across content areas?
• Compare results to last year’s school summary• Highlight in yellow, areas where there has been
significant growth. Highlight in blue, areas where there has been no growth or decline.
• Establish cross-grade and content groups to hypothesize what the school can do to close gaps and maintain increases
• Develop a school/district plan to address areas of weakness and continue areas of growth
Fun with School Level Summary Data!
![Page 62: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/62.jpg)
62
Student Level Data Files
• All demographic information for each student that was provided by the state
• The scaled score, achievement level, and subscores earned by each student in all content areas tested
• Also contain:• Performance on released items• Student questionnaire responses• Optional reports fields data
![Page 63: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/63.jpg)
63
![Page 64: New England Common Assessment Program](https://reader035.fdocuments.in/reader035/viewer/2022081513/56813b21550346895da3d9f7/html5/thumbnails/64.jpg)
64
Supporting Materials
• Guide to Using the 2006 NECAP Reports and companion PowerPoint
• Grade Level Expectations• Test Specifications documents• Release Item documents• Preparing Students for NECAP: Tips for
Teachers to Share with Students• Technical Report