Using Curricular Measures for Description & Analysis
description
Transcript of Using Curricular Measures for Description & Analysis
![Page 1: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/1.jpg)
Using Curricular Measuresfor Description &
AnalysisSEC Collaborative Meeting
Tampa, FLFebruary 21, 2005
John L. Smithson, Director, Measures of Enacted CurriculumWisconsin Center for Education Research, University of Wisconsin-Madison
Surveys of Enacted Curriculum
![Page 2: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/2.jpg)
The SEC Data - Sets
DistinctionsOn-Line Off-line
•Data collection/processing/reporting•Descriptive Data•Limited Reporting Options•Easy Access / Indiv. Results
•Analytic analyses•Unlimited reporting options•Requires data manipulation
![Page 3: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/3.jpg)
Conducting Inquiry Using SEC Data
Forms of Inquiry Collaborative or Evaluative
Teacher EnrichmentSchool ImprovementProfessional Lrng. Comm.
Program EvaluationIndicator ReportingProgram ManagementPerformance Modeling
![Page 4: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/4.jpg)
• MSP-PD Study (In progress: Blank, Smithson, Porter, Garet, Birman)
Use of SEC data for program evaluation
A short history of SEC research
• Reform-Up-Close (Porter, Kirst, Osthoff, Smithson, Schneider, 1993)
Validation of teacher self-report survey data.
• Upgrading Mathematics (Gamoran, Porter, Smithson, White, 1997)
First content analysis of assessment using content language.Predictive validity of alignment index comparing instruction & assessments
• Data on Enacted Curriculum (Blank, Porter, Smithson, 2004)
Use of SEC data to facilitate school improvement effortsFirst content analysis of state standards
![Page 5: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/5.jpg)
Alignment Relationships in Standards-based Reform
![Page 6: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/6.jpg)
Instruction
Assessment
Standards
Intersection of what is taught with what is tested.
Intersection of what is taught with what is in standards.
Intersection of whatis taught with what is in the standards
Taught,tested, and inthe standards
Alignment Relationships in Standards-based Reform
![Page 7: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/7.jpg)
A Quantitative Approach to Alignment
Porter-Smithson/SEC Alignment Process
Content analyses of curriculum documents and reports of practice by content experts using two-dimensional content language.
Multiple raters (w/ content & assessment expertise) using independent ratings in combination with team discussions.
Content Description [Topic(s) by Cognitive Demand(s)]
Yields Alignment Index based on:
![Page 8: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/8.jpg)
Alignment Analyses
The analytic power of quantitative alignment measures
• Prediction
• Program Evaluation*
• Planning
• Performance Modeling
* Not teacher evaluation
![Page 9: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/9.jpg)
Alignment Analyses
Students perform best when tested on material for which they have been provided the opportunity to learn.
Class Gains Student Gains
Alignment Index 0.451 0.259
Correlation of Alignment Index to Achievement Gains
From: Gamoran, et.al. (1997). Uprgading High School Mathematics Instruction. EEPA v19n4pp325-338.
![Page 10: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/10.jpg)
Time 0 Time 1 Time 2
Time 0 Time 1 Time 2
Learning Gains by Course Type
Learning Gains Controlling for Content
RegentsAlgebraStretch RegentsMath A/B/UCSMPGen. Mth. / Pre-alg.
From:Upgrading High School Mathematics Instruction,(Gamoran, Porter, Smithson, & White, 1997),EEPAv19n4
12
11
10.5
12
9.5
10
11.5
12
11
10.5
12
9.5
10
11.5
Explaining variation in student learning gains
![Page 11: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/11.jpg)
Alignment Index:Instruction to Standards
MathematicsAcross 4 Districts
Alignment Analyses for School Improvement
Using alignment as an outcome measure
(Measuring change in alignment over time)
Treatment 99
Control 124
Leaders 16
Counts
![Page 12: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/12.jpg)
Alignment Analyses for Planning Purposes
![Page 13: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/13.jpg)
Mapping Curriculum Materials
![Page 14: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/14.jpg)
(www.seconline.org) SEC - Online
![Page 15: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/15.jpg)
Administrative Functions:Administration Set-upReview Registrants, Completion RatesAdministrator Report Generator
![Page 16: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/16.jpg)
![Page 17: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/17.jpg)
SEC Reports
Floating Bars
Dynamic SampleSelectionDynamic DataDisaggregation
Sample andDisaggregateCounts
ComparisonCharts
IndividualResults
![Page 18: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/18.jpg)
SEC Reports: Instructional Content
Coarse Grain Maps
Content Areas
by
Cognitive Demand
![Page 19: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/19.jpg)
SEC Reports: Instructional Content
Fine Grain Maps
Topics
by
Cognitive Demand
![Page 20: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/20.jpg)
Alignment as a Systemic Tool
Fine Grain
![Page 21: Using Curricular Measures for Description & Analysis](https://reader036.fdocuments.in/reader036/viewer/2022062521/56814c4a550346895db951bb/html5/thumbnails/21.jpg)
X 1 2 3 Y 1 2 3
z 1 2 3
Alignment= 0.6
C0.1 0 0
B0 0.1 0
0.2 0 0
A0.3 0.1 0
C
0.3 0.2 0
B 0.1 0.1 0.1
AA
B
C
0.5 0.1 0.1
0 0.2 0
0.1 0 0
Calculating Alignment1-((|x-y|(1-n))/2)