Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement
description
Transcript of Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement
![Page 1: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/1.jpg)
Using Assessment Data to Monitor Teaching Effectiveness to Encourage School
Improvement
Margaret HeritageCRESST/UCLA
CBSE & ACER International Educational Conference on Assessment 25th January, 2011Delhi, India
![Page 2: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/2.jpg)
Overview
• Recent U. S. Developments• A System of Quality Assessments• Quality Process of Data Use• Data Tools• Data Use Skills and Culture
![Page 3: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/3.jpg)
Recent Developments
• Spotlight on data use in the U.S. for the past decade (e.g., Ikemoto & Marsh & 2007; Hamilton, Stecher & Klein, 2002)
• Research suggests that data use is a characteristic of effective schools and districts (e.g., Casserly, 2006; Cawelti & Protheroe, 2001; Consortium for School Networking, 2003; Laird, 2006; Supovitz & Klein, 2003; Teddlie & Reynolds, 2000; Tognieri & Anderson, 2003)
![Page 4: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/4.jpg)
Key Issues
• Use of assessment data in the service of teaching and learning requires:–Quality assessments –Quality in the process of using data derived
from assessments (Herman & Heritage, 2007)
![Page 5: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/5.jpg)
QUALITY ASSESSMENTS
![Page 6: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/6.jpg)
One size does NOT fit all
![Page 7: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/7.jpg)
Three Features of an Assessment System
• Comprehensiveness
• Coherence
• Continuity
A coherent assessment system is built on a well-structured
conceptual base – an expected learning progression, which
serves as the foundation of all assessments
A comprehensive assessment system
“provides a variety of evidence to support educational decision
making" (p. 259)
A continuous assessment system provides
“indications of student growth over time.” (p. 259).
(NCR, 2001)
![Page 8: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/8.jpg)
Student Standards
DailyWeekly
UnitQuarterly
Annually
Assessments in the System
Minute -by-minute
Adapted from Herman & Heritage, 2007
![Page 9: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/9.jpg)
Assessment Quality
• Validity
• Reliability
• Freedom from Bias
![Page 10: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/10.jpg)
QUALITY PROCESS OF DATA USE
![Page 11: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/11.jpg)
Anal
ANALYSIS OF PRESENT STATE
TRANSFORMATION PLANNING
TRANSITION TO NEW STATE
PRESENT STATE
Data analysis &
interpretation
Targeted data
collection
Identify available
data
Identify what we want to
know
OK
no
yes
Data Use Process
![Page 12: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/12.jpg)
Major Question FociHow are students performing by subject area?Question How do our students perform by grade level?
How does our students’ performance compare with the state and district? How does our students’ performance compare with demographically similar high
performing schools?
Analysis Mean or median scores Percentage of students performing at each proficiency level Identify lowest performing students and/or students performing at or below X level
Interpretation How well are we serving our students? Are some grades performing substantially better or worse than others? What are the implications for action? What students need special attention?
What are the trends in student performance by subject area over time?Question What do trends look like by grade level?
How do trends compare with the state and district? How do trends compare with demographically similar, high performing schools?
Analysis Longitudinal analysis by cohort Percent of students performing at particular proficiency levels* Note: Use mean and medians only if the tests are vertically scaled
Interpretation Are we improving? Are some grades performing substantially better or worse than others? What are the implications for action?
State Assessments
![Page 13: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/13.jpg)
How are subgroups performing currently and over time?Question How are they performing based on current results?
What do trends in performance look like? How do trends compare with the state and district? How do trends compare with demographically similar, high performing schools?
Analysis Mean or median scores Percentage of students performing at each proficiency level by subgroup Longitudinal analysis by cohort Percent of students performing at particular proficiency levels* Note: Use mean and medians only if the tests are vertically scaled
Interpretation How well are we serving our students? Are we improving? Are some grades performing substantially better or worse than others? What are the implications for action?
What are relative strengths and weaknesses in teaching and learning?Question How do our students perform by strand?
How well do our students perform on the item content clusters?Analysis Mean or median scores by strand and grade
Analyze items in clusters to identify any specific skill or understanding that might have led students to answer questions incorrectly
Interpretation What are the relative strengths and weaknesses based on the strand data? What are the relative strengths and weaknesses based on the content clusters? In what areas do we need to improve? What are the implications for action? Which students need special attention?
What evidence is there that curricular changes, new programs or instructional strategies are working?QuestionAnalysis Identify what changes you would expect to see as a result of specific innovations
Identify when you would expect to see improvements in performance?
Interpretation Does the evidence meet the expectations?
![Page 14: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/14.jpg)
Major Question Foci
How are students performing by subject area?Question How do our students perform by grade level?
How does our students’ performance compare with the state and district? How does our students’ performance compare with demographically similar high
performing schools? Are the target test results consistent with the results of the state test? How are students performing on the target tests on those areas identified as weak on the
state tests?
Analysis Mean or median scores Percentage of students performing at each proficiency level Identify lowest performing students and/or students performing at or below X level
Interpretation How well are we serving our students? Are some grades performing substantially better or worse than others? What are the implications for action? Which students need special attention? Is there evidence that our innovations are working?
Which individuals are currently struggling?Question Who are the students who are most in need?
What are their strengths and weaknesses?Analysis Identify students who are scoring at the X levelInterpretation Who needs special attention?
What attention do they need?
INTERIM ASSESSMENTS – First Administration
![Page 15: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/15.jpg)
What are relative strengths and weaknesses in teaching and learning?
Question How do our students perform by strand? How well do our students perform on the item content clusters?
Analysis Mean or median scores Analyze items in clusters to identify any specific skill or understanding that might have led
students to answer questions incorrectly
Interpretation What are the relative strengths and weaknesses based on the strand data? What are the relative strengths and weaknesses based on the content clusters? In what areas do we need to improve? What are the implications for action? What students need special attention?
What evidence is there that curricular changes, new programs or instructional strategies are working?QuestionAnalysis Identify what changes you would expect to see as a result of specific innovations
Identify when you would expect to see improvements in performance?
Interpretation Does the evidence meet the expectations?
INTERIM ASSESSMENTS – First Administration
![Page 16: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/16.jpg)
Major Question FociHow are students performing by subject area?Question How do our students perform by grade level?
How does our students’ performance compare with the state and district? How does our students’ performance compare with demographically similar high
performing schools? Have my students made progress on the item content clusters that were identified as weak
on the last target test administration? Is there an increase in achievement on the target tests in areas where we have specifically
focused on improvement since the last administration of target tests? Is there an increase in achievement on the target tests in areas where we have
implemented curricular and instructional innovations?
Analysis Mean or median scores Percentage of students performing at each proficiency level Identify lowest performing students and/or students performing at or below X level
Interpretation How well are we serving our students? Are some grades performing substantially better or worse than others? What are the implications for action? What students need special attention? Is there evidence that our innovations are working?
Which individuals are currently struggling?Question Who are the students who are most in need?
What are their strengths and weaknesses?Analysis Identify students who are scoring at the X levelInterpretation Who needs special attention?
What attention do they need?
INTERIM ASSESSMENT - Second Administration
![Page 17: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/17.jpg)
What are relative strengths and weaknesses in teaching and learning?Question How do our students perform by strand?
How well do our students perform on the item content clusters?
Analysis Mean or median scores Analyze items in clusters to identify any specific skill or understanding that might have led
students to answer questions incorrectly
Interpretation What are the relative strengths and weaknesses based on the strand data? What are the relative strengths and weaknesses based on the content clusters? In what areas do we need to improve? What are the implications for action? What students need special attention?
What evidence is there that curricular changes, new programs or instructional strategies are working?Question
Analysis Identify what changes you would expect to see as a result of specific innovations Identify when you would expect to see improvements in performance?
Interpretation Does the evidence meet the expectations?
INTERIM ASSESSMENT - Second Administration
![Page 18: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/18.jpg)
Multiple Data Sources
• What have my students learned?
• What are the strengths and weaknesses student learning?
![Page 19: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/19.jpg)
Multiple Data Sources
• Aggregated data from annual assessments
• Subscales for more specific information
• Quarterly assessments to review rate of progress and patterns
![Page 20: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/20.jpg)
Anal
ANALYSIS OF PRESENT STATE
TRANSFORMATION PLANNING
TRANSITION TO NEW STATE
PRESENT STATE
Implement Plans
Prioritize needs & options
Data analysis &
interpretation
Targeted data
collection
Identify available
data
Identify what we want to
know
Analyze & rework
Comm
unication
Build consensus & make
decisions
Collect data for monitoring & evaluation
Communication
Communication
OK
no
yes
Identify desired results
Develop targeted plan
Identify resources
Identify evidence-based interventions
Create monitoring & evaluation plan
Data Use Process
![Page 21: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/21.jpg)
Going Beyond Assessment Data(Berhardt, 1998)
Achievement
Opportunities to Learn Perceptions
Demographics
![Page 22: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/22.jpg)
DATA TOOLS
![Page 23: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/23.jpg)
The Problem
![Page 24: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/24.jpg)
Data Tools
• Store• Access• Analyze• Integrate multiple sources• Relational database
![Page 25: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/25.jpg)
Quality School Portfolio (CRESST)
23 different graphic reporting options
![Page 26: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/26.jpg)
Store student work with rubric and exemplars
Digital Portfolio
![Page 27: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/27.jpg)
Digital PortfolioStore video samples of student performance
![Page 28: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/28.jpg)
ARKANSAS LONGITUDINAL DATA SYSTEM
![Page 29: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/29.jpg)
Revised Data Analysis Framework
Increased emphasis on student cohorts
Focus on growth/progress
Specific analysis instructions
![Page 30: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/30.jpg)
Revised Data Analysis Framework
Addresses possibility of needing more data
Prompts identification of next steps
Emphasizes patterns/trends
![Page 31: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/31.jpg)
Online PD—Tool Tutorial
![Page 32: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/32.jpg)
hive Visualization
![Page 33: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/33.jpg)
DATA USE SKILLS AND CULTURE
![Page 34: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/34.jpg)
Skills (Heritage and Chen, 2005)
• Ask questions to start the investigation
• Frame drill-down questions to deepen the inquiry
• Interpretive questions
![Page 35: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/35.jpg)
Skills (Heritage and Chen, 2005)
• Select appropriate analysis (e.g., comparative, correlation, cross-sectional, longitudinal analysis)
• Interpretive skills• Set priorities and goals
![Page 36: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/36.jpg)
Data Culture
• Expectation that judicious use of evidence drives the educational practices of leaders and teachers to improve student learning
• Leaders and teachers engage in using data for systematic reflection and planning
• “Using data is the way we do things around here.”
![Page 37: Using Assessment Data to Monitor Teaching Effectiveness to Encourage School Improvement](https://reader035.fdocuments.in/reader035/viewer/2022062810/56815b65550346895dc95554/html5/thumbnails/37.jpg)
To Summarize
• Well-aligned system of high-quality assessments
• Tools that support analysis and interpretation• Skills to analyze and interpret data effectively• Attitude that data are a valuable resource for
guiding improvement