Module 4 for Administrators
description
Transcript of Module 4 for Administrators
Module 4 for Administrators
The Real Implications of VAM and Florida Standards
for Teacher Evaluation
2
Professional Development Session Alignment
Set 1Governing Board
School Leaders
Module 6 FL CCRS Math
Teachers Math
Leadership Teams Session 2
Session1
ELAData Use
Data Use ELA Math
Data Use
3
Professional Development Session Alignment
Set 2Governing Board
School Leaders
Module 7 ELA & Data Use
Module 8 Math & Data Use
Teachers Math
Leadership Teams
Session 4
Session3
ELA
AssessmentVAM &
Academic Rigor
Florida Standards
Data &ELA
Data &Math
Session 5
Session 6
4
Data
5
8 Components of Full Florida Standards Implementation
Develop an understanding of Florida’s Value-Added Model (VAM) and its implications to teachers as well as the entire charter school.
Increase awareness of the higher standards of learning and rigor the new assessments will require of students and staff.
Recognize best practices that teachers should be implementing to increase rigor in the classroom.
Develop an understanding of teacher and student behaviors that should be observed during classroom walkthroughs to ensure a rigorous classroom.
Recognize the importance of Webb’s Depth of Knowledge Levels and its connection to increased rigor.
Module Outcomes
6
Today’s Agenda
• Welcome and Introductions• Pre-Assessment• Section 1: Teacher Evaluation- The Value-
Added Model• Lunch• Section 2: Understanding Academic Rigor• Section 3: Connecting Academic Rigor
and Depth of Knowledge• Next Steps• Wrap Up and Post-Assessment
7
To Do List
Pre-Assessment
Introductory Activity
8
Guide Page
4
Teacher Evaluation: The Value-Added Model (VAM)
9
Section 1
• Research has shown that the best predictor of a teacher’s effectiveness is his or her past success in the classroom. Most other factors pale in comparison, including a teacher’s preparation route, advanced degrees, and even experience level (after the first few years).*
• The lesson is clear: to ensure that every child learns from the most effective teachers possible, schools must be able to gauge their teachers’ performance fairly and accurately.
*Jordan, Mendro, and Weerasinghe, Teacher Effects on Longitudinal Student Achievement, 1997
Why Change Teacher Evaluation Systems?
10
• “The effect of increases in teacher quality swamps the impact of any other educational investment, such as reductions in class size.” Goldhaber, 2009
• “More can be done to improve education by improving the effectiveness of teachers than by any other single factor.” Wright, Horn and Sanders, 1997
• “Having a high-quality teacher throughout elementary school can substantially offset or even eliminate the disadvantage of low socio-economic background.” Rivkin, Hanushek and Kain, 2005
Teachers Matter
11
Changing the Traditional Teacher Evaluation Model to Ensure Quality
12
Using Multiple Measures
Watch the video on changes to teacher evaluation systems. Discuss with members at your table the challenges and opportunities for your charter school with stronger teacher evaluation models.
What Changes to the Florida Teacher Evaluation System Have Been Established?
• Established by Student Success Act (SB 736) Educator Evaluation System
• Teacher and School Administrator Evaluations
Professional Practice Student Learning Growth
• FLDOE Student Growth Implementation Committee (SGIC)
13
What Is the Biggest Shift In Florida’s Teacher Evaluation System?
14
The Teacher Evaluation Model Now Has Two Major Components: Instructional Practice
• Deliberate Practices• Teacher Observation• Professional Growth• Surveys
Student Performance (VAM data)
Important to Know….It is important to note that measures used in the Florida School Grading system, student proficiency and learning gains, are not a part of the Florida Value-Added Model of student learning growth.
Student learning growth is based on the actual scale score on FCAT 2.0 tests. Although students may score in Achievement Levels 3-5, they still have room to grow.
15
• A Value-Added model attempts to measure the impact of a teacher on student learning, by accounting for other factors that may impact the learning process.
• The models do not:• Evaluate teachers based on a
single year of student performance or proficiency.
• Evaluate teachers based on simple comparisons from one year to the next.
The New Measure: Value-Added Analysis
16
Let’s Take a Break…
17
Be back in 15 minutes…
Comparing the Features of a Growth Model to a Value-Added Model
Understanding the Purpose of Student Growth Models
18
Looking at the Difference Between Growth and Proficiency
Growth (Progress)Growth models measure the amount of academic progress students make between two points in time
Proficiency (Status)A method for measuring how students perform at one point in time
19
Looking at the Difference Between Growth and Proficiency
Stu A Stu B Stu C Stu D Stu E0
50100150200250300350400450500
Prior Current
Stu F Stu G Stu H Stu I Stu J0
50100150200250300350400450500
Prior Current
Growth?
Teacher 1 Teacher 2
ProficiencyProficiency
Growth
20
Growth> Compare student to
own prior performance > Consider student
characteristics> Progress between
points> Critical to student
success
Proficiency> Compare student to
a standard> Does not consider
student characteristics> Performance at a
point in time> Critical to
postsecondary opportunity
A Better Picture of Student Learning
21
&
VAM & Student Learning Growth
• Teacher contribution to a change in a student’s achievement on standardized test
• Calculated from a measure of student learning growth over time
• Students may not be proficient, but may have met their student growth target
http://www.fldoe.org/committees/sg.asp22
23
Understanding VAM: What Are The Scores?What is the Predicted Student Score? • It is the score you would EXPECT a student to achieve based on the
student’s performance on prior tests and other information available on the student.
• A predicted score for a student is generated based on what would normally happen in an average class with a typical teacher.
What is the Student Learning Growth Score? • The difference between current test score and predicted test
score.
Student Learning Growth is the Amount Above or Below Predicted Score
24
Proficiency vs. Predicted Growth (VAM)
Stu A Stu B Stu C Stu D Stu E0
50100150200250300350400450500
Prior Current PredictedStu F Stu G Stu H Stu I Stu J
0
50
100
150
200
250
300
350
400
450
500
Prior Current Predicted
Teacher 1 Teacher 2Did Student B meet or exceed the predicted score?
Did Student J meet or exceed the predicted score?
25
• Teachers teach classes of students who enter with different levels of proficiency and possibly different student characteristics.
• Value-Added models ATTEMPT to “level the playing field” by accounting for differences in the proficiency and characteristics of students assigned to teachers.
26
Advantages of Value-Added Models
• Value-Added models are designed to MITIGATE the influence of differences among the entering classes so that schools and teachers do not have advantages or disadvantages simply as a result of the students who attend a school or are assigned to a class.
• Value-Added models are not perfect. The model will be continually reviewed by the FLDOE in case adjustments are necessary.
Focusing on Student Growth1. Discuss with others at your table the impact focusing on
student growth rather than proficiency will have at your school.2. In the Participant Guide on page 6, review the guiding
questions. Brainstorm ways that using a student growth reporting system will be more informational and beneficial than the traditional proficiency based system that is currently in place.
3. Be prepared to share with the entire group.
Activity 1: Focusing on Student Growth
27
Guide Page
6
STUDENT Characteristics
CLASSROOM Characteristics
SCHOOL Characteristics
Understanding VAM: Factors Used To Adjust Predicted Score
28
Factors Identified to “Level the Playing Field”
Understanding VAM: Student Characteristics
29
Students with Disabilities
English Language Learners Gifted Status
Attendance MobilityNumber of
Subject-Relevant Courses Enrolled
Two Years Achievement
Scores
Difference from Modal Age in
Grade
30
Understanding VAM: Factors NOT Used To Adjust Predicted Score
30
Gender Race
EthnicitySocio-
economic Status
Student Characteristics NOT directly accounted for
Since these factors already influence a student’s performance and prior performance is the predictor with the strongest weight, they are indirectly accounted for.
Student A
Prior FCAT Reading
Score: 1700
Attendance: 10 days absent
English Language Learner
Student B
Prior FCAT Reading
Score: 1700
Attendance: 0 days absent
Not an English Language Learner
How Do The Factors Affect The Predicted Scores? – An Example
31
Student A Predicted
Score1750
Student B Predicted
Score1790
How Are Student Growth ScoresCalculated for the Teacher/School?
• State calculates scores for each grade and subject separately
• Therefore, a teacher/school may have more than one score
32
Percent Ranks and Ratings• State generates a percent rank for each
teacher, school, district based currently on FCAT scores• Reading score is ranked among all
Reading scores in the state (Grades 4-10)
• Math score is ranked among all Math scores in the state (Grades 4-8, No Algebra EOC)
• Reading + Math score is ranked among Reading + Math scores in the state
33
What Are the Key Features of A Teacher Report for VAM Data?
34
35
What Are the Key Features of A Teacher Report?
36
What Are the Key Elements of A Class Report?
Amount of growth above predicted score made by student
10 out of 13 students met their expectation
Sample School Report for Students
37
Effective
Combined Scores
Sample Questions to Ask When Analyzing VAM Data
What are our strengths as a charter school?
• In what grade levels and content area(s) did we produce more than expected growth with our students?
• Is there anything special or different about the instructional strategies when delivering that content?
What are the challenge areas for our charter school?
• What is different about how this content is delivered?
• What root cause can we determine that may affect the progress in that content area?
38
Activity 2: Understanding VAM and Its Impact on Charter Schools
39
Understanding VAM: the Value-Added Model 1. Look at the table on page 7 in your Participant Guide. The table
has columns labeled: New Information, Opportunities, Challenges, and Questions for your District regarding the new teacher evaluation system which includes VAM.
2. Individually, using all the information (including new learning) fill out the table as it relates to your charter school.
3. After you have completed the table, share with other participants your thoughts and discuss the reflection questions at the bottom of the activity sheet.
Guide Page
7
Bon Appétit
40
Be back in 1 hour…
Understanding Academic Rigor
41
Section 2
How Can School Leaders Support Stronger VAM Scores?
42
• School leaders cannot change how VAM scores are calculated, but do have influence over the level of rigor in classrooms in their charter school.
• Focusing on what teachers and students are doing in the classroom will raise student achievement level and eventually affect VAM scores.
Are our students ready for more rigor?
What is Rigor?
Quickly jot down your definition of rigor and characteristics of a rigorous lesson you may see at your school. In your table group, discuss your definition with others and determine if there is a consensus.
Think of a teacher who applies rigorous practices in their classroom.
Defining Academic Rigor
43Teaching What Matters Most: Standards and Strategies for Raising Student Achievement. ASCD, 2001
“When instruction is academically rigorous, students actively explore, research and solve complex problems to develop a deep understanding of core academic concepts that reflect college readiness standards.” Oregon Small Schools Initiative
“Rigor is creating an environment in which each student is expected to learn at high levels; each student is supported so s/he can learn at high levels; and each student demonstrates learning at high levels.” Barbara Blackburn
What Is Academic Rigor?
44
Teaching What Matters Most: Standards and Strategies for Raising Student Achievement. ASCD, 2001
Rigor 101 With Barbara Blackburn
Low Expectations
Less Challenging Content-Courses
Low Level Instruction
Assignments
Poor Assessment
Results
Without Academic Rigor: Cycle of Low Expectations
45
Assessing Evidence of Academic Rigor In Your Charter School1. Review the list of standards of academic rigor in schools on pages 9-11
in your Participant Guide.
2. For each standard, assess the level of rigor (1-3) you believe best reflects your school at this time. Be sure to include evidence that supports your implementation level.
3. Once you have completed the survey, discuss the reflection questions on the Activity Sheet with members of your table group.
Activity 3: Assessing Evidence of Academic Rigor In Your Charter School
46
Guide Pages 9-11
What Does Academic Rigor Look Like In the Classroom?
Rigor Is Not…
• Something extra you assign on top of everything else
• More of the same math problems• Faster rate of instruction• More homework
Rigor Is…
• An integral part of quality instruction
• Increasing the complexity of questions that require more than one word answers
• Expanding academic challenges• Digging deeper into an
assignment to gain full understanding
47
What Should
Teachers Do?
Create an environment to support student learning at high
levels.
Ask higher order thinking
questions.
Support students with gaps, not
provide different assignments.
Assess students to allow them to
demonstrate their knowledge.
What Should Teachers Be Doing In A Rigorous Classroom?
48
Davis, P. (2013). Teaching to the Rigor of the CCSS. ASCD Webinar
What Should
Students Do?
Think deeply about tasks.
Interpret information &
synthesize knowledge.
Analyze their learning and
apply it.
Challenge themselves
intellectually.
What Should Students Be Doing In A Rigorous Classroom?
49
Davis, P. (2013). Teaching to the Rigor of the CCSS. ASCD Webinar
Getting Rigor Right, What You Should See In A Classroom1. Read the article “
Before and After the Walkthrough: What to Do to Improve Instructional Rigor”, which is a separate handout.
2. After reading the article, review the eight best practices highlighted in the article.
3. Use the Activity Sheet on pages 12-13 of your Participant Guide to assess the practices currently used at your charter school to raise the rigor of instruction to better align to the Florida Standards.
4. Determine which areas your school has been making strides to accomplish and which areas need more focus and direction. Think about if the best practices for rigor are reflected in your Florida Standards Implementation Plan. Discuss with others at your table.
Activity 4a: Getting Rigor Right, What You Should See In a Classroom
50
Guide Pages 12-13
Activity 4b: Assessing Rigor During A Lesson
51
Assessing Rigor During A Lesson
1. Watch the video of a high school class teaching a lesson on the Declaration of Independence.
2. Using the Focusing on Rigor Classroom Walkthrough Tool on pages 14-16 in the Participant Guide, assess the level of rigor during the lesson. Be sure to include evidence.
3. After the video, discuss with your table groups the observations made regarding the rigor of the lesson.
4. As a group, outline feedback that you as a school leader would provide to the teachers in the class to increase the rigor of the lesson.
Teaching the Declaration of Independence
Guide Pages 14-16
Connecting Academic Rigor and Depth of Knowledge
52
Section 3
Rigor’s Impact on Assessment
53
• Rigor in assessment is important for students to demonstrate what they know and are able to do.
• Assessing students’ depth of knowledge in a content area aligns to the expectation of the new Florida Standards and assessments.
The new Florida Standards increases Rigor and Depth of Knowledge.
Student’s Depth of Knowledge should focus on the complexity of the Florida Standards in order to successfully complete an assessment or task. The outcome is the focus of the depth of understanding.
Assessments must also accurately measure students’ depth of knowledge of the content standards.
Aligning Rigor and Depth of Knowledge (DOK) with Curriculum, Instruction, and Assessment
54
• Focuses on the complexity of content standards in order to successfully complete an assessment or task. The outcome (product) is the focus of the depth of understanding.
• Aligns to the rigor of the new Florida Standards and the higher expectations that teachers need to have for student learning.
Why Depth of Knowledge?
55
• Norman Webb (1997) developed a process and criteria for systematically analyzing the alignment between standards and assessments.
• Depth of Knowledge (DOK) is in reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product.
• DOK measures cognitive demand, or complexity, and is assigned using Levels 1 to 4.
Looking At Webb’s Depth of Knowledge
56
57
Webb’s Four Levels of Cognitive Complexity
• Recall and ReproductionLevel 1• Skills and ConceptsLevel 2• Strategic ThinkingLevel 3• Extended ThinkingLevel 4
Depth of Knowledge Overview
58
• Requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure
• Answering a Level 1 item can involve following a simple, well-known procedure or formula
DOK Level 1: Recall and Reproduction
Discuss at your table what DOK 1 would look like in a classroom. What would the teacher be doing, what would students be doing?
• Includes the engagement of some mental processing beyond recalling or reproducing a response
• Items require students to make some decisions as to how to approach the question or problem
• Actions imply more than one mental or cognitive process/step
DOK Level 2: Skills/Concepts
59
Discuss at your table what DOK 2 would look like in a classroom. What would the teacher be doing, what would students be doing?
• Requires deep understanding exhibited through planning, using evidence, and more demanding cognitive reasoning
• The cognitive demands are complex and abstract
• An assessment item that has more than one possible answer and requires students to justify the response would most likely be a Level 3
DOK Level 3: Strategic Thinking
60
Discuss at your table what DOK 3 would look like in a classroom. What would the teacher be doing, what would students be doing?
• Requires high cognitive demand and is very complex
• Students are expected to make connections, relate ideas within the content or among content areas, and select or devise one approach among many alternatives on how the situation can be solved
• Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time
DOK Level 4: Extended Thinking
61
Discuss at your table what DOK 4 would look like in a classroom. What would the teacher be doing, what would students be doing?
The Depth of Knowledge is NOT determined by the verb (Bloom’s Taxonomy), but by the context in which the verb is used and the depth of thinking required. The intended student learning outcome determines the DOK level.
Depth of Knowledge
62
Bloom
What type of thinking is needed
(verbs) to complete the task
Webb
How deeply do you need to
understand the content to be successful?
How Is the Learning Assessed?
Looking at Alignment
63
Adapted from: http://karolyeatts.com/Polk%20Math%20Institute%202008/Webb%20and%20Bloom%20Comparison.pdf
Guide Page
18
• DOK 3: Describe how the author of a short story must be cognizant of how much space he actually has to develop the elements of the story because he is confined by space..(Requires deep understanding of the elements of a short story and definition of the short story genre.)
• DOK 2: Describe the difference between a short story and a novel. (Requires cognitive processing to determine the difference in the two genres.)
• DOK 1: Describe the characteristics of a short story. (Simple recall)
Examining Depth of Knowledge
64
From: Depth of Knowledge: Promoting Rigor and Relevance in Learning. Fulton County School District
Examining Depth of Knowledge in Classrooms1. Before beginning the session, divide into groups specifically for
elementary, middle, and high school. Once in your grade alike groups, break up into smaller groups of 4-6.
2. In your Participant Guide on pages 19-26, read the overview regarding instructional practices at each DOK Level.
3. Individually, for each DOK Level, determine if the task examples are observed during classroom walkthroughs at your charter school. If they are observed, assign a Level of Use score for each task.
4. When all four DOK Levels are completed, in table groups compare your results. Discuss the Reflection Questions on page 27. Note the strategies to help support increased rigor in your school if not currently seen.
5. Reflect on what changes will be needed in your Implementation Plan to increase rigor and Depth of Knowledge.
Activity 5: Examining DOK in Classrooms
65
Guide Pages 19-27
Activity 6: Next Steps
66
Big Ideas People to Share With
Section 1: Teacher Evaluation-The Value-Added Model
Section 2: Understanding Academic Rigor
Section 3: Connecting Academic Rigor and Depth of Knowledge
What are some “big ideas” that you want to remember from today? With whom do you need to share within your charter school in order to take the next steps to support the transition to the Florida Standards?
Guide Page
28
Closing Activities
67
• Developed an understanding of Florida’s Value-Added Model (VAM) and its implications to teachers as well as the entire charter school.
Increased awareness of the higher standards of learning and rigor the new assessments will require of students and staff.
Identified best practices that teachers should be implementing to increase rigor in the classroom.
Developed an understanding of teacher and student behaviors that should be observed during classroom walkthroughs to ensure a rigorous classroom.
Recognized the importance of Webb’s Depth of Knowledge Levels and its connection to increased rigor.
Module Outcomes
68
Post-Assessment and Session Evaluation
69
Where Are You Now?
Assessing Your Learning
Guide Page
30