First Semester Jeopardy Category 1Category 2Category 3Category 4Category 5 10 20 30 40 50.
Florida Department of Education - School District of Palm ...SchoolsCategory in State...
47
Florida Department of Education Florida Department of Education Florida Department of Education Florida Department of Education Value Value‐added Model (VAM) added Model (VAM) FY2012 Using Student Growth in Teacher and School Based Administrator Evaluations
Transcript of Florida Department of Education - School District of Palm ...SchoolsCategory in State...
Florida Department of EducationFlorida Department of EducationFlorida Department of Education Florida Department of Education ValueValue‐‐added Model (VAM)added Model (VAM)
FY2012 Using Student Growth in Teacher andSchool Based Administrator Evaluations
HOW DID WE GET HERE?• Established by Student Success Act (SB 736)
Educator Evaluation System http://www.flsenate.gov/Committees/BillSummaries/2011/html/0736ED
• Teacher and School Administrator Evaluations– Professional PracticeProfessional Practice– Student Learning Growth
d h• FLDOE Student Growth Implementation Committee (SGIC) htt // fld / itt /http://www.fldoe.org/committees/sg.asp
Established by Student Success Act (SB 736)Teacher Evaluation System
Instructional
Teacher Evaluation System
2012 W i ht
InstructionalPractice (IP)
(4) Highly Effective2012 Weights
Teacher %/ %
(3) Effective(2) Needs Improvement
(1) UnsatisfactoryFinal RatingIP = 60%/SLG = 40%
AdministratorStudent Learning
Rating4‐HE3‐Eff2‐NI
IP = 50%/SLG = 50%g
Growth (SLG)(4) Highly Effective(3) Effective
1‐U
(3) Effective(2) Needs Improvement(1) Unsatisfactory
FY2012 Implementation
• Required by law• First time used in evaluations• Based only on FCAT 2.0 Reading/MathBased only on FCAT 2.0 Reading/Math• No Algebra 1 EOC student growth scoresN t h h l d i i t t ill i• No teacher or school administrator will receive a student growth rating less than Effective
IMPORTANT!IMPORTANT!IMPORTANT!IMPORTANT!It is important to note that measures used in theIt is important to note that measures used in the Florida School Grading system, student proficiency and learning gains are not a part ofproficiency and learning gains, are not a part of the Florida Value‐added Model of student learning growthlearning growth.
Student learning growth is based on the actual l FCAT 2 0 t t E if t d tscale score on FCAT 2.0 tests. Even if students
already score in Achievement Levels 3‐5, they till h tstill have room to grow.
Growth vs. Proficiency
(Progress) (Progress)
of academic how
points in
Growth vs. Proficiency
Growth (Progress) Proficiency (Status)Growth (Progress)Growth models measure the amount of academic
Proficiency (Status)A method for measuring how students perform atthe amount of academic
progress students make between two points in
how students perform at one point in time
between two points in time
Growth vs. Proficiency
500 500TEACHER 1 TEACHER 2
350
400
450
350
400
450Proficiency Proficiency
250
300
350
250
300
350Growth
100
150
200
100
150
200
0
50
100
0
50
100
Stu A Stu B Stu C Stu D Stu E
Prior Current
Stu F Stu G Stu H Stu I Stu J
Prior Current
Growth vs. Learning Gains260
257
240
250 Level 4Level 4251
243
248248 246246230
240 237
P fi i248248 246246220
Proficiency
Learning210
Learning Gain
200
Grade 6 Reading Grade 7 Reading
Growth vs. Learning Gains260
257
240
250 Level 4Level 4251
243
248248 246246230
240 237
P fi iG h248248 246246220
Proficiency
Learning
Growth
210Learning Gain
200
Grade 6 Reading Grade 7 Reading
A More Complete Picture of Student Learningof Student Learning
Growth ProficiencyGrowth> Compare student to own
prior performance
Proficiency> Compare student to
a standard&> Consider student characteristics
> Progress between points
> Does not consider student characteristics> Performance at a
& Progress between points
> Critical to student successpoint in time
> Critical to postsecondary opportunity
OverviewTeacher:
Establishes ScoreOverviewTeacher:
Aggregate of Students Taught Used to Rank
Teacher
S dUsed for EvaluationStudent
Growth
Evaluation
Establishes Score
School: Aggregate of d ll d
Used to Rank School
Students EnrolledUsed for Evaluation
VAM & Student Learning GrowthVAM & Student Learning Growth
• Contribution to a change in a student’sContribution to a change in a student s achievement on standardized test
• Calculated from a measure of student learning growth over timelearning growth over time
http://www.fldoe.org/committees/sg.asp
What is the Student Learning Growth Score?Student Learning Growth Score?
The difference between C t t t dCurrent test score and Predicted test score
What is the Predicted Student Score?Predicted Student Score?
What is the Predicted Student Score?Predicted Student Score?
Student score expectedbased on prior tests and other characteristicsother characteristics
FLDOE Value‐Added ModelVariables determining predicted scoreVariables determining predicted score
• Two or more years of prior achievement scores y p• Gifted status • Class size OTHER • Student Attendance (Days)• Mobility (number of transitions) CHARACTERISTICS?• Difference from modal age in grade (indicator of retention) Th b f bj l i hi h• The number of subject‐relevant courses in which the student is enrolled
• Homogeneity of entering test scores in the class• Homogeneity of entering test scores in the class
http://www.fldoe.org/committees/sg.asp
FLDOE Value‐Added ModelVariables determining predicted scoreVariables determining predicted score
• Two or more years of prior achievement scores y p• Gifted status • Class size • Student Attendance (Days)• Mobility (number of transitions) • Difference from modal age in grade (indicator of retention) Th b f bj l i hi h• The number of subject‐relevant courses in which the student is enrolled
• Homogeneity of entering test scores in the class• Homogeneity of entering test scores in the class
http://www.fldoe.org/committees/sg.asp
FLDOE Value‐Added ModelVariables determining predicted scoreVariables determining predicted score
• Students with Disabilities (SWD) status – Language impaired– Hearing impaired– Visually impaired
E ti l/b h i l– Emotional/behavioral– Specific learning disability– Dual sensory impaired– Autism spectrum disorder– Autism spectrum disorder– Traumatic brain injury– Other health impaired– Intellectual disabilityIntellectual disability
• English Language Learner (ELL) status – LY– LY
http://www.fldoe.org/committees/sg.asp
FLDOE Value‐Added Model
Student Success Act specifically excludes these student characteristicsexcludes these student characteristics
• GenderGender • Race
Ethnicity• Ethnicity• Socioeconomic status
What is the Student Learning Growth Score?Student Learning Growth Score?
Let’s take a look at the predicted score
What is the Predicted Student Score?900 1000…
Y12 FCAT
)TEST
700
800
(FY
Y axis is Current Year FCAT Score
RREN
T T
500 600
CU0 300
400
…10
0 200
X axis is Prior Year FCAT Score
PRIOR TEST
…100 200 300 400 500 600 700 800 900 1000…
(FY11 FCAT)
What is the Predicted Student Score?
900
Y12 FCAT
)900 1000…
TEST
800
700
(FY
700
800
Answer:Average of
RREN
T T 600
500
400
500 600
Average of current scores
of similar students
CU
400
300
0 300
400
of similar students
…10
0 200
PRIOR TEST (FY11 FCAT)
…100 200 300 400 500 600 700 800 900 1000…
Student Learning Growth is the Amount Above or Below Predicted ScoreAbove or Below Predicted Score
900
Y12 FCAT
)900 1000…
TEST
800
700PREDICTED SCORE
(FY
700
800
RREN
T T 600
500
400
PREDICTED SCORE600
500 600
CU
400
300
0 300
400
…10
0 200
PRIOR TEST (FY11 FCAT)
…100 200 300 400 500 600 700 800 900 1000…
Student Learning Growth is the Amount Above or Below Predicted ScoreAbove or Below Predicted Score
900
Y12 FCAT
)900 1000…
TEST
800
700PREDICTED SCORE
Positive Student Growth Score
ACTUAL SCORE(FY
700
800
RREN
T T 600
500
400
PREDICTED SCORE600
Negative Student Growth Score
ACTUAL SCORE
500 600
CU
400
300
ACTUAL SCORE
0 300
400
…10
0 200
PRIOR TEST (FY11 FCAT)
…100 200 300 400 500 600 700 800 900 1000…
Growth vs. Proficiency
500 500TEACHER 1 TEACHER 2
Did St d t B t Did Student J meet
350
400
450
350
400
450Did Student B meet or exceed the predicted score?
Did Student J meet or exceed the predicted score?
250
300
350
250
300
350
100
150
200
100
150
200
0
50
100
0
50
100
Stu A Stu B Stu C Stu D Stu E
Prior Current Predicted
Stu F Stu G Stu H Stu I Stu J
Prior Current Predicted
Student Growth
350 The differenceb t thResidual:
250
300between the current and predicted
Residual:The amount of growth by
Residual
200
250 predictedscores
represents the
g ya student
100
150p
value added by the teacher
0
50 and the school
Stu C
Prior Current Predicted
Determining the Student Learning Growth by School12
FCA
T)900 1000…
Example: All Grade 5
TEST
(FY
700
800
Reading Students in Florida
RREN
T T
500 600
The diagonal line is the predicted
CU
.
0 300
400
growth observed among similar
students
…10
0 200 students
PRIOR TEST DRE(FY11 FCAT)
…100 200 300 400 500 600 700 800 900 1000…
Determining the Student Learning Growth by School2 FCA
T)900 1000…
The difference
TEST
(FY1
2 700
800
between the predicted and actual scores is the growth.
RREN
T T
500 600
scores is the growth.
The average of the growth of students
CUR
0 300
400
growth of students within a school
produces the school
…10
0 200 score for a
subject/grade.
PRIOR TEST (FY11 FCAT)
…100 200 300 400 500 600 700 800 900 1000…
Student Growth Scores
Teacher VAM School Component
Student Growth ScoresTeacher/School
• State calculates scores for each grade d bj land subject separately
• Therefore, a teacher/school may have more than one scoremore than one score
Student Growth ScoresTeacher/SchoolTeacher/School
• Teacher of both 9th and 10th grade studentsg
d bjStudent Teacher
Grade Subject Count VAM 9 Reading 18 6.0410 Reading 16 0 08
Two Different10 Reading 16 ‐0.08Different Scores
Comparing Scores• Comparing scores across subjects/gradesFCAT 2 0 l h i i t i• FCAT 2.0 scale has inconsistencies12
Average Yearly Growth–FCAT 2.0 Reading
8
10
ain
g y g
6
ale Score Ga
2
4Sca
0
Grade 4 Grade 5 Grade 6 Grade 7 Grade 8 Grade 9 Grade 10
Common Score
• State creates scores on common scale• Example: Jane Doe teaches intensive reading
Grade SubjectStudent Count
Teacher VAM Common
SScore9 Reading 18 6.04 1.0710 Reading 16 0 08 0 0110 Reading 16 ‐0.08 ‐0.01
Common Score
• State creates scores on common scale• Example: Jane Doe teaches intensive reading
Grade SubjectStudent Count
Teacher VAM Common
SScore9 Reading 18 6.04 1.0710 Reading 16 0 08 0 0110 Reading 16 ‐0.08 ‐0.01
COMMON SCORE .052
Ranking Scores
90
100 87% ‐ 100% HIGHLY EFFECTIVE
70
80
Jane Doe
50
60 15% ‐ <87 % EFFECTIVE
30
40
10
20
2% ‐ <15% DEVELOPING
0
Category 1 Category 2 Category 3 Category 4Schools in State Teachers in State<2% UNSATISFACTORY
Jane Doe: Teacher Example
1 Jane Doe’s common score is 0 0521. Jane Doe s common score is 0.052
2. This score is ranked among all teachers in the state with a Reading score
3 This gives Jane a percent rank of 68 48 when3. This gives Jane a percent rank of 68.48 when compared to all other teachers in the State
4. This percent rank translates to “Effective”
Percent Ranks and RatingsPercent Ranks and Ratings
• State generates a percent rank for eachState generates a percent rank for each teacher, school, district–Reading score is ranked among all Reading scores in–Reading score is ranked among all Reading scores in the state (Grades 4‐10)
–Math score is ranked among allMath scores in theMath score is ranked among allMath scores in the state (Grades 4‐8, No Algebra)
– Reading + Math score is ranked among Reading + g g gMath scores in the state
• Percent ranks are placed on the District Student pGrowth Rating Scale
District Evaluation SystemImplementation PlanImplementation Plan
• Teachers– FCAT Teachers use teacher VAM score for students of grade/subjects taughtNon FCAT Teachers use combined reading/math– Non‐FCAT Teachers use combined reading/math school score
– Teacher assigned to 2 schools, use weighted average /of 2 schools combined reading/math scores
– Teacher assigned to more than 2 schools, use District reading/math score combinedreading/math score combined
• Administrators– Principals & Assistant Principals use combinedPrincipals & Assistant Principals use combined reading/math school score
Established by Student Success Act (SB 736)Teacher Evaluation System
Instructional
Teacher Evaluation System
2012 W i ht
InstructionalPractice (IP)
(4) Highly Effective2012 Weights
Teacher %/ %
(3) Effective(2) Needs Improvement
(1) UnsatisfactoryFinal RatingIP = 60%/SLG = 40%
AdministratorStudent Learning
Rating4‐HE3‐Eff2‐NI
IP = 50%/SLG = 50%g
Growth (SLG)(4) Highly Effective(3) Effective
1‐U
(3) Effective(2) Needs Improvement(1) Unsatisfactory
Final Teacher Evaluation Rating
Student Learning Growth (40%)(40%)
1 2 3 4)
tiona
l (60
%) 1 1 1.4 1.8 2.2
2 1.6 2 2.4 2.8Hold Harmless
Instruc
ractice
3 2.2 2.6 3 3.4Hold HarmlessI P 4 2.8 3.2 3.6 4
WEIGHTED AVERAGE
Harmless
WEIGHTED‐AVERAGEHE Eff NI U
3.2 ‐ 4.0 2.1 ‐ 3.1 1.2 ‐ 2.0 1.0 ‐ 1.1
Final Administrator Evaluation Rating
Student Learning Growth (50%)
actice
1 2 3 4
onal Pra
50%) 1 1 1.5 2 2.5
2 1.5 2 2.5 3
ofessio (5 2 1.5 2 2.5 3
3 2 2.5 3 3.5
WEIGHTED AVERAGE
Pro
4 2.5 3 3.5 4
WEIGHTED‐AVERAGEHE Eff NI U
3.2 ‐ 4.0 2.1 ‐ 3.1 1.2 ‐ 2.0 1.0 ‐ 1.1
Sample School Report
Sample teacher reportSample teacher report
Sample Teacher Report
Sample Student Report
ResourcesResources• Research & Evaluation Website (PPT)
• FLDOE Resources on Student Growth– State Educator Evaluation System (Video)y ( )
– Value‐Added Model White Paper (Word, 841KB)
– Value‐Added Model Technical Report (Word, 601KB) p ( , )
– Presentation on the Value‐Added Model (PDF, 103KB)
– FDOE ‐ VAM Course Codes used in Value‐Added Model
• Oak Tree Analogy of Value‐added
CONTACTStudent Growth (VAM)Mark Howard, Director
Research, Evaluation and Assessment,[email protected]
561‐434‐8781
Educator Evaluations Kathy Orloff, Director
Professional [email protected]
561‐776‐3696
Ed E l iEducator EvaluationsDianne Wyatt, Manager
Human Resources/Professional Developmentdiane wyatt@palmbeachschools [email protected]
561‐366‐6112
