Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio...
-
Upload
marjorie-williamson -
Category
Documents
-
view
221 -
download
0
Transcript of Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio...
![Page 1: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/1.jpg)
Quality data, sound judgements,powerful learning.
Sara Glover – Executive Director, Portfolio CoordinationStephen Fraser – Data, Outcomes and EvaluationOffice for Children and Portfolio Coordination
2010 NMR ConferenceMantra Erskine Beach Resort, Lorne,May 31st – June 1st 2010
![Page 2: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/2.jpg)
The essence of accountability is looking forward, using data to inform judgments about current performance and formulate plans for reasonable actions.
(Earl and Katz, 2006)
![Page 3: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/3.jpg)
Outline of the institute
• Working with data• Building a data story
• Break
• Building a data story contd.
• Leading change – a process to construct change theories• Engaging stakeholders• Monitoring impact
![Page 4: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/4.jpg)
A framework for thinking and action
Engagement
AnalysisPlanning
and action
Monitoring progress
Knowledge sharing and
capacity building
![Page 5: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/5.jpg)
School Level Report
Attitudes to SchoolStaff Opinion Parent Opinion
VCE Data Service (online by VCAA)
NAPLAN Data Service (online by VCAA)
School Performance SummaryPercentile/SFO Report
![Page 6: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/6.jpg)
• Build data literacy across your school – data managers, data teams
• You are not your data.
• Data is half the story; context is the other half.
• Look for the main stories. Don’t get bogged down in the detail and miss the main stories.
• Don’t react to any one particular data set, for any one particular year. Always look at a range of data sets over time. Look for a consistent story.
• Beware of small numbers (e.g. in small schools). Use the raw data (number of students) or aggregate statistics (e.g. over several years).
Working with data
![Page 7: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/7.jpg)
• Absolute scores (usually shown as vertical blue columns, e.g., mean, % “C”) are useful for;– knowing how you compare to some standard (e.g. the VELS),– monitoring over time (i.e. are the scores increasing, declining or showing
no clear pattern) and– specifying targets.
• Percentiles (usually shown as horizontal orange bars) are useful for;– comparing your own performance to other schools, and – identifying strengths and weaknesses.
Interpretation Advice (… continued)
![Page 8: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/8.jpg)
Always remember how the data are constructed
1
1.5
2
2.5
3
3.5
4
4.5
52
00
7
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
20
07
20
08
20
09
Yr 5 Yr 6 Yr 7 Yr 8 Yr 9 Yr 10 Yr 11 Yr 12
Teacher Effectiveness
1=
stro
nlg
y d
isa
gre
e &
5=
stro
ng
ly a
gre
e
0th percentile
100th percentile
10th percentile
90th percentile
50th percentile
![Page 9: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/9.jpg)
Targeting low-SES students vs Targeting low performing students Source: Masters (2009) using PISA data
![Page 10: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/10.jpg)
A comparison of NAPLAN scores for low SES students and all students
0%
2%
4%
6%
3
127
181
219
250
277
302
324
346
367
387
408
429
452
477
506
544
608
NAPLAN scaled score
pe
r ce
nt
All
Low SES*
* "low SES" defined as occupation of parent is … machine operator, hospitality staff, assistant, labourer or related worker, or not in paid work in last 12 months
![Page 11: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/11.jpg)
Leading change: using data to engage people in the change process
• Build a data story• Develop a focus• Dig into the detail• Examine practice – what are we doing now – how do we know if it’s
working?• Develop a change theory – what would happen if we did things differently?• Engage stakeholders• Do the work• Monitor impact
![Page 12: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/12.jpg)
Building a data story along a developmental continuum
Engagement
AnalysisPlanning
and action
Monitoring progress
![Page 13: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/13.jpg)
Kindergarten Participation in our community
Early Childhood Community Profile
Kindergarten Participation rate for 4 year olds across the LGA: 91%
• Less than the metropolitan average• Less than the rural average
![Page 14: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/14.jpg)
Children’s development across our community (AEDI)
Developmental Vulnerability• Physical Health and
Wellbeing: 9%• Social Competence: 10%• Emotional Maturity: 8%• Language and Cognitive:
13%• Communication and General
Knowledge: 24%
![Page 15: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/15.jpg)
Early Years Reading
English Online Interview• High proportions of
students in the lower bands
Assessment of Reading Prep – Year 2
• - declining Prep outcomes (improving towards Year 2)
![Page 16: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/16.jpg)
Reading Prep to Year 6 - VELS
• Mismatch with EOI and with AoR• Greater distribution of outcomes towards Year 6 – more at the top,
more at the tail
![Page 17: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/17.jpg)
Reading Prep to Year 6 - NAPLAN
• Improvement over the last two years• Greater distribution of outcomes towards Year 6 – more at the top, more
at the tail (consistent with VELS)
![Page 18: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/18.jpg)
Reading Year 7 to 10 - VELS
• Decline over three year period
• Greater distribution of outcomes towards Year 10 – more at the top, more at the tail (consistent with Prep to Year 6)
![Page 19: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/19.jpg)
Numeracy Year 7 to 10 - VELS
• Mixed over three year period
• Improvement at Year 10
![Page 20: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/20.jpg)
Reading Year 7 to 10 - NAPLAN
• Improvement over last two years• Greater proportions of students in the lowest two bands by Year 10
(somewhat consistent with VELS)
Year 7 Reading Year 9 Reading
![Page 21: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/21.jpg)
Numeracy Year 7 to 10 - NAPLAN
• Improvement over last two years• Stronger than Literacy• Lower proportions of students in the lowest two bands by Year 10
(somewhat consistent with VELS)
Year 7 Numeracy Year 9 Numeracy
![Page 22: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/22.jpg)
Post-Compulsory Outcomes - VCE
• Decline in English• Increase in Maths (very strong)
• Overall (allstudy) steady improvement
![Page 23: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/23.jpg)
Post-Compulsory Outcomes - VET
• High participation• High completion
![Page 24: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/24.jpg)
Post-Compulsory Outcomes - VCAL
• Low participation• High completion
![Page 25: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/25.jpg)
Transitions and Pathways
• Excellent and improving retention across the board
7-10
10-11
11-12
7-12
![Page 26: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/26.jpg)
Transitions and Pathways
• Overall – excellent proportions of students continuing with their education or training
• Declining University enrolments.
![Page 27: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/27.jpg)
Engagement and Wellbeing
Strengths• Motivated students• Positive teacher-learner
relationships
Opportunities• Stimulating learning – high
demand content – lacking• Safety an issue link with
distress
• Attendance excellent.
![Page 28: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/28.jpg)
Climate
Strengths• High Expectations• High motivation and
goal alignment
Opportunities• Collegiality and decision
making
![Page 29: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/29.jpg)
Building a data story
• We have lower kinder participation rates• We have vulnerability in our community – language/general
knowledge/communications• Literacy a challenge from the outset• We are seeing a decline in literacy outcomes at most transition points• Strong mathematics performance• We have very strong retention and pathways through to further education
and training, with declining University enrolment.
• We have engaged and motivated students – is the level of task challenge lacking?
• We have motivated teachers with high expectations• Student and staff alike have a lack of say
Out
com
esE
xper
ienc
es
![Page 30: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/30.jpg)
Dig into the detail
• What can we learn about mathematics instruction in the Secondary years – about the students undertaking further maths in VCE
• NAPLAN Data Service – Item Analysis Report (SREAMS SPA).
![Page 31: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/31.jpg)
Developing the plan…examining practice
Engagement
AnalysisPlanning
and action
Monitoring progress
![Page 32: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/32.jpg)
Examine practice – what are we doing now
• What are our relationships with kindergartens like?• What are our transition arrangements/plans like? Are teachers getting
the right information about literacy at the start of each year?• What level of involvement do teachers and students have in decisions
that affect them?
![Page 33: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/33.jpg)
Developing a change theory
• Deciding what to do.
• The most important, but often overlooked aspect of these theories is that they should be falsifiable – that is, they can be proven false.
If we do …x , then we expect to see …y.
![Page 34: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/34.jpg)
Monitoring progress…communicating successes and challenges against what we set out to do
Engagement
AnalysisPlanning
and action
Monitoring progress
![Page 35: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/35.jpg)
Monitoring Impact
• If the data doesn’t really capture what you’re trying to achieve in your school, target on the behaviours that will.
Behaviour Impact Data (if any)
If …
then …
If …
then …
If …
then …
If … x
then … y
![Page 36: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/36.jpg)
Engaging stakeholders – building ownership
• Thinking about the traction achieved by engaging stakeholders vs going it alone
• A question of how to harness the enthusiasm/effort/ownership of stakeholders at all levels
• And what information/communication tools can be used to do so
![Page 37: Quality data, sound judgements, powerful learning. Sara Glover – Executive Director, Portfolio Coordination Stephen Fraser – Data, Outcomes and Evaluation.](https://reader035.fdocuments.in/reader035/viewer/2022062309/56649e9d5503460f94b9e502/html5/thumbnails/37.jpg)
Engaging stakeholders
Key Messages Communication Tool
School Leaders
Classroom teachers
Parents
Students