Bryk may 2014 using NICs to tackle practical problems in education
-
Upload
learningemerg -
Category
Education
-
view
189 -
download
0
description
Transcript of Bryk may 2014 using NICs to tackle practical problems in education
![Page 1: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/1.jpg)
Anthony S. Bryk Master Class, University of Bristol
![Page 2: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/2.jpg)
2
Triple Aims of Educational Improvement
EFFICIENCY
EFFECTIVENESS
ENGAGEMENT
Be0er Use of Resources
Ambi<ous Learning For All Students
More Relevance
![Page 3: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/3.jpg)
How We Work Now: Tower of Babel Problem
3
![Page 4: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/4.jpg)
The Educational R&D Problem
• Accelerate Improvement Efforts
• Aim for Quality, Reliably at Scale
4
![Page 5: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/5.jpg)
How We Are Working on This
• Analogical Scavengers—The Gawande Inspiration
• Learning by Doing—Can we actually make the ideas work?
• Engaging a Larger Community
![Page 6: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/6.jpg)
An Inspiration: Improvement Science in Healthcare Protecting 5 Million from Harm,
Saving 100,000 Lives
![Page 7: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/7.jpg)
7
We can accomplish more together, than
even the best of us can do alone.
Complex systems problems that we now seek to solve
Power of Networks
![Page 8: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/8.jpg)
Networked Improvement Communities: What are they?
Integrating Two Big Ideas:
• The discipline of Improvement Science
joined to
• The Power of Networks
Accelera'ng Learning in and through Prac'ce to Improve
![Page 9: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/9.jpg)
Six Principles Guide the Work (plus useful tools to scaffold the activity)
9
Taken Together:
• Disciplined Inquiry
• Rudiments a scientific community
• Aim: systematic practice improvement
![Page 10: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/10.jpg)
I. Problem-‐ & User-‐Centered
• What is the specific problem we’re trying to solve?
• What we tend to do now: a general issue comes into view and we jump on solu<ons
![Page 11: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/11.jpg)
60-‐70% Students assigned to developmental math
course.
80% Percent of these
students that never get past this gate.
500,000 students
in every cohort will never complete college math
requirement.
11
The Problem
![Page 12: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/12.jpg)
A Solu<on Framework: Integrated Pathways
12
Through college-‐level sta5s5cs
“To-and-through” college-level quantitative reasoning
Two 1-year pathways “to and through college math”
1
2
![Page 13: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/13.jpg)
II. Varia5on in Performance is the problem to solve
• “What Works” is typically the wrong ques<on • Real Issue: Quality Improvement Ques<on
“How to advance effec<veness among diverse faculty engaging varied popula<ons of students and working in different organiza<onal contexts?”
• Goal: Achieve efficacy with reliability at scale
![Page 14: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/14.jpg)
Trad
ition
al S
eque
nce
Stat
way
Effects: Time to Complete a College Level Math Course 1 Year 2 Years
Triple the
success rate in half the time.
6%
51%
15%
![Page 15: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/15.jpg)
What is Next?
• Normal Course of Events: “It Works” – Tout success – Publish results – Hope others pick this up – Go onto our next project
![Page 16: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/16.jpg)
Varia<on in Pathways Success Rates by College (n=19)
16
1
2 3
4 5
6 7
8
9
11
12 13
14
15
17
18 19
-50%
0%
50%
100%
0% 50% 100%
Stat
way
Stu
dent
s
Non-Statway Matched Comparisons
No improvement line
We also have a failure, why? What can we learn?
Triple success rate line
![Page 17: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/17.jpg)
III. See the System to Improve it
• Put simply: It is hard to improve what we do not fully understand.
![Page 18: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/18.jpg)
How Do We Heal Medicine? Atul Gawande April, 2012
![Page 19: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/19.jpg)
![Page 20: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/20.jpg)
Gawande’s Closing Observation
Making systems work is the great task of my generation of physicians and scientists. But I would go further and say that making systems work — whether in healthcare, education, climate change, making a pathway out of poverty — is the great task of our generation as a whole.
![Page 21: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/21.jpg)
The Invisible Complexity Schooling
21
The Invisible Complexity of Schooling
![Page 22: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/22.jpg)
60-‐70% Students assigned to developmental math
course.
80% Percent of these
students that never get past this gate.
500,000 students
in every cohort will never complete college math
requirement.
22
Returning to The Presen<ng Problem
![Page 23: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/23.jpg)
The Orien<ng Problem
Extraordinarily high failure rates among students assigned to developmental math instruc<on
Consolidate the courses into a 1-‐year pathway
Real world problems from sta<s<cs as the organizer
Psycho-‐social interven<ons aimed at “produc<ve persistence”
Rapid analy<cs capacity
Faculty development
Causal Systems Analysis: Why do we con<nue to get the outcomes observed?
Primary Causes for High Failure Rates
Organizing Improvement Hypotheses
event
???
![Page 24: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/24.jpg)
Ins$tu$onal*structures*don’t*support*student*success*
Students*are*not*engaged*or*mo$vated*
The*course*material/content*is*problema$c*
Instructors*lack*skills*and*beliefs*that*students*can*succeed*
State*policy*does*not*support*student*success*
Low*success*rates*in*development*
math*func$on*as*a*gatekeeper*to*opportunity*
Students*lack*the*skills*to*succeed*
Ineffec$ve*advising*system*
Ineffec$ve*learning*support*services*
Lack*of*social*$es*to*each*other*and*to*their*instructors*
Inaccurate*placement*
High*dropAout*rates*between*courses*
Don’t*believe*they*can*learn*
Math*and*tes$ng*anxiety*
Poorly*prepared*mathema$cally*
They*don’t*know*how*to*study*math*
They*don’t*know*how*to*“navigate”*the*college*world*
Not*interes$ng*or*relevant**
Not*seen*as*useful*
Text*is*inaccessible*
Does*not*leverage*what*we*know*about*how*students*learn*
Too*many*courses*in*the*developmental*sequence*
Few*opportuni$es*to*learn*from*others*
Lack*knowledge*of*learning*theory*
Weak*pedagogy*
Don’t*believe*suppor$ng**student*success*is*their*job*
Lower*reimbursement*for*developmental*math*
Tradi$onal*transfer*requirements*impede*innova$on*
Arcane*curricular*topics*create*needless*hurdles*
Funding*based*on*enrollment*rather*than*outcomes*
OOen*taught*by*adjunct*faculty**with*liPle*professional*support*
![Page 25: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/25.jpg)
The Orien<ng Problem
Embedded literacy and language barriers
Extraordinarily high failure rates among students assigned to developmental math instruc<on
Lose large # of students at the transi<ons
Consolidate the courses into a 1-‐year pathway
Students mindsets undermine success
Real world problems from sta<s<cs as the organizer
Students “gone” before we know it
Psycho-‐social interven<ons aimed at “produc<ve persistence”
Rapid analy<cs capacity
Course material and instruc<on are not engaging
Faculty development
Analy<c Summary of Causal Systems Analysis
Primary Causes for High Failure Rates
Organizing Improvement Hypotheses
Eventually leads to a “Pathways Strategy”
![Page 26: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/26.jpg)
Pathways Driver Diagram: Organizing a Networked Improvement Community
Aim: increase from 5% to 50%, students achieving college math credit within one year of continuous enrollment
Instructional System: Organized around productive struggle, explicit connections, and
deliberate practice.
Productive Persistence:
Students develop skills and maintain positive mindsets
Language and Literacy: Students
use language in understanding
problems, reason mathematically, and communicate results
Advancing Teaching: Effective
teaching within 2 years of
implementation
Reduce transitions + assure enrollment across semesters
Deliberate focus on “Starting Strong”
Promote students’ ties to peers, faculty,
pathway
Math that matters: students see material interesting, relevant
Enhance faculty’s beliefs and relational
practices
Opening lessons engage interest, assure early
success
Direct interventions to influence student mindsets
Real-time data tracking on student engagement
Detail supportive classroom norms and social
connections
Professional development on “Starting Strong”
A Community Explicates its Causal Thinking:
A Community Explicates its Causal Thinking:
A Driver Diagram
to Organize Its Major Improvement Hypotheses
![Page 27: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/27.jpg)
Pathways Driver Diagram: Organizing a Networked Improvement Community
Aim: increase from 5% to 50%, students achieving college math credit within one year of continuous enrollment
Instructional System: Organized around productive struggle, explicit connections, and
deliberate practice.
Productive Persistence:
Students develop and maintain
positive mindsets
Language and Literacy: Students
use language in understanding
problems, reason mathematically, and communicate results
Advancing Teaching: Effective teaching within 2
years of implementation
Reduce transitions + assure enrollment across semesters
Deliberate focus on “Starting Strong”
Promote students’ ties to peers, faculty,
pathway
Math that matters: students see material interesting, relevant
Enhance faculty’s beliefs and relational
practices
Opening lessons engage interest, assure early
success
Direct interventions to influence student mindsets
Real-time data tracking on student engagement
Detail supportive classroom norms and social
connections
Professional development on “Starting Strong”
Elabora<ng Out The Driver Diagram
Produc<ve Persistence
![Page 28: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/28.jpg)
IV. You cannot improve at scale what you cannot measure
• Measureable targets: “Some is not a number; soon is not a <me”-‐-‐Valued outcome measures – But, you just can not stand at the end of the line.
• We need process measures <ed to intermediate targets.
![Page 29: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/29.jpg)
Produc<ve Persistence
Suppor<ve social rela<onships
Target: How do we measure it?
Mindsets about the value of math
Mindsets about poten<al to learn
math
Anxiety Regula<on
Study Skills Conceptual Task: reduce to 5 core ideas focus on underlying malleable causes + change evidence
Prac5cal Measurement:
reduce 900 items to 26 “you have 3 minutes”
![Page 30: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/30.jpg)
V. Accelerate Improvement: Embrace Disciplined Inquiry
• Policy Romance of the Silver Bullet – Move quickly to large scale implementa<on, but…
• We typically don’t know whether: – We can make these ideas work at all; – We have capacity and will to execute with efficacy at
scale.
• Instead, a DEED orienta<on – Quick, minimally intrusive, an empirical warrant – Mantra: Learn Fast, Fail Fast, Improve Fast!
![Page 31: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/31.jpg)
A System of Social Learning to Improve
Transla5onal Research
Interven5ons (Alpha Labs)
Will they work for community college students, and if so,
how?
Expert Prac55oner Knowledge (Subnet)
Building robust clinical
knowledge about effec<ve materials and instruc<onal prac<ces.
Learning from Network Data (Hub Analy5cs)
Learning from observed variability.
Discerning the unseen.
![Page 32: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/32.jpg)
32
Transla5onal Research
Interven5ons (Alpha Labs)
• Will they work for community college students, and if so,
how?
![Page 33: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/33.jpg)
Initial Alpha Lab: Mindset Intervention
• A carefully designed experimental intervention has changed student mindsets.
• But just because an intervention can work in one setting does not mean it will work in another.
• Need to engineer it to “fit” in instructional contexts. – Conduct rapid R&D using DEED methodology. – “Smell testing” – 4 months from small-scale test to larger scale use.
![Page 34: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/34.jpg)
Rapid Iterative DEED cycles
• Research-Practitioner Team
• Testing – Small double-blind randomized
trial in Algebra course (n = 26)
– Larger double-blind experiment (n = 288)
• Introduce to faculty network, carefully study emerging results, continue to revise, refine, and extend.
34
Roberta Carew, Statway faculty
Valencia College
![Page 35: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/35.jpg)
35
Learning from Network Data (Hub Analy5cs)
• Learning from observed
variability. Discerning the “unseen.”
![Page 36: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/36.jpg)
36
!
1. Assessing Change: Initial Evidence of Efficacy of Starting Strong Package
![Page 37: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/37.jpg)
2. Predictive Analytics—targeting support
(a simple at-risk indicator scoring 5 key items/item clusters-day 1)
37
% of who failed the end-‐of-‐term common assessment
![Page 38: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/38.jpg)
Connections to Stereotype Threat
12% 13% 14%
28%
40%
7% 11% 14%
50%
71%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Never Hardly Ever Some<mes Frequently Always
Pathways Drop
out
All students Black students
“How oqen, if ever, do you wonder: ‘Maybe I don't belong here?’”
N = 714 math students
![Page 39: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/39.jpg)
39
Expert Prac55oner Knowledge (Subnet)
Building robust clinical
knowledge about effec<ve instruc<onal materials and prac<ces.
![Page 40: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/40.jpg)
PDSA Cycle: Rapid, Small Experimental Trials
PLAN DO
ACT STUDY
The Three Ques5ons: • What specifically are we trying to accomplish?
• What change might we introduce?
• How will we know that the changes are an improvement?
![Page 41: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/41.jpg)
Improving Instructional Routines in Support of Productive Persistence: PDSA Cycles
• Faculty routines and email scripts re: absent students
• Student group noticing routine
• Effective scaffolding for group roles (rich problems)
41
![Page 42: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/42.jpg)
Sample Run Chart for a PDSA Cycle (Student Group Noticing Routine)
0%#
20%#
40%#
60%#
80%#
100%#
120%#
1/14/1
3#
1/21/1
3#
1/28/1
3#
2/4/13
#
2/11/1
3#
2/18/1
3#
2/25/1
3#
3/4/13
#
3/11/1
3#
3/18/1
3#
3/25/1
3#
4/1/13
#
4/8/13
#
4/15/1
3#
4/22/1
3#
4/29/1
3#
Perce
nt'of
'Stud
ent'in'A
/end
ance'
A/endance'(By'Day)'
Typical#a4endance#
Observed#a4endance#
n=44#Median:#0.85#
Median#
![Page 43: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/43.jpg)
Developing a Quality Process Reliably at Scale
Develop A Change
Test under mul<ple condi<ons
Test under increasingly varied
condi<ons
Make the change permanent
Ini5al Hunches
System Changes
1 school 1 administrator
5 schools Many administrators
En<re ver<cal team A more diverse group of administrators
District Wide All administrators
Seeing Task Complexity
Seeing Organiza<onal Complexity
Learning to improve feedback conversa<ons between
principals and new teachers PLAN DO
ACT STUDY
![Page 44: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/44.jpg)
A Developmental Dynamic
Hunches Theories Ideas
Ini<a<ng Resources
P D
S A
P D
S A
P D
S A P D
S A
Moving out toward More diverse condi<ons: “factor of 5 rule of thumb”
Aiming for Efficacy with Reliability at
Scale
![Page 45: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/45.jpg)
VI. Accelerate Improvement: Tap the Power of Networks
• A source of innova<on – Dig into the details: what worked, how, for whom? – Can we adap<vely integrate this into other contexts?
• Mul<ple fast replica<on – Can we make this happen with efficacy, reliably at scale?
• Innova<on diffusion—it is largely about who is connected to whom and what they think and do
A Learning Educa'onal System
![Page 46: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/46.jpg)
A A
Improvement Networks: Accelerate Learning in Prac<ce for Improvement
A
B
A A A B
A A A B
A A A B
C
(Englebart,1994)
![Page 47: Bryk may 2014 using NICs to tackle practical problems in education](https://reader034.fdocuments.in/reader034/viewer/2022051609/5479fb61b37959a22b8b4832/html5/thumbnails/47.jpg)
It is all about accelerating how we learn in and through practice to improve.