Symposium Overview Bill Penuel SRI International.

Symposium Overview

Bill PenuelSRI International

Complexity and Education

August 2005: Gathering at SFI of scholars in education inspired by how concepts of complex adaptive systems might be applied to education

Two areas of study had already begun to develop– Complexity in education: Helping K-12 students

understand complexity– Complexity of education: Studying and modeling

educational systems as complex systems Our symposium focuses on complexity in educational

systems– Applications are nascent– We are newcomers to the community– We come bearing theory and preliminary models

Theoretical Frameworks

Social capital theory – Focus on resources and expertise that individuals

access through ties to others– Resources and expertise are embedded in networks

New institutionalism– Isomorphism: Emergence of collaborative

arrangements as an adaptation to outside pressure– Norms within institutional (professional) fields that

are constituted (and sometimes transformed) in local interaction

Complex systems theory– Notion of making explicit assumptions so that

systems can be modeled– Application to “systemic reform” efforts

Models and Methods

Penuel and Riel: Integrating social network analysis into multi-level models of school change processes

Joshi: Using shadowing data to examine resources exchanged in interaction and uncovering institutionalized norms

DiBello: Using simulations to help school districts understand the costs of time away from educationally relevant tasks

Sabelli: Analyzing system levels toward developing approaches to cross-case analysis

Agency and Timescales in Education

Within education, there are agents who act at different levels of the system– Students: Classroom, Home, Out-of-School Time– Teachers: Classroom and school– Building leaders: School, district– District administrators: School, district– State and federal policy makers: State, federal

agencies– Reform intermediaries: All levels

Institutional change happens on different timescales– Age-graded classroom: 100 years– Data-driven decision-making: 2 years– Adopting a new curriculum: days, months

Applications

Common framework for analyzing social capital in schools– Gathering and interpreting network data– Common data sets

Knowledge building in educational research– Unpacking “professional community”– Cross-case analyses of schools (common language)– Agent-based simulations for researchers

Agent-based simulations for school leaders– What-if scenarios for orchestrating collaboration– Enabling leaders to understand schools and

districts as complex, multiscale systems

Teacher Networks and the Diffusion of Innovations

Bill Penuel and Margaret RielSRI International

Ken Frank and Ann KrauseMichigan State University

A Network Approach to Social Capital

Drawing on Portes (1998) and Lin (2001), we define social capital in terms of:

Ties: Interactions among faculty members in a school Resources and Expertise: The value of resources (e.g.,

curriculum) and expertise (e.g., wisdom of practice) accessible through ties to others

This definition considers social capital as useful for individual action, and secondarily as a social or collective resource.

Diffusion of innovations is an emergent characteristic of the school, which is facilitated by teacher talk and sharing of resources about teaching.

For us, analyzing networks is essential to measuring social capital:

Mapping the boundaries of networks and subgroups within networks

Including as part of our models the resources and expertise one can access through those networks

Considering the consequences in terms of changes in teacher attitudes, teaching practice, and student achievement

Several scholars have suggested that social capital has a network structure (e.g., Burt, 2000; Lin, 2001), in that valued resources are embedded within a network


Prior research– Functions of network closure

• Resources and expertise flow freely within dense networks

• Can protect a network from outside pressure– Functions of bridging

• Critical source of new knowledge and skill• People who play bridging functions can exert

considerable control over the flow of resources

Implications– Need to attend to network boundaries– Boundaries exist within and across schools


A Crystallized Sociogram

Resources, Expertise, and Consequences

Approaches to conceptualizing resources:– Access to instructional materials– Affordances of particular instructional materials (Stein &

Kim, 2006)– Resources-in-use (Cohen, Raudenbush, & Ball, 2003)– Schoolwide norms (Bryk & Schneider, 2002)

Approaches to conceptualizing expertise:– Prior experience with a reform or activity– Adaptive expertise framework (Bransford, Crawford)– Pedagogical Content Knowledge (Ball, Hill)– Formal Preparation (production-function literature)– Teacher experience*

Candidate consequences:– Teaching quality– Curriculum or reform implementation– Student achievement

Our Research

Big Questions: How do interactions with colleagues affect teachers’

beliefs and practices? What patterns of expertise flow in a school promote

the diffusion of innovations across a school? How do informal interactions combine with

professional development and intentional efforts to promote teacher collaboration?

How do teachers choose with whom to interact around practice?

Study of Schoolwide Reforms

Looking at different “home-grown” school-wide reform initiatives– Technology integration– Literacy– Data-driven decision making– Creating standards-aligned assessments

Measured social capital and self-reported influence on practice at two points in time (Penuel, Frank, & Krause, 2006)– We used “implementation levels” at Time 1 as an

indicator of expertise– We looked at how getting help from an “expert”

influenced teachers reports that their school’s initiative influenced their practice

Sample Characteristics

Schools– 23 schools selected for commitment to collaboration and

engaged in whole school reform– 13 elementary; 3 K-8; 5 middle; 2 high schools– 8 high SES <10% free/reduced price lunch– 3 low SES >80% free/reduced price lunch

Teachers– 499 teachers (with matched data for both surveys)– teaching experience: 125 <5 years; 236 6-15 years; 190

teachers 16+ years School Leaders

– 29 informal leaders from 14 of the schools: designated coaches for schoolwide initiatives; generally team or grade-level leaders

Two Analyses for Today

Case studies of two schools’ reforms– A look at the role of critical role of between-

subgroup dynamics in a school with respect to flows of resources and expertise

Hierarchical linear model looking at what predicts change in instructional practice– A look at between and within-subgroup

dynamics and their influence on teachers’ attitudes

Glade and Crosswinds

Two schools in California’s II/USP Program Similar demographics and challenges in improving

literacy outcomes for English Language Learners Adopted similar approaches to reform: Promoting

teacher community facilitated by instructional coaches

Coaches were expected to play slightly different roles Two schools met with dramatically different results:

– Glade: Still struggling for reform to take hold and gain legitimacy

– Crosswinds: Steady gains in achievement, strong shared commitment to reform goals

Comparative case study analysis set out to test rival explanations (Yin, 2003) for why

Formal Meetings at Glade and Crosswinds

Density of Interaction

Glade’s Subgroups

Veteran Group

New Immigrant Teacher Group

Crosswinds’ Subgroups

Early Elementary Group

Access through Ties to Expertise

Size and color indicate extent of use at time 1

A

B

C

D

provides help to

provides help

to

provides help

to

HLM Analyses of Innovation Diffusion

Models take into account expertise of colleagues with whom teachers interact

Three levels to model– School– Subgroup– Individual teacher

Access through Ties to Expertise

Change in A is a function of interaction with people with expertise; the greater the mean expertise of Helpers B, C, and D, the greater the change in A

B

C

D

provides help to

provides help

to

provides help to

A

Ripple around A indicates increasein use between time 1 and time 2

3-Level Model of Change in Practice

Discussion

The case study results point to the importance of the informal network in supporting or inhibiting the flow of resources and expertise across subgroups

Relationship to school (i.e., perceived collective responsibility) is filtered by experience within subgroup (i.e., fit with subgroup).

The HLM analyses are suggestive of a way that between and within-subgroup dynamics can induce changes in teachers’ practices.

The Role of Institutionalized Norms of Autonomy and Equality in Shaping Interactions of Teachers

Aasha JoshiWilliam R. PenuelSRI International

Institutionalized Norms

Norms emerge out of interactions “Specify how things should be done…designate

appropriate ways to pursue them [goals or objectives]” (Scott, 2001, p55)

Equality as a Norm in Schools

Network configuration– Many ties within and across subgroups– No single leader

Direction of help– Bias against seeking and giving help

Interaction structure– Collaborative “war” stories– Experience-swapping– Simply ignore reform (don’t believe there is anything to

learn) Focus of talk

– Common challenges and struggles (e.g., students)– Shared tasks

Autonomy as a Norm in Schools

Network configuration– Limited interaction among colleagues (Lortie, 1977)

Direction of help– Limited evidence of any kind of helping interaction

Interaction structure– Experience-swapping– Collaboration as distribution of responsibility for

tasks Focus of talk

– Talk about broad principles (e.g., standards) but not much about teaching

Creating Disequilibrium: Efforts to Create Teacher Community

Network configuration– Informal and formal leaders emerge in bridging

roles– Teachers also perform bridging roles to colleagues

Direction of help– Creates press to give and receive help

Interaction structure– Apprenticeship learning (e.g., model teaching)

Focus of talk– On teaching and on challenging and critiquing

peers’ practice

Shadowing as a Way to Study Transformation of Norms in

Interaction The shadowing task:

– Single researcher followed shadowees from arrival to the school until the end of their work day; debrief interview following the shadowing day

Documentation: – Duration– physical location (e.g., staff room, hallway, or telephone)– intentionality (e.g., scheduled or impromptu)– Topics– Participants – Comments by the participants during the interactions

Data set: – 6 schools– 14 teachers– 6 school leaders– Coded: 798 interactions

Case Study Schools

Goals and Key Strategies of Reforms in Sample Schools

School Goals Key Strategies School 1 Improving reading achievement,

especially for English Language Learners

Use of common instructional planning template Mentoring and coaching of teachers

School 2 [Charter]

Improving instructional decision-making through better use of data

Analysis of alignment among national standards, state standards, and assessments (especially in English/Language Arts) Structured process for reviewing curricular resources Frequent monitoring and assessment of students

School 3 Shared leadership with respect to instructional decision making

Consensus process for deciding on instructional strategies based on data (primarily English/Language Arts and mathemat ics) Frequent monitoring and assessment of students

School 4 [Charter]

Analyzing and coordinating effective instruction

“Cycle of Inquiry” in which teachers test and collect data on efficacy of particu lar instructional strategies

School 5 Improving reading achievement, especially for English Language Learners

Frequent monitoring and assessment of students Mentoring and coaching of teachers

School 6 Improving reading and language development

Two-way immersion curriculum Train ing and assistance from resource teachers in school

Case Study Schools

Demographic Characteristics of the Schools

School Student

Enro llment Faculty Size

(FTEs) Percent

Minority

Percent English

Language Learners

Percent Free and Reduced Price Lunch

School 1 817 39.0 88.7 70.4 100 School 2 521 27.5 55.3 27.3 18.6 School 3 297 16.4 50.2 11.1 47.5 School 4 274 15.3 16.4 5.8 5.5 School 5 724 45.0 73.2 43.6 66.4 School 6 520 26.0 71.2 27.3 49.8

Network Configuration

Direction of Help

Bi-directional help is common among both school leaders and teachers School leaders give help more than they receive help Teachers receive help more than they give help

School Leaders

(n = 6) Teachers (n = 14)

Aspect of Interaction M SD M SD t Direction of Help Shadowee is giving help 14.17 6.43 6.43 4.62 3.06** Shadowee is receiv ing help 6.00 2.68 9.43 4.16 -1.85† Bi-directional help 14.17 4.54 20.29 9.45 -1.50

Interaction Structure

Modeling as help is rare; talk as help is most common

School Leaders


Aspect of Interaction M SD M SD t Nature of Interaction Exchange of resources 11.67 6.68 6.78 4.49 1.93† Modeling instruction 2.50 2.59 3.14 2.03 -0.60 Talk 27.00 5.76 29.14 10.43 0.47

Focus of Interaction

Instruction, coordination of activities, and school-level problems are common topics of talk as help

School Leaders


Aspect of Interaction M SD M SD t Focus of Talk Instruction 4.83 4.26 6.64 3.93 -0.92 Assessment 4.50 4.04 4.21 3.19 0.17 Students 2.67 2.07 4.57 3.46 -1.25 Other colleagues 0.83 0.75 0.86 0.95 -0.05 School-level problems 5.33 2.88 5.43 4.40 -0.62 Coordination of interactions,

resources, and activities 5.50 2.59 6.29 3.56 -0.49

Uncertain/Shadowee is witness 3.17 2.32 0.79 1.12 -3.15**

Discussion

Network configuration, helping patterns show some evidence of perturbations with respect to norms of autonomy and equality– Networks are dense in schools, and interaction

among teachers is frequent, both in planned and impromptu encounters

– There is some asymmetrical help, with designated leaders providing it

In these schools at least, the nature of interaction and focus of talk there is evidence of equality and autonomy norms– Lots of “experience swapping” remains– Talk about instruction happened, but was not the

main focus of talk

Complex Systems and Educational Change

Nora Sabelli, SRI International Jay L. Lemke, University of Michigan

Balcones Conference

2001: Meeting of 4 major NSF-funded projects in which researcher-educator partnerships had sustained successful collaborations for over 10 years

Guiding Questions:– What is the relationship between educational

system, research on the system, and the models of change used in designing the work?

– What are the open problems with the existing models that could shape the future of this type of work?

– Does the existence of models of change facilitate scaling and adaptation of reform efforts?

– Is there a taxonomy of such models where these projects and others have generated knowledge?

Internal and External Complexity of Complex Adaptive Systems

A complex adaptive system is situated in an environment:

– That environment is always more complex than the system itself, and therefore, it can never be completely predictable for the system, but the system depends on regularities of the environment for maintaining [the] energy supply needed to support its internal structures and processes .

SFI Working Paper Abstract; 2003 Author: Juergen Jost Paper # 03-12-070

Changes in Paradigms Based on Complex Systems Theory

Causal loops vs. causal chains (nonlinear networks) Integrated systems vs. isolable units of analysis Dynamical models and simulations vs. input-output

modeling Unique systems vs. generic systems Emergence vs. determinacy (surprise)

Complex systems theory provides athinking tool for:

•Qualitative reasoning about complex socio-natural systems

•Making the infrastructure (human and technical) assumptions, needs and opportunities more explicit.

•Quantitative modeling and simulation

Complex Systems Theory Provides a Thinking Tool For

Qualitative reasoning about complex socio-natural systems

Making the infrastructure (human and technical) assumptions, needs and opportunities more explicit.

Quantitative modeling and simulation

Creating a Possible Framework to Make the Infrastructure Visible

Capacity to order and simplify Identification of significant features Congruence with reality Communicative power Explanation of a total process A basis for inquiry and research:

– How to build a model that specifies the relationships between concepts

Goal: Cross-case Research

• It takes a village to study a village: who’s on the team? – The education system is a system

• Cross-project cumulativity of cases: meta-models for research• Education is local, research is general

• One example of a meta-model• To annotate local case studies

Goal: Cross-Case Research

It takes a village to study a village: who’s on the team? – The education system is a system

Cross-project cumulativity of cases: meta-models for research– Education is local, research is general

One example of a meta-model– To annotate local case studies

Available resources

Incentives

Teacher Professional Development

Alignment

Distribution of internal and external resources

Instructional Leadership and coherence

IncentivesSystem options and constraints

Local education needs

How people learn (cognition)

What people learn (content)

Why people learn (context)Which people learn (equity)

Transition across Levels

StandardsInstructional workforceDemographic Trends

Standards

Coherence & Accountability

Standards

Teacher Recruitment and policies

Teacher Certification Standardized Testing

Teacher Expectations

Pedagogical Content Knowledge

Assessment data available

How is Learning Organized?

Why people learn (context) Which people learn (equity)

What people learn (content)How people learn (cognition)

Standards-based curriculumProfessional Development

Student Outcomes Instruction

Coordination

Accountability

Incentives

Resources

Standards-based Curriculum

What is Known about Learning

Evidence

Incentives and Accountability

NSF Systemic Change Drivers

Cohen et al.

Confrey et al.

How is Learning Organized?

Ups and Downs of Multiscale Self-organization

Lower levels define the range of possibilities

Higher levels constrain the emergence of consistent patterns

Intermediate levels buffer higher levels against fluctuations from below

Some Key Questions for Adequate Models

What are the range of timescales of the critical processes that enable the system to maintain itself?

What are its significant levels of organization (control hierarchies) by characteristic structures and emergent processes and patterns at each level?

What kinds of material resource and information flows connect adjacent and non-adjacent levels?

How is information transformed, filtered, re-organized, and added to from level to level?

Symposium Overview Bill Penuel SRI International.

Documents

Transcript of Symposium Overview Bill Penuel SRI International.