Learning Learning Progressions: Progressions:

Some Some Thoughts Thoughts

About What About What we dowe do

With and With and About ThemAbout Them

Jim PellegrinoJim PellegrinoUniversity of IllinoisUniversity of Illinois

at Chicagoat Chicago

• Learning is facilitated when new and existing knowledge is structured around big ideas or a conceptual framework rather than small, discrete bits of information.

• Learning develops as a continuous process with an

individual continuously making links back and forth among ideas and not in linear, discrete steps.

• Learning difficult ideas takes time and often comes together as students work on a task that forces them to synthesize ideas.

• Yet, K – 12 science curricula are generally not structured to build and cycle back on ideas.

Why Learning Progressions?

Curriculum Instruction

Assessment

Domain-Based Domain-Based Models of Learning Models of Learning & Understanding& Understanding

Importance of Domain-Based Models

Curriculum Instruction

Domain-Based Domain-Based Models of Learning Models of Learning

& Understanding& Understanding

Assessment

ObservationsObservations InterpretationInterpretation

Connecting Assessment toCurriculum & Instruction

Assessment as a Process of Reasoning from Evidence

• cognition – model of how students

represent knowledge & develop competence in the domain

• observations– tasks or situations that allow

one to observe students’ performance

• interpretation– method for making sense of

the data

observation interpretation

cognition

Must be coordinated!

Guiding Principles

• All models are wrong, some are useful» George Box

• Better an approximate answer to the right question than an accurate answer to the wrong question

» John Tukey

Assessment Design Principles

• Assessment design should always be based upon a model of student learning and a clear sense of the inferences about student competence that are desired for the particular context of use.

• The model of student learning suggests the most important aspects of comptence that one would want to make inferences about and provides clues about the types of tasks that will elicit evidence to support those inferences.

Aspects of Student Models

• Domain specific and empirically based

• Identifies cognitive performances that differentiate expert and novice learners

• Lays out one or more typical progressions toward competence including milestones or landmark performances along the way.

• Can be at various levels of detail; grain size depends on assessment purpose

Contrasting & Complementary Perspectives on “Cognition”

Targets for Explanation

(& Assessment)

Rationalist

Perspective

Sociocultural

Perspective

Nature of

Performance

Underlying Competencies

Communal

Practices

Nature of Development

Trajectories of

Learning

Trajectories of

Participation

Nature of Knowledge

Mental

Representations

Forms of Mediated Activity

Interpretation -- Making Sense of the Observational Data

• Some of the ways in which individuals typically try to make sense of the observational data– Intuition -- gut reaction– Counting number correct (4/8); generating percent correct– Assigning points to answers or assigning letter grades

• Going deeper -- getting at student strengths & weaknesses– Focus on the nature of student thinking, including

misunderstandings, not just correct responses– Create interpretive scales and rubrics that provide detail

about multiple features of student competence

• Assessment isn’t just about assigning scores– It’s a meaning making process

• Description of successively more sophisticated ways of thinking about a big idea

• Provide a framework for long-term development– Describes what it means to move towards more

expert understanding in an area– Gauge increasing competence over time – A sequence of successively more complex ways of

thinking about how an idea develops over time

• Consider how ideas build upon each other to form more complex practices or ideas

Learning Progressions

The Researcher’s Modelof Development/Change/Progress

• A central feature of all this work resides in the researcher’s beliefs about the nature of individual development or change -- IN ONE OR MORE DOMAINS!!!!;

• These beliefs affect the researcher’s choice of theoretical framing, choice of data, choice of measure, and choice of analysis tools;

• Choices interact with each other affecting the way claims are warranted and inferences are drawn;

Questions to Help Unpack Likely Assumptions

• Question 1:

– Has change been framed as systematic or random?

– All assessments have error. How does your model of change adjust for this error, or does it?

Questions to Help Unpack Likely Assumptions

• Question 2: Is development reversible?– Implications for the form of the

measurement; – Implications for the functional form of the

developmental trajectories;– The shape of the trajectory should be

hypothesized before data collection;– The shape can be empirically tested;

Questions to Help Unpack Likely Assumptions

• Question 3: Is the change unitary or multi-path?

• Your theory of learning should be able to identify and describe why certain groups of students follow different growth trajectories;

Questions to Help Unpack Likely Assumptions

• Question 4: Can we consider change to:– be a continuous, gradual, quantitative

phenomenon?– have large magnitude shifts on a

quantitative variable?– Be a progression through as series of

qualitatively distinct stages?

Questions to Help Unpack Likely Assumptions

• Question 5: Is change considered as differences in magnitude in:

– an absolute sense? – calibration? – conceptualization?

Questions to Help Unpack Likely Assumptions

• Questions 6: Is change considered a shared characteristic of a group of individuals over time? – Is it what occurs within individuals over

time?– Or both?– Keep in mind that both kinds of change can

occur simultaneously!

Questions to Help Unpack Likely Assumptions

• Question 7: If we assume that all individuals have trajectories of the same functional form are there systematic inter-individual differences in the values of the individual growth parameters?– What would this mean theoretically?

Questions to Help Unpack Likely Assumptions

• Question 8: Are there cross domain relationships in change over time?– Is the relationship between inter-individual

differences in intra-individual change over time (and the predictors of those differences) invariant across learning domains?

Questions to Help Unpack Likely Assumptions

• Question 9: Finally, is there invariance across groups with respect to the facet of change over time under investigation?– Put simply, are change patterns at the

group level constant or invariant across groups?

Ravit’s Questions ++• Nature of progression:

– Path/ paths/ landscapes?– Nature of movement -cycles, multiple states– Context dependence

• Nature of learning performances:– Integrate big ideas and practices – Quantifiable variables that measure learning outcomes

• Nature of evidence:– Can we really rely on short terms studies, will we (and if so when) need to

actually follow student learning over grades?– Wont instruction fundamentally change what students can do , and

therefore the progression

• Challenges for teaching (instructional practice)• Challenges for assessment• Challenges for curriculum design