Diana Laurillard: The Conversational Framework - an approach to Evaluating e-Assessment Patterns in...
-
Upload
yishay-mor -
Category
Education
-
view
11.700 -
download
3
description
Transcript of Diana Laurillard: The Conversational Framework - an approach to Evaluating e-Assessment Patterns in...
The Conversational Framework: An approach to evaluating e-assessment patterns in terms of learning theory
Diana Laurillard
London Knowledge Lab
Institute of Education
Scoping a vision of formative e-assessment
2
A common framework for representing learning theory
What does it take to learn (formal learning)?
What are the requirements for formative assessment?
Evaluating patterns of formative e-assessment
3
A common framework of representation
‘Instructivist’ - Teacher-focused
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Concepts
Actions
Adapt actions
Adapt Task practice environment
Reflect
Task goal
Answers
4
‘Constructionist’ - Practice-focused
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Concepts
Actions
Adapt actions
Adapt Task practice environment
Reflect
Task goal
Reflect
Revisions
Feedback
Practice environment
Answers
A common framework of representation
5
‘Constructionist’ - Practice-focused
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Practice environment
A common framework of representation
6
‘Social constructivist’ - Learner-focused
Teacher’s ideas
Learner’s ideas
Learner’s practice
Reflect
Other learner(s)
Ideas
Ideas
Practice environment
Reflect
Others’ practice
A common framework of representation
7‘Collaborative learning’ = ‘Social constructionism’?
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Practice environment
Other learner(s)
Ideas
Ideas
Reflect
Others’ practice
Adapt actions
Draft outputs
Draft outputs
A common framework of representation
8
- A Conversational FrameworkWhat does it take to learn?
Instructivism + Constructionism + Social constructivism + Collaboration
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Practice environment
Other learner(s)
Ideas
Ideas
Reflect
Others’ practice
Adapt actions
Draft outputs
Draft outputs
Concepts Answers
Reflect
QuestionsOutputs
9
The Conversational FrameworkAn attempt to draw on the learning theories developed over the
last century, and encapsulate them in a form that enables
educators to test conventional and digital methods against them
(Dewey, Piaget, Vygostsky, Bruner, Papert, Pask, Anderson, Marton, Biggs,
Brown, Lave, Wenger, Engestrom...)
10
Testing the value of digital learning
technologies against
the Conversational Framework
Lecture, Presentation, Book, Educational television, Audio… Powerpoint, Digital video, Animation, Podcast…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts
Reflect
QuestionsOutputs
Learning through acquisition…
Libraries, Catalogues, Journals, Resource banks… Online resource, Digital library, Website, Search engine…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts
Reflect
Questions
Learning through inquiry…
Tutorial, Seminar, Class discussion, Small group discussion… Online conferencing, Forum, Chat room, Discussion board…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts Answers
Reflect
QuestionsOutputs
Learning through discussion…
Problem sheet, practice exercises, project work… Interactive simulation, Spreadsheet, Data analysis tool, Game…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts Answers
Reflect
QuestionsOutputs
Learning through practice…
Answers
Outputs
Feedback
Answers
Outputs
Reflect
Laboratory, Small group work, Fieldwork, Workshop…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts Answers
Reflect
QuestionsOutputs
Learning through collaboration…
CSCL environments, Wikis, Second Life, Multiplayer games…
Reflect
Essay, program, solution, design, product, performance…
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Other learner(s)
Ideas
Ideas
Reflect
Other learner(s)
Adapt actions
Draft outputs
Draft outputs
Concepts
Reflect
Outputs
Learning through production…
Ppt, Program, Model, Website, Blog, Design, Digital video…
17
An integrated framework of our collective theories of
learning provides a powerful challenge to digital
technologies, showing that:
To support the learning process fully we need to integrate all the technological capabilities available
18
1. access explanations and presentations of the theory, ideas or concepts?2. ask questions about their understanding of the theory, etc, by providing the opportunity for answers from (i) the teacher, or (ii) their peers?3. offer their own ideas and conceptual understanding, by providing comment on them from (i) the teacher, or (ii) their peers?4. use their theoretical understanding to achieve a clear task goal by adapting their actions in the light of their understanding, or in response to comments or feedback?5. repeat practice, by providing feedback on actions that enables them to improve performance?6. repeat practice, by enabling them to share their trial actions with peers, for comparison and comment?7. reflect on the experience of the goal-action-feedback cycle, by offering repeated practice at achieving the task goal?8. discuss and debate their ideas with other learners?9. reflect on their experience, by having to articulate or produce their ideas, reports, designs, performances, etc. for presentation to their peers?10. reflect on their experience, by having to articulate or produce their ideas, reports, designs, performances, etc. for presentation to their teachers?
The Conversational Framework – Challenging a learning design
How does your pattern of learning activities motivate students to:
Shares practice attempt
19
Learner’s conception as practice = 20%
Learner’s conception =
10% teacher’s
Teacher’s conception
Other learner’(s) conceptions
The theory of the Conversational Framework - the learner’s developing experience
Teacher-designed task practice environment
Other learner’(s) conceptions as
practice
Offers answers, ideas
Questions, offers ideas
Feedback on action
Action to achieve goal
Shares practice attempt
Adapts approach to task to current conceptionAdapt a Task practice
environment for learners’ needs
Reflects on feedback
Reflects on alternative practice
Reflects on learners’ practice
Hints, comments
Present concepts
Task goal
Revises action
Presents conception as product
Questions
Learner’s conception =
20% teacher’s
Learner’s conception =
40% teacher’s
Learner’s conception as practice = 40%
Learner’s conception =
50% teacher’s
Learner’s conception as practice = 50%
Adapts approach to task to current conception
Learner’s conception as practice = 60%
Learner’s conception =
60% teacher’s
Discursive/theoretical level
Practice/practical level
“the best predictor of quality of output is the number of iterations”
Theory level
Practice level
Feasst: Formative assessment strategies
Strategy: clarifying and sharing learning intentions and criteria for success
Strategy: engineering effective classroom discussions and other learning tasks that elicit evidence of student understanding
Strategy: providing feedback (extrinsic/intrinsic) that moves learners forward
Strategy: activating students as instructional resources for one another, sharing outputs and ideas with peers
Strategy: activating students as the owners of their own learning, learner self-regulation;
20
21
Teacher’s ideas
Learner’s ideas
Practice environment
Learner’s practice
Actions
Adapt actions
Adapt Task practice environment
Task goal
Reflect
Revisions
Feedback
Practice environment
Other learner(s)
Ideas
Ideas
Reflect
Others’ practice
Adapt actions
Draft outputs
Draft outputs
Hints/comments
Reflect
QuestionsOutputs
Moments of contingency Forms of feedback to the learner
Wiliam: Formative assessment conceptsConcepts
Task goal
22
Learner’s conception as practice
Teacher’s conception
Other learner’(s)
conceptions
The Conversational Framework - the learners’ experience of formative assessment – collaborative version
Teacher-designed task
Other learner’(s)
conceptions as practice
Feedback on action
Action to achieve goal
Shares practice attempt
Adapts approach
to task
Adapt a Task practice
environment for learners’ needs
Reflects on feedback
Reflects on
alternative practice
Reflects on learners’ practice
Revises action
Adapts approach
to task
Shares practice attempt
Offers answers, ideas
Questions, offers ideas
Theory level
Practice level
Learner’s conception
Clarify learning intention
Engineering effective discussions, tasks and activities that elicit evidence of learning
Providing (intrinsic) feedback that moves learners forward
Activating students as learning resources for one another
Activating students as owners of their own learning
Activating students as owners of their own learning
Aspects of formative assessment – Dylan Wiliam
Hints, commentsProviding (extrinsic) feedback that moves learners forward
Task goal
23
Learner’s conception as practice
Teacher’s conception
Other learner’(s)
conceptions
The Conversational Framework - the learners’ experience of formative assessment – individual version
Teacher-designed task
Other learner’(s)
conceptions as practice
Feedback on action
Action to achieve goal
Shares practice attempt
Adapts approach
to task
Adapt a Task practice
environment for learners’ needs
Reflects on feedback
Reflects on
alternative practice
Reflects on learners’ practice
Revises action
Adapts approach
to task
Shares practice attempt
Offers answers, ideas
Questions, offers ideas
Theory level
Practice level
Learner’s conception
Aspects of formative assessment – Dylan Wiliam
Hints, comments
Clarify learning intention
Providing (intrinsic) feedback that moves learners forward
Activating students as owners of their own learning
Providing (extrinsic) feedback that moves learners forward
dummy
1. access explanations and presentations of the theory, ideas or concepts?2. ask questions about theory, etc, by providing the opportunity for answers from the teacher, or their peers?3. offer their own ideas and understanding, by providing comment on them from the teacher or their peers?4. use their understanding to achieve a goal by adapting their actions in the light of understanding, or feedback?5. repeat practice, by providing intrinsic feedback on actions that enables them to improve performance?6. repeat practice, by enabling them to share their trial actions with peers, for comparison and comment?7. reflect on the experience of the goal-action-feedback cycle, by offering repeated practice at achieving the goal?8. discuss and debate their ideas with other learners?9. reflect on their experience, by having to articulate or produce their ideas, for presentation to their peers?10. reflect on their experience, by having to articulate or produce their ideas for presentation to their teachers?
24
The Conversational Framework – Challenging a learning design
How does your pattern of learning activities motivate students to:
TryOnceRefineOnce pattern
Lack of immediate feedback leads to fossilisation of errors; immediate feedback can hinder effective learning
“Feedback on any remaining errors is given, along with correct answer(s)”
“Students are posed questions to elicit answers that can contain multiple errors. A mark is given which contributes to a percentage of the total mark for the question, along with detailed - yet generic- feedback on the location and type of the errors.)”
“Students are permitted a second attempt in which to refine their answer. No further attempts are permitted.”
25
Learner’s conception as practice = 20%
Learner’s conception =
10% teacher’s
Teacher’s conception
Applying the Conversational Framework to the TryOnceRefineOnce pattern
Teacher-designed task practice environment
Action to achieve goal
Adapt a Task practice environment for learners’ needs
Feedback
Task goal
Revises action
Learner’s conception =
20% teacher’s
Learner’s conception =
40% teacher’s
Learner’s conception as practice = 40%
“the best predictor of quality of output is the number of iterations”
Theory level
Practice level
Adapts approach to task to current conception
Gaps can be filled by using technology to:• Structure peer feedback to increase cycles of iterations• Provide computer-based feedback to increase practice iterations• Automate hints and comments in response to student actions
Technology (‘e’) adds speed of response, iteration, intrinsic feedback (processing), re-use, and orchestration of peer collaboration
Reflects on feedback
26
Summary
The Conversational Framework can be used to evaluate and improve
patterns of formative assessment through use of technology against
established learning theory and the formative assessment criteria
established in the Feasst project.