Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall,...
-
Upload
margery-norton -
Category
Documents
-
view
216 -
download
1
Transcript of Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall,...
![Page 1: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/1.jpg)
Assessment Feedback Systems and Interactive Learning Environments
The Open University, Walton Hall, Milton Keynes MK7 6AA
Professor Denise [email protected]
DMW Congress Colombia November 2014
![Page 2: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/2.jpg)
DMW Congress Colombia November 2014
![Page 3: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/3.jpg)
I hate marking but want the tasks and feedback to assist student learning
DMW Congress Colombia November 2014
![Page 4: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/4.jpg)
The e-Assessment and automatic feedback Challenge
• Constructivist Learning – Push
• Institutional reliability and
accountability – Pull
DMW Congress Colombia November 2014
![Page 5: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/5.jpg)
DMW Congress Colombia November 2014DMW Congress Colombia November 2014
![Page 6: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/6.jpg)
www.storiesabout.com
www.storiesabout.com/creativepdp
DMW Congress Colombia November 2014
![Page 7: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/7.jpg)
DMW Congress Colombia November 2014
![Page 8: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/8.jpg)
MCQs: Variation on a theme
DMW Congress Colombia November 2014
Example of LAPT Certainty-Based Marking
UK cabinet ministers demo exercise
showing feedback,
University College, Tony Gardner-Medwin
Drug Chart Errors and Omissions
Medicines Administration Assessment,
Chesterfield Royal Hospital
![Page 9: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/9.jpg)
MCQs: High Stakes Assessment
DMW Congress Colombia November 2014
Example of practice “Thinking Skills Assessment" (TSA) question, Admissions Interview, Cambridge Assessment, Steve Lay
Example of practice “Thinking
Skills Assessment"
(TSA) feedback, Admissions Interview,
Cambridge Assessment, Steve Lay
![Page 10: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/10.jpg)
Scaffolding and High Stakes assessment
• Math for Science
• Tutor less course
• Competency led
• No point to cheat
• Web home exam
• Invigilation technologies
DMW Congress Colombia November 2014
![Page 11: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/11.jpg)
Self diagnosis
• Basic IT skills, first year med
students (Sieber, 2009)
• Competency based testing
• Repeating tests for revision
• Enables remedial intervention
DMW Congress Colombia November 2014
![Page 12: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/12.jpg)
Theoretical drivers for Assessment and Feedback
Piaget (1930) Individual manipulating surroundings --
mental representations can be tested Induction of rules and
their testing ... Nuffield science
Cognitive psychology (70s 80s) How are these mental
representations stored? Classification development and how
this can go wrong. .... Misconceptions and mal rules........
Student modelling .... AI
Bruner (1982) assessment tasks will match competency
levels depending on level of help...SCAFFOLDING
Pask conversation model used by Laurillard
DMW Congress Colombia November 2014
![Page 13: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/13.jpg)
Scaffolding and Interactivity; The Open University Science Foundation Course
• Interaction, Feedback loops
• Tell, Explore, Check
• Predict, Look and Explain
• Entering the discourse of a subject via audio feedback
• Scaffolded text feedback (Bruner & Woods)
• SHOW ME button
DMW Congress Colombia November 2014
![Page 14: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/14.jpg)
Interactive Tasks
• Games
• Simulations
• Making the abstract concrete
• Directing the sequence of an
animation
DMW Congress Colombia November 2014
![Page 15: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/15.jpg)
Interactivity and Cognitive Change Scores
DMW Congress Colombia November 2014
![Page 16: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/16.jpg)
Theoretical drivers: Social Constructivism
• Vygotsky (1978) individuals shape cultural settings
which shapes minds no longer individual
• Activity theory (Engstrom 1987) Tool mediation
• Situated Cognition (Lave and Wenger 1991)
Authentic assessment
• Peer interaction and assessment
• Learning conversations Laurillard (2002)
DMW Congress Colombia November 2014
![Page 17: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/17.jpg)
Characteristics Descriptor
Authentic Involving real-world knowledge and skills
Personalised Tailored to the knowledge, skills and interests of each student
Negotiated Agreed between the learner and the teacher
Engaging Involving the personal interests of the students
Recognise existing skills Willing to accredit the student’s existing work
Deep Assessing deep knowledge – not memorization
Problem oriented Original tasks requiring genuine problem solving skills
Collaboratively produced Produced in partnership with fellow students
Peer and self assessed Involving self reflection and peer review
Tool supported Encouraging the use of ICT
DMW Congress Colombia November 2014
Elliott’s characteristics of Assessment 2.0 activities
![Page 18: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/18.jpg)
Authentic Assessment: Building e-portfolios on a chef’s course
DMW Congress Colombia November 2014
Food preparation for e-portfolio
Modern Apprenticeship in Hospitality and
Catering, West Suffolk College, Mike Mulvihill
Evidence of food preparation skill for e-portfolio
Modern Apprenticeship in Hospitality and Catering,
West Suffolk College, Mike Mulvihill
![Page 19: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/19.jpg)
Peer Assessment and the WebPA Tool
• Loughborough (Loddington et al, 2009)
• Self assess and peer assess with given criteria
• Group mark awarded by tutor
• Students rated:
• More timely feedback
• Reflection
• Fair rewards for hard work
• Staff rated:
• Time savings
• Administrative gains
• Automatic calculation
• Students have faith in the administrative system
DMW Congress Colombia November 2014
![Page 20: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/20.jpg)
Mobile Technologies and Assessment
• MCQs ,PDAs Valdiva &
Nussbaum(2009)
• Polls,instant surveys
• Simpson & Oliver (2007)
• Draper (2009) EVS
DMW Congress Colombia November 2014
![Page 21: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/21.jpg)
Gains from Formative Assessment
• Mean effect size on standardised tests between 0.4 to 0.7
(Black & Williams, 1998)
• Particularly effective for students who have not done well
at school http://kn.open.ac.uk/document.cfm?docid=10817
• Can keep students to timescale and motivate them
• How can we support our students to become more
reflective learners and engage in formative assessment
tasks?
DMW Congress Colombia November 2014
![Page 22: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/22.jpg)
Collaborative formative assessment with Global Warming
DMW, Institute of Educational Technology, September 1997DMW, Institute of Educational Technology, September 1997
DMW Congress Colombia November 2014
![Page 23: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/23.jpg)
Global Warming
DMW Congress Colombia November 2014
![Page 24: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/24.jpg)
Next: ‘Yoked’ apps via BuddySpace
Student A
Student B(‘yoked’, butwithout full screen sharingrequired!)
DMW Congress Colombia November 2014
![Page 25: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/25.jpg)
Global Warming: Simlink Presentation
DMW Congress Colombia November 2014
![Page 26: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/26.jpg)
LISC: Aily Fowler
Kent University ab-initio Spanish module
•Large student numbers
•Skills-based course
•Provision of sufficient formative assessment meant
unmanageable marking loads
•Impossible to provide immediate feedback
• leading to fossilisation of errors
DMW Congress Colombia November 2014
![Page 27: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/27.jpg)
The LISC solution: developed by Ali Fowler
A call system designed to enable
students to:
•Independently practise sentence
translation
•Receive immediate (and robust) feedback
on all errors
•Attend immediately to the feedback
(before fossilisation can occur)
DMW Congress Colombia November 2014
![Page 28: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/28.jpg)
How is the final mark arrived at in the LISC System?
The two submissions are unequally weighted
•Best to give more weight to the first attempt
• since this ensures that students give careful
consideration to the construction of their first
answer
• but can improve their mark by refining the answer
•The marks ratio can vary
(depending on assessment/feedback type)
• the more information given in the feedback, the
lower the weight the second mark should carry
DMW Congress Colombia November 2014
![Page 29: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/29.jpg)
Heuristics for the final mark
If the ratio is skewed too far in favour of
the first attempt…
• students are less inclined to try
hard to correct non-perfect
answers
If the ratio is skewed too far in favour of
the second attempt…
• students exhibit less care over the
construction of their initial answer
DMW Congress Colombia November 2014
![Page 30: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/30.jpg)
What about emotional support in the feedback?
• Difficult at times to receive written feedback
• Not just a cognitive response
• How can Bales help?
DMW Congress Colombia November 2014
![Page 31: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/31.jpg)
Coding into Categories
• Bales analysis
• Psychology 1950s
• Analyses talk
• Includes socio-emotive categories
• Flander’s (1970) categories
inappropriate as also includes
classroom control
DMW Congress Colombia November 2014
![Page 32: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/32.jpg)
Bales Categories
• Four main groupings
• A. Positive reactions; agreeing
and boosting the other person
• B. Directing/teaching
• C. Questions: requesting
information, clarification etc
• D. Negative reactions:
disagreement
DMW Congress Colombia November 2014
![Page 33: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/33.jpg)
Coding the tutor comments
Categories Specific Examples
Positive Reactions
A1
A2
A3
1. Shows solidarity
2. Shows tension release
3. Shows agreement
Jokes, gives help, rewards others
Laughs, shows satisfaction
Understands, concurs, complies, passively accepts
Attempted Answers
B1
B2
B3
4. Gives suggestion
5. Gives opinion
6. Gives information
Directs, proposes, controls
Evaluates, analyses, expresses feelings or wishes
Orients, repeats, clarifies, confirms
Questions
C1
C2
C3
7. Asks for information
8. Asks for opinion
9. Asks for suggestion
Requests orientation, repetition, confirmation, clarification
Requests evaluation, analysis, expression of feeling or wishes
Requests directions, proposals
Negative Reactions
D1
D2
D3
10. Shows disagreement
11. Shows tension
12. Shows antagonism
Passively rejects, resorts to formality, withholds help
Asks for help, withdraws
Deflates others, defends or asserts self
Bales’ Interaction Process
DMW Congress Colombia November 2014
![Page 34: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/34.jpg)
Identifying trends: H801
Graph to show conflated Bale’s
categories against mean number
of incidences in H801 scripts
0 5 10 15 20 25
A Pass 1
A Pass 2
A Pass 3
A Pass 4
B Pass 1
B Pass 2
B Pass 3
B Pass 4
C Pass 1
C Pass 2
C Pass 3
C Pass 4
D Pass 1
D Pass 2
D Pass 3
D Pass 4
Ba
les'
In
tera
ctio
na
l C
ate
go
rie
s a
t e
ach
Pa
ssL
leve
l
Number ofIincidences
DMW Congress Colombia November 2014
![Page 35: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/35.jpg)
Identifying trends: H801
Pie Chart to show the mean
number of incidences per pass per
conflated Bales' Interactional
Category for all four levels of pass
in H801 scripts
5.96
17.13
5.73
1.61
A
B
C
D
Key:
A = Positive reactions
B = Responses
C = Questions
D = Negative reactions
DMW Congress Colombia November 2014
![Page 36: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/36.jpg)
Identifying trends
A
B
C
D
A
B
C
D
A
B
C
D
A
B
C
D
Pie Charts to show the mean number of incidences per conflated Bales Interactional
Category for ‘Pass 1’ and ‘Pass 4’ in the following courses:
Key:A = Positive reactions C = QuestionsB = ResponsesD = Negative reactions
Pass 4
Pass 1
B820 S103
A
B
C
D
A
B
C
D
H801
DMW Congress Colombia November 2014
![Page 37: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/37.jpg)
What is Open Mentor?
“An open source mentoring tool for tutors”
“Open source” = free and easy to use, and to embed in an
institutions infrastructure and working practices
“mentoring” = designed to help people learn how to give
feedback effectively, through reflection and social networks
“tutors” = primarily intended for teaching staff, but with clear
applications for those involved in quality
DMW Congress Colombia November 2014
![Page 38: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/38.jpg)
How Open Mentor handles comments
“Good work”
“Yes, well done”
“Yes, but is this useful?”
“Can you explain what you mean”
“This does not follow”
A = positive reactions
A = positive reactions
B = attempted answers,
and not a positive reaction
C = questions
D = negative reactions
DMW Congress Colombia November 2014
![Page 39: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/39.jpg)
Explaining OpenMentor’s Rules
Four categories
•A – Positive Reactions
•B – Attempted Answers
•C – Questions
•D – Negative Reactions
DMW Congress Colombia November 2014
![Page 40: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/40.jpg)
‘A’ - Positive Reactions
Category Examples of Rules Examples of commentsA - Positive Reactions1. Shows solidarity A1 ...excellent... Excellent Conclusions.
A1 ...(good|comprehensive)... Good, you are drawing on hard facts here.A1 ...nicely... Very nicely stated. Your analysis is thorough and your
conclusions consistent regarding the attractiveness of the budget airline sector. This is a good example of critical thinking.
A1 ...well presented... Very well presented diagram with interesting information.A1 ...effective use... Effective use of the case material here.A1 …well (structured|stated)… Report very well structured.A1 ...(well|clear)(ly)*
(structured|structure|summary| summarised|presented|presentation)...
The corporate vs. business unit strategy is well presented and nicely tied to strategies.
A1 ...reasonable.... A reasonable structure as listed in your table of contents.A1 ...useful point(s)... Generally useful points in this section.
2. Shows tension release A2 ... a helpful...
A2 …(thanks|thank you)…3. Shows agreement A3 ...yes... Yes, the intellectual reactions are both real.
A3 ...indeed... Indeed – if it has one basic strategy it is surely differentiation, though it still has to control costs.
4. Praise then direction A4 good...but... Good model, good quote, but be careful about what industry you analyse ??
DMW Congress Colombia November 2014
![Page 41: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/41.jpg)
‘B’ Attempted Answers
Category Rules Examples of commentsB - Attempted Answers
B4 …perhaps… Perhaps even better here to explain the link in your mind between "analysis of stratgeies" and "strategic issues".
B4 …requires…B4 …take care… Take care with your STEP analysis not to make it too industry
focussed.B4 …useful to… Innovation is closely linked to structure and culture- it would be
useful to see some book 6-8 concepts here too.
B4 …you (might|ought)… You ought to have explicitly stated these.
B4 Don’t|never … Don’t introduce new frameworks just for the sake of it in the conclusion. The conclusion should be pulling together what went before.
B4 Please (see|refer to|look at)... Please make sure to read and understand the question correctly
5. Gives opinion B5 I (am|think)... I think I can see where you are going, though a numbered report format might have demonstrated the approach better
B5 This is.... this is an introduction rather than a “summary”B5 ...sounds...like... This sounds as if it could be very popular!!B5 ...not sure... I am not sure about the balance between the environmental
analysis and the review of the resources, capabilities (power, culture, structures and systems) as raised in the question.
B5 I (thought|agree|suggest)… I thought it was because they did not need any external input and saw a significant market sector they could address themselves.
B5 I (do|don’t) think… I don’t think this exercise has helped to develop your analysis. I also think that the development of the perspectives is superficial
6. Gives information B6 …(demonstrates|shows) this….B6 Also… Also, cross link to Leadership issues, Pettigrew on Strategic
Thinking tooB6 ...Q1... etc Q1 = 59/100
DMW Congress Colombia November 2014
![Page 42: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/42.jpg)
Is the rule set generic?
Comments Classified from Test Data
0102030405060708090
100
B820 TestSet 1
B820 TestSet 2
B820 TestSet 3
A850 M878 S809
Test Data Set
% o
f co
mm
ents
cla
ssif
ied
DMW Congress Colombia November 2014
![Page 43: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/43.jpg)
DMW Congress Colombia November 2014
![Page 44: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/44.jpg)
DMW Congress Colombia November 2014
![Page 45: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/45.jpg)
What can we learn from modelling tutors marking to construct a formative e-assessment tool?
• Open Comment project builds on
the work of OpenMentor
• Free text entry for History and
Philosophy students
• Immediate feedback (in context)
to students
• Influenced by ELIZA
(Weizenbaum, 1963)
DMW Congress Colombia November 2014
![Page 46: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/46.jpg)
Open Comment addresses the problem of free text entry
• Automated formative assessment tool
• Free text entry for students
• Automated feedback and guidance
• Open questions, divergent assessment
• No marks awarded
• For use by Arts Faculty
DMW Congress Colombia November 2014
![Page 47: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/47.jpg)
Causal models of explanation
First step:
• Identification of question types where
students exhibit causal reasoning
Looked for questions with:
• Salient variables
• Description of these variables
• Identification of trends
• Identification of relationship between
the variables i.e. causality
DMW Congress Colombia November 2014
![Page 48: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/48.jpg)
DMW Congress Colombia November 2014
![Page 49: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/49.jpg)
DMW Congress Colombia November 2014
![Page 50: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/50.jpg)
DMW Congress Colombia November 2014
![Page 51: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/51.jpg)
Praise for effort and not just ability
• Praise for ability per se can hinder
learning (Mueller & Dweck, 1998)
• Praise = being clever
• Negative feedback now
without ability
• Disempowering and demoralising
DMW Congress Colombia November 2014
![Page 52: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/52.jpg)
Praise and motivation: feedback and self regulation
• “Feeling good” feedback part of many guidelines
• Mueller & Dweck (1998) Praise for effort as well as
performance
• Raven’s test given to USA school children
• First test feedback was praise
• Second test most difficult. Half praised for effort,
half praised for ability
• Third test medium difficulty
• On the third test pupils praised effort increased score by
1. Praised for ability decreased 1 mark (scale 0-10)
DMW Congress Colombia November 2014
![Page 53: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/53.jpg)
How does feedback effect mindsets?
1. Your intelligence is something very basic about you
that you can’t change very much
2. You can learn new things but you can’t really change
how intelligent you are
3. No matter how much intelligence you have you can
always change it quite a bit
4. You can always substantially change how intelligent
you are
DMW Congress Colombia November 2014
![Page 54: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/54.jpg)
Mindsets (Dweck, 2006)
• Fixed mindset
• Super sensitive about being
wrong
• Always trying to prove
themselves
• Growth mindset
• Stretch themselves
• Confront obstacles as
challenges
• Lack of tension when
learning as they know they
are novices and can
improve
DMW Congress Colombia November 2014
![Page 55: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/55.jpg)
Stages of analysis by computer of students’ free text entry for Open Comment: Advice with respect to content (socio-emotional support stylised example)
Stage 1a: Detect errors
E.g. Incorrect dates, facts. (Incorrect inferences and
causality is dealt with below)
•Instead of concentrating on X, think about Y in order to
answer this question Recognise effort (Dweck) and
encourage to have another go
•You have done well to start answering this question but
perhaps you misunderstood it. Instead of thinking about X
which did not…….. Consider Y
DMW Congress Colombia November 2014
![Page 56: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/56.jpg)
Computer analysis continued
Stage 2a: Reveal first omission
Consider the role of Z in your answer Praise
what is correct and point out what is missing
Good but now consider the role X plays in your
answer
Stage 2b: Reveal second omission
Consider the role of P in your answer Praise
what is correct and point out what is missing
Yes but also consider P. Would it have
produced the same result if P is neglected?
DMW Congress Colombia November 2014
![Page 57: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/57.jpg)
Final stages of analysis
• Stage 3:request clarification of key point 1
• Stage 4: request further analysis of key point 1
– stages 3 and 4 repeated with all the key points
• Stage 5:request the inference from the analysis
of key point 1 if it is missing
• Stage 6:request the inference from the analysis
of key point 1 if it is not complete
• Stage 7:check the causality
• Stage 8:request all the causal factors
are weighted
DMW Congress Colombia November 2014
![Page 58: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/58.jpg)
Open Comment drivers for reflection
• Students are able to find facts similar to X
• Know how X might be disputed
• Are able to make predictions about X
• Know how to use X in an argument
• Know how far X can be pushed
• Supported with tools and strategies for effort
DMW Congress Colombia November 2014
![Page 59: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/59.jpg)
SAFeSEA
Professor Denise Whitelock
Professor John Richardson
Professor Stephen Pulman
An automated tool supporting
online writing and assessment
of essays providing
accurate targeted feedback
SAFeSEA: Supportive Automated Feedback for Short Essay Answers
DMW Congress Colombia November 2014
![Page 60: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/60.jpg)
About SAFeSEA
• No tutor support for drafts of first assignment
• Reduce dropout rate with automatic feedback?
• Effect of summaristion
• What are the beneficial factors?
• Correlate measures of learner activity and
essay improvement
• http://www.open.ac.uk/researchprojects/safesea/
DMW Congress Colombia November 2014
![Page 61: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/61.jpg)
Talk Back
• Checking understanding by ‘talk back’
• Summaries in OpenEssayist
• Key words = key ideas
• www.open.ac.uk/researchprojects/safesea/
DMW Congress Colombia November 2014
![Page 62: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/62.jpg)
OpenEssayist: What it tells you
The system’s focus is to present summaries of your own work
in different ways, to encourage you to reflect constructively on
what you have written.
In other words Open Essayist tells you from its analysis what
are the most important or key points in your essay. You can
then think about whether that was what you intended to
emphasis in your essay. If not then you can make the
appropriate changes.
A very important aspect of the OpenEssayist system is that it
will not tell you what to write, or how to rewrite sections of
your essay, or even what is correct or incorrect in your essay.
DMW Congress Colombia November 2014
![Page 63: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/63.jpg)
ESSAY
GRAPHICS
ANALYSIS
![Page 64: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/64.jpg)
OpenEssayist: How it gives feedback
Three aspects of your essay are analysed by the system:
• the structure of the essay (which paragraphs constitute
the introduction, the conclusion, the discussion
sections, etc.),
• the key words and key phrases of your essay (which
are the most important words and phrases, the ones
that are most representative of your essay's overall
meaning)
• the key sentences of your essay (which are whole
sentences that are most representative of your essay's
overall meaning).
DMW Congress Colombia November 2014
![Page 65: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/65.jpg)
DMW Congress Colombia November 2014
![Page 66: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/66.jpg)
![Page 67: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/67.jpg)
![Page 68: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/68.jpg)
![Page 69: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/69.jpg)
Sample key phrases dispersion plot
DMW Congress Colombia November 2014
![Page 70: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/70.jpg)
Visualisations as a Thinking Tool
• Visualising text (Bertin, 1981;
Johnson et al, 1993)
• Free text visualisation still
problematic
• Do users need training with
visualisations?
• Convention vs. Instruction?
DMW Congress Colombia November 2014
![Page 71: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/71.jpg)
Short text for illustration of Rainbow Diagrams
DMW Congress Colombia November 2014
![Page 72: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/72.jpg)
Sentence graph of short text
DMW Congress Colombia November 2014
![Page 73: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/73.jpg)
Pretend essay: 10 identical paragraphs
DMW Congress Colombia November 2014
![Page 74: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/74.jpg)
Pretend essay: 50 identical sentences
DMW Congress Colombia November 2014
![Page 75: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/75.jpg)
Stanford University Boothe Prize essay
DMW Congress Colombia November 2014
![Page 76: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/76.jpg)
OU essay awarded high grade
DMW Congress Colombia November 2014
![Page 77: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/77.jpg)
OU essay awarded low grade
DMW Congress Colombia November 2014
![Page 78: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/78.jpg)
Good vs. bad?
Bad:
•not densely connected•red nodes (conclusion)
not central•few links between violet (intro)
and red nodes
DMW Congress Colombia November 2014
Good:
•densely connected•red nodes (conclusion)
central•close links between violet
(intro) and red nodes
![Page 79: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/79.jpg)
Findings
“One of the clues was talking about the way the color nodes, that the red ones are at the end. If they are a good connection with a good explanation for each paragraph. They should be connected together but the darker ones should be in the middle. But you see the color groups together so for me it automatically pulls my eye to this page because all the colors are closer together and more in the middle. So that would be the student course assignment essay with the highest mark.”
“You don’t show me anybody’s text. You are not revealing anyone else’s essay. So students cannot plagiarize. But you are saying ‘Look hang on, this is the way this essay connects together.’ That’s what telling a good story is about this linking.”
DMW Congress Colombia November 2014
![Page 80: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/80.jpg)
Rainbow diagrams related to mark awarded
• Multivariate analysis of variance on marks
awarded to 45 students
• Submitted two essays
• Rainbow diagrams produced from these essays
and rated as high, medium or low attainment
• Covariate showed a significant relationship with the marks
• F(1, 43) = 5.92, p = .01 using a directional test
• Essays rated as high would be expected to receive 8.56
percentage points more than essays rated as medium
• 17.2 percentage points higher than essays rated
from rainbow diagrams as low
DMW Congress Colombia November 2014
![Page 81: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/81.jpg)
Is NLP the bridge between Computer Assisted Assessment and Learning Analytics?
• Automatic marking
• Recognising text
• Refining text
DMW Congress Colombia November 2014
![Page 82: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/82.jpg)
How about feedback first?
• Hints before writing?
• R.C.T.
• 2 essays
• F(1,41) = 3.23 p = 0.04 for hints
DMW Congress Colombia November 2014
![Page 83: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/83.jpg)
Creating teaching and learning dialogues: towards guided learning supported by technology
• Learning to judge
• Providing reassurance
• Providing a variety of signposted
routes to achieve learning goals
• Provide socio-emotive support
DMW Congress Colombia November 2014
![Page 84: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/84.jpg)
Feedback: Advice for Action
• Students must decode feedback and
then act on it Boud (2000)
• Students must have the opportunity to
act on feedback Sadler (1989)
• Gauging efficacy through student action
DMW Congress Colombia November 2014
![Page 85: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/85.jpg)
Characteristics Descriptor
Authentic Involving real-world knowledge and skills
Personalised Tailored to the knowledge, skills and interests of each student
Negotiated Agreed between the learner and the teacher
Engaging Involving the personal interests of the students
Recognise existing skills Willing to accredit the student’s existing work
Deep Assessing deep knowledge – not memorization
Problem oriented Original tasks requiring genuine problem solving skills
Collaboratively produced Produced in partnership with fellow students
Peer and self assessed Involving self reflection and peer review
Tool supported Encouraging the use of ICT
Elliott’s characteristics of Assessment 2.0 activities
Advice for Action
DMW Congress Colombia November 2014
![Page 86: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/86.jpg)
The 4Ts Pyramid
Tool Development
Training of staff
TransformationTasks
TransferLearning
DMW Congress Colombia November 2014
![Page 87: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/87.jpg)
Training
• Question development
• Quality REAQ
• Plagiarism software
• Advice for Action
• Socio e-emotive content
• Maintain empathy with the Learner
DMW Congress Colombia November 2014
![Page 88: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/88.jpg)
e-Assessment Futures
• Research issues
• Adaptive testing
• Automatic marking
• Learning Analytics Data mining
• Web 2.0, 3.0 …
DMW Congress Colombia November 2014
![Page 89: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/89.jpg)
Feedback to prompt self reflection
• Analysis must prompt “Advice for
Action”
• Self reflective discourse with
computer feedback
• Visual representation of feedback
can open a discourse between
tutor and student
• Prompt peer to peer discourse
DMW Congress Colombia November 2014
![Page 90: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/90.jpg)
DMW Congress Colombia November 2014
![Page 91: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/91.jpg)
References 1
Whitelock, D. (2010) Activating Assessment for Learning: are we on the way with Web 2.0? In M.J.W. Lee & C. McLoughlin (Eds.) Web 2.0-Based-E-Learning: Applying Social Informatics for Tertiary Teaching. IGI Global. pp. 319–342.
Whitelock, D. (2008) ‘Accelerating the assessment agenda: thinking outside the black box’. In Scheuermann, F. & Pereira, A.G. (eds.) Towards a Research Agenda on Computer-Based Assessment: Challenges and Needs for European Educational Measurement, published by the European Commission, Joint Research Centre. pp15-21 ISSN. 1018-5593
DMW Congress Colombia November 2014
![Page 92: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/92.jpg)
References 2
DMW Congress Colombia November 2014
![Page 93: Assessment Feedback Systems and Interactive Learning Environments The Open University, Walton Hall, Milton Keynes MK7 6AA Professor Denise Whitelock denise.whitelock@open.ac.uk.](https://reader037.fdocuments.in/reader037/viewer/2022110210/56649e605503460f94b5a3ff/html5/thumbnails/93.jpg)
Five assessment special issues
DMW Congress Colombia November 2014