AUTHENTICITY: TURNING THEORY INTO TASKS · Interactional Authenticity cognitive processes in...
Transcript of AUTHENTICITY: TURNING THEORY INTO TASKS · Interactional Authenticity cognitive processes in...
AUTHENTICITY:
TURNING THEORY INTO TASKS
Andrew Kitney & Sian Morgan
Direct method of teaching
Greater focus on speaking
assessment
Communicative competence
Interactional competence
Authenticity of interaction
Why did
authenticity
become
important
in assessment?
Oral interviews and authenticity
• Natural conversation?
• Construct underrepresentation (Savignon 1985, Young & Milanovic, 1998)
• Unequal power relationship
• Unscripted
Situational authenticity
real-life
task
test
task
• the degree to which features of real life tasks are reproduced in test tasks
Interactional Authenticity
cognitive
processes
in real-life
language
use
cognitive
processes
in test task
• the extent to which a test taker engages cognitive processes when completing a test task
‘A concern with situational authenticity requires test takers to respond to contexts which simulate ‘real life’ in terms of criterial parameters without necessarily replicating it exactly.
‘A concern with the interactional authenticity in a test requires that the cognitive processes candidates engage in while completing the test task are representative of, and offer adequate coverage of, the cognitive processes which would prevail in a natural (i.e. non-test) context.’ (Weir 2013: 97)
Accounting for these will allow test designers to build a validity argument for context validity and cognitive validity, part of Weir’s (2005) overall socio-cognitive framework for test validation.
Context validity and cognitive validity
Challenges for the testing organisation
• a range of authentic real life speaking and writing tasks, which elicit skills relevant to the test taker
• ensuring reliability of the scores important for the test user
• practicality of administration ( test taker and test provider
Speaking
• Criticism of Oral Proficiency Test
• Face to face
• Paired format tasks
• Range of skills – monologue, paired problem solving, developed discussion (cognitive validity)
Speaking
Cognitive challenges for the test taker
• Time pressure under normal circumstances – most utterances require an immediate response
• Planning time minimal, and monitoring + planning occurs often when TT engaged in own turn
• Interaction is dynamic, reactive and reciprocal
• Increased difficulty for lower level
(Field 2011 , p. 85/86)
How cognitive demands of tasks increase along CEFR levels
A2 B1 & B2 C1 & C2
Domain Personal Public Abstract
Control Controlled Semi-controlled Open-ended
Support Support Some support Autonomy
Interaction Transactional + Interactional + Evaluative
A2 interactive speaking task
A2 interactive speaking visual prompt
B1 interactive task prompt
B1 interactive speaking task speaking
B2 interactive speaking task
B2 interactive speaking task
Advanced interactive speaking task
• There is a cognitive dimension to writing
• There is a contextual dimension to writing
Writing
In the case of writing, what is involved?
The cognitive dimension -producing a text
a. macro-planning
b. organisation
c. micro-planning
d. translation
e. monitoring
f. revising
(Shaw & Weir 2007)
a. assess the task requirements and audience, determine goals and how to achieve them
b. order the ideas, identify relationships between them, decide on importance
c. focus on the part of the text about to be produced
d. translate ideas into linguistic form
e. monitor
f. make changes informed by the monitoring
In L2 writing, cognitive resources may be deployed differently, depending on the expertness of the writer
• Less proficient writers > more cognitive resources to local language issues
• More proficient writers > resources available for organisation and rhetorical presentation of ideas
The cognitive dimension
The cognitive dimensionProducing a text will involve – to a greater or lesser extent...
Macro-planning Assessing the task requirements, determining goals and how these might be achieved
Organisation Ordering ideas, identifying relationships between them; determining
which are central to the goals of the text, which secondary
Micro-planning Focuses on the part of the text about to be produced – at sentence level
Translation Translating ideas previously held in abstract form to linguistic form
Monitoring Checking mechanical accuracy of spelling, punctuation and syntax
Revising Making corrections or adjustments informed by monitoring. Can be after
each sentence, each paragraph or when whole text has been written.
(Shaw & Weir 2007)
The cognitive dimensionProducing a text will involve – to a greater or lesser extent...
Macro-planning Assessing the task requirements, determining goals and how these might be achieved
Organisation Ordering ideas, identifying relationships between them; determining
which are central to the goals of the text, which secondary
Micro-planning Focuses on the part of the text about to be produced – at sentence level
but also in relation to the paragraph and overall text
Translation Translating ideas previously held in abstract form to linguistic form
Monitoring • Checking mechanical accuracy of spelling, punctuation and syntax
• Examining the text to determine if it reflects the writer’s intentions
and fits with the developing argument structure of the text
Revising Making corrections or adjustments informed by monitoring. Can be after
each sentence, each paragraph or when whole text has been written.
(Shaw & Weir 2007)
‘Writing is neither simply words on a page nor an activity of solitary individuals…When we pick up a pen or sit at a word-processor we adopt and reproduce certain roles, identities and relationships to engage in particular socially-approved ways of communicating. So while every act of writing is in a sense both personal and individual, it is also interactional and social.' (Hyland 2002:48)
Writing as a social act taking place in a specifiable context (Weir 2005)
• Setting
• Audience
• Purpose
The contextual dimension
You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.
The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.
Write your report.
Write your answer in 220 – 260 words.
Designing the task
How can the task take account of these dimensions?
You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.
The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.
Write your report.
Write your answer in 220 – 260 words.
Designing the task
How can the task take account of these dimensions?
A2 C2
Linguistic
Cognitive
Socio-linguistic
Levels
Features of writing tasks at different levels
You have just finished a three-week study and work programme in an English-speaking country. You studied English language in the mornings and worked for a local company in the afternoons.
The programme organiser has asked you to write a report about your experience. In your report, you should evaluate the programme, explaining which part of the programme was more useful, and suggest changes you would recommend for next year’s programme.
Write your report.
Write your answer in 220 – 260 words.
• Functions of analysis and evaluation
• Having to select two of three concepts to cover and being able to discuss their merits relative to each other requires macro-planning, organisation and the ability to generate ideas to support an argument.
• Successfully arguing at C1 level for one concept over another is likely to involve some knowledge-transforming writing.
• use of source materials and role of reading into writing
• most academic writing is not timed: students can revise and edit, refer to sources, reflect on topic etc.
• most academic writing is judged primarily on the accuracy of content, rather than use of language.
opinions provided in the task input could be seen as a stand-in for longer texts, and allow writer to position his or her opinions in relation to them
compromise between authenticity and practicality
academic literacy versus general language proficiency
(Weigle 2002)
Possible limits to the authenticity of the task
(Cheung & Elliott 2017)
For more resources visit
Our YouTube channel Cambridge English TV for videos of our webinars and Facebook Live sessions
www.youtube.com/user/cambridgeenglishtv
Thank you!
You’re going to design a task.
1. Form groups of four.
2. Decide what your context is (test users and takers)
3. Decide which level you are going to target
4. Design a task to elicit either Speaking or Writing performance
Consider:
• What abilities within your skill you want to assess
• Context of use
• Criteria for assessment (if you have time)
5. Write the instructions and prompt
Task force challenge: Phase 1
Instructions for Phase 1 (15 minutes)
1. Form groups of eight (i.e. your group and one other)
2. Explain your task and how it is relevant to a real-life context (5 minutes each)
Phase 3: Revise task based on feedback (10 minutes)
1. Revise your task if necessary, considering any feedback you received
2. Copy task out onto yellow card and stick it on the wall for group task photo.
Phase 2: Share ideas with another group
(10 minutes)