Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment...
Transcript of Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment...
![Page 1: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/1.jpg)
1
Task-BasedLanguageAssessmentandTes1ngforProficiency:WhereDotheTwainMeet?ErwinTschirnerL.E.A.R.N.WorkshopSeptember20-21,2016UniversiAesatShadyGrove
![Page 2: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/2.jpg)
Overview
• OriginsofTask-BasedLanguageAssessment• Task-Based:ShadesofMeaning• Evidence-CenteredDesign(ECD)• RelaAonshipBetweenECDandProficiencyAssessment• ConstructoftheOPI• AssessmentUseArgument• FutureofProficiencyAssessment
2
![Page 3: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/3.jpg)
Task-BasedLanguageAssessment(TBLA)
3
![Page 4: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/4.jpg)
TheprocessofevaluaAng,inrelaAontoasetofexplicitlystatedcriteria,thequalityofthecommunica1veperformanceselicitedfromlearnersaspartofgoal-directed,
meaning-focusedlanguageuserequiringtheintegra1onofskillsand
knowledge”(Brindley,1994:74).
4
![Page 5: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/5.jpg)
“TheelicitaAonandevaluaAonoflanguageuse(acrossallmodaliAes)forexpressingandinterpreAngmeaningwithinawell-definedcommunica1vecontext(andaudience),foraclearpurpose,towardsavaluedgoalor
outcome”(Norris,2014:25).
5
![Page 6: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/6.jpg)
EarlyConsideraAons(Mislevyetal,2002)
• Disenchantmentwithdiscrete-skillsassessmentsfocusingongrammarandvocabularywithlargelydecontextualizedtestitems– RealizaAonthatlinguisAccompetence(grammarandvocabulary)
isnotsufficienttousealanguagetoachieveendsinsocialsituaAons
• Alignmentofassessmentwith(task-based,proficiency-oriented)communicaAveinstrucAon
• PosiAvewashbackofassessmentpracAcesoninstrucAon
6
![Page 7: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/7.jpg)
Advantages
• AssessestheabilitytouselanguagetoachieveendsinsocialsituaAons(Mislevyetal.,2002)
• AssessestheintegraAonoftopical,social,andpragmaAcknowledgewithknowledgeoftheformalelementsoflanguage
• AlignswithCommunicaAveLanguageTeachingandTask-basedLanguageInstrucAoninForeignLanguageEducaAon
• EnablesmeaningfulintegraAonofteaching,learning,andassessment(Wiggins1990)
7
![Page 8: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/8.jpg)
EarlyProponents
• ForeignServiceIns1tuteOralInterview(1950s)• GrantWiggins(1990):AuthenAcAssessment• GeoffBrindley(1994):Task-centeredAssessment
– Outcomes-basedassessment(1998andlater)
• LyleBachman&AdrianPalmer(1996):LanguageTesAnginPracAce
• TimMcNamara(1996):PerformanceTesAng• MichaelLong&JohnNorris(2000):Task-basedAssessment• RobertMislevy(2002):Evidence-basedDesign
8
![Page 9: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/9.jpg)
ShadesofMeaning
• Theinfluenceoftaskfeaturesontesttakers’cogniAveprocessesandresulAngperformance(Skehan,1998)
• SharedcharacterisAcssuchaslearner-centeredness,contextualizaAon,andauthenAcitybetweenassessmentandinstrucAon(Chalhoub-Deville,2001)
• AbilitytoaccomplishspecifictargettasksinparAcularcommunicaAonsegngs(Long&Norris,2000)
• AbilitytoengageinthekindsofacAviAescharacterisAcallyencounteredincommunicaAvelanguageteaching(Wiggins,1990)
9
![Page 10: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/10.jpg)
Evidence-CenteredDesign
10
![Page 11: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/11.jpg)
‘Data’become‘evidence’onlywhentheirrelevancetosomehypothesis,someinference,
someclaimisestablished.Intask-basedlanguageassessment,thismeansthatwhatwereallyneedtounderstandfirstandforemostistheinferen1al
argumentassociatedwiththeassessment.Whatisitspurpose?Whatdowewanttoknow,aboutwhatstudentsknoworcando,inwhatkindsof
situa1ons?(Mislevyetal.,2002:492-3).
11
![Page 12: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/12.jpg)
TheMeaningofTestResults
• Databecomeevidenceonlywhen:=TestResultsaiainmeaningonlywhen:• Theirrelevancetoahypothesis,aninference,aCLAIMhas
beenestablished.
Ø Toulmin’sArgumentModelØ Bachman/Palmer’s(2010)AssessmentUseArgumentØ Mislevyetal’sEvidence-CenteredDesign
12
![Page 13: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/13.jpg)
Evidence-CenteredDesign
Whatisthepurposeoftheassessment?
TaskAnalysis
Whatdowewanttoknowaboutwhatstudentsknoworcando?
Ability
Evidence
InwhatkindsofsituaAons? Task
13
![Page 14: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/14.jpg)
Mislevy,Steinberg,&Almond(2002)
![Page 15: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/15.jpg)
ExamineeModelAbility
KnowledgeSkills
EvidenceModelScoring
Measuring
TaskModelSchemastoelicit
evidence
AssemblyModelCombiningtaskstoproduceanassessment
![Page 16: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/16.jpg)
ExamineeModel
• ComplexofknowledgeandabiliAestobeassessed• Unobservable• Construct-driven(basedonSLAtheory)
– CommunicaAveCompetence(e.g.Bachman/Palmer1996)– ProficiencyLevelDescripAons(ILR,ACTFL)– Can-dostatements(ILR,ACTFL,CEFR)
• Developmentalstages/ImplicaAonalscaling
16
![Page 17: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/17.jpg)
EvidenceModel• Observablebehaviorsthatprovidesufficientevidenceaboutthe
knowledgeorabiliAeswewishtomeasure• EvaluaAonComponent
– Extractsthesalientcharacteris1csofwhatstudentssayordo(Keyaspectsoftheperformance)
– BasedonevaluaAonrules(rubricsforraAngscales)– Whatisvalued,andhowisitevaluated?
• MeasurementComponent– Accumula1onofinformaAontoupdatebeliefsaboutexaminees:What
dotheobservablevariablestellusabouttheexaminee’sabiliAes?– HowisevidencesynthesizedacrossmulApletasksanddifferent
performances?(Whendoyouknow,e.g.,whenyouhaveenough?)
17
![Page 18: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/18.jpg)
TaskModel• TasksorsituaAonsthatelicitthebehaviorsweneedas
evidence• SchemaforconstrucAnganddescribingthesituaAonsinwhich
examineesact• Task-driven(basedontask/domain/needsanalysis)
– Salientfeaturesoftasks– Howdotasksinfluence(andconstrain)performance?– Whatarekeyfeaturesoftargetlanguageuses(TLUs)intasks
(authenAcity)?• Developmentalstages/ImplicaAonalscaling
– Hierarchyoftasks
18
![Page 19: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/19.jpg)
AssemblyModel
• Mixandnumberoftasksthataresufficientforanassessment– Determinestherangeofcircumstancesthatneedtobecovered
(tobeabletogeneralize)– Controlsthedifficultyoftasks– ManageswhatinformaAonaccumulatesordoesnotaccumulate
19
![Page 20: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/20.jpg)
ExamineeModelPragmaAcComp.Socio-ling.Comp.TextCompetenceGram./Lex.Comp.
EvidenceModelSuccess/Failure
Quality:Howwell?QuanAty:Howmuch?
TaskModelGlobalTasks
Context/ContentRoleofInterlocutor
AssemblyModelChecks/ProbesRangeofTopicsRangeofTasksKindsofTasks
![Page 21: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/21.jpg)
SpeechSample
• Providescumula1veevidenceoftheexaminee’slanguageability
• RepresentaAveoftheexaminee’slanguageability– FuncAons,TextualorganizaAon,Register,Culturalcompetence– Varietyoftopicsandcontexts– ConcernforaffecAveschemata(warm-upetc.)
• IndicaAveoftheexaminee’scontrolovertheabove– Random,emerging,developing,sustained
21
![Page 22: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/22.jpg)
Task-centeredandConstruct-centered
Task-centeredperspecAve• Featuresoflanguage-usesituaAonsthatreveal:• Thelanguage-usecompetencesthatareofinterest;and• Thekindsofperformancesthatcontainevidenceabout
language-usecompetencesConstruct-centeredperspecAve• WhatperformancesinwhatsituaAonstelluswhatabout
studentabiliAes?
22
![Page 23: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/23.jpg)
TheConstructoftheOPICommunicaAveCompetence
23
![Page 24: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/24.jpg)
LanguageAbility(Bachman/Palmer1996)
• OrganizaAonalability– GrammaAcalability(sentencelevel)– Textualability(textlevel)
• PragmaAcability– FuncAonalability
• IdeaAonal,instrumental,heurisAc,imaginaAve– SociolinguisAcability
• ConvenAons,register,culturalreferences
24
![Page 25: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/25.jpg)
LanguageAbilityinOPIs
• FuncAonalability– Instrumental(negoAaAngdailyinteracAons)– IdeaAonal(describingandreporAng)– ImaginaAve(narraAngandhypothesizing)– HeurisAc(explainingandarguingapoint)
• Textualability– Word-andsentence-lengthuierances– Connectedtextsofvariouscomplexity
• SociolinguisAcability– Register,culturalcompetence
25
![Page 26: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/26.jpg)
DevelopmentalHierarchies(ILR/ACTFL)
• FuncAonalability– AskingforandprovidingsimplepersonalinformaAon– DescribingandreporAng– NarraAngandexplaining– Arguingandhypothesizing
• SociolinguisAcability– None– Oneregister– Twoormoreregisters– HighlevelofacculturaAon(controlofconvenAons)
26
![Page 27: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/27.jpg)
DevelopmentalHierarchies(ILR/ACTFL)
• OrganizaAonalability• UseoforganizaAonaltypes(texttypes)
– Stringsofwords– Sentence-lengthuierances– Stringsofsentences– Connectedspeechofvariouslengths(cohesion)
27
![Page 28: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/28.jpg)
DevelopmentoforganizaAonalabilityI
• GillianBrown(1994):Modesofunderstanding– HierarchiesofcogniAveload
• IdenAfying:words• Proceduralunderstanding:onesentenceataAme
– Fullysupportedbytheexternalworld;ParAalunderstandingissufficient;Immediatefeedback
• NarraAveunderstanding:texts• Understandingofargument:complextexts
28
![Page 29: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/29.jpg)
DevelopmentoforganizaAonalabilityII
• ManfredPienemann(1998):Processabilitytheory– GrammaAcalmemory:featureunificaAon
• Lemmaaccess:singlewords,nofeatures Words– LisAngsinglewordsorexpressions;nosenseforsyntaxor
morphology
• Categoryprocedure• Phrasalprocedure• Sentenceprocedure Sentences• Subordinateclauseprocedure Texts
29
![Page 30: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/30.jpg)
TheBiggerPictureTheAssessmentUseArgument
30
![Page 31: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/31.jpg)
Performance
Task
Consequences
Decisions
InterpretaAons
AssessmentRecords(Scores,Descriptors)
TestDevelop
men
t
InterpretaAo
nandUse
![Page 32: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/32.jpg)
ClaimTeresawilldowellinherfinalexam.
RebuialTeresahastest
anxiety.
WarrantGoodstudentsdowellinfinalexams.
BackingStaAsAcalrelaAonshipbetweenGPAandfinal
exams.Data
Teresaisagoodstudent.
RefutaAonTeresahasn‘thadtestanxietyinanyfinals.
Onaccountof
Because
Unless
ThereforeExcept
![Page 33: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/33.jpg)
ArgumentApproach
Warrants• Explicit• Generalizable• ProvideiniAalsupportthattheconnecAonbetweenthedata
andtheclaimisappropriateandlegiAmate
Backing• Providestheevidenceforthewarrant• Laws,rules,principles,facts• WidelyacceptedassumpAonsorsharedexperience
33
![Page 34: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/34.jpg)
Claim4
Claim3
Claim2
Claim1
Data
Data
Data
Data
WarrantsRebuials
WarrantsRebuials
WarrantsRebuials
WarrantsRebuials
Consequences:Beneficial
Decisions:ValuessensiAve,Equitable
InterpretaAons:Meaningful,Generalizable,ImparAal,Relevant,Sufficient
AssessmentRecords:Consistent
Performance
Task
![Page 35: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/35.jpg)
Example:InterpretaAons• Generalwarrant:Theassessmentrecordcanbeinterpreted
meaningfully.• Specificwarrant:TheraAngreflectswhattheexamineeisable
andnotabletodolinguisAcally.• Backing:Theconstructoftheassessment,i.e.proficiency,is
describedincomprehensivecan-dostatements.• Rebuial:TheconstructwasdevelopedexperienAallyandisnot
theory-based.• RepudiaAon:TheconstructalignswellwithcurrentSLA
theories.
35
![Page 36: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/36.jpg)
Performance
Task
Consequences
Decisions
InterpretaAons
AssessmentRecords(Scores,Descriptors)
TestDevelop
men
t
Impact
Reliability
Validity
![Page 37: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/37.jpg)
TheEvoluAonofthe(ACTFL)OPI
HighReliability/QualityAssurance• SubstanAveanddetailedTesAngProtocol• SubstanAveanddetailedRaAngProtocol• ExtensivetestertrainingandcerAficaAonprocedures• Extensiveongoingtesterandraternorming• BlinddoubleraAngsandarbitraAon• Quality-controlledoperaAonaltesAngandraAng(LTI)
37
![Page 38: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/38.jpg)
What’sMissing?
• …theevaluaAonoflanguageuseforexpressingmeaningwithinawell-definedcommunicaAvecontextandaudience,foraclearpurpose…(Norris,2014)
• ...meaningfulintegraAonofteaching,learning,andassessment...(Wiggins,1998);...posiAvewashbackofassessmentpracAcesoninstrucAon...(Mislevyetal.,2002)
38
TaskAnalysisDomainAnalysisInterpretaAon
CurricularTie-inConsequences
![Page 39: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/39.jpg)
SomeBestPracAceExamples
• RevisedFSISpeakingTest(2014)– TaskAnalysis
• OPI+(NSW)(2016) MarlaFedere(2016)– TaskAnalysis– CurricularTie-in
39
![Page 40: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/40.jpg)
KeyTake-Aways• Assessmentsneedtobeboth,task-basedandconstruct-based
toyieldevidenceaboutanexaminee’sabiliAes• Currentvalidity/validaAontheoriesputindoubtthenoAonofa
generalproficiencytest• Currentvalidity/validaAontheoriescallfor:
– Domain-specificproficiencytests• Evenforgenerallanguageprogramsinschools/universiAes
– SubstanAalcurricularAe-in• Thefutureofproficiencyassessmentmaybetheassessmentof
Proficiency+– Proficiencyforamission/jobplusmoregeneraldomains
40
![Page 41: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/41.jpg)
Bibliography• Bachman,L.F.,&Palmer,A.S.(1996).LanguageTes*nginPrac*ce:Designingand
DevelopingUsefulLanguageTests.Oxford:OxfordUniversityPress.• Bachman,L.F.&Palmer,A.S.(2010).LanguageAssessmentinPrac*ce:Developing
LanguageAssessmentsandJus*fyingtheirUseintheRealWorld.Oxford:OxfordUniversityPress.
• Brindley,G.(1994).Task-centredassessmentinlanguagelearning:Thepromiseandthechallenge.InBird,N.,Falvey,P.,Tsui,A.,Allison,D.&McNeill,A.,eds.,LanguageandLearning:PapersPresentedattheAnnualInterna*onalLanguageinEduca*onConference.HongKong:HongKongEducaAonDepartment,73–94.
• Brown,G.(1994).Modesofunderstanding.InG.Brown,K.Malmkjær,A.Pollii&J.Williams,eds.,LanguageandUnderstanding.Oxford:OxfordUniversityPress,9-20.
• Chalhoub-Deville,M.(2001).Task-basedassessments:CharacterisAcsandvalidityevidence.InBygate,M.,Skehan,P.andSwain,M.,eds.,ResearchingPedagogicTasks:SecondLanguageLearning,TeachingandTesAng.NewYork:PearsonEducaAon,210–228.
• Federe,M.(2016,September).AuthenAcity:FromTheorytoReality.InvitedPaper.2016LEARNTaskBasedAssessment.UniversiAesatShadyGrove,Rockville,MD.
• Long,M.H.andNorris,J.M.(2000).Task-basedlanguageteachingandassessment.InByram,M.,ed.,EncyclopediaofLanguageTeaching.London:Routledge,597–603.
41
![Page 42: Task-Based Language Assessment and Tes1ng for Proficiency ... · 1 Task-Based Language Assessment and Tes1ng for Proficiency: Where Do the Twain Meet? Erwin Tschirner L.E.A.R.N.](https://reader030.fdocuments.in/reader030/viewer/2022041300/5e0f80c317ab6d697e3200b1/html5/thumbnails/42.jpg)
Bibliography• McNamara,T.(1996).MeasuringSecondLanguagePerformance.NewYork:Longman.• Mislevy,R.J.,Steinberg,L.S.,&Almond,R.G.(2002).Designandanalysisintask-based
languageassessment.Languagetes*ng,19,477-496.• Norris,J.M.(2014,April).Howdoweassesstask-basedperformance?InvitedLARC/
CALPERtesAngandassessmentwebinar.Online:hips://larc.sdsu.edu/testassesswebinar/jnorris/Norris_Task-BasedAssessment_PPT.pdf
• Pienemann,Manfred(1998).LanguageProcessingandSecondLanguageDevelopment:ProcessabilityTheory.Amsterdam:JohnBenjamins.
• Skehan,P.(1998).ACogni*veApproachtoLanguageLearning.Oxford:OxfordUniversityPress.
• Toulmin,S.(1958).TheUsesofArgument.Cambridge:CambridgeUniversityPress.• Wiggins,G.(1990).TheCaseforAuthen*cAssessment.ERICDigest.Online:hip://
files.eric.ed.gov/fulltext/ED328611.pdf.
42