Table of Contents
Foreword by Andreas Schleicher....................................................................................................xxviii
Foreword by Chris Dede.................................................................................................................... xxx
Preface.............................................................................................................................................xxxiii
Acknowledgment..........................................................................................................................xxxviii
Volume ISection 1
Defining Real-World Skills in Technology-Rich Environments
Chapter 1TwentyFirstCenturySkillsvs.DisciplinaryStudies?........................................................................... 1
Lars Vavik, Stord/Haugesund University College, NorwayGavriel Salomon, Hiafa University, Israel
Chapter 2DigitalCompetence:ANetofLiteracies.............................................................................................. 13
Edith Avni, Toward Digital Ethics Initiative, IsraelAbraham Rotem, Toward Digital Ethics Initiative, Israel
Chapter 3TheApplicationofTransdisciplinaryTheoryandPracticetoSTEMEducation................................. 42
Susan Malone Back, Texas Tech University, USAHeather Greenhalgh-Spencer, Texas Tech University, USAKellilynn M. Frias, Texas Tech University, USA
Chapter 4TheSOARStrategiesforOnlineAcademicResearch:HelpingMiddleSchoolStudentsMeetNewStandards............................................................................................................................................... 68
Carolyn Harper Knox, University of Oregon, USALynne Anderson-Inman, University of Oregon, USAFatima Terrazas-Arellanes, University of Oregon, USAEmily Deanne Walden, University of Oregon, USABridget Hildreth, University of Oregon, USA
Chapter 5TheValueofMetacognitionandReflectivityinComputer-BasedLearningEnvironments.............. 105
Sammy Elzarka, University of La Verne, USAValerie Beltran, University of La Verne, USAJessica Decker, University of La Verne, USAMark Matzaganian, University of La Verne, USANancy T. Walker, University of La Verne, USA
Chapter 6AFrameworkforDefiningandEvaluatingTechnologyIntegrationintheInstructionofReal-WorldSkills........................................................................................................................................ 137
J. Christine Harmes, Assessment Consultant, USAJames L. Welsh, University of South Florida, USARoy J. Winkelman, University of South Florida, USA
Chapter 7EquippingAdvancedPracticeNurseswithReal-WorldSkills........................................................... 163
Patricia Eckardt, Stony Brook University, USABrenda Janotha, Stony Brook University, USAMarie Ann Marino, Stony Brook University, USADavid P. Erlanger, Stony Brook University, USADolores Cannella, Stony Brook University, USA
Section 2Technology Tools for Learning and Assessing Real-World Skills
Chapter 8SimulationsforSupportingandAssessingScienceLiteracy............................................................. 191
Edys S. Quellmalz, WestEd, USAMatt D. Silberglitt, WestEd, USABarbara C. Buckley, WestEd, USAMark T. Loveland, WestEd, USADaniel G. Brenner, WestEd, USA
Chapter 9UsingtheCollegiateLearningAssessmenttoAddresstheCollege-to-CareerSpace........................ 230
Doris Zahner, CAE, USAZachary Kornhauser, CAE, USARoger W. Benjamin, CAE, USARaffaela Wolf, CAE, USAJeffrey T. Steedle, Pearson, USA
Chapter 10Rich-MediaInteractiveSimulations:LessonsLearned...................................................................... 261
Suzanne Tsacoumis, HumRRO, USA
Chapter 11AnApproachtoDesign-BasedImplementationResearchtoInformDevelopmentofEdSphere®:ABriefHistoryabouttheEvolutionofOnePersonalizedLearningPlatform................................... 284
Carl W. Swartz, MetaMetrics, USA & University of North Carolina, USASean T. Hanlon, MetaMetrics, USAE. Lee Childress, Corinth School District, USAA. Jackson Stenner, MetaMetrics, USA & University of North Carolina, USA
Chapter 12ComputerAgentTechnologiesinCollaborativeAssessments........................................................... 319
Yigal Rosen, Harvard University, USAMaryam Mosharraf, Pearson, USA
Chapter 13AToughNuttoCrack:MeasuringCollaborativeProblemSolving................................................... 344
Lei Liu, Educational Testing Service, USAJiangang Hao, Educational Testing Service, USAAlina A. von Davier, Educational Testing Service, USAPatrick Kyllonen, Educational Testing Service, USADiego Zapata-Rivera, Educational Testing Service, USA
Chapter 14Animalia:CollaborativeScienceProblemSolvingLearningandAssessment.................................. 360
Sara Bakken, Pearson Education, USAJohn Bielinski, Pearson Education, USACheryl K. Johnson, Pearson Education, USAYigal Rosen, Harvard University, USA
Chapter 15UsingTechnologytoAssessReal-WorldProfessionalSkills:ACaseStudy..................................... 385
Belinda Brunner, Pearson, UKKirk A. Becker, Pearson, USANoel Tagoe, Chartered Institute of Management Accountants, UK
Volume II
Chapter 16AssessmentintheModernAge:ChallengesandSolutions................................................................ 415
Mahmoud Emira, City & Guilds of London Institute, UKPatrick Craven, City & Guilds of London Institute, UKSharon Frazer, City & Guilds of London Institute, UKZeeshan Rahman, City & Guilds of London Institute, UK
Chapter 17Technology-AssistedLearningforStudentswithModerateandSevereDevelopmentalDisabilities.......................................................................................................................................... 445
Diane M. Browder, University of North Carolina at Charlotte, USAAlicia Saunders, University of North Carolina at Charlotte, USAJenny Root, University of North Carolina at Charlotte, USA
Chapter 18MitigationofTestBiasinInternational,Cross-NationalAssessmentsofHigher-OrderThinkingSkills................................................................................................................................................... 472
Raffaela Wolf, CAE, USADoris Zahner, CAE, USA
Chapter 19Evidence-CenteredConceptMapinComputer-BasedAssessmentofCriticalThinking.................. 502
Yigal Rosen, Harvard University, USAMaryam Mosharraf, Pearson, USA
Chapter 20“VisittoaSmallPlanet”:AchievementsandAttitudesofHighSchoolStudentstowardsLearningonFacebook–ACaseStudy.............................................................................................................. 528
Rikki Rimor, Kibbutzim College of Education Technology and the Arts, IsraelPerla Arie, Kibbutzim College of Education Technology and the Arts, Israel
Chapter 21Cross-BorderCollaborativeLearningintheProfessionalDevelopmentofTeachers:CaseStudy–OnlineCoursefortheProfessionalDevelopmentofTeachersinaDigitalAge................................. 558
Rafi Davidson, Kaye Academic College of Education, IsraelAmnon Glassner, Kaye Academic College of Education, Israel
Section 3Automated Item Generation and Automated Scoring Techniques for Assessment and
Feedback
Chapter 22UsingAutomatedProcedurestoGenerateTestItemsThatMeasureJuniorHighScienceAchievement....................................................................................................................................... 590
Mark Gierl, University of Alberta, CanadaSyed F. Latifi, University of Alberta, CanadaHollis Lai, University of Alberta, CanadaDonna Matovinovic, CTB/McGraw-Hill, USAKeith A. Boughton, CTB/McGraw-Hill, USA
Chapter 23AutomatedScoringinAssessmentSystems....................................................................................... 611
Michael B. Bunch, Measurement Incorporated, USADavid Vaughn, Measurement Incorporated, USAShayne Miel, LightSide Labs, USA
Chapter 24AutomatedScoringofMulticomponentTasks................................................................................... 627
William Lorié, Questar Assessment, Inc., USA
Chapter 25AdvancesinAutomatedScoringofWritingforPerformanceAssessment........................................ 658
Peter W. Foltz, Pearson, USA & University of Colorado – Boulder, USA
Chapter 26UsingAutomatedFeedbacktoImproveWritingQuality:OpportunitiesandChallenges.................. 678
Joshua Wilson, University of Delaware, USAGilbert N. Andrada, Connecticut State Department of Education, USA
Section 4Analysis, Interpretation, and Use of Learning and Assessment Data from Technology
Rich Environments
Chapter 27AssessingProblemSolvinginTechnology-RichEnvironments:WhatCanWeLearnfromOnlineStrategyIndicators?............................................................................................................................ 705
Jean-Francois Rouet, University of Poitiers, FranceZsofia Vörös, University of Poitiers, FranceMatthias von Davier, Educational Testing Service, USA
Chapter 28AnalyzingProcessDatafromTechnology-RichTasks....................................................................... 724
Lisa Keller, University of Massachusetts – Amherst, USAApril L. Zenisky, University of Massachusetts – Amherst, USAXi Wang, University of Massachusetts – Amherst, USA
Chapter 29AnalyzingProcessDatafromProblem-SolvingItemswithN-Grams:InsightsfromaComputer-BasedLarge-ScaleAssessment........................................................................................................... 749
Qiwei He, Educational Testing Service, USAMatthias von Davier, Educational Testing Service, USA
Chapter 30AssessmentofTaskPersistence.......................................................................................................... 777
Kristen E. DiCerbo, Pearson, USA
Chapter 31AssessingEngagementduringtheOnlineAssessmentofReal-WorldSkills..................................... 804
J. Christine Harmes, Assessment Consultant, USASteven L. Wise, Northwest Evaluation Association, USA
Compilation of References............................................................................................................ xxxix
About the Contributors................................................................................................................. cxxix
Index................................................................................................................................................. cxlvi
Foreword
Weneedtothinkharderabouthowweprepareyoungpeoplefortomorrow’sworld.Inthepast,educa-tionwasaboutteachingstudentssomething.Now,it’saboutmakingsurethatstudentsdevelopareliablecompassandthenavigationskillstofindtheirownwaythroughanuncertain,volatile,andambiguousworld.Now,schoolsneedtopreparestudentsforaworldinwhichmostpeoplewillneedtocollaboratewithpeopleofdiverseculturaloriginsandappreciatedifferentideas,perspectivesandvalues;aworldinwhichpeopleneedtodecidehowtotrustandcollaborateacrosssuchdifferences;andaworldinwhichtheirliveswillbeaffectedbyissuesthattranscendnationalboundaries.Technologyhasbecomethekeytobridgespaceandtimeinallofthis.
Thesedays,wenolongerknowexactlyhowthingswillunfold.Weareoftensurprisedandneedtolearnfromtheextraordinary,andsometimeswemakemistakesalongtheway.Anditwilloftenbethemistakesandfailures,whenproperlyunderstood,thatcreatethecontextforlearningandgrowth.Agenerationago,teacherscouldexpectthatwhattheytaughtwouldlasttheirstudentsalifetime.Today,schoolsneedtopreparestudentsformorerapideconomicandsocialchangethaneverbefore,forjobsthathavenotyetbeencreated,tousetechnologiesthathavenotyetbeeninvented,andtosolvesocialproblemsthatwedon’tyetknowwillarise.
Howdowefostermotivated,engagedlearnerswhoarepreparedtoconquertheunforeseenchal-lengesoftomorrow,nottomentionthoseoftoday?Thedilemmaforeducatorsisthatroutinecognitiveskills—theskillsthatareeasiesttoteachandeasiesttotest—arealsotheskillsthatareeasiesttodigitize,automate,andoutsource.Thereisnoquestionthatstate-of-the-artknowledgeandskillsinadisciplinewillalwaysremainimportant.Innovativeorcreativepeoplegenerallyhavespecializedskillsinafieldofknowledgeorapractice.Andasmuchas‘learningtolearn’skillsareimportant,wealwayslearnbylearningsomething.However,educationalsuccessisnolongeraboutreproducingcontentknowledge,butaboutextrapolatingfromwhatweknowandapplyingthatknowledgeinnovelsituations.Putsim-ply,theworldnolongerrewardspeopleforwhattheyknow—Googleknowseverything—butforwhattheycandowithwhattheyknow.Becausethatisthemaindifferentiatortoday,educationtodayneedstobemuchmoreaboutwaysofthinking,involvingcreativity,criticalthinking,problem-solving,anddecision-making;aboutwaysofworking,includingcommunicationandcollaboration;abouttoolsforworking,includingthecapacitytorecognizeandexploitthepotentialofnewtechnologies;and,lastbutnotleast,aboutthesocialandemotionalskillsthathelpusliveandworktogether.
Conventionally,ourapproachtoproblemswastobreakthemdownintomanageablebitsandpiecesandthentoteachstudentsthetechniquestosolvethem.Buttodaywecreatevaluebysynthesizingthedisparatebits.Thisisaboutcuriosity,open-mindedness,andmakingconnectionsbetweenideasthatpre-
xxviii
Foreword
viouslyseemedunrelated,whichrequiresbeingfamiliarwithandreceptivetoknowledgeinotherfieldsthanourown.Ifwespendourwholelifeinasiloofasinglediscipline,wewillnotgaintheimaginativeskillstoconnectthedotswherethenextinventionwillcomefrom.
Equallyimportant,themorecontentknowledgewecansearchandaccess,themoreimportantbecomesthecapacitytomakesenseofthiscontent—thecapacityofindividualstoquestionorseektoimprovetheacceptedknowledgeandpracticesoftheirtime.Inthepast,youcouldtellstudentstolookintoanencyclopediawhentheyneededsomeinformation,andyoucouldtellthemthattheycouldgenerallyrelyonwhattheyfoundtobetrue.Today,literacyisaboutmanagingnon-linearinformationstructures,buildingyourownmentalrepresentationofinformationasyoufindyourownwaythroughhypertextontheInternet,anddealingwithambiguity—interpretingandresolvingconflictingpiecesofinformationthatwefindsomewhereontheWeb.
Perhapsmostimportantly,intoday’sschools,studentstypicallylearnindividuallyandattheendoftheschoolyear,wecertifytheirindividualachievements.Butthemoreinterdependenttheworldbe-comes,themoreweneedgreatcollaboratorsandorchestrators.Innovationtodayisrarelytheproductofindividualsworkinginisolationbutanoutcomeofhowwemobilize,share,andlinkknowledge.Intheflatworld,everythingthatisourproprietaryknowledgetodaywillbeacommodityavailabletoeveryoneelsetomorrow.Expresseddifferently,schoolsneedtodriveashiftfromaworldwhereknowledgeisstackedupsomewhere,depreciatingrapidlyinvalue,towardsaworldinwhichtheenrichingpowerofcommunicationandcollaborativeflowsisincreasing.Andtheywillneedtohelpthenextgenerationtobetterreconcileresilience(managinginanimbalancedworld)withgreatersustainability(puttingtheworldbackintobalance).
Thisisatoughagenda.Whatiscertainisthatitwillnevermaterialiseunlessweareabletoclearlyconceptualiseandmeasurethose21stcenturyknowledgeareasandskills.Withoutrigorousconcep-tualisation,wewillnotbeabletobuildmeaningfulcurriculaandpedagogiesaroundtheseknowledgeareasandskills.And,attheendoftheday,whatisassessediswhatgetstaught.Thisvolumemakesamajorstepinadvancingthisfrontier.Itexaminesarangeofskillsthatareimportant;itlooksatinnova-tivemeasurementmethodstomaketheseskillsamenabletoquantitativeassessmentinwaysthattheybecomeactivatorsofstudents’ownlearning,anditlooksathowwecanlearntodrinkfromthefirehoseofincreasingdatastreamsthatarisefromnewassessmentmodes.
Andreas Schleicher Organisation for Economic Co-Operation and Development (OECD), France
xxix
Foreword
InitslandmarkreportEducation for Life and Work in the 21st Century,theNationalResearchCouncil(2012)described“deeperlearning”asaninstructionalapproachimportantinpreparingstudentswithsophisticatedcognitive,intrapersonal,andinterpersonalskills.Theapproachesrecommendedbyadvo-catesofdeeperlearningarenotnew,andhistoricallytheseinstructionalstrategieshavebeendescribedunderavarietyofterms.Untilnow,however,theyhavebeenrarelypracticedwithintheschools(Dede,2014),resultinginthesadsituationthatstudentswhoexcelinschoolmaystruggleintherealworld.Andstudentswhostruggleinschoolarelikelytosinkintherealworld.Various“deeperlearning”ap-proachesaredescribedbelow.
• Case-basedlearninghelpsstudentsmasterabstractprinciplesandskillsthroughtheanalysisofreal-worldsituations;
• Multiple, varied representations of concepts provide different ways of explaining complicatedthings,showinghowthosedepictionsarealternativeformsofthesameunderlyingideas;
• Collaborativelearningenablesateamtocombineitsknowledgeandskillsinmakingsenseofacomplexphenomenon;
• Apprenticeshipsinvolveworkingwithamentorwhohasaspecificreal-worldroleand,overtime,enablesmasteryoftheirknowledgeandskills;
• Self-directed,life-wide,open-endedlearningisbasedonstudents’passionsandisconnectedtostudents’identitiesinwaysthatfosteracademicengagement,self-efficacy,andtenacity;
• Learning for transferemphasizes that themeasureofmastery isapplication in life rather thansimplyintheclassroom;
• Interdisciplinarystudieshelpstudentsseehowdifferingfieldscancomplementeachother,offer-ingaricherperspectiveontheworldthananysingledisciplinecanprovide;
• Personalizedlearningensuresthatstudentsreceiveinstructionandsupportsthataretailoredtotheirneedsandresponsivetotheirinterests(U.S.DepartmentofEducation,2010;Wolf,2010;Rose&Gravel,2010);
• Connectedlearningencouragesstudentstoconfrontchallengesandpursueopportunitiesthatexistoutsideoftheirclassroomsandcampuses(Itoetal.,2013);and
• Diagnosticassessmentsareembedded into learningandare formative for further learningandinstruction(Dede,2012).
Theseentailverydifferentteachingstrategiesthanthefamiliar,lecture-basedformsofinstructioncharacteristicofindustrial-eraschooling,withitsone-size-fits-allprocessingofstudents.Ratherthanrequiringrotememorizationandindividualmasteryofprescribedmaterial,theyinvolvein-depth,dif-
xxx
Foreword
ferentiatedcontent;authenticdiagnosticassessmentembeddedininstruction;activeformsoflearning,often collaborative; and learning about academic subjects linked to personal passions and infusedthroughoutlife.
Thechaptersinthisbookdemonstratethatnewtoolsandmediacanbeveryhelpfultomanyteacherswhowouldotherwisestruggletoprovidethesekindsofinstructionfordeeperlearning(Dede,2014).Byanalogy,imaginethatyouwishtovisitafriend20milesaway.Youcouldwalk(andsomepeoplewouldprefertodoso),butitwouldbemucheasiertouseabicycle,anditwouldbefareasierstilltouseacar.Inshort,teacherswhowishtopreparetheirstudentsfortherealworld,aswellasforfurtheracademics,don’thavetouseeducationaltechnology;theymayprefertowalk.Realistically,however,many,ifnotmost,teacherswillbehard-pressedtogetfromindustrial-styleinstructiontodeeperlearningwithoutthevehiclesofdigitaltools,media,andexperiences.
InanextensivereviewoftheliteratureontechnologyandteachingfortheforthcomingAmericanEducationalResearchAssociation(AERA)HandbookofResearchonTeaching(5thEdition),BarryFishmanandI(inpress)notetheimportantdistinctionbetweenusingtechnologytodoconventionalthingsbetter andusing technology todobetter things (Roschelleet al.,2000).While theremaybevalueindoingsometypesofconventionalinstructionbetter(i.e.,moreefficientlyandeffectively),therealvalueintechnologyforteachingliesinrethinkingtheenterpriseofschoolinginwaysthatunlockpowerfullearningopportunitiesandmakebetteruseoftheresourcespresentinthe21st-centuryworld.
Inourreview,weconsiderhowandunderwhatconditionstechnologycanbeproductivelyemployedbyteacherstomoreeffectivelypreparestudentsforthechallengespresentedbyarapidlyevolvingworld.Wearguethattechnologyasacatalystiseffectiveonlywhenusedtoenablelearningwithrichercontent,morepowerfulpedagogy,morevalidassessments,andlinksbetweenin-andout-of-classroomlearning.Theexaminedthefollowingtechnologiesindepth:
• Collaborationtools,includingWeb2.0technologiesandtoolsthatsupportknowledgebuilding;• Onlineandhybrideducationalenvironments,whichareincreasinglybeingusedtobroadenaccess
toeducationbutalsohavethepotentialtoshiftthewayweconceiveofteachingandlearning;• Toolsthatsupportlearnersasmakersandcreators,whichhavetheirdeeprootsinhelpingstudents
learntobecomeprogrammersofcomputers(andnotjustusersofthem);• Immersivemediathatcreatevirtualworldstosituatelearningoraugmenttherealworldwithan
overlayofcomputationalinformation;and• Gamesandsimulationsthataredesignedtoenhancestudentmotivationandlearning.
Thisbookprovidesexamplesoftheseandotherpowerfultechnologiestoaidthistypeofinstruction.Ifusedinconcert,thesedeeper-learningtechnologiescanhelppreparestudentsforlifeandworkinthe21stcentury,mirroringintheclassroomsomepowerfulmethodsofknowinganddoingthatpervadetherestofsociety.Further,theycanbeusedtocreateapractical,cost-effectivedivisionoflabor,onethatempowersteacherstoperformcomplexinstructionaltasks.Inaddition,thesemediacanaddressthelearningstrengthsandpreferencesofstudentsgrowingupinthisdigitalage,includingbridgingformalinstructionandinformallearning.And,finally,thesetechnologiescanprovidepowerfulmechanismsforteacherlearning;bywhicheducatorsdeepentheirprofessionalknowledgeandskillsinwaysthatmirrorthetypesoflearningenvironmentsthroughwhichtheywillguidetheirstudents.
xxxi
Foreword
Atatimeinhistorywhencivilizationfacescrisesthatweneedthefullcapacityofpeopleacrosstheworldtoresolve,thisvolumeprovidesanexemplarysuiteofpracticalwaystomoveforwardwithcur-ricula,instruction,andassessmentsthataretrulyorientedto21st-centurylifeandwork.
Chris Dede Harvard University, USA
REFERENCES
Dede,C.(2012).Interweaving assessments into immersive authentic simulations: Design strategies for diagnostic and instructional insights(CommissionedWhitePaperfortheETSInvitationalResearchSymposiumonTechnologyEnhancedAssessments).Princeton,NJ:EducationalTestingService.
Dede,C.(2014).The role of technology in deeper learning.NewYork,NY:JobsfortheFuture.
Fishman,B.,&Dede,C.(inpress).Teachingandtechnology:Newtoolsfornewtimes.InD.Gitomer&C.Bell(Eds.),Handbook of research on teaching(5thed.).NewYork,NY:Springer.
Ito,M.,Gutiérrez,K.,Livingstone,S.,Penuel,B.,Rhodes,J.,&Salen,K.…Watkins,S.C.(2013).Connectedlearning:Anagendaforresearchanddesign.Irvine,CA:DigitalMediaandLearningRe-searchHub.
NationalResearchCouncil.(2012).Education for life and work: Developing transferable knowledge and skills in the 21st century.Washington,DC:TheNationalAcademiesPress.Retrievedfromhttp://www.nap.edu/catalog.php?record_id=13398
Roschelle,J.M.,Pea,R.D.,Hoadley,C.M.,Gordin,D.N.,&Means,B.M.(2000).Changinghowandwhatchildrenlearninschoolwithcomputer-basedtechnologies.The Future of Children: Children and Computer Technology,10(2),76–101.doi:10.2307/1602690PMID:11255710
Rose,D.H.,&Gravel,J.W.(2010).Universaldesignforlearning.InE.Baker,P.Peterson,&B.McGaw(Eds.),International Encyclopedia of Education(3rded.).Oxford,UK:Elsevier.doi:10.1016/B978-0-08-044894-7.00719-3
U.S.DepartmentofEducation.(2010).Transforming American education: Learning powered by technol-ogy (National Educational Technology Plan 2010).Washington,DC:OfficeofEducationalTechnology,U.S.DepartmentofEducation.
Wolf, M. A. (2010, November). Innovate to educate: System [re]design for personalized learning.Washington,DC:SoftwareandInformationIndustryAssociation.
xxxii
Preface
Changesintheworldeconomy,specificallytowardinformationindustries,havechangedtheskillsetdemandofmanyjobs(OrganizationforEconomicDevelopment[OECD],2012a).Informationiscre-ated,acquired,transmitted,andused—ratherthansimplylearned—byindividuals,enterprises,organi-zations,andcommunitiestopromoteeconomicandsocialdevelopment.Majoremployersandpolicymakersare increasinglyasking teachersandeducators tohelpstudentsdevelopso-calledreal-worldskills(Gallup,2013).Whilelearningbasicnumeracyandliteracyskillsstilliscrucialtosuccessinthejobmarket,developingreal-worldskillsalsoisessentialtosuccessinthejobmarketandworldwideeconomicdevelopment.
Real-world skills, or “21st century skills,” include critical thinking, collaborative problem solv-ing,creativity,andglobalcompetency.Theseskillsthatfacilitatemasteryandapplicationofscience,mathematics,languagearts,andotherschoolsubjectswillgrowinimportanceoverthecomingdecade(NationalResearchCouncil,2012;OECD,2012a,2012b).Awiderangeofinitiativesandprogramsineducationpromotelearningandassessmentofreal-worldskills.Theseinclude,forexample,theCom-monCoreStateStandards(NationalGovernorsAssociationCenterforBestPracticesandCouncilofChiefStateSchoolOfficers,2010a,2010b),NextGenerationScienceStandards(NationalResearchCouncil,2013),CommonEuropeanFrameworkofReference(CouncilofEurope,2011),Partnershipfor21stCenturySkills(Partnershipfor21stCenturySkills,2009),EducationforLifeandWork(NationalResearchCouncil,2012),andassessmentframeworksintheProgrammeforInternationalStudentAs-sessment(PISA)(OECD,2013).
Becauseoftheimportanceofpromotingtheseskills,wehaveembarkedonajourneytocreateaHandbook of Research on Technology Tools for Real-World Skill Development.Becauseconceptionsandeducationalapplicationsofreal-worldskillsareevolvingrapidly,wehavewelcomedawiderangeofskillsintheHandbook.Thefollowingfourstrandsofskillsarerepresentedinthechapters:Think-ing skillsrefertohigher-ordercognitionanddispositionssuchascriticalthinking,complexproblemsolving,metacognition,andlearningtolearn.Social skillsrefertoattitudesandbehaviorsthatenablesuccessfulcommunicationandcollaboration.Global skillsrefertoattitudesandbehaviorsthatempha-sizetheindividual’srolein,andawarenessof,thelocalaswellastheglobalandmulticulturalenviron-ment.Digital skillsemphasizeinformationanddigitalliteraciesneededinthetechnology-richworldinwhichwelive.Similarly,thechaptersinthisHandbookdescribearangeoftechnologytoolstosupportteaching,learning,assessmentforlearning(e.g.,Stiggins,2005;Wiliam,2011),feedbackforlearning(e.g.,Hattie,&Timperley,2007;Shute,2008),andscoringofstudentresponses.Forexample,section1includeschaptersoncurriculaandframeworksforteachingreal-worldskills;thechaptersinsection2describespecifictechnologytoolsforteaching,learning,andassessingreal-worldskills;thechaptersin
xxxiii
Preface
section3describeautomatedscoringtoolsforassessmentandlearning;andsection4containschaptersontechniquesforanalyzingdatafromtechnology-basedperformanceassessments.Helpingstudentslearnreal-worldskills—thatis,tointernalizethemandusethemflexiblyacrossarangeofchallengesandcontextsintheireverydayandworklives—isasignificanteducationalchallenge.Real-worldskillscannotbetaughtinasinglecourseorinasingleyearofschooling.Andassessingreal-worldskillstoprovidefeedbacktoguidedevelopmentofthoseskillscannotbeaccomplishedusingconventional,large-scaleassessmentandscorereportingmethodsalone.Thetechnologytoolsdescribedhererepresenttherangeofcurrentanddevelopingcapabilitiesoftechnologytoolstosupportteaching,learning,assess-ment,andfeedbackforlearning.
Astechnology-richenvironmentsforteaching,learning,assessment,andfeedbackarebeingintegratedintoeducationalprocesses,thereismuchtobelearnedabouthowtoleverageadvancesintechnology,learningsciences,andassessmenttodevelopreal-worldskillsforthe21stcentury.Researchfindingsonwhatworksbestarejustemerging,possiblyduetothestrongmulti-disciplinaryapproachesrequiredtoextractthegreatestvalue.ThisHandbookisintendedtoserveasafirstbodyofresearchintheexpand-ingareaoftechnologytoolsforteaching,learning,assessment,andfeedbackonreal-worldskillsthateducatorscanturntointhecomingyearsasareference.Ouraimistobringtogethertopresearcherstosummarizeconceptsandfindings.TheHandbookcontainscontributionsofleadingresearchersinlearningscience,educationalpsychology,psychometrics,andeducationaltechnology.Assumingthatmanyreaderswillhavelittlegroundinginthosetopics,eachchapteroutlinestheoryandbasicconceptsandconnectsthemtotechnologytoolsforreal-worldskilldevelopment.WeseethisasoneofthemostcrucialcontributionsoftheHandbook,seekingtoestablishstrongtheoreticalprinciplesthatcaninformeducationalresearchandpracticeandfutureresearchanddevelopment.TheHandbookalsoprovidesbriefoverviewsineachtopicsectionformoreknowledgeablereaders.TheHandbookisorganizedintofoursections.
SECTION 1: DEFINING REAL-WORLD SKILLS IN TECHNOLOGY-RICH ENVIRONMENTS
ThesevenchaptersinSection1exploreconceptualizationofreal-worldskillsandtheroleoftechnol-ogy.Thesectionincludeschaptersoncurriculaandframeworksforteachingreal-worldskills.Toaidreadersinselectingspecificchapterstostudy,welistthetechnologytoolsdescribedinthesechapters.
Chapter 1:Aprincipledapproachfordevelopingdigitalcompetency.Chapter 2:Amodelforteachingdigitalcompetency.Chapter 3:Amodel formeasuringproblemsolvingskills inscience, technology,engineering,and
mathematics(STEM).Chapter 4:AmodelforteachingInternetresearchskills.Chapter 5:AnothermodelforteachingInternetresearchskills.Chapter 6:AmatrixforevaluatingtechnologyintegrationinK-12instructionalsettings,andteacher-
relatedprofessionaldevelopment.Chapter 7:Anonlineteam-basedlearningmodelinnursingeducation.
xxxiv
Preface
SECTION 2: TECHNOLOGY TOOLS FOR LEARNING AND ASSESSING REAL-WORLD SKILLS
Chapters8through21dealwiththecoretopicoftechnologytoolsandawiderangeofapplicationsaimedatlearningandassessingofreal-worldskills.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.
Chapter 8:Technology-richsimulationsforlearningandassessingscienceskills.Chapter 9:TheCollegiateLearningAssessment,atest toevaluatethecriticalthinkingandwritten
communicationskillsofcollegestudents.Chapter 10:Guidance,basedonlessonslearnedfromdevelopingrich-mediasimulations,forassess-
mentfororganizationstaffpromotionanddevelopment.Chapter 11:Apersonalizedlearningplatformfordevelopingearlyreading.Chapter 12:Computeragenttechnologyforassessingcollaborativeproblemsolvingskills.Chapter 13:Amodelforassessingcognitiveandsocialskillsthroughonlinecollaboration.Chapter 14:Anapproachfortechnology-richlearningandformativeassessmentofcollaborativeprob-
lemsolvingskills.Chapter 15:Aframeworkforprincipledthinkingaboutaconstructmapassessmentofahigher-order
thinkingskills.Chapter 16:Computer-basedandcomputer-assistedapproachesforassessmentofknowledgeandskills.Chapter 17:Technology toolsfor learningforstudentswithmoderateandseveredevelopmentand
intellectualdisabilities.Chapter 18:Strategiesformitigatingbiasforacomputer-administeredperformance-basedassessment
ofhigher-orderskills.Chapter 19:Anevidence-centeredconceptmapforacriticalthinkingassessment.Chapter 20:Facebookasasocialnetworkforlearning.Chapter 21:Aframeworkforteachers’professionaldevelopmentinthedigitalage.
SECTION 3: AUTOMATED ITEM GENERATION AND AUTOMATED SCORING TECHNIQUES FOR ASSESSMENT AND FEEDBACK
ThefivechaptersinSection3addressarangeoftechnologiesforautomatedscoring,automateditemgeneration,andlearnerfeedback.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.
Chapter 22:Proceduresforautomatedgenerationofscienceitems.Chapter 23:Automatedscoringapproachesfordevelopmentofwritingproficiency.Chapter 24:Aprincipledframeworkfordesigningautomatedscoringofmulticomponentassessment
tasks.Chapter 25:Automatedscoringasthebasisforfeedbacktosupportimprovementofwritingskills.Chapter 26:Automatedfeedbacktoimprovewritingquality.
xxxv
Preface
SECTION 4: ANALYSES OF PROCESS DATA IN TECHNOLOGY-RICH PERFORMANCE TASKS
Chapters27through31dealwithanalysis,interpretation,anduseoflearningandassessmentdataintechnologyenvironments.Thetechnologytoolsdescribedinthesechaptersincludethefollowing.
Chapter 27:Analysisofsolutionpathsinatechnology-richproblemsolvingassessment.Chapter 28:Analysisofsolutionpathsintechnology-richcriticalthinkingassessment.Chapter 29:Useofachi-squarefeaturesselectionalgorithm(i.e.,sequentialpatternmining)andN-
gramsrepresentationmodeltoanalyzeprocessdataintechnology-richproblemsolvingtasks.Chapter 30:Analyticmethodstoinduceapersistencemeasurefromgameplayclickstreamdataanda
designpatterntoguidefuturedevelopmentofpersistencemeasuresindigitalenvironments.Chapter 31:AnItemEngagementIndex(IEI)andStudentEngagementIndex(SEI)forassessingen-
gagementduringtheonlineassessmentofreal-worldskills.
Ourgoalincollectingandorganizingtheseexcellentchaptersistobeginaprocessofcrystalizingwhatourfieldhasaccomplishedtodateandwhatitknows,collectively,abouttechnologytoolsandhowthosetoolscanbeusedtosupportandenhanceteachingandlearningofreal-worldskills.Knowingwhatweknowshouldhelpusidentifywhatweneedtoknow.Anditshouldguidefurtherdevelopmentofpracticalapplicationsandempiricalresearchontheefficacyofusingtechnologytoolsforteaching,learning,assessing,andprovidingfeedbackaslearnersworktodeveloptheskillstheyneedfortoday’shigh-tech,higher-orderknowledgeandskillsworld.WehopethisHandbookwillserveasatooltoen-couragecollaborationsamongresearchers,educators,policymakers,employers,andthegeneralpublictopromotelearning,assessment,andpersonalizedfeedbacktechnologies.BycompilingtherichresearchandknowledgeinthisHandbook,wehopetosparkinnovationineducation.
TheHandbookisarecommendedreadingsourcetothefollowingaudiences:Educators:Thisbookwillshareessentialinsightsforpolicymakers,principals,curriculumexperts,
andteacherswhoareinterestedinbetterunderstandingthepracticalchallengesandopportunitiesinintroducingnewtechnology-richprogramsaimedtopromote learning,assessment,andfeedbackonreal-worldskills.
Researchers:Thisbookwillprovideavaluablespringboardtoresearchersinpsychology,educa-tion,assessment,andcomputersciencetoengagewiththeconceptoftechnology-richassessmentandlearningofhigher-orderthinkingskillsandworkonnewresearchdirections.Thiswillbeaidedbytheemphasisofkeygapsinexistingresearchandprovidingdetailsonwhatareasneedmorecarefulresearchandempiricalvalidation.
General audienceswithinterestinupcomingtrendsinlearning,assessment,andfeedback:Thisbookwillcoverarangeoftopicsrelatedtoreal-worldskillsandvalueofreal-worldskillsinnext-generationeducation.
xxxvi
Preface
REFERENCES
CouncilofEurope.(2011).Common European framework of references for languages: Learning, teach-ing, assessment.Strasburg:Author.
Gallup.(2013).21st century skills and the workplace: A 2013 Microsoft-Pearson Foundation study on 21st century skills and the workplace.Washington,DC:Author.
Hattie,J.,&Timperley,H.(2007).Thepoweroffeedback.Review of Educational Research,77(1),81–112.doi:10.3102/003465430298487
NationalGovernorsAssociationCenterforBestPracticesandCouncilforChiefStateSchoolOfficers.(2010a).Common core state standards for mathematics.Washington,DC:Author.
NationalGovernorsAssociationCenterforBestPracticesandCouncilforChiefStateSchoolOfficers.(2010b).Common core state standards for English language arts and literacy in history/social, science, and technical subjects.Washington,DC:Author.
NationalResearchCouncil.(2012).Education for life and work: Developing transferable knowledge and skills in the 21st century.Washington,DC:TheNationalAcademiesPress.
NationalResearchCouncil.(2013).Next generation science standards: For states, by states.Washington,DC:TheNationalAcademiesPress.
OrganizationforEconomicDevelopment(OECD). (2012a).Better skills, better jobs, better lives: A strategic approach to skills policies.OECDPublishing.
OrganizationforEconomicDevelopment(OECD).(2012b).Education at a glance 2012: OECD indica-tors.OECDPublishing.
OrganizationforEconomicDevelopment(OECD).(2013).PISA 2015 collaborative problem solving framework.OECDPublishing.
Partnershipfor21stCenturySkills.(2009).P21 framework definitions.Washington,DC:Author.
Shute,V.J.(2008).Focusonformativefeedback.Review of Educational Research,78(1),153–189.doi:10.3102/0034654307313795
Stiggins,R.J.(2005).FromformativeassessmenttoassessmentFORlearning:Apathtosuccessinstandards-basedschools.Phi Delta Kappan,87(4),324–328.doi:10.1177/003172170508700414
Wiliam,D.(2011).Embedded formative assessment.Bloomington,IN:SolutionTreePress.
xxxvii
Acknowledgment
Inthewritingandeditingthisbookwehavetothanktheconceptualvisionarieswhopushedourthinkingandcontributedgreatlytotheformationoftheideaspresentedhere.AtPearson,Dr.KimberlyO’Malley,athoughtfulleaderwhoprovidedexecutivesupportduringallthestagesofourworkontheHandbook.Ourcolleagues,Dr.PeterFoltzandDr.KatieMcClartywhohavebeenkeyindevelopingtheseideasaspartof21stCenturySkillsproject.Prof.AndreasSchleicher,OECDandProf.ChrisDede,HarvardUniversitysharedtheirinvaluableinsightsanddirectionsforfurtherresearchinForewordsectionoftheHandbook.Andthenthereistheoutstandinggroupofauthorsfromawiderangeoforganizationsandgeographieswhocontributedtheirchapterstothisvolume.Alongtheway,theauthorsgraciouslyservedaseachother’sreviewersaswepasseddraftsaround,nurturingeachother’schaptersandaddingnewperspectives.WethankyouallformakingourownworkontheHandbookagreatpleasure.
xxxviii
Editorial Advisory BoardChrisDede,Harvard University, USAKimberlyO’Malley,Pearson, USAAndreasSchleicher,Organisation for Economic Co-Operation and Development (OECD), France
About the Contributors
Yigal RosenisaSeniorResearchScientistinTheViceProvostforAdvancesinLearning(VPAL)atHarvardUniversity.HeleadstheHarvardVPALResearchTeamthatenablesfacultytocreateandimplementresearch-enabledonlinelearningexperiencesforresidentialandonlineuse,suchasHarvardXMassivelyOpenOnlineCourses.PriortojoiningHarvardhewasaSeniorResearchScientistatPearson,leadingresearchanddevelopmentofinnovativelearningandassessmenttechnologiesforreal-lifeskilldevelopment.YigalwasaHeadofAssessment&EvaluationatTimeToKnow.HewasamemberofOECD’sPISA2015AssessmentFrameworkExpertGroupandconsultedtoMicrosoft,Intel,ETS,Min-istryofEducationinIsraelandtheNationalAuthorityforMeasurement&Evaluation.YigalpreviouslyheldacademicappointmentsattheUniversityofHaifa,theOpenUniversityofIsraelandBen-GurionUniversity.In2009hereceivedtheUniversityofHaifaFacultyofEducation’sOutstandingLecturerAward.HeobtainedhisPh.D.degreefromtheUniversityofHaifa.Yigalwasapost-doctoralfellowatHarvardUniversityGraduateSchoolofEducationandatTelAvivUniversitySchoolofEducation.Hebeganhiscareerasacomputerscience,mathandscienceteacheringrades5through12.Yigalco-editedthebookduringhisworkasaSeniorResearchScientistatPearson.
Steve FerraraisVicePresident,PerformanceAssessment,andleadstheCenterforNextGenerationLearningandAssessmentinPearson’sResearchandInnovationNetwork.Thecenterdesigns,develops,validates,andconductsresearchonperformanceassessmentforsummativeandformativeusestosup-portlearning.PriortojoiningPearson,StevewasaPrincipalResearchScientistatCTB/McGraw-Hill,ManagingResearchDirectorintheassessmentprogramatAmericanInstitutesforResearch,andStateAssessmentDirectorinMaryland,andaHeadStartandhighschoolspecialeducationteacher.SteveconductspsychometricresearchanddesignslargescaleandformativeassessmentsforK-12educationalachievement.Heservesontechnicaladvisorypanelsforassessmentprogramsandresearchprojects.HewasEditorofEducationalMeasurement:IssuesandPracticeandservesonjournaleditorialadvisoryboards.Stevewasco-recipient,AERACognitionandAssessmentSpecialInterestGroup2014awardforOutstandingContributiontoPracticeinCognitionandAssessment;andco-recipient,AERADivisionD2006awardforSignificantContributionstoEducationalMeasurementandResearchMethodology.
Maryam MosharrafhasbeenworkingatPearsonsince2009.Shehasmanagedanumberofna-tionalandglobalassessmentproductlines.InhermostrecentrolesheworksasaResearchtoPracticeManager.Inthisroleshebringsinnovation,researchandnewtechnologyintoeducationalsolutionsforteachersandstudents.Maryamhasconductedresearchondeveloping innovative21stcenturyskillsassessmenttaskswhichhavebeenadministeredinTurkey,SouthAfrica,Singapore,UK,USandothercountries.SomeofherpreviousexperienceincludesworkingatProcterandGamble.Priortomoving
cxxix
About the Contributors
totheStatestoattendbusinessschool,MosharraflivedinIranandworkedattheBritishEmbassyinTehranasaninterpreter.Sheholdsamaster’sdegreeinBusinessAdministrationfromtheUniversityofTexasinAustinandabachelor’sdegreeinTranslationfromAzadUniversity.
***
Lynne Anderson-InmanisaprofessorofeducationattheUniversityofOregonandpastDirectoroftheuniversity’sCenterforAdvancedTechnologyinEducation.Dr.Anderson-Inmanisanationallyandinternationallyrecognizedexpertontheuseoftechnologytoimprovereading,writing,andstudy-ing,withspecialemphasisonstrategiesforusingtechnologytoenhancetheacademicsuccessofstrug-glinglearners.Shepioneeredtheconceptof“computer-supportedstudying,”conductingresearchonstrategy-basedinstructiontopromoteeffectivereadingandlearningonline.Recentworkhasfocusedoninvestigating“supportedetext”asameanstoimprovecontent-areareadingandstudying.Dr.Anderson-Inmanisawidelypublishedauthoroneffectiveusesoftechnologyandafrequentspeakeratnationalandinternationalconferences.
Gilbert Andrada,PhD.,StateEducationConsultant,hasbeenwiththeConnecticutStateDepart-mentofEducationfor21years.InadditiontohavingbeentheprogrammanagerfortheConnecticutBenchmark Assessment System (CBAS), his duties involve psychometric and statistical analyses,appliedresearchprojects,programevaluations,andlarge-scalestudentassessment.HeholdsaCon-necticutschooladministrator’slicense(092).GilisanactiveresearcherwithcolleaguesfromtheNeagSchoolofEducationattheUniversityofConnecticut.Heisafrequentadjunctfacultymemberteachingcoursesinstatistics,measurementandresearchmethods.GilholdsaBAinpsychologyfromCaliforniaStateUniversity,Stanislaus;anMSinPsychologyfromtheSchoolofPsychologicalSciencesatPurdueUniversity;andaPhDinEducationalPsychologyfromtheMeasurement,EvaluationandAssessmentprogramattheUniversityofConnecticut,NeagSchoolofEducation.
Perla Arie,M.Ed.HonorsDegreeinEducationalTechnologyfromKibbutzimCollegeofEducation,TechnologyandtheArts.EFLandEnglish-HebrewTranslationteacheratKugelHighSchool(Holon,Israel).Mainfieldsofinterest:Languageteaching(English,FrenchandSpanish),Innovativeteachingmethods,ComputerAssistedLearning,Web2.0toolsandFacebookuseforeducationalpurposes.
Edith Avni,Ph.D.inEducation,HaifaUniversity,Israel.ChargeofICTinEducation,MinistryofEducation,HaifaDistrict,Israel.Lecturerinprofessionaldevelopmentcoursesforeducationalleadershipandteachers.EstablishingwithDrA.Rotem“TowardDigitalEthicsInitiative”,Israel,containsoriginalarticlesrelatedtotechnology,pedagogicalandethicalaspectsinlearningandteaching.
Susan Malone Back,PhD,MBA,isaResearchScientistintheInstituteforMeasurement,Meth-odology,AnalysisandPolicyoftheCollegeofEducationatTexasTechUniversityandDirectoroftheuniversitySmallBusiness InnovationResearch/SmallBusinessTechnologyTransfer (SBIR/STTR)ResourceCenter.Shehasextensiveexperiencemanaging federal andprivate foundationgrantsandworkingwithpublic/privatepartnerships.Herareasof researcharecommunityengagement,STEMeducation,andnewfrontiersinbusinesseducation.Dr.BackreceivedherM.Ed.andPhDinEducationalPsychologyfromTempleUniversityandherMBAfromtheUniversityofDenver.
cxxx
About the Contributors
Sara Bakken’seducationcareerbeganin2001asahighschoolscienceteacher.Afterleavingtheclassroom,sheworkedasaninstructionaldesignerandadvocateforonlinelearninginboththeK-12andhighereducationenvironments.ShehasworkedextensivelywiththeTexasVirtualSchoolNetwork(TxVSN),andservedascommitteeco-chairforthe2011publicationoftheInternationalAssociationforK-12OnlineLearning(iNACOL)NationalStandardsforQualityOnlineTeaching.Shehaspresentedontheeffectiveincorporationof21stcenturyskillsintothevirtualclassroomatseveralnationalconferencesincludingtheTexasDistanceLearningAssociation(TxDLA)andiNACOL’sVirtualSchoolSympo-sium.ShenowdevelopsnewandinnovativeonlineassessmentsatPearson,whereshewonthePearsonInnovativeAssessmentIdeacompetitionin2012.SheholdsaMasterofSciencedegreeininstructionaldesignfromtheUniversityofHouston-ClearLake.
Kirk A. BeckerisaseniorresearchscientistwithPearsonandconsultingeditorfortheJournalofComputerAdaptiveTesting.Hehasoverfifteenyears’experienceconductingresearchandperformingoperationalworkinmeasurementandpsychometrics.Dr.Becker’scurrentresearchfocusesonefficientmethodsforevaluatingandcodinglargeitempoolsusingnaturallanguageprocessing,especiallywithinthecontextofadaptivetesting,LOFT,andautomatedtestassembly.Thisresearchonnaturallanguageprocessingiscurrentlybeingextendedtoautomateditemgeneration,itemmodeling,andtheevaluationofitemquality.AdditionallyKirkhascontinuedtoresearchmethodsfordeveloping,evaluating,andscoringinnovativetestitemsandperformancetasks,withcurrentresearchlookingattheapplicationofpolytomousscoringmodelsoninformationfunctionsandadaptiveadministration.Dr.BeckerhaspresentedresearchonapplicationsofnaturallanguageprocessingattheannualconferencesoftheAmeri-canEducationalResearchAssociation,theAssociationofTestPublishers,andtheNationalCouncilonMeasurementinEducation.KirkreceivedhisPh.D.inEducationalPsychologyfromtheUniversityofIllinoisatChicago.
Valerie Beltran,Ed.D.,isanAssociateProfessorofEducationattheUniversityofLaVerne.IntheK-12field,shehasbeenateacher,aresourcespecialist,andaprincipal.Attheuniversitylevel,sheteachescoursesinboththeteachercredentialandMaster’sprograms.ShechairstheMaster’sofEduca-tionSpecialEmphasisprogram.Herresearchinterestsincludewritinginstruction,bilingualeducation,onlineteaching,andstudentengagement.
Roger BenjaminhasbeenPresidentofCAE(CouncilforAidtoEducation)since2005.HewasaresearchscientistatRANDfrom1990to2005(directorofRANDEducation,1994-1999).PrevioustohisappointmenttoRAND,hewasprofessorofpoliticalscienceattheUniversityofMinnesota,1966to1983andassociatedeanandexecutiveofficer,CollegeofLiberalArts,UniversityofMinnesota,1980to1983,vicechancellorforacademicaffairsandprovostattheUniversityofPittsburgh,1983to1986,andvicepresidentforacademicaffairsandprovost,UniversityofMinnesota,1986to1988,andprofessorofpoliticalscience,1988to1990.Heistheauthororco-authorofnumerousbooks,monographsandarticlesinpoliticaleconomyandpublicpolicy,includinggovernance,strategicplanning,andeduca-tionalassessment.HislatestbookisTheNewLimitsofEducationPolicy:AvoidingATragedyoftheCommons.London:EdwardElgar,2012.HedirectsCAE,aneducationalassessmenttestingorganiza-tion,implementingperformanceassessmentsandrelated21stcenturyassessmentsthroughouttheK-16educationsystemintheUnitedStatesandbeyond.
cxxxi
About the Contributors
John Bielinskihasworkedineducationalassessmentasadeveloperandresearchersince1995.HeearnedaPhDinquantitativemethodsthroughtheEducationalPsychologydepartmentattheUniversityofMinnesota.Hisresearchincludesgenderdifferencesinmathachievement,testaccommodationvali-dation,DIF,andgrowthmodeling.SincejoiningPearsonin2002,Johnhasdirectedthedevelopmentofseveralclinicalandeducationalassessment includingtheKeyMath-3DiagnosticAssessmentandtheBruininks-OseretskyTestofMotorProficiency,2ndEdition,andaimsweb.Heiscurrentlyleadingdevelopmentonnextgenerationcurriculum-basedmeasurementandformativeassessment.Inadditiontohisresearchanddevelopmentwork,Johnvolunteersasamiddleschoolmathcoach.
Keith A. BoughtonisaSeniorResearchScientistatCTB/McGraw-HillEducation.Hisareasofspe-cializationincludeautomaticitemgeneration,adaptivetesting,mixture/latentclassmodels,hierarchicalIRTmodels,andmultidimensionalIRTmodels.
Daniel G. Brenner, a Senior Research Associate at WestEd’s Science, Technology, Engineer-ing,andMathematics(STEM)program,isCo-PrincipalInvestigatoronmultipleprojectsdevelopingsimulation-basedassessmentsinvarioussciencecontentareas:theNSF-fundedHumanBodySystemand SimScientists Assessment System: Physical Science Links projects as well as the IES-FundedSimScientistsAssessmentSystem:LifeSciencesandSimScientistsModelProgressions(GeneticsandEvolution)providingscientificcontentexpertise,managingtheassessmentprogramming,andoverseeingtheSimScientistslearningmanagementsystemthatservesover10,000studentsannually.Dr.BrennerisalsoPrincipalInvestigatorontheInstituteofEducationSciences-fundedVoyagetoGalapagosprojecttodevelopadifferentiatedassistancemodelthatisbasedonknowledgeaboutwhatkindsofassistancearemosteffectiveforstudentsofdifferingabilitiesastheyworkinaninquiry-drivenlearningenviron-mentforevolutionsciencecontent.Brenner’sresearchinterestsfocusononlineeducationandassess-ment,andontheapplicationofnewtechnologiestotheseareas,andonapplicationoftheartstoSTEMresearch.Hehasdoneextensiveresearchandprojectmanagementinacademicandindustrialresearchintheareasofbioorganicchemistryandbiopharmaceuticals.PriortojoiningWestEd,Brennerworkedasaninstructionaldesignerheadingtheproductionofover500in-depthonlinemediarichcoursesfortheAcademyofArtUniversity.Dr.BrennerreceivedanABinChemistryandVisual&EnvironmentalStudiesfromHarvardCollege,andaPhDinChemistryfromHarvardUniversity.
Diane BrowderistheLakeandEdwardP.SnyderDistinguishedProfessorofSpecialEducationattheUniversityofNorthCarolinaatCharlotte.Dr.Browderhasovertwodecadesofresearchandwritingonassessmentandinstructionforstudentswithseveredevelopmentaldisabilities.Dr.Browder’sresearchhasbeenrecognizedthroughmultipleawards.Shereceivedthe2009DistinguishedResearcherAwardinSpecialEducationfromtheAmericanEducationResearchAssociationandwasthe2009FirstCitizensBankScholarattheUniversityofNorthCarolinaatCharlotte.In2011Dr.BrowderwasrecognizedbythestateofNorthCarolinawiththeO.MaxGardnerAwardforresearchthathasmadeacontributiontohumanity.
Barbara Buckley, a Senior Research Associate at WestEd’s Science, Technology, Engineering,&Mathematics(STEM)programisPrincipleInvestigatoroftheSimScientistsHumanBodySystems(HBS)projectfundedbyNSF(DRK12),andco-PIoftwocurrentSimScientistsprojectsinthelifesci-encesfundedbyIES.Asaproponentofmodel-basedlearning,Dr.Buckleyguidestheevidence-centered
cxxxii
About the Contributors
designprocessusedbytheSimScientiststeamtocreatesimulation-basedinstructionalandassessmentsuitesthatincorporatemodel-basedlearning,complexsystemsthinking,andNextGenerationScienceStandards.PriortojoiningWestEd,Dr.BuckleywasaresearcherattheConcordConsortium,playingacentralroleintheModelingAcrosstheCurriculumProject(IERI#0115699).AtTERC,shecontributedtotheGLOBEandGlobalLabprojects.
Michael B. BunchisSeniorVice-PresidentofMeasurementIncorporated,atestdevelopmentandscoringcompanyservingthelarge-scaleassessmentandprofessionallicensure/certificationfields.Heoverseesaresearchanddevelopmentunitofover80assessmentprofessionalsandservesasacompanydirector.Dr.Bunchearnedadoctorateinpsychologywithaspecialtyinmeasurementandhumandif-ferencesfromtheUniversityofGeorgiain1976.PriortojoiningMIin1982,heservedasresearchpsychologistattheAmericanCollegeTestingProgram(ACT)andseniorprofessionalatNTSResearchCorporation.SincejoiningMI,hehasbeenactivenotonlyintheprofessionbutintheDurhamcom-munityaswell,servingonandchairingboardsoflocaleducationandhumanserviceorganizations.HeispastchairmanoftheGraduateEducationAdvancementBoardoftheUniversityofGeorgia.
Dolores CannellaearnedherPhDinSocialandHealthPsychology.SheistheDirectorofBehavioralSciencesandtheActingChairintheDepartmentofGeneralDentistryatStonyBrookUniversity,SchoolofDentalMedicine.Herteachingandresearchinterestsfocusonthedevelopmentofpatient-centeredinterprofessionalhealthcareteamstoimprovehealthoutcomes.
E. Lee ChildresshasbeensuperintendentoftheCorinthSchoolDistrictfor13years.AnativeofClarksdale,Mississippi,ChildressisagraduateofDeltaStateUniversity.HeservedasasocialstudiesteacherintheClarksdalePublicSchoolsbeforebecomingProgramManagerfortheSchoolExecutiveManagementInstitutefortheMississippiDepartmentofEducation,whichprovidedtrainingforover2,000Mississippischooladministrators.ChildresswasalsoDirectorofSchoolImprovementfortheDeSotoCountySchoolDistrictbeforeservingfornineyearsasAssistantSuperintendentfortheCorinthSchoolDistrict.Dr.ChildresshasbeenactiveinseveraleducationalorganizationsandiscurrentlyservingasaTrusteeforMississippiPublicEmployeesRetirementSystemrepresentingpublicschoolandcom-munitycollegeeducators;servesasChairmanoftheMississippiCommissiononSchoolAccreditation,President,oftheProgramforResearchandEvaluationinPublicSchools;andservesontheAdvisoryBoardfortheMississippiSchoolforMathandScience.
Rafi DavidsonreceivedhisPH.DinBiochemistryfromtheWeizmannInstituteinRechovot,Israel.Heisaseniorlecturerinthedepartmentofscience&technologyinKayeCollegeofeducation,where,for20years,heteachescoursesforB.Ed.andM.ED.inthesubjectsof“ICTineducation”,“informationliteracyandcriticalthinking”and“teachingandlearninginaninformationage”.In1999hefoundedtheCenterforEducationalTechnologyInitiativeswhichsupports,advisesandprovidespedagogical-technologicalassistancetolecturersandstudentatKayeCollege.Forthelast4yearsheisresponsiblefor thedevelopmentand implementationofprogramsforprofessionaldevelopmentof lecturersandpedagogicalsupervisorsinthecollegesforeducationinIsrael.TheseprogramsarepartofthenationalprogramfortheadaptationoftheIsraelieducationsystemtothe21stcentury.HisareasofresearchanddevelopmentincludeintegrationofICTinteachingandlearning,socialandemotionalaspectsofonlinelearningandthefactorsthatinfluencethedigitaldivide.
cxxxiii
About the Contributors
Jessica Decker isanAssistantProfessorofEducationattheUniversityofLaVerne.Previously,shehasworkedasanelementaryschoolteacher.Herareasofresearchincludeimprovingwritingskills,technology,andmentoring.
Kristen DiCerbo isaPrincipalResearchScientistintheCenterforDigitalData,Analytics,andAdaptiveLearningatPearson.Herresearchprogramcentersonhowtouseevidencefromlearneractivityingamesandsimulationstounderstandwhatlearnersknowandcando.Dr.DiCerbohasalsoengagedwithteacherstounderstandhowtobestcommunicateinformationaboutstudentperformancetoinforminstructionaldecisions.PriortojoiningPearson,KristenprovidedresearchsupporttotheNetworkingAcademiesatCiscoandworkedasacertifiedschoolpsychologist.KristenreceivedherMaster’sDegreeandPh.D.inEducationalPsychologyatArizonaStateUniversity.ShereceivedaB.A.inPsychologyandSociologyatHamiltonCollege.
Patricia EckardtisaneducatorwithaPhDinQuantitativeMethodsforResearch,EducationalPsy-chology.Shehastaughtresearchandstatisticstonursesandhealthprofessionalsacrossundergraduateandgraduatestudies.Shehasbeenaregisteredprofessionalnurseforoverthirtyyears.
Sammy ElzarkaistheDirectoroftheCenterfortheAdvancementofFacultyExcellenceattheUni-versityofLaVerneinLaVerne,California.Hehasworkedineducationfornearly20yearsincludingrolesasteacher,curriculumdeveloper,assessmentanalyst,andaccreditationliaison.Hiscurrentfocusisontheeffectiveuseofnewtechnologiesforteachingandlearning.
David P. Erlangerisaneducatorandresearchdirector.Hisinterestsincludehowtoapplyeducationaltheorytoensurestudentslearnbestinbothtraditionalandnon-traditionalsettings.Hehasafocusonidentifyingmethodsneededtomakedistancelearningeffective,especiallyforresearchstudents.
Peter FoltzisaVicePresidentforresearchanddevelopmentandworkstobringinnovativetechnolo-giestolearningandassessment.Heisoneoftheoriginaldevelopersofautomatedscoringtechnologiesandholdsapatentonmethods forscoringofwriting.Dr.Foltz’s researchhas focusedon languagecomprehension,21stCenturyskillslearningandassessment,andusesofmachinelearningandnaturallanguageprocessingineducationaltechnology.Themethodshehaspioneeredimprovestudentachieve-ment,expandstudentaccess,andmakelearningmaterialsmoreaffordable.HehasledtheframeworkdevelopmentforanewassessmentofcollaborativeproblemsolvingfortheOrganisationofEconomicCooperationandDevelopment’sProgrammeforInternationalStudentAssessment(PISA)test.AformerprofessorofpsychologyatNewMexicoStateUniversity,hehasauthoredmorethan100journalarticles,bookchapters,conferencepapers,andotherpublications.HepreviouslyworkedatBellCommunica-tionsResearchandtheLearningResearchandDevelopmentCenterattheUniversityofPittsburgh.Dr.Foltzholdsdoctorateandmaster’sdegreesinCognitivePsychologyfromtheUniversityofColorado,Boulder,andabachelor’sdegreefromLehighUniversity.
Kellilynn M. FriasisanAssistantProfessorintheMarketingDepartmentatTexasTechUniversity.Her researchagenda focusesonunderstanding the technology, firm, andenvironmental factors thatimpactafirm’sdecisionof“wheretoparticipateinthevaluechain”or“whattosell”.Thisresearchhasimplicationsforproductdevelopmentandinnovation,designofinter-firmties,andintellectualpropertyenforcement.cxxxiv
About the Contributors
Mark Gierl isProfessorofEducationalPsychologyandtheDirectorof theCentreforResearchinAppliedMeasurementandEvaluation(CRAME)attheUniversityofAlberta.Hisspecializationiseducationalandpsychologicaltesting,withanemphasisontheapplicationofcognitiveprinciplestoassessmentpractices.ProfessorGierl’scurrentresearchisfocusedonautomaticitemgenerationandautomatedessayscoring.HisresearchisfundedbytheMedicalCouncilofCanada,Elsevier,ACTInc.,andtheSocialSciencesandHumanitiesResearchCouncilofCanada.Dr.GierlholdstheTierICanadaResearchChairinEducationalMeasurement.
Amnon GlassnerwastrainedasamathteacherandthencompletedhisM.A.andPhDinthecenterofcognition,instructionandcomputersattheHebrewUniversityinJerusalem.HeparticipatedinsomeresearchesaboutlearningandargumentationintheframeofKishurimGroupintheleadershipofProf.BaruchSchwarz.Hiscurrenteducationalinterestistoleadsomenewprogressiveprogramsofteachertraining,suchasthosewhousePBL(projectorproductbasedlearning)asmaindirectionforlearningandinstruction.HeservesastheheadofEducationDepartment,theheadofInformalStudiesandapedagogicalguideinKayeAcademicCollegeofEducationinBeer-Sheva.Hiscurrentresearchesdi-rectionsincludeinfusionofcreativeandcriticalthinkingduringlearningofanycontent,learningwithPBL,learningbysuccesses,andmoderationofdialogicallearningdiscussion.
Heather Greenhalgh-Spencer isanAssistantProfessorofCurriculumand InstructionatTexasTechUniversity.Herresearchfocusesondiversityandequityissuesinonlineeducation.Thisresearchhasimplicationsforthewaysweteachinonlinespaces,andimplicationsforcreatinggreateraccessandengagementineducationforunderrepresentedstudents.
Sean T. Hanloncreatesstudent-andteacher-centerededucationaltechnologyapplicationsthatblendassessmentandinstruction.HereceivedhisundergraduatedegreeincomputersciencefromTheGeorgeWashingtonUniversity(GW),hismaster’sdegree incomputerscienceandhisPh.D. ineducationalpsychologyfromtheUniversityofNorthCarolinaatChapelHill.DuringhissenioryearatGW,SeanreceivedtheGaryBardEntrepreneurawardforBlaise,anonlineessay-writingsystemwithgesture-basedscoring.ThisapplicationwasdeployedintheGWWritingCenterandusedinthecompulsoryfreshmencompositioncourse.SincejoiningMetaMetrics,Seanhas ledthedesign,development,deployment,and enhancement of multiple educational technologies including MyWritingWeb, MyReadingWeb,MyMathWeb,LearningOasis,EdSphere,ModuleCreator,andGuidedLiteracy.Seansupplementshistechnologicalexpertisewiththeinstructionalandassessmentprinciplesnecessarytocreateresearch-basedtechnologiesthatimpacttheeverydaylivesofstudentsandteachers.
Jiangang HaoisaResearchScientistintheCenterforAdvancedPsychometricsatETS.PriortojoininginETS,hehasbeenworkingonmodelingandminingTerabyte-scaledatainphysicsatFermiNationalAcceleratorLaboratoryaftergettinghisPhDinphysicsandMAinStatisticsbothfromtheUniversityofMichigan.Hehasbeenworkingextensivelyondatamining&machinelearning,statisticalmodeling&inference,datastandardizationanddatamodelschemadesignforbigdata.Hepublishedover40papersinpeer-reviewedjournalswithover2400totalcitationsandhisworkhasbeenreported
cxxxv
About the Contributors
byWired,MIT Technology Reviewandetc.Jiangang’scurrentresearchcentersoncollaborativeproblemsolving,gameandsimulationbasedassessment,educationaldatamining,logfilestructuringandanalysis.Heisalsoapplyingnaturallanguageprocessingtechniquestoanalyzetheconversationsincollaborationtodevelopintelligent(adaptive)facilitationmechanism.
J. Christine Harmesisaconsultantintheareasofassessment,measurement,andtechnologyin-tegration.SheholdsaPh.D.withdualconcentrationsinEducationalMeasurementandResearchandInstructionalTechnology.Dr.Harmes’researchincomputer-basedtestinghasmostrecentlyfocusedonthedesign,cognitiveprocessing,memorability,andusabilityofinnovativeitems.HerrecentprojectsregardingtechnologyintegrationinK-12settingshaveincludeddevelopingtoolsforteacherprofessionaldevelopment with and use of technology, and developing technology-related assessments, observa-tiontools,andsurveys.Shehastaughtcoursesinmixedmethodsresearch,scholarlycommunication,performanceassessment,computer-basedtesting,measurementforteachers,andinstructionaldesign.
Qiwei HeisanAssociateResearchScientistinETS’sResearch&DevelopmentDivision,GlobalAssessmentCenter.Herbackgroundandexpertiseisinpsychometrics,datamining,textminingandnaturallanguageprocessingwithafocusonbuildingapplicationsforeducationalandpsychologicaltechnology.Shecurrentlyfocusesonexplorationresearchofprocessdatainlargescaleassessments.Previously,Dr.Hehasconductedresearchintheareasoftext-basedonlineassessmentforpsychiatricandpsychologicalassessmentintheUniversityofTwente,theNetherlands.ShealsowontheBestDis-sertationPrizeofAbbasFundinthefieldofpsychologicalassessmentsintheNetherlands.
Bridget Hildreth,M.F.A.,isaResearchAssistantattheCenterforAdvancedTechnologyinEduca-tionattheUniversityofOregon.ShewasasecondaryteacherandELAInterventionSpecialistforovertenyears,andshetaughtwriting,remedialELA,andeLiteracyatthecommunitycollegelevel.Herre-searchwithfederallyfundedprojects(SSOAR,SAIL,SOAR,andMeTRC)primarilyinvolvesimprovingeducationoutcomesformiddleschoolstudentswithlearningdisabilitiesthroughtheuseoftechnology.
Cheryl Johnsonhasworkedinprofessional,clinical,andeducationassessmentdevelopmentsince1998.Shehaswrittencontentandmanagedteamstosuccessfullyexecutemulti-yearassessmentresearchprojectsthatincludetheKaufmanAssessmentBatteryforChildren,2ndEdition,theKaufmanTestofEducationalAchievement,2ndand3rdEditions,SensoryProfile,2ndEdition,andaimsweb.Shecur-rentlyleadsresearchonearlyliteracy,andeffectiveprogressmonitoringtowardachievementgoalsforschool-agedchildren,asResearchDirectoratPearson.Inadditiontoherresearchwork,sheisanadvocateforMontessorieducation,approachestohelpmaximizecognitiveandphysicalperformanceinadults,andnewtechnologyandexperiencesthatfosteraperson’snaturalcuriosity.
Carolyn KnoxhasaPh.D.inDigitalLearningwithanemphasisonstudentswithspecialneedsandEnglishLearners.SheisAssistantDirectorofCATEattheUniversityofOregon.Dr.KnoxisPIoftheSteppinguptoSOAR:StrategiesforOnlineAcademicReadingproject,developingboththeproject’sstrategiesanditsvideo-basedinstructionalwebsite.ShewasaseminalpartofthedevelopmentofCATE’soriginal“computer-basedstudystrategies”forusingtechnologytoenhancetheacademicsuccessofstrugglinglearners.Sheisanationallyrecognizedexpertontheuseofdigitalnote-takingsystemsinthe
cxxxvi
About the Contributors
classroomforstudentswithdisabilities.Forthelast25years,Dr.Knoxhasbeenconductingresearchanddevelopmentprojectsthatcreateandpromoteinnovativeusesoftechnologyforlearning,innovativeinstructionalmaterials,andevidence-basedstrategiesdesignedtohelpstrugglinglearnersusetechnol-ogyforeffectivelearningandacademicsuccess.
Zach KornhauserisanAssociateMeasurementScientistattheCouncilforAidtoEducation.PriortojoiningCAEZachworkedasaSeniorResearchAssistantfortheCommunityCollegeResearchCenter.Hisresearchinterestsincludeexaminingqualityofeducationalprogrammingincollegesanduniversi-ties,understandingfactorsthataffectstudentmotivationonlow-stakesassessments,studyingstudents’useoftechnologyincollegeclassrooms,andinvestigatingfactorsthatexplainstudentperformanceincollegeoverandabovestudentability.ZachreceivedaMaster’sinAppliedDevelopmentalPsychologyfromFordhamUniversityin2011,andiscurrentlyfinishinghisdoctorateatFordham.
Patrick KyllonenisSeniorResearchDirectoroftheCenterforAcademicandWorkforceReadinessandSuccessatEducationalTestingService(ETS)inPrinceton,N.J.Centerscientistsconductinnovativeresearchon(a)highereducationassessment,(b)workforcereadiness;(c)internationallargescaleassess-ment(e.g.,ProgramforInternationalStudentAssessment;PISA);and(d)21stcenturyskillsassessment,suchascreativity,collaborativeproblemsolving,andsituationalinterviews.Dr.KyllonenreceivedhisB.A.fromSt.John’sUniversityandPh.D.fromStanfordUniversityandisauthorofGeneratingItemsforCognitiveTests(withS.Irvine,2001);LearningandIndividualDifferences(withP.L.Ackerman&R.D.Roberts,1999);ExtendingIntelligence:EnhancementandNewConstructs(withR.RobertsandL.Stankov,2008),andInnovativeAssessmentofCollaboration(withA.vonDavierandM.Zhu,forthcoming).HeisafellowoftheAmericanPsychologicalAssociationandtheAmericanEducationalResearchAssociation,recipientofTheTechnicalCooperationProgramAchievementAwardforthe“de-sign,development,andevaluationoftheTrait-SelfDescription(TSD)PersonalityInventory,”andwasacoauthoroftheNationalAcademyofSciences2012report,EducationforLifeandWork:DevelopingTransferableKnowledgeandSkillsinthe21stCentury.
Hollis LaiisAssistantProfessorofDentistryandtheDirectorofAssessmentforUndergraduateMedi-calEducationprogramattheUniversityofAlberta.Hisspecializationiseducationalandpsychologicaltesting,withanemphasisonassessmentdesignsinmedicaleducation,curriculummapping,educationaldatamining,anditemgeneration.
Syed F. LatifiisadoctoralcandidateinEducationalPsychologyattheUniversityofAlbertaandagraduateresearcherintheCenterforResearchinAppliedMeasurementandEvaluation(CRAME)atthesameuniversity.HeholdsaM.S.inComputerSciencefromLUMS,andanM.Ed.inMeasurement,EvaluationandCognitionfromUniversityofAlberta.Hehaspublishednumerouspapersandarticlesinreputedpeer-reviewedconferencesandjournals.Hiscurrentresearchinterestsincludeautomaticitemgeneration,automatedessayscoring,evolutionaryprogrammingandapplicationofswarmintelligenceinassessmentsystems.
Lei LiuisaresearchscientistfromtheCognitiveandLearningSciencesCenteratEducationalTest-ingService.Sheisleadingmultipleprojectsfocusingonthedesignofinnovativeandtechnology-richscienceassessmentsthatarecompetency-basedandNGSSaligned.Herresearchhasdrawnheavilyon
cxxxvii
About the Contributors
cognitiveandsocioconstructivistlearningtheories.Herresearchinterestisontheroleoftechnologyinlearningandassessing.Shehasdevelopedsimulation-basedlearningenvironmentandassessments,learningprogressionbasedassessments,conversation-basedassessments,andcollaborativeproblemsolvingassessments.Shehaspublishednumerous journalarticlesandbookchapters in the fieldoflearningaboutcomplexsystems,computer-supportedcollaborativelearning,andlearningsciences.SheearnedherPhDfromRutgersUniversityandherdissertationwonthebestDoctoralResearchAwardbytheNationalAssociationofResearchinScienceTeaching.
William Lorié,Ph.D.,isDirectorofEducationResearchatQuestarAssessment,Inc.AgraduateofNotreDame(B.A,Philosophy,B.S.Physics)andStanford(M.S.Statistics,Ph.D.Education),Dr.LoriéhasworkedatCTB/McGraw-Hill,ETS,theWorldBank,andMetrica,anassessmentcompanyheco-founded.Dr.LoriéhasleadR&Dforlarge-scaletestingprogramsandhasadvisedonassessment-relatedcapacitybuildingforstatesandministriesofeducation.HisfocusatQuestarisonR&Dinnovation.
Mark Loveland,aSeniorResearchAssociateatWestEd’sScience,Technology,Engineering,&Mathematics(STEM)programisCo-PrincipalInvestigatoroftheIESfundedSimScientistsAssessmentSystemandSimScientistsModelProgressionsprojects,andtheNSFfundedSimScientistsAssessments:PhysicalScienceLinksproject.Dr.Lovelandhasover20yearsofexperienceworking in scientificresearchandSTEMeducation.PriortojoiningWestEd,Dr.LovelandcoordinatedresearcheffortstotransformmiddleschoolmathematicsandscienceteachingandlearninginSanFranciscopublicschools.Hehasconductedcancerresearchandtaughtbiology,chemistry,andenvironmentalscienceinformalandinformalenvironments.
Marie Ann MarinoistheAssociateDeanforAcademicAffairsandStrategicPartnershipsattheStonyBrookUniversitySchoolofNursing.Dr.MarinoreceivedaBachelorofSciencefromMolloyCollege,aMasterofScienceinNursingfromStonyBrookUniversityandaDoctorofEducationatTeachersCollege,ColumbiaUniversity.ShecompletedaPost-DoctoralFellowshipinFamilyViolenceResearchatTheChildren’sHospital/HarvardUniversitySchoolofMedicine.Dr.Marinohasmaintainedaclinicalpracticeasapediatricnursepractitioner/forensicmedicalexaminerconductingmedicalevalu-ationsofchildvictimsofabuseandviolence.Dr.Marinoisanactivememberofseveralprofessionalorganizations,includingtheInternationalAssociationofForensicNurses.Sheisakeydriveronseveralacademic-practicepartnershipsaimedatinnovatingcaremanagementandimprovingtransitionalcare,aswellasdevelopmentofcompetenceininterprofessionalcollaborativepractice.
Donna MatovinovicisVicePresidentofTestDevelopmentatACT.Donnahasover25yearsofexperienceprovidingleadershipandstrategicdirectionforthedesignanddevelopmentofassessmentproductsandservicesacrosstheKthroughcareercontinuum.Herfocusittoensuretightalignmentbetweencontentdevelopmentandpsychometricprinciplesaswellaswithproduct,program,andcus-tomerrequirements.
Mark MatzaganianisAssociateProfessorofEducationattheUniversityofLaVerne.HeteachesintheareasofAssessmentandResearch.Hisscholarlyinterestsincludetheroleoftechnologyineduca-tion,onlinelearning,andstudentmotivation.
cxxxviii
About the Contributors
Shayne MieliscurrentlytheVicePresidentofSoftwareDevelopmentatLightSideLabs,aned-techstartupfocusedonprovidingautomatedwritingsupporttoK-12studentsaroundtheworldthroughtheuseoftheirautomatedscoringandfeedbacktechnology.HisroleatLightSideisprimarilyresearchanddevelopmentoftheautomatedassessmentandfeedbacksoftware.PriortohisexperienceatLightSideLabs,ShaynewastheSr.DirectorofAITechnologyatMeasurementIncorporated,whereheledagroupofresearchersanddevelopersinthecreationanddevelopmentofmoderndayPEG.
Edys QuellmalzisDirectorofTechnology-EnhancedAssessmentandLearningSystemsinWestEd’sMath,ScienceandTechnologyprogram.SheleadsSimScientistsprojectsfundedbyNSFandtheU.S.DepartmentofEducationrelatedtosimulation-basedsciencecurriculaandassessmentsforformativeandsummativeusesthatcanserveascomponentsofbalancedstatescienceassessmentsystems.Proj-ectsincludeCalipersII:UsingSimulationstoAssessComplexScienceLearning(NSF);Foundationsof21stCenturyScienceAssessment (NSF);ModelProgressions (IES);SimScientistsAssessments:PhysicalScienceLinks(NSF);SimScientistsCrosscuttingConcepts:Progressions inEarthSystems(NSF);SimScientistsAssessmentSystem(IES);SimScientists:InteractiveSimulation-basedLearningEnvironments(IES);MutilevelAssessmentsofScienceStandards(IES);TransformativeAssessmentsforScience(CarnegieCorporation);andIntegratingScienceSimulationsintoBalancedStateScienceAssessmentSystem(OESE).Sheco-directedthedevelopmentoftheframeworkandspecificationsforthe2014TechnologicalLiteracy(TEL)NationalAssessmentofEducationalProgressandservedontheSteeringCommitteeforthe2011NAEPWritingFramework.Shehasconsultedfornumerousstate,national,internationalassessmentprograms.ShewasAssociateDirectoroftheCenterforTechnologyandLearningatSRIInternationalandtheDirectorofAssessmentResearchandDesign.SheservedonthefacultyattheStanfordSchoolofEducationandasresearchfacultyintheUCLAGraduateSchoolofEducation.
Rikki Rimor,Ph.D.,SeniorLecturer,TechnologyinEducationGraduateProgram,DepartmentofEducationatKibbutzimCollegeofEducation,TechnologyandtheArts,Israel.Pastexperience:developerandinstructorofonlinegraduatecourse“FromSearchforInformationtoConstructionofKnowledgeinTechnology-BasedEnvironments”attheOpenUniversityofIsrael.ThiscoursewonExcellencePrizeintheOpenUniversityOnlineCoursesContest.ShewastheHeadoftheCenterforPedagogy&InnovativeTechnologies(PITEC)attheTeachersCollegeofTechnology,Tel-Aviv.ShewasalsoHeadoftheUnitforResearchandEvaluationattheDepartmentofEducationintheMunicipalityofJerusalem.Mostofherpublicationsdealwithwhyandhowtopromotecollaborativeknowledgeconstructioninonlinelearningenvironments,teachingandassessingproblemsolvinginanonlinecollaborativeenvironment,complexityofsocial interactionsincollaborativeonlinelearningenvironmentaswellasanalysisofcognitiveandmetacognitiveaspectsoflearningonFacebook.
Jenny Root,M.Ed.istheSnyderFellowatUNCCharlottewheresheisadoctoralstudentinSpe-cialEducation.SheisagraduateresearchassistantforTheSolutionsProject,anIESgrant,developingamathematicsproblem-solvingcurriculumforstudentswithseveredisabilities.Jennyhasoversevenyearsofexperienceworkingwithchildrenandadolescentswithdisabilitiesinschoolsettings.Jenny’sresearchfocusesongivingstudentswithautismandintellectualdisabilityaccesstothegeneralcur-
cxxxix
About the Contributors
riculumthroughtechnology,systematicinstruction,andpositivebehaviorsupports.Futuredirectionsfor her research include systematic instruction to increase independence within academic tasks forstudentswithdevelopmentaldisabilitiesandusingtechnologytoovercomebarrierstocriticalthinkingandproblemsolvingskills.
Abraham Rotem,Ph.DinPhysicsandElectricalEngineering,KSUUSAandBen-GurionUni-versity,Israel(1983);FounderSENTOPLtd(1990);EstablishingTeachersDevelopmentcenterwithBrancoWeissInstitute,Israel(1994).EstablishmentofthefirstsocialeducationalnetworkinIsrael,toschoolsandstudent(1998).WrotetheBook“Schoolturnsonline”withDrY.Peled(2008).Lecturerinprofessionaldevelopmentcoursesforschooladministratorsandteachers.EstablishingwithDrE.Avni“TowardDigitalEthicsInitiative”,Israel,containsoriginalarticlesrelatedtotechnology,pedagogicalandethical aspects in learningand teaching;Academicadvisor ICTprogram in Israel (MinistryofEducation)forthelastdecade.
Jean-François RouetisaSeniorresearchscientistwiththeFrenchCentreNationaldelaRechercheScientifique(NationalCenterforScientificResearch).Hewastrainedasadevelopmentalandlanguagepsychologistandhehaspublishedextensivelyonthecognitiveunderpinningsofreadingliteracyandskilledusesofinformationtechnology(e.g.,“Reading:Fromwordstomultipledocuments”,coeditedwithBrittandGoldman,Routledge,2013).Jean-FrançoisRouethasbeeninvolvedsince2006asanexpertintheOECD’sPISAandPIAACsurveysofteenageandadultliteracy,respectively.HehasservedasthedirectoroftheCenterforResearchonCognitionandLearningattheUniversityofPoitiers,France,from2004to2011.AfterhisPhD(1991),Jean-FrançoisRouetspentseveralyearsasapost-doctoralfellowattheLearningResearchandDevelopmentCenter(UniversityofPittsburgh,USA).Hehasde-velopednumerouscollaborationswithscholarsfromvariouscountriesincludingArgentina,Germany,Japan,Norway,SpainandtheUSA.HeisontheboardofseveralmajoracademicjournalsandaformerassociateeditorofLearningandInstruction.
Gavriel Salomon(PhDatStanford,1968)isprofessorofeducationalpsychologyattheHaifaUniver-sityinIsraelwhereheservedasdeanoftheFacultyofEducation.AseditorofEducationalPsychologist,hehaspublishedfourbooksandabouta150research,theory,andmethodologyjournalarticles.HeistherecipientoftheIsraelAwardforscientificachievementsandholdsanhonorarydoctoratefromtheLeuvenUniversityinBelgium.Hisfieldsaretheuseandabuseofcomputersineducationandresearchonpeaceeducationinregionsofconflict.
Alicia Saunders,Ph.D.,istheProjectCoordinatorandResearchAssociatefortheSolutionsProject,anIESgrant,developingamathematicalwordproblemsolvingcurriculumforstudentswithseveredisabilitiesattheUniversityofNorthCarolinaatCharlotte.Shehasconductedresearchintheareasofgeneralcurriculumaccess,specificallyinscience,mathematics,andEnglishlanguagearts,aswellasintechnology,includingvideomodelingandcomputer-assistedinstruction.Additionally,shehaspub-lishedmultiplepeerreviewedjournalarticlesandbookchapters,developedandconductedprofessionaldevelopment webinars on aligning instruction to alternate achievement standards and the CommonCoreStateStandards,andisaco-authorofEarlyNumeracy,amathematicscurriculumforstudentswithseveredisabilities.Dr.Saundersreceivedherbachelor’sdegreeinpsychologyfromtheUniversity
cxl
About the Contributors
ofNorthCarolinaatChapelHill,andbothhermaster’sanddoctoraldegreesinspecialeducationfromtheUniversityofNorthCarolinaatCharlotte.ShetaughtstudentswithseveredisabilitiesandautisminNorthCarolinapriortoobtainingherdoctorate.ShealsoworkedasaresearchassociateonanIESfundedgrantinvestigatingmathematicsandscienceinstructionforstudentswithseveredisabilities,andasaresearchassociateforoneofthetwonationalconsortiacalledtodevelopanalternateassessmentandcurricularmaterialsbasedonalternateachievementstandardsalignedtotheCommonCoreStateStandardsforstudentswithsignificantcognitivedisabilities.
Matt Silberglitt,aSeniorResearchAssociateatWestEd’sScience,Technology,Engineering,&Mathematics(STEM)programisCo-PrincipalInvestigatoroftheSimScientistsAssessments:PhysicalScienceLinksandSimScientistsCrosscuttingConcepts:ProgressionsinEarthScienceprojectsfundedbyNSFandtheSimScientistsAssessmentSystemsprojectfundedbyIES.Asaformerscienceteacherandexperiencedassessmentdeveloper,Mr.Silberglittprovidescontentexpertiseinphysicalscienceandexpertise in standards for educational assessments.Prior tohis currentposition,Mr.SilberglittmanageddevelopmentofscienceassessmentsforWestEdinseveralstates.PriortojoiningWestEd,Mr.SilberglittoversawdevelopmentofscienceassessmentsattheMinnesotaDepartmentofEducationanddevelopedassessmentsatDataRecognitionCorporation.Mr.Silberglittstartedhiscareerasahighschoolscienceteacher.
Jeffrey T. SteedleisaResearchScientistatPearson,whereheprovidespsychometricsupportforstatetestingprogramsandconductsresearchwiththeCenterforNextGenerationLearningandAssess-ment.Hisresearchinterestsincludestudentmotivation,comparativejudgment,itemdifficultymodeling,andscienceassessment.HeearnedaPh.D.inEducationalPsychologyandaM.S.inStatisticsfromStanfordUniversityin2008.
A. Jackson Stennerisaco-founderofMetaMetricsandisitsChiefExecutiveOfficerandChairmanoftheBoard.Withcolleagues,hedevelopedTheLexileFrameworkforReading,TheQuantileFrame-workforMathematics,andtheLexileFrameworkforWriting.Dr.Stennerhaspublishedmorethan60papers,monographsandbooks,primarilyonstatisticalandmeasurementmethodology.Currently,heisaResearchProfessorintheDepartmentofAppliedDevelopmentalandSpecialEducationProgramintheSchoolofEducationattheUniversityofNorthCarolina-ChapelHill.Dr.StennerreceivedhisPh.D.ineducationalpsychology,withanemphasisinmeasurement,researchdesignandevaluationmethodology,fromDukeUniversity,anddualundergraduatedegreesinpsychologyandeducationfromtheUniversityofMissouri–SaintLouis.HehastaughtgraduateseminarsatDukeUniversityandtheUniversityofNorthCarolinaatChapelHillandguestlecturedatseveraldozenuniversities.
Carl W. Swartzconductsresearchonmodelsandtechnologiesforblendinglearningandassessmentanduseoftechnologyasacomponentofeducationalprogramstoenhancepersonalizedlearningforstudentswithlearning,attention,andlanguagedifferences.HeiscurrentlyaResearchProfessorintheDepartmentofAppliedDevelopmentalandSpecialEducationintheSchoolofEducationattheUniversityofNorthCarolina-ChapelHillwherehewasaResearchScientistintheSchoolofMedicineandClini-calAssistantProfessorintheSchoolofEducationatTheUniversityofNorthCarolinaatChapelHill.Currently,HereceivedhisundergraduatedegreeineducationfromIndianaUniversityandhismaster’s
cxli
About the Contributors
anddoctoratedegreesineducationfromTheUniversityofNorthCarolinaatChapelHill.Dr.SwartztaughtearlyadolescentswithsevereemotionaldisturbancesandbehaviordisordersatamiddleschoolinGreenville,SouthCarolina(1983-1986).
Noel Tagoe is theExecutiveDirectorofEducationatCIMA,wherehewaspreviouslyHeadofResearchandDevelopment.PriortojoiningCIMANoelhadheldaccounting,strategyandfinancialconsultingpositionswithBPandKMPGinGhana.HehasalsotaughtatvariousuniversitiesintheUKincludingManchesterUniversityandOxfordUniversity.Noel’scurrentinterestsareindesigningprofes-sionalaccountingcurriculathataddressbusinessneedsandensuringthattheyareassessedrigorouslyandrobustlyusingtechnologywherepossible.
Fatima E. Terrazas-ArellanesisPIofProjectESCOLAR,anOSEP-fundedprojecttocreateonlinemodulesforteachingsciencetomiddleschoolstudentswithlearningdisabilities.Dr.Terrazas-ArellaneswasateacherfromMexico,coordinatingresearchprojectsinareasofSpanishliteracyinstruction,secondlanguageacquisition,andeTextsupports.Dr.Terrazas-Arellanes’areasofexpertiseincludedesigningandconductingresearchtolearnhowtechnologyaddressestheeducationalneedsofchildrenandyouths,particularlystudentswithlearningdisabilitiesandEnglishLanguageLearners.Dr.Terrazas-ArellanesreceivedherBachelorofArtsinPsychologyfromtheUniversidadAutonomadeSinaloain2000,aMaster’sofScienceinSpecialEducationfromtheUniversityofOregonin2008,aswellasaDoctorofPhilosophyfromtheUniversityofOregon’sSchoolofPsychologyin2009.
Suzanne Tsacoumishasbuiltherprofessionalcareerconductingandmanagingprojectsassociatedwiththeresearch,development,andimplementationofpersonnelassessmentsandhumancapitalsys-tems,ofteninlitigiousenvironments.Herexpertiserevolvesaroundjobanalysis,selection,promotion,leadershipassessment,andperformanceevaluation.Currently,sheisspearheadinginnovativeworkinthedevelopmentofvalid,onlinesimulations,suchasvirtualroleplaysandinteractivein-baskets,foruseinbothpromotionsystemsandself-assessmentprocesses.Sheoftenconsultsonarangeofpolicyandimplementationissues.Inadditiontohertechnicalwork,SuzanneisacorporateofficerandHumRRO’svicepresidentofbusinessdevelopment.Inthisrole,sheisresponsibleforprovidingleadershipanddi-rectiontothebusinessdevelopmentandmarketingteam.SuzannehasbeenanactivevolunteerfortheSocietyofIndustrial-OrganizationalPsychology(SIOP)andsheisafellowofSIOPandtheAmericanPsychologicalAssociation(APA).SuzannehasservedasanadjunctprofessoratbothTheGeorgeWash-ingtonUniversityandGeorgeMasonUniversity.SheearnedherPh.D.fromtheUniversityofGeorgia,specializinginIndustrial-OrganizationalPsychologyandherB.A.fromBucknellUniversity.
David Vaughnisasoftwareengineerwithovertenyearsofexperienceinthefieldofartificialintel-ligence(AI)andmachinelearning.AsaSeniorSoftwareDeveloperhedesigns,develops,andenhancesautomatedscoringtechnologies.Mr.VaughnwastheprimaryarchitectofMI’swinningsubmissionstotheAutomatedStudentAssessmentPrize(ASAP)contest,anautomatedscoringcompetitionsponsoredbytheHewlettFoundation.HesubsequentlyenhancedtheAIscoringsystemusedintheASAPcom-petitiontoimproveitsaccuracy,speed,efficiency,andscalabilityforuseontheSmarterBalancedPilotTestautomatedscoringproject.Previously,Mr.Vaughndevelopedandimplementedmachinelearningalgorithmsinavarietyofcontextsincludingemailclassification,genomeresearch,andhuman/computervoicedialoguesystems.HereceivedhisB.S.inComputerSciencefromHarvardUniversity.
cxlii
About the Contributors
Lars VavikisaprofessorininteractiveeducationaltechnologyatStord/HaugesundUniversityCollege.Hehasbeenworkingwithdesignofdigitalmediasince1980andhasbeengivenEuropeanAcademicSoftwareAwardfordevelopingofmodelingandsimulationtools.Hisworkthelast15yearshasbeendedicatedtodevelopingamasterprograminICTandlearningandtheleadershipofseveralresearchprogramas“EducationonCurriculumandTechnology”(2008-2012)and“Learninginthe21stcentury”(2012-2016)supportedbyTheResearchCouncilofNorway.
Alina von DavierisaSeniorResearchDirectorandleaderoftheCenterforAdvancedPsychometricsatETS.ShealsoisanAdjunctProfessoratFordhamUniversity.AtETS,vonDavierisresponsiblefordevelopingateamofexpertsandapsychometricresearchagendainsupportofnextgenerationofas-sessments.Computationalpsychometrics,whichincludemachinelearninganddataminingtechniques,Bayesianinferencemethods,stochasticprocessesandpsychometricmodelsarethemainsetoftoolsemployedinhercurrentwork.Shealsoworkswithpsychometricmodelsappliedtoeducationaltesting:testscoreequatingmethods,itemresponsetheorymodels,adaptivetesting.Shepublishedseveralbooks,authoredoredited;shealsopublishednumerouspapersinpeerreviewedjournals.
Matthias von DavierisaSeniorResearchDirectorinETS’sResearch&DevelopmentDivision,managingtheinternationallarge-scaleassessmentmethodologygroupintheETScenterforglobalas-sessment.AtETS,Dr.vonDaviermanagesagroupofresearchersconcernedwithmethodologicalques-tionsarisinginlarge-scaleinternationalcomparativestudiesineducation.Heiscurrentlyeditor-in-chiefoftheBritishJournalofMathematicalandStatisticalPsychologyandco-editorofthejournal“LargeScaleAssessmentsinEducation”,whichisjointlypublishedwiththeInternationalAssociationfortheEvaluationofEducationalAchievement(IEA)andETSthroughtheIEA-ETSResearchInstitute(IERI).HiscurrentworkatETSinvolvesthepsychometricmethodologiesusedinanalyzingcognitiveskillsdataandbackgrounddatafromlarge-scaleeducationalsurveys,suchastheOrganizationforEconomicCo-operationandDevelopment’supcomingPIAACandtheongoingPISA,aswellasIEA’sTIMSSandPIRLS.HisworkatETSalsoinvolvesthedevelopmentofsoftwareformultidimensionalmodelsforitemresponsedata,andtheimprovementofmodelsandestimationmethodsfortheanalysisofdatafromlargescaleeducationalsurveyassessments.PriortojoiningETS,Dr.vonDavierledaresearchgrouponcomputerassistedsciencelearning,wasco-directorofthe“Computerasatoolforlearning”sectionattheInstituteforScienceEducation(IPN)inKiel,Germany,andwasanassociatememberofthePsychometrics&MethodologyDepartmentofIPN.
Zsofia VorosiscurrentlyresearchassociateintheCenterforResearchonCognitionandLearning,CNRSandUniversityofPoitiers,France.Herresearchinvestigatescognitivedimensionsthatmayun-derlieelectronicreading,learningandproblemsolving.Previously,sheworkedasaresearchconsultantfortheEducationalTestingService.SheexploredProgrammefortheInternationalAssessmentofAdultCompetencies(PIAAC)processingdatatoidentifycognitive/metacognitiveskillsandsolutionstrategieslinkedtosuccessinvariouselectronicproblemsolvingtasks.SheearnedherPh.DinpsychologyfromtheUniversityofPoitiers,France(2009).
Emily Walden,M.A.,isaResearchAssistantattheCenterforAdvancedTechnologyinEducationattheUniversityofOregoninEugene,Oregon.Shecompletedabachelor’sdegreeinEnglishandpsy-chologyattheUniversityofOregonin2010andamaster’sdegreeindevelopmentalpsychologyatthe
cxliii
About the Contributors
UniversityofOregonin2011.Herresearchwithfederallyfundedprojects(ESTRELLAS,COPELLS,SSOAR,andESCOLAR)primarilyinvolvesimprovingeducationoutcomes,especiallyinscience,formiddleschoolstudentswithlearningdisabilitiesandEnglishlearnersthroughtheuseoftechnology.
Nancy T. WalkeriscurrentlyaProfessorofEducationattheUniversityofLaverne.Forthepastfifteenyears,shehasoverseentheteachereducationliteracyfacultyandworkedwithadjunctfacultyinonlinelearning.Herresearchhasfocusedoncontentarealiteracy,disciplinaryliteracy,visualliteracy,andonlinelearninginhighereducation.Previously,shetaughtLanguageArtsatthemiddleschoollevelforeightyearsandmentorednewteachersineffectiveliteracyinstruction.Dr.Walkerisapublishedauthorontheuseofmultipletextsintheclassroomandisafrequentspeakeratnationalandinternationalconferences.
James WelshistheDirectoroftheFloridaCenterforInstructionalTechnologyandholdsaPh.D.inLiteracyStudiesfromtheUniversityofSouthFloridaCollegeofEducation.Aformerelementaryschoolteacher,JamesistheprogramcoordinatorfortechnologyintegrationsupportintheUSFCollegeofEducationandtheprojectleaderfortheTechnologyIntegrationMatrixK-12evaluationtools.JamesconductsresearchwiththeContemporaryLiteraciesCollaborativeatUSF.Hisresearchinterestsincludeevaluationofeducationaltechnology,criticalmedialiteracy,studentcreationofmultimediatexts,andtheroleofgenreinstudentcomposition.
Joshua WilsonisanAssistantProfessorintheSchoolofEducationattheUniversityofDelaware.HeearnedhisPh.D.inSpecialEducationfromtheUniversityofConnecticut in2014.Hisresearchfocusesonmethodsofassessingandinstructingstrugglingwriters,andontheapplicationofautomatedessayevaluation(AEE)technologyinResponse toIntervention(RtI)contexts.Hecurrently teachescoursesonelementaryspecialeducationmethods.PriortoearninghisPh.D.,Dr.Wilsonwasaspecialeducationteacherforsixyears.
Roy WinkelmanhasspenttwodecadesinK-12educationandtwodecadesinhighereducation.FormerlydirectoroftheFloridaCenterforInstructionalTechnology,henowworksonspecialprojectsfortheCenterandcreatesdigitalresourcesforK-12studentsandteachers.
Steven WiseisaSeniorResearchFellowatNorthwestEvaluationAssociation.Dr.Wisehaspub-lishedextensivelyduringthepastthreedecadesinappliedmeasurement,withparticularemphasesincomputer-basedtestingandthepsychologyoftesttaking.Inaddition,hesitsontheeditorialboardofseveralacademicjournalsandprovidespsychometricconsultationtoavarietyoforganizations.Inrecentyears,Dr.Wise’sresearchhasfocusedprimarilyonmethodsforeffectivelydealingwiththemeasure-mentproblemsposedbylowexamineeeffortonachievementtests.
Raffaela WolfcompletedherPhDinResearchMethodologyfromtheUniversityofPittsburghinDecember2013.Dr.RaffaelaWolfjoinedCAEinJanuary2014asaMeasurementScientist.Inthisrole,Dr.RaffaelaWolfisinvolvedinresearchprojectsthatpertaintovalidityandreliabilitystudiesoftheCollegiateLearningAssessment-Plus.Otherresponsibilitiesincluderunningpsychometricanalysesfornationalandinternationalclients.Researchinterestsincludeequating,scaling,andlinking;itemresponsetheory;structuralequationmodeling;and,cross-nationalassessmentsinhighereducation.
cxliv
About the Contributors
Doris ZahnerjoinedCAEinMarch2011astheDirectorofTestDevelopmentandaMeasurementScientist.HerresponsibilitiesincludeoverseeingallitemdevelopmentforCAEaswellasconductingresearchstudiespertainingtotheCLA+andotherCAEassessmentinstruments.Priortoherpositionwith theCAE,shewasaProgramDirectorwithaprofessional licensureandcertification firmandmanagedavarietyofclients,includingamedicalcertificationboard,aninformationsystemsauditingcertificationprogram,variousnursingspecialties,andplasticsengineers.Shehasexperiencemanaginglarge-scaleitemdevelopmentprojectsandisknowledgeableaboutallaspectsofassessmentdevelopmentincludingtaskanalyses,testspecifications,itemwriting,examconstruction,scorereportingandequating,andstandardsetting.SheholdsaPh.D.inCognitivePsychologyandaM.S.inAppliedStatisticsfromTeachersCollege,ColumbiaUniversity.Herpost-doctoralworkattheStevensInstituteofTechnologywasontheuseofdiagramsininformationsystemsdesignandherownresearchinterestspertaintotheuseofdiagramsinprobabilityandmathematicsproblemsolving.InadditiontoherresponsibilitiesatCAE,Dr.ZahnerisanadjunctassistantprofessoratBarnardCollegewheresheteachesacourseonstatisticstoundergraduatestudentsinthesocialsciences.
Diego Zapata-Rivera,PhD,isaseniorresearchscientistintheCognitiveScienceResearchGroupatEducationalTestingService.Hisresearchfocusesoninnovationsinscorereportingandtechnology-enhancedassessment,includingworkonassessment-basedlearningenvironmentsandgame-basedas-sessments.HisresearchinterestsalsoincludeEvidence-CenteredDesign,Bayesianstudentmodeling,openstudentmodels,conversation-basedassessments,virtualcommunities,authoringtoolsandprogramevaluation.Hehasbeenacommitteememberandorganizerofinternationalconferencesandworkshopsinhisresearchareas.HeisamemberoftheBoardofSpecialReviewersoftheUserModelingandUser-AdaptedInteractionjournalandanAssociateEditoroftheIEEETransactionsonLearningTechnolo-giesJournal.Mostrecently,Dr.Zapata-RiverahasbeeninvitedtocontributehisexpertisetoprojectssponsoredbytheNationalResearchCouncil,theNationalScienceFoundationandNASA.
cxlv
Top Related