A FRAMEWORK FOR EVALUATING APPROPRIATENESS OF EDUCATIONAL TECHNOLOGY USE IN GLOBAL DEVELOPMENT PROGRAMS TheMassachusettsInstituteofTechnology,Cambridge,Massachusetts&TheIndianInstituteofManagement,Ahmedabad,India
2
TheComprehensiveInitiativeonTechnologyEvaluation(CITE)atMITisaprogramdedicatedtodevelopingmethodsforproductevaluationinglobaldevelopment.CITEisledbyaninterdisciplinaryteam,anddrawsupondiverseexpertisetoevaluateproductsanddevelopanunderstandingofwhatmakesproductssuccessfulinemergingmarkets.TheIndianInstituteofManagement,Ahmedabadwasanessentialpartnerinresearchandtestingofthiseducationaltechnologyframework.
ThisreportwasmadepossiblethroughsupportoftheUnitedStatesAgencyforInternationalDevelopment.TheopinionsexpressedhereinarethoseoftheauthorsanddonotnecessarilyreflecttheviewsoftheUnitedStatesAgencyforInternationalDevelopmentortheUSGovernment.
Table of Contents
INTRODUCTION 3DEVELOPINGAFRAMEWORK 4GENERALIZABILITYANDUSEINOTHERCONTEXTS 6USINGTHEFRAMEWORK 7COMPREHENSIVEFRAMEWORK 9CONSIDERATIONSFORFUTUREAPPLICATION 18REFERENCES 18AUTHORS&ACKNOWLEDGEMENTS 19
3
Introduction
Asdigitalmediaproliferate,andincreasingamountsofdailyworkareperformedindigitalenvironments,therearegrowingdemandsfromparents,educators,andgovernmentstodeployeducationaltechnologiesinglobaldevelopmentprograms,whetherasameanstoimprovethequalityofeducationingeneral,orastoolstofamiliarizestudentswiththetechnologiesthatwillshapetheirfuturelives.Thisdemandcanbedrivenby:
• Enthusiasmforemergingtechnologies;
• Expectations(realisticorotherwise)forwhattechnologycanachieve;
• Fearofbeingleftoutofemergingsocio-culturaldevelopments;or
• Alloftheabove.
Whileallschoolsmayshareacommongoalofeducatingstudents,thereisabroaddiversityofmeans,dependingonsuchvariablesas:
• Schoolfunding;
• Teacherpreparedness;
• Educationalphilosophy;or
• Technicalinfrastructure.
Unfortunately,manyconsumersoftechnologicalinterventions—policymakers,administrators,teachers,andparents—failtoaccountforthesevariablesinmakingdecisionsabouttheadoptionofanyparticulartechnology(Davies,2011).Andmanydeveloperscreatetechnologicalinterventionswithoutfullyunderstandingtheeducationalsystemsintowhichtheywillbeintroduced.
Therefore,thereisaneedforevaluativetoolsthatwill:
• Aidvariousstakeholdersindeterminingwhicheducationalinterventionsaremostpromisingforanyparticularcontext;and
• Helpstakeholdersevaluateinterventionsastheyareintheprocessofimplementation(i.e.formativeassessment).
Itwouldbedifficulttooverstatethechallengesinherentinanyefforttoobjectivelyevaluatetheeffectivenessofagiveneducationaltechnology.Tobegin,thenotionofwhatconstituteseffectiveeducationishighlycontested,eveninadevelopedcountrysuchastheUnitedStates,withaneducationalsystemthathashad150yearsofrelativepeaceandprosperityinwhichtoprogressandevolve.Oneneedonlylookatthelivelydebatesthatrevolvearoundsuchquestionsastheusesofstandardizedtesting,commoncorecurricula,charterschools,ortheroleofcomputersinchildren’seducationaldevelopmenttoacknowledgethatthereisnoconsensusonwhateducationalsuccesslookslike,andsimilarlynoconsensusonhowtomeasureanyputativesuccess.Evenwhenoneidentifiesthedesiredandmeasureableoutcomesofparticularintervention,thechallengesandcostsofperforming
4
scientificallyvalidassessmentsusingrandomlyassigned,singlevariabletreatmentandcontrolgroupsinsufficientlylargepopulationscanbestaggering.
Ifweshiftourattentiontothedevelopingworldwearelikelytofindevengreaterchallengesresultingfromunder-fundedschools,ashortageofprofessionaleducators,andalimitedtechnologicalinfrastructure.Inaddition,oneencountersthesamedisagreementsaboutthepurposeofeducationandwhatconstitutessoundpedagogyasarefoundinthedevelopedworld.Anymeaningfulevaluationmustaccommodateitselftoallthesefactors.
Webeginbyacknowledgingthesechallengesnotinthespiritofresignation,butrathertoavoidthegrandioseclaimsthatarealltoooftenmadeonbehalfofeducationaltechnology.Theauthorsofthisreporthaveworkedfordecadesbothdevelopingandevaluatingeducationaltechnologies.Weremainoptimisticabouttheroleoftechnologyineducation,butwealsounderstandthatthegreatestrisktotheadoptionofanytechnologymaybeunrealisticexpectations,andsubsequentdiscouragementandprematurecapitulationtodefeatism.Accordingly,thisreportidentifiesmethodsofevaluatingeducationaltechnologiesthatarebothpracticalandadaptabletoawiderangeofeducationalsettings,andthatwillresultintheadoptionanduseoftechnologiesinwaysthatarepotentiallysustainableindevelopingenvironments.
Developing a Framework
Todevelopausefultoolinsuchcomplexcontexts,wechosetodevelopaframeworkthatpullsfromandsynthesizestheexistingliteraturerelatedtothisspace,aswellasdataandfeedbackfromrealworldcontextsthatareseekingtoeffectivelychooselearningtechnologiesandwouldbenefitfromexternalsupportsandtoolsindoingso.
Frameworksofferseveralbenefits.First,theyclarifycomplexorambiguoussituations(Whetten&Cameron,2014).Agoodframeworklaysoutthedimensionsofthecomplexityoftheproblemspace—manyofwhichwouldoftenbeotherwiseoverlooked.Agoodframeworkputsalloftheseelementsanddimensionsontheradarofthepeopleinvolvedsothateachcanbeconfrontedandaddressedappropriatelyinthereal-worldcontextandproblem.Second,agoodframeworkwillalsohelppromptandsupporteffectiveengagementwitheachofthosedimensionsasitrelatestotheircontext.Inotherwords,theframeworknotonlyframestheentireproblemspace,butitframeshowtotakestepsingettingtowardsaneffectivesolution.Eveninacomplexproblemspace,frameworksserveasbothanchorsand/ortouchstonestoreturnto,providingstabilityinthemidstofconstantchange(Whetten&Cameron,2011).
Wechosetostructureourframeworkasaquestionnaire,withthequestions(andbyextension,theframeworkitself)performingtwofunctions:
1. Whenusedbyanoutsideevaluator,thequestionscanstructuretheexplorationofallthesalientelementsofaproposedintervention,oronealreadyinprocess.
2. Whenusedbyastakeholdereithercreating,oradoptinganintervention,thequestionsactaspromptstohelpthestakeholderfullyreflectontherangeofrelevantissues,someofwhichmayhavepreviouslygoneunconsidered.
5
Thoughitwouldbebeyondthescopeofthisstudy,itwasourhopethatiftheframeworkprovedeffective,wewouldseekthemeanstodevelopanon-lineversionthatcouldbeadministeredeitherbyevaluatorsorstakeholders.Thiswouldtakeadvantagesofthebranchingcapabilitiesinadigitalenvironmenttotailornavigationthroughtheframeworkbasedonuserresponses.
Thegoalfortheinitialpilotwastotesttheusabilityoftheframeworkinthefield,andtomodifyitwhereappropriatebasedonuseexperiences.
Informing the Framework Followinganextensiveliteraturereview,theframeworkwasdevelopedbyresearchersattheMassachusettsInstituteofTechnology,andthentestedinIndia,withasmallteamofinvestigatorsfromtheIndianInstituteofManagement,Ahmedabadcarryingoutfieldworkandanalysis.Theaiminthepilotwastwofold:(1)totesttheframeworkoutinvariouscontextswhereeducationaltechnologywasbeingusedandseewhetherthequestionswewereaskingwererelevantinthesecontextsand(2)whethertherewasanythingofimportancetovariousstakeholdersinthefieldthatwewerenotasking,butshouldbe.Wethereforeusedthisasaniterativeprocesstoimprovetheframeworkbasedondatafromthefield.
Tomeetourrequirementofsimultaneouslytestingandaddingtotheframework,wedecidedonholdingsemi-structuredinterviewsand/orgroupdiscussionswithvariousstakeholders—developers,implementingorfacilitatingagencies,schoolleadersoradministrators,teachers,studentsandwhereavailable,parents—insiteswherethetechnologiesstudiedhadbeenpilotedordeployed.Asemi-structuredformatallowedustoguidethediscussionsbasedontheframework,butalsotobeopentodivergencesandopen-endedresponsesthatmightleadustotopicsthatourframeworkshouldfocuson,butdidnot.However,becausethedatacollectionandanalysisfromsuchmethodsareresource-intensive,thenumberofsitesforourstudywerelimited.
Theschoolsweobservedinthepilotstudyrangedfromthemostbasictothemostmodern,including:
• AruralvillageinUttarPradeshwithoutaschoolbuildingordedicatedclassroom,whereNGOEkalVidyalayahadtrainedvolunteersinthedeliveryofrudimentarymathandreadinglessonstochildrenfromages5to10
• AruralvillageinGujarat,whereanotherNGO,PlanetRead,providedvillagerswithsubtitledBollywoodmoviestobeviewedcommunallytoreinforceliteracy
• RiversideSchool,amodernindependentschoolinAhmedabadwithfacilitiesthatwouldmatchthoseofaprogressiveprivateschoolintheU.S.;studentsusedcomputertechnologyregularlyintheirschoolwork
• AseriesofcomputerlabsinbothpublicschoolsandcommunitycentersinMumbaiandPunerunbythePrathamEducationFoundation,oneofthemostinfluentialNGOsworkingtoimproveeducationinIndia
6
Ourschoolvisitsconfirmedoursuppositionthatinanygivensettingmultiplevariableswereinplay,suchas:funding,technologicalinfrastructure,pedagogicalmodels,andteacherengagement.Foraframeworktobeeffectiveitwouldhavetohelpevaluatorsorstakeholdersdeterminethefitofagiventechnologywithallsuchvariablesaccountedfor.
Inaddition,westudiedthreeEnglishlanguagelearningtechnologyinterventionsthathadeitherbeenrecentlydeployedorwereintheprocessofdeployment,takingintoaccount(a)diversityofcontext,natureofuseandstageofimplementationand(b)accesstothevariousstakeholders.Thetechnologiesincluded:
1. EnglishHelper(EH)RightToRead:Aread-alongsoftwareforclassrooms,describedasmultisensorysinceitsimultaneouslyengagesvisionandhearing.TheprimarytargetseemstobestudentsinearlystagesoflearningtoreadEnglishasasecond(orthird)language.
2. Mindspark(MS)English:Acommercialadaptivelearningsoftware,marketedtobothschoolsandindividuals.Mindsparkisintendedforusedirectlybylearnersanditssubscriptionfeemakesitinaccessibletolower-incomegroupswhoenrolltheirchildreningovernmentschools.
3. EkStep:AprojectthataimstoaddresslearninggapsinEnglishandMathinprimaryeducationatanationalscaleusingtechnology.EkStepwasinterestingtoussinceittargetsbothformalandnon-formaleducation,andaimstoquicklyscaletomillionsoflearnersoncedeployed.
Sincethepilotwasintendedtobeaniterativeprocesstoimprovetheframework,thedatacollectedfromeachsitewasdiscussedextensivelybytheteamandusedtomakeadditionsormodificationstotheframework.Thisupdatedframeworkwasusedatthenextvisit,whethertothesamesiteoranewone.Theframeworkthatfollowsisinformedbythefindingsofthispilotstudy,whicharedetailedinthefullreportavailableatcite.mit.edu.
Generalizability and Use in Other Contexts Asinitiallydevelopedandsubsequentlymodified,theframeworkaddressesquestionsof“fit”betweenaparticularinterventionandagiveneducationalcontext.Forthesakeoffocus,wepilotedtheframeworkbyevaluatingtheuseofEnglishlanguageandliteracyproductsbeingdeployedinIndia.Theparticularcircumstancesweencounteredonthegroundinformedourthinkingandcontributedtofurtherrefinementoftheframework.
Whileitfollowsthatfurtherapplicationsoftheframeworkmightleadtofurtherrefinements,weneverthelessbelievethatthisframeworkisrobustenoughtosuccessfullybedeployedinothercontexts—betheyotheracademicdomainsorothercountries’educationalsystems.Webasethisconfidenceonthenatureofthemodificationsthatoccurredduringthepiloting.Specifically,thesemodificationstendedtointroducenewdimensionstoquestionsalreadypresentintheframework,likethoseofimplementation,equilibrium,orengagement.Thereisnothingintheexperienceofthisinitialpilottosuggesttherearesignificantgapsinitscurrentscope.
7
Using the Framework
Inthecourseofthisstudywedevelopedaframeworktobeusedbyvariousstakeholdersinassessingtheappropriatenessofneweducationalproductsorinterventionsincluding:
• Developersofnewtechnologies;
• Adoptersofnewtechnologies,includingsystem-wideadministrators,schoolprincipalsorteachers;or
• Fundersofnewtechnologies.
Theframeworkisintendedtobeusedbeforetheadoptionofanintervention,orasaformativeassessmentofthatinterventionasitisbeingdeployed.Thoughwepilotedthisframeworkinthecontextofthespecificdomainoflanguagelearning,ourlargergoalwastocreateatoolthatwouldbemorebroadlyapplicable.
Theframeworkisdesignedtoassistmultiplestakeholderstomoresuccessfullyplanandimplementaneducationaltechnologyinagivensetting.Thiscouldrangefromasingleteacherwantingtotryanewtechnologyinhisorherclassroom,toanentiredistrictorstatesystemofeducationseekingtoimplementatechnology-basedlearningprogram.
Forthecaseofindividualteachers,thismightinvolvereviewingtheelementsoftheframeworkinregardstoanyparticulareducationaltechnologytheydesiretouse,andreflectingontheanswersinordertoidentifyandaddressandbarriersthatmaycomeup.Forexample,theframeworkincludeselementssuchas"technologyresourcesrequired,"whereateachermightidentifythatthetoolinquestionrequireshigh-speedWi-Fiinorderforthestudentstoeffectivelydothetypesoflearningactivitiesintheclassroomthattheteacherimagines,andthereforetheteachermighteitherchoosetonotusethetechnologyorfindwaystoworkaroundthis.Anotherexamplewouldbe"accesstoresourcesandsupports,"andinreviewingthispartoftheframeworkteachersmightrealizethattheydon'thaveanyoneintheirschoolwhocanhelpthemandthereforeseekoutsomeoneintheirprofessionalnetworkwhoknowsaboutthistoolthattheycanconnectwithshouldtheyrunintoanyproblemswhileusingthetoolintheclassroom.
Foranyoftheelements,theindividualmightrespondtothepromptinoneoffourways:
1.Idon’tknowtheanswersyet;
2.Nothespecificconditionsarenotpresent;
3.Itwillrequireadditionalresourcestowork;or
4.Thereisafit/theconditionsarepresent
Foragivenaspectoftheframework,iftheresponseis#1,thentheuserneedstodigdeeperandbetterunderstandthetooland/orthecontextandresourcesheorsheisworkingwith.Iftheresponseis#2or#3,theuserneedstoconsiderwhatwouldneedtochange,andifthatchangedesirableorevenpossible.Iftheresponseis#4,theuserhasidentifiedanareaofstrengthforthisimplementation,whichmaybeabletohelpsupportotherareasthatarelacking.
8
Framework Overview
TEACHERS COMMUNITY,SOCIAL,POLITICAL
T.1.ComfortT.2.CompetenceT.3.OpennesstoChangeT.4.RoleT.5.ClassroomManagement
CSP.1.ImplementationCSP.2.Support
STUDENTS LEARNING
S.1.ComfortS.2.AccessS.3.OpennesstoChange
L.1.LearningGoals/ImpactonLearningL.2.PedagogyL.3.Curriculum
CULTURE INFRASTRUCTURE
C.1.CulturallyRelevancy I.1.EquipmentI.2.ElectricityI.3.Internet
SUSTAINABILITY SCALABILITY&MARKETIMPACT
SU.1.FundingSU.2.Maintenance&Repairs
SM.1.BroaderCommunityImpactSM.2.Adoption&Scaling
9
Comprehensive Framework
TEACHERS
T.1. Comfort
T.1.1. ComfortwithTechnology
Howcomfortablearetheteacherswithtechnology?Intermsofgeneraluseaswellasinaneducationalsetting.
T.1.2. ComfortwithTeachingStudentsTechnology
Howcomfortableareteachersinteachingstudentshowtousethetechnology?Asis,andthenwithadditionaltraining.
T.2. Competence
T.2.1. ProfessionalDevelopmentRequired
Howmuchlearningofthetechnologywouldteachersneed?Andwhatisthestructure?(onedayvs.multiplesessions?)
T.2.2. ResourcesforProfessionalDevelopment
Whowouldprovidetheinstruction?Outsidevs.in-schoolemployee
T.2.3. ProfessionalDevelopmentScheduling
Whenwouldtheinstructionhappen?Areadditionalworkhoursneeded?
T.2.4. ProfessionalDevelopmentCosts
Whatadditionalcostsareassociatedwiththeinstruction?Dotheteachers,school,ortechnologycompanycoverthesecosts?
T.3. OpennesstoChange
T.3.1. LearningTechnology
Areteacherswillingtolearnhowtousethetechnology?Howmuchtimearetheywillingtoputintolearnhowtousethetechnology?Isthereanassociatedjobtrainingbenefitoflearningthetechnology?
T.3.2. LearningNewPedagogies
Areteacherswillingtochangetheirpedagogytoaccommodatetheuseoftechnology?Hasitbeenmadecleartoteacherswhytheyareusingthetechnology?Isthetechnologyinalignmentwithteachers’currentlearninggoalsforstudents?Isthetechnologyinalignmentwiththeschool-widegoalsforlearning?
10
T.4. Role
T.4.1. RolewithTechnology
Whatistheroleoftheteacherintheimplementationofthetechnology?Isthetechnologyseenasan“addedresponsibility”ora“teacherreplacement”withoutanybenefits?Isthetechnologyperceivedinapositivelight,asatooltoaidinteaching/learning?Howdoestheteacherinteractwithstudentsusingthetechnology?
T.5. ClassroomManagement
T.5.1. MonitoringTechnologyUse
Howwillthetechnologyusebemonitored(sostudentscannotaccessinappropriatecontent)?Doesthetechnologycompanyputrestrictionsinplace?Aretheteacher/schoolresponsibleformonitoringcontent?Dotheyknowhowtoeffectivelysetupmonitoring?
T.5.2. DemandsbytheTechnology
Doesthetechnologycreateaburdenofextramanagementfortheteacher?Doesthetechnologymakelearningmoreefficientandeffectiveintermsoftimefortheteacher?Istheteacherawareofhowthestudentsareusingthetechnologyatanindividuallevel?Doestheteacherreceiveusageandprogressreportsorcantheymonitorusageeasily?Doesmonitoringtheusagetakealotofextraeffortfortheteacher?
STUDENTS
S.1. Comfort
S.1.1. ComfortwithTechnology
Whatdotheyknowhowtodo/whatistheircomfortlevelwithtechnology?Includingthekindsoftechnologytheyhavecomfortwith(phones,tablets,PCs,etc.)andtheactionstheyarecomfortablewithusingthetechnologyfor(i.e.wordprocessing,apps,internet,etc.)
S.1.2. StudentSupport
Howmuchinstructionwouldstudentsneedtousethetechnology?WHENwouldthishappen?Inschoolvs.afterschoolhoursWouldstudentsbewillingtocomeinafterschoolfortheinstruction?WHOwouldprovidetheinstruction?Teachersvs.outsidefacilitatorsHOWMUCHwouldtheinstructioncost?Intermsofteachertimeoroutsidefacilitatorcost/timeWillstudentsmastertechnologywithgreaterfacilitythanteachers?Couldtheyassisttheteacherinmasteringthetechnology?
11
S.2. Access
S.2.1. StudentHomeAccess
Whattechnology,ifany,dostudentshaveaccesstoathome?Inotherpublicspaces(i.e.publiclibrary,afterschoolprograms,etc.)?Dotheyshareaccessordotheyhaveindividualdevices?
S.2.2. Student-TechnologyAccessNeeds
Wouldtheyneedaccesstothetechnologyathomeaswellasinschool?Iftheyneedaccessathome,howfrequentlyandhowmuchtimeperuse?Istechnologyequallyaccessibleinallhomes?
S.2.3. EquitableAccess
Willtherebeequitableaccesstotechnologyforstudentsbetweengendersandage-levels(whereappropriate)?Howcanthisbeensured?
S.3. OpennesstoChange
S.3.1. LearningTechnology
Howwillingarestudentstousethetechnologyinschool/athome?
S.3.2. PerspectiveonNewTechnologies
Dostudentsviewtechnologyasanopportunityoraburden?Arestudentsexcitedaboutthechancetousethistechnology?Aretheynervousaboutusingthistechnology?Doesthisdependontheirage/gender?
12
COMMUNITY/SOCIAL/POLITICAL
CSP.1. Implementation
CSP.1.1. OutsideFacilitatorNeeds
Isthereafacilitatorfromoutsideoftheschoolthatisnecessarytoimplementthetechnology?Howaretheyassociatedwiththeschool?Whatisthedynamicbetweenthefacilitatorandtheteachers/students/administration?(intermsofinteractionandculture)Doestheschoolhavetopaythefacilitator?Howmuch?Whatarethemotivationsoftheoutsidefacilitator?Isthefacilitatingagencystableoverthelonghaul?
CSP.1.2. TeacherRequirements
Whatisthedegreethatthetechisimplementablewithouttheteacher?Doteachers/facilitatorsneedtoalwaysbepresent?Sometimespresent?Neverpresent?
CSP.2. Support
CSP.2.1. PerceptionsofTechnology
Howisthetechnologyperceivedbythecommunity?Opportunityvs.unnecessary
CSP.2.2. SupportoftheTechnology
Istherepoliticalsupportfortheuseofthetechnology?Inwhatform?Localgovernment?Stategovernment?
CSP.2.3. ReportingNeeds
Isitnecessarytoprovidereportstoanycommunity/political/fundingorganizations?Howoftenmustthesereportsbedone?Whatisthenecessarycontentofthereports?Doesthecompletion/accuracy/contentofthereportsdeterminefundingorsupportoftheprogram?
CSM.2.4. Governmental/AdministrativeApproval
Isgovernmentapprovalnecessarytousethetechnology?Ifso,howdoesapprovaloccur?Istherealongtimelagtogainapproval?
13
LEARNING
L.1. LearningGoals/ImpactonLearning
L.1.1. LearningGoals
Whatarethelearninggoals?(teacherandschool-based)Arethesegoalscurrentlybeingmet?Isthetechnologyappropriateforaddressingthesegoals?Isthetechnologynecessarytoachievethesegoals?Istechnologythebestmethodforachievingthesegoals?Whatlearninggoalswillnotbemetbythetechnology?
L.1.2. EvidenceofLearning
Isthereevidencethattheuseofthistechnologyaidslearning?Whatistheevidence?Isitreliable?Istheevidencegeneralizabletothiscontext?
L.1.3. MeasurementofLearning
Howwilllearningasaresultofthetechnologybemeasured?Standardassessments,pre/posttests?Qualitativemeasures?Willassessmentsalignwithexistinglearninggoalsorbetailoredtothetechnology’saffordances?
L.2. Pedagogy
L.2.1. CurrentPedagogicalModel
Whatisthepedagogicalmodelrightnow?DirectInstructionbyteachers?CollaborativeLearning?Inquiry-basedLearning?Project-basedLearningProblem-basedLearningArepedagogicalapproachesuniformacrosstheschoolordoteachershavesomeautonomyintermsofteachingstyles?
L.2.2. CurrentClassroomTools
Whattoolsareusedtoteachintheclassroom?TextbooksWorksheetsHand-outsGamesHands-onmodelsOther?Arecomputersusedintheclassroom?Ifso,how?Aremobiledevicesusedintheclassroom?Ifso,how?
L.2.3. ProposedPedagogicalModel
14
Whatshouldthepedagogicalmodellooklikewiththetechnology?Doesitneedtobeschool-wide(allteachersadopt)?Studentcenteredvs.teachercentered
L.2.4. BlendedLearningCapacity
Towhatextentareteacherspreparedtoimplementablendedlearningenvironment?Areteacherswillingtochangetheirpedagogicalpracticestoutilizethetechnology?(seeteachers>willingnesstochange)
L.2.5. ImpactonCurrentPractices
Whatwouldbetheimpactonteachingpracticeswithtechnologyintheclassroom?Newpedagogicalmodel?Needtoadapttoanewstyleofteaching?Needtoworkcloselywithasecondteacher/facilitatortohelpstudentsusethetechnology?
L.3 Curriculum
L.3.1. Technology-CurriculumFit
Howdoesthetechnologyfitwithinthecurrentcurriculum?Isitanaturaladditiontosupportlearning?Wouldthetechnologysignificantlychangethecurrentcurriculum?Islocaladaptation/modificationofthetechnologypossible,andatwhatcost?
L.3.2. Technology’sRole
Isthetechnologydesignedtobeastand-alonetoolortoprovideextrasupportforthecontentthatisalreadybeingtaught?Istheschoolwillingandabletousethetechnologyasithasbeendesignedtobeused?(i.e.throwouttheoldcurriculumifnecessary)Doesthetechnologyempowerlearnerstocreateknowledge,orrequirerelianceondominant/mainstreamsourcesofknowledge.
CULTURE
C.1. CulturalRelevancy
C.1.1. CulturallyAppropriate
Isthetechnologyculturallyappropriate?Intermsof…Content?Structure?Agelevel?Implementationmodel(i.e.doesitengageallnecessarystake-holdersInteractionbetweenstudents/teachers/genders?
15
INFRASTRUCTURE
I.1. Equipment
I.1.1. EquipmentRequired
Whatequipmentisnecessaryforthetechnology?Besidesthemainequipment(i.e.computers/tablets/other),arethereotheraccessories(keyboards,projectors,etc.)thatwouldbenecessarytousethetechnology?Howmuchimpactwouldtheadditionalequipment(accessories)haveonlearning?(i.e.aretheyessential?)
I.1.2. EquipmentSourcing
Whoisprovidingthisequipment?Aretheydonatingtheequipmentoristhereacost?Whatisthecost?Isitpaidonetimeorasanannualfee?Willthedistributorcoverrepairsandmaintenanceorwillthatbecoveredattheschoollevel?Ifcoveredattheschoollevel,whatisanestimationofthecost?Isitpaidasinsurance(annually)orasproblemsarise(feeeachtimearepairneedstobemade)?Doesthetechnologyhelpreduceotheroperationalorcapitalcosts?
I.1.3. Storage
How,andwhere,willtheequipmentbestored?Isthespacesecure?Howwillitbeaccessed?Whowillhaveaccesstoit,andwhen?
I.1.4. Maintenance
Whatongoingmaintenancecanbeanticipated?Whowillconductandmanagethismaintenance?Arethereareenoughresourcesavailabletosupportthis?
I.2 Electricity
I.2.1. ElectricityRequirements
Willelectricitybenecessary?Ifso,isitjustforcharging?Ordoesthetechnologyalwaysneedtobepluggedin?Istheelectricityreliable?(Howoftendoesitgooutattheschoolandforhowlong?)
I.3. Internet
I.3.1. InternetRequirements
Doesthetechnologyrequireinternetaccess?Ifso,whatkindofspeedisnecessaryperdevice?
I.3.2. SchoolInternetResources
Doestheschoolhaveinternetaccess?
16
Ifso,where?Howreliableisit?(alwaysworksvs.sometimesworks)Istheresufficientbandwidthtosupportthetechnology?Howmanystudentscouldbeonlineatonceusingthedevice/program?
SUSTAINABILITY
SU.1. Funding
SU.1.1. TechnologyCosts
Whatarethecostsofthetechnology?Howmuchdoesthedeveloper/donoragencypay?Howmuchdoestheschoolpay?Arestudents/familiesresponsibleforanycostsassociatedwiththetechnology?Areallfamiliesabletoaffordthesecosts?Willcostbeadeterrenttoparticipation?Aretherewaystosupportstudentswhosefamiliescan’torwon’tpaythecost?
SU.1.2. TechnologyFunding
Whatdoesthebudgetfortechnologyattheschool-levellooklike?Isthereabudgetconstructedforthetechnology(capitalexpensevs.overhead)?Isthereasustainableplantocontinuefundingthetechnologyoveraperiodoftime?Arethecostspaidannuallyoronsomeothertimescheduleorrandomlyastheyoccur?
SU.1.3 TechnologyReturnonInvestment
Whatarethetrade-offs(intermsofresourceallocation),ifany,ofimplementingthetechnology?Iftheschoolpaysasignificantprice,whataretheycuttingtohavethatmoneyavailablefortechnology?
SU.2 Maintenance&Repairs
SU.2.1. TechnologyMaintenance&Support
Willfrequentmaintenanceandrepairsbeneeded?Whatarethelikelymaintenanceandrepairneeds?Canteachers/students/communitylearntomaintainequipment?Ifnot,issupport/repaireasilyaccessible.Aretherebackupsforwhenthetechnologyfails?
SU.2.2. SupportPlan
Arethereplansandfundingforthenecessarymaintenanceandrepairs?Insurancevs.payingcostsastheyarise?Dedicatedbudgetformaintenance?
17
SU.2.3. ImplementationSupport
Dotheteachersknowhowtoreportproblemsandaccessmaintenancefortheequipment?
SCALABILITY/MARKETIMPACT
SM.1. BroaderCommunityImpact
SM.1.1. KeyStakeholders
Arethereotherstakeholdersforthistechnology(outsideoftheteachers,students,school,anddevelopers)?
SM.1.2. CommunicationPlan
Whatisthecommunicationplanforinformingthestakeholdersaboutthetechnologyandsharingitmorebroadly?
SM.1.3 BestPracticeSharing
Howwillbestpracticesbesharedthroughoutthecommunityusingthetechnology?Doesthedeveloperoradonorhaveanappropriatenetworkorchanneltoshareinformation?
SM.2 Adoption&Scaling
SM.2.1. EconomicBenefits
Arethereeconomicbenefitstousingthistechnology?Tangibleskillsforstudents,teachers,orfacilitatorsthatwouldaidearningpotentialnoworlater?Arethesebasedoncontentknowledgeordigitalliteracy?
SM.2.2. IncentivesforAdoption
Doincentivestoencouragetechnologyadoptionexist?Ifso,whataretheyandhowinfluentialarethey?
SU.2.3. AdoptionMechanisms
Doinformationalstructurestolearnaboutbenefitsandscalethetechnologyexist?
Considerations for Future Application
Torepeatapointmadeinourinitialproblemstatement,thereareanumberofvariablesthatarenotalwaysfullyaccountedforbyvariousdecisionmakersintheeducationalsystem.Theframeworkwasintentionallyconstructedasaseriesofquestionssothatatechnologydeveloper,anadministratororateachermightuseittopromptreflectionastheycontemplatethecreationof,oradoptionofanewtechnologicalintervention.
Thisseconduseoftheframeworkasaself-administeredevaluationwouldlenditselfwelltoanonlineimplementation.Usersloggingintotheframeworkwouldbepromptedtoidentifytherolestheyplayinaneducationalsystem,andthatwouldinturninfluencethenatureandsequenceofthepromptquestionstowhichtheywouldbeexposed.Certainkeyquestionswouldbeusedastriggerstoleadtoadditionalresources,betheyresearchaboutaparticulardomain,orevaluationsofexistingproducts,orresourcesforsolvingparticularimplementationproblemsidentifiedthroughtheprocessofcompletingtheframework.
Thebenefitstotheuserfromaself-administeredframeworkwouldbetwo-fold.Thefirstbenefitwouldderivefromthereflectionitfostersinusers,alertingthemtoissuestheymaynothavefullyanticipatedinadoptinganintervention.Thisbenefitwouldnotbedependentonautomatedresponsesfromtheonlineframework.Nevertheless,thebranchingpathsofanautomatedframeworkwouldmakeitmoreefficienttouse.
Thesecondbenefitwouldresultfromthecomputationalenginebehindtheonlineframework.Weanticipatethatbasedonusers’responses,theframeworkcouldcategorizetheirparticularcontextasbelongingtospecific,familiarlevelsoftechnologicalpreparednessandcapability,andcouldthenpointuserstobothresearchandproductsthathadbeenpreviouslyidentifiedasbeingrelevantforthosecircumstances.Theframeworkcouldn’tnecessarilyevaluatetheappropriatenessofaproposedinterventionifthatinterventionhadnotpreviouslybeendocumentedwithintheframework’sknowledgebase.However,ifusersweretoaddinformationastohowtheyevaluatedagiveninterventionbasedontheframeworkandwithregardtotheircontext,thatinterventionmightbeincorporatedintotheknowledgebaseforthebenefitoffutureusers.
Theframeworkwouldbeadaptabletomultiplelanguages,andcouldbemaintainedbyanetworkofNGOsoracademicinstitutionsaroundtheworld,withonlyamodestinvestmentofeffort.Properlystructured,suchanetworkmightalsofunctionasacommunityofpractice,continuallybuildingouttheframeworkandassociatedresources.Intheprocessthisnetworkwouldbecreatingandsustaininganinteractiveknowledgebase,builtuponsharedfindingsasthetoolisusedwithincreasingfrequencythroughoutthedevelopingworld.
References Davies,R.S.(2011).Understandingtechnologyliteracy:Aframeworkforevaluatingeducational
19
technologyintegration.TechTrends,55(5),45-52.
Whetten,D.&Cameron,K.(2011).DevelopingManagementSkills.PrenticeHall.
Authors & Acknowledgements PrincipalInvestigatorsEricKlopfer&AnkurSarin
ReportAuthorsScotOsterweil,PrateekShah,StaceyAllen,JenniferGroff,SaiPriyaKodidala,andIlanaSchoenfeldThisprojectwouldnothavebeenpossiblewithoutthegenerousassistanceoftheindividualsandNGOswhograciouslyofferedtheirtimeandresourcesaswevisitedtheirsitesormetwiththeminconversation.Itisourhopethattheywillfindthisreportandtheresultingframeworkusefulintheirongoingwork,andthatitwilloffersomerecompenseforthetimetheysokindlygavetous.
WearealsogratefulfortheadviceofourcolleaguesontheCITEproject,andatUSAID.Theirinputwasessentialintheinitialformulationandongoingexecutionofthisproject,andwebenefittedgreatlyfromthemethodologiestheyhavedeveloped,andtheirextensiveexperienceperformingevaluationsinotherdomains.
Top Related