Exploring Learning Analytics · 2020-01-03 · advances, the next decade is when we see data...
35
1 Exploring Learning Analytics: The Role of Data Regulation and Privacy Aishwarya V. Nadgauda Thesis Advisor: Prof. Gad Allon Engineering Advisor: Dr. Sampath Kannan EAS 499: Senior Capstone Thesis University of Pennsylvania School of Engineering and Applied Science Department of Computer and Information Science May 1, 2019
Transcript of Exploring Learning Analytics · 2020-01-03 · advances, the next decade is when we see data...
1
ExploringLearningAnalytics:TheRoleofDataRegulationandPrivacy
AishwaryaV.Nadgauda
ThesisAdvisor:Prof.GadAllonEngineeringAdvisor:Dr.SampathKannan
EAS499:SeniorCapstoneThesisUniversityofPennsylvania
SchoolofEngineeringandAppliedScienceDepartmentofComputerandInformationScience
May1,2019
2
TableofContents
Introduction...........................................................................................................................................3ScopeofthePaper............................................................................................................................5
TechnologyofLearningAnalytics...................................................................................................6Classification.....................................................................................................................................6ClassificationinLearningAnalytics.........................................................................................................7
Clustering...........................................................................................................................................8ClusteringinLearningAnalytics...............................................................................................................8
SocialNetworkAnalysis................................................................................................................9SupplementalTechnologies.........................................................................................................9DataCollectionandAnalysis......................................................................................................10CurrentLimitationsofDataCollection................................................................................................11
CurrentApplications.........................................................................................................................12CurrentState...................................................................................................................................12PerformanceMetrics....................................................................................................................13Stakeholders...................................................................................................................................14Applications.....................................................................................................................................15MOOCs...............................................................................................................................................................15
CaseStudies.....................................................................................................................................17Coursera...........................................................................................................................................................18Duolingo...........................................................................................................................................................19Signal.................................................................................................................................................................21
DataPrivacy,RegulationandEthics.............................................................................................23RegulatoryLandscape..................................................................................................................23PrivacyConcerns...........................................................................................................................24ImplicationsofPrivacyConcerns.............................................................................................29DataasaCommodity....................................................................................................................30
Conclusion.............................................................................................................................................31
Endnotes................................................................................................................................................33
3
Introduction:
Thereisn’tanindustrytodaythatisn’tbeingturnedonitsheadwiththepotentialofdataanalytics.Itseemsthateverythingisonthebrinkofbeingdiscovered,andinsightswecangainbyanalyzingthevastamountsofdatathathasbeencollectedisallthat’sneededtoputusovertheedge.WeliveinadayandagewhereeverysingleadvertisementthatweseeinthelittlepopupbanneronourGoogleChromescreenistargetedandcustomized.
Therootsofdataanalyticsasweunderstandittodaybeganinthelate1960swhen
wesawforthefirsttimecomputersbecomeapartofthedecision-makingprocess.Thiswashappeningasaresultoftheadventofaswellasdatawarehouses.Therewerealsoconsiderablestridesinsoftwareandhardwaretechnologymadeduringthisperiodoftimethatenabledfasterprocessingofdataaswellascheaperstorageofdata.
Attheheartofdataanalyticsliesstatistics.Datahasbeencollectedandsifted
throughtogarnernewinsightsforlongbeforedataanalyticsasweunderstanditnowexisted.Relationaldatabasesplayedacriticalroleinbringingtogetherdatasets.Theexpansionintonon-relationaldatabaseshasallowedforgreaterflexibilityinconnectionsandcorrelationsthatcouldbediscoveredwithinthedata.
The1980sbroughtwithitthedatawarehouseswhichhadsignificantimplications
fordatastorageand,asaresultofthat,forthescopeofdataanalytics.Thiswasaremarkableleapforwardfromdatabeingstoredonharddiskdrives.Datawarehousesallowednotonlyformoredatastoragebutalsoforfasteraccesstodata.Asaresultoftheseadvances,thenextdecadeiswhenweseedataanalyticspickup.
Dataanalyticshaspermeatedeveryindustryandtheeducationspaceisno
exception.Inrecentyearswehaveseentechnologybecomecommonplaceintheclassroom.Notebooksandtextbooksarerapidlyreplacedbytablets,laptops,smartphonesetc.Weseetheintroductionofanewmediumthroughwhichweinteractwiththestudent.Thisinturnallowsforcollectionofmoredataaboutthesestudentsandhowtheyareinteractingwiththematerialtobecollected.
Atthehighestlevelweseethatstorageandcomputationalcapabilitieshaveseenan
increase.Thesecapabilitieshaveopenedupthepossibilityofdoingalotmorewiththedatathanwaspreviouslyimagined.Thecombinationofthiswiththefactthattherehasbeenanincreaseindatacollectionintheclassroomisthatweseetheemergenceofeducationaldatamining.Thegoalofeducationaldataminingistoworktowardsimprovingthequalityandaccessibilityofeducation.
Asthepotentialforthistooccurhasincreased,fundinggoingintotheeducational
dataminingspaceisincreasingaswell.TheHewlettandGatesfoundationallocatedalmost$25milliontohighereducationprojectsofwhich$3millionwasfocusedonprojectsintheanalyticsspace[1].Eventhegovernmenthasbeguntoallocatemoremoneyintothisspace.Itisseenashavingthecapabilitytopositivelyimpacttheeconomyatagloballevel.During
4
Obama’spresidency,theMyDatainitiativeemergedwhichworkedtowardsgivingstudentsdigitalcopiesoftheiracademicrecordwhichtheycouldthendownload[2].
Thekeyconsumersofeducationtechnologyareeitherstudentsorlearners.
Studentsareindividualsinatraditionalclassroomsettingwhoareinteractingwitheducationtechnologythroughanassessmentontheirtablet.Learnerscanbeanyoneconsumingmaterialfromaneducationtechnologyplatformoutsideofaclassroomsetting.Anexampleofalearnerwouldbeanindividualwhoisworkingasasoftwareengineerandusesacourseonlinetolearnanewcodinglanguagetohelpthemwithaprojectatwork.
Whileeducationaldataminingconsidersabroadlandscapeofwaysinwhich
technologyandeducationcancollaborate,wearehoninginonlearninganalyticsspecifically.Learninganalyticsisasubsetofeducationaldataminingfocusedongaininginsightsfromthedatatoincreasethechancesofastudent’ssuccess.
Ineducationaldatamining,traditionalanalysis(e.g.clustering)areperformedon
setsofdata.Learninganalyticsisdoingthis,butalsoconsideringadditionalfactorsontopofthis.Itincorporatesinstrumentssuchasvisualizationtoolsandsocialnetworkanalysis.Thepremiseoflearninganalyticsisgoingastepfurtherthantraditionaleducationdatamining.Thereisanaspectofitthatinvolvestryingoutthetheoriesandideasthatcomeaboutasaresultofthedataminingandseeinghoweffectivetheyare.
Giventhepotentialoflearninganalytics,weseeitproliferatedinmultiplewaysfromstudentsinkindergartenthroughexecutivesincompanies.Learninganalyticsisamultidisciplinaryfielddrawinguponexpertisefromindividualsacrossbothsectors(educationandtechnology).Successfulideasandproductsinthisindustryrequiresufficientknowledgeaboutarangeoftopicsfromwhattheclassroomsettinglooksliketotheregulationaroundstudentprivacytohowtobuildawebsite.Aboutadecadeago,weseethespacegettingmoreattentionataninternationallevel.Thefirstinternationalconferenceoneducationaldataminingconvenedin2008.Thiswasfollowedbythefirstlearninganalyticsandknowledgeconferencebeingheldin2011[3].
5
ScopeofthePaper: Overthecourseofthisthesis,wewillbeginbyunderstandingthemostcommontechnologicalapproachestolearninganalyticsandwherethedataisbeingsourcedfromsuchthatthisisfeasible.Fromthere,weconsiderthecurrentapplicationsthathavebeenderivedfromthistechnologyinindustryandthefactors(suchasstakeholders,businessmodels,marketforcesandregulation)thatareshapingthewayinwhichcompaniesinthisspacework.Asthedatathatisutilizedinanalyticsplaysaveryfundamentalroleinsettingthepathforthepotentialofthetechnology,theissueofdataprivacyiscritical.Finally,wewillconsiderhowregulationinthespaceinfluencesdataprivacystandardsandtheimplicationsofthatforthegrowthoflearninganalyticsapplicationsinthefield.Theargumentwillbemadethatregulationshouldallowforcollaborationinthefieldofdataanalyticsinordertocreatelearningenvironmentsthatbenefitthemoststudents.
6
TechnologyofLearningAnalyticsThefundamentalapproachestolearninganalyticsmirrorthoseinthelargerdata
analyticsspacethatspanamultitudeofindustries.
Figure1:LearningAnalyticsSource:IADLearning[4]
AscanbeseeninFigure1,therearethreecriticalcomponentstoanalytics:data,analysisandthenaction.Dataisreferringtothebaseinformationwhichundergoestheprocessofanalysiswhichallowstheretobeadditionalinsightsthatonecouldnotgetjustbylookingatthedata.Finally,thereistheactioncomponentwhichishowchangeaprocessorreacttotheinformationthatwehavegarnered.Thisstepiscriticaltothesuccessoflearninganalyticsasitisonlythroughclearactiontakenthatstudentscangetthebenefitsthatcomeaboutasaresultofthefirsttwosteps.
Theapproachestothesecondstep,analytics,thatareusedinlearninganalyticsareclassification,clusteringandsocialnetworkanalysis.Thesemethodologiesserveasapproachedtoprediction.Predictioniswhenthereisadatapointthatwewanttoinfer,andthisisdonebyconsideringanaggregateofotherdatapointsthathavebeenlookedto.Predictivemodelinghelpstofillinthegapsthatthedatamayhave.Tobuildoutapredictionmodel,themostcommonlyusedmethodologyortheanalysisstepisclassification.
Classification:
Classificationisessentiallyapredictivemodelwherewearegivingalabeltoeachobjectinthedatasetthatisbeingconsidered.Examplesofthisincludedecisiontrees,neuralnetworks,naiveBayesianclassification,supportvectormachinesandk-nearestneighborclassification.Thegoalofthisclassificationistobeabletotakeanunidentifiedobjectandputitinthecorrectgrouping.Aswebuildoutthesemodels,cross-validationplaysanimportantrole[3].Thisisintegralindeemingtheaccuracyofthemodelateachpointintime.Forexample,inthefinancialindustrythisiswhatisusedinloangroupings—classificationallowsonetobeabletoconsideraloanapplicationandidentifyitwiththelabellow,mediumorhighrisk.
7
Figure2:SampleDecisionTreeonStudents
Source:EdurekaAspecificexampleofclassification,adecisiontreemodel,isverycommonlyusedin
thelearninganalyticsspace.Thebigpictureideaofadecisiontreeisthattherearedecisionnodes,andbasedontheoutcomeatthatpoint,wemovetooneoftwootherdecisionnodes.FigureXdisplaysanexampleofhowalearnercouldbemodeledthroughadecisiontree.Thefactorsthatonewouldbeconsideringinthiscaseareincome,ageandcreditrating.
Thesplitsinthedecisiontreemodelarechosenbasedonwhatgivesyouthemost
significanceatanypointintime.Soyouareessentiallyprioritizingthesplitsthatgiveyouthemostinformationtothetopofthedecisiontree.Thisallowsyoutogivedifferentweightagetovariousattributes.Theendresultofadecisiontreemodelisaseriesofverysimplerulesthatcanbeusedinclassification.
ClassificationinLearningAnalytics:
Lookingathowwecantakethismodelofclassificationandtranslateitintothelearninganalyticsspace,thefirststepistounderstandwhatthelabelsofobjectswouldbe.Examplesofthisinthelearninganalyticsworldarelineitemssuchasstandardizedtestscoresorgrades.Thevalueofpredictioncanreallybegainedbybeingabletoidentifywhenitismostimpactfultointerveneinastudent’slearningprocess.Additionally,itcanprovideconstructiveinsightintowhereastudent’scurrentunderstandingofthemateriallies.Predictivemodelingthroughclassificationcanfurtherbeusedtogainaperspectiveonfactorsthatcontributetoacertainoutcome.Anexampleofthisislookingatwhatdetermineswhetherastudentgraduateshighschool.
8
Asubsetofclassificationthatisutilizedfrequentlyinthelearninganalyticssectorislatentknowledgeestimation.Atthecoreofthisapproachisthatwearetryingtomeasureastudent’sknowledge,whichisnotdirectlymeasurableandmustbeinferredbasedonastudent’sperformance[3].Decisionsthatadministratorsandeducatorswouldmakebasedonthisinformationwouldincludewhetherornottheindividualshouldprogressinthecurriculumorifateachershouldintervene.
Thetwoalgorithmsmostcommonlyemployedforlatentknowledgeestimationare
BayesNetsandBayesianKnowledgeTracing.Whenthe“knowledge”thatisbeingconsideredismorecomplex,weturntoBayesNets.InBayesNets,thedataisstructuredinanacyclicgraph,whereconditionaldependenciesarerepresentedbyedges.Ifitisacasewhere,bylookingatoneproblemoronegroupofproblems,wearetryingtoseeifthereisasingleskillthatthestudenthasbeenabletomaster,thenBayesianKnowledgeTracing[5]isused.BayesianKnowledgeTracinghasfouroutputparameters:
- p-init→Whatistheprobabilityofthestudenthavinganunderstandingoftheskillpriortobeingintroducedtothismaterial?
- p-transit→Whatistheprobabilityofthestudentshowingthattheyknowtheskillgiventhattheyhavethechancetoapplyit?
- p-slip→Whatistheprobabilityofthestudentperforminganerrorwhentheyareapplyingtheskill?
- p-guess→Whatistheprobabilityofthestudentperformingtheskillcurrentlybychance?
Clustering:
Thekeydifferencebetweenclusteringandclassificationisthatinclassificationonehasaninitialdatasetthatyouarestartingwith.Thisiscalledthetrainingsetwhichiscomprisedofobjectsthatarealreadylabeledintogroups.Withclustering,theselabelsdonotexistinadvance.Hereyougothroughtheprocessofgroupingobjectstogethersuchthatyouareminimizingthe“distance”betweenthem.Thedistancebeingconsideredhereissomemeasureofthedifferencesbetweenthegroupings.
Inclustering,therearedifferentlevelsandsizesatwhichtheclusteringcanoccur—
students,grades,schoolsstudentbehaviorsetc.Therearetwokeyapproachestoclustering:hierarchicalagglomerativeclustering(HAC)andnon-hierarchical.ThedifferencebetweenthetwoapproachesliesinthattheHACmodeloperatesundertheassumptionthatclusterscanclustertogether,whichnon-hierarchicalviewsclustersasentirelyseparateentitiesthatcannotbegroupedtogetherfurther[3].
ClusteringinLearningAnalytics: Withclustering,wehavethepotentialtobuildamodelthatisabletoreproducethewayinwhichstudentsareinteractingwithonlinecoursematerial.Thisisanecessarystepforthetechnologytothenbeabletopredicthowacertainstudentprofilewillrespondtomaterialdeliveredinaspecificmanner[15].
9
Withclustering,oneofthemostcomplexstepsisdeterminingthenumberofgroupsorcategoriesintowhichthedatashouldbebucketed.Theattributesuponwhichonechoosestogroupmustbesuchthatthereisnooverlapamongstthebuckets.Inmeasuringhow“good”agivenclusteringis,wecalculatethedistancebetweenthegroups.Essentiallywhatisbeingmaximizedishomogeneitywithinagroupandheterogeneitybetweengroups.SocialNetworkAnalysis:
Inasocialnetworkanalysis,youarelookingattheinformationthatyouaretakingawayfromlearninganalyticsasagraph.Thistendstobeusedmoreofteninaquantitativeanalysis.Thiscanbehelpfulasinthelearninganalyticsfield,asistrueacrossanalyticsinmultipleindustries,weoftenrunintotheissueof“dataright,butinformationpoor”[6].Itisdifficulttogofromrawdatatohelpfulinsightsthatcanactuallyguidethedecisionsthatarebeingmade.
Thisisparticularlyhelpfulinasubsetoflearninganalyticscalledsociallearning
analytics.Sociallearninganalyticsiscenteredaroundtheideathatthelearningenvironmentisnotasiloedexperiencebetweenastudentandinstructor.Researchinthisspacelooksintohowsocialdimensionsinvolvingone’snetworkinfluencetheprocessoflearning[20].Socialnetworkanalysisisacriticalcomponentofbeingabletorecognizethenetworksthatexistandhowtheyinteractwitheachother.SupplementalTechnologies:
Learninganalyticsinvolvestakingtheinsightsgainedfromdataanalysismethods,suchasclusteringandclassification,andconsideringhowtheycanbefedintoanapplicationthatallowsitsinsightstopositivelyimpactstudentsandlearners.
Onewaythroughwhichthisisdoneisinformationvisualizationwherewetake
statisticalinformation(suchashowmanytimesapagewasvisitedorwhatpercentageofmaterialanindividualhaslookedat)andpresentitinawaythatiseasierfortheindividualwhoisprocessingtheinformationtounderstandwhatitistheyarelookingat.Fromthisinformation,wecanmoreeffortlesslycommunicatetheanswerstoquestionssuchas“whatarethekeytakeaways?”.Itessentiallydrawsattentiontotherelevantinformationsuchashighsandlows.
Anadditionalwayinwhichlearninganalyticsutilizestheawarenessbroughtabout
throughsuchplatformsisbyservingasthefoundationforarecommendersystem.Thetwomaintypesofrecommendersystemsarecontentbasedrecommendationsandcollaborativefiltering[6].Contentbasedrecommendationsisbasedoffofwhattheuserhasalreadybeenexposedto.Collaborativefilteringisbasedonwhatotheruserswhohaveviewedsimilarthingstowhatthecurrentuserhasviewed.Optimally,acombinationofthetworecommendationsystemsareused.
10
Therecommendationsystemsserveasthefoundationforpersonalizedadaptivelearning.Adaptivityiswhencoursematerialcanbechanged,butitisbasedonpredeterminedguidelines.Adaptabilityisthepersonalizationofcoursematerial,andthisiswherelearninganalyticshasthepotentialtogrowalot[6].TherehasbeenalotoffocusinthelearninganalyticsspacearoundthisasisevidentthroughexamplessuchasIntelligentTutoringSystemandAdaptiveHypermediaSystems.
Astechnologicaladvancementshavetakenplaceinpersonalizedadaptivelearning,
wehaveseentheemergenceofthePersonalLearningEnvironment,whichisfocusedontheideaofadaptability.Themainideahereisthatthelearnershouldbetheoneguidingtheenvironmentinwhichtheyarelearning[6].Assuch,theyarenotrestrictedbytheenvironmentorinstructorinwhichtheyarelearning.Learninganalyticsiscriticalforthistohappen.Thisishowwehavethepotentialtogivematerialtothelearnerinanorderandmannerinwhichtheyaremostlikelytobesuccessfulinunderstandingandretainingtheinformationpresentedtothem.Animportantpartofthisisbeingabletotellwherethelearneriscurrentlyintheirunderstandingofcertainmaterial.DataCollectionandAnalysis:
Tovaryingdegrees,datahasbeenusedtodrivelearningandinteractionintheclassroompriortothefieldoflearninganalyticspickingupspeed.Mostexamplesofthisoccuratasmallscale,suchasconsideringaserverlogtoseewhichtestquestionsstudentshadthemosttroublewith.Onlyutilizingdataatthislevelofgranularityfailstoreapthebenefitsofamoreglobalandsystematicapproachtolearninganalytics.
Asmentionedbefore,thedatasourcesthatarepoolingintolearninganalyticsplaya
criticalroleindrivingthepotentialforgrowth.Thequantityandqualityofthedataisthebaselineforthecaliberoftheinsightsthatarederived.
Sourcesofdataincludecentralizededucationsystemsanddistributedlearning
environments.CentralizededucationsystemsincludelearningmanagementsystemssuchasBlackboardwhichservesasanonlineplatformtosupplementlearning.Throughthisinformationallogsofstudentactivitiescanbecollected.Questionssuchas“Whatmaterialaretheyaccessing?”or“Howhavetheydoneontests?”canbeanswered.Adistributedlearningenvironmentisoneinwhichtheinstructorandstudentscanbeindifferentlocationandthecurriculumisstilldelivered.
Thedatautilizedinanalyticsprimarilycomefromstudentinformationsystemsand
learningmanagementsystems.StudentinformationsystemssuchasPowerSchoolofferawayforschoolstomonitornotonlystudentperformancebutalsooffersadditionalfeaturessuchasscheduling[21].LearningmanagementsystemssuchasLearnUponserveasaplatformfortheircustomerstocreateanddelivercontenttotheirstudents,whetherthosebeexecutiveorschoolchildren.
Throughastudentinformationsystem,informationcanbecollectedonthe
student’sdemographics,grades,socioeconomicstandingetc.Additionally,throughan
11
integratedlibrarysystem,wecangetdatapointsaroundtheresourcesanindividualisusing.Electronictextbooksallowsforpotentialdatacollectionaroundthewayinwhichastudentisconsumingcertainmaterialavailabletothem.Fundamentally,theinfluxoftechnologyinaclassroomsettingisdrivingthefeasibilityofthisdatacollectionandalteringthewayinwhichweareapproachingdeliveringinformationtostudents.
Tofurtherourcapacitytobuildoutpersonalizationinthisspace,morediversedata
setsmustbebuiltout.Onepotentialsourcesofdatainvolvinginteractioninaphysicallearningspacewouldbethroughwearabledevicesthatcollectdata.Asitisactuallyusedrightnow,majorityoflearninganalyticsreliesondatathatisautomaticallycaptured.Whenweconsiderprojectsintheanalyticsspacethatareofmuchlargermagnitudeinindustriessuchashealthcareandretail,wherelearninganalyticsaspirestotobe,thereisconsiderableamountofmanualeffortintermsofdatacollectionthatgoesintoit.Particularlywithhealthcare,asignificantportionofthedataaroundahuman’sgeneralhealthiscollectedmanuallybyadoctor.Thisdataisthenconglomeratedtostudy.Astheapplicationsofthespaceexpand,themodelswouldideallyaccountforamultitudeofdatainputs,includingbothautomatedandhumansources.
Thispointgainsparticularimportanceasweturntoknowledgedomainmodeling.A
domainmodelis“definingcausalrelationshipswithindata,basedonadeepunderstandingofadomain,allowingustoanticipateconsequencesandcausesofactionsandtolearnnewcausalrelationshipshiddenindata”[22].Withknowledgedomainmodeling,wehoneinonthepersonalizationaspectsoftheapplicationsinlearninganalytics[14].Indiscussingknowledgedomainmodeling,thefirststepisrecognizingthatamultidisciplinaryapproachiscrucialtoachievingsuccessinthisregard.Forexample,curriculumdevelopersshoulddrawnuponthistocapturewhatcontenteachtypeofstudentwouldbemostinclinedtobenefitfrom.CurrentLimitationsofDataCollection:
Asmentionedabove,themainsourcesofdatacurrentlycomefromlearningmanagementsystemsandstudentinformationsystems.Fromthisdata,theconclusionsthatareoftendevelopedareasfollows:thestudentwhologsintoalearningmanagementsystemmorefrequentlyismorelikelytohaveahigherscoreontheassessmentspresentedinclass.Whilethereisvaluetothisinsightinitself,ifweviewthegoaloflearninganalyticsasstudyingdatainordertocreatealearningenvironmentthatismorebeneficial,thesetypesofinsightsarenotconducivetothat.Simplytellingalowerperformingstudenttologintothelearningmanagementsystemmorefrequentlywouldnotaidthatstudentinbecomingmoresuccessful.Inordertomovetowardsmoreactionableinsights,thesourcesfromwhichwecollectdatamustbeexpandedtoincludemoreinformationonhowthestudentisactuallylearning.
12
CurrentApplications: Successinthefieldoflearninganalyticsrequiresamulti-disciplinaryapproach.Thestridesmadefromatechnologicalperspectivearejustoneoftheprongsoffurtherdevelopmentinlearninganalytics.Thelandscapeoftheindustryplaysacriticalroleindictatinghowrapidlygrowthcanoccurandwhereinthepainpointslie.Wewilllookatwherethefieldcurrentlystands,considerwhothethecorestakeholdersinlearninganalyticsareandhowtheirgoalsintersectandwheretheyareatodds.Thenconsideringthemainsectsofapplicationsactuallyinindustry,wewillexaminehowcompanieshaveleveragedtheseapplicationstogeneraterevenueandwherethereispotentialforfuturegrowth.
Figure3:ProposedLearningAnalyticsFramework
Source:IEEE[23]CurrentState:
Learninganalyticsencompassesarangeoftechniqueswhichdriveapplicationsthatalterthewayinwhichourlearningenvironmentisinformed.Asitcurrentlystands,thelearninganalyticsspaceisnotmeetingitspotentialintermsofinsightsthatitcanofferitsconsumers.Itisevidentthroughalackofoperationalefficiencythattheeducationalsectorisfallingbehindotherindustriesbynotbuildingoutaninfrastructurethatissupportiveoflearninganalytics.Oneexampleofthisisthelackofdashboardsoranalyticaltoolsatthedisposalofstudentsandeducatorsalike.Rightnowweseethecenteroftheresearchinthisspaceisprimarilyfocusedontheadaptionofcontent.Whatislackingisaclearsetofactionablestepsforbothstudentsandteachersonwhatshouldbedonetogetthebestpossibleresults.Thatwouldlooklikeanincreasedfocusonfeedbackandreflectionfor
13
boththestudentandtheinstructoraswellasallowingforeffectiveinterventionfromtheteachertothestudent.
Weseethisbeginningtoshiftinrecentyearswithreneweddiscussiononhowwe
canshifttowardsmoredatadrivendiscussionsintheeducationsector.However,aconsiderablehindrancetothisremainsthatthereisdebateregardingtheextenttowhichdatacollectionshouldoccurandtheramificationsofsharingthedata.
Acriticaldriverinthegrowthofthefieldhasbeentheincreaseofcurriculum
availableanddeliveredonlinetostudents.Oneexampleofthisisfullcoursesthatareavailableonline.Thefirstsurgeofinvolvementinthespacefromastudentperspectivecanbeseenin2012,whentherewereover6.7millionstudentsenrolledinatleastoneonlinecourse[1].Asof2016,thenumberofstudentspursuinghighereducationwhowereenrolledinanonlinecoursewasoveraquarter[12].
Inadditiontoclassesbeingofferedandmadeavailableonline,wealsohavethatstudentassessmentsandassignmentsarebeingmovedtoadigitalplatform.Thisisacriticalsteptowardsfosteringanenvironmentthatnotonlyallowsbutencouragesthepersonalizationofeducationmadefeasiblethroughlearninganalytics. Thisisoccurringinbothelementaryschoolandhighereducationsettings.Foryoungerchildren,asystemsuchasi-Readyprovidesaplatformfordifferentiatedlearning.Whati-Readisabletodoisthatit“integratespowerfulassessmentsandrichinsightswitheffectiveandengaginginstructioninreadingandmathematicstoaddressstudents’individualneeds”[24].IninstitutionsofhighereducationscompaniessuchasCanvasprovideanenvironmentforstudentstointeractwithmaterialfromlecturerecordingstoquizzesthroughasingleinterface.Onotherothersideofthis,allthisdatacanbecollectedinoneplace.PerformanceMetrics:
Whenlookingattheimpactoflearninganalytics,therearecertainmetricsthatareconsiderednoteworthywithintheindustry.Administratorsoftenlooktostudentretentionandgraduationratenumberstoseetheimpactofachangeintheexistingorganizationorprocess.Thesearenumbersthataretypicallyusedasbenchmarksintheeducationspaceand,assuch,havecarriedoverintotheeducationtechnologyspaceaswell.Thisisparticularlytruewhenthefocusisonabigpictureviewoftheorganizationandcanbeusedtocompareacrossgeographicregions.
Withlearninganalytics,weadditionallyhavetheconsiderationoflearners.Herewe
seethatanotherimportantmetricisintroduced,onethatismoreapplicableatanindividuallevel.Thisistheextenttowhichanindividualstudentisunderstandingthematerialandengagingwithit.Whilethisistheoverallgoalthatisbeingworkedtowards,themannerinwhichitismeasuredorstudiedvariesalotdependingontheapplication.Examplesofthisincludethespeedandaccuracywithwhichanassignmentiscompleted,
14
thefrequencyofengagingwiththematerialandtheextenttowhichonecontributestodiscussionsonline.
Lookingatthesamesituationfromtheinstructorsperspective,wehaveaction
research.Thisiswhere“throughiterativecyclesofaction,perceptionandevaluation,teachingcanregularlybematchedandadjustedtoalllearners’needs”[6].Thegoalofactionresearchistohavestepsyoucantaketohaveeffectivepractices[1].Whatishappeningisthatthereisanactivechangemadeinanorganization(oftenaclassroomintheeducationtechnologyspace),andtheeffectsofthatchangeareobserved.Thisintheoryprovidesinsightintoeffectiveversusnoteffectiveteachingpractices.Theseinsightscanthenbeusedtoshapewhatistaughtintheclassroomandthemannerinwhichitistaught.Stakeholders:
Therearealotofdifferentindividualswhoareconsideredtobestakeholdersinthelearninganalyticsspaceandtheyallhavedifferentobjectivesthattheywantoutoflearninganalytics.Theirdifferinggoalsandperspectives,whennotproperlyaddressed,createhindrancestotheadoptionandsupportoflearninganalyticsbykeyindividuals.Belowaresomeofthesekeygroupsaswellastheirgoalswithlearninganalytics[6].
● TeachersTheywanttounderstandhowtheycanbestcommunicatetheirmaterialtothestudents.
● StudentsTheyarefocusedonretaininginformationandimprovingtheirgrades.
● AdministrationWeseethatadministratorsoftenmakedecisionswithstudentretention,graduationratesandidentifyingstudentswhoareatriskattheforefrontoftheirmind.
Iflearninganalyticsisusedtoaddresstheseconcerns,thenextconsiderationistowhatextentexistingprocesseswouldhavetobealteredforthesetobeaddressed?Woulditsimplyrequireanadditionallayerofworkontopofwhatisbeingdone?Ordowehavetoalterourfundamentalapproachinordertobeabletoreapthefullbenefitsthatlearninganalyticshastooffer?
Aslearninganalyticsopensthedoortonewideas,therearepotentialclashesthatcouldoccuramongststakeholdersintermsofwhatfactorsshouldbeconsideredinmakingtheoptimaldecision.Anexampleofthiswouldbeiftheadministrationwantedtoknowwhichteacherisusingthemosteffectiveteachingmethod.Theteacherscouldfeelasthoughtheyareconstantlybeingassessedthroughlearninganalyticsandfeelthattheydon’thavecompletefreedomandcontrolintheirclassroom.Anotherpotentialclashofconflictsisifstudentdataisusedtobetterinformteacherdecisionmaking,butdoessoinawaythatisharmfultothestudent.
15
ApplicationsofLearningAnalyticsObjectives:
Therearearangeofsituationsinwhichlearninganalyticscanbeusedandtheireffectsvaryacrossthestakeholders.Firstly,itcanbeusedtomonitorandanalyzethestudent’sprogress.Secondly,learninganalyticsprovidesawaytoeffectivelypredictandintervene.Itgivesyoutheabilitytoseehowthestudentmightbedoinginthefutureandwhatcanbedonetokeepthemontrackor,iftheyareheadeddownapaththatisnotpositive,whatcanbedonetostopthatfromhappening.Anotherobjectiveistutoringandmentoringwhichismorespecifictoacertaintopic.Learninganalyticscanbeemployedforassessmentandfeedbackaswellasadaptation.Advancinginthisdirection,learninganalyticsalsoservesasanintegralpartofpersonalizationandrecommendation.Thisistheobjectivethatismostfocusedonthelearner.Inpersonalizationandrecommendationthereisabackandforthbetweentheknowledgepushversustheknowledgepull.Whoisleadingthewaythatthelearningishappening—theinstructororthestudent?Anotherkeyobjectiveoflearninganalyticsthatattimesisnotasevidentintheapplicationsisreflection.
Weshoulddrawupontheperformancemetricsusedintheeducationtechnology
spacetoevaluatehowagivenlearninganalyticsplatformperformswiththeobjectiveslistedabove.Traditionally,intheeducationspacewehavethatgrades,graduationratesandstudentretentionaretheprimaryfactorsconsideredinperforminganevaluation.Withtheintroductionoflearninganalytics,wecanexpandthistoconsiderotherfacetsofthespacesuchastherateatwhichastudentismakingprogressandhowimpactfulashiftincurriculumorastudent’sstudyhabitsareontheirabilitytounderstandmaterialasjudgedbasedonassessmentandfeedback.Learninganalyticsallowsustoconsiderthemoreindividuallevelperformancemetricsassociatedwitheducation.
MOOCs: MassiveOpenOnlineCoursesorMOOCsserveasaspringboardinthelearninganalyticsspace.MOOCshavegrowninpopularityinrecentyears.InunderstandingtheroletheyplayinlearninganalyticsitisimportanttounderstandwhatMOOCSare.Massiverelatestothenumberofindividualsenrolledinthecourses.Openindicatesthehighlevelofaccessibilityofthecourses.Onlineshowsthattheclassesareontheinternet.Coursesrefertothefactthatmaterialisstructuredinamannerthatismeanttobetaughttosomeoneelse.TheadvantagesofMOOCsarethattheycanbringeducationatascaleanddepththatpreviouslycouldnotbecomprehendedatitslowcost. MOOCsprovideanopportunityforcompaniesinthelearninganalyticsspacetotakearichdatasetandbegintoscratchthesurfaceoftheinsightslearninganalyticshasthepotentialtoprovide.Thisinturnallowsustoalterthewayinwhichthecurriculumisdeliveredtothelearner.
OneofthebiggestissuesfacedbyMOOCSistheirretentionrate,andconsideringwhatcanbedonetoencourageindividualstocompleteacoursetheyhavestartedisaone
16
oftheproblemslearninganalyticsislookingtosolve.Justasinschools,therearestudentsinMOOCsdrivenbybothintrinsicandextrinsicmotivation.Studentsareintrinsicallymotivatedwhentheysignupforaclassbecauseitcoversasubjectmattertheyareinterestedin.Ontheotherhand,asituationinvolvingextrinsicmotivationinaMOOCwouldbewhenanindividualhasenrolledinaclassbecausetheydesirethecertificationtheygetattheconclusionofthecourse[16].
Inanefforttogarnerthemostapplicableandgeneralizableinsightsaboutstudents
thatarepartakinginMOOCs,thelearninganalyticsclusteringapproachisoftendoneinthisspacebythetypeofstudentrelatingtotheirlevelofengagement.Onewayinwhichthathasbeendoneistheportionofthecurriculumthatthestudenthasworkedtheirwaythrough.KhalilandEbnerconsideredthefollowingcharacteristicsofstudentsinastudytheydidin2016:“completers”whofinishedtheentirelyofthecourse;“auditors”whowatchedthevideosbutdidnotdosomeoftheassignments;“disengagingstudents”wholefttheclassafterthefirstthirdofit;“sampling”whodidnotcontinuewiththematerialafterthefirsttwoweeks[16].Anotherwayinwhichstudentscanbedividedistheextenttowhichtheycompletedassignments.
InastudythatlookedtohowstudentsweremotivatedinMOOCsthatfocusedprimarilyatdifferentiatingbetweenintrinsicandextrinsicmotivation,theresultshadstrongindicationsforfutureresearchonhowtoreducedropoutratesinMOOCs.ThefindingsrevealedthatthefourmaingroupingsofstudentsmirroredthoseofCryer’sschemeofEltonwhichlookedatacommitmentofanindividualinaclassroom[16].
Figure4:Cryer’sschemeofElton
Source:ClusteringpatternsofengagementinMassiveOpenOnlineCourses[16]
17
Figure5:ClusteringanalysisofstudentsinvolvedinaMOOC
Source:ClusteringpatternsofengagementinMassiveOpenOnlineCourses[16]
ObservingtheclusteringthatcameaboutwhenconsideringstudentsthatweretakingaMOOC,theresultsoftheclustermirrorCryer’sschemeofElton.ThisdrawsastronglinkbetweentheprofilesofstudentsintheclassroomandthoseusingMOOCstobeinsomecorewaysanalogous.Thisgivesusinsightsonhowtoshiftstudentsintothetypesofstudentsthatgainmorefromanonlinecourse.Theresultsofthestudypointedtowardsincreasingintrinsicfactorsasopposedtoextrinsicfactorsasawaytoinvolvestudents[16].CaseStudies:
Aswehavediscussedthespaceoflearninganalytics,wehavedevelopedanunderstandingofthepotentialthatthistechnologyholds.Inrecentyears,therehasbeenarapidincreaseinthenumberofeducationtechnologyfirmsandthefieldhasbecomeincreasinglysaturated.Since2011wehaveseenacontinuousgrowthinmoneyputintoeducationtechnologycompanies,andfortheUnitedStatesthatnumberreachedapeakin2018hitting$1.45billion[25].Therearealotofdifferenttypesofcompaniesinthisspace,manyofthemaddressingdifferentnichemarketswithinthespace.
Asisoftenthecase,thereisagapbetweenthepotentialofthetechnologythatis
consideredinapurelytheoreticalandresearchcontextasopposedtotheactualtechnologythatacompanybuildsabusinessmodelon.Herewewillexplorethreecompanies—Coursera,DuolingoandSignal.Eachofthesefirmsisattheforefrontoftheproblemtheyconcentrateonsolving,andassuchgiveusanunderstandingofthebreadthofthefieldoflearninganalyticsasitcurrentlystands.
18
Coursera
CourseraisanexampleofplatformformassiveopenonlinecoursesorMOOCs.Theplatformsoffersonlinecoursesthecoverarangeoftopicsfrom“ManagingInnovationandDesignThinking”to“ChildNutritionandCooking.”Courserapartnerscloselywithuniversitiestohavehighlyqualifiedinstructorscuratethematerialandteachthecourses.Thecompanywasfoundedin2012andhasgrowntomorethan30millionusersin2018[26].
Figure 6: Example Curriculum for Course Figure 7: Sample Courses Offered Source: Coursera Source: Coursera
Thevideosfortheclassesbeingtaughtareavailableforfree.Thereareafew
streamsthroughwhichCourserageneratesrevenue.Tobeginwith,togetacertificateshowingthatyouhavecompletedthematerialassociatedwiththeclass,theconsumerpaysafee.Thereisanadditionalfeeforgradesandassessments,whichgoestoshowthevaliditywithwhichyouhaveengagedwiththematerialofagivencourse.Mostrecentlyin2016,theyhaveaddedamonthlysubscriptionmodelthatallowsonetogetaspecialization[26].
OnenuancetoconsideristhataMOOCsuchasCourseraofferscoursesinarangeof
differentsubjectmatters.Assuch,onemustkeepinmindhowengagementandanalyticsofengagementmightvaryfromsayEnglishtomathematics.
Courseraplaysauniqueroleinthelargerfieldofdataanalytics,asitisasourceof
oneofthelargestpublicdatasetsrelatingtostudentsandlearnersstudyingacurriculumfrommaterialonline.WecanlookatdatafromCourseraMOOCstounderstandhowpeopleareengagingwithonlinecoursework.ThedatathatisbeingconsideredwhenstudyinglearnerbehavioronCourseraincludesinformationsuchaswheretheyclick,whattheyview,howlongtheyareonapageetc.Ithasbeenfoundthatthereisanexponentialdecay
19
intheindividualsactivelyinacoursethroughoutthefirst6/10ofthedurationofthecourse.
Asthisdatasetisfurtherstudied,wewanttobeabletoanswerthefollowing
questions:• Howdostudentsinteractwiththeonlinematerial?• Canwegaugefromthedatawecancollectwhenastudentisenrolledina
MOOCwhetherornottheyknowthematerial?ThishasseriousimplicationsforthevalidityofcertificationsthatMOOCShavestartedtogive.Inordertobetterunderstandthis,theprocessthroughwhichstudentsengagewiththematerialtheyarepresentedinaMOOCisstudied.WhathasbeendiscoveredthusfarinregardstotheprocessesthatallowstudentstobesuccessfulinaMOOCisthatsuccessfulstudentswatchvideosintheorderrecommendedbytheinstructor. WewanttobeabletoconsiderlessofthedemographicinformationofthestudentandmorewhattheirhabitsarewhentheyarelearningthroughaMOOCandusethistoserveasthebaselineforhowtheyarelearning.Thisallowsforlessbiasedaswellasmoregeneralizableconclusions.Inordertobeabletodothis,processminingisused. Processmininginvolvesthreesteps.Thefirstpartistocreateamodel,whichiscalleddiscovery.Thesecondpartisconformance,whichiswhenyoucomparethemodelthathasbeendiscoveredtothedataanddeterminewhereinthedeviationslie.Thethirdpartofprocessmininginvolvedenhancement,whichiswherethemodelisalteredbasedonwhatwaspotentiallyuncoveredduringconformance[3]. AstudydonebuildingaprocessmodelonCourserausingitsdatafoundthatthefollowingtwofactorswereintegralinunderstandinglearnerbehaviorontheseMOOCS:[17]
• Watchstatus• Viewinghabit
Duolingo DuolingoisalanguagelearningplatformthatteachesEnglishspeakers11differentlanguages.Thecompanyhasawebsiteaswellasanappthathasover300millionusers[].Theirbusinessmodelissuchthattheydon’tchargeconsumerswhoutilizetheapplication.Insteadtheyderivetheirrevenueprimarilythroughadvertisements.Ifauserwishestoutilizetheapplicationwithoutadvertisements,theycanpayasubscriptionfeeforanadfreeservice.
20
Figure8:IntroductiontoalanguagecommunityinDuolingo
Source:Duolingo
Let’sconsidertheimpactofDuolingothroughastudythatexploredtheeffectivenessofDuolingoinlearningSpanishasmeasuredbyascoreonacollegeSpanishplacementexam.Inthestudy,learnerstooktheplacementtesttwice,oncebeforeandonceaftertheyworkedwithDuolingo.Overall,theconclusionwasdrawnthatDuolingowassuccessfullyabletoincreasethelearners’understandingofSpanish[29].
Figure9:InterfacetolearnSpanishinDuolingo
Source:Duolingo ThefactorsthatplayedintothesuccessofthesubjectsofthestudyrevealedaninverserelationshipbetweenhowmuchthelearnerinitiallyknewaboutthelanguageandhowmuchgainedfromDuolingo.WhenstudyingtheimpactofDuolingointhissetting,onecannotjustlookattheincreaseinthescoreinthetwotimestheexamwastaken.TheamountoftimethelearnerspentonDuolingomustalsobetakenintoaccount.So,inthese
21
situationsthemetricofpointsgainedperhourofstudyisacommonmetricusedtomeasurethesuccessoftheonlinelearning.Theoverallsentimentoftheusersonthetechnologywasthattheyweresatisfiedwithitanddidn’thaveadifficulttimelearningthesubjectmatterthroughanontraditionalavenue.Thisheldtrueacrossagegroups,levelsofeducationandracialbackgrounds. Duolingoreliesheavilyonadaptivelearning.ThebigpictureideaofadaptivelearningthatDuolingoassumesisthatthematerialthelearnerispresentedwithismodifiedbasedontheirpastperformance.Thisisdoneusingadecisiontreemodel. Theidealistgoalwiththisisapersonalizededucationsystem,whichisnotpossibleintheexistingclassroomsetting.SoyouhaveaspacewherecompanieslikeDuolingocanaddalotofvalue. Butanimportantfactortoconsiderwhenteachingmaterialthroughadecisiontreemodelisthatlearningalanguagethroughadaptivelearninghassignificantdifferencesfromlearningasubjectsuchasmathematics.Adaptivelearningisbasedoffoftwokeyassumptions.Thefirstassumptionisthatthematerialisbeinglearntinalinearfashion,and,assuch,thatitiscumulative.Thesecondassumptionisthatthematerialcanbebrokendownintobuildingblocksthatallowonetobetaughtthematerialinpieces[3].Althoughthismakessenseforasubjectsuchasmathematics,thisdoesn’tnecessarilytranslateovertolanguagesandassuchitbecomessignificantlymorecomplextoapplyanadaptivelearningmodelinteachingsomeoneanewlanguage.Alargepartofthisishowtheextenttowhichone“understands”orhas“mastery”ofasubjectisevaluated.Signal SignalwasfirstpilotedinPurdueUniversityin2006andstartedbeingusedinamultitudeofothersettingsin2009.Itisaplatformthatmonitorsastudent’sprogressandalertstheminrealtimeifthereisanareainwhichtheyshouldbedoingadditionalwork.Iftheyreceivea“signal”indicatingthattheyarenotontack,thentheirprofessorcanprovidetheminformationonwhattheycandotogetbackontrack[13].
22
Figure 10: Sample interface of Signals application
Source: Signals, Purdue University Inlookingathowthesystemworks,weseethatitisnotjusttakingintoaccounttheactualgradethestudenthas.Informationisalsousedonhowthestudenthasengagedwiththematerialtheyhavebeenlookingat.Itessentiallyusesthisinformationaswellastheiractualscoresandthencomparesittopreviousstudentstogaugehowthisstudentisperforming. ThekeypiecesofinformationthatgointothealgorithmthatSignalssystemusesarelistedbelow:
- “Performance:basedonpointsearnedonthecoursesofar- Effort:interactionwiththeVLEcomparedwithotherstudents- Prioracademichistory:includinghighschoolGPAandstandardizedtestscores- Studentcharacteristics:e.g.ageorcreditsattempted”[29]
TheimpactthatSignalhasonstudentsisthatithelpsthemrealizetheyneedto
improveandwhattheyneedtodotoimprovebeforeitistoolateforthemtocatchup.Ithelpstoovercometheissueofnotbeingabletohelpstudentsthatfallinthemiddleoftheclass(asopposedtofocusingonthetoporbottomtierstudents).TheresultsofutilizingSignalshowedthattherewere10%moreA’sandB’sgiveninclassesthathadSignal.WhenstudentswerenolongerintheSignalprogram,thestudentsthathadusedtheplatformwerestillmorelikelytogethelpoutsideofclassandmakesuretheywerekeepingup.
23
DataPrivacy,Regulation,andEthics Untilnowwehaveexaminedwhatthelearninganalyticsspacelookslikefromtheperspectiveofwhatthepotentialofthetechnologyisandhowitiscurrentlybeingusedinindustry.Anadditionaldimensiontoassessistherolethatdataprivacyandregulationplays.Themainsectionsofthisarepolicy,corporatesocialresponsibility,andpublicopinion. Thisisoneofthelargestimplicationsoftheincreaseoftechnologyintheclassrooms.Withthisintroductionoftechnologyintotheclassroomspace,thereissignificantlymoreinformationcollectedaboutstudents.Now,sincethisinformationiselectronic,itprovidestheincrediblebenefitofresearchersandtechnologistsbeingabletopoolthedataforeducationaldatamining.Ontheflipside,thereisthepotentialforthedatabeingusedwithmalintentaswellasitspreadingmoreeasilywithouttheindividualwhocreatedthedatahavingcontroloverit.Thissparksadeepcontroversyabouttheextenttowhichthisdatashouldbecollectedtobeginwith.RegulatoryLandscape: Theeducationtechnologyspacecanbemoldedfromtheinsightsoflearninganalyticstocreateanenvironmentwhereeachindividualhastheopportunitytoreachtheirmaximumpotential.Thisiswhattheeducatorshavestriventowardsforhundredsofyears.Asignificanthurdleforthistobecomearealityistheregulationthatresultsinwallsbeingbuiltaroundstudentdatathatcanmakeitparticularlydifficulttoparseforinsights.Thereislegitimatecausefortheirtobeconcernsaroundwhohasaccesstowhatdataaboutastudent,particularlyconsideringtherolethatageplayshere.Understandably,thereisalotofapprehensionthatchildrencannotsufficientlyperceiverisksthatmightbeassociatedwithdecisionsthattheyaremakingand,assuch,couldbecoercedintodoingsomethingtheyaren’tnecessarilycomfortablewith. TwomainexistinglawsthatareconnectedtotherightsthatminorshaveinregardstoprivacyintheeducationandtechnologyspacearetheFederalEducationRightsandPrivacyAct(FERPA)andtheChildren’sOnlinePrivacyProtectionAct(COPPA).
• FERPA:TheFederalEducationRightsandPrivacyActislimitedto
organizationsthatreceivefederalfundinginsomecapacity.IfanorganizationviolatesFERPA,thereisafinancialpenaltythattheythenmustpay.Theactrelatestoeducationalrecordswhichencompassthefollowingbucketsofinformation:“AccordingtoFERPA,educationrecordscontaininformationonstudentbackground,academicperformance,grades,standardizedtestresults,psychologicalevaluations,disabilityreports,andanecdotalremarksfromteachersorschoolauthoritiesregardingacademicperformanceorstudentbehavior”(FERPA,1974,20U.S.C§1232g(a)(1)(D)(3)[18].
24
Ifthisinformationistobereleasedbytheschool,thentheyhavetogetwrittenpermission.Ifthestudentisundertheageof18,thenthereleasemustcomefromaparentorguardianovertheageof18.Ifthestudentisovertheageof18,thentheycanprovidethewrittenconsentthemselves.Whenthisinformationispassedalongtoanotherorganization,incompliancewithFERPA,itisundertheunderstandingthattheinformationsharedwillnotgobeyondthat.
ThereisagrayareaanddebatearoundafewareasinFERPA.Do
emailsthatarehostedbytheschool’scloudservicesgetprotectionunderFERPA?Whatifdataisbeingcollectedtobeutilizedforsomesortofadditionalservice?
• COPPA:
o TheChildren’sOnlinePrivacyProtectionActappliestoyouthundertheageof13.UnderCOPPA,onlineproviderswhoarecollectingdatafromchildrenundertheageof13mustgarnerparentalconsentthattheycancollectthedataaboutthechild.Itisimportanttonotethatwiththisact,parentsalsohavetherightto,atanypoint,retracttherightfortheprovidertostorethisinformation.Iftheydecidetodoso,thananypreviouslycollecteddatamustalsobedeleted[18]..
o COPPAwasmostrecentlyupdatedtoincludethefollowingitemsasalsobeingconsideredpersonalinformationforwhichorganizationsmustreceiveconsent:cookies,geolocation,photographs,videosandaudiorecordings.Thisisalistthathasbeenexpandedandupdatedovertheyearsasthenumberofwaysinwhichdataaboutindividualsofthisagecanbecollectedhasgrown.
PrivacyConcerns: Aswithanynewtechnologythatemerges,wemustconsiderthelargerimplicationsthatitposesforsociety.Withlearninganalytics,thereareaplethoraofconcernsaroundthepotentialmisuseofdatathatisbeingcollectedaswellastensionaroundwhohasownershipofthedataandwhoisprivytotheinformationthatisbeingcollectedaroundthis.
Thereisageneralconsensusthattherearebenefitstothepersonalizationandadaptabilitythattheplatformsthathavecomeaboutasaresultoflearninganalyticshavetooffer.Buttobeabletocollectthedatatoperformtheanalysis,companiesfindthemselvespulledinmultipledifferentdirectionsaddressingthequestionsmentionedabove.
Theoverarchingconcernsregardingprivacyofstudentdatainlearninganalyticscanbebrokendownintothefollowingfivebuckets[18]:
25
• IssueOne:Thenetworkofpeopleandorganizationsthathaveaccesstothe
information.Ifweallowacompanyoraneducationalorganizationtocollectdataaboutastudent,doesthismakeitmorelikelythatthedatawillbepassedalongtoanotherentity?
HerewefacetheissueofifapersongivespermissiontoOrganizationAtohavetheirdata,whathappensifOrganizationAthenpassesitalongtoOrganizationB?Thereareamultitudeofcomplicatedsituationsinthelearninganalyticsspacethatcanariseasaresultofthis.Forexample,saydataisbeingcollectedonalearningmanagementsystemonhowmanyvideosastudentiswatchingandwhatportionofthevideotheyhaveviewed.Therearestudiesthatindicatethatthisiscorrelatedtostudentperformanceinthecourseandisthereforerelevantinformationtoretain.ButjustbecausethisinformationisbeneficialtotheLMStomoreaccuratelydetermineastudent’ssuccessinthecoursedoesnotmeanthattheinstructorsshouldbeprivytothisinformation.Itcouldresultintheteacherbeingbiasedfororagainstthestudent.
Anothercasetoconsiderisonewheretheargumentcouldbemade
thatinformationshouldnotbesharedwiththestudent.ReferringbacktotheearlierdiscussiononSignals,ingivingastudentaredlight,whilethisishelpfulinformationindeterminingwhereastudentstands,itcanalsoresultinthestudentfeelinghopelessandunabletoovercomethetaskaheadofthem.Byprovidingthemwiththisinformation,weruntheriskthattheyiftheydidn’tknowthis,theywouldhavekeptworkinghardandimprovedtheirscore.Butnowtheymightjustwritethemselvesoffasnotgoodenoughandstopworkingtowardssuccessinthatclassorsubject.
Thereisalsotheconsiderationofhowinformationissharedwith
thirdparties.StudentrecordsarestrictlyprotectedunderFERPA.However,considerthefactthatprospectiveemployersalreadyaskstudentsforinformationsuchastranscriptsandlettersofrecommendations,whichthestudentsthenallowthereleaseof.AlthoughanalyticalinformationgatheredaboutastudentwouldbeprotectedunderFERPA,ifthisinformationexisted,thereisahighchancethatemployerswouldwanttobeprivytoit.Theintensityofthecompetitionaroundjobswouldmostlikelyresultinascenariowherestudentsdon’thavemuchofanoptionotherthantohandoverthisinformationtoemployers.Therearethosewhoarguethatitisbeneficialtotheefficiencyoftheeconomyifinformationaboutindividualsismadeopenandthisinturnresultsingreatersocialwelfare[9].Thisexampleaddressesthefactthatthedatacollectedwiththepurposeofhelpinganindividuallearnisn’tnecessarilyonlygoingtobeusedforthatpurpose.
26
• IssueTwo:Howintrusiveistheprocessofgatheringinformation?Ifthereapointatwhichtheinformationbeingcollectedisbeneficialbuttoointrusivetobestored?
Whendeterminingwhatinformationtocollect,theprimaryconsiderationiswhetherornotitisrelevant.Inlearninganalytics,wearelookingatafieldthathasrelativelyrecentlyexistedinitscurrentformandassuchalmostanythingcouldbeconsideredtobepotentiallyrelevant.Individualswhosupportthisapproachareprioritizingtheinsightsgainedfromlearninganalyticsaboveallelse[10].Thiscouldthenbeusedasjustificationtocollectinformationaboutallaspectsofanindividual’sliferangingfromtheirpoliticalviewstotheirreligiousbeliefs.
Thebigquestionthatisposedhereisifincreasingalearningoutcomeistheonlyfactorthatorganizationsinthisspaceshouldconsiderwhengaugingwhatinformationtheyshouldandshouldnotcollect.Thisisanissuethatisparticularlyrelevantgiventhatdatacanalsobesourcesfromthingslikesocialmediaandgeolocationwhichwouldprovideschoolsanduniversitieswiththeabovementionedinformation.
• IssueThree:Weighingtheupsidesanddownsidesoftheoutcomeofinsightsfrom
analyticstodetermineifitisjustified.Takingintoconsiderationtherisks,consequencesandthemannerinwhichtheconsequencesaredistributed,doesthatatanypointoutweighthebenefitsgainedfromlearninganalytics?
Thereissubstantialworkdonethatpointsusinthedirectionofsupportingtheideathatlearninganalyticsprovidesadditionalinsightsaboutalearningenvironmentwhichallowinstitutionstoadjusttheirstructurestobettertailortheexperience.Whatweareaddressinghereiswhetherthisgoaljustifyanyunintendedconsequences?Thisbecomesparticularlyrelevantinsituationswhereweconsiderthattheprivacyconcernsofastudentdivergefromwhatisinthebestinterestsoftheinstitution.Anadditionalfactortoconsiderisifthebenefitsoflearninganalyticsarenotevenlydistributedamongststudents.
Anintegralcomponenttothisdiscussionisastudent’srighttoprivacyandtherepercussionsofhavingastudent’sprioritiesdivergefromthatoftheinstitution.Wehavediscussedheavilytheadvantagethatlearninganalyticsprovidesintermsofguidinginstructionintheclassroomanddetermininghowastudentwouldbemostsuccessful.Saythatwewerethenabletocreatealistofclassesastudentshouldtakeinordertobemostsuccessful.Now,weareguidingstudentsintotakingclassesthattheyotherwisemightnothave,whichisn’tnecessarilyinthebenefitofthestudentbutinsteadtheinstitutionatlargewhichisworkingtooptimizeonafewmetrics.
27
• IssueFour:Impactonastudent’sautonomy.Dothebenefitsfromlearninganalyticsoutweighastudent’ssayinwhatinformationisbeingcollectedaboutthem?
Herewearefacedwiththedilemmaofhowmuchcontrolanindividualshouldbeabletohaveoverdatacollectionforlearninganalytics.Giventhatitisstillarelativelynewfield,thereareisnotaclearlistofwaysinwhichthedatayouareallowingtobecollectedaboutyouisgoingtobeused. Inadditiontothedatabeingcollectedaboutthem,thereisanotherelementofwhetherstudentsshouldhavecontroloverwhatthedataisbeingusedfor.
• IssueFive:Doestheworkalignwiththegoalsoftheeducationsystem?
Relativetotheotherissues,thisoneaddressesamuchbroaderviewpoint.Atthecoreoflearninganalyticsistheideathatweareworkingtowardsabettermentofachievinglearningobjectives.Soindiscussingthisissue,wethenmustconsiderwhatthelearningobjectivesare.Todefineitasgoodgradesforstudentswouldbenarrowingthescopeofittolessthanwhatitis.Additionallearningobjectivesincludecommunicationskills,thewayinwhichoneapproachesaproblem,beinganactiveandknowledgeablememberofone’scommunityetc.Giventhatthegoalsaresuchbroadconcepts,itbecomesdifficulttoassesstowhatextenttheyhavebeenachieved.
Acommonbenchmarkusedininstitutionsofhighereducationisthestatisticsaroundjobplacements.Giventhattheinformationaroundjobplacementismosteasytomeasureandfeedintoamodel,therecouldbeundueimportancegiventothisinlearninganalytics.Anexampleofthisgoingwrongwouldbeifclassroomsfullofindividualswerepushedtotakeaclassthatwasn’tnecessarybecauseitmadethemmoreemployable.
Attheheartofalotoftheseissuesisthat,untillearninganalyticsemergedasamore
prominentdiscipline,therewasaclearlinebetweendatausedforassessingastudentandastudent’spersonalinformationsuchaswheretheyliveandhowtheywork.Aslearninganalyticsgrows,thislinecontinuestoblur.
Therearewaysinwhichtoaccommodatetomitigateanddecreasetherisksandissuesbroughtupabove.Bybeingmindfuloftherippleeffectoflearninganalytics,systemsandpoliciescanbeputinplacetoavoidscenariossuchastheonesmentionedabove.Factorstoconsiderinaddressingtheissuesarementionedbelow:
• AddressingIssueOne:Beconsciousofallpossibleimplicationsofsharinginformationwithagivenstakeholder.Whileitmightbebeneficialtoshare
28
informationwithoneparty,itcouldbedetrimentaltosharethesameinformationwithanotherparty.Assuch,theinformationgarneredfromlearninganalyticsshouldalwaysbesetupinsuchamannerthatitallowsfordifferentdegreesofaccess.Rememberthatindividualsandorganizationsoutsideoftheeducationspacemighteventuallybeprivytotheinformationbeingcollected.
• AddressingIssueTwo:Astepindrawingalineaboutwhatdatacanandcannotbecollectedwouldbetoestablishcriteriafordeterminingthisbeyondpotentialrelevance.Wehavethebasisthatdataisbeingcollectedtofurtherthelearningexperienceofanindividual.Inordertomakeitbetter,onehastobeabletochangesomethingsothattheoutcomecanbedifferent.Onepotentialcriterionisthattheinstitutionshouldonlycollectdataonthingsthattheycanactuallychangeandhelpthestudentwith.Thusifitisdeemedthataneducationalorganizationshouldnotinterveneonthebasisofreligionorpoliticalviews,thenitshouldbeimpermissibleforsuchdatatobecollectedtobeginwith.
• AddressingIssueThree:Relativetotheotherissues,thisoneismorenuancedto
address.Thisisbecause,inordertogetguidanceonthebestcourseofaction,therehastobeadecisionmadeonwhatisconsideredtobejustintermsofthebenefitsandconsequences.
• AddressingIssueFour:Thereisastrongargumenttobemadeherethatcreatinganenvironmentoftransparencywouldbefoundationalintrustbetweenindividualsandinstitutionscollectingdataaboutthem.Ifindividualsfeltasenseofcontroloverthedatathattheyweresharing,thiswouldpotentiallymakethemmoreopenwithallowingthedatatobecollected.
• AddressingIssueFive:Itisimpossibletolookatthisissuewithoutconsideringthe
otherfourproblemsdefinedabove.Toknowwhetherlearninganalyticsisanthatisworthyofthecomprisesisalargequestiontoask.Howmuchonevaluesselfautonomyandcontroloverlargerinsightscreatesatugofwarwithinthisindustry.
Therearearangeofconcernsthatareattheforefrontofdiscussionarounddataprivacyissuesinregardstodataanalytics.Theargumenthasbeenmadethatthedatacollectedissignificantlymorebeneficialtotheinstitutionthatiscollectingitthenitistothestudent[1].Furthermore,althoughbigdatacanbeseenasaddingatransparencytotheworld,thereisalsoanissueofthedecisionsthatarebeingmadeasaresultofbigdatabeingseenascomingoutofablackbox.Inthecontextoflearninganalytics,thismeansthatindividualshavelittleknowledgeofwhatinformationisbeingusedtocometowhatconclusionsaboutthem[11].
Asmentionedabovewhendiscussingthetypesofdatathatarepooledtogetherforlearninganalytics,astudent’sdemographicinformation,socioeconomicstatus,genderidentity,sexualidentityetc.couldbeused.Asasocietythathasbeenmakingprejudiceddecisionsforgenerations,weruntheriskoftheconclusionsdrawnfromthisdata
29
mirroringoldandexistingprejudices.Thiswouldinturnresultinsystematicbiases.Anadditionalconsiderationisthataspeoplebecomemoreawareofthedatathatisbeingcollectedaboutthem,theyalterthemannerinwhichtheyareactingandthisinfluencestheconclusionsthendrawnfromthedata.Thisisreferredtoasthechillingeffect[7].ImplicationsofPrivacyConcerns: Inconsideringtheprivacyissuesfacedbythelearninganalyticsspace,wecanlooktootherindustriesindeterminingbestpracticesandstandardstouseasabenchmark.Oneindustryinparticularwhereprivacyconcernsstronglydictatehowdataissharedisthehealthcarespace.Thisisanexampleoftherebeingheavyregulationthatsetstheroadmapforwhatisacceptableandwhatisnotacceptable.Ontheotherhand,whenlookingatthecommercespace,thereisconsiderablylessregulationonwhatacompanycandowiththedatatheycollectandalsowhotheycanshareitwith.However,acombinationofdataleakageissuesaswellasgrowingconsumerawarenessinregardstothemassamountofdatabeingcollectedaboutthemhasresultedinapushformoreregulationinthisspaceaswell.Itisimportanttonotethatnoneoftheregulation,noteventhatinthehealthcarespace,pointstowards“absoluteconfidentiality”[8].
Inordertohavestricterregulations,thereisprotectionofdatathatcouldbeindividuallyidentifiable.Whenthedatacannotbelinkedbacktoaspecificperson,therulesofsharingitbetweenorganizationsandthepurposesforwhichitcanbeutilizedarelooser.
Whendiscussingprivacyissues,wedrawadistinctionwithageandthereisasignificantdifferenceinthepolicyforminorsversusindividualsovertheageof18.Thequestionthenarisesifthisisafairplacetodrawadistinction?Giventhereducedamountofregulationincollectingdataonadults,itissignificantlyeasierforfirmstodoso,andassuchtheyhavemoredataaboutadultsavailabletothem.Sinceitissafertosolicitpermissionfromadultstogetdata,isthereapotentialtoregressbackwardsfromdataaboutadultsandthenusethistobuildmodelsforstudents?
Ifthesectorislessregulatedintermsofdatausage,therearesituationsinwhichinstitutionstakeituponthemselvestousethedataresponsibly.Thisisseenascorporatesocialresponsibilitydictatingthewayinwhichdataisused.Althoughtheeducationalsectorisnotonethatislightlyregulated,particularlyinrelationtolearninganalytics,theideaofcorporatesocialresponsibilitystillplaysakeyrolehere.Sincetheinnerworkingsoflearninganalyticsisseenasablackholetomostconsumers,inorderforparentstosignoffontheirchildren’sdatabeingcollected,theyhavetobelievethatthedatawillnotbeusedandtheresultswillnotbedetrimentaltotheirchild’slearningexperience.Corporatesocialresponsibilityinregardstolearninganalyticscancomeinmanydifferentforms:makingsurethestorageofthedataisheldtoahighstandardofsafety;anonymizingthedatawheneveritcanbedone;notsharingdatawiththirdparties;opencommunicationwiththeconsumersonwhatdatatheyarecollectingandtowhatenditisbeingused.
30
DataasaCommodity: Thereisanotherfacettothisdiscussionwhichtakesintoaccounthowdataisviewedinothersectors.Inamultitudeofconsumerfacingindustries,weseethatdatahasahighmonetaryvalueandissoldbycompaniesasasignificantportionoftheirrevenue.Thisideaviewsdataasacommodity.Lookingbackonthetypeofcompaniesthatwesawinthelearninganalyticsspace,thisisnotasourceofrevenueforthemgiventhestrictlimitationsoftheirdatausage. Giventhattheyareconstrainedintheirabilitytomonetizedirectlyfromtheirdatacollection,thiscouldinsomecapacitydeterthemfromenlargingthedatathattheyarecollecting.Consideringtherevenuestreamforalotofcompaniesintheanalyticsspaceiscloselytiedtotheirdatacollection,thisisaconsiderableconstraintforsomecompaniesinthethelearninganalyticsspace.
AnexampleofthiswasrevealedinaninterviewwithRobRichardson,theCEOofPocketPoints[30].PocketPointshasaphoneapplicationthatgivesstudentspointsforhavingtheirphonesoffduringclass.Thesepointscanthenberedeemedforrewardssuchasextracreditoracouponforalocalstore.Thegoalhereistooincentivizestudentstostayofftheirphonesduringclass.However,thefirmstrugglestomaintaintheirprofitmarginsandonegoodsolutionwouldbetohaveadvertisementswithintheapp.However,giventheregulationsofthespace,theyareunabletogetenoughdataonastudenttoprovidetargetedadvertising.Assuch,companiesdon’tthinkitisworththeirmarketingmoneytoadvertisethroughPocketPointsapp.Thisisnottosaythattheyshouldbeallowedtoadvertisetostudents,butrathertodrawattentiontothecomplicationsthatariseinthefieldwhendatacannotbeviewedasacommodity.
31
Conclusion Althoughregulationshouldserveasawaytosafeguardstudents’rightsandensurethattheyarenotputinharm'sway,itisincrediblybeneficialtoboththemandschoolsatlargefortheirdatatobemoreopenlyshared.Withouthavingaccesstolargepoolsofstudentdata,companiesinthespacearenotincentivizedtoinvestingrowingthescopeandapplicationsoflearninganalytics.Allthreekeystakeholdersstandtobenefitfromlearninganalyticsbecomingalargerpartofeducationalorganizations. Fromastudent’sperspective,thebenefitliesinanindividualizedcurriculum.Thetwoprimaryadvantagesofthisarethatwecangravitateawayfromthetraditionalclassroommodelofputtingmoreefforttowardsthestudentsatthetopandbottomofthegradingdistributionandprovideaplatformthatsupportsstudentsatalllevels.Thesecondstakeholder,whichistheteacher,cangainalotfromknowingwhatthemosteffectivecomponentsoftheirteachingpracticesare.Theycanthenfurtherhonetheseskillsanddevelopamoretargetedefforttowardsmakingthemostofthetimetheyhavewiththeirstudents.Consideringthethirdstakeholder,theadministrationandlargerinstitutionalgoals,theirfirstbenefitcomesfromstudentsandteachersperformingbetter.Havingstudentsgetthebesteducationtheycanisreflectedinanincreaseinstandardizedscoresthatareoftenusedatperformancemetricstomakesureagivenschoolisuptopar.Furthermore,itprovidesthemthecapacitytostreamlineprocesseswithintheinstitution,resultingingreateroperationalefficiencies.
Alloftheabovementionedpotentialgrowththatcancomeoutoflearninganalyticsiscontingentupontheorganizationsandindividualsdrivingthistechnologicaladvancementhavingaccesstostudentdata.Hereinweseecomplicationsbeginsurroundingaccesstodata,particularlyaboutminors.Thiscancreateseriousbarriersforthetechnologytobeabletoreachitsfullpotential.Thisbecomesanevenmorecomplicatedmatterwhenweconsiderthatcompaniesinthisfieldmightbedisincentivizedtocollectdatasincetheydonotseenhowtheycanbuildaprofitablebusinessmodelwiththisconstraint.
Learninganalyticshasthepotentialtomakeapositiveimpactonthefutureof
humanityinaveryfundamentalway.TherapidadvancesandhugeinvestmentsinAI,MLandothertechnologiesarepoisedtodeliverpowerfulalgorithmsandtechniquesthatwillmakeanalyticsmoreinsightfulandactionable.Theproliferationofdigitaldeviceswillcontinueandtheamountofdataavailabletoanalyzewillalsocontinuetogrowexponentially.Technologywillexistwhereitwillbepossibletoidentify,interveneandinfluenceastudent’slearningexperiencesothatitishyper-personalizedandbringsoutthebestindividualoutcomeforthatstudent.Thiscanbemadeavailabletostudentsearlyintheireducationandregardlessoftheirsocio-economicstatus.Thepositiveimplicationsofsuchachangearemanifold.Theregulatoryenvironmentwillneedtoadaptandensurethatitisenablingsuchachangeinsteadofbeinganelementoffriction.Thatwilldeterminetheimpactoflearninganalyticsmorethananyotheraspect.
32
Learninganalyticsisfacedwithacatch22—wecannotbesurewhattherisksandrewardsoflearninganalyticsapplicationslooklikeatthescaleweenvisionthemtobeuntilwehaveenoughdatatoputitintoaction.Butwecannotallowthedatatobecollecteduntilweareabletoweighrisksandrewardsandshaperegulationaccordingly.Soascompaniespioneerintothislandscape,itiscriticalthattheykeepatthecoreoftheirlearninganalyticseffortstheirsenseofcorporatesocialresponsibilityandholdtheirdatapracticestoahighstandard.Thisiscriticalinbuildingtrustwiththeotherstakeholders,anditisonlythroughthesupportofallthreestakeholdersthatthelearninganalyticsspacewillbeanenvironmentthatfostersgrowthforthesecompanies.
Wecanlookforwardintowhataclassroommightlooklike5yearsfromnow:desks
thatareinteractive,computersthatprovidepersonalizedassessments,avirtualrealitycornerwherestudentspracticedebating.Learninganalyticshasakeyroletoplayinmakingthisareality.Astechnologysteamrollsthroughthecurrentclassroom,creatinganentirelynewenvironmentthatwedidn’tevenknowwecouldaspireto,wecannotallowtheethicalimplicationsofthistolagbehind.Wecannotforgetthattheindividualsmostimpactedbythisareoftenonestooyoungtounderstandtheimplicationsofthechangeandhaveasayinit.Theirinterestsandsafetymustbeattheforefrontofdiscussionaswebuildourfutureclassroom.
33
Endnotes
[1]Rubel,AlanP.andJones,Kyle,DataAnalyticsinHigherEducation:KeyConcernsand OpenQuestions(May2016).UniversityofSt.ThomasJournalofLawandPublic Policy,2017.AvailableatSSRN:https://ssrn.com/abstract=2905985
[2]Honey,Kristen.“MyData:EmpoweringAllAmericanswithPersonalDataAccess.”NationalArchivesandRecordsAdministration,NationalArchivesandRecordsAdministration,Mar.2016,obamawhitehouse.archives.gov/blog/2016/03/15/my-data-empowering-all-americans-personal-data-access.
[3]Baker,RyanShaunJoazeirodeandPaulSalvadorInventado.“ChapterX:Educational DataMiningandLearningAnalytics.”(2014).
[4]Omedes,Jose.“LearningAnalytics2018-AnUpdatedPerspective.”IADLearning,15Feb.2018,www.iadlearning.com/learning-analytics-2018/.
[5]Yudelson,MichaelV.,etal.“IndividualizedBayesianKnowledgeTracingModels.”LectureNotesinComputerScienceArtificialIntelligenceinEducation,2013,pp.171–180.,doi:10.1007/978-3-642-39112-5_18.
[6]MohamedAmineChatti,AnnaLeaDyckhoff,UlrikSchroeder,andHendrikThüs.2012. Areferencemodelforlearninganalytics.Int.J.Technol.Enhanc.Learn.4,5/6 (January2012),318-331.DOI=http://dx.doi.org/10.1504/IJTEL.2012.051815[7]Solove,DanielJ.“ATaxonomyofPrivacy.”UniversityofPennsylvaniaLawReview,vol. 154,no.3,2006,pp.477–564.JSTOR,www.jstor.org/stable/40041279.[8]JianfengXia,MinYu,YingYang,HaoJin,"PersonalizedPrivacy-Preservingwithhigh performance:(αε)-anonymity",ComputersandCommunications(ISCC)2018IEEE Symposiumon,pp.00807-00812,2018.[9]Posner,RichardA.“AnEconomicTheoryofPrivacy.”PhilosophicalDimensionsof Privacy:AnAnthology,editedbyFerdinandDavidSchoeman,CambridgeUniversity Press,Cambridge,1984,pp.333-345.[10]Diaz,V.,andM.Brown.2012.Learninganalytics:AreportfromtheELIfocussession. ELIreportELI3027.[11]Slade,Sharon,andPaulPrinsloo.“LearningAnalytics.”AmericanBehavioralScientist, vol.57,no.10,2013,pp.1510–1529.,doi:10.1177/0002764213479366.
34
[12]“Report:OneinFourStudentsEnrolledinOnlineCourses.”OLC,25Feb.2016,onlinelearningconsortium.org/news_item/report-one-four-students-enrolled-online-courses/.
[13]“SignalsTellsStudentsHowThey'reDoingEvenbeforetheTest.”SignalsTellsStudentsHowThey'reDoingEvenbeforetheTest,www.purdue.edu/uns/x/2009b/090827ArnoldSignals.html.
[14]Siemens,George.“LearningAnalytics.”AmericanBehavioralScientist,vol.57,no.10,2013,pp.1380–1400.,doi:10.1177/0002764213498851.
[15]Bogarín,Alejandro,etal.“ClusteringforImprovingEducationalProcessMining.”ProceedinsoftheFourthInternationalConferenceonLearningAnalyticsAndKnowledge-LAK'14,2014,doi:10.1145/2567574.2567604.
[16]Khalil,Mohammad,andMartinEbner.“ClusteringPatternsofEngagementinMassiveOpenOnlineCourses(MOOCs):theUseofLearningAnalyticstoRevealStudentCategories.”JournalofComputinginHigherEducation,vol.29,no.1,2016,pp.114–132.,doi:10.1007/s12528-016-9126-9.
[17]Mukala,Patrick,etal.“LearningAnalyticsonCourseraEventData:AProcessMiningApproach.”DepartmentofMathematicsandComputerScienceEindhovenUniversityofTechnology,Eindhoven,TheNetherland,2015.
[18]Sabourin,Jennifer,etal.“StudentPrivacyandEducationalDataMining:PerspectivesfromIndustry.”InternationalConferenceonEducationalDataMining,2017,doi:10.31094/2017/1.
[19]Upasana.“DecisionTree|DecisionTreeIntroductionWithExamples.”Edureka,Edureka,22Feb.2019,www.edureka.co/blog/decision-trees/.
[20]Ferguson,Rebecca,andSimonBuckinghamShum.“SocialLearningAnalytics.”Proceedingsofthe2ndInternationalConferenceonLearningAnalyticsandKnowledge-LAK'12,2012,doi:10.1145/2330601.2330616.
[21]“PowerYourSchoolOperationstoSuccess.”PowerSchool,www.powerschool.com/ppc/student-information-system/.
[22]Hartmann,T.,Moawad,A.,Fouquet,F.,Nain,G.,Klein,J.,Traon,Y.L.,&Jézéquel,J. (2017).Model-DrivenAnalytics:ConnectingData,DomainKnowledge,and Learning.CoRR,abs/1704.01320.
[23]Malliarakis,Chris,andMayaSatratzemi.“IntegratingLearningAnalyticsinanEducationalMMORPGforComputerProgramming-IEEEConferencePublication.”
35
InstituteofElectricalandElectronicsEngineers,ieeexplore.ieee.org/abstract/document/6901446.
[24]“PersonalizedLearningforAllStudents.”PersonalizedLearningforAllStudents|CurriculumAssociates,www.curriculumassociates.com/products/i-ready.
[25]Wan,Tony.“USEdtechInvestmentsPeakAgainWith$1.45BillionRaisedin2018-EdSurgeNews.”EdSurge,EdSurge,15Jan.2019,www.edsurge.com/news/2019-01-15-us-edtech-investments-peak-again-with-1-45-billion-raised-in-2018.
[26]“ByTheNumbers:MOOCsin2018-ClassCentral.”ClassCentral'sMOOCReport,21Mar.2019,www.classcentral.com/report/mooc-stats-2018/.
[27]Lardinois,Frederic.“DuolingoHiresItsFirstChiefMarketingOfficerasActiveUserNumbersStagnatebutRevenueGrows–TechCrunch.”TechCrunch,TechCrunch,1Aug.2018,techcrunch.com/2018/08/01/duolingo-hires-its-first-chief-marketing-officer-as-active-user-numbers-stagnate/.
[28]PhilipKerr,Adaptivelearning,ELTJournal,Volume70,Issue1,January2016,Pages 88–93,https://doi.org/10.1093/elt/ccv055.
[29]“CASESTUDYA:TrafficLightsandInterventions:SignalsatPurdueUniversity.”LearningAnalyticsinHigherEducation.
[30]Richardson,Rob.“InterviewwithPocketPointsCEO.”16Apr.2019.
