MOOC Retention Rate and Motivation - Ondrej Henek › x ›...

MOOCRetentionRateandMotivationOndrejHenek

Thesis,M.Sc.ITProductDesign

1stJune2017 Supervisedby:SaluYlirisku

DepartmentofDesignandCommunicationUniversityofSouthernDenmark,Kolding

3

Thanksto

JakubRybář,SaluYlirisku,interviewparticipants,colleagues,teachers,friendsandfamilymembers

fortheirhelpandsupport.Printedby:Print&Sign,SDU

4

ABSTRACT......................................................................................................................................5

INTRODUCTION..............................................................................................................................6MASSIVEOPENONLINECOURSES.............................................................................................................7NOSTIS.................................................................................................................................................7VOCABULARY.........................................................................................................................................8

THERETENTIONRATE...................................................................................................................10MOOCSTUDENTSANDRETENTIONRATE...................................................................................................11REDEFININGRETENTIONRATE.................................................................................................................12

LONGTERMENGAGEMENTANDMOTIVATIONTHEORY...............................................................14UNDERSTANDINGMOTIVATION(MOTIVATIONTHEORIES,EXTRINSICANDINTRINSICMOTIVATION).....................14MOTIVATIONTYPESANDDIMENSIONS......................................................................................................14ENHANCINGMOTIVATIONUSINGGAMIFICATION........................................................................................16PITFALLSOFENHANCINGMOTIVATION.....................................................................................................18HOWOTHERSAPPROACHMOTIVATION?...................................................................................................18

METHODSANDDATA...................................................................................................................20THEFRAMINGOFTHISPROJECT...............................................................................................................20THEEXPLORATIONOFONLINEEDUCATION................................................................................................20THESURVEY.........................................................................................................................................21THEMOTIVATIONINTERVIEWS................................................................................................................21THEWORKSHOP...................................................................................................................................22OBSERVINGUSERBEHAVIOR...................................................................................................................22THEPROTOTYPETESTING.......................................................................................................................22

WHOARETHEMOOCUSERS?.......................................................................................................24USERSOFNOSTIS.................................................................................................................................24MOTIVATIONOFNOSTISUSERS...............................................................................................................27WHYDOPEOPLELEAVEANDNEVERCOMEBACK?.......................................................................................27

OPTIMIZATIONSFORRUSHERSANDSAMPLERS...........................................................................29THEEXPERIMENTRESULTS......................................................................................................................30

THEMOOCONBOARDINGEXPERIENCE........................................................................................32THEWIREFRAMEEXPLAINED...................................................................................................................32

THEMOTIVATIONHELPER............................................................................................................35THELACKOFTIMEPROBLEM...................................................................................................................35WHYMOTIVATIONHELPER.....................................................................................................................35

DISCUSSION.................................................................................................................................38THERESEARCHREFLECTION....................................................................................................................38OTHERCONCEPTSWORTHEXPLORING......................................................................................................38THEMOTIVATIONHELPER’SNEXTSTEP.....................................................................................................39

CONCLUSION................................................................................................................................40

REFERENCES.................................................................................................................................41

5

AbstractTheaverageretentionrateamongonlinecoursesisreportedtobearound10%,causingalotofcriticismofMOOCs.Buttheidea,thatonly10%ofusersaresuccessfulandtheremaining90%fails,iswrong.Thisthesisexploresthroughinterviewsandasurvey,whatdoesthe90%do, what are their motivations and behavior. Then, based on motivation theory andgamification, threeMOOCplatformoptimizations are designed: 1) Improved coursemainpage to convey all the important information the user needs to enroll. 2) Optimizedexperienceforpeoplefinishingacoursefasterthanusual.3)AMotivationHelperaimingtoenhancemotivation of users by a notification system. All these designs were tested andevaluated,theresultsandnextstepsarethendiscussed.

MOOCretentionrateogmotivering

Dengennemsnitligeopretholdelseomkringonlinekursererestimerettilomkring10%.Detteharmedført enmasse kritik omkringMOOCs. Ideenomkring at kun 10%af brugerneharsucces og at de resterende 90% fejler, er forkert. Gennem interviews og en rundspørge,udforskerdettespecialehvadderskermedderesterende90%.Hvaderderesmotivationerog adfærd? Baseret på motivations teorier og gamification, bliver tre MOOC platformsoptimalere designet. 1) Forbedre kursets forside så den giver al den vigtige informationbrugernebehøverforatdeltage.2)Enoptimeretoplevelseforpersonerderfærdiggøreetkursus hurtigere end normalt. 3) En motiverende hjælper der har til formål at forbedremotivationenhosbrugerneviaetnotifikationssystem.Alledissedesignserblevettestetogevalueret,resultaterneogdenæstenødvendigetrinbliverderefterdiskuteret.

6

IntroductionIn2014Ihaveco-createdtheNGONugisFinemwhichhasamotto“Thereisabetterlifeinmoreeducatedsociety”.Ourbiggestproject isthefirstpurelyCzechMassiveOpenOnlineCourseplatform–Nostis.DuringthetwoyearsofitsexistencewehavereceivedempoweringfeedbackfromouruserswhichkeptmeinvestingmytimeintoimprovingNostis.ButwhileIwasexploringthevastthemeofonlineeducation,IhaverepeatedlystumbledoveracritiqueofMOOCsbasedontheargumentthatinaverageonly10%ofenrolledstudentscompletethecourse.Thissoundedlikeanawfulnumberanditmademewonderwhatwashappeningwiththe90%.MorethanaresearchquestionIhaveconstructedaresearchdirection:

Whydopeoplesignupforanonlinecourse,whydotheyleaveandhow

canweencouragethemtostayandcompletethecourse?

ExploringandansweringthisquestionshouldhelpmycolleaguesinNugisFinemandotherMOOCcreatorsanddesignerstoprovidebetteruserexperiencetotheirusers.InthischapterIintroduceMOOCsandthenfocusonNostis.IalsoincludeavocabularyofthekeywordsusedintheMOOCworldandthisthesis.

ThesecondchapterisallaboutTheRetentionRate,whatitisandwhyitdoesnotmakesenseinthecurrentstate.ThefollowingchapterLongTermEngagementandMotivationTheorygivesatheoreticalintroductionintomotivationtheoriesandgamificationonwhichIbuiltmyenhancementsandprototypes.

Thenext chapterMethodsandData givesanoverviewofmethodsand tools I haveusedthroughoutmyresearchandhowIgatheredmydata.InchapterWhoAretheMOOCUsers?IdelveintotheworldofaMOOCstudent,exploringtheirmotivationsandbehaviorandwhypeopleleaveonlinecourses.

Following three chaptersOptimization for Rushers and Samplers, TheMOOC Onboarding

ExperienceandTheMotivationHelperintroducethewayIusedthemotivationtheoriesandtheresearchfindingsinordertoboostretentionrateandenhanceuserexperienceonNostis.IclosethisthesiswithaDiscussionevaluatingmyapproachanddescribingthenextstep,andConclusion,whereIprovideasummaryofthewholework.

“IfIciteanon-Englishtext,Iuseatranslationdirectlyinthetext,butthe

originalinnon-Englishcanstillbefoundinafootnote.”–Ondrej

Inthiskindofbox,Isharemyownexperiences.Whiletheydidn’tcontributedirectlytomyresearch,myownexperiencehasinfluencedmywork.

Chaptersummary

- Tomaintainanoverview,thereisaboxattheendofeverychapter,excepttheIntroduction,summarizingthecontents.

7

MassiveOpenOnlineCourses

MOOC1isaphenomenonofthe21stcentury,althoughtheideaofdistanteducationhasbeenaroundformorethanhundredyears.Theyear2012wascalledTheYearoftheMOOCbyTheNewYorkTimes(Pappano2012)becausetheleadinguniversitiesreleasedtheirfirstonlinecoursesandthepublicityoffreeonlineeducationboomed.Butaftertheinitialexcitement,alotofwell-groundedcriticismappeared.(J.Kim2012)

TheMOOCrevolutionwasmeanttodemocratizeeducationandbringtopknowledgetoallcornersoftheworld.Butthesehighgoalsarestillnotareality.Internetconnectionstableandfastenoughtoplayavideocourseisnotavailableinmanypartsoftheworldandthereis still the language barrier. Most of the popular courses are in English and althoughtranslations to various languages are sometimes present, the content is still notunderstandabletoeveryone.(Wildavsky2014;Glass,Shiokawa-Baklan,andSaltarelli2016)

AnotherconcernisaboutthequalityoftheeducationwithpeopleclaimingthattheMOOCversionofthecourseisinferiortotheon-campusone(Laurillard2014).Thecriticssay,thatfromapedagogicalperspective,MOOCsareonlyfocusedonthe“massive”andtheylackanypedagogicalinnovation.(Karsenti2013)

Anotherwaveofcriticismcamebecauseofverylowreportedretentionrate.Onlyaround10%ofsignedupstudentsfinishthecourse,sometimesdroppingtoastaggering2.5%(Hoetal.2014).Buttakingjustthisnumberwithoutthinkingaboutwhatitrepresentsisworthevenmorecriticism.MostoftheMOOCplatformsdonotofferanypreviewofthecontentwithoutsigningupandthesign-upbuttonalsoservesas“Remindmewhenthecoursestarts”givingtheactionseveralmeanings.

Nostis

NostisisaMOOCplatformpublishedinMay2015byNGONugisFinem2.ItsfirstcoursewasaboutFamilyLawandmorelaw-themedcoursesfollowed.Theplatformwasdesignedanddevelopedbymyself,thereforeIhavedeepinsightintohowitworksandaccesstoanydataavailable.Atthistime,wehave6coursesalready,5about lawand1aboutcreativity.WecollaboratewithprofessorsfromvariousCzechUniversitiesandexpertsinthefield.Asof23rdMay, we have 3404 registered users and 1237 certificates given among 4938 courseenrollments,thereforewecouldcalculate25%retentionrateacrossthe6courses.

1MassiveOpenOnlineCourseis:“AcourseofstudymadeavailableovertheInternetwithoutchargetoaverylargenumberofpeople.”(“MOOC”2017)2NugisFinemisaCzechNGOcreated in2015focusedoneducatinggeneralpublic.Czechwebsiteisathttps://www.nugisfinem.org

8

Figure1:DevelopmentofsignedupusersandcoursesafteracompleteredesigninJanuary2016untilMay2017.Everynew

coursebringsawaveofnewusers.

Vocabulary

SincethereareseveraltermsconnectedtoMOOCsthathavespecialmeaning,Iwouldliketointroducethemtothereadertoavoidanyconfusionfurtherinthetext.

MOOC

TheclassicalMOOCisservedonaninternetplatform,whereuserregistersandenrollsintoaMOOC.The coursehasa set startingdateandeveryweekanewblockof lecturesopenscontainingalsoaquiz.Afterscoringacertainnumberofpointsinquizzes,userisawardedbyacertificate.

Week/Block

Ablockisasetoflectures,oftencalledaweek,becauseusuallyeveryweekanewblockisaccessibletothestudent.

Lecture

ThebasicbuildingblockofMOOCs,usuallyinaformofvideo.Buttheselecturescanbealsographics,textsoradditionalmaterials.

Test/Quiz

Testoraquizisatypicalassessingtool.Userispresentedwithaquestionandmustchooseacorrectanswerfromgivenoptions.Aftersubmitting,user isawardedwithpointscountingtowardsacertificate.

Peerassessment

Peer assessment is another assessment tool with higher pedagogical value than a test(Kulkarnietal.2015).Ithasthreestages,oftenagaininaweeklyrhythm:Thefirstweekuserscompleteagiventaskbycreatingsomekindofcontent.(writinganessayorcommentary,

0

500

1000

1500

2000

2500

3000

3500

Nostistimeline

Users Courses(scaled)

9

drawingdiagrams, takingpictures,makingavideo…)The secondweek theygrade severalworksofotherpeersbygivencriteria.Andinthethirdandfinalstagetheusercanseetheirowncontentgradedbyseveralotherstudents.

Figure2:OneofthecoursesonNostis.Thecoreelementsaretitle,descriptionandanenrollbutton.Belowontheleftisalist

oflecturesdividedintoblocks.Ontheright,thereisanoverviewofpointsearnedshowingprogresstowardsthecertificate,

alistofcollaboratinginstitutionsandalistoflecturers.

10

TheRetentionRateTheretentionrateisformethemostinterestingphenomenonintheworldofMOOCs.Itisaveryloudlytalkedaboutbututterlymisunderstoodmetric.Asstatedintheintroduction,mostreports, papers and criticism isusing the value10% retention rate. (Hood, Littlejohn, andMilligan2015;BelangerandThornton2013;Adamopoulos2013)ManystudiesevendefinethetopicofMOOCswiththisnumber,copyingitfromothers,withoutconsideringhowthemetriciscreatedandtheimpactthishasondecisionmakersatuniversities.

“Notably,MOOCshavebeenplaguedbyextremelyhighdrop-outrates.”

(Zhengetal.2015)

The usual way to count retention rate is to take the number of successful students(completingcriteriaforacertificate)anddividethembythenumberofpeoplewhoenrolledto the course. The act of enrollingmeans clickingon the “Enroll” buttonbyuserwithoutknowingtheintentionsoftheuser.

Theretentionrateisoftencomparedtoenrollmenttoofflineuniversities,butthishasmanyproblems. Insheerabsolutenumbers,even1%retentionrate inaMOOC,where100000studentsenrolled,therefore1000studentsfinished,isagreatsuccesswhereclassicuniversitycourseneedsoftenseveralyearstoaccomplishthesame.WhereMOOCenrollmentrequiresemailaddressandoneclick,classicuniversitieshavetoughprerequisitesandvariousfees.ThereisalsonopenaltyinfailingaMOOC,whereasstudentsofclassicuniversitieshavetoinvestmuchmore timeand resources into thecourseand failing is thereforemuchmorecostly(Hoetal.2014).

From another perspective, continuous enrollment also skews the data. As those freshlyenrolledcannotobtainacertificateyet,buttheyappearinstatisticsnevertheless.Thishascausedatemporarydropinretentionratetoastunning2.5%onEdXplatform(Hoetal.2014).

Universitydecisionmakers

TheMOOCs on Nostis aremostlymadewith direct collaborationwith teachers.We hadproblemsnegotiatingwithauniversityasthewhole,becausethekeydecisionmakershavealmost no information aboutMOOCs and the easy to find criticism is often including thismythicalretentionratesasanargument.

11

Figure3:StudentpersistenceinBioelectricitycourseFall2012.(BelangerandThornton2013)Atypicaldistributionofstudent

engagementinaMOOC.

Moocstudentsandretentionrate

Theoriginalapproachtoretentionratehasverynarrowsuccessconditionstailoredjustforonekindofmotivation:Tocompletewholecourseandearnacertificate.Buttherecouldbemanykindsofmotivationtoclicktheenrollbutton.Thoseareforexample:

1. Watchlectures,doexercisesandearncertificate.(Theclassicalapproach.)2. Onlywatchlecturesandlearnwithoutexercises.3. Accessonlysomematerialswithinacourse(lecture,pdf,recommendation,forum…).4. Lookatwhatthecourseislike.5. LookatwhatMOOCslooklike.6. Beremindedofthestartofthecourseandthendecidewhetherdoitornot.

“Littleresearchattentionhasbeendirectedtothepopulationofstudents

whodonotfinishthecoursesinwhichtheyregister;asaresultitisdifficult

tofullycapturetheuserexperiencestiedtoretentionproblems,andmany

importantreflectionsaremissed.”(Zhengetal.2015)

Astudyofdisengagementfrom3coursesshowedaninterestingbehaviorofMOOCstudents.Kizilcec,Piech,andSchneider(2013)identified4basicsubpopulationsofMOOClearners:

1. ‘Completing’:learnerswhocompletedthemajorityoftheassessmentsofferedinthe

class.Thoughtheseparticipantsvariedinhowwelltheyperformedontheassessment,

theyallatleastattemptedtheassignments.Thisengagementpatternismostsimilar

toastudentinatraditionalclass.�

2. ‘Auditing’:learnerswhodidassessmentsinfrequentlyifatallandengagedinsteadby

watchingvideolectures.Studentsinthisclusterfollowedthecourseforthemajorityof

itsduration.Nostudentsinthisclusterobtainedcoursecredit.�

3. ‘Disengaging’:learnerswhodidassessmentsatthebe-ginningofthecoursebutthen

have a marked decrease in engagement (their engagement patterns look like

Completingatthebeginningofthecoursebutthenthestudenteitherdisappearsfrom

12

the course entirely or sparselywatches video lectures). Themoments atwhich the

learnersdisengagediffer,butitisgenerallyinthefirstthirdoftheclass.�

4. ‘Sampling’: learners who watched video lectures for only one or two assessment

periods(generallylearnersinthiscategorywatchjustasinglevideo).Thoughmany

learners“sample”atthebeginningofthecourse,therearemanyothersthatbriefly

explorethematerialwhentheclassisalreadyfullyunderway.

Thedistributionoftheseparticipantsvariedamongthe3coursesallwithdifferentdifficulty.

Figure4:Auserbehaviorbasedoncoursedifficulty.HS=highschool,UG=Undergraduate,GS=graduate

Mostinterestingformeisthesizeofsamplinggroupinrelationtocoursedifficulty.Thisresultsuggests, that over 74%probably just looked at several videos anddecided to leave thecourseforsomereason.Wecanalsoseethat inthedifficultcourse,morepeopleare justwatchinglecturesandtrytogettheknowledge(auditing),whilelesspeoplefinishthecoursewhilestayingontrack,completingassessmentsandgettingcertificate.

Thisbehaviorwasexploredfurtherbyanotherstudy(Zhengetal.2015)wheretheintentionbehindsigningupwasexploredbyconductinginterviews.Theresultssuggest,thattherearestudentswhowanttotakeonlypartoftheknowledgeinsideacourse,socalled:“learningondemand”.

“IamonlyinterestedintheAmericanhistoryduringtheColdWar,Ionly

studiedthoserelatedlecturesandskippedothers.ThatwaswhyIdidn’t

finishthecourse.Ihavealreadyreachedmygoal.”(Zhengetal.2015)

Redefiningretentionrate

Thoseexamplesofmotivationsandbehavior,different thanexpected linearadvancementtowards certificate, made me wonder whether the problem with retention rate is notsomethinginMOOCs,butthemetricitself.Althoughitmightsoundlikecheating,changingtherulesformeasuringretentionrateandsuccessofusersmakesalotofsenseinthiscase.

InspiredbyJobstobeDoneframework(Christensenetal.2016)Iwanttoselectandworkonly with those people, whose job to be done is to finish the whole course and earn

InlastfewyearsItriedtodo2courseswhereIhavefailedonpaper,butIhavefinishedthemanyway.IncoursesLearningHowtoLearnandGamification,IstoppeddoingexercisesatsomepointandthenIalsostoppedauditing,becauseIdownloadedtheremaininglecturestowatchitlater.Ileftthecoursebecauseoflackoftime,butIhavereturnedtothecontentlater.Thus,learningeverythingfromthecourse,butnotcompletingitinthegeneralsense.

13

certificate.Thenmyretentionratewouldshow,howmanypeoplehadtheintentiontofinishthecourseandthensucceeded.Andmyaimistohelpthoseuserstostayontrack.

Butwhatother jobstobedonearesatisfiedordissatisfiedbyNostis?AndshouldI ignorethose, who do not have the intention to earn certificate? To find an answer, I had tounderstandourusersandhowmotivationworks.

Theretentionratesummary:

- MOOCsareoftendefinedbylowretentionrate,whichmakesithardforourNGOtonegotiatesupportfromuniversities.

- Studies show, that MOOC users have different motivations and differentbehavior.

- Retentionrateiscalculatedbynumberofpeopleenrolleddividedbynumberofcertificatesgiven.Thisworksonlywithafractionofusersignoringmanydifferentmotivationsandbehaviors.

- Theretentionrateisflawed.Itdoesnotrepresentwhatitshouldbe.

14

LongtermengagementandmotivationtheoryEveryactionandbehaviorhavesomekindofmotivationbehindit,andenrollingintoacourseisnotanexception.Inthischapter,Ilookatthetheoreticalbasisofhowhumanmotivationworksandhowtoempowerit.ThenIdelveintogamificationtofindconcreteframeworksandtools,thatcanhelpmyusersachievetheirgoal.

Understandingmotivation(Motivationtheories,extrinsicandintrinsicmotivation)

Thefieldofmotivationaspartofthedisciplineofpsychologyhasmaturedovermanydecadesincludingwell-knownexamplesasPavlov’sconditionalpsychologyexperimentsorMaslow’spyramid.Motivation has as many definitions as points of view and various theorists areinterestedindifferentaspectsofmotivation.

Amotiveiswhatpromptsapersontoactinacertainwayoratleast

developaninclinationforspecificbehavior.–Kast’sandRosnzweig’sdefinition(Pardee1990)

Motivationisultimatelyderivedfromatensionthatresultswhenoneor

moreofourimportantneedsareunsatisfied.–Dessler’sdefinition(Pardee1990)

Tobemotivatedmeanstobemovedtodosomething.Apersonwhofeels

noimpetusorinspirationtoactisthuscharacterizedasunmotivated,

whereassomeonewhoisenergizedoractivatedtowardanendis

consideredmotivated.–(RyanandDeci2000)

IchosetousetheoriesofRichardM.RyanandEdwardL.Deci,whoareoftenmentionedincontext of gamification, motivation at work or in education. They respond well to theobjectionsofpreviousgenerationsofpsychologistsandtheirideasmadesenseincontextofmy research. (This chapter tries to explain a combination of Self-Determination Theory,Organismic Integration Theory and Cognitive Evaluation Theory. Together these theoriesdescribemotivation.)

Motivationcanbedividedintoextrinsicandintrinsicmotivation,whereintrinsicmotivationrefersto“doingsomethingbecauseitisinherentlyinterestingorenjoyable”whereasextrinsicmotivationrefersto“doingsomethingbecauseitleadstoaseparableoutcome”.(RyanandDeci2000)Motivationisnotflat,butamultidimensionalcombinationofvariousaspectsandminor motivators. It is a relation between an actor and activity and cannot be easilygeneralized.

Motivationtypesanddimensions

Intrinsicmotivationisoftenperceivedasthestrongestmotivation.Itwasfoundinpeopleandinanimalsanditisaboutlearning,trying,andplayfulcuriosity.Itcannotbecaused,butratheris facilitatedandcatalyzed.Thereareseveraltypesofextrinsicmotivation.Theyareoftenseenasthealienatedandalmost“negative”types,butthecaseisnotblackandwhite.Sincetheendofourchildhoodtheintrinsicmotivation(carelessexploration,purefunandjoy…)isoftenpushedasidebyextrinsicones(money,status,socialdemands,expectations),butthatdoesn’tmeanthatourlivesaredrivensolelybycarrotsandsticks.

15

Figure5:Visualizationofmotivationcategoriesincluding4typesoftheextrinsicmotivation.

OnFigure5wecanseethedistinctionbetweenamotivation,whichleadstonoaction,fourtypesofextrinsicmotivationandanintrinsicone.Ingeneral,themorewemovetotherightonthediagram,thestrongermotivatorsthereare.Amotivationtowardsanactivitycanshiftleftorrightintime.Thesemotivationtypesareusuallyinterwovenbecauseanactivitycanbedividedintosmallersub-activitieswithcombinationofmotivations.Forexample,anactivityof readingahistorybookcanbedivided into sub-activitieswithdifferentmotivations likesittingdownbecauseof tiredness, learningabouthistory for fun,acquiringknowledge forhistoryclass,finishingabooktocompetewithafriend’sreadingchallenge,etc.

Butmotivationismultidimensionalanditisstronglyconnectedtothethreefollowingaspectsthat play a critical role: Relatedness, Competence and Autonomy. These aspects aremaintainingintrinsicmotivationandenhancingextrinsicmotivationwhenpresent,theycan

Ifoundaniceanalogyofthemotivationtypesinreadingabook.Thiswillprobablyapplyonly to me personally, but I hope it will explain the theory better. I am intrinsicallymotivatedtoreadagoodfantasynovel,becauseIfinditfun.Thereisnoexternaloutcome,justajoy.Ialsoenjoyreadingaboutdesignwhichisstillfun,butpartlyIwanttobeabetterdesignerandchangetheworld(integratedgoal).IhavealsoreadabookaboutSpanish,whichisalanguageIwanttoimprove,becauseIwanttotravelanditwouldbequiteuseful(identification with self-set goal). I have started reading Lord of the Rings, becauseeverybodyaroundmehavealreadydidthatandIwantedtobeincludedinthediscussion(approvalfromothers).Ihavereadfewchaptersofabookfrommandatoryreadinglistinhighschooltonotfailanexam(extrinsicpunishment).Andfinally,IdidnotreadFiftyShadesofGreybecauseIhadnoinnerinterest,itdoesnotalignwithmyvalues,goals,mysocialcircledoesnotappreciateitandIwouldgetnorewardwhatsoever.

16

alsohindermotivationwhenabsent.TheRelatednessisinplaywhentheactivityisvaluedbysignificantothers,whotheactorfeelsorwouldliketofeelconnectedto.Itcouldbefamily,peergrouporasociety.Ifanactorcanrelatetosomebodywhoseesvalueintheactivity,themotivationgrows.ThesenseofCompetenceisaboutbalanceoffeelingefficacioustowardsanaction.Forexample,ifastudentfeelsthatanactivityistoohard(thestudentfeelsnotcompetentenough),themotivationtodoitdecreases.Butontheotherhand,ifthetaskistooeasy,themotivationalsodeclines.Thebiggerthegapbetweenperceivedcompetenceandthedifficultyofatask,thegreaternegativeimpactithasonmotivation.AnotheraspectisAutonomy,wheremoreautonomyleadstoaninternalizationandintrojectionofagoal,meaningthatthetypeofmotivationcanshifttowardstherightsideofthediagramandgetstronger.Iftheactorfeelsthatherbehavioriscontrolled,thenhermotivationismorelikelytodroptoweakertype.

Thismotivationtheoryexplainswellwhyandhowsomeactivitiesaresoengagingandfuntodo, but does not offer any direct tool for motivation enhancement. For this, I chose toresearchafieldofgamification,thatstudiestheengagingaspectsofgames.Itbuildsontopof the extrinsic and intrinsic motivation theories, but offers frameworks and tools forenhancingmotivationasgameelements.

Enhancingmotivationusinggamification

Intrinsicmotivation,asthestrongestmotivator,isallaboutfunandenjoyment,whichisthebasicbuildingblockofgames.BasedonRaphKoster,gamesarebasicallysystemsofpatterns,whichpeopleenjoy learningandmastering.Ourbrain iswired inaway, thattoachieveabettersurvival,itrewardsusforlearningandmasteringpatternsandskills.(Koster2013)

Figure6:JoKim'splayercategories(2012)

Butwhatmakesgamesmorefun,thanreadingatextbook?Thegamificationstudiesrevealthebasicideasbehindgamesandhowtheyworkwithhumanmotivations.Butnoteveryoneenjoyseverygame.AgamedesignerJoKimcreatedaframeworktodividepeopleintofour

17

categories based on their motivations in playing games. (A. J. Kim 2012) And comparingvariousMOOCmotivationstudies,IcanseealinktoMOOCs.AdesiretoCompeteissolvedby earning a certificate with a score mark, to Explore means having “collected” all theadditionalmaterialsandtoCollaboratemeansparticipationindiscussionandorganizationofstudygroups.Weareusuallytryingtocoveralltheaspectstoengageallplayertypes.ItseemsthatthereisaroomfortheExpressmotivatorpossiblyinassignmentsortasksinsideaMOOC.

EverygameisbuiltofGameElements,whichcouldbebrokendowntoapyramidof3levels.Inthehighestlevel,thereareGameDynamics,whichengagethecoremotivationofaplayer.It could be thought as a “grammar” of the game and it is connected to the player typesmentioned above. Dynamics could be about relationships, narratives, progression,exploration… In themiddle areGameMechanicswhich couldbe thought as “verbs” of agame.Theyaretheconcreterulesrepresentingagamedynamic.Thosecouldberewards,transactions,turns,challenges,cooperation,feedback.AndinthelowestlevelareconcreteGameComponentswhicharethemostbasicbuildingblocksofgames.Wecanthinkofthemas “nouns” which are representing and executing mechanics having the higher dynamicabove. These are for example: points, inventory, dice rolls, leaderboards, achievements,progressionbars,bossfights,maps,power-ups,superpowers,avatars,skills,health…

Figure7:Pyramidofgameelements

Engaging games are full of activity loops. These are either engagement loops, where amotivationleadstoanaction,thatgivesafeedbackwhichenhancesmotivation,seeFigure8.And then there is a progression loop, where player should have a sense of progressionthroughthecontent.Thesmallerstepstherearetotake,thelessoverwhelmingthechallengeis.Progressionloopisoftenimplementedasasetofsmallerchallengesdividedbycheckpoints(smallprogressionloops)andendingwitha“bossfight”.Afterdefeatingthelastchallenge,youprogresstoanotherlevel(biggerprogressionloops)andsoon.

18

Figure8:Engagementloop:Aninitialmotivationcausesandaction,whichresultsinafeedbackenhancingmotivationagain.

Therearemanyotherinterestingwaysinwhichgamesenhancemotivationofaplayer,thatcouldbetranslatedtoMOOCs.Forexamplestorytelling(Altarriba2014),whichcouldbeusedto link lectures together andmake the user curious about how it unfolds, progress barsshowinghowmuchtheuserhasaccomplished,collectibleachievementsorbadgesandmuchmore.Butinthecontextofthisthesis,Ichosetodescribeonlythemainprinciples,whichareimportanttounderstandmydesign.

Pitfallsofenhancingmotivation

Whilegamificationintroducestomyresearchaframingoftheconceptoffunandgivesmeaframeworktoenhancemotivation,therearealsopitfalls,thatgreatlydiminishthemotivationand must be avoided. As positive performance feedback enhances the more intrinsicmotivation(e.g.praising“goodjob”,givingthumbsup),anegativefeedbackjustmakestheuserfeellesscompetentandlowersthemotivation(e.g.emphasizingthewronganswersinatest result). Therefore, the activity loopsmustbeexecutedwell to support the feelingofmastery.

Lookingbackattheaspectofautonomy,therearemorethingstobecarefulabout.Examplesofcontrollingbehaviorwhichlimitsautonomyandthereforehindersmotivationare:tangiblerewards,directives,competitionpressure,threatsanddeadlines.Unfortunately,gamificationis often deployed only as points, badges and leaderboards (PBL) focusing on extrinsicmotivationandusingthesepotentiallydangerouscontrollingelements(Trenholm2015).Thiscanleadtoanoverjustificationeffect(Carlsonetal.2009),whereanintroductionofrewardsreplacesintrinsicmotivationbyanextrinsicone.Furthermore,withdrawaloftheserewardsdoesnotshift theextrinsicmotivationbacktoan intrinsicone,butoverall themotivationdiminishesastheserewardsareremoved.

Howothersapproachmotivation?

Gamificationisnowleadingtheonlinemotivationtrends(Deterding2012),hencethereisalotofresourcesavailable.Butitisnoteasytodothegamificationrightandtheinternetisfilledwithmanysimplifiedguidestogamificationwithoutdeeperunderstandingoftheissue.SomereportsexistongamificationincontextofMOOCs,butthisdevelopmentisveryrecentand studies are scarce. A review of several MOOC platforms (Hansch, Newman, andSchildhauer 2015) suggest, that PBL is the most widespread instrument of enhancingmotivation.AdetailedreportfromOpenHPIplatform(Willemsetal.2014)standsasagreatexampleoftheaforementionedreview.

19

TheapproachofOpenHPIplatformispartiallyoneofthose,whichItrytodifferentiatemyselffrom.Theiruseofrewardsisontheedgeofcontrollingbehaviorandthepoints,obtainedbyavarietyofverydifferentactions,arejustemptynumberswithoutanyfurthermeaning.Thepointsdonotrepresentamasteryofauser,theydonotrepresenthelpfulness,theyareablendofeverythingandnothing.Therefore,aleaderboardbasedonthesepointsdoesnotmake sense, too. Unfortunately, they have only designed their system and tested ittheoretically on personas. The actual result of their modifications, while being the mostinterestingpart,isnowheretobefoundevenaftertwoyearsafterpublishing.

AninterestingandinspiringuseofPBLhasStackOverflow3,oftenpraisedasagoodexampleofgamification(Deterding2011).Thepointsandbadgesrepresentdirectlyareputationofauser.Theonlywaytogetrewardsisbeinghelpfultoothersbyansweringquestionsthebestyoucan.Therefore,thegamificationhereemphasizesanaspectofrelatednessandamplifiesstrongertypesofmotivation.

“Likeanythingelse,gamificationcanbedonewellandcanbedonepoorly.Companiesthat

see the success of Foursquare or Farmville and just try to copy what they see, without

understandingthestakeholdersandtheirneeds,aredestinedtofail—asarethosewhothink

they can replace meaningful rewards and incentives with empty points and badges, and

manipulateusersintodoingthingstheydon’twanttodo.”-(Deterding2012)

3MoreonStackOverflowgamificationathttps://blog.codinghorror.com/the-gamification/

Themotivationtheoriessummary:

- Motivationofapersontowardsacertaintaskcouldbeextrinsic(4subtypes)orintrinsic(fun).Themotivationisstrongerthemoreintrinsicitis.

- Motivationismainlyinfluencedby:Relatedness,CompetenceandAutonomy.- Gamificationenhancesmotivationusingprincipleslike:

o Playertypes–differentfunfordifferentpeople.o Activityloops–toshowprogressandenhanceengagement.o Gameelements–frameworksandtoolstoenhancemotivation.

- CurrentresearchexploresuseofgamificationinMOOCplatforms,butoftenonlyPBLisused.Thismightleadtooverjustificationeffectandafailure.

20

MethodsanddataThischapterclarifiessome“how”and“why”questionsconcerningmyresearch.FirstIframethescopeofthisthesistomakeclearwhichaspectsofaMOOCIfocusonandwhichIneglect.Then Idescribemyapproachtotheresearch Ihavedoneandwhatmethods Ihaveused.While using variousmethods throughout the research, I havebeen analyzing thedatabywritingthistextandfindingrelevantliterature.

Theframingofthisproject

AsaprogrammerandwebsiteadministratorofNostis,IcanmakeanymodificationstotheplatformofNostis,thereforeinthecontextofthisworkIchosetoneglectthecontentandfocusontheplatform.Idonotdenytheimportanceofthecontent,contraryIthinkitisthemost important part ofMOOC, but I do not have access to the course lectors, the videoequipment,norIhavecompetencyincreatingaMOOC.Anotheressentialaspectofanonlinecourseisthelearning.Buttolearn,peoplemustvisitthecoursewebsitefirst.Therefore,myfocus is purely on motivation and encouraging particular behavior, while neglecting thepedagogical sideofMOOCs,which is still very interesting,butoutsideof thescopeofmythesis.

Figure9:Analysisofthe15interviewsbyextractinginterestingthemes.

Theexplorationofonlineeducation

Ihavestartedmyresearchatthebeginningof2016togetherwithmyclassmateJakubRybářwithadeskresearchandthen15exploratorysemi-structuredinterviews.Wewerelookingathow people learn online, what services they use and how they use them.Wewere alsotouchingtheirofflinelearningbehavior,butmostofourfocuswasontheirexperiencewithonlinelearningandMOOCs.Ourrespondentswerebetween20and28yearsold–universitystudents or fresh graduates.We conducted Skype interviewswith 9 of our Czech friends

21

rangingfrom30to80minutesand6shorterinterviewsfacetofacewithDanishstudentsorfriends.

Whilegatheringdataandre-listeningtotheinterviews,wecodedthereportedexperiencesintothemesandputthemonawall.Anaturalisticapproachofanalysis(Silverman2013)gaveusbasicoverviewofhowstudentsbehavewhilelearningonlineandtheirroughmotivations.

Thesurvey

TogetbetterunderstandingofusermotivationspecificallyonNostis,Icreatedasurveyforourcourseparticipantswheretheycouldselectmultiplereasonsfortheircourseenrollment,select their job status, field and education. This survey was based on our exploratoryinterviews andothers’ research in this area. (Breslowet al. 2013; Belanger and Thornton2013;Zhengetal.2015)IwantedtocompareouraudiencetootherMOOCsandresearchsoIincludedageintothesurvey.ItisveryeasytodeterminegenderofCzechsusingtheirnamesoIdidnotbotheruserswiththisquestion.

Figure10AlistofsurveyquestionstranslatedtoEnglish.

Respondentswere filling thesurveyduringacourse.Thesurveywasdisplayedat randomlecturehavingonly2questionsfromtheset.ThiswayItriedtominimizethedisruptionofuserflowandbyrandomizationinlecturestogetbalancedresultsoftheprogressionwithinthecourse.About1000usersparticipatedbutnoteveryonefilledallthequestions.Ianalyzedthemotivationquestionsafter158answersanddiscontinuedthispartofthesurveybecauseitdidnotmakesense.

Themotivationinterviews

The motivation results from the survey were fuzzy so I wanted to clear up my insights.BecausetheNostisusersarescatteredallaroundCzechRepublicIconducted4cellphoneinterviewswiththosewhohaverecentlyenrolledintoacourseandre-analyzeonepreviousinterview.While listening to the recordings, I took the constructivist approach, lookingat

- WhyamIhere?� Iwantacertificate� Iwantknowledge� Outofcuriosity� Becauseofwork� Becauseofstudies

- HowoldamI?- Myhighestachievededucationis:

o Primaryschoolo Highschoolo University

- WhatdoIdo?� Study� Work

[Inwhatfield?]

22

varioushints,pausesanddifferentrolesthattheparticipantstookthroughouttheinterview.Ifoundthatthesurveywastoostrictlyimposingourpre-definedcategoriesontotheusers.The motivations from the interviews were blending together having maybe one primarymotivation,butnotdenyingothermotivations.Intheinterviews,Iaskedifpeoplecoulddoanyonlinecoursewhatsoever,whatwoulditbeabout.Mostlytheytoldmeaboutatopicrelatedtotheirwork,whichtheyalsoenjoyedandfoundfuntolearnabout,thuscombiningallthemotivationcategoriestogether.

Theworkshop

IorganizedaworkshopwithstakeholdersfromNugisFinemwhereIwantedtomakesenseofmyresearchsofarandtogatherideasaboutthenextstep.Withthose5people,Istartedbypresentingmydatasofar,whileansweringtheirquestions.Onewastrulyinspiring:

“Dopeoplegothroughtheweeklyblockoflecturesinonesitting?Anddo

theywanttofinishawholecourseinoneweekend?”

WesharedourexperienceswithMOOCs,whatmadeusstayandwhatmadeusabandonit.Basedonourfindings,weideatedwaystomakepeoplestaywithacourse.WethenfocusedonNostisandwhatIcoulddointhescopeofthisthesis.

Observinguserbehavior

AbigpartofmystudyusesquantitativedatagatheredfromNostisdatabaseandanalyticaltools.Nostiscollectsdatesandtimesofeverycompletedlecture,course,earnedcertificateandtestattempt,thereforethereisahugeamountofdataforanalysis.Tomakesenseofallthisdata,IusedadirectSQLcommandstocountnumber,atrialversionofTableausoftwareandMicrosoftExceltovisualizeandanalyzethedata.Ialwaysposedaquestionandthenusedthedatatogetananswer:

- Howoftendopeoplewatchlecturesandinwhatpatterns?- Whichdaysofaweekandwhichtimesofaday?- Dothewatchmorelecturesinaroworjustfewlecturesperday?- Howquicklydopeoplefinishacourse?

Theprototypetesting

With the Motivation helper, I started by sketching the interface on a paper and thenintroducingthelastoneto2ofmyfriends.Imadethemreadthecontentandaskedthem,whatdotheythinktheyarelookingatandhowcouldtheyuseit.Thisusertestsgavemeapositiveresultregardingunderstandabilityandusability.

Ontheotherhand,oneofthemtoldmethatthefunctionalityisnotsufficientandshewouldneedadditionalfeaturesmoreresemblingacalendar.Thismademewonderwhoisexactlythetargetgroupofthishelperandhowbigitis.Idecidedtomakeafull-scaletest–toputtheprototypeonthelivewebsiteandseehowmanyuserswoulduseit.Unfortunately,Igotstuck in development and couldn’t introduce the tool. I describe the prototype in detailincludingthenextstepinthechapterMotivationhelperonpage35.

23

Figure11:Asketchofthemotivationhelperwithroughintroductionofthetoolandaninteractivetablewhereuserscan

selectthetimeofnotification.

Themethodsanddatasummary:

- This project is focused on improvements to MOOC platform Nostis to boostretentionrateusingmotivationtheories.

- Projectstartedwith15exploratoryinterviewsaboutonlineeducation- Thedemographicdataarebasedonasurveyof1000participants;Motivation

questionswereansweredby158people.- Iconducted4interviewstoclarifyusermotivation.- Anideationworkshopwith5stakeholdersshowedadirectionofmyprototypes.- UserbehaviorresultsarebasedondatafromdatabaseandGoogleAnalytics.- Itestedtheprototypesandimprovementswithabout10peopleinshortsessions

gettingtheirimmediateimpressionandfeedback.

24

WhoaretheMOOCusers?ToincreasetheretentionrateinthewayIhavespecified,Ihadtofamiliarizemyselfwithusersand understand their intentions. My main interest was their motivation and goals. Tocomparemy own findingswith different research reports in this area,mostwith Englishspeakingaudience,Iwasalsogatheringdemographicdata.Theresultsweresynthesizedfrom15exploratoryinterviews,anonlinesurveywithNostisusersand4follow-upinterviews.

UsersofNostis

AbriefanalysisofnamesofNostisusersshowsthattherearearound60%womenand40%menregistered.ThemedianageofNostisusersis26,andthehighestachievededucationisdominatedbyhigh school finishers (55%)and followedbyuniversitygraduates (36%)andpeople with basic knowledge have only 9%. But the most important discovery was thelocationofusers.

Figure12:MapofCzechRepublicshowingnumberofsessionsonnostis.org.Thedataarefromtheyear2016wherethe

audiencewasmostlyfromOlomoucarea,followedbySouthernMoravia.

Themaps(Figure12andFigure13)showinadequatedistributionacrossthecountry,whereprimaryregionsareOlomoucandSouthernMoravia. I ignorePragueonpurpose,becausemanyinternetserviceprovidersformobileconnectionarelocatedthere,hencethelocationofthesessionisalsoreportedasPrague.Thiscanhappenonpublicwifisinbusesandtrainsandmost importantly onmobile connections and similar networks without a permanentlocation.

TheregionsofOlomoucandSouthernMoraviaarealsotheregionswheremostoftheNGOmembers live andwork or study. SinceNostiswasmostly promoted by sharing on socialmediaamongfriendsandfamily,theresultsshowthelocationwhereNugisFinemfriendsandfamily are located. Also, Nugis Finem and its project Nostis are well known at PalackyUniversityofOlomouc,thereforealotofvisitsarefromOlomoucitself.

25

Figure13:Themapisfromperiodof16

thJanuary2017to16

thApril2017anditshowssessiondominanceinSouthernMoravia.

Atthebeginningofthisyear,welaunchedacoursewithMasarykUniversityinBrno(SouthernMoravia)asapartofanofficialuniversitycourse.Therefore,mostofthesessionsthisyeararefromBrno,wherestudentswereattendingthiscourseeveryweekfornewcontent.

Looking at the fields of occupation across both students and workers, next to theapproximately 20% of law students, we have a big group of users with social and civiloccupations,whicharestronglytiedtolaws.Iwasunabletofindanytraceofthisamountoflawstudentsinanyotherstudysuggesting,thatthethemeofacourseisbasicallydeterminingtheaudience.

Figure14:Surveyresultsforoccupation.Mostusersarefromthefieldoflaw,butmanyusersalsocomefromsocialorcivil

occupations,wherelawplaysabigrole.

To conclude the demographic study, I can say that the audience of Nostis courses isdeterminedbythetopicofthecourseandmarketingefforts.Thecomparisonwithstudiesfrom world-known platform containing various types of courses therefore did not make

0

10

20

30

40

50

60

70

Workonly Studyonly Work&study

Fieldsofworkorstudy(orboth)

Law Socialworks Civilservice Other Unknown

26

sense.Onepatterndiscovereddidnotmakemuch sense tome, though.Whydo the lawstudents do courses about law? The assumption was, that these users already have theknowledgeandthereforetheyarenotinterestedinNostis.Basedonthedata,Iknewwhatkindofrespondentstochooseforfurtherinterviewsandclarificationofmotivations.

Figure15:Weeklyhoursofcompletion.OnXaxisarehours,onYaxisweekdaysstartingwithSunday

TogetadeeperunderstandingofNostisusers,Ilookedattheirbehaviorpatterns.TheFigure15showspeeksofactivityonSundayafternoon,whendeadlinesareclosing,andonMondayandTuesdayafternoons,whennewcontentisavailable.Thursday,FridayandSaturdayarequitecalmincompletedlecturecount.Takingthistoanextlevel,Icouldobserveandidentifypeople,whoarehighlymotivatedtolearnandwatchlecturesrightwhentheyareavailable,orthose,whostruggletokeepwiththepace.TheFigure16suggests,thattherearetwoverydifferent learningstyles.Somepeoplespreadthe lecturesthroughentiredaysandweeks,completingonelectureatatime,andontheotherhandthereisagroupofpeople,whofinishawholeblockoflecturesinonesitting.

Figure16:Graphshowinglecturecompletionperhouronrandomsampleofusers.Bluelineisonelectureinanhour,yellow

2andthemorered,themorelectureswerecompletedperonehour.Thisdisprovesthehypothesis,thatmostpeoplewatch

severallecturesinarow,butbothbehavioralgroupsaremoreorlessequalindistribution.

27

MotivationofNostisusers

Inthesurvey,therewasamultiple-choicequestionaboutusermotivation.Thisattempttosortusers’motivationintocategoriesusedquestionsfromsimilarstudies(Zhengetal.2015).Asresults,39%ofresponsesincludeddesireforknowledge,24%wantedacertificate,20%wascuriosityand18%wasconnectedtostudyorwork.Myassumptionwas,thatthiswouldclearlydefineusergroups,butreportedmotivationswereblendedtogetherandnopatternwasclearlyvisible

Ahypothesisemergedthatthesurveycouldbemisinterpretedandtheresultsarenotcorrect.IrealizedthatevenIcouldn’tproperlyanswerthequestionandforeveryMOOCmyanswerwouldbeslightlydifferent.Therefore, Idecidedtoclarifythe ideasaboutmotivationwithmoreinterviewsonarandomsampleoffreshcourseparticipants.ThisgavemebetterinsightsintowhatpeoplewanttodoonNostis:

Dana–ApoliceofficermostlydoingpaperworkaboutlawsawNostisinTV,shelookeditupandtriedeverycourseavailable.Shefindsitinterestingbecauseofherworkbutshefindsitusefulinhereverydaylife.Shesuggeststhatit’sgoodtoknowthelawingeneral.

Renata–Formerjob-seeker,finishedschoolrecently,butfoundajobalready.Afriendfroma faculty of law suggested to do an online communismonNostis.Hermotivationwas toincreaseherchancestofindajob,butaftergettingemployed,themotivationhasfallen.

Robert–StudentofLaw,knowsNostisfromschool.Hejustwantedtorefreshhisknowledgesohestartedanonlinecourse.Hewouldliketotakeanothercourseforthesamereason–torefresh his memory to be more confident in his field. A certificate from law course ismeaningless for someone with law degree. This explains the motivation behind our bigpopulationoflawstudents.

Soňa–AteacherwhodidtheFamilyLawcoursebecauseshewastoldfromherdaughtertodo so. The themewas interesting to her and she has learned something throughout thecourse. The certificate has some kind of meaning for her; she has submitted it to heremployer.

Thisleavesmewithaconclusion,thatmotivationisnotbinary,ithasmanyaspectswhicharehardtodifferentiate.Ingeneral,peoplewantfromNostisusefulknowledgefortheirwork,study and daily life. The courses also satisfy curiosity about the topic. Any furthercategorizationwouldbebendingresultstofitartificialboxes.

Whydopeopleleaveandnevercomeback?

Theoverarchingthemeisthatthemoreapersonisinterestedinthetopicofthecourse,themorewillingsheistostickwithit.Andviceversaifthereisalmostnointerestinthetopic,thennothingcankeepthestudentaround.Fromthesideofthestudentitisoftenherlackoftime,procrastinationandvariousdistractions(Facebook,email,othertabsinbrowser…)thatkeepherfromcompleting(Zhengetal.2015).

FromtheMOOCpointofview,therearealsomanyaspectscrucialtotheretentionrate.Inthe exploratory workshop with stakeholders, we used our past experiences and currentresearch(Oakley,Poole,andNestor2016;Zhengetal.2015)tocomeupwithalistofaspects,thatarethemostcrucialtotheretentionrate:

28

Like in an ordinary lecture at school, the most influential and engaging is the lecturer(Adamopoulos2013).Whentheinformationispoorlypresented,itcanputoffeventhemostinterestedusersandviceversa;IftheuserisnotsoengagedwithathemeofaMOOC,thepresentationcanbestillsoentertaining,thatthereisnoreasontoleave.InmostMOOCs,there is a lecturer talking and possibly some kind of presentation in the background.Important is charisma of a lecturer, how clear he speaks and natural body language.Furthermore,betterthanonestaticcameraiswellthoughtmovementsandcutskeepingtheuser’sattentionalongwithapresentationandotherpost-productionimprovements.

Fromthetechnicalpointofview,audioqualityiscriticalbecauseofthespokeninformation,followedbyvideoqualityfortheadditionalnotesonthescreen.Ofcourse,ausermustfindtheMOOC,navigateto lecturesandfill testswithoutanymajorerrorsandfailures.ThesebasicwebsiterequirementsaresummarizedbyŘezáč‘spyramidofwebneeds(2016)basedonMaslow’spyramid.Thebasic idea is, thataservicemustfirstsatisfyneeds infollowingorder: usefulness, discoverability, availability, accessibility, usability and trustworthiness.Onlyaftermostlysatisfyingthoseaspects,theMOOCplatformcanbeusedwithoutcausingfrustration and can climb the pyramid higher to persuasiveness, happiness of usage andmakingabondwiththeuser.

Thetheoryoffun(Koster2013)offersanadditionalreasonswhythefuncandiminishandbecomereplacedbyboredom.FollowingisasummaryofthosecausesincontextofMOOCs–basicallyreplacing“player”by“user”and“game”by“course”:

- Theuserunderstandseverythingthetopiccanofferandthecoursebecomestooeasy.- Theuserdoesnotunderstandthetopicandcannotconnectthedots.Thecourseis

toohard.- Thepacingistooslowandnewchallengescometoolate.Theuserlosesinterestand

iseasilydistracted.- Thepacingistoofastandnewchallengescometooquickly.Theuserisgettinglost

andthecoursebecomesjustanoise.

TheMOOCUserssummary:

- UsersofNostisaredeterminedbymarketingactivitiesandcoursetopics.- Ifoundfollowinggroupsofusers:

o Lawstudentsrepeatingolderknowledgeo Peoplewhoseworkisrelatedtolaw.Theybenefitfromtheknowledgein

theirjobsanddailylives.o Therearealso peoplewhowant to impresspotentialemployerwitha

certificate,peoplewholookonlyforaspecificinformationandmore…- Userbehaviorisconnectedtovaryingmotivations,noteveryonewantstofollow

aMOOCintheclassicalsense.Therefore,alsoonNostisadifferentcalculationofretentionrateisneeded.

- Crucial aspects of engagement besides the MOOC topic are: student’s timecapacities and motivation, lecturer’s presentation abilities, audio and videoquality,camerawork,post-productionandbalanceddifficulty.

29

Optimizationsforrushersandsamplers

“SuchMOOCsshouldalsobeopentopublicatalltimes,allowingstudents

whohaveimmediatelearningneedstoaccessthecourseinstantlyandnot

waitforthenextsessionbeinglaunched.”-suggestedbyZhengetal.

(2015)

Theresearchshows,thatthereisasubstantialgroupofpeoplewhoarenotusingaMOOCinawayitwasdesigned.Iwantedtofindout,whetherthesegroupsappearalsoonNostisandifitwouldbeworthtooptimizetheuserexperiencefortheirgoals.AmongvarioususecaseswethoughtofinaworkshopwithNostisstakeholders,Ifoundtwohypothesistheeasiesttoexploretogetherbyoneexperiment:

- Rushers–people,forwhothecourseistooslowandtheywanttofinishitwholeinjustafewdays.

- Samplers–people,whoaresearchingontheinternetforaspecificinformation,whichcouldbesatisfiedbyasolelectureorasmallpartofawholeMOOC.(Kizilcec,Piech,andSchneider2013).

So far, these use caseswere not supported by Nostis, because by default, lectures fromupcoming blockswere not visible and they only appeared to a user after signing up andwaitingforaweekperblock.Therefore,notevensearchenginescouldseeallthecontentandmany lectureswerebasically invisible.Thereasonbehindthis is that learning ismoreeffective,whendoneinsmallerchunksthroughoutalongertimespan,sotheknowledgehastimeto“absorb”(Koller2012).

Inmyexperimentaldesign,Itriedtokeepthislong-termstudypattern,butalsogivetheuseranoptiontoskipthisrecommendationandtakeandtakeadifferentapproach.Idisplayeverylectureinacourse,souserscanhaveanoverviewofthecontentandsearchenginescanindexthoselectures.ButIaddedalockicontoshow,thatthecontentisprobablynotaccessible.Uponhovering,thelockedlockiconischangedtoanunlockedstate,suggestingapossibilityof interaction.Uponclickingona locked lecture,user ispresentedwithamessageaskingwhethertheuserwantstoholdontotherecommendedpace,orwhethershewantstounlockall content and proceed at her own pace. Every unlocked course was also saved to thedatabaseinordertomeasuretheusageanddeterminetheresultofthisexperiment.

Tomakethelecturesmoreeasilyfindable,inadditiontoservingthemtosearchengines,IaddedatextdescriptiontoeverypagewithlectureinoneofourMOOCs.Therefore,searchengines couldget keywords foreverypageand thendisplay the lecture in search results.Because Nostis already uses Google Analytics, a data about user acquisition could becompared from periods before and after this change and a result of the experimentdetermined.

30

Figure17:AscreenshotofanoriginalandoptimizedversionofNostis.Ontheleftthelecturesinblocksarehidden,onthe

right,therearelockiconsforaquickeraccesstotheMOOCcontent.

Theexperimentresults

TheexperimentwasrunningonNostisforthreemonthsandthedatafromdatabaseshow,thattheunlockfunctionwasused148timesoutof812newenrollments.Thatmeans18%ofusersdiscoveredandusedthisnewfeature.Buttheusercouldhavejustunlockedthecontentbut then follow in normal pace. Therefore, I did a deeper graphical analysis of lecturewatchingpatternsofthose,whohavealreadyearnedacertificate.BasedontheFigure18,Icansay,thatthisexperimentissuccessfullyprovinganexistenceofuserbase,whichwantstofinishacoursefasterthanthedefaultpace.

31

Figure18:During3months,32usersunlockedacourseandlaterearnedacertificate.Thisfigureshowshowmanylectures

werecompletedperdayforthoseusers.Everyrowisauserandtherearetwotimelines(becauseIwasunabletomergeit).

Ontheleft,thereareblacksquaresrepresentingearnedcertificates.Ontherightaretotallecturescompletedperdayincolor

gradients.Bluedotisonelecture,purpleisaround10,pink25,yellow40andthroughorangeto80lecturesinred.Wecan

seethat25peoplefinishedawholecourseinashortertime,thanthecoursewasdesignedfor,proving,thattheintroduced

optimizationmakessense.

Unfortunately,thesecondpartoftheexperiment,searchengineoptimization,didnotyieldasolidresult.ThedatainGoogleAnalyticsshowednoincreaseoftrafficfromsearchenginesdirectly to lectures.Thereasoncould lie in thechosenMOOCaboutbusiness lawandMSExcel. There is a lotof guides, tutorials, articles andothermaterials about this theme, soNostisisusuallynotinthetopresults.Toprovideclearandconclusivedata,thelecturesoftheMOOCneedstohaveauniquecontentandoptimizedtextualdescriptionincontextofpossiblesearchqueries.RecentlyaMOOCplatformCourserastartedofferingcontentsofaGamificationcoursetosamplersthroughsearchengineoptimizationandabackdooraccessproving,thatmyconceptisworthevaluating.

Optimizationsforrushersandsamplerssummary:

- Nostiswasoptimizedfortwouserbehaviorgroups:o Rushers–tounlockandfinishacoursesoonerthanrecommended.o Samplers–togetonlysomeinformation,notfinishingawholecourse.

- Theexperimentshowedapresenceofrushers,whounlockedandfinishedcourseearlier.

- Optimizationforsamplersdidnotworkasexpectedandisinconclusive.Ontheotherhand,Courseraintroducedsimilarfeatureshowingapotential.

- Nostisnowappealstomoreusergroupswithdifferentjobstobedone.

32

TheMOOCOnboardingExperience

“Bad[course],well…Itcanbeactuallygood,butthedescriptioncanbe

bad…”4–Tomáš

Toimprovetheretentionrate,Itackletheproblemofmisunderstandinginthebeginningofacourse.Usersmightleaveacourseforseveralreasons,butinspiredbyaquotefromTomáš,Iwanttominimizethedrop-outforfollowingreasons:

- Thecontentofthecourseisnotwhattheuserthoughtitis.- TheMOOCistoodifficultforausertograsp;theuserismissingsomepriorknowledge.- TheMOOCistootrivial;theuseralreadyhastheknowledge.- TheMOOCtakesmoretime,thanexpected.- Thelecturer,videooraudioqualityofthecourseisnotasexpected.

Byappealingtotherightaudienceanddeterringthewrongone,Iaimtolowerthenumberofenrolled studentsbut increase thenumberofhighlymotivatedones.Themorepeopleknowwhattoexpect,thelessfrustrationarisesandthemorepositiveemotionswillthebrandofNostisgain.Intheend,Icanonlyenhancemotivationofauser,whoisalreadymotivatedtofinishacourse.

ThewireframeinFigure19wasmadeagainwithJakubRybářbasedonourinitialexploratoryresearchandthentestedwithrandomstudentsallarounduniversitycampus.Weaskedthemaboutwhatforthemisimportanttoknowwhentheydecideforacourse(onlineoroffline),thenwepresentedthemoursketchtobesureaboutclarity.IhavethenslightlyimprovedthewireframebasedonthesynthesisofallthedataIhavegatheredincontextofthisthesis.

Thewireframeexplained

Thecoursepagewireframeisdividedintoseveralsectionsbyaline.Thetopsectionisthemost importantonecontainingatitle,veryshortclaimsaboutthecoursetoattractuser’sattention,pictureofthemainlecturerwithanameandherinstitutionandmostimportantlyaplaybuttoninthemiddlesuggestingthereisavideointhebackground.Afteractiviatingthevideo,itcomesintoforegroundandthelecturerexplainswhatthewholecourseisabout.Thisisnotonlyanattractivedescriptionofacourse,butalsoademonstrationofthepictureandsoundqualityof theMOOCandqualitiesof the lecturer. It is important for someusersasearchenginesnottoforgetthetextualdescriptionofacourse,whichIwouldpartiallyhidealsointhetopsection.

“IwanttoseethevideosoIknowthatitisnotuglyandthelecturerdoes

nothaveaboringvoice.5”–Eva

4Czechoriginal:“Blbej,třeba...Onmůžebýtprávědobrej,alemůžebýtjakoblběpopsanej.”5Czechoriginal:“No, jáchcividetotovideo,abytonebylohnusný,nebotenčlověknebylhnusnej,nebonemělblbejhlas.”

33

Figure19:Wireframeofimprovedcoursepage.

34

The second section of the wireframe pinpoints the most important data which are nowmissing on Nostis. There is a short explanation of who the course is for – law students,experiencedcivilengineers,orjustgeneralpublic.Thisisconnectedtoadifficultyleveljustbelow,whichcouldbecombinedtogether.Difficultyrepresentshowmuchconfidenceandknowledgeisrequiredfromtheuseringeneral.Andthethirdessentialinformationisabouttimeneededtocompletethecourse.Itisoptimaltodescribehowmanyhoursperweekthelecturesandassignmentswouldtake,andhowmanyweeksdoesthecourselast.Ontherightisthebuttontostartthecoursewhichemphasizesthecost–free.Somestudentssaid,thattheyhaveassociatedhighqualitycoursewithafee,andafreecoursemeanslowqualitytothem.Hence it is important toshowthequalityof thecourse first in thevideo,and thenemphasizetheno-cost.

Anothertwoblocksarededicatedtoearningthetrustoftheuser.Thereisasmallpreviewofa certificate, list of lecturers and a list of collaborating institutions. These displays ofauthoritieshaveabigimpactonuser’sdecisions.BelowaretestimonialsandratingsoftheMOOCfromusers.Inthebestscenario,thetestimonialsshowwouldbefromasimilaruserbasedonage,occupationorlocation.Themorethepeoplehaveincommon,thehighertrustthereis,nomatterhowrelevanttheinformationis(Cialdini2009).

Atthebottomofthepage,usercanfindlecturesthematicallydividedintoblocks.Thevideosshouldnotbe longer thanapproximately6minutes,becauseusers start to loseattentionaroundthattime.Also,everylectureisaprogressionloopenhancingmotivationasfastasthelectureends.Theselecturesandblocksformcheckpointsshowingtheuserhowmuchshehasaccomplishedandthatthenexttaskisjustanothereasysmallbite.

The wireframe has not yet been implemented, it should highlight the most importantinformationuserneedstosignupforacourse.Topickthemostimportantelement,itwouldbe the top video. User should be able to get all the required information from there;therefore,itiscrucialtomakeitright.

Theonboardingexperiencesummary:

- Usersoftenleaveacoursebecauseoflackoftime,boringlecturer,badaudioorvideoqualityorbecauseoftoodifficultoralreadyknowncontent.

- I created a wireframe tacking these problems by displaying all the requiredinformationupfrontinanintroductionvideoandalsotextuallyonthepage.

- Theaimistogiveenoughinformationtotheuserbeforeenrollingin.

35

TheMotivationhelperThethirdconceptIcreated,themotivationhelper,wasbasedonastakeholderexploratoryandideationworkshop.Other interestingconceptsthatemergedduringtheworkshoparedescribedintheDiscussionchapter.Themotivationhelperutilizesthemotivationtheoriesand ideas fromgamificationto increaseretentionrateofusers. Itwastestedwithseveralparticipants,unfortunatelyarealdeploymentandfollowingquantitativeevaluationwas intheendunsuccessful.

Thelackoftimeproblem

AstudybyNawrotandDoucet(2014)identifiedthebiggestthreattoMOOCengagement-lack of time (“bad time organization, conflicting real life responsibilities, too much timeconsumingcourse”).Myownunderstandingof“lackoftime”isgivingtheactivitylowpriority.IfIabsolutelywanttofinishmydegree,Iwillinvestmytimeintoit.ButifIdonothavemoneyforfood,Imightinvestmoretimeintojobseeking,ratherthanstudying.Inotherwords,Iaminthiscasemotivatedbymyprimaryneedtofeedmyself,ratherthanbeingmotivatedtolearnsomething(andpotentiallydiebyhunger).Therefore,Iseetherealchallengenotonlyas a “lack of time”, but also as a lack of motivation. This is nicely put by one of myinterviewees:

“IfIdonothavethetime,butIknowthatIREALLYneedthecourse,thenI

willjustcreatethetimeforit...”-Tomáš6

Totacklethisproblem,IdesignedaMotivationHelper–atimemanagement&notificationtoolbasedonmotivationtheoriestoencourageuserstostayontrackwithacourse.Afterenrollingintoacourseandwatchingtwolectures,theuserispresentedwithacolor-codedtable(Figure20).ThereareallweekdaysandonYaxistimeperiodsofaday:beforenoon,afternoonandevening.Usersees3pre-selectedfieldsmeaningthathewillbenotifiedatthesetimesto“Keeptrackwithacoursematerialanddeadlines!”Theusercanmodifytheselectionandtimesofnotifications.

Thebasicideaofthistoolisthenotificationmanagement,whereuserisremindedthatthereisalearningtobedone.Itshouldservethosewithtimemanagementproblems,whohaveahardtimerememberingwhattheywantedtodo,orwhogetseasilydistracted.Butthereismuchmorethanthat.Toaddressthe“lackoftime”issue, Ialsowanttoforcetheusertoconsider,whethershehasenoughtimeforthecourse.Thetoolshowshowmuchtimethecoursewillrequireandthenaskstheusertoplanwhentheyhavetimetolearn.

Whymotivationhelper

Goingbacktothemotivationtheory,thetoolkeepsinmindthecoreprinciplesofamotivationenhancers. The user has a complete control over the notification settings, thereforeautonomyispreserved.Ialsotriedtobringintoplaytheaspectofrelatednessintheformofthecolorfulgrid.Usercansee,thatthedarkerthefieldis,themorepopularisthistimeamongotherparticipants.Therefore,usershouldgetthefeeling,thatsheisnotaloneandcanseetheactualbehaviorofherpeers.Thethirdaspectiscompetence,whichIdidnotfocusoninthistool.

6TranslatedfromCzech:“Kdyžnemámčasa...tenkurz,alevím,žefaktjakopotřebuju,taksitenčasjakobyudělám[…]”

36

Figure20:ImplementationoftheMotivationHelper.

Anotherpowerfulaspectissenseofcommitmenttooneself.Withthetool,theparticipantcommitsthatsheisgoingtolearnattheplannedtime.Andahumanbrainlikestosticktoplannedgoalsandstayconsistent.Thisisaremotepartofonetypeofextrinsicmotivation,butmuchdeeperexplanationandelaborationonthistopiccanbefoundinabookInfluenceby Robert B. Cialdini (2009). This commitment should also helpme in differentiating theparticipants,whowant to finish the course the classicalway, andother behavior groups.Although this approachof calculating retention ratehasother flaws, in combinationwithothermeasures,itcanshowinterestingpatternsindisengagement.

Themotivationhelperwasgreatly inspiredbyDuolingoandtheirmobileappnotificationsthatyouget,ifyoudonotpracticeduringaday.ButbecauseNostisdoesnothavenorneedto have a mobile application, I opted for a browser notifications. This way, the user isremindedwhilejustbrowsingtheinternetandifheisofflineandthebrowserisclosed,thenotificationisnotsorelevantanyway.

Figure21:Anexampleofnotificationpersuadingausertocomebacktothecourseandlearn.

37

The notification itself is trying to amplify the user’s engagement by displaying amessagerelevanttotheselectedcourse.Thetextgivestheuserreassurance,thatwhatsheisabouttodoisgoodforher.Totakethistoanextlevel,theusercouldfirstspecify,whatisthereasonbehind taking the course and what is her motivation. Then the message could be trulypersonalized. But that would add another step to the setup making it probably overlycomplicated.

Testingthistoolshowedinterestintheconceptwithsomeusers,otherrequestedadditionalfeatureswhichwouldresembleacalendar.Butthosepeoplearealreadyquiteorganizedandtheyalreadyusesometimemanagementtoolssobuildinganewoneforthemisnottherightwaytogo.Awidertestwouldbeneededtosee,whetherthere isalsoabiggroupof lessorganizedusersforwhothistoolwasdesignedfor.

TheNostismotivationhelpersummary:

- OneofthebiggestMOOCproblemsisusers’lackoftimeandlowmotivation.- Themotivationhelperusesmotivationtheoryandgamificationtoenhanceuser’s

motivation:autonomy,relatedness,commitment,activityloops…- Usersetsupnotificationsatspecifictimeremindinghertolearn.- Ratedpositivelyinqualitativetests,butquantitativetestneededtoprove,that

theconceptisusefulforawideraudience.

38

DiscussionAsineverywork,thereareaspectsthatcouldnotbecoveredmostlyduetotimeconstraintsandmycapabilities.Thischapterdiscussessomeotherperspectives,challenges,directionsandnextstepsofmywork.

Theresearchreflection

Ihaveencounteredacriticismofmyworkregardingonlyexplorativeresearch focusedonprovingordisprovinghypotheses,ratherthangettingexactnumbersandpercentages.Ifoundapproximationsandhintsfromqualitativeresearchassufficienttomakeaconclusion.ButinthecirclesofMOOCcreators, lecturersanduniversity stakeholders,adirectnumbersandprecisemeasurementswouldmakemuchbigger impact.Forexample,whileexploring thepopulationof rushersamongNostisusers, Ididnotknowhowtocomeupwithaprecisenumber.Howtoidentifythesepeople.Those,whounlockedacourse?Those,whounlockedandgotacertificate?Those,whowatchedlecturesfromdifferentblocksinonesession?

Inordertogetbetterarguments,IshouldhavesetupsomeKeyPerformanceIndicators(KPI)andthenmeasurethose.ButthenIwouldfallintothesameproblemsastheretentionratemetricIcriticize.Therefore,IhopethatthisthesiswillbeunderstoodasaworkbroadeningtheviewofMOOCs,notonedescribingtherealityofMOOCs.

Otherconceptsworthexploring

Tocomeupwithaconcepttoworkon,IinvitedthesecondNostisco-creator,theplatformmanagerand3MOOCcreators foran ideationworkshop. In this4-hoursession,wewentthroughcurrentobstaclesforcourseparticipantsandideatedaboutpossibleapproachestotakeinordertoenhanceusermotivation.WehadthreemainideasfromwhichIchosetoworkontheMotivationHelper.Theotherswere:

AdiscussionplatformforNostis,whereuserscouldaskquestionstooneanothercollaborateandsocialize.AlthoughoftenconsideredanessentialpartofMOOCs,Nostisdoesnotyethaveanydiscussionplatformwhatsoever. Inour first course, users couldput commentsundereverylecture,butthisfeaturewasbarelyusedandwestoppedsupportingit.ThefeelingofbeingsociallyconnectedandpartofagroupisapowerfulmotivationtoolinthetheoryofExtrinsicandIntrinsicMotivationandinGamificationandinadditionaparticipants’discussioncouldgiveusavaluablefeedbackonourMOOCs.

On the other hand, current research shows that discussion forums are also a source offrustration, when there are old unanswered questions (Oakley, Poole, and Nestor 2016).Therefore, for a discussion platform to works properly, it requires maintenance frommoderatorsandlecturers,theresourcesNostisdosnothave.Anotherstudypointsout,thatonly3%ofallparticipantsengagedinadiscussion.

Anotherideawasanauto-playfeaturewith…Itwasbuiltonahypothesis,thatmostofthepeoplewatchawholeblockoflecturesinarow.Inmyresearch,Ianalyzedthisbehaviorandbecauseitwasnotmajorityofusers,Idiscardedthisidea.Butasthisgroupofusersisstillaconsiderablepercentagesuggestingthisfeatureisworthtokeepinmind.Intheend,IchosethenotificationtoolalsobecauseIwasinterestedinthetechnology,notbecauseitwouldbeobjectivelythebestchoice.

39

Themotivationhelper’snextstep

Unfortunately,IcouldnottestthenotificationfunctionoftheMotivationHelperprototypeinalong-termexperiment,becausethecurrenttechnologyinwebbrowsersdoesnotsupporttimingofnotifications.Thereisaworkaround,whereattheselectedtime,theserverwouldsendapushnotificationtothebrowser,whichthendisplaysthenotification.Butthiswayismuchmoredemandingoninfrastructureandoutofmypossibilities.

Fromthefewtests,Icanseethatthereisapotentialinthistooltohelpsomeusersinplanningtheir time and finishing a course. But I do not know if potential users are inmajority orminority.Therefore,Isuggestasanextstepafake-doortest7–sendingnewusersanemailwithalinktotheprototype.Butsincethefunctionalityisnotthereyet,astheyclick“savenotifications”,theywouldbepresentedwithanapology,thatthisfeatureisjustatest.Inthisway,Icouldobservehowmuchwouldthistoolbeusedandhowdopeopleinteractwiththeinterface.

7Awaytotestwhetheraserviceiswantedornot.Wedescribetheservicetoauser(inemailoronwebpage)andmeasurehowmanyclickedonanactionbutton(likeregister,signuporsetupnotifications).

40

ConclusionMOOCusershavevariousmotivationstoenrollintoaMOOCanddifferentbehavioroncetheydo.From19interviewsandasurveystudy,Ifoundthatsomeuserswanttogothroughalllecturesandtestsandfinishthewholecoursewithacertificate,somedon’twanttobotherwithassignmentsandonlywatchlectures,andsomeareonlylookingforacertaininformationand then they leave. But retention rate, which is understood as a user success rate, iscalculatedasnumberofcertificatesdividedbynumberofusersenrolled.Theproblemis,thatthismeasurementisfocusedonlyonasubpopulationofMOOCusers,whowanttocompletethecourse.AlthoughausermightseeasampleofMOOC,gettheinformationneededandthenleavesatisfied,theretentionraterepresentsthisuserasfailure.Therefore,thewholeretentionrateisflawed.

Mydesign is focusedon improvingretentionrateonNostisusingmotivationtheoriesandgamification. The goal was to make a feasible prototype which could be then deployeddirectlytotheliveserverasanewfunctionality.IdesignedthreeimprovementstoNostis:Anoptimizationforrushersandsamplers,aredesignedcoursemainpageandtheMotivationHelper.

Rushersarethepeoplewhowanttofinishacourseinfewdays,insteadofweeks,rushingtowardsthecertificate.Sofaritwasimpossibleduetogradualunlockingofanewcontenteveryweek.Icreatedanexperiment,wherethewholecontentofacourseisshownandbyaquickinteractionuserscanunlockallthecontentofacourse.Thisexperimentprovedtobesuccessful showing that there is a subpopulation of users who unlocked the course andfinished sooner than usual. The samplers optimization is based on making all lecturesaccessible and searchable through search engines. But due to a poor set up of thisexperiment,Icouldnotconcludeexistenceornonexistenceofsamplersnormaketheiruserexperiencebetter.

Myresearchshowedthatthecoursemainpageisthepointwherepeopledecidewhethertoenrollintoacourse,butoftenthepagelacksnecessaryinformationwhichisdiscoveredafterenrollmentandleadstodroppingout. IsynthesizedthereasonswhypeopleleaveMOOCsandthekeyinformationtheyneedtomakeaninformeddecisionandredesignedthecoursemain page.User tests proved, that thewireframe is on the right track. Furthermore, if apersonknowsfromthestart,thatthecourseisnotwhattheyneed,theydonothavetimeforit,oritistoodifficult,theywillnotenroll.Thisgoesagainsttheideatogetasmanyusersaspossible,butwhat is thebenefitofattractingauserwhichgets frustrated lateronandleavesanyway?

TheMotivationHelperisanewfeatureofferingnotificationstouserstohelpthemstayontrack.Ausercansetupnotificationsatchosentimesforcinghertothinkaboutherweeklyscheduleandmakingacommitmenttoherself.Itisbuiltontheoryofextrinsicandintrinsicmotivation,emphasizingautonomyandrelatednesstostrengthenthe initialmotivation. Italsousesactionloopsfromthefieldofgamificationandsomegameelements.Althoughfewqualitative testsproved there isapotential,a full-scalequantitative test is stillneeded toassessthesuccessofthetool.

41

ReferencesAdamopoulos,Panagiotis.2013.“WhatMakesaGreatMOOC?AnInterdisciplinaryAnalysis

ofStudentRetentioninOnlineCourses.”In,2013:21.AIS.

Altarriba,Ferran.2014.“TheRevolutionofFun:AStudyofAppliedGamesandFuninNon-Entertainment Contexts.” BA (Hons) Interaction Design dissertation.http://www.ferranaltarriba.com/docs/papers/14_sp_rf.pdf.

Belanger,Yvonne,andJessicaThornton.2013.“Bioelectricity:AQuantitativeApproachDukeUniversity’s First MOOC (2013).”http://dukespace.lib.duke.edu/dspace/handle/10161/6216.

Breslow,Lori,DavidEPritchard,JenniferDeBoer,GlendaSStump,AndrewDHo,andDanielTSeaton.2013.“StudyingLearningintheWorldwideClassroom:ResearchintoedX’sFirst MOOC.” Research & Practice in Assessment 8.http://search.proquest.com/openview/27e5ba80e54284de770427a71f05bf8d/1?pq-origsite=gscholar.

Carlson, Neil R, Donald Heth, Harold Miller, John Donahoe, and G Neil Martin. 2009.Psychology:TheScienceofBehavior.Pearson.

Christensen, ClaytonM, TaddyHall, KarenDillon, andDavid SDuncan. 2016. “KnowYourCustomers’‘JobstoBeDone.’”HarvardBusinessReview9:54–62.

Cialdini,RobertB.2009.Influence:ScienceandPractice.5thed.Boston:PearsonEducation.

Deterding, Sebastian. 2011. “Meaningful Play. Getting »Gamification« Right.” January 31.https://www.slideshare.net/dings/meaningful-play-getting-gamification-right.

Deterding,Sebastian.2012.“Gamification:DesigningforMotivation.”Interactions19(4):14.doi:10.1145/2212877.2212883.

Glass, Chris R., Mitsue S. Shiokawa-Baklan, and Andrew J. Saltarelli. 2016. “Who TakesMOOCs?” New Directions for Institutional Research 2015 (167): 41–55.doi:10.1002/ir.20153.

Hansch,Anna,ChristopherNewman,andThomasSchildhauer.2015.“FosteringEngagementwithGamification:ReviewofCurrentPracticesonOnlineLearningPlatforms.”SSRNElectronicJournal.doi:10.2139/ssrn.2694736.

Ho,AndrewDean,JustinReich,SergiyONesterko,DanielThomasSeaton,TommyMullaney,JimWaldo,andIsaacChuang.2014.“HarvardXandMITx:TheFirstYearofOpenOnlineCourses, Fall 2012-Summer 2013.” SSRN Electronic Journal.doi:10.2139/ssrn.2381263.

Hood, Nina, Allison Littlejohn, and Colin Milligan. 2015. “Context Counts: How Learners’ContextsInfluenceLearninginaMOOC.”Computers&Education91(December):83–91.doi:10.1016/j.compedu.2015.10.019.

Karsenti,Thierry.2013.“WhattheResearchSays.” InternationalJournalofTechnologiesinHigherEducation10(2):23–37.

Kim,AmyJo.2012.“SocialEngagement:Who’sPlaying?HowDoTheyliketoEngage?”Amy

Jo Kim. September 19. http://amyjokim.com/blog/2012/09/19/social-engagement-whos-playing-how-do-they-like-to-engage/.

42

Kim,Joshua.2012.“PlayingtheRoleofMOOCSkeptic:7Concerns.”InsideHigherEd.May21.https://www.insidehighered.com/blogs/technology-and-learning/playing-role-mooc-skeptic-7-concerns.

Kizilcec,RenéF.,ChrisPiech, andEmily Schneider.2013. “DeconstructingDisengagement:AnalyzingLearnerSubpopulations inMassiveOpenOnlineCourses.” In ,170.ACMPress.doi:10.1145/2460296.2460330.

Koller,Daphne.2012. “DaphneKoller:WhatWe’reLearning fromOnlineEducation.”TED.June.https://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education?language=en#t-672691.

Koster,Raph.2013.TheoryofFunforGameDesign.O’ReillyMedia,Inc.

Kulkarni,Chinmay,KohPangWei,HuyLe,DanielChia,KathrynPapadopoulos,JustinCheng,Daphne Koller, and Scott R Klemmer. 2015. “Peer and Self Assessment inMassiveOnlineClasses.”InDesignThinkingResearch,131–168.Springer.

Laurillard,Diana.2014.“FiveMythsaboutMOOCs.”TheTimesHigherEducation.

“MOOC.” 2017. Oxford Dictionaries. Oxford University Press. Accessed April 14.https://en.oxforddictionaries.com/definition/mooc.

Nawrot, Ilona, and Antoine Doucet. 2014. “Building Engagement for MOOC Students:IntroducingSupportforTimeManagementonOnlineLearningPlatforms.”In,1077–82.ACMPress.doi:10.1145/2567948.2580054.

Oakley,BarbaraAnn,DebraPoole,andMaryAnneNestor.2016.“CreatingaStickyMOOC.”OnlineLearning20(1).doi:10.24059/olj.v20i1.731.

Pappano,Laura.2012.“TheYearoftheMOOC.”TheNewYorkTimes2(12):2012.

Pardee,RonaldL.1990.“MotivationTheoriesofMaslow,Herzberg,McGregor&McClelland.A Literature Review of Selected Theories Dealing with Job Satisfaction andMotivation.”ERIC.https://eric.ed.gov/?id=ED316767.

Řezáč, Jan. 2016. Web ostrý jako břitva: návrh fungujícího webu pro webdesignery a

zadavateleprojektů.

Ryan, Richard M., and Edward L. Deci. 2000. “Intrinsic and Extrinsic Motivations: ClassicDefinitionsandNewDirections.”ContemporaryEducationalPsychology25(1):54–67.doi:10.1006/ceps.1999.1020.

Silverman,David.2013.DoingQualitativeResearch:APracticalHandbook.SAGEPublicationsLimited.

Trenholm,Zen.2015.“Points,Badges,andLeaderboards:TheGamificationFallacy.”Yu-KaiChou: Gamification & Behavioral Design. January 2.http://yukaichou.com/gamification-study/points-badges-and-leaderboards-the-gamification-fallacy/.

Wildavsky, Ben. 2014. “Evolving Toward Significance or MOOC Ado About Nothing?”InternationalEducator23(3):74–79.

Willems,C.,N.Fricke,S.Meier,R.Meissner,K.A.Rollmann,S.Voelcker,S.Woinar,andC.Meinel. 2014. “MOTIVATING THE MASSES - GAMIFIED MASSIVE OPEN ONLINE

43

COURSES ON OPENHPI.” In EDULEARN14 Proceedings, 4042–52. 6th InternationalConferenceonEducationandNewLearningTechnologies.Barcelona,Spain:IATED.

Zheng,Saijing,MaryBethRosson,PatrickCShih,andJohnMCarroll.2015.“UnderstandingStudentMotivation,BehaviorsandPerceptionsinMOOCs.”InProceedingsofthe18thACM Conference on Computer Supported Cooperative Work & Social Computing,1882–1895.ACM.

MOOC Retention Rate and Motivation - Ondrej Henek › x ›...

Documents

Transcript of MOOC Retention Rate and Motivation - Ondrej Henek › x ›...