Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale
-
Upload
daniel-davis -
Category
Education
-
view
192 -
download
0
Transcript of Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale
![Page 1: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/1.jpg)
FOLLOW THE SUCCESSFUL CROWDRAISING MOOC COMPLETION RATES THROUGH SOCIAL COMPARISON AT SCALE
DAN DAVIS IOANA JIVET RENÉ KIZILCEC LAMBDA-LAB
GUANLIANG CHEN CLAUDIA HAUFF GEERT-JAN HOUBEN
![Page 2: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/2.jpg)
MOOC LEARNING
![Page 3: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/3.jpg)
FEEDBACK
![Page 4: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/4.jpg)
SOCIAL CUES SELF-REGULATED LEARNING METACOGNITION
MOTIVATION
![Page 5: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/5.jpg)
OUTLINE 2. EXPERIMENTS3. CULTURE4. CONCLUSION
1. DESIGN
![Page 6: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/6.jpg)
1. DESIGNFEEDBACK · SOCIAL COMPARISON
![Page 7: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/7.jpg)
SOCIAL COMPARISON THEORYEstablish social and personal worth by comparing ourselves to others (Festinger, 1954)
Comparison with slightly-better-performing peers has a beneficial effect on students’ grades (Huguet, 2001)
![Page 8: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/8.jpg)
ROLE MODELSlearners who earned a passing grade in the previous edition of the MOOC
![Page 9: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/9.jpg)
FEEDBACK SYSTEM DESIGN
![Page 10: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/10.jpg)
DESIGN GOALSprovide feedback on behavior extracted from edX’s log traces afford learners the ability to intuitively understand and explore the information presented (Kay) show learners specific ways to take action and change their behavior
Traceable
Scrutable
Actionable
![Page 11: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/11.jpg)
Learning tracker
Average graduate You kAverage graduate next week
Time on theplatform
Time watchingvideos
Fraction of timespent watchingvideos while onthe platform
Videos watched
Quiz answerssubmitted
Timeliness ofquiz answersubmission
![Page 12: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/12.jpg)
FEEDBACK SYSTEM DESIGN
1. Embodied representation of multiple metrics 2. Numerous indicators displayed in a small space 3. Easily comparable
![Page 13: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/13.jpg)
ITERATIVE DESIGN1. Interactivity 2. Planning ahead 3. Framing
Framing
Feedback Dimension
Planning Ahead
Interactive
![Page 14: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/14.jpg)
2. EXPERIMENTSRESEARCH QUESTIONS · RESULTS
![Page 15: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/15.jpg)
RQ1 Does social comparison-based learner feedback lead to increased achievement and increased self-regulatory behavior for MOOC learners?
![Page 16: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/16.jpg)
Water Treatment
Urban Sewage
Responsible Innovation
Pre-Calculus
11k
8k
2k
12k
3.5%
3.8%
3.6%
1.7%
42
272
79
327
MOOC ENROLLED PASS QUIZQUESTIONSRATE
![Page 17: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/17.jpg)
5,460
5,483
—
Control
Treatment 1
Treatment 2
4,038
4,099
—
1,184
1,168
—
4,142
4,087
4,065
WaterX SewageX BusinessX CalcX
ALL ENROLLED
![Page 18: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/18.jpg)
WaterX SewageX BusinessX CalcX
1,268
1,251
—
Control
Treatment 1
Treatment 2
771
746
—
164
160
—
1,150
1,147
1,118
ACTIVE: >5 MIN IN COURSE
![Page 19: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/19.jpg)
RESULTSCOMPLETION · ENGAGEMENT · PRIOR EDU
![Page 20: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/20.jpg)
COURSE COMPLETION
![Page 21: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/21.jpg)
WaterXC 1,268 160
MOOC COND. N # PASS PASS RATE
T 1,251 18812.6%15.0%
UrbanXC 771 136T 746 165
17.6%22.1%
BusinessXC 164 46T 160 54
28.0%33.8%
OVERALLC 2,203 342T 2,157 407
15.5%18.9%
![Page 22: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/22.jpg)
WaterXC 1,268 160
MOOC COND. N # PASS PASS RATE
T 1,251 18812.6%15.0%
UrbanXC 771 136T 746 165
17.6%22.1%
BusinessXC 164 46T 160 54
28.0%33.8%
OVERALLC 2,203 342T 2,157 407
15.5%18.9% **
![Page 23: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/23.jpg)
WaterX UrbanX BusinessX Overall
Control
Treatment
10%
20%
30%
COURSE COMPLETION
![Page 24: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/24.jpg)
RQ1 Does social comparison-based learner feedback lead to increased achievement and increased self-regulatory behavior for MOOC learners?
Providing learners a comparison of their own behavior to that of previously successful peers will increase learner achievement.
H1.1
![Page 25: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/25.jpg)
RQ1 Does social comparison-based learner feedback lead to increased achievement and increased self-regulatory behavior for MOOC learners?
Providing learners a comparison of their own behavior to that of previously successful peers will increase learner achievement and engagement
H1.1
Learners will change the aspects of their behavior that the Feedback System makes them aware of.
H1.2
X
![Page 26: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/26.jpg)
RQ2 Which learners benefit most from the Feedback System?
![Page 27: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/27.jpg)
PRIOR EDUCATION
![Page 28: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/28.jpg)
WaterXHigh 13.2
MOOC PRIOR
Low 11.815.711.5
UrbanX17.416.4
21.314.0
BusinessX23.721.4
29.122.8
OVERALL16.314.4
19.513.6
HighLowHighLowHighLow
EDU
0.150.16
0.040.66
0.040.78
<0.010.36
CTRL TREAT pGRADE GRADE
![Page 29: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/29.jpg)
WaterXHigh 13.2
MOOC PRIOR CTRL TREAT
11.815.711.5
UrbanX17.416.4
21.314.0
BusinessX23.721.4
29.122.8
OVERALL16.314.4
19.513.6
High
High
High
EDU p
0.150.16
0.040.66
0.040.78
<0.010.36
<
<
<
<
GRADE GRADE
Low
Low
Low
Low
![Page 30: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/30.jpg)
RQ2 Which learners benefit most from the Feedback System?
![Page 31: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/31.jpg)
RQ2 Which learners benefit most from the Feedback System?
A: HIGHLY EDUCATED LEARNERS
![Page 32: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/32.jpg)
3. CULTURE*FRAMING · RESULTS
* PRE-REGISTERED osf.io/ys6au/
![Page 33: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/33.jpg)
RQ3Does feedback framed in line with a learner’s cultural context make it more effective?
![Page 34: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/34.jpg)
Framing
Feedback Dimension
Planning Ahead
Interactive
![Page 35: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/35.jpg)
ON TRACK
![Page 36: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/36.jpg)
ON TRACK
![Page 37: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/37.jpg)
BEHIND
![Page 38: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/38.jpg)
Framing
Feedback Dimension
Planning Ahead
Interactive
![Page 39: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/39.jpg)
CULTUREHOFSTEDE INDIVIDUALISM
![Page 40: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/40.jpg)
INDIVIDUALISM
INDIVIDUAL AS AN INDEPENDENT ACTOR WITH LOOSE SOCIAL RELATIONS
COLLECTIVISM
![Page 41: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/41.jpg)
CLOSELY KNIT SOCIAL RELATIONS AND GREAT CARE FOR THE WELL-BEING OF ALL
INDIVIDUALISMCOLLECTIVISM
![Page 42: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/42.jpg)
CULTUREHOFSTEDE INDIVIDUALISM
GELFAND TIGHTNESS
![Page 43: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/43.jpg)
HAVE MANY STRONGLY ENFORCED RULES AND LITTLE TOLERANCE FOR DEVIANCE
TIGHTLOOSE
![Page 44: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/44.jpg)
HAVE FEW STRONGLY ENFORCED RULES AND GREATER TOLERANCE FOR DEVIANCE
TIGHTLOOSE
![Page 45: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/45.jpg)
RQ3Does feedback framed in line with a learner’s cultural context make it more effective?
![Page 46: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/46.jpg)
RQ3Does feedback framed in line with a learner’s cultural context make it more effective?
Feedback in line with a learner’s cultural individualism index will increase its impact on the learner
H3.1
X
![Page 47: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/47.jpg)
RQ3Does feedback framed in line with a learner’s cultural context make it more effective?
Feedback in line with a learner’s cultural individualism index will increase its impact on the learner
H3.1
Feedback in alignment with a learner’s cultural tightness will increase its impact on the learner
H3.2
X
![Page 48: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/48.jpg)
WaterX
MOOC PASS RATE
13.8%
UrbanX 19.8%
BusinessX 31%
CalcX 5%
![Page 49: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/49.jpg)
RAISE ACTIVITY THRESHOLD5 MINUTES -> 1 HOUR
From N = 3,415 to N = 658
![Page 50: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/50.jpg)
WaterX
MOOC PASS RATE
13.8%
UrbanX 19.8%
BusinessX 31%
CalcX 16%
![Page 51: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/51.jpg)
INDIVIDUALIST FRAMING+ Activity Threshold
Increased completion rates across cultural contexts
Control: 12.8%Individualist Framing: 19.9%
55% Increase (t = 2.02, p = 0.04, N = 441)
![Page 52: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/52.jpg)
INDIVIDUALIST FRAMING+ Activity Threshold
Increased completion rates across cultural contexts
Control: 12.8%Individualist Framing: 19.9%
55% Increase (t = 2.02, p = 0.04, N = 441)
Increased completion rates especially in tight culturesControl: 12.1%Individualist Framing: 36.3%
300% Increase (t = 2.07, p = 0.04, N = 55)
![Page 53: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/53.jpg)
5. CONCLUSIONWHAT’S NEXT
![Page 54: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/54.jpg)
1. REAL-TIME 2. FRAMING 3. PERSONAS
![Page 55: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/55.jpg)
1. REAL-TIME 2. FRAMING 3. PERSONAS
![Page 56: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/56.jpg)
1. REAL-TIME 2. FRAMING 3. PERSONAS
![Page 57: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/57.jpg)
THANK YOU
BIT.LY/WIS-LEARNING-ANALYTICS
LAMBDA-LAB
DAN DAVIS IOANA JIVET RENÉ KIZILCEC
GUANLIANG CHEN CLAUDIA HAUFF GEERT-JAN HOUBEN
![Page 58: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/58.jpg)
LOW CULTURAL TIGHTNESS PREDICTS COMPLETION
p < 0.02
![Page 59: Follow the Successful Crowd: Raising MOOC Completion Rates through Social Comparison at Scale](https://reader034.fdocuments.in/reader034/viewer/2022042611/58d0fb541a28abc00b8b6181/html5/thumbnails/59.jpg)
Control 1,150 45
CONDITION N # PASS PASS RATE
3.9%
Indiv. - Promotion 1,147 62 5.4%
Collect. - Prevent. 1,118 51 4.5%
OVERALL EFFECT