20 Years: What the Data Show and What They Don't (287820054)
description
Transcript of 20 Years: What the Data Show and What They Don't (287820054)
![Page 1: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/1.jpg)
20 Years: What the Data Show and What They Don’t
Chuck Dziuban & Patsy Moskal
Center for Distributed LearningResearch Initiative for Teaching Effectiveness (RITE)
University of Central Florida
![Page 2: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/2.jpg)
About UCF• Orlando, FL• Over 63,000 students; 2nd largest university in U.S.• Carnegie classification: RU/VH Research University• 216 degree programs; 11 colleges; 11 campuses
• Academic Year 2014-15• 38% of total university student credit hours• 78% of all students took at least one online course
• 80% of all undergraduates (47,116)• 61% of all graduate students (6,469)
![Page 3: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/3.jpg)
The Evaluation PlanStudents
Faculty Institution
![Page 4: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/4.jpg)
Where’s the data??
LMS
ApplicationsSPI
SIS
Surveys
![Page 5: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/5.jpg)
The end goal…
Turning data…
…into INFORMATION
![Page 6: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/6.jpg)
Some Lessons Learned Through the Years
![Page 7: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/7.jpg)
Success
![Page 8: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/8.jpg)
Success
Summer 12 Fall 12 Spring 13 Summer 13 Fall 13 Spring 140
102030405060708090
10090 87 88 91 87 88
94 90 92 94 91 9089 87 90 90 89 89
F2F (n=647,390) Blended (n=73,629)Fully Online (n=189,208)
Perc
ent
![Page 9: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/9.jpg)
1. Does success equal learning?• Answer: No
2. Are the differences significant?• Answer: Yes
Success
![Page 10: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/10.jpg)
Question that was never asked and should have been…
• Is there selection bias in your data?• Answer: Yes
![Page 11: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/11.jpg)
Significance may not be significant
![Page 12: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/12.jpg)
The question for all ages
“How large does my sample have to be?”
![Page 13: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/13.jpg)
Statistical (classical) hypothesis tests are a function of 3 things:
1) Significance Level .05? .01? …or something else?
2) Sample Size
Tiny? Small? Medium? Large? Huge?3) Some Effect Size
A difference that means something to me∆1 –Doesn’t matter
∆2 –Really important to me
![Page 14: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/14.jpg)
How much is enough?
I don’t care about this
∆1
![Page 15: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/15.jpg)
How much is enough?
∆2I care
about this
![Page 16: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/16.jpg)
Statistical significance testing (SD = 15)Sample Size
27502500225020001750150012501000750500
x1=100 x2=101 ES=.06.01.02.03.04.05.07.10.14.20.29
Significant
NotSignificant
![Page 17: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/17.jpg)
So the strategy is…
1) Pick ∆2 first This is important to me
2) Then pick a significance level .05, .01, or something else
3) Pick a sample size that will catch ∆2 but not ∆1
![Page 18: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/18.jpg)
When treatment is not treatment
![Page 19: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/19.jpg)
Blended learning as a boundary object
Blended Learning
Evaluators Journalists
Students
Faculty
DeansLibrarians
Provosts
![Page 20: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/20.jpg)
The way we blendOnline tutorials
E-mails
WEB MODULES
CoMpUtEr LaBs
Optional websites
Tele-web software
Computer cluster room
Internet tutorials
Electronic field trips
Supplemental web
activities
Tech equipped classrooms
Evidence-based practice
Discussion
Virtual experiments
![Page 21: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/21.jpg)
The way we measure
MuLtIpLe ChOiCe
Class ranks
student ratings
Class work
Placement tests
Motivation questionnaires
Achievement tests
Cooperativeness scales
Mathematics anxiety scales
Self-confidence scales
Self-report surveys
APPREHENSION TESTSReading
power
Conceptual testsElectric circuit tests
![Page 22: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/22.jpg)
Can you predict success and
bandwidth analytics?
![Page 23: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/23.jpg)
Add one logistic regression analysis for predicting non-success DFW (n=258,212)
R2
Modality .003Course Level .022Class Size .024Gender .029Ethnicity .035Age .035SAT .034College .047High School GPA .074Cumulative GPA .405
![Page 24: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/24.jpg)
D* 1
Classification and regression tree for predicting non success in online courses
Overall Non Success
D* 3-10
D* 2
13.3%n = 70948
34.3%n = 39646
7.5%n = 31302
24.4%n = 14070
44% n = 25576
* Deciles Prediction Accuracy 94%
D* 1-2
![Page 25: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/25.jpg)
•Can you take action on GPA?•Answer: Yes and No
•Why does GPA predict?
Scarcity and the cognitive bandwidth tax
![Page 26: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/26.jpg)
Could it be scarcity?
![Page 27: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/27.jpg)
Scarcity and the cognitive bandwidth taxC
ogni
tive
Ban
d W
idth
Scarcity Factors
Tuition Housing Books Transportation Child Care
Work Add. Cost
Safety
Scarcity
![Page 28: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/28.jpg)
Scarcity and the cognitive bandwidth taxC
ogni
tive
Ban
d W
idth
Scarcity Factors
Tuition Housing Books Transportation Child Care
Work Add.Cost
Safety
Cognitive Bandwidth
![Page 29: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/29.jpg)
Is there an answer to scarcity?
![Page 30: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/30.jpg)
Harris Rosen
![Page 31: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/31.jpg)
www.tangeloparkprogram.com
![Page 32: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/32.jpg)
Program Components
Early Childhood Education
ParentLeadership
ScholarshipsVocational
Alumni
![Page 33: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/33.jpg)
Tangelo Park Crime Rates 1994-2013Standardized by 1993 Figures
94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 13
Crim
e R
ates
0
20
-20
-40
-60
-80
+3% +4%
-38%-30%
-4%
-33%
-38%-45%
-49%
-67%-61%
-35%-47%
-53%
-45%
-52%
-48%-53%-66%
-66%
![Page 34: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/34.jpg)
Completion rates for those who entered
Community college Vocational school College/ university Graduate school0
102030405060708090
100
32
8377
83
Perc
ent
![Page 35: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/35.jpg)
94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 09 10 11 12 130
102030405060708090
100
0 0
2836
5867 70 70 68 72 87 76
8288 87 86
76 73 74 74
100
7264
4233 30 30 32 28
1324
1812 13 14
24 27 26 26
Perc
ent
2-yr-old program
Scholarship
TPP Budget: 2-yr-old program and scholarship (percent of total)
![Page 36: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/36.jpg)
Return on Investment to Society
• Total 20 year investment: $10,000,000Half for Early Childhood programHalf for scholarships
• Lance Lochner, University of Western Ontario
• Return on investment: $7 for every $1 spent
![Page 37: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/37.jpg)
Come together blended research
![Page 38: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/38.jpg)
Louis Guttman meets Brené Brown meets the Center for Fiction
Literary Criticism
Image Analysis
Survey
Grounded Theory
![Page 39: 20 Years: What the Data Show and What They Don't (287820054)](https://reader036.fdocuments.in/reader036/viewer/2022062502/577ca7d11a28abea748c9275/html5/thumbnails/39.jpg)
Research Initiative for Teaching Effectiveness
For more information contact:Dr. Chuck Dziuban
(407) 823-5478 [email protected]
Dr. Patsy Moskal(407) 823-0283
http://rite.ucf.eduhttp://www.if.ucf.edu/