Jasig sakai2012 silva-lugo

31
The Advantages of Monitoring Sakai Usage By Jose L. Silva-Lugo Carl J. Perry John R. Boekenoogen e-Learning Support Services Office of Academic Technology University of Florida

Transcript of Jasig sakai2012 silva-lugo

Page 1: Jasig sakai2012 silva-lugo

The Advantages of Monitoring Sakai Usage

By

Jose L. Silva-LugoCarl J. Perry

John R. Boekenoogene-Learning Support Services

Office of Academic Technology University of Florida

Page 2: Jasig sakai2012 silva-lugo

04/14/2023

2

Introduction

The University of Florida (UF)’s contract with Blackboard WebCT was going to end in October 2010.

In Spring of 2009, UF came up with four options: Sakai, Blackboard, Moodle, and Angel.

UF ran open meetings with each vender.

UF set up test sites for faculty, staff, and students.

UF selected Sakai

Page 3: Jasig sakai2012 silva-lugo

04/14/2023

3

Introduction: Planning, Testing, & Implementation

2009 2010 2011 201

2

Process Dec Spr Sum Fall Spr Sum Fall Spr

Planning

Pilot Study

Migration

Introd. to UF

Roll Out

Workshops

Open Labs

Webinars

Page 4: Jasig sakai2012 silva-lugo

04/14/2023

4

Introduction: Training & Assistant Offer to UF

• Getting Started• Moving On• Evaluating knowledge with online assessment

• Begin a new semester• End a semester

Workshops

• 1 h consultation per week• There is no need of appointment• Every Tuesdays through the semester

Open labs

• 30 minutes per week• Every Wednesday through the semesterWebinar

Page 5: Jasig sakai2012 silva-lugo

04/14/2023

5

Introduction: The Need of Monitoring Sakai Usage

Even though the effort of promoting Sakai carried out from summer 2010 to 2011, we did not have the expected attendance.

We decided to look our records to: Find out the percentage of courses, sections, and

instructors using Sakai per college/year Identify the colleges with percentages less than 50% used

of Sakai Plot the percentage of courses, sections, and instructors

per semester Target colleges with a marketing program

Page 6: Jasig sakai2012 silva-lugo

04/14/2023

6

Methodology: Databases

Data was retrieved from the network: Registrar Office: IBM DB2 database Middleware: MySQL database

Language used to query the database: A PHP script queried both database (< 5

seconds). The same script merged and compared the

results (5-20 seconds).

Page 7: Jasig sakai2012 silva-lugo

04/14/2023

7

Methods: Points of Interest about Querying

You need to understand the database.

You have to find how to best compare the results: The Registrar DB2 had zero to many instructors associated with a

section The Registrar DB2 used postfix characters in the course name, such

as for labs

The results produced from each database were accurate and error free.

Manipulating large data sets and verifying the results were accurate.

Page 8: Jasig sakai2012 silva-lugo

04/14/2023

8

Methodology: Calculating Percentages

Registrar (Total) • Courses• Sections• Instructors

Middleware (Sakai)• Courses• Sections• Instructors

% of Courses Xx 100

Page 9: Jasig sakai2012 silva-lugo

04/14/2023

9

Methodology: Statistics

Statistical Analysis: Proportion comparison:

Between 2010 and 2011 for courses, sections, and instructors per college

Between semesters for courses, sections, and instructors

Alpha level = 0.05 Two-tailed test

Page 10: Jasig sakai2012 silva-lugo

04/14/2023

10

No. of Courses from the Registrar Office and Middleware in 2010 & 2011

  2010 2011College Registrar Middleware Percentage Registrar Middleware PercentageDESIGN, CONSTRUCTION AND PLANNING 307 91 29.64 348 256 73.56HEALTH AND HUMAN PERFORMANCE 383 91 23.76 423 282 66.67ENGINEERING 1,100 218 19.82 1,127 445 39.49JOURNALISM AND COMMUNICATIONS 311 57 18.33 289 191 66.09MEDICINE 534 92 17.23 519 116 22.35PHARMACY 244 42 17.21 296 150 50.68LIBERAL ARTS AND SCIENCES 3,138 455 14.50 2,872 1,237 43.07PUBLIC HEALTH & HEALTH PROFESSIONS 457 65 14.22 458 245 53.49BUSINESS ADMINISTRATION 645 88 13.64 653 406 62.17AGRICULTURAL AND LIFE SCIENCES 1,602 213 13.30 1,647 573 34.79CROSS-COLLEGE 36 4 11.11 47 0 0.00FINE ARTS 1,022 56 5.48 1,037 179 17.26VETERINARY MEDICINE 259 13 5.02 264 51 19.32EDUCATION 630 18 2.86 662 54 8.16DENTISTRY 244 1 0.41 279 5 1.79ACCOUNTING 64 0 0.00 65 0 0.00BUILDING CONSTRUCTION 211 0 0.00 218 0 0.00CONTINUING EDUCATION 209 0 0.00 171 0 0.00LEVIN COLLEGE OF LAW 190 0 0.00 550 11 2.00MILITARY 62 0 0.00 55 0 0.00NURSING 133 0 0.00 130 88 67.69

 Total 11,781 1,504 12.77 12,110 4,289 35.42

Page 11: Jasig sakai2012 silva-lugo

04/14/2023

11

Page 12: Jasig sakai2012 silva-lugo

04/14/2023

12

Courses: Proportion Comparison between 2010 and 2011 per College

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value ResultAGRICULTURAL 0.13 0.35 0.2419 0.0150 14.3034 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.14 0.62 0.3806 0.0270 18.0054 1.645 < 0.0001 P < 0.05DENTISTRY  0.00 0.02 0.0115 0.0093 1.4809 1.645 0.0694 P > 0.05DESIGN, CONSTRUCTION 0.30 0.74 0.5298 0.0391 11.2387 1.645 < 0.0001 P < 0.05EDUCATION 0.03 0.08 0.0557 0.0128 4.1510 1.645 < 0.0001 P < 0.05ENGINEERING 0.20 0.39 0.2977 0.0194 10.1481 1.645 < 0.0001 P < 0.05FINE ARTS 0.05 0.17 0.1141 0.0140 8.4064 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.24 0.67 0.4628 0.0352 12.2002 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.18 0.66 0.4133 0.0402 11.8711 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.02 0.0149 0.0102 1.9640 1.645 0.0250 P < 0.05LIBERAL ARTS & SCIENCES 0.14 0.43 0.2787 0.0115 25.0227 1.645 < 0.0001 P < 0.05MEDICINE 0.17 0.22 0.1975 0.0245 2.0872 1.645 0.0188 P < 0.05NURSING 0.00 0.68 0.3346 0.0582 11.6320 1.645 < 0.0001 P < 0.05PHARMACY 0.17 0.51 0.3556 0.0414 8.0846 1.645 < 0.0001 P < 0.05PUBLIC HEALTH 0.14 0.53 0.3388 0.0313 12.5489 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.05 0.19 0.1224 0.0287 4.9889 1.645 < 0.0001 P < 0.052010 0.13 0.50 0.3142 0.0060 61.6931 1.645 < 0.0001 P < 0.052011 0.35 0.50 0.4248 0.0064 22.2413 1.645 < 0.0001 P < 0.05

Page 13: Jasig sakai2012 silva-lugo

04/14/2023

13

Page 14: Jasig sakai2012 silva-lugo

04/14/2023

14

Page 15: Jasig sakai2012 silva-lugo

04/14/2023

15

Courses: Proportion Comparison between Semesters

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value Result

Spring 2010 - Summer 2010 0.00 0.04 0.0166 0.0031 13.4318 1.645 < 0.0001 P < 0.05

Summer 2010 - Summer 2011 0.04 0.29 0.1772 0.0102 24.5078 1.645 < 0.0001 P < 0.05

Fall 2010 - Spring 2011 0.30 0.37 0.3344 0.0099 6.4253 1.645 < 0.0001 P < 0.05

Fall 2010 - Summer 2011 0.30 0.29 0.2985 0.0107 1.0054 1.645 0.1562 P > 0.05

Fall 2010 - Fall 2011 0.30 0.38 0.3432 0.0099 8.1293 1.645 < 0.0001 P < 0.05

Spring 2011 - Summer 2011 0.37 0.29 0.3363 0.0111 6.7244 1.645 < 0.0001 P < 0.05

Spring 2011 - Fall 2011 0.37 0.38 0.3752 0.0102 1.6609 1.645 0.0485 P < 0.05

Summer 2011 - Fall 2011 0.29 0.38 0.3469 0.0111 8.2315 1.645 < 0.0001 P < 0.05

Page 16: Jasig sakai2012 silva-lugo

04/14/2023

16Not significantly different

Page 17: Jasig sakai2012 silva-lugo

04/14/2023

17

No. of Sections from the Registrar Office and Middleware in 2010 & 2011

2010 2011College Registrar Middleware Percentage Registrar Middleware Percentage

HEALTH AND HUMAN PERFORMANCE 628 169 26.91 894 563 62.98MEDICINE 604 134 22.19 682 165 24.19DESIGN, CONSTRUCTION AND PLANNING 577 125 21.66 582 341 58.59ENGINEERING 2467 500 20.27 2384 784 32.89PHARMACY 718 131 18.25 735 426 57.96LIBERAL ARTS AND SCIENCES 7653 1308 17.09 7632 3649 47.81CROSS-COLLEGE 66 11 16.67 71 0 0.00JOURNALISM AND COMMUNICATIONS 588 98 16.67 638 327 51.25BUSINESS ADMINISTRATION 1264 185 14.64 1306 915 70.06AGRICULTURAL AND LIFE SCIENCES 3220 457 14.19 3870 1144 29.56PUBLIC HEALTH & HEALTH PROFESSIONS 811 90 11.10 919 347 37.76FINE ARTS 1843 87 4.72 1975 273 13.82VETERINARY MEDICINE 325 14 4.31 370 63 17.03EDUCATION 1299 22 1.69 1389 78 5.62DENTISTRY 356 1 0.28 372 4 1.08ACCOUNTING 136 0 0.00 116 0 0.00BUILDING CONSTRUCTION 269 0 0.00 277 0 0.00CONTINUING EDUCATION 228 0 0.00 177 0 0.00LEVIN COLLEGE OF LAW 441 0 0.00 503 11 2.19MILITARY 90 0 0.00 79 0 0.00NURSING 279 0 0.00 279 208 74.55

Total 23,862 3,332 13.96 25,250 9,298 36.82

Page 18: Jasig sakai2012 silva-lugo

04/14/2023

18

Page 19: Jasig sakai2012 silva-lugo

04/14/2023

19

Sections: Proportion Comparison between 2010 and 2011 per College

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value ResultAGRICULTURAL 0.14 0.30 0.2258 0.0100 15.4094 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.15 0.70 0.4280 0.0195 28.3899 1.645 < 0.0001 P < 0.05DENTISTRY  0.00 0.01 0.0069 0.0061 1.2973 1.645 0.0968 P > 0.05DESIGN, CONSTRUCTION & PLANNING 0.22 0.59 0.4021 0.0288 12.8197 1.645 < 0.0001 P < 0.05EDUCATION 0.02 0.06 0.0372 0.0073 5.3689 1.645 < 0.0001 P < 0.05ENGINEERING 0.20 0.33 0.2647 0.0127 9.9592 1.645 < 0.0001 P < 0.05FINE ARTS 0.05 0.14 0.0943 0.0095 9.6172 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.27 0.63 0.4809 0.0260 13.8633 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.17 0.51 0.3467 0.0272 12.7130 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.02 0.0117 0.0070 3.1238 1.645 < 0.001 P < 0.05LIBERAL ARTS & SCIENCES 0.17 0.48 0.3243 0.0076 40.5675 1.645 < 0.0001 P < 0.05MEDICINE  0.22 0.24 0.2325 0.0236 0.8508 1.645 0.1917 P > 0.05NURSING 0.00 0.75 0.3728 0.0409 18.2102 1.645 < 0.0001 P < 0.05PHARMACY 0.18 0.58 0.3833 0.0255 15.5669 1.645 < 0.0001 P < 0.05PUBLIC HEALTH & HEALH PROFESSIONS 0.11 0.38 0.2526 0.0209 12.7359 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.04 0.17 0.1108 0.0239 5.3304 1.645 < 0.0001 P < 0.052010 0.14 0.50 0.3213 0.0043 84.9707 1.645 < 0.0001 P < 0.052011 0.37 0.50 0.4316 0.0044 28.7675 1.645 < 0.0001 P < 0.05

Page 20: Jasig sakai2012 silva-lugo

04/14/2023

20

Page 21: Jasig sakai2012 silva-lugo

04/14/2023

21

Page 22: Jasig sakai2012 silva-lugo

04/14/2023

22

Sections: Proportion Comparison between Semesters

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value Result

Spring 2010 - Summer 2010 0.00 0.05 0.0163 0.0023 21.5659 1.645 < 0.0001 P < 0.05

Summer 2010 - Summer 2011 0.05 0.24 0.1576 0.0074 25.3576 1.645 < 0.0001 P < 0.05

Fall 2010 - Spring 2011 0.31 0.40 0.3528 0.0068 12.3286 1.645 < 0.0001 P < 0.05

Fall 2010 - Summer 2011 0.31 0.24 0.2850 0.0075 9.7743 1.645 < 0.0001 P < 0.05

Fall 2010 - Fall 2011 0.31 0.42 0.3636 0.0068 15.2238 1.645 < 0.0001 P < 0.05

Spring 2011 - Summer 2011 0.40 0.24 0.3371 0.0079 19.8939 1.645 < 0.0001 P < 0.05

Spring 2011 - Fall 2011 0.40 0.42 0.4058 0.0070 2.7055 1.645 0.0034 P < 0.05

Summer 2011 - Fall 2011 0.24 0.42 0.3514 0.0079 22.3055 1.645 < 0.0001 P < 0.05

Page 23: Jasig sakai2012 silva-lugo

04/14/2023

23

No. of Instructors from the Registrar Office and Middleware in 2010 & 2011

2010 2011College Registrar Middleware Percentage Registrar Middleware Percentage

CROSS-COLLEGE 36 10 27.78 37 0 0.00HEALTH AND HUMAN PERFORMANCE 220 56 25.45 267 175 65.54ENGINEERING 697 176 25.25 758 417 55.01DESIGN, CONSTRUCTION AND PLANNING 218 46 21.10 236 129 54.66BUSINESS ADMINISTRATION 405 80 19.75 414 124 29.95JOURNALISM AND COMMUNICATIONS 286 56 19.58 318 180 56.60LIBERAL ARTS AND SCIENCES 3,364 519 15.43 3,843 1,289 33.54PUBLIC HEALTH & HEALTH PROFESSIONS 403 61 15.14 444 162 36.49AGRICULTURAL AND LIFE SCIENCES 1,003 147 14.66 1,050 388 36.95FINE ARTS 621 50 8.05 659 141 21.40VETERINARY MEDICINE 256 14 5.47 489 32 6.54MEDICINE 324 16 4.94 541 49 9.06EDUCATION 583 21 3.60 581 67 11.53PHARMACY 164 3 1.83 236 8 3.39DENTISTRY 132 1 0.76 571 4 0.70ACCOUNTING 40 0 0.00 45 0 0.00BUILDING CONSTRUCTION 67 0 0.00 82 0 0.00CONTINUING EDUCATION 12 0 0.00 5 0 0.00LEVIN COLLEGE OF LAW 256 0 0.00 297 9 3.03MILITARY 28 0 0.00 31 0 0.00NURSING 174 0 0.00 205 6 2.93

Total 9,289 1,256 13.52 11,109 3,180 28.63

Page 24: Jasig sakai2012 silva-lugo

04/14/2023

24

Page 25: Jasig sakai2012 silva-lugo

04/14/2023

25

Instructors: Proportion Comparison between 2010 and 2011 per College

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value ResultAGRICULTURAL 0.15 0.37 0.2606 0.0194 11.5043 1.645 < 0.0001 P < 0.05BUSINESS ADMINISTRATION 0.20 0.30 0.2491 0.0302 3.3741 1.645 < 0.001 P < 0.05DENTISTRY  0.01 0.01 0.0071 0.0081 0.0703 1.645 0.4721 P > 0.05DESIGN, CONSTRUCTION 0.21 0.55 0.3855 0.0457 7.3403 1.645 < 0.0001 P < 0.05EDUCATION 0.04 0.12 0.0756 0.0155 5.1170 1.645 < 0.0001 P < 0.05ENGINEERING 0.25 0.55 0.4076 0.0258 11.5416 1.645 < 0.0001 P < 0.05FINE ARTS 0.08 0.21 0.1492 0.0199 6.6968 1.645 < 0.0001 P < 0.05HEALTH & HUMAN PERFORMANCE 0.25 0.66 0.4743 0.0455 8.8171 1.645 < 0.0001 P < 0.05JOURNALISM & COMMUNICATIONS 0.20 0.57 0.3907 0.0398 9.3113 1.645 < 0.0001 P < 0.05LEVIN COLLEGE OF LAW 0.00 0.03 0.0163 0.0108 2.8082 1.645 0.0025 P < 0.05LIBERAL ARTS & SCIENCES 0.15 0.34 0.2509 0.0102 17.6964 1.645 < 0.0001 P < 0.05MEDICINE 0.05 0.09 0.0751 0.0185 2.2242 1.645 0.0132 P < 0.05NURSING 0.00 0.03 0.0158 0.0129 2.2748 1.645 0.0116 P < 0.05PHARMACY  0.02 0.03 0.0275 0.0166 0.9387 1.645 0.1736 P > 0.05PUBLIC HEALTH 0.15 0.36 0.2633 0.0303 7.0459 1.645 < 0.0001 P < 0.05VETERINARY MEDICINE 0.05 0.07 0.0617 0.0186 0.5791 1.645 0.2810 P > 0.052010 0.14 0.50 0.3152 0.0068 52.8065 1.645 < 0.0001 P < 0.052011 0.50 0.29 0.3907 0.0065 31.9009 1.645 < 0.0001 P < 0.05

Page 26: Jasig sakai2012 silva-lugo

04/14/2023

26

Page 27: Jasig sakai2012 silva-lugo

04/14/2023

27

Page 28: Jasig sakai2012 silva-lugo

04/14/2023

28

Instructors: Proportion Comparison between Semesters

Semester π1 π2 Π σ π1- π2   |Z| Zα P-value Result

Spring 2010 - Summer 2010 0.00 0.07 0.0235 0.0044 14.5203 1.645 < 0.0001 P < 0.05

Summer 2010 - Summer 2011 0.07 0.23 0.1582 0.0113 14.3211 1.645 < 0.0001 P < 0.05

Fall 2010 - Spring 2011 0.29 0.32 0.3065 0.0103 3.5117 1.645 < 0.001 P < 0.05

Fall 2010 - Summer 2011 0.29 0.23 0.2645 0.0114 5.3643 1.645 < 0.0001 P < 0.05

Fall 2010 - Fall 2011 0.29 0.28 0.2856 0.0099 0.4479 1.645 0.3264 P > 0.05

Spring 2011 - Summer 2011 0.32 0.23 0.2881 0.0115 8.4099 1.645 < 0.0001 P < 0.05

Spring 2011 - Fall 2011 0.32 0.28 0.3030 0.0099 4.0899 1.645 < 0.0001 P < 0.05

Summer 2011 - Fall 2011 0.23 0.28 0.2636 0.0111 5.1051 1.645 < 0.0001 P < 0.05

Page 29: Jasig sakai2012 silva-lugo

04/14/2023

29 Not significantly different

Significantly different

Page 30: Jasig sakai2012 silva-lugo

04/14/2023

30

Discussion

In general, course, sections, and instructors using Sakai are not monitored.

This simple procedure has the following advantages: Provide information about the proportion of

instructors, courses, and sections using Sakai. It helps to monitor the implementation process. It creates the database for marketing programs. It might be used to evaluate success at the college

and at university level.

Page 31: Jasig sakai2012 silva-lugo

04/14/2023

31

Conclusions

There was significant increase in their proportions of courses, sections, and instructors from 2010 to 2011 in the

majority of the colleges.

There was significant increase in their proportions of courses, sections, and

instructors per semester.

We have identified the colleges that need more our attention, and we are targeting

them with a marketing program.

Monitoring Sakai usage is important during the implementation and evaluation of the

overall success at the college and university level.