Climate Informed Monitoring Discussion Climate Informed Monitoring Facilitated Discussion.
Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 ›...
Transcript of Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 ›...
![Page 1: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/1.jpg)
Better-Informed Decision Making
28 August 2012
Kent Chen Market Development Director
![Page 2: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/2.jpg)
2
![Page 3: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/3.jpg)
3
“The Third Wave”- Malcomm Brown, Director of EDUCAUSE Learning Initiative
![Page 4: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/4.jpg)
4
“The Third Wave”- Malcom Brown, Director of EDUCAUSE Learning Initiative
20052000 2015
Learning Management System
Web 2.0
2010
![Page 5: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/5.jpg)
5
Why Analytics?
How effective are our online tools at producing better student outcomes?
![Page 6: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/6.jpg)
6
Why Analytics?
How effective are our online tools at producing better student outcomes?
Who are our at‐risk students and how can we help them become successful?
![Page 7: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/7.jpg)
7
Why Analytics?
How effective are our online tools at producing better student outcomes?
Who are our at‐risk students and how can we help them become successful?
Who are our most successful educators, and why are they successful?
![Page 8: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/8.jpg)
8
Information Needs Vary
DATA TO HELP ME
On DemandEasy to Access
and Easy to Digest
But There Are Common Themes
![Page 9: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/9.jpg)
9
Getting Information Today
![Page 10: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/10.jpg)
10
Academic Leadership
Top 50 Courses Course Design & Trends
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and tTrends
Assessment Coordinators
Course, program, and gen ed-level goal & rubric-based reporting
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Students By Course: • Student performance against goals
Learning Analytics – Metrics/Reporting
Course Reporting
In‐House Custom Reporting
![Page 11: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/11.jpg)
11
Course Reporting
One course, one semester viewInstructors are primary users
![Page 12: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/12.jpg)
Academic Leadership
Top 50 Courses Course Design & Trends
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and Trends
Assessment Coordinators
Instructors
By Course: • Student performance against goals• Rubric-based reporting• User activity per dayEarly warning system
Students By Course: • Student performance against goals
Learning Analytics – Metrics/Reporting
Course Reporting
In‐House Custom Reporting
![Page 13: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/13.jpg)
13
Academic Leadership
Top 50 Courses Course Design & Trends
At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:
instructor, department, subject, term
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS Usage and Trends LMS usage and trends Capacity planning Course open after term end
Assessment Coordinators
Aggregate, student, cross-course, cross-term goal performance (LMS scores)
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Course dashboard - comparison of instructor’s course design to other courses
Correlation of grades to activity
Students By Course: • Student performance against goals
Student dashboard - performance across courses and compared to peers
Learning Analytics – Metrics/Reporting
For LearnCourse Reporting
In‐House Custom Reporting
![Page 14: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/14.jpg)
Information Needs Vary
DATA TO HELP ME
On DemandEasy to Access
and Easy to Digest
But There Are Common Themes
![Page 15: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/15.jpg)
CustomerSuccess Story
![Page 16: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/16.jpg)
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
![Page 17: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/17.jpg)
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
![Page 18: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/18.jpg)
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
Impact to Lingnan University
![Page 19: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/19.jpg)
Blackboard AnalyticsCustomer
Lingnan University Key Facts
• Public University – 2,800 Students
• Small Class Size
• High International Community
Why Blackboard Analytics?
Impact to Lingnan University
![Page 20: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/20.jpg)
Thank you!
![Page 21: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/21.jpg)
APPENDIX
![Page 22: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/22.jpg)
George SiemensAssociate Director
Technology Enhanced Knowledge Research Institute
![Page 23: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/23.jpg)
Thank you!
![Page 24: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/24.jpg)
24
Best Practices
Defined Metrics
Business Rules
Derived Information
Transformation Layer
Data Warehouse
OLAP
Star Schema
Pre-Built Reports &
Dashboards
Blackboard AnalyticsA Packaged Analytics Application
Pre-Built Data Integration
SIS
![Page 25: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/25.jpg)
For Leadership and Staff
![Page 26: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/26.jpg)
Flexible Reporting
![Page 27: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/27.jpg)
For Instructors
![Page 28: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/28.jpg)
For Students
![Page 29: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/29.jpg)
29
APPENDIX
![Page 30: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/30.jpg)
30
![Page 31: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/31.jpg)
31
![Page 32: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/32.jpg)
32
![Page 33: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/33.jpg)
33
![Page 34: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/34.jpg)
34
![Page 35: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/35.jpg)
35
![Page 36: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/36.jpg)
36
![Page 37: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/37.jpg)
37
![Page 38: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/38.jpg)
38
Baseline Implementation
![Page 39: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/39.jpg)
1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 12
Rapid Deployment
Homegrown Data Warehouses
Your Data is Loaded
5 Step Approachmonths
Design and Development Implementation
ERP Vendor Toolkits at $250 per hour
committed resources
![Page 40: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/40.jpg)
40
Metrics & Dimensions for Analysis
•Gradebook•Possible Score•Score•Weight•Grade Pct•Weighted•Element Count•Highest Score•Lowest Score•Improvement•Average Score•Attempts•Days To Grade
•Goals•TBD
•Course Design•Item Count•Files Attached•VTBE Objects•VTBE Characters•File Size•Folder Depth•Links•Folders
•Course Section•Subject•Department•College•Class Number•Term•Course Career/Level•Instruction Method
•Students•Birthdate•Gender•Ethnicity•Citizenship•Current Term•Primary Major•Career/Level•Academic Level•Current Academic Standing•Primary College
•Instructors•Gender•Ethnicity•Highest Education Level•Faculty Rank•Tenure Status•Primary Department
•User Activity•Sessions
•Logins•Session Minutes•Courses•Content Items•Tools•Modules•Clicks•Pages
•Course Activity•Courses Accessed•Course Access Minutes•Session Course Sequence•Content Items•Pages
•Content Activity•Content Items Accessed•Content Item Hits•Content Access Minutes•Session Sequence•Folder (or Content) Depth
•Submissions•TBD
Learn MetricsKey SIS
Elements
![Page 41: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/41.jpg)
41
Technical Requirements
• Requires the use of Blackboard LearnTM, Release 9.1, Service Pack 4 or later
• Pre-built SIS integration with:• Oracle (PeopleSoft)• Datatel (Colleague) • SunGard Higher Education • An Integration method is provided for other SISs
• Requires a locally hosted SQL Server environment• Minimum hardware & software requirements available on
request
![Page 42: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/42.jpg)
42
Academic Leadership
At-risk analysis Submission activity Engagement vs. grades correlation Aggregate learning outcomes Top 50 courses Course design summary & trends Usage, trends & best practices by:
instructor, department, subject, term
Instructional Technology
Overall usage per date range User statistics per month Course statistics per month
LMS usage and trends Capacity planning Course open after term end
Assessment Coordinators
Course, program, and gen ed-level goal & rubric-based reporting
Goal “coverage” reporting against curriculum
Artifact repository
Aggregate, student, cross-course, cross-term goal performance (LMS scores)
Instructors
By Course: • Student performance against goals• Rubric-based reporting • User activity per dayEarly warning system
Course dashboard - comparison of instructor’s course design to other courses
Correlation of grades to activity
Students By Course: • Student performance against goals
Student dashboard - performance across courses and compared to peers
Learning Analytics – Metrics/Reporting
For LearnCourse Reporting
![Page 43: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/43.jpg)
43
Course Reporting
One course, one semester viewInstructors are primary users
![Page 44: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/44.jpg)
44
Cross-course rubric
Assessment Coordinators are primary users
![Page 45: Better-Informed Decision Makingchtl.hkbu.edu.hk › documents › elfa2013 › Session1A-S2-forweb.pdf · Better-Informed Decision Making ... George Siemens Associate Director Technology](https://reader030.fdocuments.in/reader030/viewer/2022041109/5f0cfba17e708231d43818ab/html5/thumbnails/45.jpg)
45
Cross-department, cross-college view
Many benefit – students, instructors, executives
for Learn