Jisc learning analytics update-nov2016
-
Upload
paul-bailey -
Category
Education
-
view
305 -
download
0
Transcript of Jisc learning analytics update-nov2016
The Open University, 2nd November 20168th UK Learning Analytics Network Meeting
Jisc Learning Analytics 2016
Programme10:25 – 11:15
Update on Jisc’s learning analytics programme
11:15 – 11:30
Tea / coffee
11:30 – 12:30
Learning design meets learning analytics, Dr Bart Rienties, Open University
12:30 – 13:30
Lunch
13:30 – 14:15
Parallel session 1: Legal issues for learning analytics, Andrew Cormack, Jisc
Parallel session 2: Addressing the challenges , Il-Hyun Jo, Ewha Womans University
14:15 – 15:00
Parallel session 1: The potential of blockchain , Prof John Domingue, Knowledge Media Institute, OU
The design and deployment of a learning analytics dashboard, David Evans, North Warwickshire & Hinckley College
15:00 – 15:15
Tea / coffee – Juniper/Medlar Room, The Hub
15:15 – 15:55
The Learning Analytics Community Exchange, Dr Doug Clow, Institute for Educational Technology, OU
Paul Bailey, Senior Codesign Manager, Research and DevelopmentJisc learning analytics service
http://www.slideshare.net/paul.bailey/
Where we started…
Jisc Learning Analytics 2016
Jisc Learning Analytics 2016
Effective Learning Analytics ChallengeRationale»Organisations wanted help to get started and have access to
standard tools and technologies to monitor and intervene Priorities identified»Code of Practice on legal and ethical issues»Develop basic learning analytics service with app for students»Provide a network to share knowledge and experienceTimescale»2015-16—test and develop the tools and metrics»2016-17—transition to service »Sep 2017—launch, measure impact: retention and achievement
Jisc Learning Analytics 2016
Jisc’s Learning Analytics ProjectThree core strands:
Learning Analytics Service
Toolkit Community
Jisc Learning Analytics
Learning Analytics Sophistication Model
Jisc Learning Analytics 2016
Analytics – the bigger picturehttps://docs.google.com/presentation/d/1AdBkYHO3hqEJ7W2McYIsAKzF4EgFNYJM9XGfDOTRYek/edit?usp=sharing
Michael Webb
Descriptive Analyticswhat happened?
Diagnostic Analyticswhy did it happen?
PredictiveAnalyticswhat will happen?
Prescriptive Analyticswhat should I do?
Automated Decision makingIt's done
Analytics maturity
Descriptive Analyticswhat happened? How do I compare?
Prescriptive Analyticswhat should I do?
Predictivewhat will happen?
Automatedit’s done
Data
Diagnostic Analyticswhy did it happen?
Ordered Data
Sector Transformation
Awareness
Experimentation
Organisation support
Organisational transformation
Analytics without a national approach
Sector Transformation
Awareness
Experimentation
Organisation support
Organisational transformation
Descriptive Analyticswhat happened? How do I compare?
Predictive Analyticswhat will happen?
Prescriptive Analyticswhat should I do?
Automatedit’s done
Data
Diagnostic Analyticswhy did it happen?
Ordered Data
Standardised Data
Analytics with a national approach
Sector Transformation
Awareness
Experimentation
Organisation support
Organisational transformation
Descriptive Analyticswhat happened? How do I compare?
Predictive Analyticswhat will happen?
Prescriptive Analyticswhat should I do?
Automatedit’s done
Data
Diagnostic Analyticswhy did it happen?
Ordered Data
Standardised Data
Adaptive learning etc.
Recommendation engines etc.
Predictive models, Intervention
management etcData exploration tools, processes etc
Dashboards, Benchmarking etc.
Data Warehouse, data stores
Data connectors
Analytics with a national approach
Descriptive Analytics
Predictive Analytics
Prescriptive Analytics
AutomatedDiagnostic Analytics
Standardised Data
Learning Records Warehouse
xAPI Plugins
Data transformation tools
Data and API Standards
Jisc Services
Other ProviderServices
Basic dashboards
Student App
Analytics Labs
Benchmarking services
College Analytics
Basic predictive modelling and intervention management
Procurement frameworks
Integration tools
Services for researchers
Pilot projects
Services for researchers
Pilot projects
Institutional Dashboards
Data visualisation tools
Data exploration tools
Advanced predictive modelling
Integrated intervention management
??? ???
Jisc Learning Analytics 2016
- Sector Data used in mashups:
- NSS- SCONUL- LiDP- HESA- Open Access Reporting/Deposit, - JUSP / IRUS- IRUS - IMD- Altmetrics - H index- Impact Factor - REF metrics- Jisc Collections bands &
Subscription data
Library Labs: 6 teams, 33 participants drawn from Libraries
Jisc Learning Analytics 2016
Library AnalyticsLibrary Labs
- BUT also analytics on institutional data:
- e-resource usage by type & department
- e-resource cost benchmarking- EZProxy logs- Loans- Gate entries- Acquisitions- Counter reports- Capita Decisions- Journal Citation Reports
Jisc Learning Analytics 2016
Library AnalyticsLibrary LabsBirkbeck, University of LondonSheffield Hallam UniversityUniversity of EdinburghUniversity of WarwickThe University of ManchesterUniversity of SalfordLiverpool John Moores UniversityNewcastle UniversitySouthampton Solent UniversityAnglia Ruskin University LibraryUniversity of South WalesUniversity of NottinghamBrunel University LondonKingston UniversityTeesside UniversityBodleain Libraries, University of OxfordUniversity of WolverhamptonUniversity of LeicesterUniversity of ReadingManchester Metropolitan UniversityUniversity of BathDe Montfort University
Jisc Learning Analytics 2016
Library Analytics- Mashing up Library data was difficult – SCONUL is not HESA- Many different internal systems, comparative analytics difficult - Proof of concept dashboards stimulating institutions (traffic lights)- More interest and contributions to recipes at http://github.com/jiscdev
/xapi-lib- New verbs! Eduroam, presence- Data Sharing Agreements and an experimental area in the Heidi Lab- Scope for more librarians alongside planners on Jisc’s beta BI project
Where are we now…
Jisc Learning Analytics 2016
Community: Project Blog, mailing list and network eventsBlog: http://analytics.jiscinvolve.org – over 30 blog postsMailing: [email protected] – 422 members (182 organisations)8th Network Meeting ~600+ participants
Jisc Learning Analytics 2016
http://www.jisc.ac.uk/guides/code-of-practice-for-learning-analytics
Code of Practice
http://repository.jisc.ac.uk/5661/1/Learning_Analytics_A-_Literature_Review.pdf
Library Analytics Service
Learning Analytics Service Architecture
Jisc Learning Analytics 2016
Learning analytics products and toolsLearning records warehouse – activeData Explorer – basic visualisations Student Unified Data Definition – version 1.2.7 and examples major SRS and validation tooVLE – xAPI recipe and plugins for Blackboard and MoodleAttendance tracking – xAPI recipe (being piloted soon)Student App – release 1 Dec 2016
Tribal Student Insights (10)Open Learning Analytics Processor (4)Further learning analytics product pilots (tbc)
Jisc Learning Analytics 2016
UDD Validator Tool• Customer-side UDD validation (web-based, secure access)• UDD data preparation tool for institutions• Jisc will load the historical data (once validated)• Covers current & future UDD - 1.2.7, 1.2.x, 1.3.0 etc• Links directly to UDD GitHub site (dynamic updates)• Agile approach to software functionality/ release• V1.0 - hard validation (UDD structure, optional/ mandatory fields, field contents)• Relational entities – integrity checks• Soft validation - data quality and concentration/ coverage (working with Tribal/ Unicon Marist)• Focus on key fields for predictive modelling purposes, student app• Gives control & flexibility to our members – rapidly quick data validation (Azure Cloud)
Jisc Learning Analytics 2016
ImplementationsProfile Aims Tools N
oData Sources
Teaching and research led Universities
Student retention and success
Tribal student insight/data warehouse
7 VLE (Moodle and Blackboard), student records and attendance
Teaching and research led Universities
Success and engagement
Student app 4 VLE (Moodle and Blackboard), student records
Teaching led Universities
Student retention
Open source processors/data warehouse
4 VLE (Moodle and Blackboard), student records and attendance
FE Colleges Student retention
Tribal student insight
2 VLE (Moodle), student records and attendance
Getting on-board…
https://analytics.jiscinvolve.org/wp/on-boarding/
Jisc Learning Analytics 2016
On-boarding Process
Stage 1: OrientationStage 2: DiscoveryStage 3: Culture and Organisation SetupStage 4: Data IntegrationStage 5: Implementation Planning
https://analytics.jiscinvolve.org/wp/on-boarding/
Jisc Learning Analytics 2016
Stage 1: Orientation
Stage 1. Orientation
1. Sign up to the analytics mailing listEvidence required:A list of people in your institution signed up to the mailing list
2. Review the learning analytics blog post and relevant reportsEvidence required:Notes on useful articles and posts you have found
3. Attend a Jisc webinar, network meeting or workshopEvidence required:Notes from attending a recent event
Jisc Learning Analytics 2016
Stage 2: Discovery ReadinessStage 2. Discovery 4. Decide on institutional aims for learning analyticsEvidence required: A prioritised list of your aims for learning analytics
5. Strategic alignment, senior management approval and you have a nominated project lead Evidence Required: Named sponsor from the senior management team, Named project lead and contact details, Named technical lead and contact leaded, A list of members of your working/management group
6. Undertake the readiness assessmentEvidence required :A completed readiness assessment questionnaire with your commentary on the answers
7. Arrange a verification meeting with Jisc to discuss the outcomes and possible next stepsEvidence required: Date of meeting, documentation to share and a list of people attending
Jisc Learning Analytics 2016
Discovery readinessTopic I
DQuestion Commentary Response Score
Leadership
1 The institutional senior management team is committed to using data to make decisions
Please provide a commentary on you response to each question where appropriate
0 - Hardly or not at all1 - To some extent2 - To a great extent
Leadership
2 Our vice-chancellor / principal has encouraged the institution to investigate the potential of learning analytics
0 - Hardly or not at all1 - To some extent2 - To a great extent
Leadership
3 There is a named institutional champion / lead for learning analytics
0 - No2 - Yes
Vision
4 We have identified the key performance indicators that we wish to improve with the use of data
0 - Hardly or not at all1 - To some extent2 - To a great extent
A supported review of institutional readiness
https://analytics.jiscinvolve.org/wp/on-boarding/step-6-readiness-assessment/
Jisc Learning Analytics 2016
Stage 3: Culture and Organisation SetupStage 3. Culture and Organisation Setup
8. Start to address readiness recommendationsEvidence required: Action plan to address readiness recommendations
9. Legal and ethical policy considerations in handEvidence required: List of institutional policies relevant to learning analytics; Plan to update/create policies to cover learning analytics
10. Decision on learning analytics products to pilotEvidence required: A documented list of products with an agreed rational for choices
11. Data processing agreement signedEvidence required: Signed Data Processing Agreement
12. Select student groups for the pilot and engage staff/studentsEvidence required: List of student groups/cohorts and numbers of students involved
Jisc Learning Analytics 2016
Stage 4: Data IntegrationStage 4. Data Integration 13. Undertake a data and systems audit 14. Contact Jisc to start data integration 15. Install and evaluate the VLE data plugin(s) on a test system at your institution
16. Extract student data, transform to UDD and validate.
17. Extract historical VLE (or other activity) data
18. Install VLE (or other activity) data plugin(s) on live system, activate for live data upload to LRW
19. View uploaded LRW data using data explorer to check quality
Jisc Learning Analytics 2016
Stage 4: Data collection
About the student Activity data
TinCan (xAPI)ETL
Jisc Learning Analytics 2016
Stage 5: Implementation Planning
Stage 5. Implementation Planning
20: Move to implementation StageEvidence required: An implementation plan with agreed timescales
Jisc Learning Analytics 2016
On-boarding Process
Data Visualisation Dashboards
Ready to implementReady to
implement
Jisc Learning Analytics 2016
On-boarding – get started
Stage 1: Orientation – review/doneStage 2: Discovery – mostly self-supportStage 3: Culture and Organisation Setup – Jan 2017Stage 4: Data Integration – slots from early 2017Stage 5: Implementation Planning - slots from early 2017
Further exploration…
https://www.jisc.ac.uk/rd/get-involved
Jisc Learning Analytics 2016
Co-design challenges 2017Explore our co-design challengesHelp steer our innovation work by exploring the next big ideas for technology in education and research.
Jisc Learning Analytics 2016
Data driven
learning gains
Next generation research
environment
Digital skills for
research
Should we gather more data on students, staff and buildings that would allow us to deliver
better experiences?
We think it is time for a new type of learning
environment, but what would this look like?
We think it is time for a new type of learning
environment, but what would this look like?
What would a truly digital apprenticeship look like?
Can we make better use of data to improve learning,
teaching and student outcomes?
How do we equip researchers and related staff with the skills
they need for the future of research?
The intelligent
campus The digital apprentice
Next generation
learning environment
Jisc Learning Analytics 2016
1Discuss
emergingchallenges
2
Prioritise
ideas
3Announce successful
ideas
4Report progre
ss
Identify ideas
31st Oct – 24th Nov
4th Jan– 30th Jan 6th Feb Apr/May
Release 6 challenge areas and invite Jisc members and
other experts to discuss
Audience: managers,
consumers, some leaders, other
experts
Present ideas for activities Jisc
could do and ask members which
they support
Audience: managers,
consumers, some leaders
Release 6 challenge areas and invite Jisc members and
other experts to discuss
Audience: everyone who followed the
challenge
Release 6 challenge areas and invite Jisc members and
other experts to discuss
Audience: everyone who followed the
challenge
Contacts
Paul Bailey [email protected]
Further Information: http://www.analytics.jiscinvolve.org
Join: [email protected]
Jisc Learning Analytics 2016